From david.holmes at oracle.com Mon Jun 1 05:51:25 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Jun 2020 15:51:25 +1000 Subject: RFR 8245289: Clean up offset code in JavaClasses In-Reply-To: References: Message-ID: <1968b242-412b-b41e-1767-5cc00c1c0041@oracle.com> Hi Coleen, On 28/05/2020 10:48 pm, coleen.phillimore at oracle.com wrote: > Summary: Make offset member names consistent and private, move static > initializations near owning classes > > This one is better.? I gave up on the X macros because they didn't save > typing.? The changes here were to add underscores to offset field names, > remove _in_bytes for offset accessors and add asserts that the offset is > non-zero, made java_lang_ref_Reference offset fields private, and moved > static member definitions closer to the class instead of some random > place in javaClasses.cpp. I focused on the javaClasses.* changes rather than the incidental changes due to the _in_bytes removal. src/hotspot/share/classfile/javaClasses.hpp 892 static void serialize_offsets(SerializeClosure* f) NOT_CDS_RETURN; You dropped the NOT_CDS_RETURN in the new code. --- src/hotspot/share/classfile/javaClasses.inline.hpp I'm confused about the removal of the initialization assertions. I would have expected this, for example, 33 void java_lang_String::set_coder(oop string, jbyte coder) { 34 string->byte_field_put(_coder_offset, coder); to be expressed using coder_offset(): 33 void java_lang_String::set_coder(oop string, jbyte coder) { 34 string->byte_field_put(coder_offset(), coder); so that we still get an initialization check. Though I see that initialization checks are an area where there remains a lot of inconsistency - it's not clear when we can rely solely on the "_x_offset != 0" check versus the "_initialized==true" check, and there are many methods with no initialization check at all. Thanks, David ----- > See discussion for 8243996 Remove hardcoded field offsets for more > details. > https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-May/041732.html > > Tested with tier1-6. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8245289.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8245289 > > Thanks, > Coleen From magnus.ihse.bursie at oracle.com Mon Jun 1 11:32:50 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 1 Jun 2020 13:32:50 +0200 Subject: RFR(S) : 8245610 : remove in-tree copy on gtest In-Reply-To: References: <93E621FF-580C-4FF9-8D8E-4BB33CB93A90@oracle.com> <0c5d41f4-eebd-a8ae-d537-e14a878a6663@oracle.com> <7B996E95-B57D-4F38-97F1-65FFCA2DD01E@oracle.com> <2A2FD345-F641-4D6A-85DC-6379E8582C6F@oracle.com> Message-ID: <4b855354-367d-a103-d2c9-242bca9fe5f8@oracle.com> On 2020-05-29 07:04, Thomas St?fe wrote: > Hi Igor, > > thank you for taking the time to answer my grumblings. > > Yes, comparison with jtreg is a bit crooked - it is not needed to get > a valid build. But jtreg is also maintained in the official OpenJDK > repositories; I can clone codetools/jtreg and have the correct > version. With gtests, I have to know which version OpenJDK needs, > which is not the latest version, and have to download that from > outside our repositories. Getting it wrong version may yield me > difficult to analyze build errors (plattform zoo, handicapped C++ > compilers etc). Also, updating to a new gtest version will now > involve?a lot more communicatio (a version check in configure would > help with that). > > But this is a minor complaint, I could live with that. I most dislike > the fact that I have to specify this option *every single time*, and > that omitting it is objectively wrong and may give me a false sense of > security. Omitting it gives no warning, but if my changes break gtests > I will only notice it hours later when jdk/submit results are back. > > Yes, I can hide this behind an alias or script, but I think this is > the wrong way. You guys did such a good job in making the build dead > easy: first time, it tells me exactly which Debian packets to install, > I always loved that special touch.?I specify boot jdk and maybe debug > level and I am done. In fact, the build is so easy that until recently > I did not even know we had a build documentation :) The new > --with-gtest option is a jarring break from that. I agree, Thomas. The patch went in too fast, without proper exploration of alternatives, or how this would affect the usability of the build. One of the main goals of the build system has been that it should be easy to build, and the system should be "self-instructing", so that if a dependency is missing, this should be made clear, and a suitable suggestion for correction should be printed. The gtest removal fails on both these accounts. :( I think we should change configure so that gtest is a required dependency, unless specifically disabled. We can look? for gtest in standard locations, like /usr/src/googletest, where it is installed by e.g. "sudo apt install googletest". Installation instructions, such as this apt command, must also be added. (I'd appreciate feedback on what the package is called on RH/Fedora!) We could also consider checking for an environment variable, similar to how the boot JDK looks at JAVA_HOME, so you can set up a non-standard path in your environment, and do not have to pass it to configure as a flag all the time. And we also need to fix the RunTest framework, as Ren? pointed out, so that if you try to run gtests without the gtest library, you need to get a proper error message that describes the step you need to take to be able to run gtest tests. /Magnus > > In my mind, gtest is in the same ballpark as the freetype library on > Windows. That has always been a bit annoying but that was only > Windows. Something you cannot rely the OS library mechanism to deliver > but have to download and place yourself and tell the build about it. > > I wonder whether we can find a compromise somehow. Maybe something > like an?agreed on structure and place for a "companion directory", > containing build dependencies like these. Its location could be by > convention in a clear relation to the OpenJDK sources (e.g. just > beside it) and configure would look for the libraries in there by > default. Like this: > > - openjdk-source > ? ?- configure > ? ?- Makefile > ? ?- .. > - openjdk-builddeps > ? - google-test > ? - freetype > ? - .. > > This would also lend itself very well to a maintained repository of > build dependencies somewhere (not necessarily maintained by Oracle). > We would have similar ease of use as with platform libraries, which by > default are also expected in a standard place in the?file system. > > Thanks, Thomas > > > On Fri, May 29, 2020 at 5:20 AM Igor Ignatyev > > wrote: > > Hi Thomas, > > I regret that this patch made you unhappier. I fully agree that > all hotspot developers are to always build *and* run gtest tests, > yet not all people who work on OpenJDK project are hotspot developers. > > I also believe that all OpenJDK developers are to run tests > related to the area they are changing. The vast majority of such > tests are jtreg tests. And jtreg, for better or worse, is a > counterexample to your last paragraph -- it's an essential part of > OpenJDK, but it doesn't come in a form of well established > library, and none of known to me platforms have jtreg in their > package manager, so you do have to manually download jtreg and > specify its location one way or another. I understand that there > is a difference b/w gtest and jtreg, as jtreg isn't really need at > build time. However the proper way to run any of OpenJDK tests is > by `make test` and for it to work you need to either executed > configure w/ --with-jtreg or specify JT_HOME on each invocation of > `make test`, where the latter is a preferred way. From that point > of view, gtest and jtreg aren't different, you need to download > both manually and specify their location at configure-time. > > with that being said, I truly understand the inconvenience it > causes to you, yet given you need to download gtest only once and > it's fairly easy to hide `with-gtest` behind a script or an alias > like?configure_openjdk='bash ./configure > --with-gtest=$GTEST_HOME', I hope it won't cause problems for you > and won't discourage you in anyways. > > Thanks, > -- Igor > >> On May 28, 2020, at 12:31 AM, Thomas St?fe >> > wrote: >> >> Hi, >> >> I know the?boat has sailed, I missed this one. But I am a bit >> unhappy about this. >> >> I always build with gtests; I think it makes no sense to not >> build with gtest. Even if you do not want to run the gtests?(and >> it makes sense to always do since its a good first-base validity >> test), hotspot developers should always build them since changes >> in the hotspot sources may break hotspot gtests. hotspot source >> and gtests are a unity. >> >> So if it makes no sense to not build gtest, I should not need a >> special option to specify gtest location - I'd argue that the >> default case should not require special?options. >> >> The JBS issue states that "it can be treated like any other >> external dependencies" but this comparison does not hold - almost >> all other dependencies come in the form of well established >> libraries with standard headers and standard installation >> locations which can be coded as default values. On a recent >> mainstream platform I do not have to specify any paths to >> libraries to build OpenJDK. This is now different, I always have >> to manually download gtests and specify gtest location. This is >> regrettable. >> >> Thanks, Thomas >> >> >> On Tue, May 26, 2020 at 2:27 PM Magnus Ihse Bursie >> > > wrote: >> >> >> >> On 2020-05-25 19:53, Igor Ignatyev wrote: >> > Hi Magnus, >> > >> >> On May 25, 2020, at 7:46 AM, Magnus Ihse Bursie >> >> > >> >> > >> wrote: >> >> >> >> Looks basically good to me. >> > thanks for your review! >> >> >> >> We need to document the dependency on gtest, and how to >> retrieve it. >> >> I recommend you add a section next to the JTReg >> information (under >> >> the "## Running Tests" heading) in doc/building.md. I >> think you >> >> should include basically the same information as you did >> in your >> >> follow-up mail, that was informative and clear. >> > that's a good suggestion, I've added a small paragraph to >> 'Running >> > Tests' in doc/building.md and regenerated corresponding >> .html file -- >> > http://cr.openjdk.java.net/~iignatyev/8245610/webrev.doc/ >> > please let me know if you think something should be added >> or reworded. >> Looks great! Thank you. >> >> /Magnus >> > >> >> >> >> I assume the most suitable replacement for developers who >> are used to >> >> add a '--disable-hotspot-gtest' to e.g. a pre-definied jib >> >> configuration is to now use '--without-gtest'. This will >> need to be >> >> communicated, perhaps to both hotspot-dev and jdk-dev. >> > sure, after this gets integrated, I'll let both hotspot-dev >> and >> > jdk-dev aliases know which changes might be required to >> enable/disable >> > hotspot gtest tests compilation. >> > >> > Thanks. >> > -- Igor >> > >> >> >> >> /Magnus >> >> >> >> On 2020-05-22 20:12, Igor Ignatyev wrote: >> >>> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00/ >> >>>> 132 lines changed: 80 ins; 36 del; 16 mod >> >>> >> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00%2bremoval/ >> >>>> 57482 lines changed: 80 ins; 57385 del; 17 mod; >> >>> Hi all, >> >>> >> >>> could you please review this small (if you ignore removal >> part) >> >>> patch which removes in-tree copy of gtest >> (test/fmw/gtest) and >> >>> updates makefiles to use one provided thru an added >> configure option >> >>> `--with-gtest`? the previously used configure option >> >>> `--enable-hotspot-gtest` gets depricated. >> >>> >> >>> the patch also compiles gtest and gmock source code into >> a static >> >>> library so they can be compiled w/ all warnings disabled. >> >>> >> >>> from JBS: >> >>>> w/ JEP 381 (JDK-8241787 / JDK-8244224) being integrated, >> all >> >>>> compilers used by OpenJDK became supported by gtest >> out-of-box, so >> >>>> there is no need to have our copy of gtest framework >> (including >> >>>> gmock) within OpenJDK source tree. instead, it can be >> treated like >> >>>> any other external dependencies, and a pointer to the >> directory w/ >> >>>> gtest code can be passed via configure options. >> >>> >> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245610 >> >>> webrevs: >> >>> ?- http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00/ >> >>> (meaningful changes) >> >>> ?- >> >>> >> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00%2bremoval/ >> >> >>> (all changes) >> >>> testing: >> >>> - gtest tests on {linux,windows,macosx}-x64; >> >>> - tier[1-5] builds which include but not limited to >> linux-aarch64, >> >>> linux-arm32, linux-x64-zero >> >>> >> >>> Thanks, >> >>> -- Igor >> >>> >> >> >> > >> > From coleen.phillimore at oracle.com Mon Jun 1 12:16:59 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 1 Jun 2020 08:16:59 -0400 Subject: RFR 8245289: Clean up offset code in JavaClasses In-Reply-To: <1968b242-412b-b41e-1767-5cc00c1c0041@oracle.com> References: <1968b242-412b-b41e-1767-5cc00c1c0041@oracle.com> Message-ID: <4df986e9-5cf0-352c-487a-075e6902e70a@oracle.com> Hi David, I already checked this in. On 6/1/20 1:51 AM, David Holmes wrote: > Hi Coleen, > > On 28/05/2020 10:48 pm, coleen.phillimore at oracle.com wrote: >> Summary: Make offset member names consistent and private, move static >> initializations near owning classes >> >> This one is better.? I gave up on the X macros because they didn't >> save typing.? The changes here were to add underscores to offset >> field names, remove _in_bytes for offset accessors and add asserts >> that the offset is non-zero, made java_lang_ref_Reference offset >> fields private, and moved static member definitions closer to the >> class instead of some random place in javaClasses.cpp. > > I focused on the javaClasses.* changes rather than the incidental > changes due to the _in_bytes removal. > > src/hotspot/share/classfile/javaClasses.hpp > > ?892?? static void serialize_offsets(SerializeClosure* f) NOT_CDS_RETURN; I did remove that unintentionally in a merge.? If the minimal build fails, we can fix it. > > You dropped the NOT_CDS_RETURN in the new code. > > --- > > src/hotspot/share/classfile/javaClasses.inline.hpp > > I'm confused about the removal of the initialization assertions. I > would have expected this, for example, > > ?33 void java_lang_String::set_coder(oop string, jbyte coder) { > ?34?? string->byte_field_put(_coder_offset, coder); > > to be expressed using coder_offset(): > > ?33 void java_lang_String::set_coder(oop string, jbyte coder) { > ?34?? string->byte_field_put(coder_offset(), coder); > > so that we still get an initialization check. Though I see that > initialization checks are an area where there remains a lot of > inconsistency - it's not clear when we can rely solely on the > "_x_offset != 0" check versus the "_initialized==true" check, and > there are many methods with no initialization check at all. Yeah, the initialization check seemed pointless to me in a lot of places where we've already created an instance of a java.lang.Blah and we're just doing getters and setters on them. One could do further cleanup with the initialization fields, which seem redundant with checking that the offset is non-zero, and keep cleaning this up,? I presume.? This is where I stopped. Thanks, Coleen > > Thanks, > David > ----- > >> See discussion for 8243996 Remove hardcoded field offsets for more >> details. >> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-May/041732.html >> >> Tested with tier1-6. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8245289.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8245289 >> >> Thanks, >> Coleen From igor.ignatyev at oracle.com Mon Jun 1 14:54:18 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 1 Jun 2020 07:54:18 -0700 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: References: Message-ID: <88ED55CF-092F-4FAC-ACFD-17D3CA869C75@oracle.com> ping? -- Igor > On May 27, 2020, at 10:25 AM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >> 8 lines changed: 2 ins; 0 del; 6 mod; > > Hi all, > > could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? > from JBS: >> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` > the patch > - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts > - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP > > JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 > testing: tier1 > webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 > > Thanks, > -- Igor From thomas.stuefe at gmail.com Mon Jun 1 16:01:47 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 1 Jun 2020 18:01:47 +0200 Subject: RFR(S) : 8245610 : remove in-tree copy on gtest In-Reply-To: <4b855354-367d-a103-d2c9-242bca9fe5f8@oracle.com> References: <93E621FF-580C-4FF9-8D8E-4BB33CB93A90@oracle.com> <0c5d41f4-eebd-a8ae-d537-e14a878a6663@oracle.com> <7B996E95-B57D-4F38-97F1-65FFCA2DD01E@oracle.com> <2A2FD345-F641-4D6A-85DC-6379E8582C6F@oracle.com> <4b855354-367d-a103-d2c9-242bca9fe5f8@oracle.com> Message-ID: Hi Magnus, On Mon, Jun 1, 2020 at 1:35 PM Magnus Ihse Bursie < magnus.ihse.bursie at oracle.com> wrote: > > > On 2020-05-29 07:04, Thomas St?fe wrote: > > Hi Igor, > > thank you for taking the time to answer my grumblings. > > Yes, comparison with jtreg is a bit crooked - it is not needed to get a > valid build. But jtreg is also maintained in the official OpenJDK > repositories; I can clone codetools/jtreg and have the correct version. > With gtests, I have to know which version OpenJDK needs, which is not the > latest version, and have to download that from outside our repositories. > Getting it wrong version may yield me difficult to analyze build errors > (plattform zoo, handicapped C++ compilers etc). Also, updating to a new > gtest version will now involve a lot more communicatio (a version check in > configure would help with that). > > But this is a minor complaint, I could live with that. I most dislike the > fact that I have to specify this option *every single time*, and that > omitting it is objectively wrong and may give me a false sense of security. > Omitting it gives no warning, but if my changes break gtests I will only > notice it hours later when jdk/submit results are back. > > Yes, I can hide this behind an alias or script, but I think this is the > wrong way. You guys did such a good job in making the build dead easy: > first time, it tells me exactly which Debian packets to install, I always > loved that special touch. I specify boot jdk and maybe debug level and I am > done. In fact, the build is so easy that until recently I did not even know > we had a build documentation :) The new --with-gtest option is a jarring > break from that. > > I agree, Thomas. The patch went in too fast, without proper exploration of > alternatives, or how this would affect the usability of the build. > > One of the main goals of the build system has been that it should be easy > to build, and the system should be "self-instructing", so that if a > dependency is missing, this should be made clear, and a suitable suggestion > for correction should be printed. > > The gtest removal fails on both these accounts. :( > > I think we should change configure so that gtest is a required dependency, > unless specifically disabled. We can look for gtest in standard locations, > like /usr/src/googletest, where it is installed by e.g. "sudo apt install > googletest". > I fear that we are more reliant on a specific version of googletest than is the case with standard libraries. Just a gut feeling, but the fact that we cannot use the latest googletest version is a strong indicator. So using the OS-provided version may be the wrong one, which may be a constant source of annoying build errors. Installation instructions, such as this apt command, must also be added. > (I'd appreciate feedback on what the package is called on RH/Fedora!) We > could also consider checking for an environment variable, similar to how > the boot JDK looks at JAVA_HOME, so you can set up a non-standard path in > your environment, and do not have to pass it to configure as a flag all the > time. > > And we also need to fix the RunTest framework, as Ren? pointed out, so > that if you try to run gtests without the gtest library, you need to get a > proper error message that describes the step you need to take to be able to > run gtest tests. > > /Magnus > > > Thanks, Thomas > In my mind, gtest is in the same ballpark as the freetype library on > Windows. That has always been a bit annoying but that was only Windows. > Something you cannot rely the OS library mechanism to deliver but have to > download and place yourself and tell the build about it. > > I wonder whether we can find a compromise somehow. Maybe something like > an agreed on structure and place for a "companion directory", containing > build dependencies like these. Its location could be by convention in a > clear relation to the OpenJDK sources (e.g. just beside it) and configure > would look for the libraries in there by default. Like this: > > - openjdk-source > - configure > - Makefile > - .. > - openjdk-builddeps > - google-test > - freetype > - .. > > This would also lend itself very well to a maintained repository of build > dependencies somewhere (not necessarily maintained by Oracle). We would > have similar ease of use as with platform libraries, which by default are > also expected in a standard place in the file system. > > Thanks, Thomas > > > On Fri, May 29, 2020 at 5:20 AM Igor Ignatyev > wrote: > >> Hi Thomas, >> >> I regret that this patch made you unhappier. I fully agree that all >> hotspot developers are to always build *and* run gtest tests, yet not all >> people who work on OpenJDK project are hotspot developers. >> >> I also believe that all OpenJDK developers are to run tests related to >> the area they are changing. The vast majority of such tests are jtreg >> tests. And jtreg, for better or worse, is a counterexample to your last >> paragraph -- it's an essential part of OpenJDK, but it doesn't come in a >> form of well established library, and none of known to me platforms have >> jtreg in their package manager, so you do have to manually download jtreg >> and specify its location one way or another. I understand that there is a >> difference b/w gtest and jtreg, as jtreg isn't really need at build time. >> However the proper way to run any of OpenJDK tests is by `make test` and >> for it to work you need to either executed configure w/ --with-jtreg or >> specify JT_HOME on each invocation of `make test`, where the latter is a >> preferred way. From that point of view, gtest and jtreg aren't different, >> you need to download both manually and specify their location at >> configure-time. >> >> with that being said, I truly understand the inconvenience it causes to >> you, yet given you need to download gtest only once and it's fairly easy to >> hide `with-gtest` behind a script or an alias like configure_openjdk='bash >> ./configure --with-gtest=$GTEST_HOME', I hope it won't cause problems for >> you and won't discourage you in anyways. >> >> Thanks, >> -- Igor >> >> On May 28, 2020, at 12:31 AM, Thomas St?fe >> wrote: >> >> Hi, >> >> I know the boat has sailed, I missed this one. But I am a bit unhappy >> about this. >> >> I always build with gtests; I think it makes no sense to not build with >> gtest. Even if you do not want to run the gtests (and it makes sense to >> always do since its a good first-base validity test), hotspot developers >> should always build them since changes in the hotspot sources may break >> hotspot gtests. hotspot source and gtests are a unity. >> >> So if it makes no sense to not build gtest, I should not need a special >> option to specify gtest location - I'd argue that the default case should >> not require special options. >> >> The JBS issue states that "it can be treated like any other external >> dependencies" but this comparison does not hold - almost all other >> dependencies come in the form of well established libraries with standard >> headers and standard installation locations which can be coded as default >> values. On a recent mainstream platform I do not have to specify any paths >> to libraries to build OpenJDK. This is now different, I always have to >> manually download gtests and specify gtest location. This is regrettable. >> >> Thanks, Thomas >> >> >> On Tue, May 26, 2020 at 2:27 PM Magnus Ihse Bursie < >> magnus.ihse.bursie at oracle.com> wrote: >> >>> >>> >>> On 2020-05-25 19:53, Igor Ignatyev wrote: >>> > Hi Magnus, >>> > >>> >> On May 25, 2020, at 7:46 AM, Magnus Ihse Bursie >>> >> >> >> > wrote: >>> >> >>> >> Looks basically good to me. >>> > thanks for your review! >>> >> >>> >> We need to document the dependency on gtest, and how to retrieve it. >>> >> I recommend you add a section next to the JTReg information (under >>> >> the "## Running Tests" heading) in doc/building.md. I think you >>> >> should include basically the same information as you did in your >>> >> follow-up mail, that was informative and clear. >>> > that's a good suggestion, I've added a small paragraph to 'Running >>> > Tests' in doc/building.md and regenerated corresponding .html file -- >>> > http://cr.openjdk.java.net/~iignatyev/8245610/webrev.doc/ >>> > please let me know if you think something should be added or reworded. >>> Looks great! Thank you. >>> >>> /Magnus >>> > >>> >> >>> >> I assume the most suitable replacement for developers who are used to >>> >> add a '--disable-hotspot-gtest' to e.g. a pre-definied jib >>> >> configuration is to now use '--without-gtest'. This will need to be >>> >> communicated, perhaps to both hotspot-dev and jdk-dev. >>> > sure, after this gets integrated, I'll let both hotspot-dev and >>> > jdk-dev aliases know which changes might be required to enable/disable >>> > hotspot gtest tests compilation. >>> > >>> > Thanks. >>> > -- Igor >>> > >>> >> >>> >> /Magnus >>> >> >>> >> On 2020-05-22 20:12, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00/ >>> >>>> 132 lines changed: 80 ins; 36 del; 16 mod >>> >>> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00%2bremoval/ >>> >>>> 57482 lines changed: 80 ins; 57385 del; 17 mod; >>> >>> Hi all, >>> >>> >>> >>> could you please review this small (if you ignore removal part) >>> >>> patch which removes in-tree copy of gtest (test/fmw/gtest) and >>> >>> updates makefiles to use one provided thru an added configure option >>> >>> `--with-gtest`? the previously used configure option >>> >>> `--enable-hotspot-gtest` gets depricated. >>> >>> >>> >>> the patch also compiles gtest and gmock source code into a static >>> >>> library so they can be compiled w/ all warnings disabled. >>> >>> >>> >>> from JBS: >>> >>>> w/ JEP 381 (JDK-8241787 / JDK-8244224) being integrated, all >>> >>>> compilers used by OpenJDK became supported by gtest out-of-box, so >>> >>>> there is no need to have our copy of gtest framework (including >>> >>>> gmock) within OpenJDK source tree. instead, it can be treated like >>> >>>> any other external dependencies, and a pointer to the directory w/ >>> >>>> gtest code can be passed via configure options. >>> >>> >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245610 >>> >>> webrevs: >>> >>> - http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00/ >>> >>> (meaningful changes) >>> >>> - >>> >>> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00%2bremoval/ >>> >>> (all changes) >>> >>> testing: >>> >>> - gtest tests on {linux,windows,macosx}-x64; >>> >>> - tier[1-5] builds which include but not limited to linux-aarch64, >>> >>> linux-arm32, linux-x64-zero >>> >>> >>> >>> Thanks, >>> >>> -- Igor >>> >>> >>> >> >>> > >>> >>> >> > From harold.seigel at oracle.com Tue Jun 2 14:30:58 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 2 Jun 2020 10:30:58 -0400 Subject: Fwd: Re: RFR: JDK-8227046: compiler implementation for sealed classes, JDK-8227047: Javadoc for sealed types and JDK-8227044: javax.lang.model for sealed classes In-Reply-To: <518ad004-fa16-178d-a9cf-c4e81d1b00d6@oracle.com> References: <518ad004-fa16-178d-a9cf-c4e81d1b00d6@oracle.com> Message-ID: Thanks to all the hotspot reviewers, also! Harold -------- Forwarded Message -------- Subject: Re: RFR: JDK-8227046: compiler implementation for sealed classes, JDK-8227047: Javadoc for sealed types and JDK-8227044: javax.lang.model for sealed classes Date: Tue, 2 Jun 2020 10:06:23 -0400 From: Vicente Romero To: compiler-dev at openjdk.java.net , Amber dev , javadoc-dev , core-libs-dev thanks to all the reviewers, the code for sealed classes has already been pushed to mainline. Vicente On 5/18/20 6:42 PM, Vicente Romero wrote: > Hi, > > Please review this patch for the compiler, javadoc and > javax.lang.model support for the JEP 360 Sealed Classes (Preview). The > changes are provided at [1], which implements the latest JLS for > sealed types [2]. The patch also include the needed changes to javadoc > and javax.lang.model to support sealed types. The CSR witht the > changes in the javax.lang.model spec is at [3]. The sealed types JEP > is accessible at [4]. There is an ongoing review for the VM and > core-libs code of sealed types [5] and that code hasn't been included > in this webrev, > > Thanks, > Vicente > > [1] http://cr.openjdk.java.net/~vromero/8227046/webrev.00/ > [2] > http://cr.openjdk.java.net/~gbierman/jep360/jep360-20200513/specs/sealed-classes-jls.html > > [3] https://bugs.openjdk.java.net/browse/JDK-8244367 > [4] https://openjdk.java.net/jeps/360 > [5] > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-May/066440.html From david.holmes at oracle.com Wed Jun 3 06:06:36 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Jun 2020 16:06:36 +1000 Subject: RFR[15] JDK-8243114: Implement montgomery{Multiply, Square}intrinsics on Windows In-Reply-To: <2442a046-6640-f759-6330-e971405f0c78@oracle.com> References: <725e8e5a-71b5-7d6e-4d3a-7c799a52ab56@redhat.com> <2442a046-6640-f759-6330-e971405f0c78@oracle.com> Message-ID: <6cb701cd-5652-6b9d-8051-b37adb3ce63f@oracle.com> Hi Simon, On 23/05/2020 12:04 am, Sean Mullan wrote: > Cross-posting to hotspot-dev for additional review since the code > changes are in hotspot. > > --Sean > > On 5/21/20 1:24 PM, Simon Tooke wrote: >> Hello, >> >> I'd like to request a review for: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8243114 >> >> Webrev: http://cr.openjdk.java.net/~stooke/webrevs/jdk-8243114-jdk/00/00/ >> >> This change implements the given intrinsics on Windows. >> >> The Windows toolchain has no 128 bit integer types, and no inline asm >> (on x64 and Arm).? In addition, 'long' is 4 bytes, not 8, as it is >> with gcc.? This patch had to change some of the linux implementation >> to account for these limitations. I can't really comment on the intrinsics themselves but a couple of suggestions: - use explicitly sized types e.g. uint64_t instead of unsigned long long - use the existing NOINLINE macro for the _declspec(noinline) The conditional compilation in this code has me quite confused. Looking at the existing code we have: 3680 #ifndef _WINDOWS 3681 3682 #define ASM_SUBTRACT 3683 3684 #ifdef ASM_SUBTRACT ... 3702 #else // ASM_SUBTRACT ... 3719 #endif // ! ASM_SUBTRACT So all the above code is only built on not-Windows, and we unconditionally define ASM_SUBTRACT, so lines 3702 to 3719 appear to be dead code! I'm not at all sure whether that is actually a bug and the windows ifndef should have had an endif after line 3682; or whether we can just simplify the code. Thanks, David >> >> My gratitude for Andrew Haley for doing the heavy lifting at the core >> of this patch. >> >> The patch, if accepted, will be offered to 11u as a potential >> backport. The changes apply cleanly modulo some line number changes. >> >> As for the speedup, this test case: >> >> BigInteger base = BigInteger.ONE.shiftLeft(1024); >> long count = LongStream.rangeClosed(2, 100_000) >> ???? .mapToObj(n -> BigInteger.valueOf(n).add(base)) >> ???? .filter(i -> i.isProbablePrime(50)) >> ???? .count(); >> >> goes from 1 minute 20 seconds down to about 35 seconds om my VM, over >> multiple runs.? As is the case on other platforms where the intrinsics >> are supported, they will be enabled by default on Windows. >> >> Thank you for your time, >> >> -Simon Tooke >> >> >> Principle Software Engineer, >> >> Red Hat Canada >> >> >> From stefan.karlsson at oracle.com Wed Jun 3 10:24:35 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 3 Jun 2020 12:24:35 +0200 Subject: RFR: 8246434: Threads::print_on_error assumes that the heap has been set up Message-ID: <6432e09b-24d2-14dc-240a-339674d46ead@oracle.com> Hi all, Please review this trivial patch to null check the access to the heap, when printing threads to the hs_err files. https://cr.openjdk.java.net/~stefank/8246434/webrev.01 https://bugs.openjdk.java.net/browse/JDK-8246434 Output from an initialization crash without this fix: --- Other Threads: [error occurred during error reporting (printing all threads), id 0xb, SIGSEGV (0xb) at pc=0x0000000104cbb609] --- with the fix: --- Other Threads: =>0x00007ff10980e220 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=7171, stack(0x00007000083dc000,0x00007000084dc000)] --- Thanks, StefanK From erik.osterlund at oracle.com Wed Jun 3 11:26:33 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 3 Jun 2020 13:26:33 +0200 Subject: RFR: 8246442: nmethod::can_convert_to_zombie() asserts when not called by the sweeper Message-ID: <4e9a0ca4-c55a-7bc6-309c-386fe3015dda@oracle.com> Hi, nmethod::can_convert_to_zombie() used to be called only by the sweeper, after it has checked that an nmethod is not entrant. Due to the single call site, there was an assert checking that the nmethod should be not entrant (the condition just checked). However, it is now used in nmethod::post_compiled_method_load_event(), where the result of the assert is racy. When an nmethod is created, the load event will never be able to see the nmethod spuriously flipping to zombie, because there simply are no safepoint polls/transitions between the nmethod being created and the load event firing. However, the jvmti_GenerateEvents can observe such spurious flips to zombie. It is completely harmless, because the whole operation runs under the CodeCache_lock, which prevents further nmethods from being flushed. But they can flip to zombie, which is indeed exactly what the check is design to test. Therefore, this is just a false positive assertion failure. Proposed fix: make the assert only apply when called from the sweeper thread. Webrev: http://cr.openjdk.java.net/~eosterlund/8246442/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8246442 Thanks, /Erik From david.holmes at oracle.com Wed Jun 3 11:37:02 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Jun 2020 21:37:02 +1000 Subject: RFR: 8246434: Threads::print_on_error assumes that the heap has been set up In-Reply-To: <6432e09b-24d2-14dc-240a-339674d46ead@oracle.com> References: <6432e09b-24d2-14dc-240a-339674d46ead@oracle.com> Message-ID: Looks good and trivial. Thanks, David On 3/06/2020 8:24 pm, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to null check the access to the heap, > when printing threads to the hs_err files. > > https://cr.openjdk.java.net/~stefank/8246434/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8246434 > > Output from an initialization crash without this fix: > --- > Other Threads: > > [error occurred during error reporting (printing all threads), id 0xb, > SIGSEGV (0xb) at pc=0x0000000104cbb609] > --- > > with the fix: > --- > Other Threads: > > =>0x00007ff10980e220 (exited) JavaThread "Unknown thread" > [_thread_in_vm, id=7171, stack(0x00007000083dc000,0x00007000084dc000)] > --- > > Thanks, > StefanK From david.holmes at oracle.com Wed Jun 3 11:40:08 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Jun 2020 21:40:08 +1000 Subject: RFR: 8246442: nmethod::can_convert_to_zombie() asserts when not called by the sweeper In-Reply-To: <4e9a0ca4-c55a-7bc6-309c-386fe3015dda@oracle.com> References: <4e9a0ca4-c55a-7bc6-309c-386fe3015dda@oracle.com> Message-ID: Hi Erik, LGTM. Your explanation and solution seems entirely reasonable. Thanks, David On 3/06/2020 9:26 pm, Erik ?sterlund wrote: > Hi, > > nmethod::can_convert_to_zombie() used to be called only by the sweeper, > after it has checked that > an nmethod is not entrant. Due to the single call site, there was an > assert checking that the nmethod > should be not entrant (the condition just checked). > However, it is now used in nmethod::post_compiled_method_load_event(), > where the result of the assert is racy. > > When an nmethod is created, the load event will never be able to see the > nmethod spuriously flipping > to zombie, because there simply are no safepoint polls/transitions > between the nmethod being created > and the load event firing. However, the jvmti_GenerateEvents can observe > such spurious flips to zombie. > It is completely harmless, because the whole operation runs under the > CodeCache_lock, which prevents > further nmethods from being flushed. But they can flip to zombie, which > is indeed exactly what the > check is design to test. Therefore, this is just a false positive > assertion failure. > > Proposed fix: make the assert only apply when called from the sweeper > thread. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8246442/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8246442 > > Thanks, > /Erik From stefan.karlsson at oracle.com Wed Jun 3 11:51:27 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 3 Jun 2020 13:51:27 +0200 Subject: RFR: 8246434: Threads::print_on_error assumes that the heap has been set up In-Reply-To: References: <6432e09b-24d2-14dc-240a-339674d46ead@oracle.com> Message-ID: Thanks, David. StefanK On 2020-06-03 13:37, David Holmes wrote: > Looks good and trivial. > > Thanks, > David > > On 3/06/2020 8:24 pm, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to null check the access to the >> heap, when printing threads to the hs_err files. >> >> https://cr.openjdk.java.net/~stefank/8246434/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8246434 >> >> Output from an initialization crash without this fix: >> --- >> Other Threads: >> >> [error occurred during error reporting (printing all threads), id >> 0xb, SIGSEGV (0xb) at pc=0x0000000104cbb609] >> --- >> >> with the fix: >> --- >> Other Threads: >> >> =>0x00007ff10980e220 (exited) JavaThread "Unknown thread" >> [_thread_in_vm, id=7171, stack(0x00007000083dc000,0x00007000084dc000)] >> --- >> >> Thanks, >> StefanK From erik.osterlund at oracle.com Wed Jun 3 12:28:16 2020 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Wed, 3 Jun 2020 14:28:16 +0200 Subject: RFR: 8246442: nmethod::can_convert_to_zombie() asserts when not called by the sweeper In-Reply-To: References: Message-ID: <339F942C-9A25-4E47-BA94-932394C6ED95@oracle.com> Hi David, Thanks for the review. /Erik > On 3 Jun 2020, at 13:40, David Holmes wrote: > > ?Hi Erik, > > LGTM. > > Your explanation and solution seems entirely reasonable. > > Thanks, > David > >> On 3/06/2020 9:26 pm, Erik ?sterlund wrote: >> Hi, >> nmethod::can_convert_to_zombie() used to be called only by the sweeper, after it has checked that >> an nmethod is not entrant. Due to the single call site, there was an assert checking that the nmethod >> should be not entrant (the condition just checked). >> However, it is now used in nmethod::post_compiled_method_load_event(), where the result of the assert is racy. >> When an nmethod is created, the load event will never be able to see the nmethod spuriously flipping >> to zombie, because there simply are no safepoint polls/transitions between the nmethod being created >> and the load event firing. However, the jvmti_GenerateEvents can observe such spurious flips to zombie. >> It is completely harmless, because the whole operation runs under the CodeCache_lock, which prevents >> further nmethods from being flushed. But they can flip to zombie, which is indeed exactly what the >> check is design to test. Therefore, this is just a false positive assertion failure. >> Proposed fix: make the assert only apply when called from the sweeper thread. >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8246442/webrev.00/ >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8246442 >> Thanks, >> /Erik From daniel.daugherty at oracle.com Wed Jun 3 13:31:14 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 3 Jun 2020 09:31:14 -0400 Subject: RFR: 8246442: nmethod::can_convert_to_zombie() asserts when not called by the sweeper In-Reply-To: <4e9a0ca4-c55a-7bc6-309c-386fe3015dda@oracle.com> References: <4e9a0ca4-c55a-7bc6-309c-386fe3015dda@oracle.com> Message-ID: <2fe85560-9325-8940-e365-2a11ea9d617f@oracle.com> On 6/3/20 7:26 AM, Erik ?sterlund wrote: > Hi, > > nmethod::can_convert_to_zombie() used to be called only by the > sweeper, after it has checked that > an nmethod is not entrant. Due to the single call site, there was an > assert checking that the nmethod > should be not entrant (the condition just checked). > However, it is now used in nmethod::post_compiled_method_load_event(), > where the result of the assert is racy. > > When an nmethod is created, the load event will never be able to see > the nmethod spuriously flipping > to zombie, because there simply are no safepoint polls/transitions > between the nmethod being created > and the load event firing. However, the jvmti_GenerateEvents can > observe such spurious flips to zombie. > It is completely harmless, because the whole operation runs under the > CodeCache_lock, which prevents > further nmethods from being flushed. But they can flip to zombie, > which is indeed exactly what the > check is design to test. Therefore, this is just a false positive > assertion failure. > > Proposed fix: make the assert only apply when called from the sweeper > thread. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8246442/webrev.00/ src/hotspot/share/code/nmethod.cpp ??? No comments. Thumbs up. I think Leonid filed a similar bug the other day: JDK-8245877 assert(_value != __null) failed: resolving NULL _value in JvmtiExport::post_compiled_method_load so I've added Serguei to this review thread since JDK-8245877 is assigned to him. Dan > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8246442 > > Thanks, > /Erik From sgehwolf at redhat.com Wed Jun 3 13:46:46 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 03 Jun 2020 15:46:46 +0200 Subject: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> Message-ID: Hi Matthias, On Fri, 2020-05-22 at 13:36 +0000, Baesken, Matthias wrote: > > Anyway, I think we should change the test code similar to this: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/01/webrev/ > > Hi Severin, this gave me : > > Checking OperatingSystemMXBean > Exception in thread "main" java.lang.IllegalAccessError: class CheckOperatingSystemMXBean (in unnamed module @0x5acf9800) cannot access class jdk.internal.platform.Container (in module java.base) because module java.base does not export jdk.internal.platform to unnamed module @0x5acf9800 > at CheckOperatingSystemMXBean.main(CheckOperatingSystemMXBean.java:33) > > (a bit unexpected because you added it to the test, I though this would be sufficient !?) OK, reproduced. FWIW, 01 was an untested patch ;-) Please try this updated webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/02/webrev/ It works fine here for cgroups v2 and cgroups v1. > (plus we still need Jie's adjustment to fix getFreeSwapSpaceSize ) Yes, but I'd rather fix this as a separate issue. I don't think it's related. Thanks, Severin > -----Original Message----- > From: Severin Gehwolf > Sent: Mittwoch, 20. Mai 2020 21:30 > To: Baesken, Matthias ; jiefu(??) < > jiefu at tencent.com>; David Holmes ; > 'hotspot-dev at openjdk.java.net' > Cc: Bob Vandette > Subject: Re: RFR [XS]: 8244500: jtreg test error in > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > Hi Matthias, > > On Wed, 2020-05-20 at 11:07 +0000, Baesken, Matthias wrote: > > Hi Severin , > > > > > FWIW, parts of this work-around got added by JDK-8236617. Looking > > > at > > > the bug it was observed that: > > > > > > containerMetrics.getMemoryAndSwapLimit() == 0 and > > > containerMetrics.getMemoryLimit() returned a large value. > > > > > > So then with JDK-8231111 I suspect > > > containerMetrics.getMemoryAndSwapLimit() now returns -1 and, > > > thus, > > > getTotalSwapSpaceSize0() is being used instead of the container > > > values. > > > > I added some tracing output > > to OperatingSystemImpl.getTotalSwapSpaceSize() and get these > > values : > > > > containerMetrics.getMemoryAndSwapLimit():-1 > > containerMetrics.getMemoryLimit():-1 > > ****** getTotalSwapSpaceSize0() returned 16106061824 > > Thanks for this. This seems to confirm my suspicion. > > So my working theory here is that the system where you are observing > the issue now is similar to the one when JDK-8236617 was discovered. > For some reason containerMetrics.getMemoryAndSwapLimit() returned 0 > for > the case when the necessary files were missing (which was wrong IMO). > After JDK-8231111 there is a clear distinction for this case: -1 is > being returned. That makes more sense to me, as there wouldn't be any > container limit to take into account. > > Your trace output seems to confirm that: > > containerMetrics.getMemoryAndSwapLimit():-1 > containerMetrics.getMemoryLimit():-1 > > > > So far the theory, I'd like to gather more data before we come up > > > with > > > another patch, though. In JDK-8236617 it wasn't investigated why > > > containerMetrics.getMemoryAndSwapLimit() returned 0 on such a > > > system. > > > It would be good to know why. Could it be that file > > > 'memory.memsw.limit_in_bytes' doesn't exist on such a system? > > > > True, it does not exist : > > > > ls -alL /sys/fs/cgroup/memory/ > > total 0 > > drwxr-xr-x 3 root root 0 Mar 16 11:00 . > > drwxr-xr-x 11 root root 260 Mar 16 11:00 .. > > -rw-r--r-- 1 root root 0 Mar 16 11:00 cgroup.clone_children > > --w--w--w- 1 root root 0 Mar 16 11:00 cgroup.event_control > > -rw-r--r-- 1 root root 0 May 20 13:00 cgroup.procs > > -r--r--r-- 1 root root 0 Mar 16 11:00 cgroup.sane_behavior > > drwxr-xr-x 2 root root 0 May 20 12:52 docker > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.failcnt > > --w------- 1 root root 0 Mar 16 11:00 memory.force_empty > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.limit_in_bytes > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.low_limit_in_bytes > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.max_usage_in_bytes > > -rw-r--r-- 1 root root 0 Mar 16 11:00 > > memory.move_charge_at_immigrate > > -r--r--r-- 1 root root 0 Mar 16 11:00 memory.numa_stat > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.oom_control > > ---------- 1 root root 0 Mar 16 11:00 memory.pressure_level > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.soft_limit_in_bytes > > -r--r--r-- 1 root root 0 Mar 16 11:00 memory.stat > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.swappiness > > -r--r--r-- 1 root root 0 Mar 16 11:00 memory.usage_in_bytes > > -rw-r--r-- 1 root root 0 May 20 13:00 memory.use_hierarchy > > -rw-r--r-- 1 root root 0 Mar 16 11:00 notify_on_release > > -rw-r--r-- 1 root root 0 Mar 16 11:00 release_agent > > -rw-r--r-- 1 root root 0 Mar 16 11:00 tasks > > Hmm, is this inside a container on this system with --memory and -- > memory-swap being specified on the docker command line? > > Anyway, I think we should change the test code similar to this: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/01/webrev/ > > If this works for you, then that's what I'd suggest as a fix. > > Note that I haven't tested this patch. I need to recreate my set-up > for > proper docker testing here :-/ > > Thanks, > Severin > > > Best regards, Matthias > > > > > > > > -----Original Message----- > > From: Severin Gehwolf > > Sent: Montag, 18. Mai 2020 12:09 > > To: jiefu(??) ; David Holmes < > > david.holmes at oracle.com>; Baesken, Matthias < > > matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' < > > hotspot-dev at openjdk.java.net> > > Cc: Bob Vandette > > Subject: Re: RFR [XS]: 8244500: jtreg test error in > > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > > > Hi, > > > > On Mon, 2020-05-18 at 06:15 +0000, jiefu(??) wrote: > > > Hi David, > > > > > > This bug should be introduced by JDK-8242480. > > > I'm sorry for that. > > > > > > And I think the logic in > > > src/jdk.management/unix/classes/com/sun/management/internal/Opera > > > tingSystemImpl.java should be fixed. > > > > > > In my fix for JDK-8242480, I had ignored the case for kernels > > > which don't support "--memory-swap 150M". > > > In that case, containerMetrics.getMemoryAndSwapLimit()[1] will > > > return -1. > > > > It appears so that containerMetrics.getMemoryAndSwapLimit() > > returned 0 > > prior JDK-8231111 and -1 after. I'm not convinced JDK-8242480 > > introduced this. > > > > > I didn't notice this bug because all our kernels support -- > > > memory-swap. > > > Fortunately, Matthias found this bug and will fix it. > > > > I'm still not sure what's happening in Matthias' case on that > > system. > > It would be best to first try to understand the actual problem > > better. > > The info I have is: > > > > - It's a cgroup v1 system > > - The system now returns large values for > > OperatingSystemMXBean.getTotalSwapSpaceSize() and > > OperatingSystemMXBean.getFreeSwapSpaceSize() > > > > It would be good to know what the actual return values are for the > > container metrics on this system with swap limit disabled at kernel > > level. And also whether or not relevant cgroup files exist on such > > a > > system (within docker). > > > > FWIW, parts of this work-around got added by JDK-8236617. Looking > > at > > the bug it was observed that: > > > > containerMetrics.getMemoryAndSwapLimit() == 0 and > > containerMetrics.getMemoryLimit() returned a large value. > > > > So then with JDK-8231111 I suspect > > containerMetrics.getMemoryAndSwapLimit() now returns -1 and, thus, > > getTotalSwapSpaceSize0() is being used instead of the container > > values. > > > > So far the theory, I'd like to gather more data before we come up > > with > > another patch, though. In JDK-8236617 it wasn't investigated why > > containerMetrics.getMemoryAndSwapLimit() returned 0 on such a > > system. > > It would be good to know why. Could it be that file > > 'memory.memsw.limit_in_bytes' doesn't exist on such a system? > > > > Unfortunately, I don't know what config one needs in order to > > reproduce > > so we'll have to rely on Matthias to gather the info. > > > > Thanks, > > Severin > > > > > Thanks a lot. > > > Best regards, > > > Jie > > > > > > [1] > > > http://hg.openjdk.java.net/jdk/jdk/file/a95911422005/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l73 > > > > > > ?On 2020/5/18, 1:48 PM, "David Holmes" > > > wrote: > > > > > > Hi Matthias, > > > > > > On 16/05/2020 12:59 am, Baesken, Matthias wrote: > > > > Hello David , I get the warning always on this linux > > > machine (I think it depends on machine setup and/or > > > kernel version). > > > > > > > >> Surely it must be specified to return 0 or some such > > > sentinel in such a case? > > > > > > > > The doc just says this : > > > > > > > > 62 /** > > > > 63 * Returns the total amount of swap space in bytes. > > > > 64 * > > > > 65 * @return the total amount of swap space in bytes. > > > > 66 */ > > > > 67 public long getTotalSwapSpaceSize(); > > > > > > > > ( however it returned previously 0 before the latest > > > changes in the container support code came in ). > > > > > > That sounds like a bug was introduced to me then. > > > > > > > > > > Regarding the getFreeSwapSpaceSize issue , Jie provided a > > > change > > > to src/jdk.management/unix/classes/com/sun/management/internal/O > > > peratingSystemImpl.java > > > > Because of this I could remove the try-construct . > > > > > > > > New webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8244500.3/ > > > > > > I'm quite confused by your changes. Your first version simply > > > adjusted > > > the test to check for the warning. But now you are also > > > changing the > > > main logic. I can't assess the validity of what you are doing > > > sorry. > > > > > > I've cc'd Severin and Bob as I think they need to evaluate > > > what the real > > > problem is here and when it was introduced. > > > > > > David > > > ---- > > > > > > > Best regards, Matthias > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: David Holmes > > > > Sent: Donnerstag, 14. Mai 2020 07:02 > > > > To: Baesken, Matthias ; > > > 'hotspot-dev at openjdk.java.net' > > > > Cc: jiefu(??) > > > > Subject: Re: RFR [XS]: 8244500: jtreg test error in > > > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > > > > > > > On 14/05/2020 12:28 am, Baesken, Matthias wrote: > > > >> Hello, here is a new webrev removing the comment : > > > >> > > > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8244500.1/ > > > >> > > > >> It removes the comment , and it also handles 0 for > > > getFreeSwapSpaceSize > > > >> ( this can occur after JDK-8242480 : Negative value > > > may be returned by getFreeSwapSpaceSize() in the docker was > > > pushed recently ). > > > > > > > > Sorry Matthias but I still see this partly as a bug in the > > > underlying > > > > container support code. If you get the warning then what > > > does > > > > getTotalSwapSpaceSize() return? Surely it must be specified > > > to return 0 > > > > or some such sentinel in such a case? > > > > > > > > Other comments: > > > > > > > > ! try { > > > > ! > > > > > > > out.shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: > > > > [1-9][0-9]+"); > > > > ! } catch(RuntimeException ex) { // allow 0, see > > > 8242480 > > > > ! > > > > > > > out.shouldContain("OperatingSystemMXBean.getFreeSwapSpaceSize: > > > 0"); > > > > ! } > > > > > > > > can't you just change the original regex to match >= 0? > > > > > > > > Style nit: avoid historical comments like "see 8242480" and > > > "since > > > > 8231111 ...". > > > > > > > > Thanks, > > > > David > > > > > > > >> Best regards, Matthias > > > >> > > > >> -----Original Message----- > > > >> From: Baesken, Matthias > > > >> Sent: Mittwoch, 13. Mai 2020 10:11 > > > >> To: 'David Holmes' ; > > > 'hotspot-dev at openjdk.java.net' > > > >> Subject: RE: RFR [XS]: 8244500: jtreg test error in > > > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > > >> > > > >> > > > >>> But according to the comment immediately before your > > > changes: > > > >>> > > > >>> // in case of warnings like : "Your kernel does not > > > support swap limit > > > >>> capabilities or the cgroup is not mounted. Memory limited > > > without swap." > > > >>> // the getTotalSwapSpaceSize does not return the expected > > > result, but 0 > > > >>> > > > >>> we should be returning zero. So this seems to be a bug in > > > the > > > >>> implementation not in the test. > > > >> > > > >> > > > >> Hi David , I think in case of "Your kernel does not > > > support swap limit capabilities ... " we just do not get > > > the values > > > >> that are normally expected. > > > >> Previously we indeed got "0" . > > > >> Probably the comment should be deleted . > > > >> > > > >> Best regards, Matthias > > > >> > > > > > > > > > From erik.osterlund at oracle.com Wed Jun 3 14:08:03 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 3 Jun 2020 16:08:03 +0200 Subject: RFR: 8246442: nmethod::can_convert_to_zombie() asserts when not called by the sweeper In-Reply-To: <2fe85560-9325-8940-e365-2a11ea9d617f@oracle.com> References: <4e9a0ca4-c55a-7bc6-309c-386fe3015dda@oracle.com> <2fe85560-9325-8940-e365-2a11ea9d617f@oracle.com> Message-ID: <418878dd-8d92-f6dc-54fe-9b8108110bdf@oracle.com> Hi Dan, Thanks for the review. /Erik On 2020-06-03 15:31, Daniel D. Daugherty wrote: > On 6/3/20 7:26 AM, Erik ?sterlund wrote: >> Hi, >> >> nmethod::can_convert_to_zombie() used to be called only by the >> sweeper, after it has checked that >> an nmethod is not entrant. Due to the single call site, there was an >> assert checking that the nmethod >> should be not entrant (the condition just checked). >> However, it is now used in >> nmethod::post_compiled_method_load_event(), where the result of the >> assert is racy. >> >> When an nmethod is created, the load event will never be able to see >> the nmethod spuriously flipping >> to zombie, because there simply are no safepoint polls/transitions >> between the nmethod being created >> and the load event firing. However, the jvmti_GenerateEvents can >> observe such spurious flips to zombie. >> It is completely harmless, because the whole operation runs under the >> CodeCache_lock, which prevents >> further nmethods from being flushed. But they can flip to zombie, >> which is indeed exactly what the >> check is design to test. Therefore, this is just a false positive >> assertion failure. >> >> Proposed fix: make the assert only apply when called from the sweeper >> thread. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8246442/webrev.00/ > > src/hotspot/share/code/nmethod.cpp > ??? No comments. > > Thumbs up. > > I think Leonid filed a similar bug the other day: > > JDK-8245877 assert(_value != __null) failed: resolving NULL _value in > JvmtiExport::post_compiled_method_load > > so I've added Serguei to this review thread since JDK-8245877 > is assigned to him. > > Dan > > > >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8246442 >> >> Thanks, >> /Erik > From vladimir.kozlov at oracle.com Wed Jun 3 15:20:34 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 3 Jun 2020 08:20:34 -0700 Subject: RFR: 8246442: nmethod::can_convert_to_zombie() asserts when not called by the sweeper In-Reply-To: References: <4e9a0ca4-c55a-7bc6-309c-386fe3015dda@oracle.com> Message-ID: <32487f5b-dc1e-ab44-fff5-fc72cf91eac1@oracle.com> +1 Thanks, Vladimir On 6/3/20 4:40 AM, David Holmes wrote: > Hi Erik, > > LGTM. > > Your explanation and solution seems entirely reasonable. > > Thanks, > David > > On 3/06/2020 9:26 pm, Erik ?sterlund wrote: >> Hi, >> >> nmethod::can_convert_to_zombie() used to be called only by the sweeper, after it has checked that >> an nmethod is not entrant. Due to the single call site, there was an assert checking that the nmethod >> should be not entrant (the condition just checked). >> However, it is now used in nmethod::post_compiled_method_load_event(), where the result of the assert is racy. >> >> When an nmethod is created, the load event will never be able to see the nmethod spuriously flipping >> to zombie, because there simply are no safepoint polls/transitions between the nmethod being created >> and the load event firing. However, the jvmti_GenerateEvents can observe such spurious flips to zombie. >> It is completely harmless, because the whole operation runs under the CodeCache_lock, which prevents >> further nmethods from being flushed. But they can flip to zombie, which is indeed exactly what the >> check is design to test. Therefore, this is just a false positive assertion failure. >> >> Proposed fix: make the assert only apply when called from the sweeper thread. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8246442/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8246442 >> >> Thanks, >> /Erik From coleen.phillimore at oracle.com Wed Jun 3 17:23:26 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 3 Jun 2020 13:23:26 -0400 Subject: RFR (M) 8246340: Move SystemDictionary GC roots into OopStorage Message-ID: Summary: use vm_global() OopStorage for system dictionary roots removed SystemDictionary::oops_do in all the GCs. Tested with tier1-6. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8246340.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8246340 Thanks, Coleen From vkempik at azul.com Wed Jun 3 17:38:53 2020 From: vkempik at azul.com (Vladimir Kempik) Date: Wed, 3 Jun 2020 17:38:53 +0000 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> Message-ID: Hello Can somebody please review this simple change ? Thanks > 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): > > Adding hotspot-runtime-dev > >> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik ???????(?): >> >> >> Hello >> Please review a fix for JDK-8243470 >> >> Long time ago as part of JEP284: New HotSpot Build System this fix was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >> At that time it was decided to lower optimisation level for unsafe.cpp from O2 to O1 only for clang on Macosx. >> >> I suppose it was done due to issues in Set/Get helper functions where too optimistic optimisations were eliminating some null pointer checks. it was probably a clang bug. >> That issue could be checked with test jdk/test/sun/misc/CopyMemory.java. >> >> I believe that workaround (going from O2 to O1) produced this issue - https://bugs.openjdk.java.net/browse/JDK-8234963 (Thread.getStackTrace is slow with clang). >> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >> >> Here I propose the patch which eliminates that workaround for clang 8+. >> >> I have tested clang versions 8/9/9.1/10, all of them showed good results: >> 1) CopyMemory test passes fine on 11/14/15. >> 2) jdk11/jdk14 passed tck. Regression testing were good as well. jdk15: no new failures in tck. >> 3) The testRandom "benchmark" from 8234963 shows great improvements on my machine, going down from ~200 seconds to ~150 seconds (the newer clang the better result). For comparision, gcc built libjvm for jdk11 shows ~130 seconds on my machine. >> >> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >> >> getStackTrace benchmark: http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >> >> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >> >> Thanks, Vladimir > From daniel.daugherty at oracle.com Wed Jun 3 17:47:19 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 3 Jun 2020 13:47:19 -0400 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> Message-ID: Adding build-dev at ... since this is a build system change. As for the right HotSpot team, I'm not sure who should be making the call on changing the way unsafe.cpp is built for macOSX. My guess when I moved the bug to hotspot/runtime was that someone on the Runtime team would know, but... Dan On 6/3/20 1:38 PM, Vladimir Kempik wrote: > Hello > > Can somebody please review this simple change ? > > Thanks > >> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >> >> Adding hotspot-runtime-dev >> >>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik ???????(?): >>> >>> >>> Hello >>> Please review a fix for JDK-8243470 >>> >>> Long time ago as part of JEP284: New HotSpot Build System this fix was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>> At that time it was decided to lower optimisation level for unsafe.cpp from O2 to O1 only for clang on Macosx. >>> >>> I suppose it was done due to issues in Set/Get helper functions where too optimistic optimisations were eliminating some null pointer checks. it was probably a clang bug. >>> That issue could be checked with test jdk/test/sun/misc/CopyMemory.java. >>> >>> I believe that workaround (going from O2 to O1) produced this issue - https://bugs.openjdk.java.net/browse/JDK-8234963 (Thread.getStackTrace is slow with clang). >>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>> >>> Here I propose the patch which eliminates that workaround for clang 8+. >>> >>> I have tested clang versions 8/9/9.1/10, all of them showed good results: >>> 1) CopyMemory test passes fine on 11/14/15. >>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. jdk15: no new failures in tck. >>> 3) The testRandom "benchmark" from 8234963 shows great improvements on my machine, going down from ~200 seconds to ~150 seconds (the newer clang the better result). For comparision, gcc built libjvm for jdk11 shows ~130 seconds on my machine. >>> >>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>> >>> getStackTrace benchmark: http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>> >>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>> >>> Thanks, Vladimir From magnus.ihse.bursie at oracle.com Wed Jun 3 18:07:16 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 3 Jun 2020 20:07:16 +0200 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> Message-ID: Thanks for cc:ing us. This is not the correct way to check for compiler versions. Nor is it the correct place. I don't have time for a full reply tonight, but will return with a better reply tomorrow. /Magnus On 2020-06-03 19:47, Daniel D. Daugherty wrote: > Adding build-dev at ... since this is a build system change. > > As for the right HotSpot team, I'm not sure who should be making > the call on changing the way unsafe.cpp is built for macOSX. My guess > when I moved the bug to hotspot/runtime was that someone on the Runtime > team would know, but... > > Dan > > > > On 6/3/20 1:38 PM, Vladimir Kempik wrote: >> Hello >> >> Can somebody please review this simple change ? >> >> Thanks >> >>> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >>> >>> Adding hotspot-runtime-dev >>> >>>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik >>>> ???????(?): >>>> >>>> >>>> Hello >>>> Please review a fix for JDK-8243470 >>>> >>>> Long time ago as part of JEP284: New HotSpot Build System this fix >>>> was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>>> At that time it was decided to lower optimisation level for >>>> unsafe.cpp from O2 to O1 only for clang on Macosx. >>>> >>>> I suppose it was done due to issues in Set/Get helper >>>> functions where too optimistic optimisations were eliminating some >>>> null pointer checks. it was probably a clang bug. >>>> That issue could be checked with test >>>> jdk/test/sun/misc/CopyMemory.java. >>>> >>>> I believe that workaround (going from O2 to O1) produced this issue >>>> - https://bugs.openjdk.java.net/browse/JDK-8234963 >>>> (Thread.getStackTrace is slow with clang). >>>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>>> >>>> Here I propose the patch which eliminates that workaround for clang >>>> 8+. >>>> >>>> I have tested clang versions 8/9/9.1/10, all of them showed good >>>> results: >>>> 1) CopyMemory test passes fine on 11/14/15. >>>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. >>>> jdk15: no new failures in tck. >>>> 3) The testRandom "benchmark" from 8234963 shows great improvements >>>> on my machine, going down from ~200 seconds to ~150 seconds (the >>>> newer clang the better result). For comparision, gcc built libjvm >>>> for jdk11 shows ~130 seconds on my machine. >>>> >>>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>>> >>>> getStackTrace benchmark: >>>> http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>>> >>>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>>> >>>> Thanks, Vladimir > From gerard.ziemski at oracle.com Wed Jun 3 19:59:37 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Wed, 3 Jun 2020 14:59:37 -0500 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> Message-ID: <5fabaaee-dc10-0fe5-823a-72295d932559@oracle.com> hi Vladimir, I will take a look to see if I can help review this... On 6/3/20 12:38 PM, Vladimir Kempik wrote: > Hello > > Can somebody please review this simple change ? > > Thanks > >> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >> >> Adding hotspot-runtime-dev >> >>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik ???????(?): >>> >>> >>> Hello >>> Please review a fix for JDK-8243470 >>> >>> Long time ago as part of JEP284: New HotSpot Build System this fix was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>> At that time it was decided to lower optimisation level for unsafe.cpp from O2 to O1 only for clang on Macosx. >>> >>> I suppose it was done due to issues in Set/Get helper functions where too optimistic optimisations were eliminating some null pointer checks. it was probably a clang bug. >>> That issue could be checked with test jdk/test/sun/misc/CopyMemory.java. >>> >>> I believe that workaround (going from O2 to O1) produced this issue - https://bugs.openjdk.java.net/browse/JDK-8234963 (Thread.getStackTrace is slow with clang). >>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>> >>> Here I propose the patch which eliminates that workaround for clang 8+. >>> >>> I have tested clang versions 8/9/9.1/10, all of them showed good results: >>> 1) CopyMemory test passes fine on 11/14/15. >>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. jdk15: no new failures in tck. >>> 3) The testRandom "benchmark" from 8234963 shows great improvements on my machine, going down from ~200 seconds to ~150 seconds (the newer clang the better result). For comparision, gcc built libjvm for jdk11 shows ~130 seconds on my machine. >>> >>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>> >>> getStackTrace benchmark: http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>> >>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>> >>> Thanks, Vladimir From igor.ignatyev at oracle.com Wed Jun 3 21:30:52 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 3 Jun 2020 14:30:52 -0700 Subject: RFR(S) : 8246494 : introduce vm.flagless at-requires property Message-ID: http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 > 70 lines changed: 66 ins; 0 del; 4 mod Hi all, could you please review the patch which introduces a new @requires property to filter out the tests which ignore externally provided JVM flags? the idea behind this patch is to have a way to clearly mark tests which ignore flags, so a) it's obvious that they don't execute a flag-guarded code/feature, and extra care should be taken to use them to verify any flag-guarded changed; b) they can be easily excluded from runs w/ flags. @requires and VMProps allows us to achieve both, so it's been decided to add a new property `vm.flagless`. `vm.flagless` is set to false if there are any XX flags other than `-XX:MaxRAMPercentage` and `-XX:CreateCoredumpOnCrash` (which are known to be set almost always) or any X flags other `-Xmixed`; in other words any tests w/ `@requires vm.flagless` will be excluded from runs w/ any other X / XX flags passed via `-vmoption` / `-javaoption`. in rare cases, when one still wants to run the tests marked by `vm.flagless` w/ external flags, `vm.flagless` can be forcefully set to true by setting any value to `TEST_VM_FLAGLESS` env. variable. this patch adds necessary common changes and marks common tests, namely Scimark, GTestWrapper and TestNativeProcessBuilder. Component-specific tests will be marked separately by the corresponding subtasks of 8151707[1]. please note, the patch depends on CODETOOLS-7902336[2], which will be included in the next jtreg version, so this patch is to be integrated only after jtreg5.1 is promoted and we switch to use it by 8246387[3]. JBS: https://bugs.openjdk.java.net/browse/JDK-8246494 webrev: http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 testing: marked tests w/ different XX and X flags w/ and w/o TEST_VM_FLAGLESS env. var, and w/o any flags [1] https://bugs.openjdk.java.net/browse/JDK-8151707 [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7902336 [3] https://bugs.openjdk.java.net/browse/JDK-8246387 Thanks, -- Igor From david.holmes at oracle.com Wed Jun 3 23:02:28 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Jun 2020 09:02:28 +1000 Subject: RFR(S) : 8246494 : introduce vm.flagless at-requires property In-Reply-To: References: Message-ID: <4584c046-ed5b-e1b9-f16b-3d4383cf1001@oracle.com> Hi Igor, On 4/06/2020 7:30 am, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 >> 70 lines changed: 66 ins; 0 del; 4 mod > > Hi all, > > could you please review the patch which introduces a new @requires property to filter out the tests which ignore externally provided JVM flags? > > the idea behind this patch is to have a way to clearly mark tests which ignore flags, so > a) it's obvious that they don't execute a flag-guarded code/feature, and extra care should be taken to use them to verify any flag-guarded changed; > b) they can be easily excluded from runs w/ flags. So all such tests should be using driver mode, and further the VMs they then exec don't use any of the APIs that include the jtreg test arguments. Okay this seems reasonable in what it does. Thanks, David > @requires and VMProps allows us to achieve both, so it's been decided to add a new property `vm.flagless`. `vm.flagless` is set to false if there are any XX flags other than `-XX:MaxRAMPercentage` and `-XX:CreateCoredumpOnCrash` (which are known to be set almost always) or any X flags other `-Xmixed`; in other words any tests w/ `@requires vm.flagless` will be excluded from runs w/ any other X / XX flags passed via `-vmoption` / `-javaoption`. in rare cases, when one still wants to run the tests marked by `vm.flagless` w/ external flags, `vm.flagless` can be forcefully set to true by setting any value to `TEST_VM_FLAGLESS` env. variable. > > this patch adds necessary common changes and marks common tests, namely Scimark, GTestWrapper and TestNativeProcessBuilder. Component-specific tests will be marked separately by the corresponding subtasks of 8151707[1]. > > please note, the patch depends on CODETOOLS-7902336[2], which will be included in the next jtreg version, so this patch is to be integrated only after jtreg5.1 is promoted and we switch to use it by 8246387[3]. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8246494 > webrev: http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 > testing: marked tests w/ different XX and X flags w/ and w/o TEST_VM_FLAGLESS env. var, and w/o any flags > > [1] https://bugs.openjdk.java.net/browse/JDK-8151707 > [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7902336 > [3] https://bugs.openjdk.java.net/browse/JDK-8246387 > > Thanks, > -- Igor > From igor.ignatyev at oracle.com Thu Jun 4 01:05:07 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 3 Jun 2020 18:05:07 -0700 Subject: RFR(S) : 8246494 : introduce vm.flagless at-requires property In-Reply-To: <4584c046-ed5b-e1b9-f16b-3d4383cf1001@oracle.com> References: <4584c046-ed5b-e1b9-f16b-3d4383cf1001@oracle.com> Message-ID: <5430D545-BE0C-4022-9468-D6EAFF7BAC78@oracle.com> Hi David, > So all such tests should be using driver mode, and further the VMs they then exec don't use any of the APIs that include the jtreg test arguments. correct, and 8151707's subtasks are going to mark only such tests (and tests which should be using driver-mode, but can't due to external factors, remember these follow-up fixes for my use driver-mode? ;) ). there are two more (a bit controversial) use cases where we can consider usage of vm.flagless: - some of debugger-debuggee tests have debugger executed w/ external flags, but don't pass these flags to debuggee; and in most cases, it doesn't seem to be right, so arguable all such tests should be updated to use driver mode to run debugger and then marked w/ vm.flagless. I know that svc team was doing some cleanup in this area recently, and given it's require more investigation w.r.t the tests' intent, I don't plan to do it as a part of 8151707, and instead will create follow up RFEs/tasks. - a unit-like tests which don't ignore flags, but weren't designed to be run w/ external flags; most of jfr tests can be used as an example: you can run w/ any flags, but they might fail as they assert things which happen only in certain configurations and these configurations are written in jtreg test descriptions. currently, these tests are marked w/ jfr k/w and it's advised not to run them w/ any external flags, yet I know that some people successfully do that to test their configurations. given the set of configurations which satisfies needs of jfr tests is much bigger than the configurations listed in the tests, I kinda feel sympathetic to people doing that, on the other hand, it's unsupported and I'd prefer us to express (and enforce) that more clearly. again, given the possible controversiality and need for a broader discussion, I'm planning to file an issue for jfr tests and follow up later w/ interested parties. to sum up, 8151707's subtasks are going to mark *only* obvious and non-controversial cases. for all other cases, the JBS entries are to be filed and followed up on. Cheers, -- Igor > On Jun 3, 2020, at 4:02 PM, David Holmes wrote: > > Hi Igor, > > On 4/06/2020 7:30 am, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 >>> 70 lines changed: 66 ins; 0 del; 4 mod >> Hi all, >> could you please review the patch which introduces a new @requires property to filter out the tests which ignore externally provided JVM flags? >> the idea behind this patch is to have a way to clearly mark tests which ignore flags, so >> a) it's obvious that they don't execute a flag-guarded code/feature, and extra care should be taken to use them to verify any flag-guarded changed; >> b) they can be easily excluded from runs w/ flags. > > So all such tests should be using driver mode, and further the VMs they then exec don't use any of the APIs that include the jtreg test arguments. > > Okay this seems reasonable in what it does. > > Thanks, > David > >> @requires and VMProps allows us to achieve both, so it's been decided to add a new property `vm.flagless`. `vm.flagless` is set to false if there are any XX flags other than `-XX:MaxRAMPercentage` and `-XX:CreateCoredumpOnCrash` (which are known to be set almost always) or any X flags other `-Xmixed`; in other words any tests w/ `@requires vm.flagless` will be excluded from runs w/ any other X / XX flags passed via `-vmoption` / `-javaoption`. in rare cases, when one still wants to run the tests marked by `vm.flagless` w/ external flags, `vm.flagless` can be forcefully set to true by setting any value to `TEST_VM_FLAGLESS` env. variable. >> this patch adds necessary common changes and marks common tests, namely Scimark, GTestWrapper and TestNativeProcessBuilder. Component-specific tests will be marked separately by the corresponding subtasks of 8151707[1]. >> please note, the patch depends on CODETOOLS-7902336[2], which will be included in the next jtreg version, so this patch is to be integrated only after jtreg5.1 is promoted and we switch to use it by 8246387[3]. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8246494 >> webrev: http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 >> testing: marked tests w/ different XX and X flags w/ and w/o TEST_VM_FLAGLESS env. var, and w/o any flags >> [1] https://bugs.openjdk.java.net/browse/JDK-8151707 >> [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7902336 >> [3] https://bugs.openjdk.java.net/browse/JDK-8246387 >> Thanks, >> -- Igor From coleen.phillimore at oracle.com Thu Jun 4 11:39:50 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Jun 2020 07:39:50 -0400 Subject: RFR (M) 8246340: Move SystemDictionary GC roots into OopStorage In-Reply-To: References: Message-ID: <3e329509-af13-b44b-e730-7f9bcf6d521c@oracle.com> Offline, Erik pointed out that I missed a system_dictionary tag in parallelGC and to not initialize OopStorage static data members, so here's an incremental that I reran tier1 tests on. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8246340.02.incr/webrev Thanks, Coleen On 6/3/20 1:23 PM, coleen.phillimore at oracle.com wrote: > Summary: use vm_global() OopStorage for system dictionary roots > removed SystemDictionary::oops_do in all the GCs. > > Tested with tier1-6. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8246340.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8246340 > > Thanks, > Coleen From magnus.ihse.bursie at oracle.com Thu Jun 4 13:04:04 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 4 Jun 2020 15:04:04 +0200 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> Message-ID: On 2020-06-03 20:07, Magnus Ihse Bursie wrote: > Thanks for cc:ing us. > > This is not the correct way to check for compiler versions. Nor is it > the correct place. > > I don't have time for a full reply tonight, but will return with a > better reply tomorrow. Ok, I have now looked more into this. You are apparently want to check for Xcode version, not clang version. This is not the same. Note that clang can be used on Linux as well. Apple is basically dropping the normal clang versioning and replacing it with their own, messed-up version which somewhat resembles the corresponding Xcode version, but is not identical. Horray. :-/ (For those interested, have a look at https://gist.github.com/yamaya/2924292.) Anyway. We do not support Xcode prior to 8. You seem to have tested from 8 and up, so this patch should be applied unconditionally. That is, just remove the -O1 special thing. /Magnus > > /Magnus > > On 2020-06-03 19:47, Daniel D. Daugherty wrote: >> Adding build-dev at ... since this is a build system change. >> >> As for the right HotSpot team, I'm not sure who should be making >> the call on changing the way unsafe.cpp is built for macOSX. My guess >> when I moved the bug to hotspot/runtime was that someone on the Runtime >> team would know, but... >> >> Dan >> >> >> >> On 6/3/20 1:38 PM, Vladimir Kempik wrote: >>> Hello >>> >>> Can somebody please review this simple change ? >>> >>> Thanks >>> >>>> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >>>> >>>> Adding hotspot-runtime-dev >>>> >>>>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik >>>>> ???????(?): >>>>> >>>>> >>>>> Hello >>>>> Please review a fix for JDK-8243470 >>>>> >>>>> Long time ago as part of JEP284: New HotSpot Build System this fix >>>>> was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>>>> At that time it was decided to lower optimisation level for >>>>> unsafe.cpp from O2 to O1 only for clang on Macosx. >>>>> >>>>> I suppose it was done due to issues in Set/Get helper >>>>> functions where too optimistic optimisations were eliminating some >>>>> null pointer checks. it was probably a clang bug. >>>>> That issue could be checked with test >>>>> jdk/test/sun/misc/CopyMemory.java. >>>>> >>>>> I believe that workaround (going from O2 to O1) produced this >>>>> issue - https://bugs.openjdk.java.net/browse/JDK-8234963 >>>>> (Thread.getStackTrace is slow with clang). >>>>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>>>> >>>>> Here I propose the patch which eliminates that workaround for >>>>> clang 8+. >>>>> >>>>> I have tested clang versions 8/9/9.1/10, all of them showed good >>>>> results: >>>>> 1) CopyMemory test passes fine on 11/14/15. >>>>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. >>>>> jdk15: no new failures in tck. >>>>> 3) The testRandom "benchmark" from 8234963 shows great >>>>> improvements on my machine, going down from ~200 seconds to ~150 >>>>> seconds (the newer clang the better result). For comparision, gcc >>>>> built libjvm for jdk11 shows ~130 seconds on my machine. >>>>> >>>>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>>>> >>>>> getStackTrace benchmark: >>>>> http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>>>> >>>>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>>>> >>>>> Thanks, Vladimir >> > From aph at redhat.com Thu Jun 4 14:08:45 2020 From: aph at redhat.com (Andrew Haley) Date: Thu, 4 Jun 2020 15:08:45 +0100 Subject: RFR[15] JDK-8243114: Implement montgomery{Multiply, Square}intrinsics on Windows In-Reply-To: References: <725e8e5a-71b5-7d6e-4d3a-7c799a52ab56@redhat.com> <2442a046-6640-f759-6330-e971405f0c78@oracle.com> <6cb701cd-5652-6b9d-8051-b37adb3ce63f@oracle.com> Message-ID: On 04/06/2020 14:35, Simon Tooke wrote: > Yes, this hurt to type.? A previous review suggested using julong, is > that acceptable to you? > > (an aside: I'm now wondering if there is other code in the JDK that > assumes long is 64bits - which is not true on all platforms...) That was just me, I think. I knew that this code would take some fixing up on Windows so I postponed portability issues until then. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From matthias.baesken at sap.com Thu Jun 4 14:11:59 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 4 Jun 2020 14:11:59 +0000 Subject: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> Message-ID: HI Severin, with your new patch I run into : java.lang.RuntimeException: 'OperatingSystemMXBean.getTotalSwapSpaceSize: [1-9][0-9]+' missing from stdout/stderr at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:189) at TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMemoryAwareness.java:165) at TestMemoryAwareness.main(TestMemoryAwareness.java:66) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) at java.base/java.lang.Thread.run(Thread.java:832) Did you really mean in line 165 of TestMemoryAwareness.java out.shouldContain("OperatingSystemMXBean.getTotalSwapSpaceSize: [1-9][0-9]+"); ? When I fix it (to shouldMatch , this was probably meant by you) I get the next error - java.lang.RuntimeException: 'Metrics.getMemoryLimit() == -1' missing from stdout/stderr I have Metrics.getMemoryAndSwapLimit() == -1 Metrics.getMemoryLimit() == 104857600 Metrics.getMemoryAndSwapUsage() == -1 Metrics.getMemoryUsage() == 30670848 Could we omit the out.shouldContain("Metrics.getMemoryLimit() == -1"); check ? >> (plus we still need Jie's adjustment to fix getFreeSwapSpaceSize ) > >Yes, but I'd rather fix this as a separate issue. I don't think it's >related. Hmm okay, probably better to open a new one. Jie - should I open a separate JBS issue ? Or do you want to do it ( you proposed the patch for the getFreeSwapSpaceSize issue) ? Best regards, Matthias -----Original Message----- From: Severin Gehwolf Sent: Mittwoch, 3. Juni 2020 15:47 To: Baesken, Matthias ; jiefu(??) ; David Holmes ; 'hotspot-dev at openjdk.java.net' Cc: Bob Vandette Subject: Re: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java Hi Matthias, On Fri, 2020-05-22 at 13:36 +0000, Baesken, Matthias wrote: > > Anyway, I think we should change the test code similar to this: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/01/webrev/ > > Hi Severin, this gave me : > > Checking OperatingSystemMXBean > Exception in thread "main" java.lang.IllegalAccessError: class CheckOperatingSystemMXBean (in unnamed module @0x5acf9800) cannot access class jdk.internal.platform.Container (in module java.base) because module java.base does not export jdk.internal.platform to unnamed module @0x5acf9800 > at CheckOperatingSystemMXBean.main(CheckOperatingSystemMXBean.java:33) > > (a bit unexpected because you added it to the test, I though this would be sufficient !?) OK, reproduced. FWIW, 01 was an untested patch ;-) Please try this updated webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/02/webrev/ It works fine here for cgroups v2 and cgroups v1. > (plus we still need Jie's adjustment to fix getFreeSwapSpaceSize ) Yes, but I'd rather fix this as a separate issue. I don't think it's related. Thanks, Severin > -----Original Message----- > From: Severin Gehwolf > Sent: Mittwoch, 20. Mai 2020 21:30 > To: Baesken, Matthias ; jiefu(??) < > jiefu at tencent.com>; David Holmes ; > 'hotspot-dev at openjdk.java.net' > Cc: Bob Vandette > Subject: Re: RFR [XS]: 8244500: jtreg test error in > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > Hi Matthias, > > On Wed, 2020-05-20 at 11:07 +0000, Baesken, Matthias wrote: > > Hi Severin , > > > > > FWIW, parts of this work-around got added by JDK-8236617. Looking > > > at > > > the bug it was observed that: > > > > > > containerMetrics.getMemoryAndSwapLimit() == 0 and > > > containerMetrics.getMemoryLimit() returned a large value. > > > > > > So then with JDK-8231111 I suspect > > > containerMetrics.getMemoryAndSwapLimit() now returns -1 and, > > > thus, > > > getTotalSwapSpaceSize0() is being used instead of the container > > > values. > > > > I added some tracing output > > to OperatingSystemImpl.getTotalSwapSpaceSize() and get these > > values : > > > > containerMetrics.getMemoryAndSwapLimit():-1 > > containerMetrics.getMemoryLimit():-1 > > ****** getTotalSwapSpaceSize0() returned 16106061824 > > Thanks for this. This seems to confirm my suspicion. > > So my working theory here is that the system where you are observing > the issue now is similar to the one when JDK-8236617 was discovered. > For some reason containerMetrics.getMemoryAndSwapLimit() returned 0 > for > the case when the necessary files were missing (which was wrong IMO). > After JDK-8231111 there is a clear distinction for this case: -1 is > being returned. That makes more sense to me, as there wouldn't be any > container limit to take into account. > > Your trace output seems to confirm that: > > containerMetrics.getMemoryAndSwapLimit():-1 > containerMetrics.getMemoryLimit():-1 > > > > So far the theory, I'd like to gather more data before we come up > > > with > > > another patch, though. In JDK-8236617 it wasn't investigated why > > > containerMetrics.getMemoryAndSwapLimit() returned 0 on such a > > > system. > > > It would be good to know why. Could it be that file > > > 'memory.memsw.limit_in_bytes' doesn't exist on such a system? > > > > True, it does not exist : > > > > ls -alL /sys/fs/cgroup/memory/ > > total 0 > > drwxr-xr-x 3 root root 0 Mar 16 11:00 . > > drwxr-xr-x 11 root root 260 Mar 16 11:00 .. > > -rw-r--r-- 1 root root 0 Mar 16 11:00 cgroup.clone_children > > --w--w--w- 1 root root 0 Mar 16 11:00 cgroup.event_control > > -rw-r--r-- 1 root root 0 May 20 13:00 cgroup.procs > > -r--r--r-- 1 root root 0 Mar 16 11:00 cgroup.sane_behavior > > drwxr-xr-x 2 root root 0 May 20 12:52 docker > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.failcnt > > --w------- 1 root root 0 Mar 16 11:00 memory.force_empty > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.limit_in_bytes > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.low_limit_in_bytes > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.max_usage_in_bytes > > -rw-r--r-- 1 root root 0 Mar 16 11:00 > > memory.move_charge_at_immigrate > > -r--r--r-- 1 root root 0 Mar 16 11:00 memory.numa_stat > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.oom_control > > ---------- 1 root root 0 Mar 16 11:00 memory.pressure_level > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.soft_limit_in_bytes > > -r--r--r-- 1 root root 0 Mar 16 11:00 memory.stat > > -rw-r--r-- 1 root root 0 Mar 16 11:00 memory.swappiness > > -r--r--r-- 1 root root 0 Mar 16 11:00 memory.usage_in_bytes > > -rw-r--r-- 1 root root 0 May 20 13:00 memory.use_hierarchy > > -rw-r--r-- 1 root root 0 Mar 16 11:00 notify_on_release > > -rw-r--r-- 1 root root 0 Mar 16 11:00 release_agent > > -rw-r--r-- 1 root root 0 Mar 16 11:00 tasks > > Hmm, is this inside a container on this system with --memory and -- > memory-swap being specified on the docker command line? > > Anyway, I think we should change the test code similar to this: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/01/webrev/ > > If this works for you, then that's what I'd suggest as a fix. > > Note that I haven't tested this patch. I need to recreate my set-up > for > proper docker testing here :-/ > > Thanks, > Severin > > > Best regards, Matthias > > > > > > > > -----Original Message----- > > From: Severin Gehwolf > > Sent: Montag, 18. Mai 2020 12:09 > > To: jiefu(??) ; David Holmes < > > david.holmes at oracle.com>; Baesken, Matthias < > > matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' < > > hotspot-dev at openjdk.java.net> > > Cc: Bob Vandette > > Subject: Re: RFR [XS]: 8244500: jtreg test error in > > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > > > Hi, > > > > On Mon, 2020-05-18 at 06:15 +0000, jiefu(??) wrote: > > > Hi David, > > > > > > This bug should be introduced by JDK-8242480. > > > I'm sorry for that. > > > > > > And I think the logic in > > > src/jdk.management/unix/classes/com/sun/management/internal/Opera > > > tingSystemImpl.java should be fixed. > > > > > > In my fix for JDK-8242480, I had ignored the case for kernels > > > which don't support "--memory-swap 150M". > > > In that case, containerMetrics.getMemoryAndSwapLimit()[1] will > > > return -1. > > > > It appears so that containerMetrics.getMemoryAndSwapLimit() > > returned 0 > > prior JDK-8231111 and -1 after. I'm not convinced JDK-8242480 > > introduced this. > > > > > I didn't notice this bug because all our kernels support -- > > > memory-swap. > > > Fortunately, Matthias found this bug and will fix it. > > > > I'm still not sure what's happening in Matthias' case on that > > system. > > It would be best to first try to understand the actual problem > > better. > > The info I have is: > > > > - It's a cgroup v1 system > > - The system now returns large values for > > OperatingSystemMXBean.getTotalSwapSpaceSize() and > > OperatingSystemMXBean.getFreeSwapSpaceSize() > > > > It would be good to know what the actual return values are for the > > container metrics on this system with swap limit disabled at kernel > > level. And also whether or not relevant cgroup files exist on such > > a > > system (within docker). > > > > FWIW, parts of this work-around got added by JDK-8236617. Looking > > at > > the bug it was observed that: > > > > containerMetrics.getMemoryAndSwapLimit() == 0 and > > containerMetrics.getMemoryLimit() returned a large value. > > > > So then with JDK-8231111 I suspect > > containerMetrics.getMemoryAndSwapLimit() now returns -1 and, thus, > > getTotalSwapSpaceSize0() is being used instead of the container > > values. > > > > So far the theory, I'd like to gather more data before we come up > > with > > another patch, though. In JDK-8236617 it wasn't investigated why > > containerMetrics.getMemoryAndSwapLimit() returned 0 on such a > > system. > > It would be good to know why. Could it be that file > > 'memory.memsw.limit_in_bytes' doesn't exist on such a system? > > > > Unfortunately, I don't know what config one needs in order to > > reproduce > > so we'll have to rely on Matthias to gather the info. > > > > Thanks, > > Severin > > > > > Thanks a lot. > > > Best regards, > > > Jie > > > > > > [1] > > > http://hg.openjdk.java.net/jdk/jdk/file/a95911422005/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l73 > > > > > > ?On 2020/5/18, 1:48 PM, "David Holmes" > > > wrote: > > > > > > Hi Matthias, > > > > > > On 16/05/2020 12:59 am, Baesken, Matthias wrote: > > > > Hello David , I get the warning always on this linux > > > machine (I think it depends on machine setup and/or > > > kernel version). > > > > > > > >> Surely it must be specified to return 0 or some such > > > sentinel in such a case? > > > > > > > > The doc just says this : > > > > > > > > 62 /** > > > > 63 * Returns the total amount of swap space in bytes. > > > > 64 * > > > > 65 * @return the total amount of swap space in bytes. > > > > 66 */ > > > > 67 public long getTotalSwapSpaceSize(); > > > > > > > > ( however it returned previously 0 before the latest > > > changes in the container support code came in ). > > > > > > That sounds like a bug was introduced to me then. > > > > > > > > > > Regarding the getFreeSwapSpaceSize issue , Jie provided a > > > change > > > to src/jdk.management/unix/classes/com/sun/management/internal/O > > > peratingSystemImpl.java > > > > Because of this I could remove the try-construct . > > > > > > > > New webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8244500.3/ > > > > > > I'm quite confused by your changes. Your first version simply > > > adjusted > > > the test to check for the warning. But now you are also > > > changing the > > > main logic. I can't assess the validity of what you are doing > > > sorry. > > > > > > I've cc'd Severin and Bob as I think they need to evaluate > > > what the real > > > problem is here and when it was introduced. > > > > > > David > > > ---- > > > > > > > Best regards, Matthias > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: David Holmes > > > > Sent: Donnerstag, 14. Mai 2020 07:02 > > > > To: Baesken, Matthias ; > > > 'hotspot-dev at openjdk.java.net' > > > > Cc: jiefu(??) > > > > Subject: Re: RFR [XS]: 8244500: jtreg test error in > > > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > > > > > > > On 14/05/2020 12:28 am, Baesken, Matthias wrote: > > > >> Hello, here is a new webrev removing the comment : > > > >> > > > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8244500.1/ > > > >> > > > >> It removes the comment , and it also handles 0 for > > > getFreeSwapSpaceSize > > > >> ( this can occur after JDK-8242480 : Negative value > > > may be returned by getFreeSwapSpaceSize() in the docker was > > > pushed recently ). > > > > > > > > Sorry Matthias but I still see this partly as a bug in the > > > underlying > > > > container support code. If you get the warning then what > > > does > > > > getTotalSwapSpaceSize() return? Surely it must be specified > > > to return 0 > > > > or some such sentinel in such a case? > > > > > > > > Other comments: > > > > > > > > ! try { > > > > ! > > > > > > > out.shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: > > > > [1-9][0-9]+"); > > > > ! } catch(RuntimeException ex) { // allow 0, see > > > 8242480 > > > > ! > > > > > > > out.shouldContain("OperatingSystemMXBean.getFreeSwapSpaceSize: > > > 0"); > > > > ! } > > > > > > > > can't you just change the original regex to match >= 0? > > > > > > > > Style nit: avoid historical comments like "see 8242480" and > > > "since > > > > 8231111 ...". > > > > > > > > Thanks, > > > > David > > > > > > > >> Best regards, Matthias > > > >> > > > >> -----Original Message----- > > > >> From: Baesken, Matthias > > > >> Sent: Mittwoch, 13. Mai 2020 10:11 > > > >> To: 'David Holmes' ; > > > 'hotspot-dev at openjdk.java.net' > > > >> Subject: RE: RFR [XS]: 8244500: jtreg test error in > > > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > > >> > > > >> > > > >>> But according to the comment immediately before your > > > changes: > > > >>> > > > >>> // in case of warnings like : "Your kernel does not > > > support swap limit > > > >>> capabilities or the cgroup is not mounted. Memory limited > > > without swap." > > > >>> // the getTotalSwapSpaceSize does not return the expected > > > result, but 0 > > > >>> > > > >>> we should be returning zero. So this seems to be a bug in > > > the > > > >>> implementation not in the test. > > > >> > > > >> > > > >> Hi David , I think in case of "Your kernel does not > > > support swap limit capabilities ... " we just do not get > > > the values > > > >> that are normally expected. > > > >> Previously we indeed got "0" . > > > >> Probably the comment should be deleted . > > > >> > > > >> Best regards, Matthias > > > >> > > > > > > > > > From erik.osterlund at oracle.com Thu Jun 4 14:18:06 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 4 Jun 2020 16:18:06 +0200 Subject: RFR: 8246442: nmethod::can_convert_to_zombie() asserts when not called by the sweeper In-Reply-To: <32487f5b-dc1e-ab44-fff5-fc72cf91eac1@oracle.com> References: <4e9a0ca4-c55a-7bc6-309c-386fe3015dda@oracle.com> <32487f5b-dc1e-ab44-fff5-fc72cf91eac1@oracle.com> Message-ID: <27244811-35de-38ee-739e-a24d601fc4a0@oracle.com> Hi Vladimir, Thanks for the review. /Erik On 2020-06-03 17:20, Vladimir Kozlov wrote: > +1 > > Thanks, > Vladimir > > On 6/3/20 4:40 AM, David Holmes wrote: >> Hi Erik, >> >> LGTM. >> >> Your explanation and solution seems entirely reasonable. >> >> Thanks, >> David >> >> On 3/06/2020 9:26 pm, Erik ?sterlund wrote: >>> Hi, >>> >>> nmethod::can_convert_to_zombie() used to be called only by the >>> sweeper, after it has checked that >>> an nmethod is not entrant. Due to the single call site, there was an >>> assert checking that the nmethod >>> should be not entrant (the condition just checked). >>> However, it is now used in >>> nmethod::post_compiled_method_load_event(), where the result of the >>> assert is racy. >>> >>> When an nmethod is created, the load event will never be able to see >>> the nmethod spuriously flipping >>> to zombie, because there simply are no safepoint polls/transitions >>> between the nmethod being created >>> and the load event firing. However, the jvmti_GenerateEvents can >>> observe such spurious flips to zombie. >>> It is completely harmless, because the whole operation runs under >>> the CodeCache_lock, which prevents >>> further nmethods from being flushed. But they can flip to zombie, >>> which is indeed exactly what the >>> check is design to test. Therefore, this is just a false positive >>> assertion failure. >>> >>> Proposed fix: make the assert only apply when called from the >>> sweeper thread. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8246442/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8246442 >>> >>> Thanks, >>> /Erik From erik.osterlund at oracle.com Thu Jun 4 14:20:15 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 4 Jun 2020 16:20:15 +0200 Subject: RFR (M) 8246340: Move SystemDictionary GC roots into OopStorage In-Reply-To: <3e329509-af13-b44b-e730-7f9bcf6d521c@oracle.com> References: <3e329509-af13-b44b-e730-7f9bcf6d521c@oracle.com> Message-ID: <89594df7-fcb8-1444-0db7-da069a919d57@oracle.com> Hi Coleen, Looks good! Thanks for the nice cleanup! /Erik On 2020-06-04 13:39, coleen.phillimore at oracle.com wrote: > > Offline, Erik pointed out that I missed a system_dictionary tag in > parallelGC and to not initialize OopStorage static data members, so > here's an incremental that I reran tier1 tests on. > > open webrev at > http://cr.openjdk.java.net/~coleenp/2020/8246340.02.incr/webrev > > Thanks, > Coleen > > On 6/3/20 1:23 PM, coleen.phillimore at oracle.com wrote: >> Summary: use vm_global() OopStorage for system dictionary roots >> removed SystemDictionary::oops_do in all the GCs. >> >> Tested with tier1-6. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8246340.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8246340 >> >> Thanks, >> Coleen > From lois.foltan at oracle.com Thu Jun 4 14:45:03 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 4 Jun 2020 10:45:03 -0400 Subject: RFR (M) 8246340: Move SystemDictionary GC roots into OopStorage In-Reply-To: <89594df7-fcb8-1444-0db7-da069a919d57@oracle.com> References: <3e329509-af13-b44b-e730-7f9bcf6d521c@oracle.com> <89594df7-fcb8-1444-0db7-da069a919d57@oracle.com> Message-ID: +1.? Looks good Coleen! Lois On 6/4/2020 10:20 AM, Erik ?sterlund wrote: > Hi Coleen, > > Looks good! Thanks for the nice cleanup! > > /Erik > > On 2020-06-04 13:39, coleen.phillimore at oracle.com wrote: >> >> Offline, Erik pointed out that I missed a system_dictionary tag in >> parallelGC and to not initialize OopStorage static data members, so >> here's an incremental that I reran tier1 tests on. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8246340.02.incr/webrev >> >> Thanks, >> Coleen >> >> On 6/3/20 1:23 PM, coleen.phillimore at oracle.com wrote: >>> Summary: use vm_global() OopStorage for system dictionary roots >>> removed SystemDictionary::oops_do in all the GCs. >>> >>> Tested with tier1-6. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8246340.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8246340 >>> >>> Thanks, >>> Coleen >> > From coleen.phillimore at oracle.com Thu Jun 4 14:58:31 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Jun 2020 10:58:31 -0400 Subject: RFR (M) 8246340: Move SystemDictionary GC roots into OopStorage In-Reply-To: <89594df7-fcb8-1444-0db7-da069a919d57@oracle.com> References: <3e329509-af13-b44b-e730-7f9bcf6d521c@oracle.com> <89594df7-fcb8-1444-0db7-da069a919d57@oracle.com> Message-ID: <32ad9053-0177-6e19-9814-0fea9e795154@oracle.com> Thanks Erik, thank you for the initiative to do this. Coleen On 6/4/20 10:20 AM, Erik ?sterlund wrote: > Hi Coleen, > > Looks good! Thanks for the nice cleanup! > > /Erik > > On 2020-06-04 13:39, coleen.phillimore at oracle.com wrote: >> >> Offline, Erik pointed out that I missed a system_dictionary tag in >> parallelGC and to not initialize OopStorage static data members, so >> here's an incremental that I reran tier1 tests on. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8246340.02.incr/webrev >> >> Thanks, >> Coleen >> >> On 6/3/20 1:23 PM, coleen.phillimore at oracle.com wrote: >>> Summary: use vm_global() OopStorage for system dictionary roots >>> removed SystemDictionary::oops_do in all the GCs. >>> >>> Tested with tier1-6. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8246340.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8246340 >>> >>> Thanks, >>> Coleen >> > From coleen.phillimore at oracle.com Thu Jun 4 15:01:32 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Jun 2020 11:01:32 -0400 Subject: RFR (M) 8246340: Move SystemDictionary GC roots into OopStorage In-Reply-To: References: <3e329509-af13-b44b-e730-7f9bcf6d521c@oracle.com> <89594df7-fcb8-1444-0db7-da069a919d57@oracle.com> Message-ID: <850b664e-13c0-6013-134d-6ea41c437778@oracle.com> Thank you for reviewing, Lois! Coleen On 6/4/20 10:45 AM, Lois Foltan wrote: > +1. Looks good Coleen! > Lois > > On 6/4/2020 10:20 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Looks good! Thanks for the nice cleanup! >> >> /Erik >> >> On 2020-06-04 13:39, coleen.phillimore at oracle.com wrote: >>> >>> Offline, Erik pointed out that I missed a system_dictionary tag in >>> parallelGC and to not initialize OopStorage static data members, so >>> here's an incremental that I reran tier1 tests on. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8246340.02.incr/webrev >>> >>> Thanks, >>> Coleen >>> >>> On 6/3/20 1:23 PM, coleen.phillimore at oracle.com wrote: >>>> Summary: use vm_global() OopStorage for system dictionary roots >>>> removed SystemDictionary::oops_do in all the GCs. >>>> >>>> Tested with tier1-6. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8246340.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8246340 >>>> >>>> Thanks, >>>> Coleen >>> >> > From jiefu at tencent.com Thu Jun 4 14:54:22 2020 From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=) Date: Thu, 4 Jun 2020 14:54:22 +0000 Subject: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java(Internet mail) In-Reply-To: References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> Message-ID: <519C6E4F-BFA9-4166-9AB6-B76901E1FAF9@tencent.com> Hi Matthias, ?On 2020/6/4, 10:12 PM, "Baesken, Matthias" wrote: >> (plus we still need Jie's adjustment to fix getFreeSwapSpaceSize ) > >Yes, but I'd rather fix this as a separate issue. I don't think it's >related. Hmm okay, probably better to open a new one. Jie - should I open a separate JBS issue ? Or do you want to do it ( you proposed the patch for the getFreeSwapSpaceSize issue) ? Please feel free to open a separate JBS issue. And many thanks for fixing it. Best regards, Jie From sgehwolf at redhat.com Thu Jun 4 16:56:06 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 04 Jun 2020 18:56:06 +0200 Subject: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> Message-ID: <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> Hi Matthias, Thanks for running this to get better diagnostics! On Thu, 2020-06-04 at 14:11 +0000, Baesken, Matthias wrote: > HI Severin, with your new patch I run into : > > java.lang.RuntimeException: 'OperatingSystemMXBean.getTotalSwapSpaceSize: [1-9][0-9]+' missing from stdout/stderr > > at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:189) > at TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMemoryAwareness.java:165) > at TestMemoryAwareness.main(TestMemoryAwareness.java:66) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) > at java.base/java.lang.Thread.run(Thread.java:832) > > Did you really mean in line 165 of TestMemoryAwareness.java > out.shouldContain("OperatingSystemMXBean.getTotalSwapSpaceSize: [1-9][0-9]+"); > ? > When I fix it (to shouldMatch , this was probably meant by you) Yes, that's what I meant. > I get the next error - > java.lang.RuntimeException: 'Metrics.getMemoryLimit() == -1' missing from stdout/stderr > > I have > > Metrics.getMemoryAndSwapLimit() == -1 > Metrics.getMemoryLimit() == 104857600 > Metrics.getMemoryAndSwapUsage() == -1 > Metrics.getMemoryUsage() == 30670848 OK. I've added this info to the bug and some more bits and pieces. I believe there isn't much we can do but to bail out and let the test pass... It would be good to only pass the test that way on a system without kernel support for swap limits. Here would be my suggestion (hopefully this one isn't too restrictive again): http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ If we changed OperatingSystemMXbean as suggested by Bob to return 0 in this case for getTotalSwapSpaceSize(), it would be incorrect for the case as described in the bug (--memory-swap=-1). That would actually be unlimited swap. > Could we omit the > > out.shouldContain("Metrics.getMemoryLimit() == -1"); > > check ? We could. I'd rather be more specific and use expectedMemoryLimit instead, though. That's what the webrev above does. Feel free to change it in a way so that it passes on the affected system as I'd have to keep guessing :) In conclusion: This seems a test bug to me. Thanks, Severin From igor.ignatyev at oracle.com Thu Jun 4 17:21:35 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 4 Jun 2020 10:21:35 -0700 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: <88ED55CF-092F-4FAC-ACFD-17D3CA869C75@oracle.com> References: <88ED55CF-092F-4FAC-ACFD-17D3CA869C75@oracle.com> Message-ID: piny? -- Igor > On Jun 1, 2020, at 7:54 AM, Igor Ignatyev wrote: > > ping? > -- Igor > >> On May 27, 2020, at 10:25 AM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >>> 8 lines changed: 2 ins; 0 del; 6 mod; >> >> Hi all, >> >> could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? >> from JBS: >>> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` >> the patch >> - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts >> - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 >> testing: tier1 >> webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >> >> Thanks, >> -- Igor > From igor.ignatyev at oracle.com Thu Jun 4 20:23:35 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 4 Jun 2020 13:23:35 -0700 Subject: RFR(S/L) : 8244614 : cleanup keywords used/available in hotspot testbase In-Reply-To: <8F94C0EC-3470-4252-A18B-72F68FDA3BC5@oracle.com> References: <939A28A1-5CEA-438D-A2EE-E547E9C7D57E@oracle.com> <8F94C0EC-3470-4252-A18B-72F68FDA3BC5@oracle.com> Message-ID: <238C8726-BEFB-480B-8D02-D986EB068033@oracle.com> ping? -- Igor > On May 29, 2020, at 10:49 AM, Igor Ignatyev wrote: > > ouch,.. it appears that this tier1 execution wasn't in vain. I realized that all my queries were limited to the tests `jtreg -l` gave me, which excludes tests which can't be w/ my default build on my localhost... > http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01-02 updates 93 more tests. > > -- Igor > >> On May 29, 2020, at 10:14 AM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>> 5 lines changed: 4 ins; 0 del; 1 mod; >> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>> 930 lines changed: 3 ins; 204 del; 723 mod; >> >> >> Hi all, >> >> given the absense of comments, I assume that nobody has any strong feeling these to-be-removed keywords , so I'm repurposing this thread as RFR. >> >> the biggest part of the patch was done by sed: >> - removal of $key : gsed -ri "s/(@key.*) $key/\1/g" {} ; where $key is in nmt cte_test, jcmd, regression >> - removal of empty @key : gsed -ri '/^ \* @key\s*$/d' >> >> the manually done changes are limited to: >> - removal of keywords from TEST.ROOT >> - adding k/w description to TEST.ROOT >> - adding @bug to redefclass031 >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244614 >> webrevs: >> - part done by sed: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.sed >>> 594 lines changed: 0 ins; 205 del; 389 mod; >> >> - part done by me: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>> 5 lines changed: 4 ins; 0 del; 1 mod; >> >> - whole (incl. copyright year update) : http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>> 930 lines changed: 3 ins; 204 del; 723 mod; >> >> >> testing: >> - grep-ed for removed keywords >> - jtreg -l >> - tier1 (just for the good luck) >> >> Thanks, >> -- Igor >> >> >> >>> On May 22, 2020, at 3:42 PM, Igor Ignatyev wrote: >>> >>> Hi all, >>> >>> I'm planning to clean up jtreg keywords that we have in test/hotspot/jtreg as most of them seem to be useless, confuse and make people to add these keywords in vain. After looking at how hotspot k/w are used, as well as which k/w are available and used by other test suites, I'd like to propose the following: >>> - keep and document 'headful', 'stress' as they are useful for test selection; >>> - keep and document 'intermittent', 'randomness' as they bring extra information about tests stability/reproducibility/determinism; >>> - remove 'nmt', 'gc', 'metaspace' as they are used only in obvious places and aren't informative; >>> - remove 'cte_test' as it's an artifact of old days and doesn't bring any useful information anymore; >>> - remove 'jcmd', not all tests which depend on jcmd has this k/w, so it can't be used for test selection/filtering; there is also no obvious reasons to single out jcmd from other binaries bundled w/ JDK >>> - remove 'regression' as the usage of at-bug is more common and convey the same meaning; and for two tests which have regression k/w, but don't have at-bug: >>> - mark vmTestbase/nsk/jvmti/RedefineClasses/redefclass031/TestDescription.java w/ at-bug 6318850; >>> - don't mark runtime/NMT/ReleaseNoCommit.java as it wasn't integrated as a true regression test, but rather as a function test for NMT w/ bunch of other NMT tests (where aren't marked w/ 'regression' k/w) by JDK-8054952. >>> >>> I'd like to receive some feedback on the proposed changes before I (more likely sed) remove keywords from 500+ tests and spend time testing that. >>> >>> [1] is the preliminary webrev just to illustrate the change, it only updates TEST.ROOT file and adds @bug to redefclass031 test. >>> >>> Thanks, >>> -- Igor >>> >>> for reference, k/w available in other test suites: >>> jdk : 2d dnd headful i18n intermittent printer randomness jfr >>> langtools: intermittent randomness >>> jaxp: intermittent randomness >>> >>> [1] http://cr.openjdk.java.net/~iignatyev//8244614/webrev.00 >>> >>> >> > From leonid.mesnik at oracle.com Thu Jun 4 22:55:46 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 4 Jun 2020 15:55:46 -0700 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: References: <88ED55CF-092F-4FAC-ACFD-17D3CA869C75@oracle.com> Message-ID: <50a6910b-f566-5824-ca01-7ceea7562ddf@oracle.com> Looks good. Leonid On 6/4/20 10:21 AM, Igor Ignatyev wrote: > piny? > -- Igor > >> On Jun 1, 2020, at 7:54 AM, Igor Ignatyev wrote: >> >> ping? >> -- Igor >> >>> On May 27, 2020, at 10:25 AM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >>>> 8 lines changed: 2 ins; 0 del; 6 mod; >>> Hi all, >>> >>> could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? >>> from JBS: >>>> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` >>> the patch >>> - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts >>> - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 >>> testing: tier1 >>> webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >>> >>> Thanks, >>> -- Igor From david.holmes at oracle.com Fri Jun 5 04:35:50 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 5 Jun 2020 14:35:50 +1000 Subject: RFR[15] JDK-8243114: Implement montgomery{Multiply, Square}intrinsics on Windows In-Reply-To: References: <725e8e5a-71b5-7d6e-4d3a-7c799a52ab56@redhat.com> <2442a046-6640-f759-6330-e971405f0c78@oracle.com> <6cb701cd-5652-6b9d-8051-b37adb3ce63f@oracle.com> Message-ID: <44d1daf2-7124-9b4b-b039-d35d1f8e96c6@oracle.com> Hi Simon, On 4/06/2020 11:35 pm, Simon Tooke wrote: > Hello, David, and thanks for the review! > > I've responded to your comments below, and intend to post a new patch > for review today or tomorrow. > > Thanks again, > > -Simon > > On 2020-06-03 2:06 a.m., David Holmes wrote: >> Hi Simon, >> >> On 23/05/2020 12:04 am, Sean Mullan wrote: >>> Cross-posting to hotspot-dev for additional review since the code >>> changes are in hotspot. >>> >>> --Sean >>> >>> On 5/21/20 1:24 PM, Simon Tooke wrote: >>>> Hello, >>>> >>>> I'd like to request a review for: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243114 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~stooke/webrevs/jdk-8243114-jdk/00/00/ >>>> >>>> This change implements the given intrinsics on Windows. >>>> >>>> The Windows toolchain has no 128 bit integer types, and no inline >>>> asm (on x64 and Arm).? In addition, 'long' is 4 bytes, not 8, as it >>>> is with gcc.? This patch had to change some of the linux >>>> implementation to account for these limitations. >> >> I can't really comment on the intrinsics themselves but a couple of >> suggestions: >> >> - use explicitly sized types e.g. uint64_t instead of unsigned long long > > Yes, this hurt to type.? A previous review suggested using julong, is > that acceptable to you? j* types should only be used when dealing with values that come from or go to Java. Obviously julong is a misnomer as Java doesn't have an unsigned type, but in general this is something we are trying to fix up in the codebase. If these arrays are extracted from Java arrays then using a j* type would be appropriate, but then I would expect jlong, unless the algorithm explicitly requires unsigned types? If so julong would be acceptable. > (an aside: I'm now wondering if there is other code in the JDK that > assumes long is 64bits - which is not true on all platforms...) There has been such code, but hopefully there is no remaining actively used code with that bug. There are using of "long" that should be eradicated (there's an open JBS issue for that as I recall) but the remaining uses don't seem to require the long be 64-bit. >> - use the existing NOINLINE macro for the _declspec(noinline) > Thanks, will do. >> >> The conditional compilation in this code has me quite confused. >> Looking at the existing code we have: >> >> 3680 #ifndef _WINDOWS >> 3681 >> 3682 #define ASM_SUBTRACT >> 3683 >> 3684 #ifdef ASM_SUBTRACT >> ... >> 3702 #else // ASM_SUBTRACT >> ... >> 3719 #endif // ! ASM_SUBTRACT >> >> So all the above code is only built on not-Windows, and we >> unconditionally define ASM_SUBTRACT, so lines 3702 to 3719 appear to >> be dead code! I'm not at all sure whether that is actually a bug and >> the windows ifndef should have had an endif after line 3682; or >> whether we can just simplify the code. > The dead code existed prior to this patch, so I wasn't proposing > removing it.? I'm happy to do so if that's for the best.? As far as I > can tell, it's for implementing these intrinsics on gcc platforms > without assembler. AFAICS Andrew originally had the ASM_SUBTRACT parts, with the unconditional #define, but there was no explanation in the review thread as to why the unused code remained present. Then before integration all the code was wrapped by the ifndef Windows to exclude it from Windows. I think it needs to be fixed as you are making changes to the Windows part and it is very hard to establish how the dead code should look in that case. Thanks, David >> >> Thanks, >> David >> >>>> >>>> My gratitude for Andrew Haley for doing the heavy lifting at the >>>> core of this patch. >>>> >>>> The patch, if accepted, will be offered to 11u as a potential >>>> backport. The changes apply cleanly modulo some line number changes. >>>> >>>> As for the speedup, this test case: >>>> >>>> BigInteger base = BigInteger.ONE.shiftLeft(1024); >>>> long count = LongStream.rangeClosed(2, 100_000) >>>> ???? .mapToObj(n -> BigInteger.valueOf(n).add(base)) >>>> ???? .filter(i -> i.isProbablePrime(50)) >>>> ???? .count(); >>>> >>>> goes from 1 minute 20 seconds down to about 35 seconds om my VM, >>>> over multiple runs.? As is the case on other platforms where the >>>> intrinsics are supported, they will be enabled by default on Windows. >>>> >>>> Thank you for your time, >>>> >>>> -Simon Tooke >>>> >>>> >>>> Principal Software Engineer, >>>> >>>> Red Hat Canada >>>> >>>> >>>> >> > From kim.barrett at oracle.com Fri Jun 5 07:52:44 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 5 Jun 2020 03:52:44 -0400 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot Message-ID: [Changes are only to the build system, but since the changes have jdk-wide effect I?ve cc?d what I think are the relevant dev lists.] This change is part of JEP 347; the intent is to target JDK 16. Please review this change to the building of C++ code in the JDK to enable the use of C++14 language features. This change does not make any code changes to use new features provided by C++11 or C++14. This requires trimming the supported compiler versions, moving the minimum supported versions forward (significantly, in some cases). The new minimums are based on compiler documentation rather than testing. It may be that more recent versions are actually required. This change needs to be tested on platforms not supported by Oracle. The JEP test plan includes additional Oracle testing beyond what I?ve done. CR: https://bugs.openjdk.java.net/browse/JDK-8246032 Webrev: https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ Testing: mach5 tier1-5 on Oracle supported platforms. Performance testing showed no significant changes in either direction. Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le From vladimir.x.ivanov at oracle.com Fri Jun 5 08:02:17 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 5 Jun 2020 11:02:17 +0300 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: References: Message-ID: Looks good. Best regards, Vladimir Ivanov On 27.05.2020 20:25, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >> 8 lines changed: 2 ins; 0 del; 6 mod; > > Hi all, > > could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? > from JBS: >> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` > the patch > - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts > - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP > > JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 > testing: tier1 > webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 > > Thanks, > -- Igor > From matthias.baesken at sap.com Fri Jun 5 08:14:54 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 5 Jun 2020 08:14:54 +0000 Subject: 8246648: issue with OperatingSystemImpl getFreeSwapSpaceSize in docker after 8242480 Message-ID: >> (plus we still need Jie's adjustment to fix getFreeSwapSpaceSize ) > >Please feel free to open a separate JBS issue. >And many thanks for fixing it. > Hello, In the TestMemoryAwareness.java docker related test the following issue has been observed on a Linux machine: java.lang.RuntimeException: 'OperatingSystemMXBean\\.getFreeSwapSpaceSize: [1-9][0-9]+' missing from stdout/stderr at jdk.test.lib.process.OutputAnalyzer.shouldMatch(OutputAnalyzer.java:306) at TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMemoryAwareness.java:154) at TestMemoryAwareness.main(TestMemoryAwareness.java:65) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) at java.base/java.lang.Thread.run(Thread.java:832) Jie Fu proposed an adjustment to OperatingSystemImpl getFreeSwapSpaceSize to address this. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8246648 http://cr.openjdk.java.net/~mbaesken/webrevs/8246648.0/ Best regards, Matthias -----Original Message----- From: jiefu(??) Sent: Donnerstag, 4. Juni 2020 16:54 To: Baesken, Matthias ; Severin Gehwolf ; David Holmes ; 'hotspot-dev at openjdk.java.net' Cc: Bob Vandette Subject: Re: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java(Internet mail) Hi Matthias, ?On 2020/6/4, 10:12 PM, "Baesken, Matthias" wrote: >> (plus we still need Jie's adjustment to fix getFreeSwapSpaceSize ) > >Yes, but I'd rather fix this as a separate issue. I don't think it's >related. Hmm okay, probably better to open a new one. Jie - should I open a separate JBS issue ? Or do you want to do it ( you proposed the patch for the getFreeSwapSpaceSize issue) ? Please feel free to open a separate JBS issue. And many thanks for fixing it. Best regards, Jie From per.liden at oracle.com Fri Jun 5 08:20:22 2020 From: per.liden at oracle.com (Per Liden) Date: Fri, 5 Jun 2020 10:20:22 +0200 Subject: RFR(S) : 8246494 : introduce vm.flagless at-requires property In-Reply-To: References: Message-ID: <5b1cac8c-7e9b-195a-edfa-6ab972e32bf0@oracle.com> Hi Igor, When looking at the follow-up sub-tasks for this, I see for example this: http://cr.openjdk.java.net/~iignatyev/8246499/webrev.00/test/hotspot/jtreg/gc/z/TestSmallHeap.java.udiff.html Maybe I'm misunderstanding how this is supposed to work, but it looks like this test would now _not_ be executed if I do: make TEST=test/hotspot/jtreg/gc/z/TestSmallHeap.java JTREG="VM_OPTIONS=-XX:+UseZGC" Is that so? In that case, that seems incorrect. cheers, Per On 6/3/20 11:30 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 >> 70 lines changed: 66 ins; 0 del; 4 mod > > Hi all, > > could you please review the patch which introduces a new @requires property to filter out the tests which ignore externally provided JVM flags? > > the idea behind this patch is to have a way to clearly mark tests which ignore flags, so > a) it's obvious that they don't execute a flag-guarded code/feature, and extra care should be taken to use them to verify any flag-guarded changed; > b) they can be easily excluded from runs w/ flags. > > @requires and VMProps allows us to achieve both, so it's been decided to add a new property `vm.flagless`. `vm.flagless` is set to false if there are any XX flags other than `-XX:MaxRAMPercentage` and `-XX:CreateCoredumpOnCrash` (which are known to be set almost always) or any X flags other `-Xmixed`; in other words any tests w/ `@requires vm.flagless` will be excluded from runs w/ any other X / XX flags passed via `-vmoption` / `-javaoption`. in rare cases, when one still wants to run the tests marked by `vm.flagless` w/ external flags, `vm.flagless` can be forcefully set to true by setting any value to `TEST_VM_FLAGLESS` env. variable. > > this patch adds necessary common changes and marks common tests, namely Scimark, GTestWrapper and TestNativeProcessBuilder. Component-specific tests will be marked separately by the corresponding subtasks of 8151707[1]. > > please note, the patch depends on CODETOOLS-7902336[2], which will be included in the next jtreg version, so this patch is to be integrated only after jtreg5.1 is promoted and we switch to use it by 8246387[3]. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8246494 > webrev: http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 > testing: marked tests w/ different XX and X flags w/ and w/o TEST_VM_FLAGLESS env. var, and w/o any flags > > [1] https://bugs.openjdk.java.net/browse/JDK-8151707 > [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7902336 > [3] https://bugs.openjdk.java.net/browse/JDK-8246387 > > Thanks, > -- Igor > From sgehwolf at redhat.com Fri Jun 5 09:00:52 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 05 Jun 2020 11:00:52 +0200 Subject: 8246648: issue with OperatingSystemImpl getFreeSwapSpaceSize in docker after 8242480 In-Reply-To: References: Message-ID: <2e640705a69d8b6f71506c1bed16cae7467429d3.camel@redhat.com> On Fri, 2020-06-05 at 08:14 +0000, Baesken, Matthias wrote: > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8246648 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8246648.0/ Prior this patch OperatingSystemMXBean.getFreeSwapSpaceSize() would return 0 (-1 - memLimit < 0), on a system with a kernel which doesn't support swap limits since -1 will be returned for Metrics.getMemoryAndSwapLimit(). After this patch it would return the system value. Similarly, for a system with explicit --memory-swap=-1 (unlimited) it would return 0 prior patch, and the system value post-patch. In both cases, pre/post-patch not all combinations are correct. However, it seems reasonable to account for the -1 retval of memSwapLimit explicitly and be well-behaved for the explicit unlimited swap value configuration and only be ill-behaved for a kernel not supporting swap limits (rare?). In which case, a warning is being printed when running the container with a swap limit. In summary: Patch looks OK to me. Aside: 55 public long getTotalSwapSpaceSize() { 56 if (containerMetrics != null) { 57 long limit = containerMetrics.getMemoryAndSwapLimit(); 58 // The memory limit metrics is not available if JVM runs on Linux host (not in a docker container) 59 // or if a docker container was started without specifying a memory limit (without '--memory=' 60 // Docker option). In latter case there is no limit on how much memory the container can use and 61 // it can use as much memory as the host's OS allows. 62 long memLimit = containerMetrics.getMemoryLimit(); 63 if (limit >= 0 && memLimit >= 0) { 64 // we see a limit == 0 on some machines where "kernel does not support swap limit capabilities" This comment --^ is wrong. Should be "we see a limit == -1 ..." 65 return (limit < memLimit) ? 0 : limit - memLimit; 66 } Not sure if you want to change that as part of this patch. Thanks, Severin From aph at redhat.com Fri Jun 5 09:02:59 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 5 Jun 2020 10:02:59 +0100 Subject: RFR[15] JDK-8243114: Implement montgomery{Multiply, Square}intrinsics on Windows In-Reply-To: <44d1daf2-7124-9b4b-b039-d35d1f8e96c6@oracle.com> References: <725e8e5a-71b5-7d6e-4d3a-7c799a52ab56@redhat.com> <2442a046-6640-f759-6330-e971405f0c78@oracle.com> <6cb701cd-5652-6b9d-8051-b37adb3ce63f@oracle.com> <44d1daf2-7124-9b4b-b039-d35d1f8e96c6@oracle.com> Message-ID: <6522bd5e-0a8e-b858-8cba-9ab51ff712a6@redhat.com> On 05/06/2020 05:35, David Holmes wrote: > If these arrays are extracted from Java arrays Yes, they are. > then using a j* type would be appropriate, but then I would expect > jlong, unless the algorithm explicitly requires unsigned types? It does. All of the code handling bignums uses unsigned words. > If so julong would be acceptable. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From matthias.baesken at sap.com Fri Jun 5 10:17:28 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 5 Jun 2020 10:17:28 +0000 Subject: 8246648: issue with OperatingSystemImpl getFreeSwapSpaceSize in docker after 8242480 In-Reply-To: <2e640705a69d8b6f71506c1bed16cae7467429d3.camel@redhat.com> References: <2e640705a69d8b6f71506c1bed16cae7467429d3.camel@redhat.com> Message-ID: Hi Severin, >In summary: Patch looks OK to me. Thanks for the review. May I have a second review ? >This comment --^ is wrong. Should be "we see a limit == -1 ..." > > 65 return (limit < memLimit) ? 0 : limit - memLimit; > 66 } > >Not sure if you want to change that as part of this patch. Better not in this change . Best regards, Matthias From matthias.baesken at sap.com Fri Jun 5 10:30:57 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 5 Jun 2020 10:30:57 +0000 Subject: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> Message-ID: > Here would be my suggestion > (hopefully this one isn't too restrictive again): > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ Hi Severin, this worked nicely . Thanks for providing this . Best regards, Matthias -----Original Message----- From: Severin Gehwolf Sent: Donnerstag, 4. Juni 2020 18:56 To: Baesken, Matthias ; jiefu(??) ; David Holmes ; 'hotspot-dev at openjdk.java.net' Cc: Bob Vandette Subject: Re: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java Hi Matthias, Thanks for running this to get better diagnostics! On Thu, 2020-06-04 at 14:11 +0000, Baesken, Matthias wrote: > HI Severin, with your new patch I run into : > > java.lang.RuntimeException: 'OperatingSystemMXBean.getTotalSwapSpaceSize: [1-9][0-9]+' missing from stdout/stderr > > at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:189) > at TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMemoryAwareness.java:165) > at TestMemoryAwareness.main(TestMemoryAwareness.java:66) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) > at java.base/java.lang.Thread.run(Thread.java:832) > > Did you really mean in line 165 of TestMemoryAwareness.java > out.shouldContain("OperatingSystemMXBean.getTotalSwapSpaceSize: [1-9][0-9]+"); > ? > When I fix it (to shouldMatch , this was probably meant by you) Yes, that's what I meant. > I get the next error - > java.lang.RuntimeException: 'Metrics.getMemoryLimit() == -1' missing from stdout/stderr > > I have > > Metrics.getMemoryAndSwapLimit() == -1 > Metrics.getMemoryLimit() == 104857600 > Metrics.getMemoryAndSwapUsage() == -1 > Metrics.getMemoryUsage() == 30670848 OK. I've added this info to the bug and some more bits and pieces. I believe there isn't much we can do but to bail out and let the test pass... It would be good to only pass the test that way on a system without kernel support for swap limits. Here would be my suggestion (hopefully this one isn't too restrictive again): http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ If we changed OperatingSystemMXbean as suggested by Bob to return 0 in this case for getTotalSwapSpaceSize(), it would be incorrect for the case as described in the bug (--memory-swap=-1). That would actually be unlimited swap. > Could we omit the > > out.shouldContain("Metrics.getMemoryLimit() == -1"); > > check ? We could. I'd rather be more specific and use expectedMemoryLimit instead, though. That's what the webrev above does. Feel free to change it in a way so that it passes on the affected system as I'd have to keep guessing :) In conclusion: This seems a test bug to me. Thanks, Severin From sgehwolf at redhat.com Fri Jun 5 14:07:17 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 05 Jun 2020 16:07:17 +0200 Subject: RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> Message-ID: <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> Hi Matthias, On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: > > Here would be my suggestion > > (hopefully this one isn't too restrictive again): > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ > > Hi Severin, this worked nicely . > Thanks for providing this . Great. If somebody else is willing to OK this, I'll push it. Thanks, Severin From igor.ignatyev at oracle.com Fri Jun 5 14:37:06 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 5 Jun 2020 07:37:06 -0700 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: References: Message-ID: <78150322-D75B-4BDA-BD07-6715DE99D0E3@oracle.com> Vladimir, Leonid, Thank you for your reviews! pushed. -- Igor > On Jun 4, 2020, at 3:55 PM, Leonid Mesnik wrote: > > Looks good. > > Leonid > On Jun 5, 2020, at 1:02 AM, Vladimir Ivanov wrote: > > Looks good. > > Best regards, > Vladimir Ivanov > > On 27.05.2020 20:25, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >>> 8 lines changed: 2 ins; 0 del; 6 mod; >> Hi all, >> could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? >> from JBS: >>> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` >> the patch >> - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts >> - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP >> JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 >> testing: tier1 >> webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >> Thanks, >> -- Igor From igor.ignatyev at oracle.com Fri Jun 5 16:10:37 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 5 Jun 2020 09:10:37 -0700 Subject: RFR(S) : 8246494 : introduce vm.flagless at-requires property In-Reply-To: <5b1cac8c-7e9b-195a-edfa-6ab972e32bf0@oracle.com> References: <5b1cac8c-7e9b-195a-edfa-6ab972e32bf0@oracle.com> Message-ID: Hi Per, you are reading this correctly, make TEST=test/hotspot/jtreg/gc/z/TestSmallHeap.java JTREG="VM_OPTIONS=-XX:+UseZGC" won't execute gc/z/TestSmallHeap.java; and I don't see it to be incorrect. Let me try to explain why using gc/z/TestSmallHeap.java as a running example. A hotspot test is expected not to be just runnable in an out-of-box configuration, but also to serve its purpose as much as possible (which is not always 100% given some tests require special build flavor, environment setup, etc); in other words, a test is to at least have all necessary VM flags within it and not to hope that someone will provide them. gc/z/TestSmallHeap.java does that, it explicitly selects zGC, so there is no need for -XX:+UseZGC to achieve that. Given this test can be run only when zGC can be selected, it @requires vm.gc.Z, which is set to true if zGC is already explicitly selected or if zGC is available and no other GC is specified, and the latter holds for an out-of-box configuration (assuming that zGC is available in the JVM under test); thus, again, you don't have to specify -XX:+UseZGC to run this test. So there are no "technical" reasons to run gc/z/TestSmallHeap.java (or any other gc/z/ tests) with -XX:+UseZGC. The proposed patches don't change that fact in any way. The patches exclude the tests that ignore external VM flags from execution if any significant VM flags are specified. gc/z/TestSmallHeap.java ignores all externally provided VM flags, including -XX:+UseZGC. And although in the case of -XX:+UseZGC, it's harmless, in almost all other cases it's not. Just to give you a few examples: Let's say you are fixing a bug in zGC which could be reproduced by gc/z/TestSmallHeap.java. You came up with two alternative solutions, one of which is guarded by `if (UseNewCode)`. To test these solutions, you ran gc/z tests twice: with -XX:+UseZGC -XX:+UseNewCode, and all tests passed; with XX:+UseZGC, and many tests (but not gc/z/TestSmallHeap.java) failed. So based on these results, you decided that the guarded solution is perfect, cleaned up the code, sent it out for review, got it pushed, and minutes later found out that gc/z/TestSmallHeap.java and some other tests which ignore VM flags failed. It would take you some time, to realize that you hadn't tested your UseNewCode solution by these tests. Yet were these tests excluded from your testing, it would be much easier for you to spot that and react accordingly. Here is another scenario, you decided to change the default value of ZUncommit, so you ran different tests with `XX:+UseZGC -XX:-ZUncommit`, all green, you pushed a trivial change s/true/false in z_globals.hpp, next thing you knew a bunch of zGC specific tests failed in CI. And again, these were the tests that silently ignored `XX:+UseZGC -XX:-ZUncommit`. Or a slight variation, zGC-supported was added to a future JIT, gc/z tests were run with the flag combination which enabled the future JIT, all passed, the victory was declared; N releases later; default JIT got changed to the future JIT; the next CI build is a disaster, with lots of tests failing from the bugs which had not been found N/2 years ago. Although I understand that it might take some getting used to from you and others who used to run gc/x tests with -XX:+Use${X}GC, I am certain that this will improve the overall quality of hotspot, save not only machine time (from running these tests with other flags) but engineers time from analyzing surprising failures, and increase confidence and trust in the hotspot test suite. In a word, I can see how this can be a bit surprising, yet still less surprising than the current behavior, but I don't see it as incorrect, it just surfaces limitations of certain tests. From my (slightly biased) point of view, it's the right thing to do. Thanks. -- Igor > On Jun 5, 2020, at 1:20 AM, Per Liden wrote: > > Hi Igor, > > When looking at the follow-up sub-tasks for this, I see for example this: > > http://cr.openjdk.java.net/~iignatyev/8246499/webrev.00/test/hotspot/jtreg/gc/z/TestSmallHeap.java.udiff.html > > Maybe I'm misunderstanding how this is supposed to work, but it looks like this test would now _not_ be executed if I do: > > make TEST=test/hotspot/jtreg/gc/z/TestSmallHeap.java JTREG="VM_OPTIONS=-XX:+UseZGC" > > Is that so? In that case, that seems incorrect. > > cheers, > Per > > On 6/3/20 11:30 PM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 >>> 70 lines changed: 66 ins; 0 del; 4 mod >> Hi all, >> could you please review the patch which introduces a new @requires property to filter out the tests which ignore externally provided JVM flags? >> the idea behind this patch is to have a way to clearly mark tests which ignore flags, so >> a) it's obvious that they don't execute a flag-guarded code/feature, and extra care should be taken to use them to verify any flag-guarded changed; >> b) they can be easily excluded from runs w/ flags. >> @requires and VMProps allows us to achieve both, so it's been decided to add a new property `vm.flagless`. `vm.flagless` is set to false if there are any XX flags other than `-XX:MaxRAMPercentage` and `-XX:CreateCoredumpOnCrash` (which are known to be set almost always) or any X flags other `-Xmixed`; in other words any tests w/ `@requires vm.flagless` will be excluded from runs w/ any other X / XX flags passed via `-vmoption` / `-javaoption`. in rare cases, when one still wants to run the tests marked by `vm.flagless` w/ external flags, `vm.flagless` can be forcefully set to true by setting any value to `TEST_VM_FLAGLESS` env. variable. >> this patch adds necessary common changes and marks common tests, namely Scimark, GTestWrapper and TestNativeProcessBuilder. Component-specific tests will be marked separately by the corresponding subtasks of 8151707[1]. >> please note, the patch depends on CODETOOLS-7902336[2], which will be included in the next jtreg version, so this patch is to be integrated only after jtreg5.1 is promoted and we switch to use it by 8246387[3]. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8246494 >> webrev: http://cr.openjdk.java.net/~iignatyev//8246494/webrev.00 >> testing: marked tests w/ different XX and X flags w/ and w/o TEST_VM_FLAGLESS env. var, and w/o any flags >> [1] https://bugs.openjdk.java.net/browse/JDK-8151707 >> [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7902336 >> [3] https://bugs.openjdk.java.net/browse/JDK-8246387 >> Thanks, >> -- Igor From bob.vandette at oracle.com Fri Jun 5 18:41:33 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 5 Jun 2020 14:41:33 -0400 Subject: 8246648: issue with OperatingSystemImpl getFreeSwapSpaceSize in docker after 8242480 In-Reply-To: References: <2e640705a69d8b6f71506c1bed16cae7467429d3.camel@redhat.com> Message-ID: I?m good with this fix. Bob. > On Jun 5, 2020, at 6:17 AM, Baesken, Matthias wrote: > > Hi Severin, > >> In summary: Patch looks OK to me. > > Thanks for the review. > May I have a second review ? > > > >> This comment --^ is wrong. Should be "we see a limit == -1 ..." >> >> 65 return (limit < memLimit) ? 0 : limit - memLimit; >> 66 } >> >> Not sure if you want to change that as part of this patch. > > Better not in this change . > > Best regards, Matthias From magnus.ihse.bursie at oracle.com Fri Jun 5 23:57:25 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Sat, 6 Jun 2020 01:57:25 +0200 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> References: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> Message-ID: <511ed9e8-c35b-aa04-12ed-a1df2dd116b8@oracle.com> On 2020-06-05 13:59, Jim Laskey wrote: > I know there was a discussion about this elsewhere but I would like to take the opportunity to correct this now > > make//autoconf/flags-cflags.m4:241 > > elif test "x$TOOLCHAIN_TYPE" = xclang; then > if test "x$OPENJDK_TARGET_OS" = xmacosx; then > # On MacOSX we optimize for size, something > # we should do for all platforms? > C_O_FLAG_HIGHEST_JVM="-Os" > C_O_FLAG_HIGHEST="-Os" > C_O_FLAG_HI="-Os" > C_O_FLAG_NORM="-Os" > C_O_FLAG_DEBUG_JVM="" > else > C_O_FLAG_HIGHEST_JVM="-O3" > C_O_FLAG_HIGHEST="-O3" > C_O_FLAG_HI="-O3" > C_O_FLAG_NORM="-O2" > C_O_FLAG_DEBUG_JVM="-O0" > fi > C_O_FLAG_SIZE="-Os" > C_O_FLAG_DEBUG="-O0" > C_O_FLAG_NONE="-O0" > elif test "x$TOOLCHAIN_TYPE" = xxlc; then > > should be changed to > > elif test "x$TOOLCHAIN_TYPE" = xclang; then > C_O_FLAG_HIGHEST_JVM="-O3" > C_O_FLAG_HIGHEST="-O3" > C_O_FLAG_HI="-O3" > C_O_FLAG_NORM="-O2" > C_O_FLAG_DEBUG_JVM="-O0" > C_O_FLAG_SIZE="-Os" > C_O_FLAG_DEBUG="-O0" > C_O_FLAG_NONE="-O0" > elif test "x$TOOLCHAIN_TYPE" = xxlc; then > > MacOSX has been paying a historic and significant performance penalty for no valid reason. This might be a valid change, but it has nothing to do with C++14, and changing it at the same time will increase risk for unrelated strange errors. Please open a separate JBS issue for this requested change. /Magnus > > Otherwise +1. > > Cheers, > > -- Jim > > > >> On Jun 5, 2020, at 4:52 AM, Kim Barrett wrote: >> >> [Changes are only to the build system, but since the changes have jdk-wide >> effect I?ve cc?d what I think are the relevant dev lists.] >> >> This change is part of JEP 347; the intent is to target JDK 16. >> >> Please review this change to the building of C++ code in the JDK to >> enable the use of C++14 language features. This change does not make >> any code changes to use new features provided by C++11 or C++14. >> >> This requires trimming the supported compiler versions, moving the >> minimum supported versions forward (significantly, in some cases). >> The new minimums are based on compiler documentation rather than >> testing. It may be that more recent versions are actually required. >> >> This change needs to be tested on platforms not supported by Oracle. >> The JEP test plan includes additional Oracle testing beyond what I?ve done. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8246032 >> >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ >> >> Testing: >> mach5 tier1-5 on Oracle supported platforms. >> >> Performance testing showed no significant changes in either direction. >> >> Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le >> From magnus.ihse.bursie at oracle.com Fri Jun 5 23:57:54 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Sat, 6 Jun 2020 01:57:54 +0200 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: Message-ID: On 2020-06-05 09:52, Kim Barrett wrote: > [Changes are only to the build system, but since the changes have jdk-wide > effect I?ve cc?d what I think are the relevant dev lists.] > > This change is part of JEP 347; the intent is to target JDK 16. > > Please review this change to the building of C++ code in the JDK to > enable the use of C++14 language features. This change does not make > any code changes to use new features provided by C++11 or C++14. > > This requires trimming the supported compiler versions, moving the > minimum supported versions forward (significantly, in some cases). > The new minimums are based on compiler documentation rather than > testing. It may be that more recent versions are actually required. > > This change needs to be tested on platforms not supported by Oracle. > The JEP test plan includes additional Oracle testing beyond what I?ve done. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8246032 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ Looks good to me. /Magnus > > Testing: > mach5 tier1-5 on Oracle supported platforms. > > Performance testing showed no significant changes in either direction. > > Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le > From kim.barrett at oracle.com Sat Jun 6 00:21:03 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 5 Jun 2020 20:21:03 -0400 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <511ed9e8-c35b-aa04-12ed-a1df2dd116b8@oracle.com> References: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> <511ed9e8-c35b-aa04-12ed-a1df2dd116b8@oracle.com> Message-ID: <50E319B9-428D-46CE-A68C-5D2F31A2556E@oracle.com> > On Jun 5, 2020, at 7:57 PM, Magnus Ihse Bursie wrote: > > On 2020-06-05 13:59, Jim Laskey wrote: >> I know there was a discussion about this elsewhere but I would like to take the opportunity to correct this now >> >> make//autoconf/flags-cflags.m4:241 >> >> [?] >> MacOSX has been paying a historic and significant performance penalty for no valid reason. > This might be a valid change, but it has nothing to do with C++14, and changing it at the same time will increase risk for unrelated strange errors. Please open a separate JBS issue for this requested change. I was going to say much the same thing. I don?t want to add that to the C++14 mix. I don?t even know for sure that it?s a desirable change. I?ve worked on projects that got better (measured) performance from -Os than from other global alternatives. One would need to do a bunch of performance testing and might need to go more deeply into the options to really optimize things. From kim.barrett at oracle.com Sat Jun 6 00:21:56 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 5 Jun 2020 20:21:56 -0400 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: Message-ID: > On Jun 5, 2020, at 7:57 PM, Magnus Ihse Bursie wrote: > > On 2020-06-05 09:52, Kim Barrett wrote: >> [Changes are only to the build system, but since the changes have jdk-wide >> effect I?ve cc?d what I think are the relevant dev lists.] >> >> This change is part of JEP 347; the intent is to target JDK 16. >> >> Please review this change to the building of C++ code in the JDK to >> enable the use of C++14 language features. This change does not make >> any code changes to use new features provided by C++11 or C++14. >> >> This requires trimming the supported compiler versions, moving the >> minimum supported versions forward (significantly, in some cases). >> The new minimums are based on compiler documentation rather than >> testing. It may be that more recent versions are actually required. >> >> This change needs to be tested on platforms not supported by Oracle. >> The JEP test plan includes additional Oracle testing beyond what I?ve done. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8246032 >> >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ > Looks good to me. Thanks. > > /Magnus > >> >> Testing: >> mach5 tier1-5 on Oracle supported platforms. >> >> Performance testing showed no significant changes in either direction. >> >> Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le From thomas.stuefe at gmail.com Mon Jun 8 06:56:39 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 8 Jun 2020 08:56:39 +0200 Subject: RFR(S) : 8245610 : remove in-tree copy on gtest In-Reply-To: References: <93E621FF-580C-4FF9-8D8E-4BB33CB93A90@oracle.com> <0c5d41f4-eebd-a8ae-d537-e14a878a6663@oracle.com> <7B996E95-B57D-4F38-97F1-65FFCA2DD01E@oracle.com> <2A2FD345-F641-4D6A-85DC-6379E8582C6F@oracle.com> <4b855354-367d-a103-d2c9-242bca9fe5f8@oracle.com> Message-ID: Taking this up again, why could we just not roll back this change if it was ill advised? Also, it may be me, but where do I find the official build documentation? Googling "building openjdk" gives me a number of hits, neither one seems official - the top hit seems to be one from Magnus' private cr directory :) but since it does not mention google test at all I don't think this is recent. Thanks, Thomas On Mon, Jun 1, 2020 at 6:01 PM Thomas St?fe wrote: > Hi Magnus, > > On Mon, Jun 1, 2020 at 1:35 PM Magnus Ihse Bursie < > magnus.ihse.bursie at oracle.com> wrote: > >> >> >> On 2020-05-29 07:04, Thomas St?fe wrote: >> >> Hi Igor, >> >> thank you for taking the time to answer my grumblings. >> >> Yes, comparison with jtreg is a bit crooked - it is not needed to get a >> valid build. But jtreg is also maintained in the official OpenJDK >> repositories; I can clone codetools/jtreg and have the correct version. >> With gtests, I have to know which version OpenJDK needs, which is not the >> latest version, and have to download that from outside our repositories. >> Getting it wrong version may yield me difficult to analyze build errors >> (plattform zoo, handicapped C++ compilers etc). Also, updating to a new >> gtest version will now involve a lot more communicatio (a version check in >> configure would help with that). >> >> But this is a minor complaint, I could live with that. I most dislike the >> fact that I have to specify this option *every single time*, and that >> omitting it is objectively wrong and may give me a false sense of security. >> Omitting it gives no warning, but if my changes break gtests I will only >> notice it hours later when jdk/submit results are back. >> >> Yes, I can hide this behind an alias or script, but I think this is the >> wrong way. You guys did such a good job in making the build dead easy: >> first time, it tells me exactly which Debian packets to install, I always >> loved that special touch. I specify boot jdk and maybe debug level and I am >> done. In fact, the build is so easy that until recently I did not even know >> we had a build documentation :) The new --with-gtest option is a jarring >> break from that. >> >> I agree, Thomas. The patch went in too fast, without proper exploration >> of alternatives, or how this would affect the usability of the build. >> >> One of the main goals of the build system has been that it should be easy >> to build, and the system should be "self-instructing", so that if a >> dependency is missing, this should be made clear, and a suitable suggestion >> for correction should be printed. >> >> The gtest removal fails on both these accounts. :( >> >> I think we should change configure so that gtest is a required >> dependency, unless specifically disabled. We can look for gtest in >> standard locations, like /usr/src/googletest, where it is installed by e.g. >> "sudo apt install googletest". >> > > I fear that we are more reliant on a specific version of googletest than > is the case with standard libraries. Just a gut feeling, but the fact that > we cannot use the latest googletest version is a strong indicator. So using > the OS-provided version may be the wrong one, which may be a constant > source of annoying build errors. > > Installation instructions, such as this apt command, must also be added. >> (I'd appreciate feedback on what the package is called on RH/Fedora!) We >> could also consider checking for an environment variable, similar to how >> the boot JDK looks at JAVA_HOME, so you can set up a non-standard path in >> your environment, and do not have to pass it to configure as a flag all the >> time. >> >> And we also need to fix the RunTest framework, as Ren? pointed out, so >> that if you try to run gtests without the gtest library, you need to get a >> proper error message that describes the step you need to take to be able to >> run gtest tests. >> >> /Magnus >> >> >> > Thanks, Thomas > > >> In my mind, gtest is in the same ballpark as the freetype library on >> Windows. That has always been a bit annoying but that was only Windows. >> Something you cannot rely the OS library mechanism to deliver but have to >> download and place yourself and tell the build about it. >> >> I wonder whether we can find a compromise somehow. Maybe something like >> an agreed on structure and place for a "companion directory", containing >> build dependencies like these. Its location could be by convention in a >> clear relation to the OpenJDK sources (e.g. just beside it) and configure >> would look for the libraries in there by default. Like this: >> >> - openjdk-source >> - configure >> - Makefile >> - .. >> - openjdk-builddeps >> - google-test >> - freetype >> - .. >> >> This would also lend itself very well to a maintained repository of build >> dependencies somewhere (not necessarily maintained by Oracle). We would >> have similar ease of use as with platform libraries, which by default are >> also expected in a standard place in the file system. >> >> Thanks, Thomas >> >> >> On Fri, May 29, 2020 at 5:20 AM Igor Ignatyev >> wrote: >> >>> Hi Thomas, >>> >>> I regret that this patch made you unhappier. I fully agree that all >>> hotspot developers are to always build *and* run gtest tests, yet not all >>> people who work on OpenJDK project are hotspot developers. >>> >>> I also believe that all OpenJDK developers are to run tests related to >>> the area they are changing. The vast majority of such tests are jtreg >>> tests. And jtreg, for better or worse, is a counterexample to your last >>> paragraph -- it's an essential part of OpenJDK, but it doesn't come in a >>> form of well established library, and none of known to me platforms have >>> jtreg in their package manager, so you do have to manually download jtreg >>> and specify its location one way or another. I understand that there is a >>> difference b/w gtest and jtreg, as jtreg isn't really need at build time. >>> However the proper way to run any of OpenJDK tests is by `make test` and >>> for it to work you need to either executed configure w/ --with-jtreg or >>> specify JT_HOME on each invocation of `make test`, where the latter is a >>> preferred way. From that point of view, gtest and jtreg aren't different, >>> you need to download both manually and specify their location at >>> configure-time. >>> >>> with that being said, I truly understand the inconvenience it causes to >>> you, yet given you need to download gtest only once and it's fairly easy to >>> hide `with-gtest` behind a script or an alias like configure_openjdk='bash >>> ./configure --with-gtest=$GTEST_HOME', I hope it won't cause problems for >>> you and won't discourage you in anyways. >>> >>> Thanks, >>> -- Igor >>> >>> On May 28, 2020, at 12:31 AM, Thomas St?fe >>> wrote: >>> >>> Hi, >>> >>> I know the boat has sailed, I missed this one. But I am a bit unhappy >>> about this. >>> >>> I always build with gtests; I think it makes no sense to not build with >>> gtest. Even if you do not want to run the gtests (and it makes sense to >>> always do since its a good first-base validity test), hotspot developers >>> should always build them since changes in the hotspot sources may break >>> hotspot gtests. hotspot source and gtests are a unity. >>> >>> So if it makes no sense to not build gtest, I should not need a special >>> option to specify gtest location - I'd argue that the default case should >>> not require special options. >>> >>> The JBS issue states that "it can be treated like any other external >>> dependencies" but this comparison does not hold - almost all other >>> dependencies come in the form of well established libraries with standard >>> headers and standard installation locations which can be coded as default >>> values. On a recent mainstream platform I do not have to specify any paths >>> to libraries to build OpenJDK. This is now different, I always have to >>> manually download gtests and specify gtest location. This is regrettable. >>> >>> Thanks, Thomas >>> >>> >>> On Tue, May 26, 2020 at 2:27 PM Magnus Ihse Bursie < >>> magnus.ihse.bursie at oracle.com> wrote: >>> >>>> >>>> >>>> On 2020-05-25 19:53, Igor Ignatyev wrote: >>>> > Hi Magnus, >>>> > >>>> >> On May 25, 2020, at 7:46 AM, Magnus Ihse Bursie >>>> >> >>> >> > wrote: >>>> >> >>>> >> Looks basically good to me. >>>> > thanks for your review! >>>> >> >>>> >> We need to document the dependency on gtest, and how to retrieve it. >>>> >> I recommend you add a section next to the JTReg information (under >>>> >> the "## Running Tests" heading) in doc/building.md. I think you >>>> >> should include basically the same information as you did in your >>>> >> follow-up mail, that was informative and clear. >>>> > that's a good suggestion, I've added a small paragraph to 'Running >>>> > Tests' in doc/building.md and regenerated corresponding .html file -- >>>> > http://cr.openjdk.java.net/~iignatyev/8245610/webrev.doc/ >>>> > please let me know if you think something should be added or reworded. >>>> Looks great! Thank you. >>>> >>>> /Magnus >>>> > >>>> >> >>>> >> I assume the most suitable replacement for developers who are used >>>> to >>>> >> add a '--disable-hotspot-gtest' to e.g. a pre-definied jib >>>> >> configuration is to now use '--without-gtest'. This will need to be >>>> >> communicated, perhaps to both hotspot-dev and jdk-dev. >>>> > sure, after this gets integrated, I'll let both hotspot-dev and >>>> > jdk-dev aliases know which changes might be required to >>>> enable/disable >>>> > hotspot gtest tests compilation. >>>> > >>>> > Thanks. >>>> > -- Igor >>>> > >>>> >> >>>> >> /Magnus >>>> >> >>>> >> On 2020-05-22 20:12, Igor Ignatyev wrote: >>>> >>> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00/ >>>> >>>> 132 lines changed: 80 ins; 36 del; 16 mod >>>> >>> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00%2bremoval/ >>>> >>>> 57482 lines changed: 80 ins; 57385 del; 17 mod; >>>> >>> Hi all, >>>> >>> >>>> >>> could you please review this small (if you ignore removal part) >>>> >>> patch which removes in-tree copy of gtest (test/fmw/gtest) and >>>> >>> updates makefiles to use one provided thru an added configure >>>> option >>>> >>> `--with-gtest`? the previously used configure option >>>> >>> `--enable-hotspot-gtest` gets depricated. >>>> >>> >>>> >>> the patch also compiles gtest and gmock source code into a static >>>> >>> library so they can be compiled w/ all warnings disabled. >>>> >>> >>>> >>> from JBS: >>>> >>>> w/ JEP 381 (JDK-8241787 / JDK-8244224) being integrated, all >>>> >>>> compilers used by OpenJDK became supported by gtest out-of-box, so >>>> >>>> there is no need to have our copy of gtest framework (including >>>> >>>> gmock) within OpenJDK source tree. instead, it can be treated like >>>> >>>> any other external dependencies, and a pointer to the directory w/ >>>> >>>> gtest code can be passed via configure options. >>>> >>> >>>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245610 >>>> >>> webrevs: >>>> >>> - http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00/ >>>> >>> (meaningful changes) >>>> >>> - >>>> >>> http://cr.openjdk.java.net/~iignatyev/8245610/webrev.00%2bremoval/ >>>> >>> (all changes) >>>> >>> testing: >>>> >>> - gtest tests on {linux,windows,macosx}-x64; >>>> >>> - tier[1-5] builds which include but not limited to linux-aarch64, >>>> >>> linux-arm32, linux-x64-zero >>>> >>> >>>> >>> Thanks, >>>> >>> -- Igor >>>> >>> >>>> >> >>>> > >>>> >>>> >>> >> From david.holmes at oracle.com Mon Jun 8 09:33:23 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 8 Jun 2020 19:33:23 +1000 Subject: RFR[15] JDK-8243114: Implement montgomery{Multiply, Square}intrinsics on Windows In-Reply-To: <101b5758-3660-d2e4-e20a-ffc362907b45@redhat.com> References: <725e8e5a-71b5-7d6e-4d3a-7c799a52ab56@redhat.com> <2442a046-6640-f759-6330-e971405f0c78@oracle.com> <6cb701cd-5652-6b9d-8051-b37adb3ce63f@oracle.com> <44d1daf2-7124-9b4b-b039-d35d1f8e96c6@oracle.com> <101b5758-3660-d2e4-e20a-ffc362907b45@redhat.com> Message-ID: <2b3e7ee7-185e-3140-583f-3c0a39fbe8b5@oracle.com> Hi Simon, On 6/06/2020 6:46 am, Simon Tooke wrote: > Thanks again for the review. > > As per your and Andrew Haley's comments, I have updated the webrev: > > - used NOINLINE > > - used julong > > - deleted the block of unused code. > > Please let me know what you think. > > updated webrev: > http://cr.openjdk.java.net/~stooke/webrevs/jdk-8243114-jdk/01/01/ All my comments have been addressed - thanks. David > Thanks, > > -Simon > > > > On 2020-06-05 12:35 a.m., David Holmes wrote: >> Hi Simon, >> >> On 4/06/2020 11:35 pm, Simon Tooke wrote: >>> Hello, David, and thanks for the review! >>> >>> I've responded to your comments below, and intend to post a new patch >>> for review today or tomorrow. >>> >>> Thanks again, >>> >>> -Simon >>> >>> On 2020-06-03 2:06 a.m., David Holmes wrote: >>>> Hi Simon, >>>> >>>> On 23/05/2020 12:04 am, Sean Mullan wrote: >>>>> Cross-posting to hotspot-dev for additional review since the code >>>>> changes are in hotspot. >>>>> >>>>> --Sean >>>>> >>>>> On 5/21/20 1:24 PM, Simon Tooke wrote: >>>>>> Hello, >>>>>> >>>>>> I'd like to request a review for: >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243114 >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~stooke/webrevs/jdk-8243114-jdk/00/00/ >>>>>> >>>>>> This change implements the given intrinsics on Windows. >>>>>> >>>>>> The Windows toolchain has no 128 bit integer types, and no inline >>>>>> asm (on x64 and Arm).? In addition, 'long' is 4 bytes, not 8, as >>>>>> it is with gcc.? This patch had to change some of the linux >>>>>> implementation to account for these limitations. >>>> >>>> I can't really comment on the intrinsics themselves but a couple of >>>> suggestions: >>>> >>>> - use explicitly sized types e.g. uint64_t instead of unsigned long >>>> long >>> >>> Yes, this hurt to type.? A previous review suggested using julong, is >>> that acceptable to you? >> >> j* types should only be used when dealing with values that come from >> or go to Java. Obviously julong is a misnomer as Java doesn't have an >> unsigned type, but in general this is something we are trying to fix >> up in the codebase. If these arrays are extracted from Java arrays >> then using a j* type would be appropriate, but then I would expect >> jlong, unless the algorithm explicitly requires unsigned types? If so >> julong would be acceptable. >> >>> (an aside: I'm now wondering if there is other code in the JDK that >>> assumes long is 64bits - which is not true on all platforms...) >> >> There has been such code, but hopefully there is no remaining actively >> used code with that bug. There are using of "long" that should be >> eradicated (there's an open JBS issue for that as I recall) but the >> remaining uses don't seem to require the long be 64-bit. >> >>>> - use the existing NOINLINE macro for the _declspec(noinline) >>> Thanks, will do. >>>> >>>> The conditional compilation in this code has me quite confused. >>>> Looking at the existing code we have: >>>> >>>> 3680 #ifndef _WINDOWS >>>> 3681 >>>> 3682 #define ASM_SUBTRACT >>>> 3683 >>>> 3684 #ifdef ASM_SUBTRACT >>>> ... >>>> 3702 #else // ASM_SUBTRACT >>>> ... >>>> 3719 #endif // ! ASM_SUBTRACT >>>> >>>> So all the above code is only built on not-Windows, and we >>>> unconditionally define ASM_SUBTRACT, so lines 3702 to 3719 appear to >>>> be dead code! I'm not at all sure whether that is actually a bug and >>>> the windows ifndef should have had an endif after line 3682; or >>>> whether we can just simplify the code. >>> The dead code existed prior to this patch, so I wasn't proposing >>> removing it.? I'm happy to do so if that's for the best.? As far as I >>> can tell, it's for implementing these intrinsics on gcc platforms >>> without assembler. >> >> AFAICS Andrew originally had the ASM_SUBTRACT parts, with the >> unconditional #define, but there was no explanation in the review >> thread as to why the unused code remained present. Then before >> integration all the code was wrapped by the ifndef Windows to exclude >> it from Windows. >> >> I think it needs to be fixed as you are making changes to the Windows >> part and it is very hard to establish how the dead code should look in >> that case. >> >> Thanks, >> David >> >>>> >>>> Thanks, >>>> David >>>> >>>>>> >>>>>> My gratitude for Andrew Haley for doing the heavy lifting at the >>>>>> core of this patch. >>>>>> >>>>>> The patch, if accepted, will be offered to 11u as a potential >>>>>> backport. The changes apply cleanly modulo some line number changes. >>>>>> >>>>>> As for the speedup, this test case: >>>>>> >>>>>> BigInteger base = BigInteger.ONE.shiftLeft(1024); >>>>>> long count = LongStream.rangeClosed(2, 100_000) >>>>>> ???? .mapToObj(n -> BigInteger.valueOf(n).add(base)) >>>>>> ???? .filter(i -> i.isProbablePrime(50)) >>>>>> ???? .count(); >>>>>> >>>>>> goes from 1 minute 20 seconds down to about 35 seconds om my VM, >>>>>> over multiple runs.? As is the case on other platforms where the >>>>>> intrinsics are supported, they will be enabled by default on Windows. >>>>>> >>>>>> Thank you for your time, >>>>>> >>>>>> -Simon Tooke >>>>>> >>>>>> >>>>>> Principal Software Engineer, >>>>>> >>>>>> Red Hat Canada >>>>>> >>>>>> >>>>>> >>>> >>> >> > From aph at redhat.com Mon Jun 8 11:19:59 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 8 Jun 2020 12:19:59 +0100 Subject: RFR[15] JDK-8243114: Implement montgomery{Multiply, Square}intrinsics on Windows In-Reply-To: <101b5758-3660-d2e4-e20a-ffc362907b45@redhat.com> References: <725e8e5a-71b5-7d6e-4d3a-7c799a52ab56@redhat.com> <2442a046-6640-f759-6330-e971405f0c78@oracle.com> <6cb701cd-5652-6b9d-8051-b37adb3ce63f@oracle.com> <44d1daf2-7124-9b4b-b039-d35d1f8e96c6@oracle.com> <101b5758-3660-d2e4-e20a-ffc362907b45@redhat.com> Message-ID: On 05/06/2020 21:46, Simon Tooke wrote: > As per your and Andrew Haley's comments, I have updated the webrev: > > - used NOINLINE > > - used julong > > - deleted the block of unused code. > > Please let me know what you think. > > updated webrev: > http://cr.openjdk.java.net/~stooke/webrevs/jdk-8243114-jdk/01/01/ That looks clean, especially since it adds support for Windows with essentially no impact on the Linux code. I'm not sure I can Review it, though, because I'm one of the co-authors. It'd be nice if someone else familiar with the arithmetic (someone on security-dev?) could take a quick look. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.joelsson at oracle.com Mon Jun 8 13:12:27 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 8 Jun 2020 06:12:27 -0700 Subject: RFR(S) : 8245610 : remove in-tree copy on gtest In-Reply-To: References: <93E621FF-580C-4FF9-8D8E-4BB33CB93A90@oracle.com> <0c5d41f4-eebd-a8ae-d537-e14a878a6663@oracle.com> <7B996E95-B57D-4F38-97F1-65FFCA2DD01E@oracle.com> <2A2FD345-F641-4D6A-85DC-6379E8582C6F@oracle.com> <4b855354-367d-a103-d2c9-242bca9fe5f8@oracle.com> Message-ID: <07d49481-2f1f-4325-4063-dc3773322d81@oracle.com> On 2020-06-07 23:56, Thomas St?fe wrote: > Also, it may be me, but where do I find the official build > documentation? Googling "building openjdk" gives me a number of hits, > neither one seems official - the top hit seems to be one from Magnus' > private cr directory :) but since it does not mention google test at > all I don't think this is recent. > The official building doc is found in the source tree in doc/building.[md|html]. /Erik From thomas.stuefe at gmail.com Mon Jun 8 13:17:15 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 8 Jun 2020 15:17:15 +0200 Subject: RFR(S) : 8245610 : remove in-tree copy on gtest In-Reply-To: <07d49481-2f1f-4325-4063-dc3773322d81@oracle.com> References: <93E621FF-580C-4FF9-8D8E-4BB33CB93A90@oracle.com> <0c5d41f4-eebd-a8ae-d537-e14a878a6663@oracle.com> <7B996E95-B57D-4F38-97F1-65FFCA2DD01E@oracle.com> <2A2FD345-F641-4D6A-85DC-6379E8582C6F@oracle.com> <4b855354-367d-a103-d2c9-242bca9fe5f8@oracle.com> <07d49481-2f1f-4325-4063-dc3773322d81@oracle.com> Message-ID: On Mon, Jun 8, 2020 at 3:14 PM Erik Joelsson wrote: > On 2020-06-07 23:56, Thomas St?fe wrote: > > Also, it may be me, but where do I find the official build > > documentation? Googling "building openjdk" gives me a number of hits, > > neither one seems official - the top hit seems to be one from Magnus' > > private cr directory :) but since it does not mention google test at > > all I don't think this is recent. > > > The official building doc is found in the source tree in > doc/building.[md|html]. > > /Erik > Thank you Erik. From igor.ignatyev at oracle.com Mon Jun 8 16:07:20 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 8 Jun 2020 09:07:20 -0700 Subject: RFR(S/L) : 8244614 : cleanup keywords used/available in hotspot testbase In-Reply-To: <238C8726-BEFB-480B-8D02-D986EB068033@oracle.com> References: <939A28A1-5CEA-438D-A2EE-E547E9C7D57E@oracle.com> <8F94C0EC-3470-4252-A18B-72F68FDA3BC5@oracle.com> <238C8726-BEFB-480B-8D02-D986EB068033@oracle.com> Message-ID: ping? -- Igor > On Jun 4, 2020, at 1:23 PM, Igor Ignatyev wrote: > > ping? > > -- Igor > >> On May 29, 2020, at 10:49 AM, Igor Ignatyev wrote: >> >> ouch,.. it appears that this tier1 execution wasn't in vain. I realized that all my queries were limited to the tests `jtreg -l` gave me, which excludes tests which can't be w/ my default build on my localhost... >> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01-02 updates 93 more tests. >> >> -- Igor >> >>> On May 29, 2020, at 10:14 AM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>> >>> >>> Hi all, >>> >>> given the absense of comments, I assume that nobody has any strong feeling these to-be-removed keywords , so I'm repurposing this thread as RFR. >>> >>> the biggest part of the patch was done by sed: >>> - removal of $key : gsed -ri "s/(@key.*) $key/\1/g" {} ; where $key is in nmt cte_test, jcmd, regression >>> - removal of empty @key : gsed -ri '/^ \* @key\s*$/d' >>> >>> the manually done changes are limited to: >>> - removal of keywords from TEST.ROOT >>> - adding k/w description to TEST.ROOT >>> - adding @bug to redefclass031 >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244614 >>> webrevs: >>> - part done by sed: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.sed >>>> 594 lines changed: 0 ins; 205 del; 389 mod; >>> >>> - part done by me: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>> >>> - whole (incl. copyright year update) : http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>> >>> >>> testing: >>> - grep-ed for removed keywords >>> - jtreg -l >>> - tier1 (just for the good luck) >>> >>> Thanks, >>> -- Igor >>> >>> >>> >>>> On May 22, 2020, at 3:42 PM, Igor Ignatyev wrote: >>>> >>>> Hi all, >>>> >>>> I'm planning to clean up jtreg keywords that we have in test/hotspot/jtreg as most of them seem to be useless, confuse and make people to add these keywords in vain. After looking at how hotspot k/w are used, as well as which k/w are available and used by other test suites, I'd like to propose the following: >>>> - keep and document 'headful', 'stress' as they are useful for test selection; >>>> - keep and document 'intermittent', 'randomness' as they bring extra information about tests stability/reproducibility/determinism; >>>> - remove 'nmt', 'gc', 'metaspace' as they are used only in obvious places and aren't informative; >>>> - remove 'cte_test' as it's an artifact of old days and doesn't bring any useful information anymore; >>>> - remove 'jcmd', not all tests which depend on jcmd has this k/w, so it can't be used for test selection/filtering; there is also no obvious reasons to single out jcmd from other binaries bundled w/ JDK >>>> - remove 'regression' as the usage of at-bug is more common and convey the same meaning; and for two tests which have regression k/w, but don't have at-bug: >>>> - mark vmTestbase/nsk/jvmti/RedefineClasses/redefclass031/TestDescription.java w/ at-bug 6318850; >>>> - don't mark runtime/NMT/ReleaseNoCommit.java as it wasn't integrated as a true regression test, but rather as a function test for NMT w/ bunch of other NMT tests (where aren't marked w/ 'regression' k/w) by JDK-8054952. >>>> >>>> I'd like to receive some feedback on the proposed changes before I (more likely sed) remove keywords from 500+ tests and spend time testing that. >>>> >>>> [1] is the preliminary webrev just to illustrate the change, it only updates TEST.ROOT file and adds @bug to redefclass031 test. >>>> >>>> Thanks, >>>> -- Igor >>>> >>>> for reference, k/w available in other test suites: >>>> jdk : 2d dnd headful i18n intermittent printer randomness jfr >>>> langtools: intermittent randomness >>>> jaxp: intermittent randomness >>>> >>>> [1] http://cr.openjdk.java.net/~iignatyev//8244614/webrev.00 >>>> >>>> >>> >> > From gerard.ziemski at oracle.com Mon Jun 8 18:18:15 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 8 Jun 2020 13:18:15 -0500 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: <5fabaaee-dc10-0fe5-823a-72295d932559@oracle.com> References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> <5fabaaee-dc10-0fe5-823a-72295d932559@oracle.com> Message-ID: <79dadca8-c34b-6923-9c6c-58df3c17220d@oracle.com> hi Vladimir, My testing show no problem with the "-O2" flag, however, using just "-Os" (i.e. dropping "-O1" from JvmOverrideFiles.gmk") shows about the same performance as adding "-O2", and in fact it is slightly fastest for the TestRandom use case: -Os -O1 (the current default for unsafe.cpp) takes 196s -Os (the default for most hotspot files) takes 158s -O2 takes 161s -O3 takes 161s Can you reproduce my findings? cheers On 6/3/20 2:59 PM, gerard ziemski wrote: > hi Vladimir, I will take a look to see if I can help review this... > > > On 6/3/20 12:38 PM, Vladimir Kempik wrote: >> Hello >> >> Can somebody please review this simple change ? >> >> Thanks >> >>> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >>> >>> Adding hotspot-runtime-dev >>> >>>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik >>>> ???????(?): >>>> >>>> >>>> Hello >>>> Please review a fix for JDK-8243470 >>>> >>>> Long time ago as part of JEP284: New HotSpot Build System this fix >>>> was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>>> At that time it was decided to lower optimisation level for >>>> unsafe.cpp from O2 to O1 only for clang on Macosx. >>>> >>>> I suppose it was done due to issues in Set/Get helper >>>> functions where too optimistic optimisations were eliminating some >>>> null pointer checks. it was probably a clang bug. >>>> That issue could be checked with test >>>> jdk/test/sun/misc/CopyMemory.java. >>>> >>>> I believe that workaround (going from O2 to O1) produced this issue >>>> - https://bugs.openjdk.java.net/browse/JDK-8234963 >>>> (Thread.getStackTrace is slow with clang). >>>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>>> >>>> Here I propose the patch which eliminates that workaround for clang >>>> 8+. >>>> >>>> I have tested clang versions 8/9/9.1/10, all of them showed good >>>> results: >>>> 1) CopyMemory test passes fine on 11/14/15. >>>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. >>>> jdk15: no new failures in tck. >>>> 3) The testRandom "benchmark" from 8234963 shows great improvements >>>> on my machine, going down from ~200 seconds to ~150 seconds (the >>>> newer clang the better result). For comparision, gcc built libjvm >>>> for jdk11 shows ~130 seconds on my machine. >>>> >>>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>>> >>>> getStackTrace benchmark: >>>> http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>>> >>>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>>> >>>> Thanks, Vladimir > From stefan.karlsson at oracle.com Mon Jun 8 18:21:07 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 8 Jun 2020 20:21:07 +0200 Subject: RFR [XS]: 8244196: adjust output in os_linux In-Reply-To: References: Message-ID: <2eaf4b16-a4b1-cb68-f21e-e7cf129e1a73@oracle.com> Hi all, I encountered the output of this today, and I think the current output looks pretty bad. From my hs_err file I have this: CPU:total 32 (initial active 32) (8 cores per cpu, 2 threads per core) ... much more ... Online cpus: 0-31 Offline cpus: cache level: 1 cache type: Data cache size: 32K cache coherency line size: 64 cache level: 1 cache type: Instruction cache size: 32K cache coherency line size: 64 cache level: 2 cache type: Unified cache size: 256K cache coherency line size: 64 cache level: 3 cache type: Unified cache size: 20480K cache coherency line size: 64 BIOS frequency limitation: Frequency switch latency (ns): 0 Available cpu frequencies: Maximum cpu frequency: 3200000 Minimum cpu frequency: 1200000 Current cpu frequency: 2799113 Available governors: performance powersave Current governor: powersave Core performance/turbo boost: It's not easy at all to read. Would you mind if I changed this to: Online cpus: 0-31 Offline cpus: cache level: 1 cache type: Data cache size: 32K cache coherency line size: 64 cache level: 1 cache type: Instruction cache size: 32K cache coherency line size: 64 cache level: 2 cache type: Unified cache size: 256K cache coherency line size: 64 cache level: 3 cache type: Unified cache size: 20480K cache coherency line size: 64 BIOS frequency limitation: Frequency switch latency (ns): 0 Available cpu frequencies: Maximum cpu frequency: 3200000 Minimum cpu frequency: 1200000 Current cpu frequency: 1870913 Available governors: performance powersave Current governor: powersave Core performance/turbo boost: I don't think we should be too afraid of changing the lines here. We've have been changing other parts of the hs_err files, and if anyone is scraping this info, I think they'll need to adapt. Thanks, StefanK On 2020-04-30 17:46, Baesken, Matthias wrote: > Hello, please review this small change to os_linux.cpp . > > It uses the recently introduced _print_ascii_file_h > Function t more places (makes the code a bit shorter ) . > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8244196 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8244196.0/ > > Thanks, Matthias From coleen.phillimore at oracle.com Mon Jun 8 19:07:27 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 8 Jun 2020 15:07:27 -0400 Subject: RFR(S/L) : 8244614 : cleanup keywords used/available in hotspot testbase In-Reply-To: References: <939A28A1-5CEA-438D-A2EE-E547E9C7D57E@oracle.com> <8F94C0EC-3470-4252-A18B-72F68FDA3BC5@oracle.com> <238C8726-BEFB-480B-8D02-D986EB068033@oracle.com> Message-ID: Hi Igor, Looks good to me.? Many of those keywords were very historical and nobody cares about them.? Honestly I don't actually know how/if tests are/can be run by specifying keywords. thanks, Coleen On 6/8/20 12:07 PM, Igor Ignatyev wrote: > ping? > -- Igor > >> On Jun 4, 2020, at 1:23 PM, Igor Ignatyev wrote: >> >> ping? >> >> -- Igor >> >>> On May 29, 2020, at 10:49 AM, Igor Ignatyev wrote: >>> >>> ouch,.. it appears that this tier1 execution wasn't in vain. I realized that all my queries were limited to the tests `jtreg -l` gave me, which excludes tests which can't be w/ my default build on my localhost... >>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01-02 updates 93 more tests. >>> >>> -- Igor >>> >>>> On May 29, 2020, at 10:14 AM, Igor Ignatyev wrote: >>>> >>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>>> >>>> Hi all, >>>> >>>> given the absense of comments, I assume that nobody has any strong feeling these to-be-removed keywords , so I'm repurposing this thread as RFR. >>>> >>>> the biggest part of the patch was done by sed: >>>> - removal of $key : gsed -ri "s/(@key.*) $key/\1/g" {} ; where $key is in nmt cte_test, jcmd, regression >>>> - removal of empty @key : gsed -ri '/^ \* @key\s*$/d' >>>> >>>> the manually done changes are limited to: >>>> - removal of keywords from TEST.ROOT >>>> - adding k/w description to TEST.ROOT >>>> - adding @bug to redefclass031 >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244614 >>>> webrevs: >>>> - part done by sed: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.sed >>>>> 594 lines changed: 0 ins; 205 del; 389 mod; >>>> - part done by me: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>>> - whole (incl. copyright year update) : http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>>> >>>> testing: >>>> - grep-ed for removed keywords >>>> - jtreg -l >>>> - tier1 (just for the good luck) >>>> >>>> Thanks, >>>> -- Igor >>>> >>>> >>>> >>>>> On May 22, 2020, at 3:42 PM, Igor Ignatyev wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I'm planning to clean up jtreg keywords that we have in test/hotspot/jtreg as most of them seem to be useless, confuse and make people to add these keywords in vain. After looking at how hotspot k/w are used, as well as which k/w are available and used by other test suites, I'd like to propose the following: >>>>> - keep and document 'headful', 'stress' as they are useful for test selection; >>>>> - keep and document 'intermittent', 'randomness' as they bring extra information about tests stability/reproducibility/determinism; >>>>> - remove 'nmt', 'gc', 'metaspace' as they are used only in obvious places and aren't informative; >>>>> - remove 'cte_test' as it's an artifact of old days and doesn't bring any useful information anymore; >>>>> - remove 'jcmd', not all tests which depend on jcmd has this k/w, so it can't be used for test selection/filtering; there is also no obvious reasons to single out jcmd from other binaries bundled w/ JDK >>>>> - remove 'regression' as the usage of at-bug is more common and convey the same meaning; and for two tests which have regression k/w, but don't have at-bug: >>>>> - mark vmTestbase/nsk/jvmti/RedefineClasses/redefclass031/TestDescription.java w/ at-bug 6318850; >>>>> - don't mark runtime/NMT/ReleaseNoCommit.java as it wasn't integrated as a true regression test, but rather as a function test for NMT w/ bunch of other NMT tests (where aren't marked w/ 'regression' k/w) by JDK-8054952. >>>>> >>>>> I'd like to receive some feedback on the proposed changes before I (more likely sed) remove keywords from 500+ tests and spend time testing that. >>>>> >>>>> [1] is the preliminary webrev just to illustrate the change, it only updates TEST.ROOT file and adds @bug to redefclass031 test. >>>>> >>>>> Thanks, >>>>> -- Igor >>>>> >>>>> for reference, k/w available in other test suites: >>>>> jdk : 2d dnd headful i18n intermittent printer randomness jfr >>>>> langtools: intermittent randomness >>>>> jaxp: intermittent randomness >>>>> >>>>> [1] http://cr.openjdk.java.net/~iignatyev//8244614/webrev.00 >>>>> >>>>> From igor.ignatyev at oracle.com Mon Jun 8 19:20:37 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 8 Jun 2020 12:20:37 -0700 Subject: RFR(S/L) : 8244614 : cleanup keywords used/available in hotspot testbase In-Reply-To: References: <939A28A1-5CEA-438D-A2EE-E547E9C7D57E@oracle.com> <8F94C0EC-3470-4252-A18B-72F68FDA3BC5@oracle.com> <238C8726-BEFB-480B-8D02-D986EB068033@oracle.com> Message-ID: <1D9CFB02-BA89-43BB-B109-3C33E718CCAC@oracle.com> Hi Coleen, Thanks for your review! > Honestly I don't actually know how/if tests are/can be run by specifying keywords. JFYI, jtreg has -k/-keywords option[1] for that, and you can use JTREG=KEYWORDS=<...>[2] to specify this expression if you use run-test framework. [1] > -k: | -keywords: > A keyword boolean expression for test selection. The > expression can contain keyword names, combined with & (and), > | (or), ! (not) and parentheses. [2] https://hg.openjdk.java.net/jdk/jdk/raw-file/ac47db0c7ccb/doc/testing.html#keywords Thanks, -- Igor > On Jun 8, 2020, at 12:07 PM, coleen.phillimore at oracle.com wrote: > > > Hi Igor, > > Looks good to me. Many of those keywords were very historical and nobody cares about them. Honestly I don't actually know how/if tests are/can be run by specifying keywords. > > thanks, > Coleen > > On 6/8/20 12:07 PM, Igor Ignatyev wrote: >> ping? >> -- Igor >> >>> On Jun 4, 2020, at 1:23 PM, Igor Ignatyev wrote: >>> >>> ping? >>> >>> -- Igor >>> >>>> On May 29, 2020, at 10:49 AM, Igor Ignatyev wrote: >>>> >>>> ouch,.. it appears that this tier1 execution wasn't in vain. I realized that all my queries were limited to the tests `jtreg -l` gave me, which excludes tests which can't be w/ my default build on my localhost... >>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01-02 updates 93 more tests. >>>> >>>> -- Igor >>>> >>>>> On May 29, 2020, at 10:14 AM, Igor Ignatyev wrote: >>>>> >>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>>>> >>>>> Hi all, >>>>> >>>>> given the absense of comments, I assume that nobody has any strong feeling these to-be-removed keywords , so I'm repurposing this thread as RFR. >>>>> >>>>> the biggest part of the patch was done by sed: >>>>> - removal of $key : gsed -ri "s/(@key.*) $key/\1/g" {} ; where $key is in nmt cte_test, jcmd, regression >>>>> - removal of empty @key : gsed -ri '/^ \* @key\s*$/d' >>>>> >>>>> the manually done changes are limited to: >>>>> - removal of keywords from TEST.ROOT >>>>> - adding k/w description to TEST.ROOT >>>>> - adding @bug to redefclass031 >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244614 >>>>> webrevs: >>>>> - part done by sed: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.sed >>>>>> 594 lines changed: 0 ins; 205 del; 389 mod; >>>>> - part done by me: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>>>> - whole (incl. copyright year update) : http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>>>> >>>>> testing: >>>>> - grep-ed for removed keywords >>>>> - jtreg -l >>>>> - tier1 (just for the good luck) >>>>> >>>>> Thanks, >>>>> -- Igor >>>>> >>>>> >>>>> >>>>>> On May 22, 2020, at 3:42 PM, Igor Ignatyev wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> I'm planning to clean up jtreg keywords that we have in test/hotspot/jtreg as most of them seem to be useless, confuse and make people to add these keywords in vain. After looking at how hotspot k/w are used, as well as which k/w are available and used by other test suites, I'd like to propose the following: >>>>>> - keep and document 'headful', 'stress' as they are useful for test selection; >>>>>> - keep and document 'intermittent', 'randomness' as they bring extra information about tests stability/reproducibility/determinism; >>>>>> - remove 'nmt', 'gc', 'metaspace' as they are used only in obvious places and aren't informative; >>>>>> - remove 'cte_test' as it's an artifact of old days and doesn't bring any useful information anymore; >>>>>> - remove 'jcmd', not all tests which depend on jcmd has this k/w, so it can't be used for test selection/filtering; there is also no obvious reasons to single out jcmd from other binaries bundled w/ JDK >>>>>> - remove 'regression' as the usage of at-bug is more common and convey the same meaning; and for two tests which have regression k/w, but don't have at-bug: >>>>>> - mark vmTestbase/nsk/jvmti/RedefineClasses/redefclass031/TestDescription.java w/ at-bug 6318850; >>>>>> - don't mark runtime/NMT/ReleaseNoCommit.java as it wasn't integrated as a true regression test, but rather as a function test for NMT w/ bunch of other NMT tests (where aren't marked w/ 'regression' k/w) by JDK-8054952. >>>>>> >>>>>> I'd like to receive some feedback on the proposed changes before I (more likely sed) remove keywords from 500+ tests and spend time testing that. >>>>>> >>>>>> [1] is the preliminary webrev just to illustrate the change, it only updates TEST.ROOT file and adds @bug to redefclass031 test. >>>>>> >>>>>> Thanks, >>>>>> -- Igor >>>>>> >>>>>> for reference, k/w available in other test suites: >>>>>> jdk : 2d dnd headful i18n intermittent printer randomness jfr >>>>>> langtools: intermittent randomness >>>>>> jaxp: intermittent randomness >>>>>> >>>>>> [1] http://cr.openjdk.java.net/~iignatyev//8244614/webrev.00 >>>>>> >>>>>> > From philip.race at oracle.com Mon Jun 8 20:07:17 2020 From: philip.race at oracle.com (Philip Race) Date: Mon, 08 Jun 2020 13:07:17 -0700 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: Message-ID: <5EDE9A75.6020504@oracle.com> Hi Kim, Can we amend the JEP itself to be not solely about hotspot. Since it affects other areas I think that 1) They may need to be compiled with C++14 enabled whether they use new features or not 2) They may want - or need - to adopt C++11 or C++14 features too. You already know that soon we will upgrade the harfbuzz 3rd party library used by 2D and it will bring along the need to use C++11 features and I had no plan to propose a JEP for that. So I don't want to be asked why we didn't do one when hotspot did ! So this really ought to be a JDK JEP whilst of course explaining that hotspot is expected to be the primary adopter. BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says > For Solaris: Add the |-std=c++14| compiler option. ..... So I am sure there is still some room to update the JEP :-) -phil. On 6/5/20, 12:52 AM, Kim Barrett wrote: > [Changes are only to the build system, but since the changes have jdk-wide > effect I?ve cc?d what I think are the relevant dev lists.] > > This change is part of JEP 347; the intent is to target JDK 16. > > Please review this change to the building of C++ code in the JDK to > enable the use of C++14 language features. This change does not make > any code changes to use new features provided by C++11 or C++14. > > This requires trimming the supported compiler versions, moving the > minimum supported versions forward (significantly, in some cases). > The new minimums are based on compiler documentation rather than > testing. It may be that more recent versions are actually required. > > This change needs to be tested on platforms not supported by Oracle. > The JEP test plan includes additional Oracle testing beyond what I?ve done. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8246032 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ > > Testing: > mach5 tier1-5 on Oracle supported platforms. > > Performance testing showed no significant changes in either direction. > > Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le > From patric.hedlin at oracle.com Mon Jun 8 22:23:54 2020 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Tue, 9 Jun 2020 00:23:54 +0200 Subject: RFR(XS): 8246689: Enable independent compressed oops/class ptrs on Aarch64 Message-ID: Dear all, I would like to ask for help to review the following change/update: Issue:? https://bugs.openjdk.java.net/browse/JDK-8246689 Webrev: http://cr.openjdk.java.net/~phedlin/tr8246689/ Aarch64 was not included in JDK-8241825. This small patch enables the feature on Aarch64 and updates the associated test-case (in tier2). This patch is an attempt to make tier2 all green for linux-aarch64, that we want to run in regular testing (see JDK-8246691). Testing: tier1-2 and a number of runs of RunThese30M. Best regards, Patric From mikhailo.seledtsov at oracle.com Mon Jun 8 23:22:19 2020 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 8 Jun 2020 16:22:19 -0700 Subject: RFR(S/L) : 8244614 : cleanup keywords used/available in hotspot testbase In-Reply-To: References: <939A28A1-5CEA-438D-A2EE-E547E9C7D57E@oracle.com> <8F94C0EC-3470-4252-A18B-72F68FDA3BC5@oracle.com> <238C8726-BEFB-480B-8D02-D986EB068033@oracle.com> Message-ID: <507f08af-d226-4b92-3fdf-6055cf1622ab@oracle.com> +1 Misha On 6/8/20 12:07 PM, coleen.phillimore at oracle.com wrote: > > Hi Igor, > > Looks good to me.? Many of those keywords were very historical and > nobody cares about them.? Honestly I don't actually know how/if tests > are/can be run by specifying keywords. > > thanks, > Coleen > > On 6/8/20 12:07 PM, Igor Ignatyev wrote: >> ping? >> -- Igor >> >>> On Jun 4, 2020, at 1:23 PM, Igor Ignatyev >>> wrote: >>> >>> ping? >>> >>> -- Igor >>> >>>> On May 29, 2020, at 10:49 AM, Igor Ignatyev >>>> wrote: >>>> >>>> ouch,.. it appears that this tier1 execution wasn't in vain. I >>>> realized that all my queries were limited to the tests `jtreg -l` >>>> gave me, which excludes tests which can't be w/ my default build on >>>> my localhost... >>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01-02 updates >>>> 93 more tests. >>>> >>>> -- Igor >>>> >>>>> On May 29, 2020, at 10:14 AM, Igor Ignatyev >>>>> wrote: >>>>> >>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>>>> >>>>> Hi all, >>>>> >>>>> given the absense of comments, I assume that nobody has any strong >>>>> feeling these to-be-removed keywords , so I'm repurposing this >>>>> thread as RFR. >>>>> >>>>> the biggest part of the patch was done by sed: >>>>> - removal of $key : gsed -ri "s/(@key.*) $key/\1/g" {} ; where >>>>> $key is in nmt cte_test, jcmd, regression >>>>> - removal of empty @key : gsed -ri '/^ \* @key\s*$/d' >>>>> >>>>> the manually done changes are limited to: >>>>> - removal of keywords from TEST.ROOT >>>>> - adding k/w description to TEST.ROOT >>>>> - adding @bug to redefclass031 >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244614 >>>>> webrevs: >>>>> - part done by sed: >>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.sed >>>>>> 594 lines changed: 0 ins; 205 del; 389 mod; >>>>> - part done by me: >>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>>>> - whole (incl. copyright year update) : >>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>>>> >>>>> testing: >>>>> - grep-ed for removed keywords >>>>> - jtreg -l >>>>> - tier1 (just for the good luck) >>>>> >>>>> Thanks, >>>>> -- Igor >>>>> >>>>> >>>>> >>>>>> On May 22, 2020, at 3:42 PM, Igor Ignatyev >>>>>> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> I'm planning to clean up jtreg keywords that we have in >>>>>> test/hotspot/jtreg as most of them seem to be useless, confuse >>>>>> and make people to add these keywords in vain. After looking at >>>>>> how hotspot k/w are used, as well as which k/w are available and >>>>>> used by other test suites, I'd like to propose the following: >>>>>> - keep and document 'headful', 'stress' as they are useful for >>>>>> test selection; >>>>>> - keep and document 'intermittent', 'randomness' as they bring >>>>>> extra information about tests stability/reproducibility/determinism; >>>>>> - remove 'nmt', 'gc', 'metaspace' as they are used only in >>>>>> obvious places and aren't informative; >>>>>> - remove 'cte_test' as it's an artifact of old days and doesn't >>>>>> bring any useful information anymore; >>>>>> - remove 'jcmd', not all tests which depend on jcmd has this k/w, >>>>>> so it can't be used for test selection/filtering; there is also >>>>>> no obvious reasons to single out jcmd from other binaries bundled >>>>>> w/ JDK >>>>>> - remove 'regression' as the usage of at-bug is more common and >>>>>> convey the same meaning; and for two tests which have regression >>>>>> k/w, but don't have at-bug: >>>>>> - mark >>>>>> vmTestbase/nsk/jvmti/RedefineClasses/redefclass031/TestDescription.java >>>>>> w/ at-bug 6318850; >>>>>> - don't mark runtime/NMT/ReleaseNoCommit.java as it wasn't >>>>>> integrated as a true regression test, but rather as a function >>>>>> test for NMT w/ bunch of other NMT tests (where aren't marked w/ >>>>>> 'regression' k/w) by JDK-8054952. >>>>>> >>>>>> I'd like to receive some feedback on the proposed changes before >>>>>> I (more likely sed) remove keywords from 500+ tests and spend >>>>>> time testing that. >>>>>> >>>>>> [1] is the preliminary webrev just to illustrate the change, it >>>>>> only updates TEST.ROOT file and adds @bug to redefclass031 test. >>>>>> >>>>>> Thanks, >>>>>> -- Igor >>>>>> >>>>>> for reference, k/w available in other test suites: >>>>>> jdk : 2d dnd headful i18n intermittent printer randomness jfr >>>>>> langtools: intermittent randomness >>>>>> jaxp: intermittent randomness >>>>>> >>>>>> [1] http://cr.openjdk.java.net/~iignatyev//8244614/webrev.00 >>>>>> >>>>>> > From thomas.stuefe at gmail.com Tue Jun 9 04:30:04 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 9 Jun 2020 06:30:04 +0200 Subject: [Mach5] mach5-one-stuefe-JDK-8243535-2-20200608-1919-11619857: FAILED In-Reply-To: <1820269980.2145.1591647002117@ol7-services-908985.s2.javaplatfo1iad.oraclevcn.com> References: <1820269980.2145.1591647002117@ol7-services-908985.s2.javaplatfo1iad.oraclevcn.com> Message-ID: Hi, Could someone from Oracle with access to the submit jobs please give me more information about the failed aarch64 builds in this test? Thank you! Thomas On Mon, Jun 8, 2020 at 10:10 PM wrote: > Job: mach5-one-stuefe-JDK-8243535-2-20200608-1919-11619857 BuildId: > 2020-06-08-1918471.thomas.stuefe.sourceNo failed testsTasks Summary > > - FAILED: 0 > - EXECUTED_WITH_FAILURE: 4 > - PASSED: 76 > - HARNESS_ERROR: 0 > - UNABLE_TO_RUN: 18 > - KILLED: 0 > - NA: 0 > - NOTHING_TO_RUN: 0Build 1 Unable to run > - linux-aarch64-install-linux-aarch64-build-signing-19 Dependency > task failed: mach5...-11619857-linux-aarch64-linux-aarch64-build-2 > 4 Executed with failure > - linux-aarch64-linux-aarch64-build-2 error while building, return > value: 2 > - linux-aarch64-debug-linux-aarch64-build-3 error while building, > return value: 2 > - linux-aarch64-open-linux-aarch64-build-4 error while building, > return value: 2 > - linux-aarch64-open-debug-linux-aarch64-build-5 error while > building, return value: 2 > Test 17 Unable to run > - tier1-product-open_test_hotspot_jtreg_tier1_common-linux-aarch64-23 > Dependency task failed: > mach5...-11619857-linux-aarch64-linux-aarch64-build-2 > - tier1-debug-open_test_hotspot_jtreg_tier1_common-linux-aarch64-debug-31 > Dependency task failed: > mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > - tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-linux-aarch64-debug-35 > Dependency task failed: > mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > - tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-linux-aarch64-debug-39 > Dependency task failed: > mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > - tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-linux-aarch64-debug-43 > Dependency task failed: > mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > - tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-aarch64-debug-47 > Dependency task failed: > mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > - tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-linux-aarch64-debug-51 > Dependency task failed: > mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > - tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-linux-aarch64-debug-55 > Dependency task failed: > mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > - tier1-product-open_test_hotspot_jtreg_tier1_gc_gcbasher-linux-aarch64-27 > Dependency task failed: > mach5...-11619857-linux-aarch64-linux-aarch64-build-2 > - tier1-debug-open_test_hotspot_jtreg_tier1_gc_gcbasher-linux-aarch64-debug-59 > Dependency task failed: > mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > - See all 17... > > From igor.ignatyev at oracle.com Tue Jun 9 04:35:28 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 8 Jun 2020 21:35:28 -0700 Subject: [Mach5] mach5-one-stuefe-JDK-8243535-2-20200608-1919-11619857: FAILED In-Reply-To: References: <1820269980.2145.1591647002117@ol7-services-908985.s2.javaplatfo1iad.oraclevcn.com> Message-ID: <7A686E30-ADA9-4C39-BAB1-6D0E4F7E0767@oracle.com> Hi Thomas, all 4 build logs have the following error: > workspace/open/src/hotspot/os/posix/os_posix.cpp: In static member function 'static void os::split_reserved_memory(char*, size_t, size_t)': > workspace/open/src/hotspot/os/posix/os_posix.cpp:375:15: error: 'record_virtual_memory_split_reserved' is not a member of 'MemTracker' > 375 | MemTracker::record_virtual_memory_split_reserved(base, size, split); > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > lib/CompileJvm.gmk:149: recipe for target 'workspace/build/linux-aarch64/buildjdk/hotspot/variant-server/libjvm/objs/os_posix.o' failed HTH, -- Igor > On Jun 8, 2020, at 9:30 PM, Thomas St?fe wrote: > > Hi, > > Could someone from Oracle with access to the submit jobs please give me > more information about the failed aarch64 builds in this test? > > Thank you! > > Thomas > > > On Mon, Jun 8, 2020 at 10:10 PM wrote: > >> Job: mach5-one-stuefe-JDK-8243535-2-20200608-1919-11619857 BuildId: >> 2020-06-08-1918471.thomas.stuefe.sourceNo failed testsTasks Summary >> >> - FAILED: 0 >> - EXECUTED_WITH_FAILURE: 4 >> - PASSED: 76 >> - HARNESS_ERROR: 0 >> - UNABLE_TO_RUN: 18 >> - KILLED: 0 >> - NA: 0 >> - NOTHING_TO_RUN: 0Build 1 Unable to run >> - linux-aarch64-install-linux-aarch64-build-signing-19 Dependency >> task failed: mach5...-11619857-linux-aarch64-linux-aarch64-build-2 >> 4 Executed with failure >> - linux-aarch64-linux-aarch64-build-2 error while building, return >> value: 2 >> - linux-aarch64-debug-linux-aarch64-build-3 error while building, >> return value: 2 >> - linux-aarch64-open-linux-aarch64-build-4 error while building, >> return value: 2 >> - linux-aarch64-open-debug-linux-aarch64-build-5 error while >> building, return value: 2 >> Test 17 Unable to run >> - tier1-product-open_test_hotspot_jtreg_tier1_common-linux-aarch64-23 >> Dependency task failed: >> mach5...-11619857-linux-aarch64-linux-aarch64-build-2 >> - tier1-debug-open_test_hotspot_jtreg_tier1_common-linux-aarch64-debug-31 >> Dependency task failed: >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 >> - tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-linux-aarch64-debug-35 >> Dependency task failed: >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 >> - tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-linux-aarch64-debug-39 >> Dependency task failed: >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 >> - tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-linux-aarch64-debug-43 >> Dependency task failed: >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 >> - tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-aarch64-debug-47 >> Dependency task failed: >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 >> - tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-linux-aarch64-debug-51 >> Dependency task failed: >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 >> - tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-linux-aarch64-debug-55 >> Dependency task failed: >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 >> - tier1-product-open_test_hotspot_jtreg_tier1_gc_gcbasher-linux-aarch64-27 >> Dependency task failed: >> mach5...-11619857-linux-aarch64-linux-aarch64-build-2 >> - tier1-debug-open_test_hotspot_jtreg_tier1_gc_gcbasher-linux-aarch64-debug-59 >> Dependency task failed: >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 >> - See all 17... >> >> From igor.ignatyev at oracle.com Tue Jun 9 04:37:12 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 8 Jun 2020 21:37:12 -0700 Subject: RFR(S/L) : 8244614 : cleanup keywords used/available in hotspot testbase In-Reply-To: <507f08af-d226-4b92-3fdf-6055cf1622ab@oracle.com> References: <939A28A1-5CEA-438D-A2EE-E547E9C7D57E@oracle.com> <8F94C0EC-3470-4252-A18B-72F68FDA3BC5@oracle.com> <238C8726-BEFB-480B-8D02-D986EB068033@oracle.com> <507f08af-d226-4b92-3fdf-6055cf1622ab@oracle.com> Message-ID: <71214CEC-864D-48E7-848C-F91A66C5A31B@oracle.com> thank you Misha, pushed. -- Igor > On Jun 8, 2020, at 4:22 PM, mikhailo.seledtsov at oracle.com wrote: > > +1 > > Misha > > On 6/8/20 12:07 PM, coleen.phillimore at oracle.com wrote: >> >> Hi Igor, >> >> Looks good to me. Many of those keywords were very historical and nobody cares about them. Honestly I don't actually know how/if tests are/can be run by specifying keywords. >> >> thanks, >> Coleen >> >> On 6/8/20 12:07 PM, Igor Ignatyev wrote: >>> ping? >>> -- Igor >>> >>>> On Jun 4, 2020, at 1:23 PM, Igor Ignatyev wrote: >>>> >>>> ping? >>>> >>>> -- Igor >>>> >>>>> On May 29, 2020, at 10:49 AM, Igor Ignatyev wrote: >>>>> >>>>> ouch,.. it appears that this tier1 execution wasn't in vain. I realized that all my queries were limited to the tests `jtreg -l` gave me, which excludes tests which can't be w/ my default build on my localhost... >>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01-02 updates 93 more tests. >>>>> >>>>> -- Igor >>>>> >>>>>> On May 29, 2020, at 10:14 AM, Igor Ignatyev wrote: >>>>>> >>>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>>>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>>>>> http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>>>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>>>>> >>>>>> Hi all, >>>>>> >>>>>> given the absense of comments, I assume that nobody has any strong feeling these to-be-removed keywords , so I'm repurposing this thread as RFR. >>>>>> >>>>>> the biggest part of the patch was done by sed: >>>>>> - removal of $key : gsed -ri "s/(@key.*) $key/\1/g" {} ; where $key is in nmt cte_test, jcmd, regression >>>>>> - removal of empty @key : gsed -ri '/^ \* @key\s*$/d' >>>>>> >>>>>> the manually done changes are limited to: >>>>>> - removal of keywords from TEST.ROOT >>>>>> - adding k/w description to TEST.ROOT >>>>>> - adding @bug to redefclass031 >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244614 >>>>>> webrevs: >>>>>> - part done by sed: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.sed >>>>>>> 594 lines changed: 0 ins; 205 del; 389 mod; >>>>>> - part done by me: http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01.manual >>>>>>> 5 lines changed: 4 ins; 0 del; 1 mod; >>>>>> - whole (incl. copyright year update) : http://cr.openjdk.java.net/~iignatyev//8244614/webrev.01 >>>>>>> 930 lines changed: 3 ins; 204 del; 723 mod; >>>>>> >>>>>> testing: >>>>>> - grep-ed for removed keywords >>>>>> - jtreg -l >>>>>> - tier1 (just for the good luck) >>>>>> >>>>>> Thanks, >>>>>> -- Igor >>>>>> >>>>>> >>>>>> >>>>>>> On May 22, 2020, at 3:42 PM, Igor Ignatyev wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I'm planning to clean up jtreg keywords that we have in test/hotspot/jtreg as most of them seem to be useless, confuse and make people to add these keywords in vain. After looking at how hotspot k/w are used, as well as which k/w are available and used by other test suites, I'd like to propose the following: >>>>>>> - keep and document 'headful', 'stress' as they are useful for test selection; >>>>>>> - keep and document 'intermittent', 'randomness' as they bring extra information about tests stability/reproducibility/determinism; >>>>>>> - remove 'nmt', 'gc', 'metaspace' as they are used only in obvious places and aren't informative; >>>>>>> - remove 'cte_test' as it's an artifact of old days and doesn't bring any useful information anymore; >>>>>>> - remove 'jcmd', not all tests which depend on jcmd has this k/w, so it can't be used for test selection/filtering; there is also no obvious reasons to single out jcmd from other binaries bundled w/ JDK >>>>>>> - remove 'regression' as the usage of at-bug is more common and convey the same meaning; and for two tests which have regression k/w, but don't have at-bug: >>>>>>> - mark vmTestbase/nsk/jvmti/RedefineClasses/redefclass031/TestDescription.java w/ at-bug 6318850; >>>>>>> - don't mark runtime/NMT/ReleaseNoCommit.java as it wasn't integrated as a true regression test, but rather as a function test for NMT w/ bunch of other NMT tests (where aren't marked w/ 'regression' k/w) by JDK-8054952. >>>>>>> >>>>>>> I'd like to receive some feedback on the proposed changes before I (more likely sed) remove keywords from 500+ tests and spend time testing that. >>>>>>> >>>>>>> [1] is the preliminary webrev just to illustrate the change, it only updates TEST.ROOT file and adds @bug to redefclass031 test. >>>>>>> >>>>>>> Thanks, >>>>>>> -- Igor >>>>>>> >>>>>>> for reference, k/w available in other test suites: >>>>>>> jdk : 2d dnd headful i18n intermittent printer randomness jfr >>>>>>> langtools: intermittent randomness >>>>>>> jaxp: intermittent randomness >>>>>>> >>>>>>> [1] http://cr.openjdk.java.net/~iignatyev//8244614/webrev.00 >>>>>>> >>>>>>> >> From thomas.stuefe at gmail.com Tue Jun 9 05:02:22 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 9 Jun 2020 07:02:22 +0200 Subject: [Mach5] mach5-one-stuefe-JDK-8243535-2-20200608-1919-11619857: FAILED In-Reply-To: <7A686E30-ADA9-4C39-BAB1-6D0E4F7E0767@oracle.com> References: <1820269980.2145.1591647002117@ol7-services-908985.s2.javaplatfo1iad.oraclevcn.com> <7A686E30-ADA9-4C39-BAB1-6D0E4F7E0767@oracle.com> Message-ID: Thank you Igor! On Tue, Jun 9, 2020 at 6:37 AM Igor Ignatyev wrote: > Hi Thomas, > > all 4 build logs have the following error: > > > workspace/open/src/hotspot/os/posix/os_posix.cpp: In static member > function 'static void os::split_reserved_memory(char*, size_t, size_t)': > > workspace/open/src/hotspot/os/posix/os_posix.cpp:375:15: error: > 'record_virtual_memory_split_reserved' is not a member of 'MemTracker' > > 375 | MemTracker::record_virtual_memory_split_reserved(base, size, > split); > > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > lib/CompileJvm.gmk:149: recipe for target > 'workspace/build/linux-aarch64/buildjdk/hotspot/variant-server/libjvm/objs/os_posix.o' > failed > > HTH, > -- Igor > > > > On Jun 8, 2020, at 9:30 PM, Thomas St?fe > wrote: > > > > Hi, > > > > Could someone from Oracle with access to the submit jobs please give me > > more information about the failed aarch64 builds in this test? > > > > Thank you! > > > > Thomas > > > > > > On Mon, Jun 8, 2020 at 10:10 PM wrote: > > > >> Job: mach5-one-stuefe-JDK-8243535-2-20200608-1919-11619857 BuildId: > >> 2020-06-08-1918471.thomas.stuefe.sourceNo failed testsTasks Summary > >> > >> - FAILED: 0 > >> - EXECUTED_WITH_FAILURE: 4 > >> - PASSED: 76 > >> - HARNESS_ERROR: 0 > >> - UNABLE_TO_RUN: 18 > >> - KILLED: 0 > >> - NA: 0 > >> - NOTHING_TO_RUN: 0Build 1 Unable to run > >> - linux-aarch64-install-linux-aarch64-build-signing-19 Dependency > >> task failed: mach5...-11619857-linux-aarch64-linux-aarch64-build-2 > >> 4 Executed with failure > >> - linux-aarch64-linux-aarch64-build-2 error while building, return > >> value: 2 > >> - linux-aarch64-debug-linux-aarch64-build-3 error while building, > >> return value: 2 > >> - linux-aarch64-open-linux-aarch64-build-4 error while building, > >> return value: 2 > >> - linux-aarch64-open-debug-linux-aarch64-build-5 error while > >> building, return value: 2 > >> Test 17 Unable to run > >> - > tier1-product-open_test_hotspot_jtreg_tier1_common-linux-aarch64-23 > >> Dependency task failed: > >> mach5...-11619857-linux-aarch64-linux-aarch64-build-2 > >> - > tier1-debug-open_test_hotspot_jtreg_tier1_common-linux-aarch64-debug-31 > >> Dependency task failed: > >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > >> - > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-linux-aarch64-debug-35 > >> Dependency task failed: > >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > >> - > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-linux-aarch64-debug-39 > >> Dependency task failed: > >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > >> - > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-linux-aarch64-debug-43 > >> Dependency task failed: > >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > >> - > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-aarch64-debug-47 > >> Dependency task failed: > >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > >> - > tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-linux-aarch64-debug-51 > >> Dependency task failed: > >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > >> - > tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-linux-aarch64-debug-55 > >> Dependency task failed: > >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > >> - > tier1-product-open_test_hotspot_jtreg_tier1_gc_gcbasher-linux-aarch64-27 > >> Dependency task failed: > >> mach5...-11619857-linux-aarch64-linux-aarch64-build-2 > >> - > tier1-debug-open_test_hotspot_jtreg_tier1_gc_gcbasher-linux-aarch64-debug-59 > >> Dependency task failed: > >> mach5...857-linux-aarch64-debug-linux-aarch64-build-3 > >> - See all 17... > >> > >> > > From david.holmes at oracle.com Tue Jun 9 05:21:12 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 9 Jun 2020 15:21:12 +1000 Subject: RFR [XS]: 8244196: adjust output in os_linux In-Reply-To: <2eaf4b16-a4b1-cb68-f21e-e7cf129e1a73@oracle.com> References: <2eaf4b16-a4b1-cb68-f21e-e7cf129e1a73@oracle.com> Message-ID: <153e947b-60a5-a474-e274-a7600ee58d2c@oracle.com> Hi Stefan, On 9/06/2020 4:21 am, Stefan Karlsson wrote: > Hi all, > > I encountered the output of this today, and I think the current output > looks pretty bad. From my hs_err file I have this: > > CPU:total 32 (initial active 32) (8 cores per cpu, 2 threads per core) > ... much more ... > Online cpus: > 0-31 > Offline cpus: > > cache level: > 1 > cache type: > Data > cache size: > 32K > cache coherency line size: > 64 > cache level: > 1 > cache type: > Instruction > cache size: > 32K > cache coherency line size: > 64 > cache level: > 2 > cache type: > Unified > cache size: > 256K > cache coherency line size: > 64 > cache level: > 3 > cache type: > Unified > cache size: > 20480K > cache coherency line size: > 64 > BIOS frequency limitation: > > Frequency switch latency (ns): > 0 > Available cpu frequencies: > > Maximum cpu frequency: > 3200000 > Minimum cpu frequency: > 1200000 > Current cpu frequency: > 2799113 > Available governors: > performance powersave > Current governor: > powersave > Core performance/turbo boost: > > > It's not easy at all to read. Would you mind if I changed this to: > Online cpus: 0-31 > Offline cpus: > cache level: 1 > cache type: Data > cache size: 32K > cache coherency line size: 64 > cache level: 1 > cache type: Instruction > cache size: 32K > cache coherency line size: 64 > cache level: 2 > cache type: Unified > cache size: 256K > cache coherency line size: 64 > cache level: 3 > cache type: Unified > cache size: 20480K > cache coherency line size: 64 > BIOS frequency limitation: > Frequency switch latency (ns): 0 > Available cpu frequencies: > Maximum cpu frequency: 3200000 > Minimum cpu frequency: 1200000 > Current cpu frequency: 1870913 > Available governors: performance powersave > Current governor: powersave > Core performance/turbo boost: > > I don't think we should be too afraid of changing the lines here. We've > have been changing other parts of the hs_err files, and if anyone is > scraping this info, I think they'll need to adapt. This is mostly my fault. I was focusing on the changes to os::Linux::print_proc_sys_info which originally removed a lot of newlines, which I thought may cause issues for anyone scraping the info. So Matthias fixed that. But the fix moved the newline printing inside _print_ascii_file_h which then caused all the extra newlines that you are complaining about. :( Sorry about that. Given I was concerned about simply removing the newline in all cases before, I have to remain concerned about removing them all now. I think we need to restore this: static void _print_ascii_file_h(const char* header, const char* filename, outputStream* st) { ! st->print("%s", header); (rather than st->print_cr) and then add the \n in the callers that want the header on a line by itself. David ----- > Thanks, > StefanK > > On 2020-04-30 17:46, Baesken, Matthias wrote: >> Hello, please review this small change to os_linux.cpp? . >> >> It? uses the? recently introduced??? _print_ascii_file_h >> Function t more places (makes the code? a bit? shorter ) . >> >> >> Bug/webrev : >> >> https://bugs.openjdk.java.net/browse/JDK-8244196 >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8244196.0/ >> >> Thanks, Matthias > From erik.osterlund at oracle.com Tue Jun 9 06:04:45 2020 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Tue, 9 Jun 2020 08:04:45 +0200 Subject: RFR(XS): 8246689: Enable independent compressed oops/class ptrs on Aarch64 In-Reply-To: References: Message-ID: <7D757361-AFF8-40E7-A8F2-ECA711DCB110@oracle.com> Hi Patric, This looks good. Thank you for trying this out on AArch64. Only Graal to go now, then we can remove the switch and have support for this feature be expected on all platforms. /Erik > On 9 Jun 2020, at 00:24, Patric Hedlin wrote: > > ?Dear all, > > I would like to ask for help to review the following change/update: > > Issue: https://bugs.openjdk.java.net/browse/JDK-8246689 > Webrev: http://cr.openjdk.java.net/~phedlin/tr8246689/ > > > Aarch64 was not included in JDK-8241825. This small patch enables the feature on Aarch64 and updates the associated test-case (in tier2). This patch is an attempt to make tier2 all green for linux-aarch64, that we want to run in regular testing (see JDK-8246691). > > Testing: tier1-2 and a number of runs of RunThese30M. > > > Best regards, > Patric From matthias.baesken at sap.com Tue Jun 9 06:48:43 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 9 Jun 2020 06:48:43 +0000 Subject: RFR [XS]: 8244196: adjust output in os_linux In-Reply-To: <2eaf4b16-a4b1-cb68-f21e-e7cf129e1a73@oracle.com> References: <2eaf4b16-a4b1-cb68-f21e-e7cf129e1a73@oracle.com> Message-ID: > It's not easy at all to read. Would you mind if I changed this to: Hi Stefan, I think your idea to use a more compact output is fine . Best regards, Matthias -----Original Message----- From: Stefan Karlsson Sent: Montag, 8. Juni 2020 20:21 To: Baesken, Matthias ; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR [XS]: 8244196: adjust output in os_linux Hi all, I encountered the output of this today, and I think the current output looks pretty bad. From my hs_err file I have this: CPU:total 32 (initial active 32) (8 cores per cpu, 2 threads per core) ... much more ... Online cpus: 0-31 Offline cpus: cache level: 1 cache type: Data cache size: 32K cache coherency line size: 64 cache level: 1 cache type: Instruction cache size: 32K cache coherency line size: 64 cache level: 2 cache type: Unified cache size: 256K cache coherency line size: 64 cache level: 3 cache type: Unified cache size: 20480K cache coherency line size: 64 BIOS frequency limitation: Frequency switch latency (ns): 0 Available cpu frequencies: Maximum cpu frequency: 3200000 Minimum cpu frequency: 1200000 Current cpu frequency: 2799113 Available governors: performance powersave Current governor: powersave Core performance/turbo boost: It's not easy at all to read. Would you mind if I changed this to: Online cpus: 0-31 Offline cpus: cache level: 1 cache type: Data cache size: 32K cache coherency line size: 64 cache level: 1 cache type: Instruction cache size: 32K cache coherency line size: 64 cache level: 2 cache type: Unified cache size: 256K cache coherency line size: 64 cache level: 3 cache type: Unified cache size: 20480K cache coherency line size: 64 BIOS frequency limitation: Frequency switch latency (ns): 0 Available cpu frequencies: Maximum cpu frequency: 3200000 Minimum cpu frequency: 1200000 Current cpu frequency: 1870913 Available governors: performance powersave Current governor: powersave Core performance/turbo boost: I don't think we should be too afraid of changing the lines here. We've have been changing other parts of the hs_err files, and if anyone is scraping this info, I think they'll need to adapt. Thanks, StefanK On 2020-04-30 17:46, Baesken, Matthias wrote: > Hello, please review this small change to os_linux.cpp . > > It uses the recently introduced _print_ascii_file_h > Function t more places (makes the code a bit shorter ) . > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8244196 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8244196.0/ > > Thanks, Matthias From stefan.karlsson at oracle.com Tue Jun 9 07:57:18 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 09:57:18 +0200 Subject: RFR [XS]: 8244196: adjust output in os_linux In-Reply-To: <153e947b-60a5-a474-e274-a7600ee58d2c@oracle.com> References: <2eaf4b16-a4b1-cb68-f21e-e7cf129e1a73@oracle.com> <153e947b-60a5-a474-e274-a7600ee58d2c@oracle.com> Message-ID: On 2020-06-09 07:21, David Holmes wrote: > Hi Stefan, > > On 9/06/2020 4:21 am, Stefan Karlsson wrote: >> Hi all, >> >> I encountered the output of this today, and I think the current >> output looks pretty bad. From my hs_err file I have this: >> >> CPU:total 32 (initial active 32) (8 cores per cpu, 2 threads per >> core) ... much more ... >> Online cpus: >> 0-31 >> Offline cpus: >> >> cache level: >> 1 >> cache type: >> Data >> cache size: >> 32K >> cache coherency line size: >> 64 >> cache level: >> 1 >> cache type: >> Instruction >> cache size: >> 32K >> cache coherency line size: >> 64 >> cache level: >> 2 >> cache type: >> Unified >> cache size: >> 256K >> cache coherency line size: >> 64 >> cache level: >> 3 >> cache type: >> Unified >> cache size: >> 20480K >> cache coherency line size: >> 64 >> BIOS frequency limitation: >> >> Frequency switch latency (ns): >> 0 >> Available cpu frequencies: >> >> Maximum cpu frequency: >> 3200000 >> Minimum cpu frequency: >> 1200000 >> Current cpu frequency: >> 2799113 >> Available governors: >> performance powersave >> Current governor: >> powersave >> Core performance/turbo boost: >> >> >> It's not easy at all to read. Would you mind if I changed this to: >> Online cpus: 0-31 >> Offline cpus: >> cache level: 1 >> cache type: Data >> cache size: 32K >> cache coherency line size: 64 >> cache level: 1 >> cache type: Instruction >> cache size: 32K >> cache coherency line size: 64 >> cache level: 2 >> cache type: Unified >> cache size: 256K >> cache coherency line size: 64 >> cache level: 3 >> cache type: Unified >> cache size: 20480K >> cache coherency line size: 64 >> BIOS frequency limitation: >> Frequency switch latency (ns): 0 >> Available cpu frequencies: >> Maximum cpu frequency: 3200000 >> Minimum cpu frequency: 1200000 >> Current cpu frequency: 1870913 >> Available governors: performance powersave >> Current governor: powersave >> Core performance/turbo boost: >> >> I don't think we should be too afraid of changing the lines here. >> We've have been changing other parts of the hs_err files, and if >> anyone is scraping this info, I think they'll need to adapt. > > This is mostly my fault. I was focusing on the changes to > os::Linux::print_proc_sys_info which originally removed a lot of > newlines, which I thought may cause issues for anyone scraping the > info. So Matthias fixed that. But the fix moved the newline printing > inside _print_ascii_file_h which then caused all the extra newlines > that you are complaining about. :( Sorry about that. > > Given I was concerned about simply removing the newline in all cases > before, I have to remain concerned about removing them all now. I > think we need to restore this: > > ? static void _print_ascii_file_h(const char* header, const char* > filename, outputStream* st) { > !?? st->print("%s", header); > > (rather than st->print_cr) and then add the \n in the callers that > want the header on a line by itself. OK. Thanks! I'll prepare a patch to clean this up for JDK 15. StefanK > > David > ----- > >> Thanks, >> StefanK >> >> On 2020-04-30 17:46, Baesken, Matthias wrote: >>> Hello, please review this small change to os_linux.cpp? . >>> >>> It? uses the? recently introduced??? _print_ascii_file_h >>> Function t more places (makes the code? a bit? shorter ) . >>> >>> >>> Bug/webrev : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8244196 >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8244196.0/ >>> >>> Thanks, Matthias >> From stefan.karlsson at oracle.com Tue Jun 9 07:57:35 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 09:57:35 +0200 Subject: RFR [XS]: 8244196: adjust output in os_linux In-Reply-To: References: <2eaf4b16-a4b1-cb68-f21e-e7cf129e1a73@oracle.com> Message-ID: <6bb34323-54d0-40b4-7134-3a0d9b3a438d@oracle.com> On 2020-06-09 08:48, Baesken, Matthias wrote: >> It's not easy at all to read. Would you mind if I changed this to: > Hi Stefan, I think your idea to use a more compact output is fine . Thanks! StefanK > > Best regards, Matthias > > > -----Original Message----- > From: Stefan Karlsson > Sent: Montag, 8. Juni 2020 20:21 > To: Baesken, Matthias ; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR [XS]: 8244196: adjust output in os_linux > > Hi all, > > I encountered the output of this today, and I think the current output > looks pretty bad. From my hs_err file I have this: > > CPU:total 32 (initial active 32) (8 cores per cpu, 2 threads per core) > ... much more ... > Online cpus: > 0-31 > Offline cpus: > > cache level: > 1 > cache type: > Data > cache size: > 32K > cache coherency line size: > 64 > cache level: > 1 > cache type: > Instruction > cache size: > 32K > cache coherency line size: > 64 > cache level: > 2 > cache type: > Unified > cache size: > 256K > cache coherency line size: > 64 > cache level: > 3 > cache type: > Unified > cache size: > 20480K > cache coherency line size: > 64 > BIOS frequency limitation: > > Frequency switch latency (ns): > 0 > Available cpu frequencies: > > Maximum cpu frequency: > 3200000 > Minimum cpu frequency: > 1200000 > Current cpu frequency: > 2799113 > Available governors: > performance powersave > Current governor: > powersave > Core performance/turbo boost: > > > It's not easy at all to read. Would you mind if I changed this to: > Online cpus: 0-31 > Offline cpus: > cache level: 1 > cache type: Data > cache size: 32K > cache coherency line size: 64 > cache level: 1 > cache type: Instruction > cache size: 32K > cache coherency line size: 64 > cache level: 2 > cache type: Unified > cache size: 256K > cache coherency line size: 64 > cache level: 3 > cache type: Unified > cache size: 20480K > cache coherency line size: 64 > BIOS frequency limitation: > Frequency switch latency (ns): 0 > Available cpu frequencies: > Maximum cpu frequency: 3200000 > Minimum cpu frequency: 1200000 > Current cpu frequency: 1870913 > Available governors: performance powersave > Current governor: powersave > Core performance/turbo boost: > > I don't think we should be too afraid of changing the lines here. We've > have been changing other parts of the hs_err files, and if anyone is > scraping this info, I think they'll need to adapt. > > Thanks, > StefanK > > On 2020-04-30 17:46, Baesken, Matthias wrote: >> Hello, please review this small change to os_linux.cpp . >> >> It uses the recently introduced _print_ascii_file_h >> Function t more places (makes the code a bit shorter ) . >> >> >> Bug/webrev : >> >> https://bugs.openjdk.java.net/browse/JDK-8244196 >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8244196.0/ >> >> Thanks, Matthias From stefan.karlsson at oracle.com Tue Jun 9 10:17:29 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 12:17:29 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files Message-ID: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> Hi all, Please review this patch to clean up newlines and whitespaces in our hs_err files. ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ ?https://bugs.openjdk.java.net/browse/JDK-8246926 I've uploaded before and after hs_err files here: ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log Main cleanups: - Numerous one-liners where written as two lines: Old === Online cpus: 0-31 Offline cpus: cache level: 1 cache type: Data cache size: 32K cache coherency line size: 64 <... many more ...> New === Online cpus: 0-31 Offline cpus: cache level: 1 cache type: Data cache size: 32K cache coherency line size: 64 - Many lines missed a space after colon: Old === VM state:not at safepoint (not fully initialized) New === VM state: not at safepoint (not fully initialized) - Newline after OS The code looked for various files (on Linux) and dumped their contents. The first line ended up on the OS: line, but the rest ended afterwards Example from Linux: OS:DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.04 DISTRIB_CODENAME=focal DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" uname:Linux Example from macOS: OS:uname:Darwin I couldn't find a quick and clean fix for this, so I simply opted to add a newline after "OS:". - There are a number of cr() calls (or ?\n" prints) that are used to add a complete blank line between sections. I moved a few of those out of the lower-level print functions, so that the caller could decide if there's a need for extra newlines. Thanks, StefanK From stefan.karlsson at oracle.com Tue Jun 9 11:25:34 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 13:25:34 +0200 Subject: RFR: 8247201: Print potential pointer value of readable stack memory in hs_err file Message-ID: Hi all, Please review this patch to print potential pointer value of readable stack memory in hs_err files. https://cr.openjdk.java.net/~stefank/8247201/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8247201 I often see these lines in hs_err file: stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable memory: 7c cb b3 05 01 00 00 00 and have to manually transpose the bytes to see if they match an important pointer value. I'd like to always print the full pointer value, if possible: stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable memory: 0x0000000105b3cb7c | 7c cb b3 05 01 00 00 00 The suggested patch only prints this value if the dereferenced address is aligned, otherwise it skips it. Side-note: the code that determines if the memory is readable only checks 4 bytes. So if we have a pointer at the end last few bytes of a page, and an inaccessible page after that, it seems like the pre-existing code will crash. Thanks, StefanK From stefan.karlsson at oracle.com Tue Jun 9 12:16:07 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 14:16:07 +0200 Subject: RFR: 8247201: Print potential pointer value of readable stack memory in hs_err file In-Reply-To: References: Message-ID: <55a7582b-0021-6009-14a3-98d042debe31@oracle.com> On 2020-06-09 13:25, Stefan Karlsson wrote: > Hi all, > > Please review this patch to print potential pointer value of readable > stack memory in hs_err files. > > https://cr.openjdk.java.net/~stefank/8247201/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247201 > > I often see these lines in hs_err file: > stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable > memory: 7c cb b3 05 01 00 00 00 > > and have to manually transpose the bytes to see if they match an > important pointer value. > > I'd like to always print the full pointer value, if possible: > stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable > memory: 0x0000000105b3cb7c | 7c cb b3 05 01 00 00 00 > > The suggested patch only prints this value if the dereferenced address > is aligned, otherwise it skips it. > > Side-note: the code that determines if the memory is readable only > checks 4 bytes. So if we have a pointer at the end last few bytes of a > page, and an inaccessible page after that, it seems like the > pre-existing code will crash. Ignore this. The reads are guarded by the align_up adjustment. StefanK > > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue Jun 9 12:49:10 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 14:49:10 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> Message-ID: <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> One small update. I'd like to add a new line between the heap printing and G1's heap region printing: ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02 This now prints: Heap: ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, 0x0000000800000000) ? region size 16384K, 0 young (0K), 0 survivors (0K) ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved 1056768K ? class space??? used 4K, capacity 384K, committed 384K, reserved 1048576K Heap Regions: E=young(eden), S=young(survivor), O=old, HS=humongous(starts), HC=humongous(continues), CS=collection set, F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start (previous, next) |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked Instead of: Heap: ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, 0x0000000800000000) ? region size 16384K, 0 young (0K), 0 survivors (0K) ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved 1056768K ? class space??? used 4K, capacity 384K, committed 384K, reserved 1048576K Heap Regions: E=young(eden), S=young(survivor), O=old, HS=humongous(starts), HC=humongous(continues), CS=collection set, F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start (previous, next) |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked Thanks, StefanK On 2020-06-09 12:17, Stefan Karlsson wrote: > Hi all, > > Please review this patch to clean up newlines and whitespaces in our > hs_err files. > > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ > ?https://bugs.openjdk.java.net/browse/JDK-8246926 > > I've uploaded before and after hs_err files here: > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log > > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log > > > Main cleanups: > - Numerous one-liners where written as two lines: > > Old > === > Online cpus: > 0-31 > Offline cpus: > > cache level: > 1 > cache type: > Data > cache size: > 32K > cache coherency line size: > 64 > <... many more ...> > > New > === > Online cpus: 0-31 > Offline cpus: > cache level: 1 > cache type: Data > cache size: 32K > cache coherency line size: 64 > > > - Many lines missed a space after colon: > > Old > === > VM state:not at safepoint (not fully initialized) > > New > === > VM state: not at safepoint (not fully initialized) > > > - Newline after OS > > The code looked for various files (on Linux) and dumped their > contents. The first line ended up on the OS: line, but the rest ended > afterwards > > Example from Linux: > OS:DISTRIB_ID=Ubuntu > DISTRIB_RELEASE=20.04 > DISTRIB_CODENAME=focal > DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" > uname:Linux > > Example from macOS: > OS:uname:Darwin > > I couldn't find a quick and clean fix for this, so I simply opted to > add a newline after "OS:". > > > - There are a number of cr() calls (or ?\n" prints) that are used to > add a complete blank line between sections. I moved a few of those out > of the lower-level print functions, so that the caller could decide if > there's a need for extra newlines. > > Thanks, > StefanK From erik.osterlund at oracle.com Tue Jun 9 12:58:03 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 9 Jun 2020 14:58:03 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> Message-ID: <07bf212a-5e73-f029-da82-0cd2eeccc79c@oracle.com> Looks good. /Erik On 2020-06-09 14:49, Stefan Karlsson wrote: > One small update. I'd like to add a new line between the heap printing > and G1's heap region printing: > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02 > > This now prints: > > Heap: > ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, > 0x0000000800000000) > ? region size 16384K, 0 young (0K), 0 survivors (0K) > ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved > 1056768K > ? class space??? used 4K, capacity 384K, committed 384K, reserved > 1048576K > > Heap Regions: E=young(eden), S=young(survivor), O=old, > HS=humongous(starts), HC=humongous(continues), CS=collection set, > F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start > (previous, next) > |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| > F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked > |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| > F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked > > Instead of: > > Heap: > ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, > 0x0000000800000000) > ? region size 16384K, 0 young (0K), 0 survivors (0K) > ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved > 1056768K > ? class space??? used 4K, capacity 384K, committed 384K, reserved > 1048576K > Heap Regions: E=young(eden), S=young(survivor), O=old, > HS=humongous(starts), HC=humongous(continues), CS=collection set, > F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start > (previous, next) > |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| > F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked > |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| > F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked > > Thanks, > StefanK > > On 2020-06-09 12:17, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to clean up newlines and whitespaces in our >> hs_err files. >> >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >> >> I've uploaded before and after hs_err files here: >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >> >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >> >> >> Main cleanups: >> - Numerous one-liners where written as two lines: >> >> Old >> === >> Online cpus: >> 0-31 >> Offline cpus: >> >> cache level: >> 1 >> cache type: >> Data >> cache size: >> 32K >> cache coherency line size: >> 64 >> <... many more ...> >> >> New >> === >> Online cpus: 0-31 >> Offline cpus: >> cache level: 1 >> cache type: Data >> cache size: 32K >> cache coherency line size: 64 >> >> >> - Many lines missed a space after colon: >> >> Old >> === >> VM state:not at safepoint (not fully initialized) >> >> New >> === >> VM state: not at safepoint (not fully initialized) >> >> >> - Newline after OS >> >> The code looked for various files (on Linux) and dumped their >> contents. The first line ended up on the OS: line, but the rest ended >> afterwards >> >> Example from Linux: >> OS:DISTRIB_ID=Ubuntu >> DISTRIB_RELEASE=20.04 >> DISTRIB_CODENAME=focal >> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >> uname:Linux >> >> Example from macOS: >> OS:uname:Darwin >> >> I couldn't find a quick and clean fix for this, so I simply opted to >> add a newline after "OS:". >> >> >> - There are a number of cr() calls (or ?\n" prints) that are used to >> add a complete blank line between sections. I moved a few of those >> out of the lower-level print functions, so that the caller could >> decide if there's a need for extra newlines. >> >> Thanks, >> StefanK > From stefan.karlsson at oracle.com Tue Jun 9 12:59:38 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 14:59:38 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <07bf212a-5e73-f029-da82-0cd2eeccc79c@oracle.com> References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> <07bf212a-5e73-f029-da82-0cd2eeccc79c@oracle.com> Message-ID: Thanks, Erik. StefanK On 2020-06-09 14:58, Erik ?sterlund wrote: > Looks good. > > /Erik > > On 2020-06-09 14:49, Stefan Karlsson wrote: >> One small update. I'd like to add a new line between the heap >> printing and G1's heap region printing: >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02 >> >> This now prints: >> >> Heap: >> ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >> 0x0000000800000000) >> ? region size 16384K, 0 young (0K), 0 survivors (0K) >> ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >> 1056768K >> ? class space??? used 4K, capacity 384K, committed 384K, reserved >> 1048576K >> >> Heap Regions: E=young(eden), S=young(survivor), O=old, >> HS=humongous(starts), HC=humongous(continues), CS=collection set, >> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >> (previous, next) >> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| >> F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| >> F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >> >> Instead of: >> >> Heap: >> ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >> 0x0000000800000000) >> ? region size 16384K, 0 young (0K), 0 survivors (0K) >> ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >> 1056768K >> ? class space??? used 4K, capacity 384K, committed 384K, reserved >> 1048576K >> Heap Regions: E=young(eden), S=young(survivor), O=old, >> HS=humongous(starts), HC=humongous(continues), CS=collection set, >> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >> (previous, next) >> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| >> F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| >> F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >> >> Thanks, >> StefanK >> >> On 2020-06-09 12:17, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to clean up newlines and whitespaces in our >>> hs_err files. >>> >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >>> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >>> >>> I've uploaded before and after hs_err files here: >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >>> >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >>> >>> >>> Main cleanups: >>> - Numerous one-liners where written as two lines: >>> >>> Old >>> === >>> Online cpus: >>> 0-31 >>> Offline cpus: >>> >>> cache level: >>> 1 >>> cache type: >>> Data >>> cache size: >>> 32K >>> cache coherency line size: >>> 64 >>> <... many more ...> >>> >>> New >>> === >>> Online cpus: 0-31 >>> Offline cpus: >>> cache level: 1 >>> cache type: Data >>> cache size: 32K >>> cache coherency line size: 64 >>> >>> >>> - Many lines missed a space after colon: >>> >>> Old >>> === >>> VM state:not at safepoint (not fully initialized) >>> >>> New >>> === >>> VM state: not at safepoint (not fully initialized) >>> >>> >>> - Newline after OS >>> >>> The code looked for various files (on Linux) and dumped their >>> contents. The first line ended up on the OS: line, but the rest >>> ended afterwards >>> >>> Example from Linux: >>> OS:DISTRIB_ID=Ubuntu >>> DISTRIB_RELEASE=20.04 >>> DISTRIB_CODENAME=focal >>> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >>> uname:Linux >>> >>> Example from macOS: >>> OS:uname:Darwin >>> >>> I couldn't find a quick and clean fix for this, so I simply opted to >>> add a newline after "OS:". >>> >>> >>> - There are a number of cr() calls (or ?\n" prints) that are used to >>> add a complete blank line between sections. I moved a few of those >>> out of the lower-level print functions, so that the caller could >>> decide if there's a need for extra newlines. >>> >>> Thanks, >>> StefanK >> > From erik.osterlund at oracle.com Tue Jun 9 13:00:28 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 9 Jun 2020 15:00:28 +0200 Subject: RFR: 8247201: Print potential pointer value of readable stack memory in hs_err file In-Reply-To: References: Message-ID: Looks good. /Erik On 2020-06-09 13:25, Stefan Karlsson wrote: > Hi all, > > Please review this patch to print potential pointer value of readable > stack memory in hs_err files. > > https://cr.openjdk.java.net/~stefank/8247201/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247201 > > I often see these lines in hs_err file: > stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable > memory: 7c cb b3 05 01 00 00 00 > > and have to manually transpose the bytes to see if they match an > important pointer value. > > I'd like to always print the full pointer value, if possible: > stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable > memory: 0x0000000105b3cb7c | 7c cb b3 05 01 00 00 00 > > The suggested patch only prints this value if the dereferenced address > is aligned, otherwise it skips it. > > Side-note: the code that determines if the memory is readable only > checks 4 bytes. So if we have a pointer at the end last few bytes of a > page, and an inaccessible page after that, it seems like the > pre-existing code will crash. > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue Jun 9 13:12:37 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 15:12:37 +0200 Subject: RFR: 8247201: Print potential pointer value of readable stack memory in hs_err file In-Reply-To: References: Message-ID: <8c04e74f-8901-92d7-857a-d4164d6377c3@oracle.com> Thanks, Erik. StefanK On 2020-06-09 15:00, Erik ?sterlund wrote: > Looks good. > > /Erik > > On 2020-06-09 13:25, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to print potential pointer value of readable >> stack memory in hs_err files. >> >> https://cr.openjdk.java.net/~stefank/8247201/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247201 >> >> I often see these lines in hs_err file: >> stack at sp + 2 slots: 0x00007000042e6930 points into unknown >> readable memory: 7c cb b3 05 01 00 00 00 >> >> and have to manually transpose the bytes to see if they match an >> important pointer value. >> >> I'd like to always print the full pointer value, if possible: >> stack at sp + 2 slots: 0x00007000042e6930 points into unknown >> readable memory: 0x0000000105b3cb7c | 7c cb b3 05 01 00 00 00 >> >> The suggested patch only prints this value if the dereferenced >> address is aligned, otherwise it skips it. >> >> Side-note: the code that determines if the memory is readable only >> checks 4 bytes. So if we have a pointer at the end last few bytes of >> a page, and an inaccessible page after that, it seems like the >> pre-existing code will crash. >> >> Thanks, >> StefanK > From david.holmes at oracle.com Tue Jun 9 13:25:55 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 9 Jun 2020 23:25:55 +1000 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> Message-ID: <339ec19a-d45d-0045-5717-c30d4ecf0dbe@oracle.com> Hi Stefan, I think the new file looks better but I am worried about the disruption this may cause to existing scraping tools that may be in use. The SAP folk seem to care most about the content and layout in this area so if they are okay with it then I'm okay with it. Thanks, David ----- On 9/06/2020 10:49 pm, Stefan Karlsson wrote: > One small update. I'd like to add a new line between the heap printing > and G1's heap region printing: > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02 > > This now prints: > > Heap: > ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, > 0x0000000800000000) > ? region size 16384K, 0 young (0K), 0 survivors (0K) > ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved > 1056768K > ? class space??? used 4K, capacity 384K, committed 384K, reserved 1048576K > > Heap Regions: E=young(eden), S=young(survivor), O=old, > HS=humongous(starts), HC=humongous(continues), CS=collection set, > F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start > (previous, next) > |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| > F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked > |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| > F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked > > Instead of: > > Heap: > ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, > 0x0000000800000000) > ? region size 16384K, 0 young (0K), 0 survivors (0K) > ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved > 1056768K > ? class space??? used 4K, capacity 384K, committed 384K, reserved 1048576K > Heap Regions: E=young(eden), S=young(survivor), O=old, > HS=humongous(starts), HC=humongous(continues), CS=collection set, > F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start > (previous, next) > |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| > F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked > |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| > F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked > > Thanks, > StefanK > > On 2020-06-09 12:17, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to clean up newlines and whitespaces in our >> hs_err files. >> >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >> >> I've uploaded before and after hs_err files here: >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >> >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >> >> >> Main cleanups: >> - Numerous one-liners where written as two lines: >> >> Old >> === >> Online cpus: >> 0-31 >> Offline cpus: >> >> cache level: >> 1 >> cache type: >> Data >> cache size: >> 32K >> cache coherency line size: >> 64 >> <... many more ...> >> >> New >> === >> Online cpus: 0-31 >> Offline cpus: >> cache level: 1 >> cache type: Data >> cache size: 32K >> cache coherency line size: 64 >> >> >> - Many lines missed a space after colon: >> >> Old >> === >> VM state:not at safepoint (not fully initialized) >> >> New >> === >> VM state: not at safepoint (not fully initialized) >> >> >> - Newline after OS >> >> The code looked for various files (on Linux) and dumped their >> contents. The first line ended up on the OS: line, but the rest ended >> afterwards >> >> Example from Linux: >> OS:DISTRIB_ID=Ubuntu >> DISTRIB_RELEASE=20.04 >> DISTRIB_CODENAME=focal >> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >> uname:Linux >> >> Example from macOS: >> OS:uname:Darwin >> >> I couldn't find a quick and clean fix for this, so I simply opted to >> add a newline after "OS:". >> >> >> - There are a number of cr() calls (or ?\n" prints) that are used to >> add a complete blank line between sections. I moved a few of those out >> of the lower-level print functions, so that the caller could decide if >> there's a need for extra newlines. >> >> Thanks, >> StefanK > From david.holmes at oracle.com Tue Jun 9 13:36:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 9 Jun 2020 23:36:52 +1000 Subject: RFR: 8247201: Print potential pointer value of readable stack memory in hs_err file In-Reply-To: References: Message-ID: <08ff111e-8547-4386-2308-eb773cdb14e8@oracle.com> Hi Stefan, Looks good! Thanks, David On 9/06/2020 9:25 pm, Stefan Karlsson wrote: > Hi all, > > Please review this patch to print potential pointer value of readable > stack memory in hs_err files. > > https://cr.openjdk.java.net/~stefank/8247201/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247201 > > I often see these lines in hs_err file: > stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable > memory: 7c cb b3 05 01 00 00 00 > > and have to manually transpose the bytes to see if they match an > important pointer value. > > I'd like to always print the full pointer value, if possible: > stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable > memory: 0x0000000105b3cb7c | 7c cb b3 05 01 00 00 00 > > The suggested patch only prints this value if the dereferenced address > is aligned, otherwise it skips it. > > Side-note: the code that determines if the memory is readable only > checks 4 bytes. So if we have a pointer at the end last few bytes of a > page, and an inaccessible page after that, it seems like the > pre-existing code will crash. > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue Jun 9 13:41:50 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 15:41:50 +0200 Subject: RFR: 8247201: Print potential pointer value of readable stack memory in hs_err file In-Reply-To: <08ff111e-8547-4386-2308-eb773cdb14e8@oracle.com> References: <08ff111e-8547-4386-2308-eb773cdb14e8@oracle.com> Message-ID: <4fb2af93-aa10-b380-78f3-b2424aeec79c@oracle.com> Thanks, David. StefanK On 2020-06-09 15:36, David Holmes wrote: > Hi Stefan, > > Looks good! > > Thanks, > David > > On 9/06/2020 9:25 pm, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to print potential pointer value of readable >> stack memory in hs_err files. >> >> https://cr.openjdk.java.net/~stefank/8247201/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247201 >> >> I often see these lines in hs_err file: >> stack at sp + 2 slots: 0x00007000042e6930 points into unknown >> readable memory: 7c cb b3 05 01 00 00 00 >> >> and have to manually transpose the bytes to see if they match an >> important pointer value. >> >> I'd like to always print the full pointer value, if possible: >> stack at sp + 2 slots: 0x00007000042e6930 points into unknown >> readable memory: 0x0000000105b3cb7c | 7c cb b3 05 01 00 00 00 >> >> The suggested patch only prints this value if the dereferenced >> address is aligned, otherwise it skips it. >> >> Side-note: the code that determines if the memory is readable only >> checks 4 bytes. So if we have a pointer at the end last few bytes of >> a page, and an inaccessible page after that, it seems like the >> pre-existing code will crash. >> >> Thanks, >> StefanK From thomas.schatzl at oracle.com Tue Jun 9 13:54:32 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 9 Jun 2020 15:54:32 +0200 Subject: RFR: 8247201: Print potential pointer value of readable stack memory in hs_err file In-Reply-To: References: Message-ID: Hi, On 09.06.20 13:25, Stefan Karlsson wrote: > Hi all, > > Please review this patch to print potential pointer value of readable > stack memory in hs_err files. > > https://cr.openjdk.java.net/~stefank/8247201/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247201 > > I often see these lines in hs_err file: > stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable > memory: 7c cb b3 05 01 00 00 00 > > and have to manually transpose the bytes to see if they match an > important pointer value. > > I'd like to always print the full pointer value, if possible: > stack at sp + 2 slots: 0x00007000042e6930 points into unknown readable > memory: 0x0000000105b3cb7c | 7c cb b3 05 01 00 00 00 > > The suggested patch only prints this value if the dereferenced address > is aligned, otherwise it skips it. > > Side-note: the code that determines if the memory is readable only > checks 4 bytes. So if we have a pointer at the end last few bytes of a > page, and an inaccessible page after that, it seems like the > pre-existing code will crash. > lgtm. Thomas From stefan.karlsson at oracle.com Tue Jun 9 13:54:32 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 15:54:32 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <339ec19a-d45d-0045-5717-c30d4ecf0dbe@oracle.com> References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> <339ec19a-d45d-0045-5717-c30d4ecf0dbe@oracle.com> Message-ID: Hi David, On 2020-06-09 15:25, David Holmes wrote: > Hi Stefan, > > I think the new file looks better but I am worried about the > disruption this may cause to existing scraping tools that may be in use. Thanks. IMHO, we shouldn't let the crash report files bit rot and start to look like a mess either. The hs_err files change all the time, and potential scraping tools need to adapt. Just like GC log scraping tools have to adapt to new releases. > The SAP folk seem to care most about the content and layout in this > area so if they are okay with it then I'm okay with it. OK. Thanks, StefanK > > Thanks, > David > ----- > > On 9/06/2020 10:49 pm, Stefan Karlsson wrote: >> One small update. I'd like to add a new line between the heap >> printing and G1's heap region printing: >> ??https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ >> ??https://cr.openjdk.java.net/~stefank/8246926/webrev.02 >> >> This now prints: >> >> Heap: >> ??garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >> 0x0000000800000000) >> ?? region size 16384K, 0 young (0K), 0 survivors (0K) >> ??Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >> 1056768K >> ?? class space??? used 4K, capacity 384K, committed 384K, reserved >> 1048576K >> >> Heap Regions: E=young(eden), S=young(survivor), O=old, >> HS=humongous(starts), HC=humongous(continues), CS=collection set, >> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >> (previous, next) >> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| >> F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| >> F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >> >> Instead of: >> >> Heap: >> ??garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >> 0x0000000800000000) >> ?? region size 16384K, 0 young (0K), 0 survivors (0K) >> ??Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >> 1056768K >> ?? class space??? used 4K, capacity 384K, committed 384K, reserved >> 1048576K >> Heap Regions: E=young(eden), S=young(survivor), O=old, >> HS=humongous(starts), HC=humongous(continues), CS=collection set, >> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >> (previous, next) >> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| >> F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| >> F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >> >> Thanks, >> StefanK >> >> On 2020-06-09 12:17, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to clean up newlines and whitespaces in our >>> hs_err files. >>> >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >>> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >>> >>> I've uploaded before and after hs_err files here: >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >>> >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >>> >>> >>> Main cleanups: >>> - Numerous one-liners where written as two lines: >>> >>> Old >>> === >>> Online cpus: >>> 0-31 >>> Offline cpus: >>> >>> cache level: >>> 1 >>> cache type: >>> Data >>> cache size: >>> 32K >>> cache coherency line size: >>> 64 >>> <... many more ...> >>> >>> New >>> === >>> Online cpus: 0-31 >>> Offline cpus: >>> cache level: 1 >>> cache type: Data >>> cache size: 32K >>> cache coherency line size: 64 >>> >>> >>> - Many lines missed a space after colon: >>> >>> Old >>> === >>> VM state:not at safepoint (not fully initialized) >>> >>> New >>> === >>> VM state: not at safepoint (not fully initialized) >>> >>> >>> - Newline after OS >>> >>> The code looked for various files (on Linux) and dumped their >>> contents. The first line ended up on the OS: line, but the rest >>> ended afterwards >>> >>> Example from Linux: >>> OS:DISTRIB_ID=Ubuntu >>> DISTRIB_RELEASE=20.04 >>> DISTRIB_CODENAME=focal >>> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >>> uname:Linux >>> >>> Example from macOS: >>> OS:uname:Darwin >>> >>> I couldn't find a quick and clean fix for this, so I simply opted to >>> add a newline after "OS:". >>> >>> >>> - There are a number of cr() calls (or ?\n" prints) that are used to >>> add a complete blank line between sections. I moved a few of those >>> out of the lower-level print functions, so that the caller could >>> decide if there's a need for extra newlines. >>> >>> Thanks, >>> StefanK >> From stefan.karlsson at oracle.com Tue Jun 9 13:55:34 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 15:55:34 +0200 Subject: RFR: 8247201: Print potential pointer value of readable stack memory in hs_err file In-Reply-To: References: Message-ID: <6e05720c-d744-add3-e35c-3e483af6d015@oracle.com> Thanks, Thomas. StefanK On 2020-06-09 15:54, Thomas Schatzl wrote: > Hi, > > On 09.06.20 13:25, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to print potential pointer value of readable >> stack memory in hs_err files. >> >> https://cr.openjdk.java.net/~stefank/8247201/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247201 >> >> I often see these lines in hs_err file: >> stack at sp + 2 slots: 0x00007000042e6930 points into unknown >> readable memory: 7c cb b3 05 01 00 00 00 >> >> and have to manually transpose the bytes to see if they match an >> important pointer value. >> >> I'd like to always print the full pointer value, if possible: >> stack at sp + 2 slots: 0x00007000042e6930 points into unknown >> readable memory: 0x0000000105b3cb7c | 7c cb b3 05 01 00 00 00 >> >> The suggested patch only prints this value if the dereferenced >> address is aligned, otherwise it skips it. >> >> Side-note: the code that determines if the memory is readable only >> checks 4 bytes. So if we have a pointer at the end last few bytes of >> a page, and an inaccessible page after that, it seems like the >> pre-existing code will crash. >> > > ? lgtm. > > Thomas From stefan.karlsson at oracle.com Tue Jun 9 15:05:01 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 17:05:01 +0200 Subject: RFR: 8247236: fieldDescriptor::print_on_for prints extra newline after NULL Message-ID: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> Hi all, Please review this small patch to remove a newline that is incorrectly printed after NULL. https://cr.openjdk.java.net/~stefank/8247236/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8247236 Ioi reported that this bug could be seen by running: --- (gdb) call pp(m) "Executing pp" java.lang.Class {0x00000007bf0008b0} - klass: 'java/lang/Class' ?- ---- fields (total size 14 words): ?- private volatile transient 'classRedefinedCount' 'I' @12 0 ?- private volatile transient strict 'cachedConstructor' 'Ljava/lang/reflect/Constructor;' @40 NULL ?(0) ?- private transient 'name' 'Ljava/lang/String;' @44 NULL ?(0) There are newlines between NULL and (0). --- This can be seen whenever an InstanceKlass is printed and one of the oops are NULL. There's no newline if the object is non-null or if printing primitives. Note, that this is only printed in non-product builds, making it less likely that someone depends on the output format. After the patch, the lines look like this: ?- private volatile transient strict 'cachedConstructor' 'Ljava/lang/reflect/Constructor;' @40 NULL (0) ?- private transient 'name' 'Ljava/lang/String;' @44 NULL (0) Thanks, StefanK From ioi.lam at oracle.com Tue Jun 9 15:11:52 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 9 Jun 2020 08:11:52 -0700 Subject: RFR: 8247236: fieldDescriptor::print_on_for prints extra newline after NULL In-Reply-To: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> References: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> Message-ID: <49f0eefa-f586-2c8c-1e47-9933960c7ee3@oracle.com> Thanks Stefan for fixing this pet peeve of mime! It looks good to me. - Ioi On 6/9/20 8:05 AM, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to remove a newline that is incorrectly > printed after NULL. > > https://cr.openjdk.java.net/~stefank/8247236/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247236 > > Ioi reported that this bug could be seen by running: > > --- > (gdb) call pp(m) > "Executing pp" > java.lang.Class > {0x00000007bf0008b0} - klass: 'java/lang/Class' > ?- ---- fields (total size 14 words): > ?- private volatile transient 'classRedefinedCount' 'I' @12 0 > ?- private volatile transient strict 'cachedConstructor' > 'Ljava/lang/reflect/Constructor;' @40 NULL > ?(0) > ?- private transient 'name' 'Ljava/lang/String;' @44 NULL > ?(0) > > There are newlines between NULL and (0). > --- > > This can be seen whenever an InstanceKlass is printed and one of the > oops are NULL. There's no newline if the object is non-null or if > printing primitives. Note, that this is only printed in non-product > builds, making it less likely that someone depends on the output format. > > After the patch, the lines look like this: > ?- private volatile transient strict 'cachedConstructor' > 'Ljava/lang/reflect/Constructor;' @40 NULL (0) > ?- private transient 'name' 'Ljava/lang/String;' @44 NULL (0) > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue Jun 9 15:26:08 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 17:26:08 +0200 Subject: RFR: 8247236: fieldDescriptor::print_on_for prints extra newline after NULL In-Reply-To: <49f0eefa-f586-2c8c-1e47-9933960c7ee3@oracle.com> References: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> <49f0eefa-f586-2c8c-1e47-9933960c7ee3@oracle.com> Message-ID: <8789edcd-85f5-babe-d125-a80794b6f0cc@oracle.com> Thanks for reviewing! StefanK On 2020-06-09 17:11, Ioi Lam wrote: > Thanks Stefan for fixing this pet peeve of mime! It looks good to me. > > - Ioi > > On 6/9/20 8:05 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this small patch to remove a newline that is >> incorrectly printed after NULL. >> >> https://cr.openjdk.java.net/~stefank/8247236/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247236 >> >> Ioi reported that this bug could be seen by running: >> >> --- >> (gdb) call pp(m) >> "Executing pp" >> java.lang.Class >> {0x00000007bf0008b0} - klass: 'java/lang/Class' >> ?- ---- fields (total size 14 words): >> ?- private volatile transient 'classRedefinedCount' 'I' @12 0 >> ?- private volatile transient strict 'cachedConstructor' >> 'Ljava/lang/reflect/Constructor;' @40 NULL >> ?(0) >> ?- private transient 'name' 'Ljava/lang/String;' @44 NULL >> ?(0) >> >> There are newlines between NULL and (0). >> --- >> >> This can be seen whenever an InstanceKlass is printed and one of the >> oops are NULL. There's no newline if the object is non-null or if >> printing primitives. Note, that this is only printed in non-product >> builds, making it less likely that someone depends on the output format. >> >> After the patch, the lines look like this: >> ?- private volatile transient strict 'cachedConstructor' >> 'Ljava/lang/reflect/Constructor;' @40 NULL (0) >> ?- private transient 'name' 'Ljava/lang/String;' @44 NULL (0) >> >> Thanks, >> StefanK > From thomas.schatzl at oracle.com Tue Jun 9 15:27:36 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 9 Jun 2020 17:27:36 +0200 Subject: RFR: 8247236: fieldDescriptor::print_on_for prints extra newline after NULL In-Reply-To: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> References: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> Message-ID: <2d9f61dc-197b-06a5-6391-b39b96c848f5@oracle.com> Hi Stefan, On 09.06.20 17:05, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to remove a newline that is incorrectly > printed after NULL. > > https://cr.openjdk.java.net/~stefank/8247236/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247236 > > Ioi reported that this bug could be seen by running: > > --- > (gdb) call pp(m) > "Executing pp" > java.lang.Class > {0x00000007bf0008b0} - klass: 'java/lang/Class' > ?- ---- fields (total size 14 words): > ?- private volatile transient 'classRedefinedCount' 'I' @12 0 > ?- private volatile transient strict 'cachedConstructor' > 'Ljava/lang/reflect/Constructor;' @40 NULL > ?(0) > ?- private transient 'name' 'Ljava/lang/String;' @44 NULL > ?(0) > > There are newlines between NULL and (0). > --- > > This can be seen whenever an InstanceKlass is printed and one of the > oops are NULL. There's no newline if the object is non-null or if > printing primitives. Note, that this is only printed in non-product > builds, making it less likely that someone depends on the output format. > > After the patch, the lines look like this: > ?- private volatile transient strict 'cachedConstructor' > 'Ljava/lang/reflect/Constructor;' @40 NULL (0) > ?- private transient 'name' 'Ljava/lang/String;' @44 NULL (0) thanks a lot for fixing up the hs_err log output. Looks good. Thomas From stefan.karlsson at oracle.com Tue Jun 9 15:30:15 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 17:30:15 +0200 Subject: RFR: 8247236: fieldDescriptor::print_on_for prints extra newline after NULL In-Reply-To: <2d9f61dc-197b-06a5-6391-b39b96c848f5@oracle.com> References: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> <2d9f61dc-197b-06a5-6391-b39b96c848f5@oracle.com> Message-ID: <693abfc4-2808-46d4-56bc-fd68150902e1@oracle.com> Thanks for reviewing! StefanK On 2020-06-09 17:27, Thomas Schatzl wrote: > Hi Stefan, > > On 09.06.20 17:05, Stefan Karlsson wrote: >> Hi all, >> >> Please review this small patch to remove a newline that is >> incorrectly printed after NULL. >> >> https://cr.openjdk.java.net/~stefank/8247236/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247236 >> >> Ioi reported that this bug could be seen by running: >> >> --- >> (gdb) call pp(m) >> "Executing pp" >> java.lang.Class >> {0x00000007bf0008b0} - klass: 'java/lang/Class' >> ??- ---- fields (total size 14 words): >> ??- private volatile transient 'classRedefinedCount' 'I' @12 0 >> ??- private volatile transient strict 'cachedConstructor' >> 'Ljava/lang/reflect/Constructor;' @40 NULL >> ??(0) >> ??- private transient 'name' 'Ljava/lang/String;' @44 NULL >> ??(0) >> >> There are newlines between NULL and (0). >> --- >> >> This can be seen whenever an InstanceKlass is printed and one of the >> oops are NULL. There's no newline if the object is non-null or if >> printing primitives. Note, that this is only printed in non-product >> builds, making it less likely that someone depends on the output format. >> >> After the patch, the lines look like this: >> ??- private volatile transient strict 'cachedConstructor' >> 'Ljava/lang/reflect/Constructor;' @40 NULL (0) >> ??- private transient 'name' 'Ljava/lang/String;' @44 NULL (0) > > thanks a lot for fixing up the hs_err log output. Looks good. > > Thomas From igor.ignatyev at oracle.com Tue Jun 9 15:56:31 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Jun 2020 08:56:31 -0700 Subject: RFR(S) : 8246387 : switch to jtreg 5.1 Message-ID: http://cr.openjdk.java.net/~iignatyev/8246387/webrev.00/ > 110 lines changed: 6 ins; 89 del; 15 mod; Hi all, could you please review the patch which switches jtreg used in jdk/jdk to the newly promoted jtreg5.1? as before, the patch changes 'requiredVersion' property in all jtreg test suites to have more consistent results. besides changing the version in jib-profiles and requiredVersion in all TEST.ROOT, the patch also - removes no-op methods from GatherDiagnosticInfoObserver. in jtharness 6.0-b11, and Harness.Observer::finishedTesting() method got deprecated, but instead of suppressing the warning, we can just remove it and other empty methods as Harness.Observer interface now provides no-op implementations for all its methods; - removes j.t.lib.Utils::getTestName method. this method has a (now) wrong assumption that testng classes are available only if we are executing a testng test. the method was used only by appcds tests to get a test specific filename, and these tests were updated to use a new supported way to get a testname via `test.name` system property; - updates test/hotspot/jtreg/runtime/condy/staticInit/TestInitException.java test to workaround CODETOOLS-7902686[1] . jasm from asmtools 7.0-b08 incorrectly sets full file path on windows as SourceFile, so the test was temporarily updated to use regexp which matches both full path and just basename. webrev: http://cr.openjdk.java.net/~iignatyev/8246387/webrev.00/ testing: tier1-4 JBS: https://bugs.openjdk.java.net/browse/JDK-8246387 [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902686 (fixed in asmtools7.0-b09, yet jtreg5.1 uses asmtools7.0-b08, so effectively can be fixed only w/ switch to a next version of jtreg) Thanks, -- Igor From erik.joelsson at oracle.com Tue Jun 9 16:10:02 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 9 Jun 2020 09:10:02 -0700 Subject: RFR(S) : 8246387 : switch to jtreg 5.1 In-Reply-To: References: Message-ID: <5f6301b8-a928-0673-e619-35050cf08be6@oracle.com> Looks good to me. /Erik On 2020-06-09 08:56, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8246387/webrev.00/ >> 110 lines changed: 6 ins; 89 del; 15 mod; > Hi all, > > could you please review the patch which switches jtreg used in jdk/jdk to the newly promoted jtreg5.1? as before, the patch changes 'requiredVersion' property in all jtreg test suites to have more consistent results. > > besides changing the version in jib-profiles and requiredVersion in all TEST.ROOT, the patch also > - removes no-op methods from GatherDiagnosticInfoObserver. in jtharness 6.0-b11, and Harness.Observer::finishedTesting() method got deprecated, but instead of suppressing the warning, we can just remove it and other empty methods as Harness.Observer interface now provides no-op implementations for all its methods; > - removes j.t.lib.Utils::getTestName method. this method has a (now) wrong assumption that testng classes are available only if we are executing a testng test. the method was used only by appcds tests to get a test specific filename, and these tests were updated to use a new supported way to get a testname via `test.name` system property; > - updates test/hotspot/jtreg/runtime/condy/staticInit/TestInitException.java test to workaround CODETOOLS-7902686[1] . jasm from asmtools 7.0-b08 incorrectly sets full file path on windows as SourceFile, so the test was temporarily updated to use regexp which matches both full path and just basename. > > > webrev: http://cr.openjdk.java.net/~iignatyev/8246387/webrev.00/ > testing: tier1-4 > JBS: https://bugs.openjdk.java.net/browse/JDK-8246387 > > [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902686 > (fixed in asmtools7.0-b09, yet jtreg5.1 uses asmtools7.0-b08, so effectively can be fixed only w/ switch to a next version of jtreg) > > Thanks, > -- Igor From ioi.lam at oracle.com Tue Jun 9 16:22:57 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 9 Jun 2020 09:22:57 -0700 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> <339ec19a-d45d-0045-5717-c30d4ecf0dbe@oracle.com> Message-ID: The changes look good to me. From a scraping tool's perspective, relevant info broken up in multiple lines is 1000 times more difficult to process (you can't just grep anymore). ??? cache level: ??? 1 ??? cache type: ??? Data I think this change will make people's life a lot easier. Thanks - Ioi On 6/9/20 6:54 AM, Stefan Karlsson wrote: > Hi David, > > On 2020-06-09 15:25, David Holmes wrote: >> Hi Stefan, >> >> I think the new file looks better but I am worried about the >> disruption this may cause to existing scraping tools that may be in use. > > Thanks. IMHO, we shouldn't let the crash report files bit rot and > start to look like a mess either. The hs_err files change all the > time, and potential scraping tools need to adapt. Just like GC log > scraping tools have to adapt to new releases. > >> The SAP folk seem to care most about the content and layout in this >> area so if they are okay with it then I'm okay with it. > OK. > > Thanks, > StefanK > >> >> Thanks, >> David >> ----- >> >> On 9/06/2020 10:49 pm, Stefan Karlsson wrote: >>> One small update. I'd like to add a new line between the heap >>> printing and G1's heap region printing: >>> ??https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ >>> ??https://cr.openjdk.java.net/~stefank/8246926/webrev.02 >>> >>> This now prints: >>> >>> Heap: >>> ??garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >>> 0x0000000800000000) >>> ?? region size 16384K, 0 young (0K), 0 survivors (0K) >>> ??Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >>> 1056768K >>> ?? class space??? used 4K, capacity 384K, committed 384K, reserved >>> 1048576K >>> >>> Heap Regions: E=young(eden), S=young(survivor), O=old, >>> HS=humongous(starts), HC=humongous(continues), CS=collection set, >>> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >>> (previous, next) >>> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| >>> 0%| F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >>> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| >>> 0%| F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >>> >>> Instead of: >>> >>> Heap: >>> ??garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >>> 0x0000000800000000) >>> ?? region size 16384K, 0 young (0K), 0 survivors (0K) >>> ??Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >>> 1056768K >>> ?? class space??? used 4K, capacity 384K, committed 384K, reserved >>> 1048576K >>> Heap Regions: E=young(eden), S=young(survivor), O=old, >>> HS=humongous(starts), HC=humongous(continues), CS=collection set, >>> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >>> (previous, next) >>> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| >>> 0%| F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >>> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| >>> 0%| F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >>> >>> Thanks, >>> StefanK >>> >>> On 2020-06-09 12:17, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to clean up newlines and whitespaces in >>>> our hs_err files. >>>> >>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >>>> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >>>> >>>> I've uploaded before and after hs_err files here: >>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >>>> >>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >>>> >>>> >>>> Main cleanups: >>>> - Numerous one-liners where written as two lines: >>>> >>>> Old >>>> === >>>> Online cpus: >>>> 0-31 >>>> Offline cpus: >>>> >>>> cache level: >>>> 1 >>>> cache type: >>>> Data >>>> cache size: >>>> 32K >>>> cache coherency line size: >>>> 64 >>>> <... many more ...> >>>> >>>> New >>>> === >>>> Online cpus: 0-31 >>>> Offline cpus: >>>> cache level: 1 >>>> cache type: Data >>>> cache size: 32K >>>> cache coherency line size: 64 >>>> >>>> >>>> - Many lines missed a space after colon: >>>> >>>> Old >>>> === >>>> VM state:not at safepoint (not fully initialized) >>>> >>>> New >>>> === >>>> VM state: not at safepoint (not fully initialized) >>>> >>>> >>>> - Newline after OS >>>> >>>> The code looked for various files (on Linux) and dumped their >>>> contents. The first line ended up on the OS: line, but the rest >>>> ended afterwards >>>> >>>> Example from Linux: >>>> OS:DISTRIB_ID=Ubuntu >>>> DISTRIB_RELEASE=20.04 >>>> DISTRIB_CODENAME=focal >>>> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >>>> uname:Linux >>>> >>>> Example from macOS: >>>> OS:uname:Darwin >>>> >>>> I couldn't find a quick and clean fix for this, so I simply opted >>>> to add a newline after "OS:". >>>> >>>> >>>> - There are a number of cr() calls (or ?\n" prints) that are used >>>> to add a complete blank line between sections. I moved a few of >>>> those out of the lower-level print functions, so that the caller >>>> could decide if there's a need for extra newlines. >>>> >>>> Thanks, >>>> StefanK >>> > From stefan.karlsson at oracle.com Tue Jun 9 16:36:03 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 18:36:03 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> <339ec19a-d45d-0045-5717-c30d4ecf0dbe@oracle.com> Message-ID: Thanks, Ioi. StefanK On 2020-06-09 18:22, Ioi Lam wrote: > The changes look good to me. > > From a scraping tool's perspective, relevant info broken up in > multiple lines is 1000 times more difficult to process (you can't just > grep anymore). > > ??? cache level: > ??? 1 > ??? cache type: > ??? Data > > I think this change will make people's life a lot easier. > > > Thanks > - Ioi > > > > On 6/9/20 6:54 AM, Stefan Karlsson wrote: >> Hi David, >> >> On 2020-06-09 15:25, David Holmes wrote: >>> Hi Stefan, >>> >>> I think the new file looks better but I am worried about the >>> disruption this may cause to existing scraping tools that may be in >>> use. >> >> Thanks. IMHO, we shouldn't let the crash report files bit rot and >> start to look like a mess either. The hs_err files change all the >> time, and potential scraping tools need to adapt. Just like GC log >> scraping tools have to adapt to new releases. >> >>> The SAP folk seem to care most about the content and layout in this >>> area so if they are okay with it then I'm okay with it. >> OK. >> >> Thanks, >> StefanK >> >>> >>> Thanks, >>> David >>> ----- >>> >>> On 9/06/2020 10:49 pm, Stefan Karlsson wrote: >>>> One small update. I'd like to add a new line between the heap >>>> printing and G1's heap region printing: >>>> ??https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ >>>> ??https://cr.openjdk.java.net/~stefank/8246926/webrev.02 >>>> >>>> This now prints: >>>> >>>> Heap: >>>> ??garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >>>> 0x0000000800000000) >>>> ?? region size 16384K, 0 young (0K), 0 survivors (0K) >>>> ??Metaspace?????? used 4K, capacity 4480K, committed 4480K, >>>> reserved 1056768K >>>> ?? class space??? used 4K, capacity 384K, committed 384K, reserved >>>> 1048576K >>>> >>>> Heap Regions: E=young(eden), S=young(survivor), O=old, >>>> HS=humongous(starts), HC=humongous(continues), CS=collection set, >>>> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >>>> (previous, next) >>>> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| >>>> 0%| F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >>>> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| >>>> 0%| F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >>>> >>>> Instead of: >>>> >>>> Heap: >>>> ??garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >>>> 0x0000000800000000) >>>> ?? region size 16384K, 0 young (0K), 0 survivors (0K) >>>> ??Metaspace?????? used 4K, capacity 4480K, committed 4480K, >>>> reserved 1056768K >>>> ?? class space??? used 4K, capacity 384K, committed 384K, reserved >>>> 1048576K >>>> Heap Regions: E=young(eden), S=young(survivor), O=old, >>>> HS=humongous(starts), HC=humongous(continues), CS=collection set, >>>> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >>>> (previous, next) >>>> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| >>>> 0%| F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >>>> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| >>>> 0%| F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >>>> >>>> Thanks, >>>> StefanK >>>> >>>> On 2020-06-09 12:17, Stefan Karlsson wrote: >>>>> Hi all, >>>>> >>>>> Please review this patch to clean up newlines and whitespaces in >>>>> our hs_err files. >>>>> >>>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >>>>> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >>>>> >>>>> I've uploaded before and after hs_err files here: >>>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >>>>> >>>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >>>>> >>>>> >>>>> Main cleanups: >>>>> - Numerous one-liners where written as two lines: >>>>> >>>>> Old >>>>> === >>>>> Online cpus: >>>>> 0-31 >>>>> Offline cpus: >>>>> >>>>> cache level: >>>>> 1 >>>>> cache type: >>>>> Data >>>>> cache size: >>>>> 32K >>>>> cache coherency line size: >>>>> 64 >>>>> <... many more ...> >>>>> >>>>> New >>>>> === >>>>> Online cpus: 0-31 >>>>> Offline cpus: >>>>> cache level: 1 >>>>> cache type: Data >>>>> cache size: 32K >>>>> cache coherency line size: 64 >>>>> >>>>> >>>>> - Many lines missed a space after colon: >>>>> >>>>> Old >>>>> === >>>>> VM state:not at safepoint (not fully initialized) >>>>> >>>>> New >>>>> === >>>>> VM state: not at safepoint (not fully initialized) >>>>> >>>>> >>>>> - Newline after OS >>>>> >>>>> The code looked for various files (on Linux) and dumped their >>>>> contents. The first line ended up on the OS: line, but the rest >>>>> ended afterwards >>>>> >>>>> Example from Linux: >>>>> OS:DISTRIB_ID=Ubuntu >>>>> DISTRIB_RELEASE=20.04 >>>>> DISTRIB_CODENAME=focal >>>>> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >>>>> uname:Linux >>>>> >>>>> Example from macOS: >>>>> OS:uname:Darwin >>>>> >>>>> I couldn't find a quick and clean fix for this, so I simply opted >>>>> to add a newline after "OS:". >>>>> >>>>> >>>>> - There are a number of cr() calls (or ?\n" prints) that are used >>>>> to add a complete blank line between sections. I moved a few of >>>>> those out of the lower-level print functions, so that the caller >>>>> could decide if there's a need for extra newlines. >>>>> >>>>> Thanks, >>>>> StefanK >>>> >> > From coleen.phillimore at oracle.com Tue Jun 9 16:46:09 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 9 Jun 2020 12:46:09 -0400 Subject: RFR: 8247236: fieldDescriptor::print_on_for prints extra newline after NULL In-Reply-To: <49f0eefa-f586-2c8c-1e47-9933960c7ee3@oracle.com> References: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> <49f0eefa-f586-2c8c-1e47-9933960c7ee3@oracle.com> Message-ID: <4a7221c3-a85f-226b-287f-27dee9853a10@oracle.com> This change looks good and trivial. thanks, Coleen On 6/9/20 11:11 AM, Ioi Lam wrote: > Thanks Stefan for fixing this pet peeve of mime! It looks good to me. > > - Ioi > > On 6/9/20 8:05 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this small patch to remove a newline that is >> incorrectly printed after NULL. >> >> https://cr.openjdk.java.net/~stefank/8247236/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247236 >> >> Ioi reported that this bug could be seen by running: >> >> --- >> (gdb) call pp(m) >> "Executing pp" >> java.lang.Class >> {0x00000007bf0008b0} - klass: 'java/lang/Class' >> ?- ---- fields (total size 14 words): >> ?- private volatile transient 'classRedefinedCount' 'I' @12 0 >> ?- private volatile transient strict 'cachedConstructor' >> 'Ljava/lang/reflect/Constructor;' @40 NULL >> ?(0) >> ?- private transient 'name' 'Ljava/lang/String;' @44 NULL >> ?(0) >> >> There are newlines between NULL and (0). >> --- >> >> This can be seen whenever an InstanceKlass is printed and one of the >> oops are NULL. There's no newline if the object is non-null or if >> printing primitives. Note, that this is only printed in non-product >> builds, making it less likely that someone depends on the output format. >> >> After the patch, the lines look like this: >> ?- private volatile transient strict 'cachedConstructor' >> 'Ljava/lang/reflect/Constructor;' @40 NULL (0) >> ?- private transient 'name' 'Ljava/lang/String;' @44 NULL (0) >> >> Thanks, >> StefanK > From stefan.karlsson at oracle.com Tue Jun 9 16:49:35 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 18:49:35 +0200 Subject: RFR: 8247236: fieldDescriptor::print_on_for prints extra newline after NULL In-Reply-To: <4a7221c3-a85f-226b-287f-27dee9853a10@oracle.com> References: <481d1cdc-2d97-f357-240c-26362f2211cd@oracle.com> <49f0eefa-f586-2c8c-1e47-9933960c7ee3@oracle.com> <4a7221c3-a85f-226b-287f-27dee9853a10@oracle.com> Message-ID: <67a4c832-795d-d8c5-9ada-31de0f487c49@oracle.com> Thanks, Coleen. StefanK On 2020-06-09 18:46, coleen.phillimore at oracle.com wrote: > > This change looks good and trivial. > thanks, > Coleen > > On 6/9/20 11:11 AM, Ioi Lam wrote: >> Thanks Stefan for fixing this pet peeve of mime! It looks good to me. >> >> - Ioi >> >> On 6/9/20 8:05 AM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this small patch to remove a newline that is >>> incorrectly printed after NULL. >>> >>> https://cr.openjdk.java.net/~stefank/8247236/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8247236 >>> >>> Ioi reported that this bug could be seen by running: >>> >>> --- >>> (gdb) call pp(m) >>> "Executing pp" >>> java.lang.Class >>> {0x00000007bf0008b0} - klass: 'java/lang/Class' >>> ?- ---- fields (total size 14 words): >>> ?- private volatile transient 'classRedefinedCount' 'I' @12 0 >>> ?- private volatile transient strict 'cachedConstructor' >>> 'Ljava/lang/reflect/Constructor;' @40 NULL >>> ?(0) >>> ?- private transient 'name' 'Ljava/lang/String;' @44 NULL >>> ?(0) >>> >>> There are newlines between NULL and (0). >>> --- >>> >>> This can be seen whenever an InstanceKlass is printed and one of the >>> oops are NULL. There's no newline if the object is non-null or if >>> printing primitives. Note, that this is only printed in non-product >>> builds, making it less likely that someone depends on the output >>> format. >>> >>> After the patch, the lines look like this: >>> ?- private volatile transient strict 'cachedConstructor' >>> 'Ljava/lang/reflect/Constructor;' @40 NULL (0) >>> ?- private transient 'name' 'Ljava/lang/String;' @44 NULL (0) >>> >>> Thanks, >>> StefanK >> > From stefan.karlsson at oracle.com Tue Jun 9 19:03:10 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 21:03:10 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> Message-ID: <94163b93-0de4-0d38-c416-6e63dd51a641@oracle.com> And a small test fix to match the new format: ?https://cr.openjdk.java.net/~stefank/8246926/webrev.03.delta/ ?https://cr.openjdk.java.net/~stefank/8246926/webrev.03/ StefanK On 2020-06-09 14:49, Stefan Karlsson wrote: > One small update. I'd like to add a new line between the heap printing > and G1's heap region printing: > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02 > > This now prints: > > Heap: > ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, > 0x0000000800000000) > ? region size 16384K, 0 young (0K), 0 survivors (0K) > ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved > 1056768K > ? class space??? used 4K, capacity 384K, committed 384K, reserved > 1048576K > > Heap Regions: E=young(eden), S=young(survivor), O=old, > HS=humongous(starts), HC=humongous(continues), CS=collection set, > F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start > (previous, next) > |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| > F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked > |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| > F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked > > Instead of: > > Heap: > ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, > 0x0000000800000000) > ? region size 16384K, 0 young (0K), 0 survivors (0K) > ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved > 1056768K > ? class space??? used 4K, capacity 384K, committed 384K, reserved > 1048576K > Heap Regions: E=young(eden), S=young(survivor), O=old, > HS=humongous(starts), HC=humongous(continues), CS=collection set, > F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start > (previous, next) > |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| > F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked > |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| > F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked > > Thanks, > StefanK > > On 2020-06-09 12:17, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to clean up newlines and whitespaces in our >> hs_err files. >> >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >> >> I've uploaded before and after hs_err files here: >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >> >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >> >> >> Main cleanups: >> - Numerous one-liners where written as two lines: >> >> Old >> === >> Online cpus: >> 0-31 >> Offline cpus: >> >> cache level: >> 1 >> cache type: >> Data >> cache size: >> 32K >> cache coherency line size: >> 64 >> <... many more ...> >> >> New >> === >> Online cpus: 0-31 >> Offline cpus: >> cache level: 1 >> cache type: Data >> cache size: 32K >> cache coherency line size: 64 >> >> >> - Many lines missed a space after colon: >> >> Old >> === >> VM state:not at safepoint (not fully initialized) >> >> New >> === >> VM state: not at safepoint (not fully initialized) >> >> >> - Newline after OS >> >> The code looked for various files (on Linux) and dumped their >> contents. The first line ended up on the OS: line, but the rest ended >> afterwards >> >> Example from Linux: >> OS:DISTRIB_ID=Ubuntu >> DISTRIB_RELEASE=20.04 >> DISTRIB_CODENAME=focal >> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >> uname:Linux >> >> Example from macOS: >> OS:uname:Darwin >> >> I couldn't find a quick and clean fix for this, so I simply opted to >> add a newline after "OS:". >> >> >> - There are a number of cr() calls (or ?\n" prints) that are used to >> add a complete blank line between sections. I moved a few of those >> out of the lower-level print functions, so that the caller could >> decide if there's a need for extra newlines. >> >> Thanks, >> StefanK > From erik.osterlund at oracle.com Tue Jun 9 19:48:46 2020 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Tue, 9 Jun 2020 21:48:46 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <94163b93-0de4-0d38-c416-6e63dd51a641@oracle.com> References: <94163b93-0de4-0d38-c416-6e63dd51a641@oracle.com> Message-ID: <3A2BDA71-E01E-412A-8E23-348D3C08AD8C@oracle.com> Still good. /Erik > On 9 Jun 2020, at 21:03, Stefan Karlsson wrote: > > ?And a small test fix to match the new format: > https://cr.openjdk.java.net/~stefank/8246926/webrev.03.delta/ > https://cr.openjdk.java.net/~stefank/8246926/webrev.03/ > > StefanK > >> On 2020-06-09 14:49, Stefan Karlsson wrote: >> One small update. I'd like to add a new line between the heap printing and G1's heap region printing: >> https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ >> https://cr.openjdk.java.net/~stefank/8246926/webrev.02 >> >> This now prints: >> >> Heap: >> garbage-first heap total 2064384K, used 0K [0x0000000082000000, 0x0000000800000000) >> region size 16384K, 0 young (0K), 0 survivors (0K) >> Metaspace used 4K, capacity 4480K, committed 4480K, reserved 1056768K >> class space used 4K, capacity 384K, committed 384K, reserved 1048576K >> >> Heap Regions: E=young(eden), S=young(survivor), O=old, HS=humongous(starts), HC=humongous(continues), CS=collection set, F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start (previous, next) >> | 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| F| |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >> | 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| F| |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >> >> Instead of: >> >> Heap: >> garbage-first heap total 2064384K, used 0K [0x0000000082000000, 0x0000000800000000) >> region size 16384K, 0 young (0K), 0 survivors (0K) >> Metaspace used 4K, capacity 4480K, committed 4480K, reserved 1056768K >> class space used 4K, capacity 384K, committed 384K, reserved 1048576K >> Heap Regions: E=young(eden), S=young(survivor), O=old, HS=humongous(starts), HC=humongous(continues), CS=collection set, F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start (previous, next) >> | 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| F| |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >> | 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| F| |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >> >> Thanks, >> StefanK >> >>> On 2020-06-09 12:17, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to clean up newlines and whitespaces in our hs_err files. >>> >>> https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8246926 >>> >>> I've uploaded before and after hs_err files here: >>> https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >>> https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >>> >>> Main cleanups: >>> - Numerous one-liners where written as two lines: >>> >>> Old >>> === >>> Online cpus: >>> 0-31 >>> Offline cpus: >>> >>> cache level: >>> 1 >>> cache type: >>> Data >>> cache size: >>> 32K >>> cache coherency line size: >>> 64 >>> <... many more ...> >>> >>> New >>> === >>> Online cpus: 0-31 >>> Offline cpus: >>> cache level: 1 >>> cache type: Data >>> cache size: 32K >>> cache coherency line size: 64 >>> >>> >>> - Many lines missed a space after colon: >>> >>> Old >>> === >>> VM state:not at safepoint (not fully initialized) >>> >>> New >>> === >>> VM state: not at safepoint (not fully initialized) >>> >>> >>> - Newline after OS >>> >>> The code looked for various files (on Linux) and dumped their contents. The first line ended up on the OS: line, but the rest ended afterwards >>> >>> Example from Linux: >>> OS:DISTRIB_ID=Ubuntu >>> DISTRIB_RELEASE=20.04 >>> DISTRIB_CODENAME=focal >>> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >>> uname:Linux >>> >>> Example from macOS: >>> OS:uname:Darwin >>> >>> I couldn't find a quick and clean fix for this, so I simply opted to add a newline after "OS:". >>> >>> >>> - There are a number of cr() calls (or ?\n" prints) that are used to add a complete blank line between sections. I moved a few of those out of the lower-level print functions, so that the caller could decide if there's a need for extra newlines. >>> >>> Thanks, >>> StefanK >> > From stefan.karlsson at oracle.com Tue Jun 9 19:52:59 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 9 Jun 2020 21:52:59 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <3A2BDA71-E01E-412A-8E23-348D3C08AD8C@oracle.com> References: <94163b93-0de4-0d38-c416-6e63dd51a641@oracle.com> <3A2BDA71-E01E-412A-8E23-348D3C08AD8C@oracle.com> Message-ID: <26c01cb8-808d-eb3c-e16e-c4ebd9e3e5ac@oracle.com> Thanks. StefanK On 2020-06-09 21:48, Erik ?sterlund wrote: > Still good. > > /Erik > >> On 9 Jun 2020, at 21:03, Stefan Karlsson wrote: >> >> ?And a small test fix to match the new format: >> https://cr.openjdk.java.net/~stefank/8246926/webrev.03.delta/ >> https://cr.openjdk.java.net/~stefank/8246926/webrev.03/ >> >> StefanK >> >>> On 2020-06-09 14:49, Stefan Karlsson wrote: >>> One small update. I'd like to add a new line between the heap printing and G1's heap region printing: >>> https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ >>> https://cr.openjdk.java.net/~stefank/8246926/webrev.02 >>> >>> This now prints: >>> >>> Heap: >>> garbage-first heap total 2064384K, used 0K [0x0000000082000000, 0x0000000800000000) >>> region size 16384K, 0 young (0K), 0 survivors (0K) >>> Metaspace used 4K, capacity 4480K, committed 4480K, reserved 1056768K >>> class space used 4K, capacity 384K, committed 384K, reserved 1048576K >>> >>> Heap Regions: E=young(eden), S=young(survivor), O=old, HS=humongous(starts), HC=humongous(continues), CS=collection set, F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start (previous, next) >>> | 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| F| |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >>> | 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| F| |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >>> >>> Instead of: >>> >>> Heap: >>> garbage-first heap total 2064384K, used 0K [0x0000000082000000, 0x0000000800000000) >>> region size 16384K, 0 young (0K), 0 survivors (0K) >>> Metaspace used 4K, capacity 4480K, committed 4480K, reserved 1056768K >>> class space used 4K, capacity 384K, committed 384K, reserved 1048576K >>> Heap Regions: E=young(eden), S=young(survivor), O=old, HS=humongous(starts), HC=humongous(continues), CS=collection set, F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start (previous, next) >>> | 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| F| |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >>> | 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| F| |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >>> >>> Thanks, >>> StefanK >>> >>>> On 2020-06-09 12:17, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to clean up newlines and whitespaces in our hs_err files. >>>> >>>> https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8246926 >>>> >>>> I've uploaded before and after hs_err files here: >>>> https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >>>> https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >>>> >>>> Main cleanups: >>>> - Numerous one-liners where written as two lines: >>>> >>>> Old >>>> === >>>> Online cpus: >>>> 0-31 >>>> Offline cpus: >>>> >>>> cache level: >>>> 1 >>>> cache type: >>>> Data >>>> cache size: >>>> 32K >>>> cache coherency line size: >>>> 64 >>>> <... many more ...> >>>> >>>> New >>>> === >>>> Online cpus: 0-31 >>>> Offline cpus: >>>> cache level: 1 >>>> cache type: Data >>>> cache size: 32K >>>> cache coherency line size: 64 >>>> >>>> >>>> - Many lines missed a space after colon: >>>> >>>> Old >>>> === >>>> VM state:not at safepoint (not fully initialized) >>>> >>>> New >>>> === >>>> VM state: not at safepoint (not fully initialized) >>>> >>>> >>>> - Newline after OS >>>> >>>> The code looked for various files (on Linux) and dumped their contents. The first line ended up on the OS: line, but the rest ended afterwards >>>> >>>> Example from Linux: >>>> OS:DISTRIB_ID=Ubuntu >>>> DISTRIB_RELEASE=20.04 >>>> DISTRIB_CODENAME=focal >>>> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >>>> uname:Linux >>>> >>>> Example from macOS: >>>> OS:uname:Darwin >>>> >>>> I couldn't find a quick and clean fix for this, so I simply opted to add a newline after "OS:". >>>> >>>> >>>> - There are a number of cr() calls (or ?\n" prints) that are used to add a complete blank line between sections. I moved a few of those out of the lower-level print functions, so that the caller could decide if there's a need for extra newlines. >>>> >>>> Thanks, >>>> StefanK From patric.hedlin at oracle.com Tue Jun 9 20:16:09 2020 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Tue, 9 Jun 2020 22:16:09 +0200 Subject: RFR(XS): 8246689: Enable independent compressed oops/class ptrs on Aarch64 In-Reply-To: <7D757361-AFF8-40E7-A8F2-ECA711DCB110@oracle.com> References: <7D757361-AFF8-40E7-A8F2-ECA711DCB110@oracle.com> Message-ID: Thanks Erik. Would you say the change is trivial... /Patric On 2020-06-09 08:04, Erik ?sterlund wrote: > Hi Patric, > > This looks good. Thank you for trying this out on AArch64. Only Graal to go now, then we can remove the switch and have support for this feature be expected on all platforms. > > /Erik > >> On 9 Jun 2020, at 00:24, Patric Hedlin wrote: >> >> ?Dear all, >> >> I would like to ask for help to review the following change/update: >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8246689 >> Webrev: http://cr.openjdk.java.net/~phedlin/tr8246689/ >> >> >> Aarch64 was not included in JDK-8241825. This small patch enables the feature on Aarch64 and updates the associated test-case (in tier2). This patch is an attempt to make tier2 all green for linux-aarch64, that we want to run in regular testing (see JDK-8246691). >> >> Testing: tier1-2 and a number of runs of RunThese30M. >> >> >> Best regards, >> Patric From erik.osterlund at oracle.com Tue Jun 9 20:18:07 2020 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Tue, 9 Jun 2020 22:18:07 +0200 Subject: RFR(XS): 8246689: Enable independent compressed oops/class ptrs on Aarch64 In-Reply-To: References: Message-ID: <4CA9A178-5A1A-4E1D-97DF-0BB88592E25B@oracle.com> Hi Patric, Yes, it?s trivial. Ship it! /Erik > On 9 Jun 2020, at 22:16, Patric Hedlin wrote: > > ?Thanks Erik. > > Would you say the change is trivial... > > /Patric > >> On 2020-06-09 08:04, Erik ?sterlund wrote: >> Hi Patric, >> >> This looks good. Thank you for trying this out on AArch64. Only Graal to go now, then we can remove the switch and have support for this feature be expected on all platforms. >> >> /Erik >> >>>> On 9 Jun 2020, at 00:24, Patric Hedlin wrote: >>> >>> ?Dear all, >>> >>> I would like to ask for help to review the following change/update: >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8246689 >>> Webrev: http://cr.openjdk.java.net/~phedlin/tr8246689/ >>> >>> >>> Aarch64 was not included in JDK-8241825. This small patch enables the feature on Aarch64 and updates the associated test-case (in tier2). This patch is an attempt to make tier2 all green for linux-aarch64, that we want to run in regular testing (see JDK-8246691). >>> >>> Testing: tier1-2 and a number of runs of RunThese30M. >>> >>> >>> Best regards, >>> Patric From david.holmes at oracle.com Tue Jun 9 22:08:26 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Jun 2020 08:08:26 +1000 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <94163b93-0de4-0d38-c416-6e63dd51a641@oracle.com> References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> <94163b93-0de4-0d38-c416-6e63dd51a641@oracle.com> Message-ID: <7208764e-c142-f6f0-ffb4-ea5ce1ba27e7@oracle.com> Test fix is good. David On 10/06/2020 5:03 am, Stefan Karlsson wrote: > And a small test fix to match the new format: > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.03.delta/ > ?https://cr.openjdk.java.net/~stefank/8246926/webrev.03/ > > StefanK > > On 2020-06-09 14:49, Stefan Karlsson wrote: >> One small update. I'd like to add a new line between the heap printing >> and G1's heap region printing: >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ >> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02 >> >> This now prints: >> >> Heap: >> ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >> 0x0000000800000000) >> ? region size 16384K, 0 young (0K), 0 survivors (0K) >> ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >> 1056768K >> ? class space??? used 4K, capacity 384K, committed 384K, reserved >> 1048576K >> >> Heap Regions: E=young(eden), S=young(survivor), O=old, >> HS=humongous(starts), HC=humongous(continues), CS=collection set, >> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >> (previous, next) >> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| >> F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| >> F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >> >> Instead of: >> >> Heap: >> ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >> 0x0000000800000000) >> ? region size 16384K, 0 young (0K), 0 survivors (0K) >> ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >> 1056768K >> ? class space??? used 4K, capacity 384K, committed 384K, reserved >> 1048576K >> Heap Regions: E=young(eden), S=young(survivor), O=old, >> HS=humongous(starts), HC=humongous(continues), CS=collection set, >> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >> (previous, next) >> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| 0%| >> F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| 0%| >> F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >> >> Thanks, >> StefanK >> >> On 2020-06-09 12:17, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to clean up newlines and whitespaces in our >>> hs_err files. >>> >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >>> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >>> >>> I've uploaded before and after hs_err files here: >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >>> >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >>> >>> >>> Main cleanups: >>> - Numerous one-liners where written as two lines: >>> >>> Old >>> === >>> Online cpus: >>> 0-31 >>> Offline cpus: >>> >>> cache level: >>> 1 >>> cache type: >>> Data >>> cache size: >>> 32K >>> cache coherency line size: >>> 64 >>> <... many more ...> >>> >>> New >>> === >>> Online cpus: 0-31 >>> Offline cpus: >>> cache level: 1 >>> cache type: Data >>> cache size: 32K >>> cache coherency line size: 64 >>> >>> >>> - Many lines missed a space after colon: >>> >>> Old >>> === >>> VM state:not at safepoint (not fully initialized) >>> >>> New >>> === >>> VM state: not at safepoint (not fully initialized) >>> >>> >>> - Newline after OS >>> >>> The code looked for various files (on Linux) and dumped their >>> contents. The first line ended up on the OS: line, but the rest ended >>> afterwards >>> >>> Example from Linux: >>> OS:DISTRIB_ID=Ubuntu >>> DISTRIB_RELEASE=20.04 >>> DISTRIB_CODENAME=focal >>> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >>> uname:Linux >>> >>> Example from macOS: >>> OS:uname:Darwin >>> >>> I couldn't find a quick and clean fix for this, so I simply opted to >>> add a newline after "OS:". >>> >>> >>> - There are a number of cr() calls (or ?\n" prints) that are used to >>> add a complete blank line between sections. I moved a few of those >>> out of the lower-level print functions, so that the caller could >>> decide if there's a need for extra newlines. >>> >>> Thanks, >>> StefanK >> > From igor.ignatyev at oracle.com Tue Jun 9 23:47:51 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Jun 2020 16:47:51 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API Message-ID: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 > > 38 lines changed: 8 ins; 16 del; 14 mod; Hi all, could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? testing: test/hotspot/jtreg/serviceability webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 Thanks, -- Igor From brian.burkhalter at oracle.com Wed Jun 10 00:18:38 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 9 Jun 2020 17:18:38 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> Message-ID: <2AF18884-0407-455C-9BE2-4355D3D9AD6C@oracle.com> Hi Igor, > On Jun 9, 2020, at 4:47 PM, Igor Ignatyev wrote: > > could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? > > testing: test/hotspot/jtreg/serviceability > webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 The NIO changes look all right to me. Brian From alexey.menkov at oracle.com Wed Jun 10 01:36:11 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 9 Jun 2020 18:36:11 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> Message-ID: Hi Igor, In SATestUtils.java you do var bb = ... Files.readAllBytes(...) ... and then use bb[0] if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. And looks like the caller doesn't expect it (it catches IOException). --alex On 06/09/2020 16:47, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >> >> 38 lines changed: 8 ins; 16 del; 14 mod; > > Hi all, > > could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? > > testing: test/hotspot/jtreg/serviceability > webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 > > Thanks, > -- Igor > From david.holmes at oracle.com Wed Jun 10 02:06:32 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Jun 2020 12:06:32 +1000 Subject: RFR(S) : 8246387 : switch to jtreg 5.1 In-Reply-To: References: Message-ID: Hi Igor, All of these changes seem okay to me. Thanks, David On 10/06/2020 1:56 am, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8246387/webrev.00/ >> 110 lines changed: 6 ins; 89 del; 15 mod; > > Hi all, > > could you please review the patch which switches jtreg used in jdk/jdk to the newly promoted jtreg5.1? as before, the patch changes 'requiredVersion' property in all jtreg test suites to have more consistent results. > > besides changing the version in jib-profiles and requiredVersion in all TEST.ROOT, the patch also > - removes no-op methods from GatherDiagnosticInfoObserver. in jtharness 6.0-b11, and Harness.Observer::finishedTesting() method got deprecated, but instead of suppressing the warning, we can just remove it and other empty methods as Harness.Observer interface now provides no-op implementations for all its methods; > - removes j.t.lib.Utils::getTestName method. this method has a (now) wrong assumption that testng classes are available only if we are executing a testng test. the method was used only by appcds tests to get a test specific filename, and these tests were updated to use a new supported way to get a testname via `test.name` system property; > - updates test/hotspot/jtreg/runtime/condy/staticInit/TestInitException.java test to workaround CODETOOLS-7902686[1] . jasm from asmtools 7.0-b08 incorrectly sets full file path on windows as SourceFile, so the test was temporarily updated to use regexp which matches both full path and just basename. > > > webrev: http://cr.openjdk.java.net/~iignatyev/8246387/webrev.00/ > testing: tier1-4 > JBS: https://bugs.openjdk.java.net/browse/JDK-8246387 > > [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902686 > (fixed in asmtools7.0-b09, yet jtreg5.1 uses asmtools7.0-b08, so effectively can be fixed only w/ switch to a next version of jtreg) > > Thanks, > -- Igor > From igor.ignatyev at oracle.com Wed Jun 10 03:11:48 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Jun 2020 20:11:48 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> Message-ID: Hi Alex, as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. -- Igor > On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: > > Hi Igor, > > In SATestUtils.java you do > > var bb = ... Files.readAllBytes(...) ... > and then use bb[0] > > if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. > And looks like the caller doesn't expect it (it catches IOException). > > --alex > > On 06/09/2020 16:47, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>> >>> 38 lines changed: 8 ins; 16 del; 14 mod; >> Hi all, >> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >> testing: test/hotspot/jtreg/serviceability >> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >> Thanks, >> -- Igor From igor.ignatyev at oracle.com Wed Jun 10 03:17:20 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Jun 2020 20:17:20 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <2AF18884-0407-455C-9BE2-4355D3D9AD6C@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> <2AF18884-0407-455C-9BE2-4355D3D9AD6C@oracle.com> Message-ID: Hi Brian, thank you for your review. Cheers, -- Igor > On Jun 9, 2020, at 5:18 PM, Brian Burkhalter wrote: > > Hi Igor, > >> On Jun 9, 2020, at 4:47 PM, Igor Ignatyev > wrote: >> >> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >> >> testing: test/hotspot/jtreg/serviceability >> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 > The NIO changes look all right to me. > > Brian From kim.barrett at oracle.com Wed Jun 10 06:26:50 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 10 Jun 2020 02:26:50 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <5EDE9A75.6020504@oracle.com> References: <5EDE9A75.6020504@oracle.com> Message-ID: > On Jun 8, 2020, at 4:07 PM, Philip Race wrote: > > Hi Kim, > > Can we amend the JEP itself to be not solely about hotspot. > Since it affects other areas I think that > 1) They may need to be compiled with C++14 enabled whether they use new features or not > 2) They may want - or need - to adopt C++11 or C++14 features too. > > You already know that soon we will upgrade the harfbuzz 3rd party library used by 2D > and it will bring along the need to use C++11 features and I had no plan to propose a JEP for that. > So I don't want to be asked why we didn't do one when hotspot did ! > So this really ought to be a JDK JEP whilst of course explaining that hotspot is expected > to be the primary adopter. I think this is at least mostly covered by the following paragraph from the JEP: (This JEP does not propose any usage or style changes for C++ code in the JDK that is outside of HotSpot. The rules are already different for HotSpot and non-HotSpot code. For example, C++ exceptions are used in some non-HotSpot code, but are disallowed in HotSpot by build-time options. However, build consistency requirements will make the newer language features available for use in all C++ code in the JDK.) That?s why I cc?d the RFR to several mailing lists in addition to hotspot-dev. Is there something additional you would like to add? > BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says > > For Solaris: Add the -std=c++14 compiler option. ..... > > So I am sure there is still some room to update the JEP :-) Yeah, I have some updates to make. I also need to do a bit of work on the feature lists. From stefan.karlsson at oracle.com Wed Jun 10 06:55:31 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 10 Jun 2020 08:55:31 +0200 Subject: RFR: 8246926: Clean up newlines and whitespaces in hs_err files In-Reply-To: <7208764e-c142-f6f0-ffb4-ea5ce1ba27e7@oracle.com> References: <69611584-2b78-4d82-319f-5f2fe66f2ecd@oracle.com> <0c6f9c87-640b-c5fa-d708-c5df9cd9c14f@oracle.com> <94163b93-0de4-0d38-c416-6e63dd51a641@oracle.com> <7208764e-c142-f6f0-ffb4-ea5ce1ba27e7@oracle.com> Message-ID: <82f5ef27-b4ae-5888-7641-9393649742a2@oracle.com> Thanks, David. StefanK On 2020-06-10 00:08, David Holmes wrote: > Test fix is good. > > David > > On 10/06/2020 5:03 am, Stefan Karlsson wrote: >> And a small test fix to match the new format: >> ??https://cr.openjdk.java.net/~stefank/8246926/webrev.03.delta/ >> ??https://cr.openjdk.java.net/~stefank/8246926/webrev.03/ >> >> StefanK >> >> On 2020-06-09 14:49, Stefan Karlsson wrote: >>> One small update. I'd like to add a new line between the heap >>> printing and G1's heap region printing: >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02.delta/ >>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.02 >>> >>> This now prints: >>> >>> Heap: >>> ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >>> 0x0000000800000000) >>> ? region size 16384K, 0 young (0K), 0 survivors (0K) >>> ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >>> 1056768K >>> ? class space??? used 4K, capacity 384K, committed 384K, reserved >>> 1048576K >>> >>> Heap Regions: E=young(eden), S=young(survivor), O=old, >>> HS=humongous(starts), HC=humongous(continues), CS=collection set, >>> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >>> (previous, next) >>> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| >>> 0%| F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >>> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| >>> 0%| F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >>> >>> Instead of: >>> >>> Heap: >>> ?garbage-first heap?? total 2064384K, used 0K [0x0000000082000000, >>> 0x0000000800000000) >>> ? region size 16384K, 0 young (0K), 0 survivors (0K) >>> ?Metaspace?????? used 4K, capacity 4480K, committed 4480K, reserved >>> 1056768K >>> ? class space??? used 4K, capacity 384K, committed 384K, reserved >>> 1048576K >>> Heap Regions: E=young(eden), S=young(survivor), O=old, >>> HS=humongous(starts), HC=humongous(continues), CS=collection set, >>> F=free, OA=open archive, CA=closed archive, TAMS=top-at-mark-start >>> (previous, next) >>> |?? 0|0x0000000082000000, 0x0000000082000000, 0x0000000083000000| >>> 0%| F|? |TAMS 0x0000000082000000, 0x0000000082000000| Untracked >>> |?? 1|0x0000000083000000, 0x0000000083000000, 0x0000000084000000| >>> 0%| F|? |TAMS 0x0000000083000000, 0x0000000083000000| Untracked >>> >>> Thanks, >>> StefanK >>> >>> On 2020-06-09 12:17, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to clean up newlines and whitespaces in >>>> our hs_err files. >>>> >>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/ >>>> ?https://bugs.openjdk.java.net/browse/JDK-8246926 >>>> >>>> I've uploaded before and after hs_err files here: >>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid146249.before.log >>>> >>>> ?https://cr.openjdk.java.net/~stefank/8246926/webrev.01/hs_err_pid196998.after.log >>>> >>>> >>>> Main cleanups: >>>> - Numerous one-liners where written as two lines: >>>> >>>> Old >>>> === >>>> Online cpus: >>>> 0-31 >>>> Offline cpus: >>>> >>>> cache level: >>>> 1 >>>> cache type: >>>> Data >>>> cache size: >>>> 32K >>>> cache coherency line size: >>>> 64 >>>> <... many more ...> >>>> >>>> New >>>> === >>>> Online cpus: 0-31 >>>> Offline cpus: >>>> cache level: 1 >>>> cache type: Data >>>> cache size: 32K >>>> cache coherency line size: 64 >>>> >>>> >>>> - Many lines missed a space after colon: >>>> >>>> Old >>>> === >>>> VM state:not at safepoint (not fully initialized) >>>> >>>> New >>>> === >>>> VM state: not at safepoint (not fully initialized) >>>> >>>> >>>> - Newline after OS >>>> >>>> The code looked for various files (on Linux) and dumped their >>>> contents. The first line ended up on the OS: line, but the rest >>>> ended afterwards >>>> >>>> Example from Linux: >>>> OS:DISTRIB_ID=Ubuntu >>>> DISTRIB_RELEASE=20.04 >>>> DISTRIB_CODENAME=focal >>>> DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS" >>>> uname:Linux >>>> >>>> Example from macOS: >>>> OS:uname:Darwin >>>> >>>> I couldn't find a quick and clean fix for this, so I simply opted >>>> to add a newline after "OS:". >>>> >>>> >>>> - There are a number of cr() calls (or ?\n" prints) that are used >>>> to add a complete blank line between sections. I moved a few of >>>> those out of the lower-level print functions, so that the caller >>>> could decide if there's a need for extra newlines. >>>> >>>> Thanks, >>>> StefanK >>> >> From leo.korinth at oracle.com Wed Jun 10 13:02:00 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 10 Jun 2020 15:02:00 +0200 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations Message-ID: Hi, I have created a RFR in the hotspot-gc-dev mailing list that mostly changes stuff in G1. However I am also doing changes in atomic.hpp (mostly adding an AtomicValue<> class) so there might be interest outside gc. If you have interest, please follow that thread. Thanks, Leo From igor.ignatyev at oracle.com Wed Jun 10 18:28:14 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 10 Jun 2020 11:28:14 -0700 Subject: RFR(S) : 8246387 : switch to jtreg 5.1 In-Reply-To: References: Message-ID: Erik, David, thank you for your reviews, pushed. Cheers, -- Igor > On Jun 9, 2020, at 7:06 PM, David Holmes wrote: > > Hi Igor, > > All of these changes seem okay to me. > > Thanks, > David > On Jun 9, 2020, at 9:10 AM, Erik Joelsson wrote: > > Looks good to me. > > /Erik > > On 10/06/2020 1:56 am, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8246387/webrev.00/ >>> 110 lines changed: 6 ins; 89 del; 15 mod; >> Hi all, >> could you please review the patch which switches jtreg used in jdk/jdk to the newly promoted jtreg5.1? as before, the patch changes 'requiredVersion' property in all jtreg test suites to have more consistent results. >> besides changing the version in jib-profiles and requiredVersion in all TEST.ROOT, the patch also >> - removes no-op methods from GatherDiagnosticInfoObserver. in jtharness 6.0-b11, and Harness.Observer::finishedTesting() method got deprecated, but instead of suppressing the warning, we can just remove it and other empty methods as Harness.Observer interface now provides no-op implementations for all its methods; >> - removes j.t.lib.Utils::getTestName method. this method has a (now) wrong assumption that testng classes are available only if we are executing a testng test. the method was used only by appcds tests to get a test specific filename, and these tests were updated to use a new supported way to get a testname via `test.name` system property; >> - updates test/hotspot/jtreg/runtime/condy/staticInit/TestInitException.java test to workaround CODETOOLS-7902686[1] . jasm from asmtools 7.0-b08 incorrectly sets full file path on windows as SourceFile, so the test was temporarily updated to use regexp which matches both full path and just basename. >> webrev: http://cr.openjdk.java.net/~iignatyev/8246387/webrev.00/ >> testing: tier1-4 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8246387 >> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902686 >> (fixed in asmtools7.0-b09, yet jtreg5.1 uses asmtools7.0-b08, so effectively can be fixed only w/ switch to a next version of jtreg) >> Thanks, >> -- Igor From alexey.menkov at oracle.com Wed Jun 10 19:20:52 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 10 Jun 2020 12:20:52 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> Message-ID: <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> Hi Igor, On 06/09/2020 20:11, Igor Ignatyev wrote: > Hi Alex, > > as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. Yes, the test still fails in the case, but if I see IndexOutOfBoundsException (or something similar) as a test failure reason, my first thought that this is the test issue. Could you please add the checks. --alex > > -- Igor > >> On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: >> >> Hi Igor, >> >> In SATestUtils.java you do >> >> var bb = ... Files.readAllBytes(...) ... >> and then use bb[0] >> >> if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. >> And looks like the caller doesn't expect it (it catches IOException). >> >> --alex >> >> On 06/09/2020 16:47, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>> >>>> 38 lines changed: 8 ins; 16 del; 14 mod; >>> Hi all, >>> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >>> testing: test/hotspot/jtreg/serviceability >>> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >>> Thanks, >>> -- Igor > From igor.ignatyev at oracle.com Wed Jun 10 19:29:30 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 10 Jun 2020 12:29:30 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> Message-ID: Hi Alex, sure, here is the incremental diff, so now Error is thrown if bb arrays are empty -- http://cr.openjdk.java.net/~iignatyev//8183040/webrev.0-1 Thanks, -- Igor > On Jun 10, 2020, at 12:20 PM, Alex Menkov wrote: > > Hi Igor, > > On 06/09/2020 20:11, Igor Ignatyev wrote: >> Hi Alex, >> as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. > > Yes, the test still fails in the case, but if I see IndexOutOfBoundsException (or something similar) as a test failure reason, my first thought that this is the test issue. > Could you please add the checks. > > --alex > >> -- Igor >>> On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: >>> >>> Hi Igor, >>> >>> In SATestUtils.java you do >>> >>> var bb = ... Files.readAllBytes(...) ... >>> and then use bb[0] >>> >>> if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. >>> And looks like the caller doesn't expect it (it catches IOException). >>> >>> --alex >>> >>> On 06/09/2020 16:47, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>> >>>>> 38 lines changed: 8 ins; 16 del; 14 mod; >>>> Hi all, >>>> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >>>> testing: test/hotspot/jtreg/serviceability >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >>>> Thanks, >>>> -- Igor From alexey.menkov at oracle.com Wed Jun 10 20:05:01 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 10 Jun 2020 13:05:01 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> Message-ID: <64f236d0-4ec1-8024-b3b9-e9f67141b4ad@oracle.com> Hi Igor, LGTM. --alex On 06/10/2020 12:29, Igor Ignatyev wrote: > Hi Alex, > > sure, here is the incremental diff, so now Error is thrown if bb arrays are empty -- http://cr.openjdk.java.net/~iignatyev//8183040/webrev.0-1 > > Thanks, > -- Igor > >> On Jun 10, 2020, at 12:20 PM, Alex Menkov wrote: >> >> Hi Igor, >> >> On 06/09/2020 20:11, Igor Ignatyev wrote: >>> Hi Alex, >>> as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. >> >> Yes, the test still fails in the case, but if I see IndexOutOfBoundsException (or something similar) as a test failure reason, my first thought that this is the test issue. >> Could you please add the checks. >> >> --alex >> >>> -- Igor >>>> On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: >>>> >>>> Hi Igor, >>>> >>>> In SATestUtils.java you do >>>> >>>> var bb = ... Files.readAllBytes(...) ... >>>> and then use bb[0] >>>> >>>> if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. >>>> And looks like the caller doesn't expect it (it catches IOException). >>>> >>>> --alex >>>> >>>> On 06/09/2020 16:47, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>>> >>>>>> 38 lines changed: 8 ins; 16 del; 14 mod; >>>>> Hi all, >>>>> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >>>>> testing: test/hotspot/jtreg/serviceability >>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >>>>> Thanks, >>>>> -- Igor > From igor.ignatyev at oracle.com Wed Jun 10 20:19:51 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 10 Jun 2020 13:19:51 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <64f236d0-4ec1-8024-b3b9-e9f67141b4ad@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> <64f236d0-4ec1-8024-b3b9-e9f67141b4ad@oracle.com> Message-ID: thanks Alex, pushed. -- Igor > On Jun 10, 2020, at 1:05 PM, Alex Menkov wrote: > > Hi Igor, > > LGTM. > > --alex > > On 06/10/2020 12:29, Igor Ignatyev wrote: >> Hi Alex, >> sure, here is the incremental diff, so now Error is thrown if bb arrays are empty -- http://cr.openjdk.java.net/~iignatyev//8183040/webrev.0-1 >> Thanks, >> -- Igor >>> On Jun 10, 2020, at 12:20 PM, Alex Menkov wrote: >>> >>> Hi Igor, >>> >>> On 06/09/2020 20:11, Igor Ignatyev wrote: >>>> Hi Alex, >>>> as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. >>> >>> Yes, the test still fails in the case, but if I see IndexOutOfBoundsException (or something similar) as a test failure reason, my first thought that this is the test issue. >>> Could you please add the checks. >>> >>> --alex >>> >>>> -- Igor >>>>> On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: >>>>> >>>>> Hi Igor, >>>>> >>>>> In SATestUtils.java you do >>>>> >>>>> var bb = ... Files.readAllBytes(...) ... >>>>> and then use bb[0] >>>>> >>>>> if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. >>>>> And looks like the caller doesn't expect it (it catches IOException). >>>>> >>>>> --alex >>>>> >>>>> On 06/09/2020 16:47, Igor Ignatyev wrote: >>>>>> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>>>> >>>>>>> 38 lines changed: 8 ins; 16 del; 14 mod; >>>>>> Hi all, >>>>>> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >>>>>> testing: test/hotspot/jtreg/serviceability >>>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >>>>>> Thanks, >>>>>> -- Igor From david.holmes at oracle.com Thu Jun 11 02:58:25 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 Jun 2020 12:58:25 +1000 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: References: Message-ID: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> Hi Leo, On 10/06/2020 11:02 pm, Leo Korinth wrote: > Hi, I have created a RFR in the hotspot-gc-dev mailing list that mostly > changes stuff in G1. However I am also doing changes in atomic.hpp > (mostly adding an AtomicValue<> class) so there might be interest > outside gc. If you have interest, please follow that thread. If you are changing atomic.hpp then please cross-post to hotspot-runtime-dev. Or even cross-post here on hotspot-dev as there should be a broad consensus on how to handle this in the VM. Thanks, David > Thanks, > Leo From patric.hedlin at oracle.com Thu Jun 11 08:24:17 2020 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Thu, 11 Jun 2020 10:24:17 +0200 Subject: RFR(S): 8247200: assert((unsigned)fpargs < 32) Message-ID: <40b781a4-5a1f-bd83-0dab-00dc2890e06d@oracle.com> Dear all, I would like to ask for help to review the following change/update: Issue:? https://bugs.openjdk.java.net/browse/JDK-8247200 Webrev: http://cr.openjdk.java.net/~phedlin/tr8247200/ Removing assert and some associated dead code. Testing: tier1-3,6 Best regards, Patric From leo.korinth at oracle.com Thu Jun 11 10:47:26 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 11 Jun 2020 12:47:26 +0200 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> Message-ID: On 11/06/2020 04:58, David Holmes wrote: > Hi Leo, > > On 10/06/2020 11:02 pm, Leo Korinth wrote: >> Hi, I have created a RFR in the hotspot-gc-dev mailing list that >> mostly changes stuff in G1. However I am also doing changes in >> atomic.hpp (mostly adding an AtomicValue<> class) so there might be >> interest outside gc. If you have interest, please follow that thread. > > If you are changing atomic.hpp then please cross-post to > hotspot-runtime-dev. Or even cross-post here on hotspot-dev as there > should be a broad consensus on how to handle this in the VM. That was what I tried to do by sending the email. I want to have the discussion in one place though, so I thought it was better announcing the RFR on hotspot-gc-dev on this mailing list than potentially creating two review threads by cross-posting. Next time I will cross post and tell people to reply on hotspot-gc-dev if that is preferable. Is it? I have attached the email below: Hi, could I have a review for this change that adds AtomicValue<> to atomic.hpp and uses it in G1? I am adding an AtomicValue<> to atomic.hpp. This is an opaque type (the "value" is private) and that protect against non-atomic operations being used by mistake. AtomicValue methods are proxied to the corresponding (all-static) Atomic:: methods. All operations are explicitly atomic and in a type safe manner with semantics defined in enum atomic_memory_order (memory_order_conservative by default). Instead of using field variables as volatile, I change them to be of AtomicValue<> type. No need to verify that += (for example) will result in an atomic instruction on all supported compilers. I have some open questions regarding the exported fields in vmStructs_g1.hpp. Today, volatile fields are sometimes "exported" as volatile (and sometimes not, and I guess this is by mistake). I choose to export them all as non-volatile. From what I can see the volatile specific part only casts to void* (only documentation?). Java code is unchanged and still access them as the unwrapped values (static assert in atomic.hpp guarantees that memory layout is the same for T and AtomicValue). I think this is okay, but would like feedback on all this. The change is presented as a two part change. The first part changes all volatile to AtomicValue, the other part removes the AtomicValue part on non-field accesses. By doing it two part I will not forget to transform some operations by mistake. Copyright years will be updated when all other changes are approved. How about pushing this after 15 is branched off and thus have it for 16? Enhancement: https://bugs.openjdk.java.net/browse/JDK-8247213 Webrev: http://cr.openjdk.java.net/~lkorinth/8247213/0/part1 http://cr.openjdk.java.net/~lkorinth/8247213/0/part2 http://cr.openjdk.java.net/~lkorinth/8247213/0/full Testing: tier1-3 Thanks, Leo > > Thanks, > David > >> Thanks, >> Leo From kim.barrett at oracle.com Thu Jun 11 12:12:41 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Jun 2020 08:12:41 -0400 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> Message-ID: <13764FCB-FAF5-4B77-B25C-57FC0D49537A@oracle.com> > On Jun 11, 2020, at 6:47 AM, Leo Korinth wrote: > > On 11/06/2020 04:58, David Holmes wrote: >> Hi Leo, >> On 10/06/2020 11:02 pm, Leo Korinth wrote: >>> Hi, I have created a RFR in the hotspot-gc-dev mailing list that mostly changes stuff in G1. However I am also doing changes in atomic.hpp (mostly adding an AtomicValue<> class) so there might be interest outside gc. If you have interest, please follow that thread. >> If you are changing atomic.hpp then please cross-post to hotspot-runtime-dev. Or even cross-post here on hotspot-dev as there should be a broad consensus on how to handle this in the VM. > > That was what I tried to do by sending the email. I want to have the discussion in one place though, so I thought it was better announcing the RFR on hotspot-gc-dev on this mailing list than potentially creating two review threads by cross-posting. > > Next time I will cross post and tell people to reply on hotspot-gc-dev if that is preferable. Is it? I have attached the email below: For cross-hotspot-component things, I think it?s better to just have the discussion on hotspot-dev rather than on multiple component lists that almost invariably don?t all get all of the messages because of the behavior of somebody?s email client. Having the discussion on the hotspot-gc-dev list is almost certainly wrong in at least this case, as I know there are people who would be interested who don?t subscribe to the gc-specific list. From leo.korinth at oracle.com Thu Jun 11 12:41:48 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 11 Jun 2020 14:41:48 +0200 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <13764FCB-FAF5-4B77-B25C-57FC0D49537A@oracle.com> References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> <13764FCB-FAF5-4B77-B25C-57FC0D49537A@oracle.com> Message-ID: <0a06cd90-4f3c-57ab-199e-af9a6cec6076@oracle.com> On 11/06/2020 14:12, Kim Barrett wrote: >> On Jun 11, 2020, at 6:47 AM, Leo Korinth wrote: >> >> On 11/06/2020 04:58, David Holmes wrote: >>> Hi Leo, >>> On 10/06/2020 11:02 pm, Leo Korinth wrote: >>>> Hi, I have created a RFR in the hotspot-gc-dev mailing list that mostly changes stuff in G1. However I am also doing changes in atomic.hpp (mostly adding an AtomicValue<> class) so there might be interest outside gc. If you have interest, please follow that thread. >>> If you are changing atomic.hpp then please cross-post to hotspot-runtime-dev. Or even cross-post here on hotspot-dev as there should be a broad consensus on how to handle this in the VM. >> >> That was what I tried to do by sending the email. I want to have the discussion in one place though, so I thought it was better announcing the RFR on hotspot-gc-dev on this mailing list than potentially creating two review threads by cross-posting. >> >> Next time I will cross post and tell people to reply on hotspot-gc-dev if that is preferable. Is it? I have attached the email below: > > For cross-hotspot-component things, I think it?s better to just have the discussion on hotspot-dev > rather than on multiple component lists that almost invariably don?t all get all of the messages > because of the behavior of somebody?s email client. > > Having the discussion on the hotspot-gc-dev list is almost certainly wrong in at least this case, > as I know there are people who would be interested who don?t subscribe to the gc-specific list. > Okay, lets continue the discussion here on hotspot-dev. Sorry for starting this badly. Thanks, Leo From david.holmes at oracle.com Thu Jun 11 13:34:11 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 Jun 2020 23:34:11 +1000 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> Message-ID: <0250fab1-c743-7077-0485-2d188609cb42@oracle.com> Hi Leo, On 11/06/2020 8:47 pm, Leo Korinth wrote: > On 11/06/2020 04:58, David Holmes wrote: >> Hi Leo, >> >> On 10/06/2020 11:02 pm, Leo Korinth wrote: >>> Hi, I have created a RFR in the hotspot-gc-dev mailing list that >>> mostly changes stuff in G1. However I am also doing changes in >>> atomic.hpp (mostly adding an AtomicValue<> class) so there might be >>> interest outside gc. If you have interest, please follow that thread. >> >> If you are changing atomic.hpp then please cross-post to >> hotspot-runtime-dev. Or even cross-post here on hotspot-dev as there >> should be a broad consensus on how to handle this in the VM. > > That was what I tried to do by sending the email. I want to have the > discussion in one place though, so I thought it was better announcing > the RFR on hotspot-gc-dev on this mailing list than potentially creating > two review threads by cross-posting. You can't expect all hotspot engineers to subscribe to hotspot-gc-dev to have this conversation. Either use a common list, like hotspot-dev (that we should all be subscribed to) or else cross-post. Thanks, David ----- > Next time I will cross post and tell people to reply on hotspot-gc-dev > if that is preferable. Is it? I have attached the email below: > > > Hi, could I have a review for this change that adds AtomicValue<> to > atomic.hpp and uses it in G1? > > I am adding an AtomicValue<> to atomic.hpp. This is an opaque type (the > "value" is private) and that protect against non-atomic operations being > used by mistake. AtomicValue methods are proxied to the corresponding > (all-static) Atomic:: methods. All operations are explicitly atomic and > in a type safe manner with semantics defined in enum atomic_memory_order > (memory_order_conservative by default). > > Instead of using field variables as volatile, I change them to be of > AtomicValue<> type. No need to verify that += (for example) will result > in an atomic instruction on all supported compilers. > > I have some open questions regarding the exported fields in > vmStructs_g1.hpp. Today, volatile fields are sometimes "exported" as > volatile (and sometimes not, and I guess this is by mistake). I choose > to export them all as non-volatile. From what I can see the volatile > specific part only casts to void* (only documentation?). Java code is > unchanged and still access them as the unwrapped values (static assert > in atomic.hpp guarantees that memory layout is the same for T and > AtomicValue). I think this is okay, but would like feedback on all this. > > The change is presented as a two part change. The first part changes all > volatile to AtomicValue, the other part removes the AtomicValue part on > non-field accesses. By doing it two part I will not forget to transform > some operations by mistake. > > Copyright years will be updated when all other changes are approved. > > How about pushing this after 15 is branched off and thus have it for 16? > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8247213 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8247213/0/part1 > http://cr.openjdk.java.net/~lkorinth/8247213/0/part2 > http://cr.openjdk.java.net/~lkorinth/8247213/0/full > > Testing: > ? tier1-3 > > Thanks, > Leo >> >> Thanks, >> David >> >>> Thanks, >>> Leo From leo.korinth at oracle.com Thu Jun 11 15:51:36 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 11 Jun 2020 17:51:36 +0200 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <0250fab1-c743-7077-0485-2d188609cb42@oracle.com> References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> <0250fab1-c743-7077-0485-2d188609cb42@oracle.com> Message-ID: On 11/06/2020 15:34, David Holmes wrote: > Hi Leo, > > On 11/06/2020 8:47 pm, Leo Korinth wrote: >> On 11/06/2020 04:58, David Holmes wrote: >>> Hi Leo, >>> >>> On 10/06/2020 11:02 pm, Leo Korinth wrote: >>>> Hi, I have created a RFR in the hotspot-gc-dev mailing list that >>>> mostly changes stuff in G1. However I am also doing changes in >>>> atomic.hpp (mostly adding an AtomicValue<> class) so there might be >>>> interest outside gc. If you have interest, please follow that thread. >>> >>> If you are changing atomic.hpp then please cross-post to >>> hotspot-runtime-dev. Or even cross-post here on hotspot-dev as there >>> should be a broad consensus on how to handle this in the VM. >> >> That was what I tried to do by sending the email. I want to have the >> discussion in one place though, so I thought it was better announcing >> the RFR on hotspot-gc-dev on this mailing list than potentially >> creating two review threads by cross-posting. > > You can't expect all hotspot engineers to subscribe to hotspot-gc-dev to > have this conversation. Either use a common list, like hotspot-dev (that > we should all be subscribed to) or else cross-post. I realize that now --- sorry --- please continue here on hotspot-dev. I closed the thread on hotspot-gc-dev. /Leo > > Thanks, > David > ----- > >> Next time I will cross post and tell people to reply on hotspot-gc-dev >> if that is preferable. Is it? I have attached the email below: >> >> >> Hi, could I have a review for this change that adds AtomicValue<> to >> atomic.hpp and uses it in G1? >> >> I am adding an AtomicValue<> to atomic.hpp. This is an opaque type >> (the "value" is private) and that protect against non-atomic >> operations being used by mistake. AtomicValue methods are proxied to >> the corresponding (all-static) Atomic:: methods. All operations are >> explicitly atomic and in a type safe manner with semantics defined in >> enum atomic_memory_order >> (memory_order_conservative by default). >> >> Instead of using field variables as volatile, I change them to be of >> AtomicValue<> type. No need to verify that += (for example) will >> result in an atomic instruction on all supported compilers. >> >> I have some open questions regarding the exported fields in >> vmStructs_g1.hpp. Today, volatile fields are sometimes "exported" as >> volatile (and sometimes not, and I guess this is by mistake). I choose >> to export them all as non-volatile. From what I can see the volatile >> specific part only casts to void* (only documentation?). Java code is >> unchanged and still access them as the unwrapped values (static assert >> in atomic.hpp guarantees that memory layout is the same for T and >> AtomicValue). I think this is okay, but would like feedback on all >> this. >> >> The change is presented as a two part change. The first part changes >> all volatile to AtomicValue, the other part removes the AtomicValue >> part on non-field accesses. By doing it two part I will not forget to >> transform some operations by mistake. >> >> Copyright years will be updated when all other changes are approved. >> >> How about pushing this after 15 is branched off and thus have it for 16? >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8247213 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8247213/0/part1 >> http://cr.openjdk.java.net/~lkorinth/8247213/0/part2 >> http://cr.openjdk.java.net/~lkorinth/8247213/0/full >> >> Testing: >> ?? tier1-3 >> >> Thanks, >> Leo >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Leo From vladimir.kozlov at oracle.com Thu Jun 11 16:56:23 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 11 Jun 2020 09:56:23 -0700 Subject: RFR(S): 8247200: [aarch64] assert((unsigned)fpargs < 32) In-Reply-To: <40b781a4-5a1f-bd83-0dab-00dc2890e06d@oracle.com> References: <40b781a4-5a1f-bd83-0dab-00dc2890e06d@oracle.com> Message-ID: CCing to aarch64 mailing list. Vladimir On 6/11/20 1:24 AM, Patric Hedlin wrote: > Dear all, > > I would like to ask for help to review the following change/update: > > Issue:? https://bugs.openjdk.java.net/browse/JDK-8247200 > Webrev: http://cr.openjdk.java.net/~phedlin/tr8247200/ > > > Removing assert and some associated dead code. > > > Testing: tier1-3,6 > > > Best regards, > Patric From erik.osterlund at oracle.com Fri Jun 12 07:29:16 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 12 Jun 2020 09:29:16 +0200 Subject: [16] RFR: 8247319: Remove on-stack nmethod hotness counter sampling from safepoints Message-ID: <8feb88a7-6bab-f694-cf24-2a83c3461f33@oracle.com> Hi, The sweeper is moving away from using safepoints for its heuristics. It used to count safepoints to figure out when to sweep, but no longer does that. At the same time, we have for a while been removing more and more safepoints. Safepoints are becoming increasingly rare events, dominated by when we need to GC (GuaranteedSafepointInterval is going to disappear). The frequency of how often we need to GC does not have an obvious connection to how often we need to sweep the code cache... any more. What still remains from the safepoint-based heuristics is the nmethod hotness counter sampling that is performed in safepoint cleanup. I would like to get rid of this. The rationale is that the use of hotness counters is kicking in when the code cache is starting to fill up quite a bit, and there is a need to kill off nmethods heuristically, rather than because they are invalid. But when the code cache fills up, we sweep more and more aggressively. And during these sweeper cycles, we perform nmethod marking using handshakes. That operation also fills in hotness counters for all sampled nmethods. In other words, when there is need for acting on the hotness counters, we are in a state where we may be getting more nmethod hotness counter sample information from the sweeping cycles than we are from safepoint sampling. Conversely, when code cache pressure is high and we need more samples, we might end up getting very few from safepoint based sampling (because the heap is large). The correlation between safepoint frequency and code cache pressure is simply not there any more. And for us to walk all stacks in the system in every single safepoint (which for ZGC is starting to dominate pauses when I remove our stack sampling from safepoints), there better be a really good reason to do this sampling in safepoints. And there simply isn't. So I propose we delete it, in favour of using the hotness counter samples we get from the sweeping cycles instead, that are indeed proportional in frequency, to the code cache pressure. Bug: https://bugs.openjdk.java.net/browse/JDK-8247319 Webrev: http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00/ Thanks, /Erik From erik.osterlund at oracle.com Fri Jun 12 07:37:31 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 12 Jun 2020 09:37:31 +0200 Subject: [16] RFR: 8247319: Remove on-stack nmethod hotness counter sampling from safepoints In-Reply-To: <7bedf284-c070-a091-cce2-fc00e2047440@oracle.com> References: <8feb88a7-6bab-f694-cf24-2a83c3461f33@oracle.com> <7bedf284-c070-a091-cce2-fc00e2047440@oracle.com> Message-ID: Hi David, Thanks! :) /Erik On 2020-06-12 09:36, David Holmes wrote: > Hi Erik, > > I can't comment on your rationale as I know nothing about how the > hotness counters operate, but I can verify that what you propose > indeed removes the processing from the safepoint cleanup logic. :) So > in that limited sense LGTM. > > Cheers, > David > ----- > > On 12/06/2020 5:29 pm, Erik ?sterlund wrote: >> Hi, >> >> The sweeper is moving away from using safepoints for its heuristics. >> It used to count safepoints to figure out when to sweep, but no >> longer does that. At the same time, we have for a while been removing >> more and more safepoints. Safepoints are becoming increasingly rare >> events, dominated by when we need to GC (GuaranteedSafepointInterval >> is going to disappear). The frequency of how often we need to GC does >> not have an obvious connection to how often we need to sweep the code >> cache... any more. >> >> What still remains from the safepoint-based heuristics is the nmethod >> hotness counter sampling that is performed in safepoint cleanup. I >> would like to get rid of this. >> The rationale is that the use of hotness counters is kicking in when >> the code cache is starting to fill up quite a bit, and there is a >> need to kill off nmethods heuristically, rather than because they are >> invalid. But when the code cache fills up, we sweep more and more >> aggressively. And during these sweeper cycles, we perform nmethod >> marking using handshakes. That operation also fills in hotness >> counters for all sampled nmethods. >> >> In other words, when there is need for acting on the hotness >> counters, we are in a state where we may be getting more nmethod >> hotness counter sample information from the sweeping cycles than we >> are from safepoint sampling. Conversely, when code cache pressure is >> high and we need more samples, we might end up getting very few from >> safepoint based sampling (because the heap is large). The correlation >> between safepoint frequency and code cache pressure is simply not >> there any more. And for us to walk all stacks in the system in every >> single safepoint (which for ZGC is starting to dominate pauses when I >> remove our stack sampling from safepoints), there better be a really >> good reason to do this sampling in safepoints. And there simply >> isn't. So I propose we delete it, in favour of using the hotness >> counter samples we get from the sweeping cycles instead, that are >> indeed proportional in frequency, to the code cache pressure. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8247319 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00/ >> >> Thanks, >> /Erik From david.holmes at oracle.com Fri Jun 12 07:36:29 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 12 Jun 2020 17:36:29 +1000 Subject: [16] RFR: 8247319: Remove on-stack nmethod hotness counter sampling from safepoints In-Reply-To: <8feb88a7-6bab-f694-cf24-2a83c3461f33@oracle.com> References: <8feb88a7-6bab-f694-cf24-2a83c3461f33@oracle.com> Message-ID: <7bedf284-c070-a091-cce2-fc00e2047440@oracle.com> Hi Erik, I can't comment on your rationale as I know nothing about how the hotness counters operate, but I can verify that what you propose indeed removes the processing from the safepoint cleanup logic. :) So in that limited sense LGTM. Cheers, David ----- On 12/06/2020 5:29 pm, Erik ?sterlund wrote: > Hi, > > The sweeper is moving away from using safepoints for its heuristics. It > used to count safepoints to figure out when to sweep, but no longer does > that. At the same time, we have for a while been removing more and more > safepoints. Safepoints are becoming increasingly rare events, dominated > by when we need to GC (GuaranteedSafepointInterval is going to > disappear). The frequency of how often we need to GC does not have an > obvious connection to how often we need to sweep the code cache... any > more. > > What still remains from the safepoint-based heuristics is the nmethod > hotness counter sampling that is performed in safepoint cleanup. I would > like to get rid of this. > The rationale is that the use of hotness counters is kicking in when the > code cache is starting to fill up quite a bit, and there is a need to > kill off nmethods heuristically, rather than because they are invalid. > But when the code cache fills up, we sweep more and more aggressively. > And during these sweeper cycles, we perform nmethod marking using > handshakes. That operation also fills in hotness counters for all > sampled nmethods. > > In other words, when there is need for acting on the hotness counters, > we are in a state where we may be getting more nmethod hotness counter > sample information from the sweeping cycles than we are from safepoint > sampling. Conversely, when code cache pressure is high and we need more > samples, we might end up getting very few from safepoint based sampling > (because the heap is large). The correlation between safepoint frequency > and code cache pressure is simply not there any more. And for us to walk > all stacks in the system in every single safepoint (which for ZGC is > starting to dominate pauses when I remove our stack sampling from > safepoints), there better be a really good reason to do this sampling in > safepoints. And there simply isn't. So I propose we delete it, in favour > of using the hotness counter samples we get from the sweeping cycles > instead, that are indeed proportional in frequency, to the code cache > pressure. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8247319 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00/ > > Thanks, > /Erik From stefan.karlsson at oracle.com Fri Jun 12 09:33:48 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 12 Jun 2020 11:33:48 +0200 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting Message-ID: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> Hi all, Please review this patch to fix the NMT accounting of CHeap GrowableArray allocations. https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8247470 The initial reason why I started fixing this was that I found code like this: src/hotspot/share/classfile/javaClasses.cpp: ??? new (ResourceObj::C_HEAP, mtModule) GrowableArray(500, true); Before this patch, the GrowableArray constructor looked like this: GrowableArray(int initial_size, bool C_heap = false, MEMFLAGS F = mtInternal) : GenericGrowableArray(initial_size, 0, C_heap, F) { _data = (E*)raw_allocate(sizeof(E)); and: GenericGrowableArray(int initial_size, int initial_len, bool c_heap, MEMFLAGS flags = mtNone) { _len = initial_len; _max = initial_size; _memflags = flags; and: void* GenericGrowableArray::raw_allocate(int elementSize) { ... return (void*)AllocateHeap(byte_size, _memflags); This means that the GrowableArray instance was accounted to mtModule, but the backing memory for the array was accounted to mtInternal. My proposal for fixing this is to fuse the (c_heap, flags) duality into one parameter (flags). The call sites are then changed like this: For resource allocations: GrowableArray(...) -> GrowableArray(...) GrowableArray(... /* c_heap */ false) -> GrowableArray(...) GrowableArray(... /* c_heap */ false, /* flags */ mtXX) -> GrowableArray(...) For CHeap allocations: GrowableArray(... /* c_heap */ true, /* flags */ mtXX) -> GrowableArray(..., mtXX) GrowableArray(... /* c_heap */ true) -> GrowableArray(..., mtInternal) That is, if you specify a "valid" mtXX tag it invokes the CHeap allocation path. If you don't specify a mtXX tag, then 'flags' get the mtNone default value. This is a marker for the code that no CHeap allocation should be done. While properly tagging all GrowableArray allocations, I went ahead and changed most GrowableArray mtInternal tags to the appropriate values. I also introduced a new mtServiceability tag, since a lot of the mtInternal tags were in areas dealing with serviceability code. I could split this out into a separate patch, if necessary. This hasn't been fully tested yet, because I'd like to get some early feedback. Thanks, StefanK From magnus.ihse.bursie at oracle.com Fri Jun 12 12:21:56 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 12 Jun 2020 14:21:56 +0200 Subject: RFR: JDK-8150828 Consider using '-fno-asynchronous-unwind-tables' to reduce the size of libjvm.so by 10 percent Message-ID: <499b6643-09df-89a3-a790-58839f42a5a9@oracle.com> From Volker's bug report: "We are building and linking the libjvm.so on Linux with -fnoexceptions because we currently don't use C++ exception handling in the HotSpot. Nevertheless, g++ generates unwind tables (i.e. .eh_frame sections) in the object files and shared libraries which can not be stripped from the binary. In the case of libjvm.so, these sections consume 10% of the whole library. It is possible to omit the creation of these sections by using the '-fno-asynchronous-unwind-tables' option during compilation and linking. Ive verified that this indeed reduces the size of libjvm.so by 10% on Linux/x86_64 for a product build: -rwxrwxr-x 1 simonis simonis 18798859 Feb 24 18:32 hotspot/linux_amd64_compiler2/product/libjvm.so -rwxrwxr-x 1 simonis simonis 17049867 Feb 25 18:12 hotspot_no_unwind/linux_amd64_compiler2/product/libjvm.so The gcc documentation mentions that the unwind information is used "for stack unwinding from asynchronous events (such as debugger or garbage collector)". But various references [1,2] also mention that using '-fno-asynchronous-unwind-tables' together with '-g' will force gcc to create this information in the debug sections of the object files (i.e. .debug_frame) which can easily be stripped from the object files and libraries. As we build the product version of the libjvm.so with '-g' anyway, I'd suggest to use '-fno-asynchronous-unwind-tables' to reduce its size. I've done some quick tests (debugging, creation of hs_err files) with a product version of libjvm.so which was build with '-fno-asynchronous-unwind-tables' and couldn't find any draw backs. I could observe that all the date from the current .eh_frame sections has bee moved to the .debug_frame sections in the stripped out data of the libjvm.debuginfo file." The patch itself is trivial, see below. Hotspot folks: Are there any reasons why we should not do it? I've waited for JDK 16 to push this; if something unexpected? turns up during the development of JDK 16 (if anything, it's odd corner cases that might be a problem), we can always revert this. Bug: https://bugs.openjdk.java.net/browse/JDK-8150828 Patch inline: diff --git a/make/autoconf/flags-cflags.m4 b/make/autoconf/flags-cflags.m4 --- a/make/autoconf/flags-cflags.m4 +++ b/make/autoconf/flags-cflags.m4 @@ -442,7 +442,8 @@ ?? if test "x$TOOLCHAIN_TYPE" = xgcc || test "x$TOOLCHAIN_TYPE" = xclang; then ???? # COMMON to gcc and clang ???? TOOLCHAIN_CFLAGS_JVM="-pipe -fno-rtti -fno-exceptions \ -??????? -fvisibility=hidden -fno-strict-aliasing -fno-omit-frame-pointer" +??????? -fvisibility=hidden -fno-strict-aliasing -fno-omit-frame-pointer \ +??????? -fno-asynchronous-unwind-tables" ?? fi ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then /Magnus From erik.joelsson at oracle.com Fri Jun 12 13:09:44 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 12 Jun 2020 06:09:44 -0700 Subject: RFR: JDK-8150828 Consider using '-fno-asynchronous-unwind-tables' to reduce the size of libjvm.so by 10 percent In-Reply-To: <499b6643-09df-89a3-a790-58839f42a5a9@oracle.com> References: <499b6643-09df-89a3-a790-58839f42a5a9@oracle.com> Message-ID: Looks good to me at least. /Erik On 2020-06-12 05:21, Magnus Ihse Bursie wrote: > From Volker's bug report: > > "We are building and linking the libjvm.so on Linux with > -fnoexceptions because we currently don't use C++ exception handling > in the HotSpot. > > Nevertheless, g++ generates unwind tables (i.e. .eh_frame sections) in > the object files and shared libraries which can not be stripped from > the binary. In the case of libjvm.so, these sections consume 10% of > the whole library. > > It is possible to omit the creation of these sections by using the > '-fno-asynchronous-unwind-tables' option during compilation and > linking. Ive verified that this indeed reduces the size of libjvm.so > by 10% on Linux/x86_64 for a product build: > > -rwxrwxr-x 1 simonis simonis 18798859 Feb 24 18:32 > hotspot/linux_amd64_compiler2/product/libjvm.so > -rwxrwxr-x 1 simonis simonis 17049867 Feb 25 18:12 > hotspot_no_unwind/linux_amd64_compiler2/product/libjvm.so > > The gcc documentation mentions that the unwind information is used > "for stack unwinding from asynchronous events (such as debugger or > garbage collector)". But various references [1,2] also mention that > using '-fno-asynchronous-unwind-tables' together with '-g' will force > gcc to create this information in the debug sections of the object > files (i.e. .debug_frame) which can easily be stripped from the object > files and libraries. > > As we build the product version of the libjvm.so with '-g' anyway, I'd > suggest to use '-fno-asynchronous-unwind-tables' to reduce its size. > > I've done some quick tests (debugging, creation of hs_err files) with > a product version of libjvm.so which was build with > '-fno-asynchronous-unwind-tables' and couldn't find any draw backs. I > could observe that all the date from the current .eh_frame sections > has bee moved to the .debug_frame sections in the stripped out data of > the libjvm.debuginfo file." > > The patch itself is trivial, see below. > > Hotspot folks: Are there any reasons why we should not do it? I've > waited for JDK 16 to push this; if something unexpected? turns up > during the development of JDK 16 (if anything, it's odd corner cases > that might be a problem), we can always revert this. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8150828 > Patch inline: > diff --git a/make/autoconf/flags-cflags.m4 > b/make/autoconf/flags-cflags.m4 > --- a/make/autoconf/flags-cflags.m4 > +++ b/make/autoconf/flags-cflags.m4 > @@ -442,7 +442,8 @@ > ?? if test "x$TOOLCHAIN_TYPE" = xgcc || test "x$TOOLCHAIN_TYPE" = > xclang; then > ???? # COMMON to gcc and clang > ???? TOOLCHAIN_CFLAGS_JVM="-pipe -fno-rtti -fno-exceptions \ > -??????? -fvisibility=hidden -fno-strict-aliasing > -fno-omit-frame-pointer" > +??????? -fvisibility=hidden -fno-strict-aliasing > -fno-omit-frame-pointer \ > +??????? -fno-asynchronous-unwind-tables" > ?? fi > > ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then > > /Magnus From matthias.baesken at sap.com Fri Jun 12 12:03:46 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 12 Jun 2020 12:03:46 +0000 Subject: RFR: 8247466: AIX link error undefined MetaspaceShared::_use_optimized_module_handling Message-ID: Hello, please review this AIX (and cds) related build fix. It seems that recent changes started to reference MetaspaceShared:: from other files, but metaspaceShared.cpp is excluded in the AIX build (because it is has cds disabled). Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8247466 http://cr.openjdk.java.net/~mbaesken/webrevs/8247466.0/ Thanks, Matthias From vladimir.kozlov at ORACLE.COM Fri Jun 12 19:35:49 2020 From: vladimir.kozlov at ORACLE.COM (Vladimir Kozlov) Date: Fri, 12 Jun 2020 12:35:49 -0700 Subject: [16] RFR: 8247319: Remove on-stack nmethod hotness counter sampling from safepoints In-Reply-To: <8feb88a7-6bab-f694-cf24-2a83c3461f33@oracle.com> References: <8feb88a7-6bab-f694-cf24-2a83c3461f33@oracle.com> Message-ID: <11ecfe57-9142-93e4-d566-57effe29eb84@oracle.com> I agree with changes. Please also update comments [1] and clean up leftover [2] from 8244658 [3]. Thanks, Vladimir [1] http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/runtime/sweeper.hpp#l38 http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/code/nmethod.hpp#l260 [2] http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/runtime/vmOperations.hpp#l113 [3] https://bugs.openjdk.java.net/browse/JDK-8244658 On 6/12/20 12:29 AM, Erik ?sterlund wrote: > Hi, > > The sweeper is moving away from using safepoints for its heuristics. It used to count safepoints to figure out when to > sweep, but no longer does that. At the same time, we have for a while been removing more and more safepoints. Safepoints > are becoming increasingly rare events, dominated by when we need to GC (GuaranteedSafepointInterval is going to > disappear). The frequency of how often we need to GC does not have an obvious connection to how often we need to sweep > the code cache... any more. > > What still remains from the safepoint-based heuristics is the nmethod hotness counter sampling that is performed in > safepoint cleanup. I would like to get rid of this. > The rationale is that the use of hotness counters is kicking in when the code cache is starting to fill up quite a bit, > and there is a need to kill off nmethods heuristically, rather than because they are invalid. But when the code cache > fills up, we sweep more and more aggressively. And during these sweeper cycles, we perform nmethod marking using > handshakes. That operation also fills in hotness counters for all sampled nmethods. > > In other words, when there is need for acting on the hotness counters, we are in a state where we may be getting more > nmethod hotness counter sample information from the sweeping cycles than we are from safepoint sampling. Conversely, > when code cache pressure is high and we need more samples, we might end up getting very few from safepoint based > sampling (because the heap is large). The correlation between safepoint frequency and code cache pressure is simply not > there any more. And for us to walk all stacks in the system in every single safepoint (which for ZGC is starting to > dominate pauses when I remove our stack sampling from safepoints), there better be a really good reason to do this > sampling in safepoints. And there simply isn't. So I propose we delete it, in favour of using the hotness counter > samples we get from the sweeping cycles instead, that are indeed proportional in frequency, to the code cache pressure. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8247319 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00/ > > Thanks, > /Erik From kim.barrett at oracle.com Sat Jun 13 03:06:18 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 12 Jun 2020 23:06:18 -0400 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> Message-ID: > On Jun 12, 2020, at 5:33 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this patch to fix the NMT accounting of CHeap GrowableArray allocations. > > https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247470 The GrowableArray changes look good to me. I?ve only spot-checked some of the usage changes, but didn?t find any problems. From igor.ignatyev at oracle.com Sat Jun 13 03:36:27 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 12 Jun 2020 20:36:27 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place Message-ID: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ > 796 lines changed: 200 ins; 588 del; 8 mod; Hi all, could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also - problem lists HexPrinterTest.java on windows due to JDK-8247521 - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 testing: test/lib-test on {windows,linux,macosx}-x64 Thanks, -- Igor From igor.ignatyev at oracle.com Sat Jun 13 03:38:27 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 12 Jun 2020 20:38:27 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: adding build-dev > On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >> 796 lines changed: 200 ins; 588 del; 8 mod; > > Hi all, > > could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? > > besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also > - problem lists HexPrinterTest.java on windows due to JDK-8247521 > - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) > - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) > - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder > - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag > - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ > - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) > > webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ > JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 > testing: test/lib-test on {windows,linux,macosx}-x64 > > Thanks, > -- Igor > > From igor.ignatyev at oracle.com Sat Jun 13 04:10:32 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 12 Jun 2020 21:10:32 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: testing revealed that LingeredAppTest.java required some love, incremental webrev w/ the fixes for LingeredAppTest -- http://cr.openjdk.java.net/~iignatyev//8211977/webrev.0-1 -- Igor > On Jun 12, 2020, at 8:38 PM, Igor Ignatyev wrote: > > adding build-dev > >> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>> 796 lines changed: 200 ins; 588 del; 8 mod; >> >> Hi all, >> >> could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? >> >> besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also >> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >> - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) >> - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) >> - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder >> - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag >> - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ >> - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >> testing: test/lib-test on {windows,linux,macosx}-x64 >> >> Thanks, >> -- Igor >> >> > From coleen.phillimore at oracle.com Sat Jun 13 16:01:37 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Sat, 13 Jun 2020 12:01:37 -0400 Subject: RFR: 8247466: AIX link error undefined MetaspaceShared::_use_optimized_module_handling In-Reply-To: References: Message-ID: <1f5612b7-dc8b-69d6-0449-618344ba558c@oracle.com> Hi this change seems to overlap the change for 8247388: Minimal build broken after JDK-8240245 (undefined reference to `MetaspaceShared::_use_optimized_module_handling') That Ao Ki is pushing. I think this change and Ao's should be pushed to the jdk/jdk15 stabilization repository. This change looks okay also, but we really should refactor how INCLUDE_CDS is included.? It seems you shouldn't compile filemap.cpp if CDS isn't included.? Not in this change though. Coleen On 6/12/20 8:03 AM, Baesken, Matthias wrote: > Hello, please review this AIX (and cds) related build fix. > > It seems that recent changes started to reference MetaspaceShared:: from other files, but metaspaceShared.cpp is excluded in the AIX build > (because it is has cds disabled). > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8247466 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8247466.0/ > > Thanks, Matthias From ioi.lam at oracle.com Sun Jun 14 02:14:01 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 13 Jun 2020 19:14:01 -0700 Subject: RFR: 8247466: AIX link error undefined MetaspaceShared::_use_optimized_module_handling In-Reply-To: References: Message-ID: Hi Matthias, I think this should be fixed by JDK-8247388, which has been integrated into the jdk15 repo (and should be synced to jdk/jdk soon). http://hg.openjdk.java.net/jdk/jdk15/rev/a7ad60e19730 It has the same changes in arguments.cpp as in your version. I think your changes in systemDictionary.cpp is not necessary, because the function is already enclosed in INCLUDE_CDS. Also, the change in filemap.cpp is not necessary because this file is omitted when CDS is not enabled. If JDK-8247388 indeed fixes the AIX build, please close JDK-8247466 as a duplicate. Thanks - Ioi On 6/12/20 5:03 AM, Baesken, Matthias wrote: > Hello, please review this AIX (and cds) related build fix. > > It seems that recent changes started to reference MetaspaceShared:: from other files, but metaspaceShared.cpp is excluded in the AIX build > (because it is has cds disabled). > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8247466 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8247466.0/ > > Thanks, Matthias From kim.barrett at oracle.com Sun Jun 14 03:48:01 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 13 Jun 2020 23:48:01 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> Message-ID: > On Jun 10, 2020, at 2:26 AM, Kim Barrett wrote: > >> On Jun 8, 2020, at 4:07 PM, Philip Race wrote: > >> BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says >>> For Solaris: Add the -std=c++14 compiler option. ..... >> >> So I am sure there is still some room to update the JEP :-) > > Yeah, I have some updates to make. I also need to do a bit of work on the feature lists. I think the JEP updates are done now. From philip.race at oracle.com Sun Jun 14 04:45:37 2020 From: philip.race at oracle.com (Philip Race) Date: Sat, 13 Jun 2020 21:45:37 -0700 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> Message-ID: <5EE5AB71.20505@oracle.com> Kim, Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. Please rename the JEP. -phil. On 6/13/20, 8:48 PM, Kim Barrett wrote: >> On Jun 10, 2020, at 2:26 AM, Kim Barrett wrote: >> >>> On Jun 8, 2020, at 4:07 PM, Philip Race wrote: >>> BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says >>>> For Solaris: Add the -std=c++14 compiler option. ..... >>> So I am sure there is still some room to update the JEP :-) >> Yeah, I have some updates to make. I also need to do a bit of work on the feature lists. > I think the JEP updates are done now. > From kim.barrett at oracle.com Sun Jun 14 08:51:09 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 14 Jun 2020 04:51:09 -0400 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <4611e386-2d3f-efaa-fdce-920c7d3d5b85@oracle.com> References: <4611e386-2d3f-efaa-fdce-920c7d3d5b85@oracle.com> Message-ID: <7DE4D1D2-ECA8-4980-B665-781BB0E36B2F@oracle.com> > On Jun 10, 2020, at 8:46 AM, Leo Korinth wrote: > > Hi, could I have a review for this change that adds AtomicValue<> to atomic.hpp and uses it in G1? > > I am adding an AtomicValue<> to atomic.hpp. This is an opaque type (the "value" is private) and that protect against non-atomic operations being used by mistake. AtomicValue methods are proxied to the corresponding (all-static) Atomic:: methods. All operations are explicitly atomic and in a type safe manner with semantics defined in enum atomic_memory_order > (memory_order_conservative by default). > > Instead of using field variables as volatile, I change them to be of AtomicValue<> type. No need to verify that += (for example) will result in an atomic instruction on all supported compilers. > > I have some open questions regarding the exported fields in vmStructs_g1.hpp. Today, volatile fields are sometimes "exported" as volatile (and sometimes not, and I guess this is by mistake). I choose to export them all as non-volatile. From what I can see the volatile specific part only casts to void* (only documentation?). Java code is unchanged and still access them as the unwrapped values (static assert in atomic.hpp guarantees that memory layout is the same for T and AtomicValue). I think this is okay, but would like feedback on all this. > > The change is presented as a two part change. The first part changes all volatile to AtomicValue, the other part removes the AtomicValue part on non-field accesses. By doing it two part I will not forget to transform some operations by mistake. > > Copyright years will be updated when all other changes are approved. > > How about pushing this after 15 is branched off and thus have it for 16? > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8247213 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8247213/0/part1 > http://cr.openjdk.java.net/~lkorinth/8247213/0/part2 > http://cr.openjdk.java.net/~lkorinth/8247213/0/full > > Testing: > tier1-3 > > Thanks, > Leo I think having a class for atomic members and variables is good, using the compiler to help keep us honest about what we're doing. So I generally like this change, despite the number of comments below. First a few general comments, then a bunch of detailed comments on specific changes. ------------------------------------------------------------------------------ I kind of wish this had been broken up into smaller chunks. Looking at a large number of changes involving atomic operations all in one go is pretty tiring. ------------------------------------------------------------------------------ I hope that this will not be a G1-specific usage, nor even a GC-specific usage, but rather will eventually be used throughout HotSpot. If not, then I'm not sure this change should be made; I wouldn't like having the differing styles of volatile members in some code and AtomicValue in other code (other than as a temporary state as changes are incrementally made). (That doesn't mean that I think all atomic operations should involve AtomicValue, and the static Atomic functions are no longer needed. I think there are some places where forcing things into the former mold is problematic and the Atomic functions will still have direct uses.) ------------------------------------------------------------------------------ The name of the class has generated some discussion. (Full disclosure: I suggested AtomicValue.) "AtomicValue" follows the style guide of using CamelCase for type names. But there are lots of counter-examples. "Atomic" is a bit strange, since then we'd have a class that is both a thin wrapper with member functions and an AllStatic with similarly named static member functions taking an additional explicit argument. The alternative would be to give this class the "Atomic" name and rename the existing "Atomic", but that would be pretty disruptive. atomic is short and aligns nicely with Atomic, but names that differ only in case may be considered a problem. Brevity doesn't seem super critical here; the type name is mostly used as the type for data members and global variables, and rarely appears in function signatures. ------------------------------------------------------------------------------ There are a number of places where plain arithmetic was being performed on relevant variables (mostly pre/post increment/decrement). I think most of these were intentionally not using Atomic arithmetic, because they are in single-threaded (mutex or safepoint) contexts. This change currently transforms those all into atomic inc/add/&etc. I've been waffling about this, and have noted occurrences in the detailed comments below. There is a (probably often negligable, though maybe not always) performance cost to that transformation, but that's not what concerns me. When I see an atomic operation, I expect there's something interesting happening and ordering or atomicity is important. Using atomic operations unnecessarily breaks that expectation, making the code less clear. That doesn't mean we should use pre/post increment/decrement operators and such for "non-atomic" uses though; we could have other names that are more explicit. (Note that atomic arithmetic operations with relaxed memory order are not the same thing as non-atomic arithmetic operations.) Also note that during review I found one place where I think the use of non-atomic arithmetic may have been a mistake, in which case the change to atomic arithmetic would be correct and necessary. ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 42 #include "utilities/debug.hpp" Out of order. ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 434 template 435 struct Atomic::StoreImpl< 436 D, T, 437 PlatformOp, 438 typename EnableIf<(IsIntegral::value || IsRegisteredEnum::value) && (IsIntegral::value || IsRegisteredEnum::value)>::type> ... 442 D value = new_value; Allowing implicit conversions here was discussed and rejected during the templatization of Atomic. While I think allowing some conversions here would be good, I think this is much too loose. I think a change of this type should be dealt with separately. Also, the line length for the EnableIf line seems rather excessive to me. ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 943 AtomicValue(const AtomicValue& a); // = delete I think copy-assign should also be forbidden. Why not use NONCOPYABLE? ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 946 STATIC_ASSERT(sizeof(AtomicValue) == sizeof(D)); // ensure the same memory layout I think there are more requirements than same-size. But maybe they fall out of usage, since the Atomic operations will provide the additional requirements. ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 949 inline void inc(atomic_memory_order order=memory_order_conservative) { I think there should be whitespace around "=". ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 940 class AtomicValue { I think I'd like the order of the load/store operations rearranged: load, load_acquire store, release_store, release_store_fence I also think I'd like inc/dec near add/sub. ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 1007 template 1008 static AtomicValue* cast_to_atomic_ptr(T* nonatomic); Declared but not defined. ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 1011 template 1012 static AtomicValue* cast_to_atomic_ptr(T* nonatomic) { I would really like to not have this. I'd like to see how far we can get without it. ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 940 class AtomicValue { Should it be default constructible? If so, should the default constructor guarantee value initialization? I think having a trivial default constructor is needed to avoid UB in some uses. See discussion of G1BOT's _offset_array below. ------------------------------------------------------------------------------ src/hotspot/share/runtime/atomic.hpp 957 inline D load() const { 961 template 962 inline void store(T store_value) { I wonder if load_relaxed and relaxed_store or something like that might be better names? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1BlockOffsetTable.cpp 45 _offset_array = cast_to_atomic_ptr((u_char*) bot_reserved.start()); This is currently the only use of cast_to_atomic_ptr. It leads to UB since the (non-trivial) constructor for AtomicValue is never called for any of the objects in the array. I think the call to memset_with_concurrent_readers is also UB. I *think* having a trivial default constructor solves the UB problem, but that isn't possible until C++11. Rather than just casting to a pointer to AtomicValue<>, this could iterate over the array memory, doing placement new construction of the elements. If we gave AtomicValue<> a non-trivial but empty default constructor (so not initializing the value), then such an iteration should be optimized away. We'd still have the UB from memset_with_cr, though that could use an iteration with the one-arg constructor instead. Or ignore the technical UB and just keep using memset_with_cr. And that gets rid of the one use (so far) of cast_to_atomic_ptr. The problem, of course, is that having a trivial/empty default constructor means one could unintentionally construct without initialization. I think a private trivial default constructor would deal with that, though again that requires C++11. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CodeCacheRemSet.cpp 39 AtomicValue G1CodeRootSetTable::_purge_list(NULL); Extra space between type and name. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.cpp 2023 _old_marking_cycles_started.load() == _old_marking_cycles_completed.load() + 1, Mis-indented. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.cpp 2027 _old_marking_cycles_started.inc(); This was _old_marking_cycles_started++. The membars introduced by using inc() aren't needed here. It is sufficient here to use _old_marking_cycles_started.store(_old_marking_cycles_started.load() + 1) This is the only write, and it's in the vmthread in a safepoint. Similarly in increment_old_marking_cycles_completed, except here we have the only write serialized by holding G1OldGCCount_lock. I also think _old_marking_cycles_started might not need any special handling at all. The additional membars don't really matter for performance, since these operations are very rare. But they catch they eye and make one wonder what is being ordered here and why. Maybe some commentary would suffice? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.cpp 4677 void G1CollectedHeap::increase_used(size_t bytes) { 4681 void G1CollectedHeap::decrease_used(size_t bytes) { I think these functions are mutexed, so the additional membars from using AtomicValue<>::add/sub aren't needed. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectionSet.cpp 129 _collection_set_regions[_collection_set_cur_length.fetch_and_add((size_t)1)] = hr->hrm_index(); I think unnecessary membars. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectionSet.cpp 191 size_t len = _collection_set_cur_length.load(); 192 OrderAccess::loadload(); This could just be a load_acquire(). ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectionSet.cpp 332 _collection_set_cur_length.inc(); I think unnecessary membars. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1ConcurrentMark.cpp 179 _chunks_in_chunk_list.inc(); 199 _chunks_in_chunk_list.dec(); I think unnecessary membars. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1ConcurrentMark.hpp 466 AtomicValue* _top_at_rebuild_starts; [pre-existing] Not sure, but I think this doesn't need to be atomic / didn't need to have a volatile qualifier in there. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1FullGCAdjustTask.hpp 42 AtomicValue _references_done; // Atomic counter / bool Member name no longer aligned with others adjacent. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1ParallelCleaning.hpp 32 AtomicValue _cleaning_claimed; [pre-existing] This looks like it ought to be bool rather than int, and change values from 1/0 to true/false. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RegionsOnNodes.cpp 44 _count_per_node[node_index].inc(); Previously not atomic; was that a bug? If not a bug, then not clear why these array elements were volatile. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1SurvivorRegions.cpp 67 _used_bytes.add(used_bytes); I think unnecessary membars. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionManager.cpp 663 for (uint i = 0; i < _n_regions; ++i) { 664 new (&_claims[i]) AtomicValue(Unclaimed); 665 } Even if AtomicValue has a trivial default destructor (as discussed above) to avoid UB when using memset, this change is an improvement. The old memset code only works because Unclaimed == 0. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionRemSet.cpp 54 fl = _free_list.load(); `fl = res;` instead? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionType.hpp 94 AtomicValue _tag; It's not immediately obvious why _tag needs to be atomic. Unless I've missed something, there don't appear to be anything other than relaxed load/store operations. It's not even obvious that those might occur concurrently. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Sun Jun 14 23:15:20 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 14 Jun 2020 19:15:20 -0400 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions Message-ID: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> Please review this change to oopDesc and Metadata classes, removing unnecessary volatile qualifiers from some member functions. This change also makes virtual overrides of the relevant functions explicitly virtual. While there, removed unnecessary workaround in oopDesc::klass_or_null_acquire. After this change, egrep -r -H --exclude-dir=.hg "\)\s*(const |)\s*volatile" src/hotspot finds nothing; there seem to be no remaining volatile qualified methods. CR: https://bugs.openjdk.java.net/browse/JDK-8247524 Webrev: https://cr.openjdk.java.net/~kbarrett/8247524/open.00/ Testing: mach5 tier1-3 From david.holmes at oracle.com Mon Jun 15 04:00:37 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 Jun 2020 14:00:37 +1000 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> <0250fab1-c743-7077-0485-2d188609cb42@oracle.com> Message-ID: Hi Leo, Thanks for undertaking this effort! Some higher-level comments at this stage just looking at the introduction of AtomicValue. As Kim has pointed out it may not always be appropriate to change all fields subject to atomic operations to be converted to AtomicValue. There is also the issue of fields accessed in a mix of locked and lock-free mechanisms (particularly in the runtime where we have locked writes and lock-free reads in a number of places, though sometimes combined with atomic updates). If such fields are converted to AtomicValue then I think we may still need a way to perform some raw operations on them; or else we chose to leave them as currently defined and apply Atomic ops directly when needed. (There is a parallel in the Java APIs where we can declare a field as an AtomicX variable, or else we can declare the field simply as X, and use an AtomicFieldUpdater to perform the atomic operations.) I like to imagine AtomicValue as a library-based alternative to a true "atomic" keyword in a language, but I'm torn between conciseness of expression and clarity of action. If we had an atomic keyword then: atomic int x = 0; ... x = 3; // this is an atomic store x++; // this is an atomic post-increment which would then argue for: AtomicValue x = 0; ... x = 3; // this is an atomic store x++; // this is an atomic post-increment rather than the explicit x.store(3); // this is an atomic store x.inc(); // this is an atomic post-increment in particular having explicit load() and store() operations makes for very unwieldy expressions IMHO. That said I know some people prefer it if atomic operations stand out very clearly. So I think we need to get a general consensus on what shape AtomicValue should take and exactly how we propose to use it (or not) across all of hotspot. And perhaps, as Kim alluded, the addition of AtomicValue should be split out from its application to any particular subsystem. Thanks, David ----- On 12/06/2020 1:51 am, Leo Korinth wrote: >>> Hi, could I have a review for this change that adds AtomicValue<> to >>> atomic.hpp and uses it in G1? >>> >>> I am adding an AtomicValue<> to atomic.hpp. This is an opaque type >>> (the "value" is private) and that protect against non-atomic >>> operations being used by mistake. AtomicValue methods are proxied to >>> the corresponding (all-static) Atomic:: methods. All operations are >>> explicitly atomic and in a type safe manner with semantics defined in >>> enum atomic_memory_order >>> (memory_order_conservative by default). >>> >>> Instead of using field variables as volatile, I change them to be of >>> AtomicValue<> type. No need to verify that += (for example) will >>> result in an atomic instruction on all supported compilers. >>> >>> I have some open questions regarding the exported fields in >>> vmStructs_g1.hpp. Today, volatile fields are sometimes "exported" as >>> volatile (and sometimes not, and I guess this is by mistake). I >>> choose to export them all as non-volatile. From what I can see the >>> volatile specific part only casts to void* (only documentation?). >>> Java code is unchanged and still access them as the unwrapped values >>> (static assert in atomic.hpp guarantees that memory layout is the >>> same for T and AtomicValue). I think this is okay, but would like >>> feedback on all this. >>> >>> The change is presented as a two part change. The first part changes >>> all volatile to AtomicValue, the other part removes the AtomicValue >>> part on non-field accesses. By doing it two part I will not forget to >>> transform some operations by mistake. >>> >>> Copyright years will be updated when all other changes are approved. >>> >>> How about pushing this after 15 is branched off and thus have it for 16? >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8247213 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~lkorinth/8247213/0/part1 >>> http://cr.openjdk.java.net/~lkorinth/8247213/0/part2 >>> http://cr.openjdk.java.net/~lkorinth/8247213/0/full >>> >>> Testing: >>> ?? tier1-3 >>> >>> Thanks, >>> Leo >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> Leo From stefan.karlsson at oracle.com Mon Jun 15 05:59:29 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 15 Jun 2020 07:59:29 +0200 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> Message-ID: Looks good. StefanK On 2020-06-15 01:15, Kim Barrett wrote: > Please review this change to oopDesc and Metadata classes, removing > unnecessary volatile qualifiers from some member functions. > > This change also makes virtual overrides of the relevant functions > explicitly virtual. > > While there, removed unnecessary workaround in oopDesc::klass_or_null_acquire. > > After this change, > egrep -r -H --exclude-dir=.hg "\)\s*(const |)\s*volatile" src/hotspot > finds nothing; there seem to be no remaining volatile qualified methods. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8247524 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8247524/open.00/ > > Testing: > mach5 tier1-3 > From stefan.karlsson at oracle.com Mon Jun 15 06:11:38 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 15 Jun 2020 08:11:38 +0200 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> Message-ID: <7524f899-42c8-9388-b2fa-6dece16336fc@oracle.com> Thanks, Kim. StefanK On 2020-06-13 05:06, Kim Barrett wrote: >> On Jun 12, 2020, at 5:33 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to fix the NMT accounting of CHeap GrowableArray allocations. >> >> https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247470 > The GrowableArray changes look good to me. > > I?ve only spot-checked some of the usage changes, but didn?t find any problems. > From david.holmes at oracle.com Mon Jun 15 06:23:12 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 Jun 2020 16:23:12 +1000 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> Hi Igor, I found this all a bit hard to follow. It would have been clearer if broken up into distinct pieces. On 13/06/2020 2:10 pm, Igor Ignatyev wrote: > testing revealed that LingeredAppTest.java required some love, incremental webrev w/ the fixes for LingeredAppTest -- http://cr.openjdk.java.net/~iignatyev//8211977/webrev.0-1 > > -- Igor > >> On Jun 12, 2020, at 8:38 PM, Igor Ignatyev wrote: >> >> adding build-dev >> >>> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>>> 796 lines changed: 200 ins; 588 del; 8 mod; >>> >>> Hi all, >>> >>> could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? >>> >>> besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. You did not modify hotspot/jtreg/TEST.groups but it refers to: testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ Plus it should probably refer to the new native_sanity tests somewhere. >>> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >>> - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) How was it built before ? >>> - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) >>> - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder >>> - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag >>> - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ >>> - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ test/lib-test/TEST.ROOT The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. make/test/JtregNativeLibTest.gmk You used the wrong license header - makefiles don't use the classpath exception. Otherwise changes appear to be okay. Thanks, David ----- >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: test/lib-test on {windows,linux,macosx}-x64 >>> >>> Thanks, >>> -- Igor >>> >>> >> > From kim.barrett at oracle.com Mon Jun 15 06:48:42 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jun 2020 02:48:42 -0400 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> Message-ID: > On Jun 15, 2020, at 1:59 AM, Stefan Karlsson wrote: > > Looks good. Thanks. > StefanK > > On 2020-06-15 01:15, Kim Barrett wrote: >> Please review this change to oopDesc and Metadata classes, removing >> unnecessary volatile qualifiers from some member functions. >> >> This change also makes virtual overrides of the relevant functions >> explicitly virtual. >> >> While there, removed unnecessary workaround in oopDesc::klass_or_null_acquire. >> >> After this change, >> egrep -r -H --exclude-dir=.hg "\)\s*(const |)\s*volatile" src/hotspot >> finds nothing; there seem to be no remaining volatile qualified methods. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8247524 >> >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8247524/open.00/ >> >> Testing: >> mach5 tier1-3 From david.holmes at oracle.com Mon Jun 15 06:59:19 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 Jun 2020 16:59:19 +1000 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> Message-ID: <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> Hi Kim, On 15/06/2020 9:15 am, Kim Barrett wrote: > Please review this change to oopDesc and Metadata classes, removing > unnecessary volatile qualifiers from some member functions. > > This change also makes virtual overrides of the relevant functions > explicitly virtual. > > While there, removed unnecessary workaround in oopDesc::klass_or_null_acquire. > > After this change, > egrep -r -H --exclude-dir=.hg "\)\s*(const |)\s*volatile" src/hotspot > finds nothing; there seem to be no remaining volatile qualified methods. I must admit I'm quite confused as to why anyone would ever have placed a volatile modifier on a method that returns a constant true or false ??? What could it possibly mean? On the other hand for a mutable field e.g.: size_t size() const volatile { return _word_size; } I can see an expectation that the volatile would force the compiler to always reload _word_size, even if the field itself was not declared volatile (which it isn't). Isn't this how we are now handling things via Atomic::load - where the field is no longer flagged volatile but the load is so that we still tell the compiler not to make assumptions about potentially caching the field? Thanks, David > CR: > https://bugs.openjdk.java.net/browse/JDK-8247524 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8247524/open.00/ > > Testing: > mach5 tier1-3 > From erik.osterlund at oracle.com Mon Jun 15 07:36:08 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 15 Jun 2020 09:36:08 +0200 Subject: [16] RFR: 8247319: Remove on-stack nmethod hotness counter sampling from safepoints In-Reply-To: <11ecfe57-9142-93e4-d566-57effe29eb84@oracle.com> References: <8feb88a7-6bab-f694-cf24-2a83c3461f33@oracle.com> <11ecfe57-9142-93e4-d566-57effe29eb84@oracle.com> Message-ID: Hi Vladimir, Thank you for reviewing. Here is a new webrev with your suggested commend changes and cleanup: http://cr.openjdk.java.net/~eosterlund/8247319/webrev.01/ Incremental: http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00.01/ Thanks, /Erik On 2020-06-12 21:35, Vladimir Kozlov wrote: > I agree with changes. > > Please also update comments [1] and clean up leftover [2] from 8244658 > [3]. > > Thanks, > Vladimir > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/runtime/sweeper.hpp#l38 > http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/code/nmethod.hpp#l260 > > [2] > http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/runtime/vmOperations.hpp#l113 > [3] https://bugs.openjdk.java.net/browse/JDK-8244658 > > On 6/12/20 12:29 AM, Erik ?sterlund wrote: >> Hi, >> >> The sweeper is moving away from using safepoints for its heuristics. >> It used to count safepoints to figure out when to sweep, but no >> longer does that. At the same time, we have for a while been removing >> more and more safepoints. Safepoints are becoming increasingly rare >> events, dominated by when we need to GC (GuaranteedSafepointInterval >> is going to disappear). The frequency of how often we need to GC does >> not have an obvious connection to how often we need to sweep the code >> cache... any more. >> >> What still remains from the safepoint-based heuristics is the nmethod >> hotness counter sampling that is performed in safepoint cleanup. I >> would like to get rid of this. >> The rationale is that the use of hotness counters is kicking in when >> the code cache is starting to fill up quite a bit, and there is a >> need to kill off nmethods heuristically, rather than because they are >> invalid. But when the code cache fills up, we sweep more and more >> aggressively. And during these sweeper cycles, we perform nmethod >> marking using handshakes. That operation also fills in hotness >> counters for all sampled nmethods. >> >> In other words, when there is need for acting on the hotness >> counters, we are in a state where we may be getting more nmethod >> hotness counter sample information from the sweeping cycles than we >> are from safepoint sampling. Conversely, when code cache pressure is >> high and we need more samples, we might end up getting very few from >> safepoint based sampling (because the heap is large). The correlation >> between safepoint frequency and code cache pressure is simply not >> there any more. And for us to walk all stacks in the system in every >> single safepoint (which for ZGC is starting to dominate pauses when I >> remove our stack sampling from safepoints), there better be a really >> good reason to do this sampling in safepoints. And there simply >> isn't. So I propose we delete it, in favour of using the hotness >> counter samples we get from the sweeping cycles instead, that are >> indeed proportional in frequency, to the code cache pressure. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8247319 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00/ >> >> Thanks, >> /Erik From kim.barrett at oracle.com Mon Jun 15 08:56:20 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jun 2020 04:56:20 -0400 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> Message-ID: > On Jun 15, 2020, at 2:59 AM, David Holmes wrote: > > Hi Kim, > > On 15/06/2020 9:15 am, Kim Barrett wrote: >> Please review this change to oopDesc and Metadata classes, removing >> unnecessary volatile qualifiers from some member functions. >> This change also makes virtual overrides of the relevant functions >> explicitly virtual. >> While there, removed unnecessary workaround in oopDesc::klass_or_null_acquire. >> After this change, >> egrep -r -H --exclude-dir=.hg "\)\s*(const |)\s*volatile" src/hotspot >> finds nothing; there seem to be no remaining volatile qualified methods. > > I must admit I'm quite confused as to why anyone would ever have placed a volatile modifier on a method that returns a constant true or false ??? What could it possibly mean? The only thing I can think of is that these were called from other member functions that were themselves volatile qualified. But there aren?t any of those anymore... > On the other hand for a mutable field e.g.: > > size_t size() const volatile { return _word_size; } > > I can see an expectation that the volatile would force the compiler to always reload _word_size, even if the field itself was not declared volatile (which it isn't). Isn't this how we are now handling things via Atomic::load - where the field is no longer flagged volatile but the load is so that we still tell the compiler not to make assumptions about potentially caching the field? The ?preferred" way to do that today would be to make the member volatile and use Atomic accesses (possibly relaxed if no ordering is needed). Or just rely on the volatile qualified member (though that?s no longer ?preferred?). (And AtomicValue<> might be used in the future, instead of a volatile member.) But in this case, none of that matters. The _word_size is initialized by the constructor and is never modified - there are no calls to set_size. I should have removed set_size but forgot; I?ll fix that now. And it looks like there are a whole bunch of unused or seemingly useless functions in Metabase. > > Thanks, > David > >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8247524 >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8247524/open.00/ >> Testing: >> mach5 tier1-3 From aph at redhat.com Mon Jun 15 09:17:07 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 15 Jun 2020 10:17:07 +0100 Subject: RFR(S): 8247200: [aarch64] assert((unsigned)fpargs < 32) In-Reply-To: References: <40b781a4-5a1f-bd83-0dab-00dc2890e06d@oracle.com> Message-ID: <61acad9c-98a1-46b2-8e24-d3cb84fa2628@redhat.com> On 11/06/2020 17:56, Vladimir Kozlov wrote: > On 6/11/20 1:24 AM, Patric Hedlin wrote: >> Dear all, >> >> I would like to ask for help to review the following change/update: >> >> Issue:? https://bugs.openjdk.java.net/browse/JDK-8247200 >> Webrev: http://cr.openjdk.java.net/~phedlin/tr8247200/ >> >> >> Removing assert and some associated dead code. Looks good. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From patric.hedlin at oracle.com Mon Jun 15 10:22:08 2020 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Mon, 15 Jun 2020 12:22:08 +0200 Subject: RFR(S): 8247200: [aarch64] assert((unsigned)fpargs < 32) In-Reply-To: <61acad9c-98a1-46b2-8e24-d3cb84fa2628@redhat.com> References: <40b781a4-5a1f-bd83-0dab-00dc2890e06d@oracle.com> <61acad9c-98a1-46b2-8e24-d3cb84fa2628@redhat.com> Message-ID: Thanks for reviewing Andrew. /Patric On 2020-06-15 11:17, Andrew Haley wrote: > On 11/06/2020 17:56, Vladimir Kozlov wrote: >> On 6/11/20 1:24 AM, Patric Hedlin wrote: >>> Dear all, >>> >>> I would like to ask for help to review the following change/update: >>> >>> Issue:? https://bugs.openjdk.java.net/browse/JDK-8247200 >>> Webrev: http://cr.openjdk.java.net/~phedlin/tr8247200/ >>> >>> >>> Removing assert and some associated dead code. > Looks good. > From kim.barrett at oracle.com Mon Jun 15 11:41:02 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jun 2020 07:41:02 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <5EE5AB71.20505@oracle.com> References: <5EDE9A75.6020504@oracle.com> <5EE5AB71.20505@oracle.com> Message-ID: > On Jun 14, 2020, at 12:45 AM, Philip Race wrote: > > Kim, > > > Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. > Please rename the JEP. I think this JEP is primarily about updating the HotSpot-specific subset of C++ and usage guidance to include some C++14 features. HotSpot usage is already different in some ways from C++ code in other parts of the JDK. For example, HotSpot code doesn't use global operator new; that isn't likely to change, and non-HotSpot code doesn't have access to the alternatives defined and used but not exported by HotSpot. As a necessary prerequisite for that, the JDK build system is being updated to enable the use of C++14 features. It's probably possible to limit the scope of the language change to HotSpot, but that doesn't seem useful. It also probably wasn't possible while Solaris was still in the mix. It is expected that groups responsible for other parts of the JDK will also want to take advantage of that build change, but I think it's up to those other groups to decide on an adoption strategy. I have no reason to think the choices being suggested here as appropriate for HotSpot are appropriate JDK-wide. I can't say whether a JEP is generally needed; maybe it's simply "C++14 - yes" in some areas. Also, some of the reasons for this being a JEP no longer apply. For example, Solaris was going to require at least a major ABI change, and at the time the JEP was created there was no path to C++14 for the AIX/PPC port; both of those needed visibility and discussion that seemed best provided via the JEP process. > -phil. > > On 6/13/20, 8:48 PM, Kim Barrett wrote: >>> On Jun 10, 2020, at 2:26 AM, Kim Barrett wrote: >>> >>>> On Jun 8, 2020, at 4:07 PM, Philip Race wrote: >>>> BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says >>>>> For Solaris: Add the -std=c++14 compiler option. ..... >>>> So I am sure there is still some room to update the JEP :-) >>> Yeah, I have some updates to make. I also need to do a bit of work on the feature lists. >> I think the JEP updates are done now. From coleen.phillimore at oracle.com Mon Jun 15 11:55:41 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 15 Jun 2020 07:55:41 -0400 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> Message-ID: This change looks good.? Those volatiles were probably added to quiet the compiler for some callers with volatiles.? This new approach is much better. Don't worry about other changes to Metabase; it's gone in Elastic Metaspace. thanks, Coleen On 6/15/20 4:56 AM, Kim Barrett wrote: >> On Jun 15, 2020, at 2:59 AM, David Holmes wrote: >> >> Hi Kim, >> >> On 15/06/2020 9:15 am, Kim Barrett wrote: >>> Please review this change to oopDesc and Metadata classes, removing >>> unnecessary volatile qualifiers from some member functions. >>> This change also makes virtual overrides of the relevant functions >>> explicitly virtual. >>> While there, removed unnecessary workaround in oopDesc::klass_or_null_acquire. >>> After this change, >>> egrep -r -H --exclude-dir=.hg "\)\s*(const |)\s*volatile" src/hotspot >>> finds nothing; there seem to be no remaining volatile qualified methods. >> I must admit I'm quite confused as to why anyone would ever have placed a volatile modifier on a method that returns a constant true or false ??? What could it possibly mean? > The only thing I can think of is that these were called from other member functions that > were themselves volatile qualified. But there aren?t any of those anymore... > >> On the other hand for a mutable field e.g.: >> >> size_t size() const volatile { return _word_size; } >> >> I can see an expectation that the volatile would force the compiler to always reload _word_size, even if the field itself was not declared volatile (which it isn't). Isn't this how we are now handling things via Atomic::load - where the field is no longer flagged volatile but the load is so that we still tell the compiler not to make assumptions about potentially caching the field? > The ?preferred" way to do that today would be to make the member volatile and use Atomic > accesses (possibly relaxed if no ordering is needed). Or just rely on the volatile qualified > member (though that?s no longer ?preferred?). (And AtomicValue<> might be used in the > future, instead of a volatile member.) > > But in this case, none of that matters. The _word_size is initialized by the constructor and > is never modified - there are no calls to set_size. I should have removed set_size but > forgot; I?ll fix that now. And it looks like there are a whole bunch of unused or seemingly > useless functions in Metabase. > >> Thanks, >> David >> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8247524 >>> Webrev: >>> https://cr.openjdk.java.net/~kbarrett/8247524/open.00/ >>> Testing: >>> mach5 tier1-3 > From coleen.phillimore at oracle.com Mon Jun 15 12:09:19 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 15 Jun 2020 08:09:19 -0400 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> Message-ID: https://cr.openjdk.java.net/~stefank/8247470/webrev.01/src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp.udiff.html I thought there was an mtJVMTI tag or that I filed a bug to add one but I guess not.? mtServiceability is fine.? Thank you for fixing these. https://cr.openjdk.java.net/~stefank/8247470/webrev.01/src/hotspot/share/runtime/unhandledOops.cpp.udiff.html Maybe mtThread? since the unhandled oops are added to the thread, so we can account for them there. I really like this change. Thank you! Coleen On 6/12/20 5:33 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to fix the NMT accounting of CHeap > GrowableArray allocations. > > https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247470 > > The initial reason why I started fixing this was that I found code > like this: > > src/hotspot/share/classfile/javaClasses.cpp: > ??? new (ResourceObj::C_HEAP, mtModule) GrowableArray(500, true); > > Before this patch, the GrowableArray constructor looked like this: > > GrowableArray(int initial_size, bool C_heap = false, MEMFLAGS F = > mtInternal) > : GenericGrowableArray(initial_size, 0, C_heap, F) { > _data = (E*)raw_allocate(sizeof(E)); > > and: > > GenericGrowableArray(int initial_size, int initial_len, bool c_heap, > MEMFLAGS flags = mtNone) { _len = initial_len; _max = initial_size; > _memflags = flags; > > and: > void* GenericGrowableArray::raw_allocate(int elementSize) { > ... > return (void*)AllocateHeap(byte_size, _memflags); > > This means that the GrowableArray instance was accounted to mtModule, > but the backing memory for the array was accounted to mtInternal. > > My proposal for fixing this is to fuse the (c_heap, flags) duality > into one parameter (flags). The call sites are then changed like this: > > For resource allocations: > GrowableArray(...) -> GrowableArray(...) > GrowableArray(... /* c_heap */ false) -> GrowableArray(...) > GrowableArray(... /* c_heap */ false, /* flags */ mtXX) -> > GrowableArray(...) > > For CHeap allocations: > GrowableArray(... /* c_heap */ true, /* flags */ mtXX) -> > GrowableArray(..., mtXX) > GrowableArray(... /* c_heap */ true) -> GrowableArray(..., mtInternal) > > That is, if you specify a "valid" mtXX tag it invokes the CHeap > allocation path. If you don't specify a mtXX tag, then 'flags' get the > mtNone default value. This is a marker for the code that no CHeap > allocation should be done. > > While properly tagging all GrowableArray allocations, I went ahead and > changed most GrowableArray mtInternal tags to the appropriate values. > I also introduced a new mtServiceability tag, since a lot of the > mtInternal tags were in areas dealing with serviceability code. I > could split this out into a separate patch, if necessary. > > This hasn't been fully tested yet, because I'd like to get some early > feedback. > > Thanks, > StefanK From kim.barrett at oracle.com Mon Jun 15 12:16:22 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jun 2020 08:16:22 -0400 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> Message-ID: <049302EB-1F4F-46BD-9770-167E46AA5CA3@oracle.com> > On Jun 15, 2020, at 7:55 AM, coleen.phillimore at oracle.com wrote: > > > This change looks good. Those volatiles were probably added to quiet the compiler for some callers with volatiles. This new approach is much better. Thanks. > Don't worry about other changes to Metabase; it's gone in Elastic Metaspace. Thanks for the update. I haven?t been following the Metaspace revamp discussion. From thomas.schatzl at oracle.com Mon Jun 15 12:22:49 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Jun 2020 14:22:49 +0200 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> Message-ID: <347f46ca-953f-c4f3-c147-ba093d936746@oracle.com> Hi, On 13.06.20 05:06, Kim Barrett wrote: >> On Jun 12, 2020, at 5:33 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to fix the NMT accounting of CHeap GrowableArray allocations. >> >> https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247470 > > The GrowableArray changes look good to me. > > I?ve only spot-checked some of the usage changes, but didn?t find any problems. > looks good to me. Thanks, Thomas From stefan.karlsson at oracle.com Mon Jun 15 12:25:01 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 15 Jun 2020 14:25:01 +0200 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> Message-ID: <1fd4bc42-6a92-50f3-2b07-39dfcd5d4a71@oracle.com> On 2020-06-15 14:09, coleen.phillimore at oracle.com wrote: > > https://cr.openjdk.java.net/~stefank/8247470/webrev.01/src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp.udiff.html > > > I thought there was an mtJVMTI tag or that I filed a bug to add one > but I guess not.? mtServiceability is fine.? Thank you for fixing these. > > https://cr.openjdk.java.net/~stefank/8247470/webrev.01/src/hotspot/share/runtime/unhandledOops.cpp.udiff.html > > > Maybe mtThread? since the unhandled oops are added to the thread, so > we can account for them there. Sure, I'll change it. > > I really like this change. Thanks for reviewing! StefanK > > Thank you! > Coleen > > > On 6/12/20 5:33 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to fix the NMT accounting of CHeap >> GrowableArray allocations. >> >> https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247470 >> >> The initial reason why I started fixing this was that I found code >> like this: >> >> src/hotspot/share/classfile/javaClasses.cpp: >> ??? new (ResourceObj::C_HEAP, mtModule) GrowableArray(500, >> true); >> >> Before this patch, the GrowableArray constructor looked like this: >> >> GrowableArray(int initial_size, bool C_heap = false, MEMFLAGS F = >> mtInternal) >> : GenericGrowableArray(initial_size, 0, C_heap, F) { >> _data = (E*)raw_allocate(sizeof(E)); >> >> and: >> >> GenericGrowableArray(int initial_size, int initial_len, bool c_heap, >> MEMFLAGS flags = mtNone) { _len = initial_len; _max = initial_size; >> _memflags = flags; >> >> and: >> void* GenericGrowableArray::raw_allocate(int elementSize) { >> ... >> return (void*)AllocateHeap(byte_size, _memflags); >> >> This means that the GrowableArray instance was accounted to mtModule, >> but the backing memory for the array was accounted to mtInternal. >> >> My proposal for fixing this is to fuse the (c_heap, flags) duality >> into one parameter (flags). The call sites are then changed like this: >> >> For resource allocations: >> GrowableArray(...) -> GrowableArray(...) >> GrowableArray(... /* c_heap */ false) -> GrowableArray(...) >> GrowableArray(... /* c_heap */ false, /* flags */ mtXX) -> >> GrowableArray(...) >> >> For CHeap allocations: >> GrowableArray(... /* c_heap */ true, /* flags */ mtXX) -> >> GrowableArray(..., mtXX) >> GrowableArray(... /* c_heap */ true) -> GrowableArray(..., mtInternal) >> >> That is, if you specify a "valid" mtXX tag it invokes the CHeap >> allocation path. If you don't specify a mtXX tag, then 'flags' get >> the mtNone default value. This is a marker for the code that no CHeap >> allocation should be done. >> >> While properly tagging all GrowableArray allocations, I went ahead >> and changed most GrowableArray mtInternal tags to the appropriate >> values. I also introduced a new mtServiceability tag, since a lot of >> the mtInternal tags were in areas dealing with serviceability code. I >> could split this out into a separate patch, if necessary. >> >> This hasn't been fully tested yet, because I'd like to get some early >> feedback. >> >> Thanks, >> StefanK > From stefan.karlsson at oracle.com Mon Jun 15 12:25:27 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 15 Jun 2020 14:25:27 +0200 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: <347f46ca-953f-c4f3-c147-ba093d936746@oracle.com> References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> <347f46ca-953f-c4f3-c147-ba093d936746@oracle.com> Message-ID: <45f642e0-43c4-b930-22f3-dc5eff04181e@oracle.com> Thanks, Thomas. StefanK On 2020-06-15 14:22, Thomas Schatzl wrote: > Hi, > > On 13.06.20 05:06, Kim Barrett wrote: >>> On Jun 12, 2020, at 5:33 AM, Stefan Karlsson >>> wrote: >>> >>> Hi all, >>> >>> Please review this patch to fix the NMT accounting of CHeap >>> GrowableArray allocations. >>> >>> https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8247470 >> >> The GrowableArray changes look good to me. >> >> I?ve only spot-checked some of the usage changes, but didn?t find any >> problems. >> > > ? looks good to me. > > Thanks, > ? Thomas From thomas.stuefe at gmail.com Mon Jun 15 12:27:35 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 15 Jun 2020 14:27:35 +0200 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: <049302EB-1F4F-46BD-9770-167E46AA5CA3@oracle.com> References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> <049302EB-1F4F-46BD-9770-167E46AA5CA3@oracle.com> Message-ID: This looks good to me too. Cheers, Thomas P.S. note that we at SAP seem to be cut off from hotspot-dev ml since 5-6 weeks, we just noticed that. So if we have been weirdly quiet, that is the reason. On Mon, Jun 15, 2020 at 2:16 PM Kim Barrett wrote: > > On Jun 15, 2020, at 7:55 AM, coleen.phillimore at oracle.com wrote: > > > > > > This change looks good. Those volatiles were probably added to quiet > the compiler for some callers with volatiles. This new approach is much > better. > > Thanks. > > > Don't worry about other changes to Metabase; it's gone in Elastic > Metaspace. > > Thanks for the update. I haven?t been following the Metaspace revamp > discussion. > > From kim.barrett at oracle.com Mon Jun 15 12:29:39 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jun 2020 08:29:39 -0400 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> <049302EB-1F4F-46BD-9770-167E46AA5CA3@oracle.com> Message-ID: <53B4F7EE-7F44-486F-9EC0-07D21E4BD009@oracle.com> > On Jun 15, 2020, at 8:27 AM, Thomas St?fe wrote: > > This looks good to me too. Thanks. > > Cheers, Thomas > > P.S. note that we at SAP seem to be cut off from hotspot-dev ml since 5-6 weeks, we just noticed that. So if we have been weirdly quiet, that is the reason. > > > > On Mon, Jun 15, 2020 at 2:16 PM Kim Barrett wrote: > > On Jun 15, 2020, at 7:55 AM, coleen.phillimore at oracle.com wrote: > > > > > > This change looks good. Those volatiles were probably added to quiet the compiler for some callers with volatiles. This new approach is much better. > > Thanks. > > > Don't worry about other changes to Metabase; it's gone in Elastic Metaspace. > > Thanks for the update. I haven?t been following the Metaspace revamp discussion. From coleen.phillimore at oracle.com Mon Jun 15 12:39:31 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 15 Jun 2020 08:39:31 -0400 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <7DE4D1D2-ECA8-4980-B665-781BB0E36B2F@oracle.com> References: <4611e386-2d3f-efaa-fdce-920c7d3d5b85@oracle.com> <7DE4D1D2-ECA8-4980-B665-781BB0E36B2F@oracle.com> Message-ID: <598ae299-13c6-5713-74ea-2be5285c2c7c@oracle.com> I like the concept of this.? I really want the compiler to do to the work to verify that if something is interestingly concurrent, we shouldn't use plain loads and stores of it. The name AtomicValue<> seems as good as any.? Can AtomicValue be declared in its own header file: runtime/atomicValue.hpp please? On 6/14/20 4:51 AM, Kim Barrett wrote: >> On Jun 10, 2020, at 8:46 AM, Leo Korinth wrote: >> >> Hi, could I have a review for this change that adds AtomicValue<> to atomic.hpp and uses it in G1? >> >> I am adding an AtomicValue<> to atomic.hpp. This is an opaque type (the "value" is private) and that protect against non-atomic operations being used by mistake. AtomicValue methods are proxied to the corresponding (all-static) Atomic:: methods. All operations are explicitly atomic and in a type safe manner with semantics defined in enum atomic_memory_order >> (memory_order_conservative by default). >> >> Instead of using field variables as volatile, I change them to be of AtomicValue<> type. No need to verify that += (for example) will result in an atomic instruction on all supported compilers. >> >> I have some open questions regarding the exported fields in vmStructs_g1.hpp. Today, volatile fields are sometimes "exported" as volatile (and sometimes not, and I guess this is by mistake). I choose to export them all as non-volatile. From what I can see the volatile specific part only casts to void* (only documentation?). Java code is unchanged and still access them as the unwrapped values (static assert in atomic.hpp guarantees that memory layout is the same for T and AtomicValue). I think this is okay, but would like feedback on all this. The SA doesn't really need anything to be volatile. I think they are there to get vmStructs to compile for volatile fields but don't have any meaning for SA since it is used to look at core files or unresponsive VMs.? I don't see the void* cast that you are referring to in the macros, which doesn't mean it's not there.? The SA seems to read these values as their underlying types so if you don't get errors in test/hotspot/jtreg/serviceability/sa, the changes are fine.? Hopefully you won't have to add AtomicValue to SA. thanks, Coleen >> >> The change is presented as a two part change. The first part changes all volatile to AtomicValue, the other part removes the AtomicValue part on non-field accesses. By doing it two part I will not forget to transform some operations by mistake. >> >> Copyright years will be updated when all other changes are approved. >> >> How about pushing this after 15 is branched off and thus have it for 16? >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8247213 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8247213/0/part1 >> http://cr.openjdk.java.net/~lkorinth/8247213/0/part2 >> http://cr.openjdk.java.net/~lkorinth/8247213/0/full >> >> Testing: >> tier1-3 >> >> Thanks, >> Leo > I think having a class for atomic members and variables is good, using > the compiler to help keep us honest about what we're doing. So I > generally like this change, despite the number of comments below. > > First a few general comments, then a bunch of detailed comments on > specific changes. > > ------------------------------------------------------------------------------ > > I kind of wish this had been broken up into smaller chunks. Looking > at a large number of changes involving atomic operations all in one go > is pretty tiring. > > ------------------------------------------------------------------------------ > > I hope that this will not be a G1-specific usage, nor even a > GC-specific usage, but rather will eventually be used throughout > HotSpot. If not, then I'm not sure this change should be made; I > wouldn't like having the differing styles of volatile members in some > code and AtomicValue in other code (other than as a temporary state as > changes are incrementally made). (That doesn't mean that I think all > atomic operations should involve AtomicValue, and the static Atomic > functions are no longer needed. I think there are some places where > forcing things into the former mold is problematic and the Atomic > functions will still have direct uses.) > > ------------------------------------------------------------------------------ > > The name of the class has generated some discussion. (Full disclosure: > I suggested AtomicValue.) "AtomicValue" follows the style guide of > using CamelCase for type names. But there are lots of > counter-examples. "Atomic" is a bit strange, since then we'd have a > class that is both a thin wrapper with member functions and an > AllStatic with similarly named static member functions taking an > additional explicit argument. The alternative would be to give this > class the "Atomic" name and rename the existing "Atomic", but that > would be pretty disruptive. atomic is short and aligns nicely with > Atomic, but names that differ only in case may be considered a > problem. Brevity doesn't seem super critical here; the type name is > mostly used as the type for data members and global variables, and > rarely appears in function signatures. > > ------------------------------------------------------------------------------ > > There are a number of places where plain arithmetic was being > performed on relevant variables (mostly pre/post increment/decrement). > I think most of these were intentionally not using Atomic arithmetic, > because they are in single-threaded (mutex or safepoint) contexts. > > This change currently transforms those all into atomic inc/add/&etc. > I've been waffling about this, and have noted occurrences in the > detailed comments below. There is a (probably often negligable, though > maybe not always) performance cost to that transformation, but that's > not what concerns me. When I see an atomic operation, I expect there's > something interesting happening and ordering or atomicity is > important. Using atomic operations unnecessarily breaks that > expectation, making the code less clear. That doesn't mean we should > use pre/post increment/decrement operators and such for "non-atomic" > uses though; we could have other names that are more explicit. (Note > that atomic arithmetic operations with relaxed memory order are not > the same thing as non-atomic arithmetic operations.) > > Also note that during review I found one place where I think the use > of non-atomic arithmetic may have been a mistake, in which case the > change to atomic arithmetic would be correct and necessary. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 42 #include "utilities/debug.hpp" > > Out of order. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 434 template > 435 struct Atomic::StoreImpl< > 436 D, T, > 437 PlatformOp, > 438 typename EnableIf<(IsIntegral::value || IsRegisteredEnum::value) && (IsIntegral::value || IsRegisteredEnum::value)>::type> > > ... > 442 D value = new_value; > > Allowing implicit conversions here was discussed and rejected during > the templatization of Atomic. While I think allowing some conversions > here would be good, I think this is much too loose. I think a change > of this type should be dealt with separately. > > Also, the line length for the EnableIf line seems rather excessive to me. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 943 AtomicValue(const AtomicValue& a); // = delete > > I think copy-assign should also be forbidden. Why not use NONCOPYABLE? > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 946 STATIC_ASSERT(sizeof(AtomicValue) == sizeof(D)); // ensure the same memory layout > > I think there are more requirements than same-size. > But maybe they fall out of usage, since the Atomic operations will > provide the additional requirements. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 949 inline void inc(atomic_memory_order order=memory_order_conservative) { > > I think there should be whitespace around "=". > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 940 class AtomicValue { > > I think I'd like the order of the load/store operations rearranged: > > load, load_acquire > store, release_store, release_store_fence > > I also think I'd like inc/dec near add/sub. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 1007 template > 1008 static AtomicValue* cast_to_atomic_ptr(T* nonatomic); > > Declared but not defined. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 1011 template > 1012 static AtomicValue* cast_to_atomic_ptr(T* nonatomic) { > > I would really like to not have this. I'd like to see how far we can > get without it. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 940 class AtomicValue { > > Should it be default constructible? If so, should the default > constructor guarantee value initialization? I think having a trivial > default constructor is needed to avoid UB in some uses. See discussion > of G1BOT's _offset_array below. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 957 inline D load() const { > > 961 template > 962 inline void store(T store_value) { > > I wonder if load_relaxed and relaxed_store or something like that > might be better names? > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1BlockOffsetTable.cpp > 45 _offset_array = cast_to_atomic_ptr((u_char*) bot_reserved.start()); > > This is currently the only use of cast_to_atomic_ptr. It leads to UB > since the (non-trivial) constructor for AtomicValue is never called > for any of the objects in the array. I think the call to > memset_with_concurrent_readers is also UB. I *think* having a trivial > default constructor solves the UB problem, but that isn't possible > until C++11. > > Rather than just casting to a pointer to AtomicValue<>, this could > iterate over the array memory, doing placement new construction of the > elements. If we gave AtomicValue<> a non-trivial but empty default > constructor (so not initializing the value), then such an iteration > should be optimized away. We'd still have the UB from memset_with_cr, > though that could use an iteration with the one-arg constructor instead. > Or ignore the technical UB and just keep using memset_with_cr. > > And that gets rid of the one use (so far) of cast_to_atomic_ptr. > > The problem, of course, is that having a trivial/empty default > constructor means one could unintentionally construct without > initialization. I think a private trivial default constructor would > deal with that, though again that requires C++11. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CodeCacheRemSet.cpp > 39 AtomicValue G1CodeRootSetTable::_purge_list(NULL); > > Extra space between type and name. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 2023 _old_marking_cycles_started.load() == _old_marking_cycles_completed.load() + 1, > > Mis-indented. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 2027 _old_marking_cycles_started.inc(); > > This was _old_marking_cycles_started++. The membars introduced by > using inc() aren't needed here. It is sufficient here to use > _old_marking_cycles_started.store(_old_marking_cycles_started.load() + 1) > This is the only write, and it's in the vmthread in a safepoint. > > Similarly in increment_old_marking_cycles_completed, except here we > have the only write serialized by holding G1OldGCCount_lock. > > I also think _old_marking_cycles_started might not need any special > handling at all. > > The additional membars don't really matter for performance, since > these operations are very rare. But they catch they eye and make one > wonder what is being ordered here and why. Maybe some commentary > would suffice? > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 4677 void G1CollectedHeap::increase_used(size_t bytes) { > 4681 void G1CollectedHeap::decrease_used(size_t bytes) { > > I think these functions are mutexed, so the additional membars from > using AtomicValue<>::add/sub aren't needed. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectionSet.cpp > 129 _collection_set_regions[_collection_set_cur_length.fetch_and_add((size_t)1)] = hr->hrm_index(); > > I think unnecessary membars. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectionSet.cpp > 191 size_t len = _collection_set_cur_length.load(); > 192 OrderAccess::loadload(); > > This could just be a load_acquire(). > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectionSet.cpp > 332 _collection_set_cur_length.inc(); > > I think unnecessary membars. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 179 _chunks_in_chunk_list.inc(); > 199 _chunks_in_chunk_list.dec(); > > I think unnecessary membars. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1ConcurrentMark.hpp > 466 AtomicValue* _top_at_rebuild_starts; > > [pre-existing] > Not sure, but I think this doesn't need to be atomic / didn't need to > have a volatile qualifier in there. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1FullGCAdjustTask.hpp > 42 AtomicValue _references_done; // Atomic counter / bool > > Member name no longer aligned with others adjacent. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1ParallelCleaning.hpp > 32 AtomicValue _cleaning_claimed; > > [pre-existing] > This looks like it ought to be bool rather than int, and change values > from 1/0 to true/false. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1RegionsOnNodes.cpp > 44 _count_per_node[node_index].inc(); > > Previously not atomic; was that a bug? If not a bug, then not clear > why these array elements were volatile. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1SurvivorRegions.cpp > 67 _used_bytes.add(used_bytes); > > I think unnecessary membars. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionManager.cpp > 663 for (uint i = 0; i < _n_regions; ++i) { > 664 new (&_claims[i]) AtomicValue(Unclaimed); > 665 } > > Even if AtomicValue has a trivial default destructor (as discussed > above) to avoid UB when using memset, this change is an improvement. > The old memset code only works because Unclaimed == 0. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionRemSet.cpp > 54 fl = _free_list.load(); > > `fl = res;` instead? > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionType.hpp > 94 AtomicValue _tag; > > It's not immediately obvious why _tag needs to be atomic. Unless I've > missed something, there don't appear to be anything other than relaxed > load/store operations. It's not even obvious that those might occur > concurrently. > > ------------------------------------------------------------------------------ > From magnus.ihse.bursie at oracle.com Mon Jun 15 12:57:43 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 15 Jun 2020 14:57:43 +0200 Subject: RFR: JDK-8150828 Consider using '-fno-asynchronous-unwind-tables' to reduce the size of libjvm.so by 10 percent In-Reply-To: References: <499b6643-09df-89a3-a790-58839f42a5a9@oracle.com> Message-ID: <7e34525c-2e94-1a04-a7f8-3565e40867b4@oracle.com> On 2020-06-12 15:09, Erik Joelsson wrote: > Looks good to me at least. Thank you Erik. Any hotspotters who care to comment? /Magnus > > /Erik > > On 2020-06-12 05:21, Magnus Ihse Bursie wrote: >> From Volker's bug report: >> >> "We are building and linking the libjvm.so on Linux with >> -fnoexceptions because we currently don't use C++ exception handling >> in the HotSpot. >> >> Nevertheless, g++ generates unwind tables (i.e. .eh_frame sections) >> in the object files and shared libraries which can not be stripped >> from the binary. In the case of libjvm.so, these sections consume 10% >> of the whole library. >> >> It is possible to omit the creation of these sections by using the >> '-fno-asynchronous-unwind-tables' option during compilation and >> linking. Ive verified that this indeed reduces the size of libjvm.so >> by 10% on Linux/x86_64 for a product build: >> >> -rwxrwxr-x 1 simonis simonis 18798859 Feb 24 18:32 >> hotspot/linux_amd64_compiler2/product/libjvm.so >> -rwxrwxr-x 1 simonis simonis 17049867 Feb 25 18:12 >> hotspot_no_unwind/linux_amd64_compiler2/product/libjvm.so >> >> The gcc documentation mentions that the unwind information is used >> "for stack unwinding from asynchronous events (such as debugger or >> garbage collector)". But various references [1,2] also mention that >> using '-fno-asynchronous-unwind-tables' together with '-g' will force >> gcc to create this information in the debug sections of the object >> files (i.e. .debug_frame) which can easily be stripped from the >> object files and libraries. >> >> As we build the product version of the libjvm.so with '-g' anyway, >> I'd suggest to use '-fno-asynchronous-unwind-tables' to reduce its size. >> >> I've done some quick tests (debugging, creation of hs_err files) with >> a product version of libjvm.so which was build with >> '-fno-asynchronous-unwind-tables' and couldn't find any draw backs. I >> could observe that all the date from the current .eh_frame sections >> has bee moved to the .debug_frame sections in the stripped out data >> of the libjvm.debuginfo file." >> >> The patch itself is trivial, see below. >> >> Hotspot folks: Are there any reasons why we should not do it? I've >> waited for JDK 16 to push this; if something unexpected turns up >> during the development of JDK 16 (if anything, it's odd corner cases >> that might be a problem), we can always revert this. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8150828 >> Patch inline: >> diff --git a/make/autoconf/flags-cflags.m4 >> b/make/autoconf/flags-cflags.m4 >> --- a/make/autoconf/flags-cflags.m4 >> +++ b/make/autoconf/flags-cflags.m4 >> @@ -442,7 +442,8 @@ >> ?? if test "x$TOOLCHAIN_TYPE" = xgcc || test "x$TOOLCHAIN_TYPE" = >> xclang; then >> ???? # COMMON to gcc and clang >> ???? TOOLCHAIN_CFLAGS_JVM="-pipe -fno-rtti -fno-exceptions \ >> -??????? -fvisibility=hidden -fno-strict-aliasing >> -fno-omit-frame-pointer" >> +??????? -fvisibility=hidden -fno-strict-aliasing >> -fno-omit-frame-pointer \ >> +??????? -fno-asynchronous-unwind-tables" >> ?? fi >> >> ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then >> >> /Magnus From david.holmes at oracle.com Mon Jun 15 13:09:14 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 Jun 2020 23:09:14 +1000 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> Message-ID: <74c08544-8257-8bd9-9f3c-df694fecaea0@oracle.com> On 15/06/2020 6:56 pm, Kim Barrett wrote: >> On Jun 15, 2020, at 2:59 AM, David Holmes wrote: >> >> Hi Kim, >> >> On 15/06/2020 9:15 am, Kim Barrett wrote: >>> Please review this change to oopDesc and Metadata classes, removing >>> unnecessary volatile qualifiers from some member functions. >>> This change also makes virtual overrides of the relevant functions >>> explicitly virtual. >>> While there, removed unnecessary workaround in oopDesc::klass_or_null_acquire. >>> After this change, >>> egrep -r -H --exclude-dir=.hg "\)\s*(const |)\s*volatile" src/hotspot >>> finds nothing; there seem to be no remaining volatile qualified methods. >> >> I must admit I'm quite confused as to why anyone would ever have placed a volatile modifier on a method that returns a constant true or false ??? What could it possibly mean? > > The only thing I can think of is that these were called from other member functions that > were themselves volatile qualified. But there aren?t any of those anymore... Okay. Good to see this weirdness gone! >> On the other hand for a mutable field e.g.: >> >> size_t size() const volatile { return _word_size; } >> >> I can see an expectation that the volatile would force the compiler to always reload _word_size, even if the field itself was not declared volatile (which it isn't). Isn't this how we are now handling things via Atomic::load - where the field is no longer flagged volatile but the load is so that we still tell the compiler not to make assumptions about potentially caching the field? > > The ?preferred" way to do that today would be to make the member volatile and use Atomic > accesses (possibly relaxed if no ordering is needed). Or just rely on the volatile qualified > member (though that?s no longer ?preferred?). (And AtomicValue<> might be used in the > future, instead of a volatile member.) So I'm reminded that there is currently disagreement about what the new "preferred way" for this, but lets leave that for the AtomicValue discussion. :) > But in this case, none of that matters. The _word_size is initialized by the constructor and > is never modified - there are no calls to set_size. I should have removed set_size but > forgot; I?ll fix that now. And it looks like there are a whole bunch of unused or seemingly > useless functions in Metabase. I saw set_size, hence my query, but didn't look that it was actually used :) I'm fine with the current set of changes. Thanks, David ----- >> >> Thanks, >> David >> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8247524 >>> Webrev: >>> https://cr.openjdk.java.net/~kbarrett/8247524/open.00/ >>> Testing: >>> mach5 tier1-3 > > From david.holmes at oracle.com Mon Jun 15 13:15:20 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 Jun 2020 23:15:20 +1000 Subject: RFR: JDK-8150828 Consider using '-fno-asynchronous-unwind-tables' to reduce the size of libjvm.so by 10 percent In-Reply-To: <7e34525c-2e94-1a04-a7f8-3565e40867b4@oracle.com> References: <499b6643-09df-89a3-a790-58839f42a5a9@oracle.com> <7e34525c-2e94-1a04-a7f8-3565e40867b4@oracle.com> Message-ID: <8b349ced-f313-f42c-b3ed-00c9727f4f07@oracle.com> Hi Magnus, On 15/06/2020 10:57 pm, Magnus Ihse Bursie wrote: > On 2020-06-12 15:09, Erik Joelsson wrote: >> Looks good to me at least. > Thank you Erik. > > Any hotspotters who care to comment? I'd like a little more assurance that nothing is broken than just: "I've done some quick tests (debugging, creation of hs_err files)" I don't know whether these tables play any part in stack walking. Thanks, David ----- > > /Magnus >> >> /Erik >> >> On 2020-06-12 05:21, Magnus Ihse Bursie wrote: >>> From Volker's bug report: >>> >>> "We are building and linking the libjvm.so on Linux with >>> -fnoexceptions because we currently don't use C++ exception handling >>> in the HotSpot. >>> >>> Nevertheless, g++ generates unwind tables (i.e. .eh_frame sections) >>> in the object files and shared libraries which can not be stripped >>> from the binary. In the case of libjvm.so, these sections consume 10% >>> of the whole library. >>> >>> It is possible to omit the creation of these sections by using the >>> '-fno-asynchronous-unwind-tables' option during compilation and >>> linking. Ive verified that this indeed reduces the size of libjvm.so >>> by 10% on Linux/x86_64 for a product build: >>> >>> -rwxrwxr-x 1 simonis simonis 18798859 Feb 24 18:32 >>> hotspot/linux_amd64_compiler2/product/libjvm.so >>> -rwxrwxr-x 1 simonis simonis 17049867 Feb 25 18:12 >>> hotspot_no_unwind/linux_amd64_compiler2/product/libjvm.so >>> >>> The gcc documentation mentions that the unwind information is used >>> "for stack unwinding from asynchronous events (such as debugger or >>> garbage collector)". But various references [1,2] also mention that >>> using '-fno-asynchronous-unwind-tables' together with '-g' will force >>> gcc to create this information in the debug sections of the object >>> files (i.e. .debug_frame) which can easily be stripped from the >>> object files and libraries. >>> >>> As we build the product version of the libjvm.so with '-g' anyway, >>> I'd suggest to use '-fno-asynchronous-unwind-tables' to reduce its size. >>> >>> I've done some quick tests (debugging, creation of hs_err files) with >>> a product version of libjvm.so which was build with >>> '-fno-asynchronous-unwind-tables' and couldn't find any draw backs. I >>> could observe that all the date from the current .eh_frame sections >>> has bee moved to the .debug_frame sections in the stripped out data >>> of the libjvm.debuginfo file." >>> >>> The patch itself is trivial, see below. >>> >>> Hotspot folks: Are there any reasons why we should not do it? I've >>> waited for JDK 16 to push this; if something unexpected turns up >>> during the development of JDK 16 (if anything, it's odd corner cases >>> that might be a problem), we can always revert this. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8150828 >>> Patch inline: >>> diff --git a/make/autoconf/flags-cflags.m4 >>> b/make/autoconf/flags-cflags.m4 >>> --- a/make/autoconf/flags-cflags.m4 >>> +++ b/make/autoconf/flags-cflags.m4 >>> @@ -442,7 +442,8 @@ >>> ?? if test "x$TOOLCHAIN_TYPE" = xgcc || test "x$TOOLCHAIN_TYPE" = >>> xclang; then >>> ???? # COMMON to gcc and clang >>> ???? TOOLCHAIN_CFLAGS_JVM="-pipe -fno-rtti -fno-exceptions \ >>> -??????? -fvisibility=hidden -fno-strict-aliasing >>> -fno-omit-frame-pointer" >>> +??????? -fvisibility=hidden -fno-strict-aliasing >>> -fno-omit-frame-pointer \ >>> +??????? -fno-asynchronous-unwind-tables" >>> ?? fi >>> >>> ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then >>> >>> /Magnus > From erik.joelsson at oracle.com Mon Jun 15 13:16:11 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 15 Jun 2020 06:16:11 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: <8e9289a9-7dc7-cde3-cf87-e93580a82fc4@oracle.com> Hello Igor, In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). /Erik On 2020-06-12 21:10, Igor Ignatyev wrote: > testing revealed that LingeredAppTest.java required some love, incremental webrev w/ the fixes for LingeredAppTest -- http://cr.openjdk.java.net/~iignatyev//8211977/webrev.0-1 > > -- Igor > >> On Jun 12, 2020, at 8:38 PM, Igor Ignatyev wrote: >> >> adding build-dev >> >>> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>>> 796 lines changed: 200 ins; 588 del; 8 mod; >>> Hi all, >>> >>> could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? >>> >>> besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also >>> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >>> - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) >>> - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) >>> - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder >>> - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag >>> - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ >>> - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: test/lib-test on {windows,linux,macosx}-x64 >>> >>> Thanks, >>> -- Igor >>> >>> From magnus.ihse.bursie at oracle.com Mon Jun 15 14:34:47 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 15 Jun 2020 16:34:47 +0200 Subject: RFR: JDK-8150828 Consider using '-fno-asynchronous-unwind-tables' to reduce the size of libjvm.so by 10 percent In-Reply-To: <8b349ced-f313-f42c-b3ed-00c9727f4f07@oracle.com> References: <499b6643-09df-89a3-a790-58839f42a5a9@oracle.com> <7e34525c-2e94-1a04-a7f8-3565e40867b4@oracle.com> <8b349ced-f313-f42c-b3ed-00c9727f4f07@oracle.com> Message-ID: On 2020-06-15 15:15, David Holmes wrote: > Hi Magnus, > > On 15/06/2020 10:57 pm, Magnus Ihse Bursie wrote: >> On 2020-06-12 15:09, Erik Joelsson wrote: >>> Looks good to me at least. >> Thank you Erik. >> >> Any hotspotters who care to comment? > > I'd like a little more assurance that nothing is broken than just: > > "I've done some quick tests (debugging, creation of hs_err files)" > > I don't know whether these tables play any part in stack walking. I thought Volker's bug report were quite comprehensive. Unwind tables are only used for C++ exception handling. This is not used by Hotspot. (Why gcc still generates those when they are not needed is a bit of a mystery...) If we add -g (which we do), gcc will create the data needed for debugging in the debug sections, from where we strip it to debuginfo files. If hs_err files are generated correctly, and it is possible to debug hotspot, I think that covers the areas that could be problematic? If anything else can be broken by this, it is likely to be some odd corner-case. That's why I want this integrated at the start of JDK 16, so we have plenty of time to discover any potential issues, and if necessary, revert the change (or fix the problematic case). /Magnus > > Thanks, > David > ----- > >> >> /Magnus >>> >>> /Erik >>> >>> On 2020-06-12 05:21, Magnus Ihse Bursie wrote: >>>> From Volker's bug report: >>>> >>>> "We are building and linking the libjvm.so on Linux with >>>> -fnoexceptions because we currently don't use C++ exception >>>> handling in the HotSpot. >>>> >>>> Nevertheless, g++ generates unwind tables (i.e. .eh_frame sections) >>>> in the object files and shared libraries which can not be stripped >>>> from the binary. In the case of libjvm.so, these sections consume >>>> 10% of the whole library. >>>> >>>> It is possible to omit the creation of these sections by using the >>>> '-fno-asynchronous-unwind-tables' option during compilation and >>>> linking. Ive verified that this indeed reduces the size of >>>> libjvm.so by 10% on Linux/x86_64 for a product build: >>>> >>>> -rwxrwxr-x 1 simonis simonis 18798859 Feb 24 18:32 >>>> hotspot/linux_amd64_compiler2/product/libjvm.so >>>> -rwxrwxr-x 1 simonis simonis 17049867 Feb 25 18:12 >>>> hotspot_no_unwind/linux_amd64_compiler2/product/libjvm.so >>>> >>>> The gcc documentation mentions that the unwind information is used >>>> "for stack unwinding from asynchronous events (such as debugger or >>>> garbage collector)". But various references [1,2] also mention that >>>> using '-fno-asynchronous-unwind-tables' together with '-g' will >>>> force gcc to create this information in the debug sections of the >>>> object files (i.e. .debug_frame) which can easily be stripped from >>>> the object files and libraries. >>>> >>>> As we build the product version of the libjvm.so with '-g' anyway, >>>> I'd suggest to use '-fno-asynchronous-unwind-tables' to reduce its >>>> size. >>>> >>>> I've done some quick tests (debugging, creation of hs_err files) >>>> with a product version of libjvm.so which was build with >>>> '-fno-asynchronous-unwind-tables' and couldn't find any draw backs. >>>> I could observe that all the date from the current .eh_frame >>>> sections has bee moved to the .debug_frame sections in the stripped >>>> out data of the libjvm.debuginfo file." >>>> >>>> The patch itself is trivial, see below. >>>> >>>> Hotspot folks: Are there any reasons why we should not do it? I've >>>> waited for JDK 16 to push this; if something unexpected turns up >>>> during the development of JDK 16 (if anything, it's odd corner >>>> cases that might be a problem), we can always revert this. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8150828 >>>> Patch inline: >>>> diff --git a/make/autoconf/flags-cflags.m4 >>>> b/make/autoconf/flags-cflags.m4 >>>> --- a/make/autoconf/flags-cflags.m4 >>>> +++ b/make/autoconf/flags-cflags.m4 >>>> @@ -442,7 +442,8 @@ >>>> ?? if test "x$TOOLCHAIN_TYPE" = xgcc || test "x$TOOLCHAIN_TYPE" = >>>> xclang; then >>>> ???? # COMMON to gcc and clang >>>> ???? TOOLCHAIN_CFLAGS_JVM="-pipe -fno-rtti -fno-exceptions \ >>>> -??????? -fvisibility=hidden -fno-strict-aliasing >>>> -fno-omit-frame-pointer" >>>> +??????? -fvisibility=hidden -fno-strict-aliasing >>>> -fno-omit-frame-pointer \ >>>> +??????? -fno-asynchronous-unwind-tables" >>>> ?? fi >>>> >>>> ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then >>>> >>>> /Magnus >> From magnus.ihse.bursie at oracle.com Mon Jun 15 15:12:45 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 15 Jun 2020 17:12:45 +0200 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: A few comments: This seems like code copied from elsewhere: 57 # This evaluation is expensive and should only be done if this target was 58 # explicitly called. 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ Please use space after comma. /Magnus On 2020-06-13 05:38, Igor Ignatyev wrote: > adding build-dev > >> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>> 796 lines changed: 200 ins; 588 del; 8 mod; >> Hi all, >> >> could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? >> >> besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also >> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >> - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) >> - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) >> - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder >> - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag >> - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ >> - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >> testing: test/lib-test on {windows,linux,macosx}-x64 >> >> Thanks, >> -- Igor >> >> From vladimir.kozlov at oracle.com Mon Jun 15 17:44:18 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 15 Jun 2020 10:44:18 -0700 Subject: [16] RFR: 8247319: Remove on-stack nmethod hotness counter sampling from safepoints In-Reply-To: References: <8feb88a7-6bab-f694-cf24-2a83c3461f33@oracle.com> <11ecfe57-9142-93e4-d566-57effe29eb84@oracle.com> Message-ID: Good. Thanks, Vladimir On 6/15/20 12:36 AM, Erik ?sterlund wrote: > Hi Vladimir, > > Thank you for reviewing. > > Here is a new webrev with your suggested commend changes and cleanup: > http://cr.openjdk.java.net/~eosterlund/8247319/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00.01/ > > Thanks, > /Erik > > On 2020-06-12 21:35, Vladimir Kozlov wrote: >> I agree with changes. >> >> Please also update comments [1] and clean up leftover [2] from 8244658 [3]. >> >> Thanks, >> Vladimir >> >> [1] http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/runtime/sweeper.hpp#l38 >> http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/code/nmethod.hpp#l260 >> >> [2] http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/runtime/vmOperations.hpp#l113 >> [3] https://bugs.openjdk.java.net/browse/JDK-8244658 >> >> On 6/12/20 12:29 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> The sweeper is moving away from using safepoints for its heuristics. It used to count safepoints to figure out when >>> to sweep, but no longer does that. At the same time, we have for a while been removing more and more safepoints. >>> Safepoints are becoming increasingly rare events, dominated by when we need to GC (GuaranteedSafepointInterval is >>> going to disappear). The frequency of how often we need to GC does not have an obvious connection to how often we >>> need to sweep the code cache... any more. >>> >>> What still remains from the safepoint-based heuristics is the nmethod hotness counter sampling that is performed in >>> safepoint cleanup. I would like to get rid of this. >>> The rationale is that the use of hotness counters is kicking in when the code cache is starting to fill up quite a >>> bit, and there is a need to kill off nmethods heuristically, rather than because they are invalid. But when the code >>> cache fills up, we sweep more and more aggressively. And during these sweeper cycles, we perform nmethod marking >>> using handshakes. That operation also fills in hotness counters for all sampled nmethods. >>> >>> In other words, when there is need for acting on the hotness counters, we are in a state where we may be getting more >>> nmethod hotness counter sample information from the sweeping cycles than we are from safepoint sampling. Conversely, >>> when code cache pressure is high and we need more samples, we might end up getting very few from safepoint based >>> sampling (because the heap is large). The correlation between safepoint frequency and code cache pressure is simply >>> not there any more. And for us to walk all stacks in the system in every single safepoint (which for ZGC is starting >>> to dominate pauses when I remove our stack sampling from safepoints), there better be a really good reason to do this >>> sampling in safepoints. And there simply isn't. So I propose we delete it, in favour of using the hotness counter >>> samples we get from the sweeping cycles instead, that are indeed proportional in frequency, to the code cache pressure. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8247319 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00/ >>> >>> Thanks, >>> /Erik > From stefan.karlsson at oracle.com Mon Jun 15 18:05:50 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 15 Jun 2020 20:05:50 +0200 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> Message-ID: <1d198eaf-7cc7-882f-c53e-3e477f9a0ba4@oracle.com> Hi all, Updated webrevs: https://cr.openjdk.java.net/~stefank/8247470/webrev.02.delta/ https://cr.openjdk.java.net/~stefank/8247470/webrev.02/ Testing showed that the previous patch was incomplete. I changed the signature and expected the compiler to show all places I needed to update. The intention was that the compiler should see that true/false wasn't a MEMFLAGS and complain. Unfortunately, there was a problem with the following pattern: ? new (ResourceObj::C_HEAP, mtCode)GrowableArray(0, true); which matches the following constructor: // Resource allocation GrowableArray(Thread* thread, int initial_size) : GenericGrowableArray(initial_size, 0, mtNone) { and yields thread == NULL (0) and initial_size == 1 (true). There were only two places that used the Thread* parameter. This special constructor seems to have been a performance optimization at some point. But we've made Thread::current() much cheaper, and neither of the affected call sites seems to be performance sensitive. I opted to simplify the code and removed that constructor. The systemDictionaryShared.cpp was needed after I rebased the patch. Thanks, StefanK On 2020-06-12 11:33, Stefan Karlsson wrote: > Hi all, > > Please review this patch to fix the NMT accounting of CHeap > GrowableArray allocations. > > https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247470 > > The initial reason why I started fixing this was that I found code > like this: > > src/hotspot/share/classfile/javaClasses.cpp: > ??? new (ResourceObj::C_HEAP, mtModule) GrowableArray(500, true); > > Before this patch, the GrowableArray constructor looked like this: > > GrowableArray(int initial_size, bool C_heap = false, MEMFLAGS F = > mtInternal) > : GenericGrowableArray(initial_size, 0, C_heap, F) { > _data = (E*)raw_allocate(sizeof(E)); > > and: > > GenericGrowableArray(int initial_size, int initial_len, bool c_heap, > MEMFLAGS flags = mtNone) { _len = initial_len; _max = initial_size; > _memflags = flags; > > and: > void* GenericGrowableArray::raw_allocate(int elementSize) { > ... > return (void*)AllocateHeap(byte_size, _memflags); > > This means that the GrowableArray instance was accounted to mtModule, > but the backing memory for the array was accounted to mtInternal. > > My proposal for fixing this is to fuse the (c_heap, flags) duality > into one parameter (flags). The call sites are then changed like this: > > For resource allocations: > GrowableArray(...) -> GrowableArray(...) > GrowableArray(... /* c_heap */ false) -> GrowableArray(...) > GrowableArray(... /* c_heap */ false, /* flags */ mtXX) -> > GrowableArray(...) > > For CHeap allocations: > GrowableArray(... /* c_heap */ true, /* flags */ mtXX) -> > GrowableArray(..., mtXX) > GrowableArray(... /* c_heap */ true) -> GrowableArray(..., mtInternal) > > That is, if you specify a "valid" mtXX tag it invokes the CHeap > allocation path. If you don't specify a mtXX tag, then 'flags' get the > mtNone default value. This is a marker for the code that no CHeap > allocation should be done. > > While properly tagging all GrowableArray allocations, I went ahead and > changed most GrowableArray mtInternal tags to the appropriate values. > I also introduced a new mtServiceability tag, since a lot of the > mtInternal tags were in areas dealing with serviceability code. I > could split this out into a separate patch, if necessary. > > This hasn't been fully tested yet, because I'd like to get some early > feedback. > > Thanks, > StefanK From coleen.phillimore at oracle.com Mon Jun 15 18:16:38 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 15 Jun 2020 14:16:38 -0400 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: <1d198eaf-7cc7-882f-c53e-3e477f9a0ba4@oracle.com> References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> <1d198eaf-7cc7-882f-c53e-3e477f9a0ba4@oracle.com> Message-ID: <46509fa1-1674-da88-a2d0-2530d7da6d77@oracle.com> Looks good.? Now there are no ambiguous constructors? I was looking to see if the GrowableArray code in os_linux.cpp was in windows/bsd, but it's not but os_windows includes growableArray.hpp.? I wonder if you could try taking it out?? It seems related to this change.? If not, it's not important. thanks, Coleen On 6/15/20 2:05 PM, Stefan Karlsson wrote: > Hi all, > > Updated webrevs: > > https://cr.openjdk.java.net/~stefank/8247470/webrev.02.delta/ > https://cr.openjdk.java.net/~stefank/8247470/webrev.02/ > > Testing showed that the previous patch was incomplete. I changed the > signature and expected the compiler to show all places I needed to > update. The intention was that the compiler should see that true/false > wasn't a MEMFLAGS and complain. Unfortunately, there was a problem > with the following pattern: > > ? new (ResourceObj::C_HEAP, mtCode)GrowableArray(0, true); > > which matches the following constructor: > > // Resource allocation > GrowableArray(Thread* thread, int initial_size) : > GenericGrowableArray(initial_size, 0, mtNone) { > > and yields thread == NULL (0) and initial_size == 1 (true). > > There were only two places that used the Thread* parameter. This > special constructor seems to have been a performance optimization at > some point. But we've made Thread::current() much cheaper, and neither > of the affected call sites seems to be performance sensitive. I opted > to simplify the code and removed that constructor. > > The systemDictionaryShared.cpp was needed after I rebased the patch. > > Thanks, > StefanK > > > On 2020-06-12 11:33, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to fix the NMT accounting of CHeap >> GrowableArray allocations. >> >> https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247470 >> >> The initial reason why I started fixing this was that I found code >> like this: >> >> src/hotspot/share/classfile/javaClasses.cpp: >> ??? new (ResourceObj::C_HEAP, mtModule) GrowableArray(500, >> true); >> >> Before this patch, the GrowableArray constructor looked like this: >> >> GrowableArray(int initial_size, bool C_heap = false, MEMFLAGS F = >> mtInternal) >> : GenericGrowableArray(initial_size, 0, C_heap, F) { >> _data = (E*)raw_allocate(sizeof(E)); >> >> and: >> >> GenericGrowableArray(int initial_size, int initial_len, bool c_heap, >> MEMFLAGS flags = mtNone) { _len = initial_len; _max = initial_size; >> _memflags = flags; >> >> and: >> void* GenericGrowableArray::raw_allocate(int elementSize) { >> ... >> return (void*)AllocateHeap(byte_size, _memflags); >> >> This means that the GrowableArray instance was accounted to mtModule, >> but the backing memory for the array was accounted to mtInternal. >> >> My proposal for fixing this is to fuse the (c_heap, flags) duality >> into one parameter (flags). The call sites are then changed like this: >> >> For resource allocations: >> GrowableArray(...) -> GrowableArray(...) >> GrowableArray(... /* c_heap */ false) -> GrowableArray(...) >> GrowableArray(... /* c_heap */ false, /* flags */ mtXX) -> >> GrowableArray(...) >> >> For CHeap allocations: >> GrowableArray(... /* c_heap */ true, /* flags */ mtXX) -> >> GrowableArray(..., mtXX) >> GrowableArray(... /* c_heap */ true) -> GrowableArray(..., mtInternal) >> >> That is, if you specify a "valid" mtXX tag it invokes the CHeap >> allocation path. If you don't specify a mtXX tag, then 'flags' get >> the mtNone default value. This is a marker for the code that no CHeap >> allocation should be done. >> >> While properly tagging all GrowableArray allocations, I went ahead >> and changed most GrowableArray mtInternal tags to the appropriate >> values. I also introduced a new mtServiceability tag, since a lot of >> the mtInternal tags were in areas dealing with serviceability code. I >> could split this out into a separate patch, if necessary. >> >> This hasn't been fully tested yet, because I'd like to get some early >> feedback. >> >> Thanks, >> StefanK > From erik.osterlund at oracle.com Mon Jun 15 19:29:28 2020 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Mon, 15 Jun 2020 21:29:28 +0200 Subject: [16] RFR: 8247319: Remove on-stack nmethod hotness counter sampling from safepoints In-Reply-To: References: Message-ID: <37B5AE30-C4F5-46E8-99C4-F3CD00627558@oracle.com> Hi Vladimir, Thanks for the review. /Erik > On 15 Jun 2020, at 19:44, Vladimir Kozlov wrote: > > ?Good. > > Thanks, > Vladimir > >> On 6/15/20 12:36 AM, Erik ?sterlund wrote: >> Hi Vladimir, >> Thank you for reviewing. >> Here is a new webrev with your suggested commend changes and cleanup: >> http://cr.openjdk.java.net/~eosterlund/8247319/webrev.01/ >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00.01/ >> Thanks, >> /Erik >>> On 2020-06-12 21:35, Vladimir Kozlov wrote: >>> I agree with changes. >>> >>> Please also update comments [1] and clean up leftover [2] from 8244658 [3]. >>> >>> Thanks, >>> Vladimir >>> >>> [1] http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/runtime/sweeper.hpp#l38 >>> http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/code/nmethod.hpp#l260 >>> >>> [2] http://hg.openjdk.java.net/jdk/jdk/file/09c54ee458aa/src/hotspot/share/runtime/vmOperations.hpp#l113 >>> [3] https://bugs.openjdk.java.net/browse/JDK-8244658 >>> >>> On 6/12/20 12:29 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> The sweeper is moving away from using safepoints for its heuristics. It used to count safepoints to figure out when to sweep, but no longer does that. At the same time, we have for a while been removing more and more safepoints. Safepoints are becoming increasingly rare events, dominated by when we need to GC (GuaranteedSafepointInterval is going to disappear). The frequency of how often we need to GC does not have an obvious connection to how often we need to sweep the code cache... any more. >>>> >>>> What still remains from the safepoint-based heuristics is the nmethod hotness counter sampling that is performed in safepoint cleanup. I would like to get rid of this. >>>> The rationale is that the use of hotness counters is kicking in when the code cache is starting to fill up quite a bit, and there is a need to kill off nmethods heuristically, rather than because they are invalid. But when the code cache fills up, we sweep more and more aggressively. And during these sweeper cycles, we perform nmethod marking using handshakes. That operation also fills in hotness counters for all sampled nmethods. >>>> >>>> In other words, when there is need for acting on the hotness counters, we are in a state where we may be getting more nmethod hotness counter sample information from the sweeping cycles than we are from safepoint sampling. Conversely, when code cache pressure is high and we need more samples, we might end up getting very few from safepoint based sampling (because the heap is large). The correlation between safepoint frequency and code cache pressure is simply not there any more. And for us to walk all stacks in the system in every single safepoint (which for ZGC is starting to dominate pauses when I remove our stack sampling from safepoints), there better be a really good reason to do this sampling in safepoints. And there simply isn't. So I propose we delete it, in favour of using the hotness counter samples we get from the sweeping cycles instead, that are indeed proportional in frequency, to the code cache pressure. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8247319 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8247319/webrev.00/ >>>> >>>> Thanks, >>>> /Erik From stefan.karlsson at oracle.com Mon Jun 15 20:37:14 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 15 Jun 2020 22:37:14 +0200 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: <46509fa1-1674-da88-a2d0-2530d7da6d77@oracle.com> References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> <1d198eaf-7cc7-882f-c53e-3e477f9a0ba4@oracle.com> <46509fa1-1674-da88-a2d0-2530d7da6d77@oracle.com> Message-ID: <0f4f33d3-8b32-61fd-ef04-3bbd80b45290@oracle.com> On 2020-06-15 20:16, coleen.phillimore at oracle.com wrote: > > Looks good.? Now there are no ambiguous constructors? Maybe this one could have been problematic: GrowableArray(Arena* arena, int initial_size, int initial_len, const E& filler) If some code previously did, say: GrowableArray(0, 0, 0, false) and tried to match the old: GrowableArray(int initial_size, int initial_len, const E& filler, bool C_heap = false, MEMFLAGS memflags = mtNone) it could have called the arena version. I temporarily added a dummy parameter like this: GrowableArray(Arena* arena, GADummy dummy, int initial_size, int initial_len, const E& filler) and fixed all call-sites where arena GrowableArrays are used. It compiles on Linux x64. I'll verify that it builds on the other platforms as well. > > I was looking to see if the GrowableArray code in os_linux.cpp was in > windows/bsd, but it's not but os_windows includes growableArray.hpp.? > I wonder if you could try taking it out?? It seems related to this > change.? If not, it's not important. I'll take it out and let the full test of this run overnight. Thanks, StefanK > > thanks, > Coleen > > On 6/15/20 2:05 PM, Stefan Karlsson wrote: >> Hi all, >> >> Updated webrevs: >> >> https://cr.openjdk.java.net/~stefank/8247470/webrev.02.delta/ >> https://cr.openjdk.java.net/~stefank/8247470/webrev.02/ >> >> Testing showed that the previous patch was incomplete. I changed the >> signature and expected the compiler to show all places I needed to >> update. The intention was that the compiler should see that >> true/false wasn't a MEMFLAGS and complain. Unfortunately, there was a >> problem with the following pattern: >> >> ? new (ResourceObj::C_HEAP, mtCode)GrowableArray(0, true); >> >> which matches the following constructor: >> >> // Resource allocation >> GrowableArray(Thread* thread, int initial_size) : >> GenericGrowableArray(initial_size, 0, mtNone) { >> >> and yields thread == NULL (0) and initial_size == 1 (true). >> >> There were only two places that used the Thread* parameter. This >> special constructor seems to have been a performance optimization at >> some point. But we've made Thread::current() much cheaper, and >> neither of the affected call sites seems to be performance sensitive. >> I opted to simplify the code and removed that constructor. >> >> The systemDictionaryShared.cpp was needed after I rebased the patch. >> >> Thanks, >> StefanK >> >> >> On 2020-06-12 11:33, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to fix the NMT accounting of CHeap >>> GrowableArray allocations. >>> >>> https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8247470 >>> >>> The initial reason why I started fixing this was that I found code >>> like this: >>> >>> src/hotspot/share/classfile/javaClasses.cpp: >>> ??? new (ResourceObj::C_HEAP, mtModule) GrowableArray(500, >>> true); >>> >>> Before this patch, the GrowableArray constructor looked like this: >>> >>> GrowableArray(int initial_size, bool C_heap = false, MEMFLAGS F = >>> mtInternal) >>> : GenericGrowableArray(initial_size, 0, C_heap, F) { >>> _data = (E*)raw_allocate(sizeof(E)); >>> >>> and: >>> >>> GenericGrowableArray(int initial_size, int initial_len, bool c_heap, >>> MEMFLAGS flags = mtNone) { _len = initial_len; _max = initial_size; >>> _memflags = flags; >>> >>> and: >>> void* GenericGrowableArray::raw_allocate(int elementSize) { >>> ... >>> return (void*)AllocateHeap(byte_size, _memflags); >>> >>> This means that the GrowableArray instance was accounted to >>> mtModule, but the backing memory for the array was accounted to >>> mtInternal. >>> >>> My proposal for fixing this is to fuse the (c_heap, flags) duality >>> into one parameter (flags). The call sites are then changed like this: >>> >>> For resource allocations: >>> GrowableArray(...) -> GrowableArray(...) >>> GrowableArray(... /* c_heap */ false) -> GrowableArray(...) >>> GrowableArray(... /* c_heap */ false, /* flags */ mtXX) -> >>> GrowableArray(...) >>> >>> For CHeap allocations: >>> GrowableArray(... /* c_heap */ true, /* flags */ mtXX) -> >>> GrowableArray(..., mtXX) >>> GrowableArray(... /* c_heap */ true) -> GrowableArray(..., mtInternal) >>> >>> That is, if you specify a "valid" mtXX tag it invokes the CHeap >>> allocation path. If you don't specify a mtXX tag, then 'flags' get >>> the mtNone default value. This is a marker for the code that no >>> CHeap allocation should be done. >>> >>> While properly tagging all GrowableArray allocations, I went ahead >>> and changed most GrowableArray mtInternal tags to the appropriate >>> values. I also introduced a new mtServiceability tag, since a lot of >>> the mtInternal tags were in areas dealing with serviceability code. >>> I could split this out into a separate patch, if necessary. >>> >>> This hasn't been fully tested yet, because I'd like to get some >>> early feedback. >>> >>> Thanks, >>> StefanK >> > From kim.barrett at oracle.com Mon Jun 15 20:53:42 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jun 2020 16:53:42 -0400 Subject: RFR: 8247524: Remove unnecessary volatile qualifiers from member functions In-Reply-To: <74c08544-8257-8bd9-9f3c-df694fecaea0@oracle.com> References: <36D6EDAC-218A-4572-A737-7591CEEA4413@oracle.com> <153a39d3-3ac6-d5aa-9538-ee2e591555a0@oracle.com> <74c08544-8257-8bd9-9f3c-df694fecaea0@oracle.com> Message-ID: <07D3ECB4-35CD-45AE-A10C-726B8569BC1C@oracle.com> > On Jun 15, 2020, at 9:09 AM, David Holmes wrote: > > On 15/06/2020 6:56 pm, Kim Barrett wrote: >>> On the other hand for a mutable field e.g.: >>> >>> size_t size() const volatile { return _word_size; } >>> >>> I can see an expectation that the volatile would force the compiler to always reload _word_size, even if the field itself was not declared volatile (which it isn't). Isn't this how we are now handling things via Atomic::load - where the field is no longer flagged volatile but the load is so that we still tell the compiler not to make assumptions about potentially caching the field? >> The ?preferred" way to do that today would be to make the member volatile and use Atomic >> accesses (possibly relaxed if no ordering is needed). Or just rely on the volatile qualified >> member (though that?s no longer ?preferred?). (And AtomicValue<> might be used in the >> future, instead of a volatile member.) > > So I'm reminded that there is currently disagreement about what the new "preferred way" for this, but lets leave that for the AtomicValue discussion. :) Hence my use of quotations :) And yes, let?s have that discussion over there. >> But in this case, none of that matters. The _word_size is initialized by the constructor and >> is never modified - there are no calls to set_size. I should have removed set_size but >> forgot; I?ll fix that now. And it looks like there are a whole bunch of unused or seemingly >> useless functions in Metabase. > > I saw set_size, hence my query, but didn't look that it was actually used :) > > I'm fine with the current set of changes. Thanks. From erik.joelsson at oracle.com Mon Jun 15 21:00:25 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 15 Jun 2020 14:00:25 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: On 2020-06-15 08:12, Magnus Ihse Bursie wrote: > A few comments: > > This seems like code copied from elsewhere: > > ? 57 # This evaluation is expensive and should only be done if this > target was > ? 58 # explicitly called. > ? 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) > > I don't agree that this is an expensive evaluation. Furthermore, the > makefile is only called for building the testlib and for making > images, so in worst case it's just the image part that would get a > penalty (although I highly doubt there is any). > This construct was added by me in the original file he copied from, so Igor is just following existing patterns. The SetupTestFilesCompilation calls result in rather big $(shell find...) calls. If you want to keep rebuilds of "make images" as fast as possible, these are certainly relevant optimizations. /Erik > ? 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ > > Please use space after comma. > > /Magnus > > On 2020-06-13 05:38, Igor Ignatyev wrote: >> adding build-dev >> >>> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev >>> wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>>> 796 lines changed: 200 ins; 588 del; 8 mod; >>> Hi all, >>> >>> could you please review this small patch which puts all tests for >>> testlibrary classes into one location under /test/lib-test? >>> >>> besides moving tests from test/jdk/lib/testlibrary and >>> test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also >>> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >>> - introduces make targets to build native parts for the tests in >>> test/lib-test (needed b/c one test has a native part) >>> - adds randomness k/w to test/lib-test (as it's used by >>> RandomGeneratorTest.java) >>> - makes Test class used by TestNativeProcessBuilder a static nested >>> class of TestNativeProcessBuilder >>> - updates LingeredAppTest to use @build instead of @compile and adds >>> necessary @library tag >>> - removes AssertsTest.java, OutputAnalyzerTest.java from >>> test/hotspot/jtreg/testlibrary_tests as they are either identical or >>> lesser that the same tests from test/jdk/lib/testlibrary/ >>> - merges >>> test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and? >>> test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds >>> test cases for `firstMatch` to the superier copy from >>> test/jdk/lib/testlibrary) >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: test/lib-test on {windows,linux,macosx}-x64 >>> >>> Thanks, >>> -- Igor >>> >>> > From mandy.chung at oracle.com Mon Jun 15 21:58:19 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 15 Jun 2020 14:58:19 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records Message-ID: This patch is joint contribution from Christoph Dreis [1] and me. Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 This proposes to make final fields in records notmodifiable via reflection.? Field::set throws IAE if a Field is not modifiable.? Thecurrent specification specifies the following Fields not modifiable: - static final fields in any class - final fields in a hidden class The spec of Field::set is changed to specify that records are not modifiable via reflection. ?Noe that no change in Field::setAccessible(true), i.e. it will succeed to allow existing frameworks to have read access to final fields in records.? Just no write access. VarHandle does not support write access if it's a static final field or an instance final field. This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. No change is made in JNI.? JNI could be considered to disallow modification of final fields in records and hidden classes (static and instance fields).? But JNI has superpower and the current spec already allows to modify the value of a final static field even after it's constant folded (via JNI SetField and SetStaticField), then all bets are off.? This should be re-visited when we consider "final is truly final" for all classes. Make final fields in records not modifiable via reflection enables JIT optimization as these final fields are effectively truly final. This change impacts 3rd-party frameworks including 3rd-party serialization framework that rely on core reflection `setAccessible` or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to construct records but not using the canonical constructor. These frameworks would need to be updated to construct records via its canonical constructor as done by the Java serialization. I see this change gives a good opportunity to engage the maintainers of the serialization frameworks and work together to support new features including records, inline classes and the new serialization mechanism and which I think it is worth the investment. This is a low risk enhancement.? I'd like to request approval for a late enhancement in JDK 15.? It extends the pre-existing code path with refactoring the hidden classes to prepare for new kinds of classes with trusted final fields.? The change is straight-forward. Can this wait to integrate in JDK 16? ? It's important to get this enhancement in when record is a preview feature that we can get feedback and give time to 3rd-party serialization frameworks to look into adding the support for records.? If we delayed this change in 16 and records exit preview, it would be bad for frameworks if they verify that they support records in 15 but fail in 16.? OTOH the risk of this patch is low. Performance Impact I addressed the performance concern I raised earlier myself.? For reflection, VM creates the reflective Field objects and fills in MemberName when resolving a member.? VM will tag if this Field/MemberName is trusted final field.? I think this is a cleaner approach rather than in each place to check for final static and final fields in hidden or record class to determine if it has write access or not. `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) Unsafe has been allowing access of static final fields of any classes but isTrustedFinalField is not limited to instance fields (2) Unsafe disallows access to all fields in a hidden class (not limited to trusted final fields).? So it follows the precedence and simply checks if the declaring class is a record. `Class::isRecord` calls `Class::getSuperclass` to check if it's a subtype of `Record`.? As `Class::getSuperclass` is intrinsified, the call on isRecord on a normal class is fast. Christoph has contributed the microbenchmarks that confirm that no performance regression. Thanks Mandy [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html From igor.ignatyev at oracle.com Tue Jun 16 00:39:38 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 15 Jun 2020 17:39:38 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> Message-ID: <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> @David, Erik, Magnus, please find the answers to your comments at the bottom of this email. @all, David's and Erik's comments made me realize that some parts of the original patch were stashed away and didn't make it to webrev.00. I'm truly sorry for the confusion and inconvenience. I also agree w/ David that it's hard to follow/review, and my gaffe just made it worse, therefore I'd like to start it over again from a clean sate http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 > 833 lines changed: 228 ins; 591 del; 14 mod; could you please review the patch which puts all tests for common testlibrary classes (which is ones located in /test/lib) into one location under /test/lib-test? please note this intentionally doesn't move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . to simplify review, the patch has been split into several webrevs, with each of them accomplishes a more-or-less distinct thing, but is not necessary self-contained: 0. trivial move of tests from test/jdk and test/hotspot/jtreg test suites to test/lib-test: http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ 1. removal of hotspot's AssertsTest, OutputAnalyzerReportingTest and "merge" of hotspot's and jdk's OutputAnalyzerTest: http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ 2. simplification of TestNativeProcessBuilder tests: converts Test class used by TestNativeProcessBuilder into a static nested class: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder 3. makefiles changes to build native parts of lib-test tests. in past, there were no tests w/ native parts in this test suite, TestNativeProcessBuilder is the 1st such test; this part also removes now unneeded lines from hotspot test suite makefile: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest 4. clean ups in hotspot test suite: adjusted location of SimpleClassFileLoadHookTest test, which is a test for hotspot-specific test library (located in /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from TEST.groups: http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ 5. LingeredAppTest fix (which apparently has never been run before): http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ 6. changes to make test/lib-test a fully supported and working test suite, and add in into tier1, includes adding ProblemList, TEST.groups file, and 'randomness' k/w: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 testing: - make test TEST=tier1 locally on macosx - test/lib-test on {windows,linux,macosx}-x64 - tier1 job (in progress) Thanks, -- Igor > On Jun 14, 2020, at 11:23 PM, David Holmes wrote: > <...> > This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. this is intentionally, ctw test-library is hotspot specific, hence its tests are to reside in hotspot test suite (until we decide to move the library to /test/lib), the same is true for SimpleClassFileLoadHookTest. > > You did not modify hotspot/jtreg/TEST.groups but it refers to: > testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ > Plus it should probably refer to the new native_sanity tests somewhere. the actual version of the patch removed reference to TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and had TestNativeProcessBuilder moved to /test/test-lib, so no updates w.r.t. native_sanity are needed > The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. /test/lib-test/TEST.ROOT file was created by JDK-8243010 on 2020-04-16, hence the added copyright has 2020 as the initial copyright year. > You used the wrong license header - makefiles don't use the classpath exception. JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which also have classpath exception. quick grep showed that make directory has 794 files which has '"Classpath" exception', out of 850 which contain 'GNU General Public License version 2' string. And although I agree that makefiles shouldn't have the classpath exception, I'd prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up to build team to decide how it's best to handle. > On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie wrote: > > A few comments: > > This seems like code copied from elsewhere: > 57 # This evaluation is expensive and should only be done if this target was > 58 # explicitly called. > 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) > I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). right, JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which has this comment, I don't know enough details to say if it's expensive or not, yet if you insist and there is a consensus within build team, I can remove the comment. > > 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ > Please use space after comma. this again was preexisting in JtregNativeJdk.gmk, added the space nevertheless. > On Jun 15, 2020, at 6:16 AM, Erik Joelsson wrote: > > In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). the lines were updated to define BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher From david.holmes at oracle.com Tue Jun 16 02:29:55 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Jun 2020 12:29:55 +1000 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: Hi Mandy, Christoph, The code changes here all look okay to me. The idea of introducing the notion of "trusted final fields" seems quite reasonable. I made one editorial comment on the CSR request. I'm unclear if the new TEST.properties file needs a copyright notice and header. We have 706 such files in the repo and 554 (mostly hotspot) do have the copyright notice and header. Thanks, David On 16/06/2020 7:58 am, Mandy Chung wrote: > This patch is joint contribution from Christoph Dreis [1] and me. > > Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 > > This proposes to make final fields in records notmodifiable via > reflection.? Field::set throws IAE if a Field is not modifiable. > Thecurrent specification specifies the following Fields not modifiable: > - static final fields in any class > - final fields in a hidden class > > The spec of Field::set is changed to specify that records are not > modifiable via reflection. > ?Noe that no change in Field::setAccessible(true), i.e. it will succeed > to allow existing frameworks to have read access to final fields in > records.? Just no write access. > > VarHandle does not support write access if it's a static final field or > an instance final field. > > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), then > all bets are off.? This should be re-visited when we consider "final is > truly final" for all classes. > > Make final fields in records not modifiable via reflection enables JIT > optimization as these final fields are effectively truly final. > > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` or > `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. > > I see this change gives a good opportunity to engage the maintainers of > the serialization frameworks and work together to support new features > including records, inline classes and the new serialization mechanism > and which I think it is worth the investment. > > This is a low risk enhancement.? I'd like to request approval for a late > enhancement in JDK 15.? It extends the pre-existing code path with > refactoring the hidden classes to prepare for new kinds of classes with > trusted final fields.? The change is straight-forward. > > Can this wait to integrate in JDK 16? > > ? It's important to get this enhancement in when record is a preview > feature that we can get feedback and give time to 3rd-party > serialization frameworks to look into adding the support for records. If > we delayed this change in 16 and records exit preview, it would be bad > for frameworks if they verify that they support records in 15 but fail > in 16.? OTOH the risk of this patch is low. > > Performance Impact > > I addressed the performance concern I raised earlier myself.? For > reflection, VM creates the reflective Field objects and fills in > MemberName when resolving a member.? VM will tag if this > Field/MemberName is trusted final field.? I think this is a cleaner > approach rather than in each place to check for final static and final > fields in hidden or record class to determine if it has write access or > not. > > `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) > Unsafe has been allowing access of static final fields of any classes > but isTrustedFinalField is not limited to instance fields (2) Unsafe > disallows access to all fields in a hidden class (not limited to trusted > final fields).? So it follows the precedence and simply checks if the > declaring class is a record. `Class::isRecord` calls > `Class::getSuperclass` to check if it's a subtype of `Record`.? As > `Class::getSuperclass` is intrinsified, the call on isRecord on a normal > class is fast. Christoph has contributed the microbenchmarks that > confirm that no performance regression. > > Thanks > Mandy > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html > From david.holmes at oracle.com Tue Jun 16 04:09:48 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Jun 2020 14:09:48 +1000 Subject: RFR: JDK-8150828 Consider using '-fno-asynchronous-unwind-tables' to reduce the size of libjvm.so by 10 percent In-Reply-To: References: <499b6643-09df-89a3-a790-58839f42a5a9@oracle.com> <7e34525c-2e94-1a04-a7f8-3565e40867b4@oracle.com> <8b349ced-f313-f42c-b3ed-00c9727f4f07@oracle.com> Message-ID: <30048f9a-1672-2353-072f-875b0b991c7c@oracle.com> On 16/06/2020 12:34 am, Magnus Ihse Bursie wrote: > On 2020-06-15 15:15, David Holmes wrote: >> Hi Magnus, >> >> On 15/06/2020 10:57 pm, Magnus Ihse Bursie wrote: >>> On 2020-06-12 15:09, Erik Joelsson wrote: >>>> Looks good to me at least. >>> Thank you Erik. >>> >>> Any hotspotters who care to comment? >> >> I'd like a little more assurance that nothing is broken than just: >> >> "I've done some quick tests (debugging, creation of hs_err files)" >> >> I don't know whether these tables play any part in stack walking. > I thought Volker's bug report were quite comprehensive. > > Unwind tables are only used for C++ exception handling. This is not used > by Hotspot. (Why gcc still generates those when they are not needed is a > bit of a mystery...) Part of my concern as well. It was not clear from the bug report that these are _only_ used by C++ exceptions, otherwise why would gcc generate them when we have C++ exceptions disabled? > If we add -g (which we do), gcc will create the data needed for > debugging in the debug sections, from where we strip it to debuginfo files. And in a non-debug build what does a stacktrace look like? It is correct but non-symbolic? > If hs_err files are generated correctly, and it is possible to debug > hotspot, I think that covers the areas that could be problematic? Yes that is the requirement, but I'm not clear to what extent these requirements have been verified. Have different hs_err generation contexts been examined or only the synchronous ones produced by the test flag? What kind of debugging has been checked? > If anything else can be broken by this, it is likely to be some odd > corner-case. That's why I want this integrated at the start of JDK 16, > so we have plenty of time to discover any potential issues, and if > necessary, revert the change (or fix the problematic case). We obviously have a different view on where the pre-integration testing bar should be set. Please at least ensure mach5 tiers 1-3 pass before pushing. Thanks, David > /Magnus >> >> Thanks, >> David >> ----- >> >>> >>> /Magnus >>>> >>>> /Erik >>>> >>>> On 2020-06-12 05:21, Magnus Ihse Bursie wrote: >>>>> From Volker's bug report: >>>>> >>>>> "We are building and linking the libjvm.so on Linux with >>>>> -fnoexceptions because we currently don't use C++ exception >>>>> handling in the HotSpot. >>>>> >>>>> Nevertheless, g++ generates unwind tables (i.e. .eh_frame sections) >>>>> in the object files and shared libraries which can not be stripped >>>>> from the binary. In the case of libjvm.so, these sections consume >>>>> 10% of the whole library. >>>>> >>>>> It is possible to omit the creation of these sections by using the >>>>> '-fno-asynchronous-unwind-tables' option during compilation and >>>>> linking. Ive verified that this indeed reduces the size of >>>>> libjvm.so by 10% on Linux/x86_64 for a product build: >>>>> >>>>> -rwxrwxr-x 1 simonis simonis 18798859 Feb 24 18:32 >>>>> hotspot/linux_amd64_compiler2/product/libjvm.so >>>>> -rwxrwxr-x 1 simonis simonis 17049867 Feb 25 18:12 >>>>> hotspot_no_unwind/linux_amd64_compiler2/product/libjvm.so >>>>> >>>>> The gcc documentation mentions that the unwind information is used >>>>> "for stack unwinding from asynchronous events (such as debugger or >>>>> garbage collector)". But various references [1,2] also mention that >>>>> using '-fno-asynchronous-unwind-tables' together with '-g' will >>>>> force gcc to create this information in the debug sections of the >>>>> object files (i.e. .debug_frame) which can easily be stripped from >>>>> the object files and libraries. >>>>> >>>>> As we build the product version of the libjvm.so with '-g' anyway, >>>>> I'd suggest to use '-fno-asynchronous-unwind-tables' to reduce its >>>>> size. >>>>> >>>>> I've done some quick tests (debugging, creation of hs_err files) >>>>> with a product version of libjvm.so which was build with >>>>> '-fno-asynchronous-unwind-tables' and couldn't find any draw backs. >>>>> I could observe that all the date from the current .eh_frame >>>>> sections has bee moved to the .debug_frame sections in the stripped >>>>> out data of the libjvm.debuginfo file." >>>>> >>>>> The patch itself is trivial, see below. >>>>> >>>>> Hotspot folks: Are there any reasons why we should not do it? I've >>>>> waited for JDK 16 to push this; if something unexpected turns up >>>>> during the development of JDK 16 (if anything, it's odd corner >>>>> cases that might be a problem), we can always revert this. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8150828 >>>>> Patch inline: >>>>> diff --git a/make/autoconf/flags-cflags.m4 >>>>> b/make/autoconf/flags-cflags.m4 >>>>> --- a/make/autoconf/flags-cflags.m4 >>>>> +++ b/make/autoconf/flags-cflags.m4 >>>>> @@ -442,7 +442,8 @@ >>>>> ?? if test "x$TOOLCHAIN_TYPE" = xgcc || test "x$TOOLCHAIN_TYPE" = >>>>> xclang; then >>>>> ???? # COMMON to gcc and clang >>>>> ???? TOOLCHAIN_CFLAGS_JVM="-pipe -fno-rtti -fno-exceptions \ >>>>> -??????? -fvisibility=hidden -fno-strict-aliasing >>>>> -fno-omit-frame-pointer" >>>>> +??????? -fvisibility=hidden -fno-strict-aliasing >>>>> -fno-omit-frame-pointer \ >>>>> +??????? -fno-asynchronous-unwind-tables" >>>>> ?? fi >>>>> >>>>> ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then >>>>> >>>>> /Magnus >>> > From david.holmes at oracle.com Tue Jun 16 07:14:59 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Jun 2020 17:14:59 +1000 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: Hi Igor, On 16/06/2020 10:39 am, Igor Ignatyev wrote: > @David, Erik, Magnus, > > please find the answers to your comments at the bottom of this email. > > @all, > > David's and Erik's comments made me realize that some parts of the > original patch were stashed away and didn't make it to webrev.00. I'm > truly sorry for the confusion and inconvenience. I also agree w/ David > that it's hard to follow/review, and my?gaffe just made it worse, > therefore I'd like to start it over again from a clean sate > > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >> 833 lines changed: 228 ins; 591 del; 14 mod; > > could you please review the patch which puts all tests for common > testlibrary classes (which is ones located in /test/lib) into one > location under /test/lib-test? please note this intentionally doesn't > move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific > testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . Ok. > to simplify review, the patch has been split into several webrevs, with > each of them accomplishes a more-or-less distinct thing, but is not > necessary self-contained: Many thanks for doing this! > 0. trivial move of tests from test/jdk and test/hotspot/jtreg test > suites to test/lib-test: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ Looks good. > 1. removal of hotspot's AssertsTest,?OutputAnalyzerReportingTest and > "merge" of hotspot's and jdk's OutputAnalyzerTest: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ Looks good. > 2. simplification of TestNativeProcessBuilder tests: converts Test class > used by TestNativeProcessBuilder into a static nested class: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder Looks good. > 3. makefiles changes to build native parts of lib-test tests. in past, > there were no tests w/ native parts in this test suite, > TestNativeProcessBuilder is the 1st such test; this part also removes > now unneeded lines from hotspot test suite makefile: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest Hmm okay. Not sure this needed its own category of build logic but ... Aside: Makefiles should not have the classpath exception version of the license header. But they all do for some reason. I'll check if we need to do a separate cleanup of that. > 4. clean ups in hotspot test suite: adjusted location > of?SimpleClassFileLoadHookTest test, which is a test for > hotspot-specific test library (located in > /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to > /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from > TEST.groups: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ Looks good. Took me a while to understand the connection to the test library here. > 5. LingeredAppTest fix (which apparently has never been run before): > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ Someone from serviceability should evaluate this test. It may not be needed. > 6. changes to make test/lib-test a fully supported and working test > suite, and add in into tier1, ?includes adding ProblemList, TEST.groups > file, and 'randomness' k/w: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ Seems okay. Thanks, David ----- > webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 > JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 > testing: > ?- make test TEST=tier1 locally on macosx > ?- test/lib-test on ?{windows,linux,macosx}-x64 > ?- tier1 job (in progress) > > Thanks, > -- Igor > > >> On Jun 14, 2020, at 11:23 PM, David Holmes > > wrote: >> <...> >> This doesn't seem to move everything under >> test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. > this is intentionally, ctw test-library is hotspot specific, hence its > tests are to reside in hotspot test suite (until we decide to move the > library to /test/lib), the same is true for?SimpleClassFileLoadHookTest. >> >> You did not modify hotspot/jtreg/TEST.groups but it refers to: >> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >> Plus it should probably refer to the new native_sanity tests somewhere. > the actual version of the patch removed reference to > TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and > had TestNativeProcessBuilder moved to /test/test-lib, so no updates > w.r.t. native_sanity are needed > >> The newly added copyright notice needs to have the correct initial >> copyright year based on when the file was first added to the repo. > ?/test/lib-test/TEST.ROOT file was created by JDK-8243010 on > 2020-04-16, hence the added copyright has 2020 as the initial copyright > year. > >> You used the wrong license header - makefiles don't use the classpath >> exception. > JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk which > also have classpath exception. quick grep showed that make directory has > 794 files which has '"Classpath" exception', out of 850 which contain > 'GNU General Public License version 2' string. And although I agree that > makefiles shouldn't have the classpath exception, I'd prefer to keep > JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up > to build team to decide how it's best to handle. > > >> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie >> > >> wrote: >> >> A few comments: >> >> This seems like code copied from elsewhere: >> 57 # This evaluation is expensive and should only be done if this target was >> 58 # explicitly called. >> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >> I don't agree that this is an expensive evaluation. Furthermore, the >> makefile is only called for building the testlib and for making >> images, so in worst case it's just the image part that would get a >> penalty (although I highly doubt there is any). > right, JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk > which has this comment, I don't know enough details to say if it's > expensive or not, yet if you insist and there is a consensus within > build team, I can remove the comment. > >> >> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >> Please use space after comma. > this again was preexisting in?JtregNativeJdk.gmk, added the space > nevertheless. > > >> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > > wrote: >> >> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or >> adjusted if linking to libjvm is actually needed). > the lines were updated to define > BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher > > From stefan.karlsson at oracle.com Tue Jun 16 07:40:36 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 16 Jun 2020 09:40:36 +0200 Subject: RFR: 8247470: Fix CHeap GrowableArray NMT accounting In-Reply-To: <0f4f33d3-8b32-61fd-ef04-3bbd80b45290@oracle.com> References: <735e538f-d91c-17b7-3f8a-943abecca068@oracle.com> <1d198eaf-7cc7-882f-c53e-3e477f9a0ba4@oracle.com> <46509fa1-1674-da88-a2d0-2530d7da6d77@oracle.com> <0f4f33d3-8b32-61fd-ef04-3bbd80b45290@oracle.com> Message-ID: Tier1 testing is complete, a large part of tier2 and tier3 is complete but some platforms are blocked up. I think this is enough testing of this, and I'm going to push this now. StefanK On 2020-06-15 22:37, Stefan Karlsson wrote: > On 2020-06-15 20:16, coleen.phillimore at oracle.com wrote: >> >> Looks good.? Now there are no ambiguous constructors? > > Maybe this one could have been problematic: > GrowableArray(Arena* arena, int initial_size, int initial_len, const > E& filler) > > If some code previously did, say: > GrowableArray(0, 0, 0, false) > > and tried to match the old: > GrowableArray(int initial_size, int initial_len, const E& filler, bool > C_heap = false, MEMFLAGS memflags = mtNone) > > it could have called the arena version. > > I temporarily added a dummy parameter like this: > GrowableArray(Arena* arena, GADummy dummy, int initial_size, int > initial_len, const E& filler) > > and fixed all call-sites where arena GrowableArrays are used. It > compiles on Linux x64. I'll verify that it builds on the other > platforms as well. > >> >> I was looking to see if the GrowableArray code in os_linux.cpp was in >> windows/bsd, but it's not but os_windows includes growableArray.hpp.? >> I wonder if you could try taking it out?? It seems related to this >> change.? If not, it's not important. > > I'll take it out and let the full test of this run overnight. > > Thanks, > StefanK >> >> thanks, >> Coleen >> >> On 6/15/20 2:05 PM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Updated webrevs: >>> >>> https://cr.openjdk.java.net/~stefank/8247470/webrev.02.delta/ >>> https://cr.openjdk.java.net/~stefank/8247470/webrev.02/ >>> >>> Testing showed that the previous patch was incomplete. I changed the >>> signature and expected the compiler to show all places I needed to >>> update. The intention was that the compiler should see that >>> true/false wasn't a MEMFLAGS and complain. Unfortunately, there was >>> a problem with the following pattern: >>> >>> ? new (ResourceObj::C_HEAP, mtCode)GrowableArray(0, true); >>> >>> which matches the following constructor: >>> >>> // Resource allocation >>> GrowableArray(Thread* thread, int initial_size) : >>> GenericGrowableArray(initial_size, 0, mtNone) { >>> >>> and yields thread == NULL (0) and initial_size == 1 (true). >>> >>> There were only two places that used the Thread* parameter. This >>> special constructor seems to have been a performance optimization at >>> some point. But we've made Thread::current() much cheaper, and >>> neither of the affected call sites seems to be performance >>> sensitive. I opted to simplify the code and removed that constructor. >>> >>> The systemDictionaryShared.cpp was needed after I rebased the patch. >>> >>> Thanks, >>> StefanK >>> >>> >>> On 2020-06-12 11:33, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to fix the NMT accounting of CHeap >>>> GrowableArray allocations. >>>> >>>> https://cr.openjdk.java.net/~stefank/8247470/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8247470 >>>> >>>> The initial reason why I started fixing this was that I found code >>>> like this: >>>> >>>> src/hotspot/share/classfile/javaClasses.cpp: >>>> ??? new (ResourceObj::C_HEAP, mtModule) GrowableArray(500, >>>> true); >>>> >>>> Before this patch, the GrowableArray constructor looked like this: >>>> >>>> GrowableArray(int initial_size, bool C_heap = false, MEMFLAGS F = >>>> mtInternal) >>>> : GenericGrowableArray(initial_size, 0, C_heap, F) { >>>> _data = (E*)raw_allocate(sizeof(E)); >>>> >>>> and: >>>> >>>> GenericGrowableArray(int initial_size, int initial_len, bool >>>> c_heap, MEMFLAGS flags = mtNone) { _len = initial_len; _max = >>>> initial_size; _memflags = flags; >>>> >>>> and: >>>> void* GenericGrowableArray::raw_allocate(int elementSize) { >>>> ... >>>> return (void*)AllocateHeap(byte_size, _memflags); >>>> >>>> This means that the GrowableArray instance was accounted to >>>> mtModule, but the backing memory for the array was accounted to >>>> mtInternal. >>>> >>>> My proposal for fixing this is to fuse the (c_heap, flags) duality >>>> into one parameter (flags). The call sites are then changed like this: >>>> >>>> For resource allocations: >>>> GrowableArray(...) -> GrowableArray(...) >>>> GrowableArray(... /* c_heap */ false) -> GrowableArray(...) >>>> GrowableArray(... /* c_heap */ false, /* flags */ mtXX) -> >>>> GrowableArray(...) >>>> >>>> For CHeap allocations: >>>> GrowableArray(... /* c_heap */ true, /* flags */ mtXX) -> >>>> GrowableArray(..., mtXX) >>>> GrowableArray(... /* c_heap */ true) -> GrowableArray(..., mtInternal) >>>> >>>> That is, if you specify a "valid" mtXX tag it invokes the CHeap >>>> allocation path. If you don't specify a mtXX tag, then 'flags' get >>>> the mtNone default value. This is a marker for the code that no >>>> CHeap allocation should be done. >>>> >>>> While properly tagging all GrowableArray allocations, I went ahead >>>> and changed most GrowableArray mtInternal tags to the appropriate >>>> values. I also introduced a new mtServiceability tag, since a lot >>>> of the mtInternal tags were in areas dealing with serviceability >>>> code. I could split this out into a separate patch, if necessary. >>>> >>>> This hasn't been fully tested yet, because I'd like to get some >>>> early feedback. >>>> >>>> Thanks, >>>> StefanK >>> >> > From forax at univ-mlv.fr Tue Jun 16 09:49:19 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 16 Jun 2020 11:49:19 +0200 (CEST) Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <1229321025.1095655.1592300959835.JavaMail.zimbra@u-pem.fr> Hi Mndy, Looks good, I don't like too much the fact that there is a new boolean field in j.l.r.Field instead of using the modifiers like in the VM counterpart, but it will require masking and unmasking the modifiers which is a far bigger change, too big to worth it in my opinion. So +1 R?mi ----- Mail original ----- > De: "mandy chung" > ?: "hotspot-dev" , "core-libs-dev" , "amber-dev" > > Envoy?: Lundi 15 Juin 2020 23:58:19 > Objet: (15) RFR: JDK-8247444: Trust final fields in records > This patch is joint contribution from Christoph Dreis [1] and me. > > Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 > > This proposes to make final fields in records notmodifiable via > reflection.? Field::set throws IAE if a Field is not modifiable. > Thecurrent specification specifies the following Fields not modifiable: > - static final fields in any class > - final fields in a hidden class > > The spec of Field::set is changed to specify that records are not > modifiable via reflection. > ?Noe that no change in Field::setAccessible(true), i.e. it will succeed > to allow existing frameworks to have read access to final fields in > records.? Just no write access. > > VarHandle does not support write access if it's a static final field or > an instance final field. > > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), then > all bets are off.? This should be re-visited when we consider "final is > truly final" for all classes. > > Make final fields in records not modifiable via reflection enables JIT > optimization as these final fields are effectively truly final. > > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` or > `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. > > I see this change gives a good opportunity to engage the maintainers of > the serialization frameworks and work together to support new features > including records, inline classes and the new serialization mechanism > and which I think it is worth the investment. > > This is a low risk enhancement.? I'd like to request approval for a late > enhancement in JDK 15.? It extends the pre-existing code path with > refactoring the hidden classes to prepare for new kinds of classes with > trusted final fields.? The change is straight-forward. > > Can this wait to integrate in JDK 16? > > ? It's important to get this enhancement in when record is a preview > feature that we can get feedback and give time to 3rd-party > serialization frameworks to look into adding the support for records. > If we delayed this change in 16 and records exit preview, it would be > bad for frameworks if they verify that they support records in 15 but > fail in 16.? OTOH the risk of this patch is low. > > Performance Impact > > I addressed the performance concern I raised earlier myself.? For > reflection, VM creates the reflective Field objects and fills in > MemberName when resolving a member.? VM will tag if this > Field/MemberName is trusted final field.? I think this is a cleaner > approach rather than in each place to check for final static and final > fields in hidden or record class to determine if it has write access or > not. > > `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) > Unsafe has been allowing access of static final fields of any classes > but isTrustedFinalField is not limited to instance fields (2) Unsafe > disallows access to all fields in a hidden class (not limited to trusted > final fields).? So it follows the precedence and simply checks if the > declaring class is a record. `Class::isRecord` calls > `Class::getSuperclass` to check if it's a subtype of `Record`.? As > `Class::getSuperclass` is intrinsified, the call on isRecord on a normal > class is fast. Christoph has contributed the microbenchmarks that > confirm that no performance regression. > > Thanks > Mandy > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html From erik.joelsson at oracle.com Tue Jun 16 13:06:49 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 16 Jun 2020 06:06:49 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: On 2020-06-15 17:39, Igor Ignatyev wrote: > @David, Erik, Magnus, > > please find the answers to your comments at the bottom of this email. > > @all, > > David's and Erik's comments made me realize that some parts of the > original patch were stashed away and didn't make it to webrev.00. I'm > truly sorry for the confusion and inconvenience. I also agree w/ David > that it's hard to follow/review, and my?gaffe just made it worse, > therefore I'd like to start it over again from a clean sate > > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 Build changes look good to me now, except for a missing newline in Main.gmk. (No need for new review.) /Erik >> 833 lines changed: 228 ins; 591 del; 14 mod; > > could you please review the patch which puts all tests for common > testlibrary classes (which is ones located in /test/lib) into one > location under /test/lib-test? please note this intentionally doesn't > move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific > testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . > > to simplify review, the patch has been split into several webrevs, > with each of them accomplishes a more-or-less distinct thing, but is > not necessary self-contained: > > 0. trivial move of tests from test/jdk and test/hotspot/jtreg test > suites to test/lib-test: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ > > 1. removal of hotspot's AssertsTest,?OutputAnalyzerReportingTest and > "merge" of hotspot's and jdk's OutputAnalyzerTest: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ > > 2. simplification of TestNativeProcessBuilder tests: converts Test > class used by TestNativeProcessBuilder into a static nested class: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder > > 3. makefiles changes to build native parts of lib-test tests. in past, > there were no tests w/ native parts in this test suite, > TestNativeProcessBuilder is the 1st such test; this part also removes > now unneeded lines from hotspot test suite makefile: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest > > 4. clean ups in hotspot test suite: adjusted location > of?SimpleClassFileLoadHookTest test, which is a test for > hotspot-specific test library (located in > /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved > to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates > from TEST.groups: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ > > 5. LingeredAppTest fix (which apparently has never been run before): > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ > > 6. changes to make test/lib-test a fully supported and working test > suite, and add in into tier1, ?includes adding ProblemList, > TEST.groups file, and 'randomness' k/w: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ > > webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 > JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 > testing: > ?- make test TEST=tier1 locally on macosx > ?- test/lib-test on ?{windows,linux,macosx}-x64 > ?- tier1 job (in progress) > > Thanks, > -- Igor > > >> On Jun 14, 2020, at 11:23 PM, David Holmes > > wrote: >> <...> >> This doesn't seem to move everything under >> test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. > this is intentionally, ctw test-library is hotspot specific, hence its > tests are to reside in hotspot test suite (until we decide to move the > library to /test/lib), the same is true for?SimpleClassFileLoadHookTest. >> >> You did not modify hotspot/jtreg/TEST.groups but it refers to: >> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >> Plus it should probably refer to the new native_sanity tests somewhere. > the actual version of the patch removed reference to > TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and > had TestNativeProcessBuilder moved to /test/test-lib, so no updates > w.r.t. native_sanity are needed > >> The newly added copyright notice needs to have the correct initial >> copyright year based on when the file was first added to the repo. > ?/test/lib-test/TEST.ROOT file was created by JDK-8243010 on > 2020-04-16, hence the added copyright has 2020 as the initial > copyright year. > >> You used the wrong license header - makefiles don't use the classpath >> exception. > JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk which > also have classpath exception. quick grep showed that make directory > has 794 files which has '"Classpath" exception', out of 850 which > contain 'GNU General Public License version 2' string. And although I > agree that makefiles shouldn't have the classpath exception, I'd > prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and > would leave it up to build team to decide how it's best to handle. > > >> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie >> > > wrote: >> >> A few comments: >> >> This seems like code copied from elsewhere: >> 57 # This evaluation is expensive and should only be done if this target was >> 58 # explicitly called. >> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >> I don't agree that this is an expensive evaluation. Furthermore, the >> makefile is only called for building the testlib and for making >> images, so in worst case it's just the image part that would get a >> penalty (although I highly doubt there is any). > right, JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk > which has this comment, I don't know enough details to say if it's > expensive or not, yet if you insist and there is a consensus within > build team, I can remove the comment. > >> >> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >> Please use space after comma. > this again was preexisting in?JtregNativeJdk.gmk, added the space > nevertheless. > > >> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > > wrote: >> >> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or >> adjusted if linking to libjvm is actually needed). > the lines were updated to define > BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher > > From magnus.ihse.bursie at oracle.com Tue Jun 16 14:03:28 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 16 Jun 2020 16:03:28 +0200 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: On 2020-06-16 15:06, Erik Joelsson wrote: > > > On 2020-06-15 17:39, Igor Ignatyev wrote: >> @David, Erik, Magnus, >> >> please find the answers to your comments at the bottom of this email. >> >> @all, >> >> David's and Erik's comments made me realize that some parts of the >> original patch were stashed away and didn't make it to webrev.00. I'm >> truly sorry for the confusion and inconvenience. I also agree w/ >> David that it's hard to follow/review, and my?gaffe just made it >> worse, therefore I'd like to start it over again from a clean sate >> >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 > > Build changes look good to me now, except for a missing newline in > Main.gmk. (No need for new review.) > Ditto. /Magnus > > /Erik > >>> 833 lines changed: 228 ins; 591 del; 14 mod; >> >> could you please review the patch which puts all tests for common >> testlibrary classes (which is ones located in /test/lib) into one >> location under /test/lib-test? please note this intentionally doesn't >> move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific >> testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . >> >> to simplify review, the patch has been split into several webrevs, >> with each of them accomplishes a more-or-less distinct thing, but is >> not necessary self-contained: >> >> 0. trivial move of tests from test/jdk and test/hotspot/jtreg test >> suites to test/lib-test: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ >> >> 1. removal of hotspot's AssertsTest,?OutputAnalyzerReportingTest and >> "merge" of hotspot's and jdk's OutputAnalyzerTest: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ >> >> 2. simplification of TestNativeProcessBuilder tests: converts Test >> class used by TestNativeProcessBuilder into a static nested class: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder >> >> 3. makefiles changes to build native parts of lib-test tests. in >> past, there were no tests w/ native parts in this test suite, >> TestNativeProcessBuilder is the 1st such test; this part also removes >> now unneeded lines from hotspot test suite makefile: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest >> >> 4. clean ups in hotspot test suite: adjusted location >> of?SimpleClassFileLoadHookTest test, which is a test for >> hotspot-specific test library (located in >> /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved >> to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates >> from TEST.groups: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ >> >> 5. LingeredAppTest fix (which apparently has never been run before): >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ >> >> 6. changes to make test/lib-test a fully supported and working test >> suite, and add in into tier1, ?includes adding ProblemList, >> TEST.groups file, and 'randomness' k/w: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >> testing: >> ?- make test TEST=tier1 locally on macosx >> ?- test/lib-test on ?{windows,linux,macosx}-x64 >> ?- tier1 job (in progress) >> >> Thanks, >> -- Igor >> >> >>> On Jun 14, 2020, at 11:23 PM, David Holmes >> > wrote: >>> <...> >>> This doesn't seem to move everything under >>> test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. >> this is intentionally, ctw test-library is hotspot specific, hence >> its tests are to reside in hotspot test suite (until we decide to >> move the library to /test/lib), the same is true >> for?SimpleClassFileLoadHookTest. >>> >>> You did not modify hotspot/jtreg/TEST.groups but it refers to: >>> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >>> Plus it should probably refer to the new native_sanity tests somewhere. >> the actual version of the patch removed reference to >> TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups >> and had TestNativeProcessBuilder moved to /test/test-lib, so no >> updates w.r.t. native_sanity are needed >> >>> The newly added copyright notice needs to have the correct initial >>> copyright year based on when the file was first added to the repo. >> ?/test/lib-test/TEST.ROOT file was created by JDK-8243010 on >> 2020-04-16, hence the added copyright has 2020 as the initial >> copyright year. >> >>> You used the wrong license header - makefiles don't use the >>> classpath exception. >> JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk which >> also have classpath exception. quick grep showed that make directory >> has 794 files which has '"Classpath" exception', out of 850 which >> contain 'GNU General Public License version 2' string. And although I >> agree that makefiles shouldn't have the classpath exception, I'd >> prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and >> would leave it up to build team to decide how it's best to handle. >> >> >>> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie >>> >> > wrote: >>> >>> A few comments: >>> >>> This seems like code copied from elsewhere: >>> 57 # This evaluation is expensive and should only be done if this target was >>> 58 # explicitly called. >>> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >>> I don't agree that this is an expensive evaluation. Furthermore, the >>> makefile is only called for building the testlib and for making >>> images, so in worst case it's just the image part that would get a >>> penalty (although I highly doubt there is any). >> right, JtregNativeLibTest.gmk is based >> on?make/test/JtregNativeJdk.gmk which has this comment, I don't know >> enough details to say if it's expensive or not, yet if you insist and >> there is a consensus within build team, I can remove the comment. >> >>> >>> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >>> Please use space after comma. >> this again was preexisting in?JtregNativeJdk.gmk, added the space >> nevertheless. >> >> >>> On Jun 15, 2020, at 6:16 AM, Erik Joelsson >> > wrote: >>> >>> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed >>> (or adjusted if linking to libjvm is actually needed). >> the lines were updated to define >> BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher >> >> From mandy.chung at oracle.com Tue Jun 16 16:42:45 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 Jun 2020 09:42:45 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <1229321025.1095655.1592300959835.JavaMail.zimbra@u-pem.fr> References: <1229321025.1095655.1592300959835.JavaMail.zimbra@u-pem.fr> Message-ID: <97446f83-1e00-13bd-14f2-d32620ff072f@oracle.com> On 6/16/20 2:49 AM, Remi Forax wrote: > Hi Mndy, > Looks good, > I don't like too much the fact that there is a new boolean field in j.l.r.Field instead of using the modifiers like in the VM counterpart, > but it will require masking and unmasking the modifiers which is a far bigger change, too big to worth it in my opinion. Adding a new field instead of using the modifiers is a deliberate choice from my side.? It's higher risk to overload the modifiers to include VM-specific flags.?? I am all for a better clean up (valhalla lworld does use the modifiers to cover a flag indicating if it's flattened which I am not satisfied either). > So +1 Thanks Remi. Mandy > R?mi > > ----- Mail original ----- >> De: "mandy chung" >> ?: "hotspot-dev" , "core-libs-dev" , "amber-dev" >> >> Envoy?: Lundi 15 Juin 2020 23:58:19 >> Objet: (15) RFR: JDK-8247444: Trust final fields in records >> This patch is joint contribution from Christoph Dreis [1] and me. >> >> Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >> >> This proposes to make final fields in records notmodifiable via >> reflection.? Field::set throws IAE if a Field is not modifiable. >> Thecurrent specification specifies the following Fields not modifiable: >> - static final fields in any class >> - final fields in a hidden class >> >> The spec of Field::set is changed to specify that records are not >> modifiable via reflection. >> ?Noe that no change in Field::setAccessible(true), i.e. it will succeed >> to allow existing frameworks to have read access to final fields in >> records.? Just no write access. >> >> VarHandle does not support write access if it's a static final field or >> an instance final field. >> >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >> >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static and >> instance fields).? But JNI has superpower and the current spec already >> allows to modify the value of a final static field even after it's >> constant folded (via JNI SetField and SetStaticField), then >> all bets are off.? This should be re-visited when we consider "final is >> truly final" for all classes. >> >> Make final fields in records not modifiable via reflection enables JIT >> optimization as these final fields are effectively truly final. >> >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` or >> `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via its >> canonical constructor as done by the Java serialization. >> >> I see this change gives a good opportunity to engage the maintainers of >> the serialization frameworks and work together to support new features >> including records, inline classes and the new serialization mechanism >> and which I think it is worth the investment. >> >> This is a low risk enhancement.? I'd like to request approval for a late >> enhancement in JDK 15.? It extends the pre-existing code path with >> refactoring the hidden classes to prepare for new kinds of classes with >> trusted final fields.? The change is straight-forward. >> >> Can this wait to integrate in JDK 16? >> >> ? It's important to get this enhancement in when record is a preview >> feature that we can get feedback and give time to 3rd-party >> serialization frameworks to look into adding the support for records. >> If we delayed this change in 16 and records exit preview, it would be >> bad for frameworks if they verify that they support records in 15 but >> fail in 16.? OTOH the risk of this patch is low. >> >> Performance Impact >> >> I addressed the performance concern I raised earlier myself.? For >> reflection, VM creates the reflective Field objects and fills in >> MemberName when resolving a member.? VM will tag if this >> Field/MemberName is trusted final field.? I think this is a cleaner >> approach rather than in each place to check for final static and final >> fields in hidden or record class to determine if it has write access or >> not. >> >> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >> Unsafe has been allowing access of static final fields of any classes >> but isTrustedFinalField is not limited to instance fields (2) Unsafe >> disallows access to all fields in a hidden class (not limited to trusted >> final fields).? So it follows the precedence and simply checks if the >> declaring class is a record. `Class::isRecord` calls >> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >> `Class::getSuperclass` is intrinsified, the call on isRecord on a normal >> class is fast. Christoph has contributed the microbenchmarks that >> confirm that no performance regression. >> >> Thanks >> Mandy >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html From igor.ignatyev at oracle.com Tue Jun 16 16:45:20 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 16 Jun 2020 09:45:20 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: <287C88A1-3464-484E-AF66-7822394CA125@oracle.com> Hi David, thanks for your review. re: LingeredAppTest, I agree that the test doesn't look very useful, yet I'd remind that the goal of this (and other test in /test/lib-test) is to (sanity) test testlibrary in order to easily spot bugs in testlibrary in a clear manner so one would not have to investigate failures of actual "product" tests and go thru their sometimes convoluted logic just to find out that the problem was in LingeredApp. w/ that being said, this test can do a better job in testing LingeredApp, so I'll file a JBS ticket against hotspo/svc to evaluate/improve/discuss the fate of the test. Thanks, -- Igor > On Jun 16, 2020, at 12:14 AM, David Holmes wrote: > > Hi Igor, > > On 16/06/2020 10:39 am, Igor Ignatyev wrote: >> @David, Erik, Magnus, >> please find the answers to your comments at the bottom of this email. >> @all, >> David's and Erik's comments made me realize that some parts of the original patch were stashed away and didn't make it to webrev.00. I'm truly sorry for the confusion and inconvenience. I also agree w/ David that it's hard to follow/review, and my gaffe just made it worse, therefore I'd like to start it over again from a clean sate >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >>> 833 lines changed: 228 ins; 591 del; 14 mod; >> could you please review the patch which puts all tests for common testlibrary classes (which is ones located in /test/lib) into one location under /test/lib-test? please note this intentionally doesn't move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . > > Ok. > >> to simplify review, the patch has been split into several webrevs, with each of them accomplishes a more-or-less distinct thing, but is not necessary self-contained: > > Many thanks for doing this! > >> 0. trivial move of tests from test/jdk and test/hotspot/jtreg test suites to test/lib-test: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ > > Looks good. > >> 1. removal of hotspot's AssertsTest, OutputAnalyzerReportingTest and "merge" of hotspot's and jdk's OutputAnalyzerTest: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ > > Looks good. > >> 2. simplification of TestNativeProcessBuilder tests: converts Test class used by TestNativeProcessBuilder into a static nested class: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder > > Looks good. > >> 3. makefiles changes to build native parts of lib-test tests. in past, there were no tests w/ native parts in this test suite, TestNativeProcessBuilder is the 1st such test; this part also removes now unneeded lines from hotspot test suite makefile: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest > > Hmm okay. Not sure this needed its own category of build logic but ... > > Aside: Makefiles should not have the classpath exception version of the license header. But they all do for some reason. I'll check if we need to do a separate cleanup of that. > >> 4. clean ups in hotspot test suite: adjusted location of SimpleClassFileLoadHookTest test, which is a test for hotspot-specific test library (located in /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from TEST.groups: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ > > Looks good. Took me a while to understand the connection to the test library here. > >> 5. LingeredAppTest fix (which apparently has never been run before): >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ > > Someone from serviceability should evaluate this test. It may not be needed. > >> 6. changes to make test/lib-test a fully supported and working test suite, and add in into tier1, includes adding ProblemList, TEST.groups file, and 'randomness' k/w: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ > > Seems okay. > > Thanks, > David > ----- > >> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >> testing: >> - make test TEST=tier1 locally on macosx >> - test/lib-test on {windows,linux,macosx}-x64 >> - tier1 job (in progress) >> Thanks, >> -- Igor >>> On Jun 14, 2020, at 11:23 PM, David Holmes > wrote: >>> <...> >>> This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. >> this is intentionally, ctw test-library is hotspot specific, hence its tests are to reside in hotspot test suite (until we decide to move the library to /test/lib), the same is true for SimpleClassFileLoadHookTest. >>> >>> You did not modify hotspot/jtreg/TEST.groups but it refers to: >>> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >>> Plus it should probably refer to the new native_sanity tests somewhere. >> the actual version of the patch removed reference to TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and had TestNativeProcessBuilder moved to /test/test-lib, so no updates w.r.t. native_sanity are needed >>> The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. >> /test/lib-test/TEST.ROOT file was created by JDK-8243010 on 2020-04-16, hence the added copyright has 2020 as the initial copyright year. >>> You used the wrong license header - makefiles don't use the classpath exception. >> JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which also have classpath exception. quick grep showed that make directory has 794 files which has '"Classpath" exception', out of 850 which contain 'GNU General Public License version 2' string. And although I agree that makefiles shouldn't have the classpath exception, I'd prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up to build team to decide how it's best to handle. >>> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie > wrote: >>> >>> A few comments: >>> >>> This seems like code copied from elsewhere: >>> 57 # This evaluation is expensive and should only be done if this target was >>> 58 # explicitly called. >>> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >>> I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). >> right, JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which has this comment, I don't know enough details to say if it's expensive or not, yet if you insist and there is a consensus within build team, I can remove the comment. >>> >>> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >>> Please use space after comma. >> this again was preexisting in JtregNativeJdk.gmk, added the space nevertheless. >>> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > wrote: >>> >>> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). >> the lines were updated to define BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher From igor.ignatyev at oracle.com Tue Jun 16 16:52:25 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 16 Jun 2020 09:52:25 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: <9CD66F39-7EE0-441E-BE9C-14C3C5CF5317@oracle.com> Magnus, Erik, thanks for your reviews, pushed w/ a newline being added at L#654 of make/Main.gmk. Cheers, -- Igor > On Jun 16, 2020, at 7:03 AM, Magnus Ihse Bursie wrote: > > On 2020-06-16 15:06, Erik Joelsson wrote: >> >> On 2020-06-15 17:39, Igor Ignatyev wrote: >>> @David, Erik, Magnus, >>> >>> please find the answers to your comments at the bottom of this email. >>> >>> @all, >>> >>> David's and Erik's comments made me realize that some parts of the original patch were stashed away and didn't make it to webrev.00. I'm truly sorry for the confusion and inconvenience. I also agree w/ David that it's hard to follow/review, and my gaffe just made it worse, therefore I'd like to start it over again from a clean sate >>> >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 Build changes look good to me now, except for a missing newline in Main.gmk. (No need for new review.) >> > > Ditto. > > /Magnus >> >> /Erik >> >>>> 833 lines changed: 228 ins; 591 del; 14 mod; >>> >>> could you please review the patch which puts all tests for common testlibrary classes (which is ones located in /test/lib) into one location under /test/lib-test? please note this intentionally doesn't move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . >>> >>> to simplify review, the patch has been split into several webrevs, with each of them accomplishes a more-or-less distinct thing, but is not necessary self-contained: >>> >>> 0. trivial move of tests from test/jdk and test/hotspot/jtreg test suites to test/lib-test: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ >>> >>> 1. removal of hotspot's AssertsTest, OutputAnalyzerReportingTest and "merge" of hotspot's and jdk's OutputAnalyzerTest: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ >>> >>> 2. simplification of TestNativeProcessBuilder tests: converts Test class used by TestNativeProcessBuilder into a static nested class: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder >>> >>> 3. makefiles changes to build native parts of lib-test tests. in past, there were no tests w/ native parts in this test suite, TestNativeProcessBuilder is the 1st such test; this part also removes now unneeded lines from hotspot test suite makefile: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest >>> >>> 4. clean ups in hotspot test suite: adjusted location of SimpleClassFileLoadHookTest test, which is a test for hotspot-specific test library (located in /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from TEST.groups: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ >>> >>> 5. LingeredAppTest fix (which apparently has never been run before): >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ >>> >>> 6. changes to make test/lib-test a fully supported and working test suite, and add in into tier1, includes adding ProblemList, TEST.groups file, and 'randomness' k/w: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: >>> - make test TEST=tier1 locally on macosx >>> - test/lib-test on {windows,linux,macosx}-x64 >>> - tier1 job (in progress) >>> >>> Thanks, >>> -- Igor >>> >>> >>>> On Jun 14, 2020, at 11:23 PM, David Holmes > wrote: >>>> <...> >>> >>>> This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. >>> this is intentionally, ctw test-library is hotspot specific, hence its tests are to reside in hotspot test suite (until we decide to move the library to /test/lib), the same is true for SimpleClassFileLoadHookTest. >>>> >>>> You did not modify hotspot/jtreg/TEST.groups but it refers to: >>>> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >>>> Plus it should probably refer to the new native_sanity tests somewhere. >>> the actual version of the patch removed reference to TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and had TestNativeProcessBuilder moved to /test/test-lib, so no updates w.r.t. native_sanity are needed >>> >>>> The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. >>> >>> /test/lib-test/TEST.ROOT file was created by JDK-8243010 on 2020-04-16, hence the added copyright has 2020 as the initial copyright year. >>> >>>> You used the wrong license header - makefiles don't use the classpath exception. >>> JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which also have classpath exception. quick grep showed that make directory has 794 files which has '"Classpath" exception', out of 850 which contain 'GNU General Public License version 2' string. And although I agree that makefiles shouldn't have the classpath exception, I'd prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up to build team to decide how it's best to handle. >>> >>> >>>> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie > wrote: >>>> >>>> A few comments: >>>> >>>> This seems like code copied from elsewhere: >>>> 57 # This evaluation is expensive and should only be done if this target was >>>> 58 # explicitly called. >>>> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >>>> I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). >>> right, JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which has this comment, I don't know enough details to say if it's expensive or not, yet if you insist and there is a consensus within build team, I can remove the comment. >>> >>>> >>>> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >>>> Please use space after comma. >>> this again was preexisting in JtregNativeJdk.gmk, added the space nevertheless. >>> >>> >>>> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > wrote: >>>> >>>> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). >>> the lines were updated to define BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher >>> >>> > From coleen.phillimore at oracle.com Tue Jun 16 17:32:29 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 16 Jun 2020 13:32:29 -0400 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <97446f83-1e00-13bd-14f2-d32620ff072f@oracle.com> References: <1229321025.1095655.1592300959835.JavaMail.zimbra@u-pem.fr> <97446f83-1e00-13bd-14f2-d32620ff072f@oracle.com> Message-ID: <76b69541-ea2b-bf4a-932a-30151de689bc@oracle.com> On 6/16/20 12:42 PM, Mandy Chung wrote: > > > On 6/16/20 2:49 AM, Remi Forax wrote: >> Hi Mndy, >> Looks good, >> I don't like too much the fact that there is a new boolean field in >> j.l.r.Field instead of using the modifiers like in the VM counterpart, >> but it will require masking and unmasking the modifiers which is a >> far bigger change, too big to worth it in my opinion. > > Adding a new field instead of using the modifiers is a deliberate > choice from my side.? It's higher risk to overload the modifiers to > include VM-specific flags. Thank you for this.? As far as I can see the trusted final field doesn't add any footprint to the java.lang.reflect.Field instance. The VM runtime changes look okay to me, excluding the methodHandles.* changes which I don't know that much about. Coleen > I am all for a better clean up (valhalla lworld does use the modifiers > to cover a flag indicating if it's flattened which I am not satisfied > either). > >> So +1 > > Thanks Remi. > > Mandy > >> R?mi >> >> ----- Mail original ----- >>> De: "mandy chung" >>> ?: "hotspot-dev" , "core-libs-dev" >>> , "amber-dev" >>> >>> Envoy?: Lundi 15 Juin 2020 23:58:19 >>> Objet: (15) RFR: JDK-8247444: Trust final fields in records >>> This patch is joint contribution from Christoph Dreis [1] and me. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >>> >>> This proposes to make final fields in records notmodifiable via >>> reflection.? Field::set throws IAE if a Field is not modifiable. >>> Thecurrent specification specifies the following Fields not modifiable: >>> - static final fields in any class >>> - final fields in a hidden class >>> >>> The spec of Field::set is changed to specify that records are not >>> modifiable via reflection. >>> ??Noe that no change in Field::setAccessible(true), i.e. it will >>> succeed >>> to allow existing frameworks to have read access to final fields in >>> records.? Just no write access. >>> >>> VarHandle does not support write access if it's a static final field or >>> an instance final field. >>> >>> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >>> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >>> >>> No change is made in JNI.? JNI could be considered to disallow >>> modification of final fields in records and hidden classes (static and >>> instance fields).? But JNI has superpower and the current spec already >>> allows to modify the value of a final static field even after it's >>> constant folded (via JNI SetField and SetStaticField), then >>> all bets are off.? This should be re-visited when we consider "final is >>> truly final" for all classes. >>> >>> Make final fields in records not modifiable via reflection enables JIT >>> optimization as these final fields are effectively truly final. >>> >>> This change impacts 3rd-party frameworks including 3rd-party >>> serialization framework that rely on core reflection `setAccessible` or >>> `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >>> construct records but not using the canonical constructor. >>> These frameworks would need to be updated to construct records via its >>> canonical constructor as done by the Java serialization. >>> >>> I see this change gives a good opportunity to engage the maintainers of >>> the serialization frameworks and work together to support new features >>> including records, inline classes and the new serialization mechanism >>> and which I think it is worth the investment. >>> >>> This is a low risk enhancement.? I'd like to request approval for a >>> late >>> enhancement in JDK 15.? It extends the pre-existing code path with >>> refactoring the hidden classes to prepare for new kinds of classes with >>> trusted final fields.? The change is straight-forward. >>> >>> Can this wait to integrate in JDK 16? >>> >>> ?? It's important to get this enhancement in when record is a preview >>> feature that we can get feedback and give time to 3rd-party >>> serialization frameworks to look into adding the support for records. >>> If we delayed this change in 16 and records exit preview, it would be >>> bad for frameworks if they verify that they support records in 15 but >>> fail in 16.? OTOH the risk of this patch is low. >>> >>> Performance Impact >>> >>> I addressed the performance concern I raised earlier myself. For >>> reflection, VM creates the reflective Field objects and fills in >>> MemberName when resolving a member.? VM will tag if this >>> Field/MemberName is trusted final field.? I think this is a cleaner >>> approach rather than in each place to check for final static and final >>> fields in hidden or record class to determine if it has write access or >>> not. >>> >>> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >>> Unsafe has been allowing access of static final fields of any classes >>> but isTrustedFinalField is not limited to instance fields (2) Unsafe >>> disallows access to all fields in a hidden class (not limited to >>> trusted >>> final fields).? So it follows the precedence and simply checks if the >>> declaring class is a record. `Class::isRecord` calls >>> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >>> `Class::getSuperclass` is intrinsified, the call on isRecord on a >>> normal >>> class is fast. Christoph has contributed the microbenchmarks that >>> confirm that no performance regression. >>> >>> Thanks >>> Mandy >>> [1] >>> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html >>> > From mandy.chung at oracle.com Tue Jun 16 23:06:02 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 Jun 2020 16:06:02 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <76b69541-ea2b-bf4a-932a-30151de689bc@oracle.com> References: <1229321025.1095655.1592300959835.JavaMail.zimbra@u-pem.fr> <97446f83-1e00-13bd-14f2-d32620ff072f@oracle.com> <76b69541-ea2b-bf4a-932a-30151de689bc@oracle.com> Message-ID: Thanks Coleen for the review. Mandy On 6/16/20 10:32 AM, coleen.phillimore at oracle.com wrote: > > > On 6/16/20 12:42 PM, Mandy Chung wrote: >> >> >> On 6/16/20 2:49 AM, Remi Forax wrote: >>> Hi Mndy, >>> Looks good, >>> I don't like too much the fact that there is a new boolean field in >>> j.l.r.Field instead of using the modifiers like in the VM counterpart, >>> but it will require masking and unmasking the modifiers which is a >>> far bigger change, too big to worth it in my opinion. >> >> Adding a new field instead of using the modifiers is a deliberate >> choice from my side.? It's higher risk to overload the modifiers to >> include VM-specific flags. > > Thank you for this.? As far as I can see the trusted final field > doesn't add any footprint to the java.lang.reflect.Field instance. > > The VM runtime changes look okay to me, excluding the methodHandles.* > changes which I don't know that much about. > > Coleen > >> I am all for a better clean up (valhalla lworld does use the >> modifiers to cover a flag indicating if it's flattened which I am not >> satisfied either). >> >>> So +1 >> >> Thanks Remi. >> >> Mandy >> >>> R?mi >>> >>> ----- Mail original ----- >>>> De: "mandy chung" >>>> ?: "hotspot-dev" , "core-libs-dev" >>>> , "amber-dev" >>>> >>>> Envoy?: Lundi 15 Juin 2020 23:58:19 >>>> Objet: (15) RFR: JDK-8247444: Trust final fields in records >>>> This patch is joint contribution from Christoph Dreis [1] and me. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >>>> >>>> This proposes to make final fields in records notmodifiable via >>>> reflection.? Field::set throws IAE if a Field is not modifiable. >>>> Thecurrent specification specifies the following Fields not >>>> modifiable: >>>> - static final fields in any class >>>> - final fields in a hidden class >>>> >>>> The spec of Field::set is changed to specify that records are not >>>> modifiable via reflection. >>>> ??Noe that no change in Field::setAccessible(true), i.e. it will >>>> succeed >>>> to allow existing frameworks to have read access to final fields in >>>> records.? Just no write access. >>>> >>>> VarHandle does not support write access if it's a static final >>>> field or >>>> an instance final field. >>>> >>>> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >>>> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >>>> >>>> No change is made in JNI.? JNI could be considered to disallow >>>> modification of final fields in records and hidden classes (static and >>>> instance fields).? But JNI has superpower and the current spec already >>>> allows to modify the value of a final static field even after it's >>>> constant folded (via JNI SetField and SetStaticField), >>>> then >>>> all bets are off.? This should be re-visited when we consider >>>> "final is >>>> truly final" for all classes. >>>> >>>> Make final fields in records not modifiable via reflection enables JIT >>>> optimization as these final fields are effectively truly final. >>>> >>>> This change impacts 3rd-party frameworks including 3rd-party >>>> serialization framework that rely on core reflection >>>> `setAccessible` or >>>> `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >>>> construct records but not using the canonical constructor. >>>> These frameworks would need to be updated to construct records via its >>>> canonical constructor as done by the Java serialization. >>>> >>>> I see this change gives a good opportunity to engage the >>>> maintainers of >>>> the serialization frameworks and work together to support new features >>>> including records, inline classes and the new serialization mechanism >>>> and which I think it is worth the investment. >>>> >>>> This is a low risk enhancement.? I'd like to request approval for a >>>> late >>>> enhancement in JDK 15.? It extends the pre-existing code path with >>>> refactoring the hidden classes to prepare for new kinds of classes >>>> with >>>> trusted final fields.? The change is straight-forward. >>>> >>>> Can this wait to integrate in JDK 16? >>>> >>>> ?? It's important to get this enhancement in when record is a preview >>>> feature that we can get feedback and give time to 3rd-party >>>> serialization frameworks to look into adding the support for records. >>>> If we delayed this change in 16 and records exit preview, it would be >>>> bad for frameworks if they verify that they support records in 15 but >>>> fail in 16.? OTOH the risk of this patch is low. >>>> >>>> Performance Impact >>>> >>>> I addressed the performance concern I raised earlier myself. For >>>> reflection, VM creates the reflective Field objects and fills in >>>> MemberName when resolving a member.? VM will tag if this >>>> Field/MemberName is trusted final field.? I think this is a cleaner >>>> approach rather than in each place to check for final static and final >>>> fields in hidden or record class to determine if it has write >>>> access or >>>> not. >>>> >>>> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >>>> Unsafe has been allowing access of static final fields of any classes >>>> but isTrustedFinalField is not limited to instance fields (2) Unsafe >>>> disallows access to all fields in a hidden class (not limited to >>>> trusted >>>> final fields).? So it follows the precedence and simply checks if the >>>> declaring class is a record. `Class::isRecord` calls >>>> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >>>> `Class::getSuperclass` is intrinsified, the call on isRecord on a >>>> normal >>>> class is fast. Christoph has contributed the microbenchmarks that >>>> confirm that no performance regression. >>>> >>>> Thanks >>>> Mandy >>>> [1] >>>> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html >>>> >> > From mandy.chung at oracle.com Tue Jun 16 23:06:40 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 Jun 2020 16:06:40 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <1e7832e9-1398-bb67-c270-b079b06aa588@oracle.com> Hi David, Thanks for the review. On 6/15/20 7:29 PM, David Holmes wrote: > Hi Mandy, Christoph, > > The code changes here all look okay to me. The idea of introducing the > notion of "trusted final fields" seems quite reasonable. > > I made one editorial comment on the CSR request. > Thanks. > I'm unclear if the new TEST.properties file needs a copyright notice > and header. We have 706 such files in the repo and 554 (mostly > hotspot) do have the copyright notice and header. > I will find out the recommendation and update you. Mandy > Thanks, > David > > On 16/06/2020 7:58 am, Mandy Chung wrote: >> This patch is joint contribution from Christoph Dreis [1] and me. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >> >> This proposes to make final fields in records notmodifiable via >> reflection.? Field::set throws IAE if a Field is not modifiable. >> Thecurrent specification specifies the following Fields not modifiable: >> - static final fields in any class >> - final fields in a hidden class >> >> The spec of Field::set is changed to specify that records are not >> modifiable via reflection. >> ??Noe that no change in Field::setAccessible(true), i.e. it will >> succeed to allow existing frameworks to have read access to final >> fields in records.? Just no write access. >> >> VarHandle does not support write access if it's a static final field >> or an instance final field. >> >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >> >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static >> and instance fields).? But JNI has superpower and the current spec >> already allows to modify the value of a final static field even after >> it's constant folded (via JNI SetField and >> SetStaticField), then all bets are off.? This should be >> re-visited when we consider "final is truly final" for all classes. >> >> Make final fields in records not modifiable via reflection enables >> JIT optimization as these final fields are effectively truly final. >> >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` >> or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via >> its canonical constructor as done by the Java serialization. >> >> I see this change gives a good opportunity to engage the maintainers >> of the serialization frameworks and work together to support new >> features including records, inline classes and the new serialization >> mechanism and which I think it is worth the investment. >> >> This is a low risk enhancement.? I'd like to request approval for a >> late enhancement in JDK 15.? It extends the pre-existing code path >> with refactoring the hidden classes to prepare for new kinds of >> classes with trusted final fields.? The change is straight-forward. >> >> Can this wait to integrate in JDK 16? >> >> ?? It's important to get this enhancement in when record is a preview >> feature that we can get feedback and give time to 3rd-party >> serialization frameworks to look into adding the support for records. >> If we delayed this change in 16 and records exit preview, it would be >> bad for frameworks if they verify that they support records in 15 but >> fail in 16.? OTOH the risk of this patch is low. >> >> Performance Impact >> >> I addressed the performance concern I raised earlier myself. For >> reflection, VM creates the reflective Field objects and fills in >> MemberName when resolving a member.? VM will tag if this >> Field/MemberName is trusted final field.? I think this is a cleaner >> approach rather than in each place to check for final static and >> final fields in hidden or record class to determine if it has write >> access or not. >> >> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >> Unsafe has been allowing access of static final fields of any classes >> but isTrustedFinalField is not limited to instance fields (2) Unsafe >> disallows access to all fields in a hidden class (not limited to >> trusted final fields).? So it follows the precedence and simply >> checks if the declaring class is a record. `Class::isRecord` calls >> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >> `Class::getSuperclass` is intrinsified, the call on isRecord on a >> normal class is fast. Christoph has contributed the microbenchmarks >> that confirm that no performance regression. >> >> Thanks >> Mandy >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html >> From david.holmes at oracle.com Wed Jun 17 00:17:21 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 Jun 2020 10:17:21 +1000 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: <287C88A1-3464-484E-AF66-7822394CA125@oracle.com> References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> <287C88A1-3464-484E-AF66-7822394CA125@oracle.com> Message-ID: <5f7e286a-e2ee-e544-3c6a-62d1f066895f@oracle.com> On 17/06/2020 2:45 am, Igor Ignatyev wrote: > Hi David, > > thanks for your review. re: LingeredAppTest, I agree that the test doesn't look very useful, yet I'd remind that the goal of this (and other test in /test/lib-test) is to (sanity) test testlibrary in order to easily spot bugs in testlibrary in a clear manner so one would not have to investigate failures of actual "product" tests and go thru their sometimes convoluted logic just to find out that the problem was in LingeredApp. w/ that being said, this test can do a better job in testing LingeredApp, so I'll file a JBS ticket against hotspo/svc to evaluate/improve/discuss the fate of the test. Given the test had not previously been run (it wouldn't even compile as far as I could see) it's validity and utility has to be called into question. But a follow up bug is fine for that as long it it currently works reliably. Thanks, David > Thanks, > -- Igor > >> On Jun 16, 2020, at 12:14 AM, David Holmes wrote: >> >> Hi Igor, >> >> On 16/06/2020 10:39 am, Igor Ignatyev wrote: >>> @David, Erik, Magnus, >>> please find the answers to your comments at the bottom of this email. >>> @all, >>> David's and Erik's comments made me realize that some parts of the original patch were stashed away and didn't make it to webrev.00. I'm truly sorry for the confusion and inconvenience. I also agree w/ David that it's hard to follow/review, and my gaffe just made it worse, therefore I'd like to start it over again from a clean sate >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >>>> 833 lines changed: 228 ins; 591 del; 14 mod; >>> could you please review the patch which puts all tests for common testlibrary classes (which is ones located in /test/lib) into one location under /test/lib-test? please note this intentionally doesn't move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . >> >> Ok. >> >>> to simplify review, the patch has been split into several webrevs, with each of them accomplishes a more-or-less distinct thing, but is not necessary self-contained: >> >> Many thanks for doing this! >> >>> 0. trivial move of tests from test/jdk and test/hotspot/jtreg test suites to test/lib-test: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ >> >> Looks good. >> >>> 1. removal of hotspot's AssertsTest, OutputAnalyzerReportingTest and "merge" of hotspot's and jdk's OutputAnalyzerTest: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ >> >> Looks good. >> >>> 2. simplification of TestNativeProcessBuilder tests: converts Test class used by TestNativeProcessBuilder into a static nested class: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder >> >> Looks good. >> >>> 3. makefiles changes to build native parts of lib-test tests. in past, there were no tests w/ native parts in this test suite, TestNativeProcessBuilder is the 1st such test; this part also removes now unneeded lines from hotspot test suite makefile: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest >> >> Hmm okay. Not sure this needed its own category of build logic but ... >> >> Aside: Makefiles should not have the classpath exception version of the license header. But they all do for some reason. I'll check if we need to do a separate cleanup of that. >> >>> 4. clean ups in hotspot test suite: adjusted location of SimpleClassFileLoadHookTest test, which is a test for hotspot-specific test library (located in /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from TEST.groups: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ >> >> Looks good. Took me a while to understand the connection to the test library here. >> >>> 5. LingeredAppTest fix (which apparently has never been run before): >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ >> >> Someone from serviceability should evaluate this test. It may not be needed. >> >>> 6. changes to make test/lib-test a fully supported and working test suite, and add in into tier1, includes adding ProblemList, TEST.groups file, and 'randomness' k/w: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ >> >> Seems okay. >> >> Thanks, >> David >> ----- >> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: >>> - make test TEST=tier1 locally on macosx >>> - test/lib-test on {windows,linux,macosx}-x64 >>> - tier1 job (in progress) >>> Thanks, >>> -- Igor >>>> On Jun 14, 2020, at 11:23 PM, David Holmes > wrote: >>>> <...> >>>> This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. >>> this is intentionally, ctw test-library is hotspot specific, hence its tests are to reside in hotspot test suite (until we decide to move the library to /test/lib), the same is true for SimpleClassFileLoadHookTest. >>>> >>>> You did not modify hotspot/jtreg/TEST.groups but it refers to: >>>> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >>>> Plus it should probably refer to the new native_sanity tests somewhere. >>> the actual version of the patch removed reference to TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and had TestNativeProcessBuilder moved to /test/test-lib, so no updates w.r.t. native_sanity are needed >>>> The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. >>> /test/lib-test/TEST.ROOT file was created by JDK-8243010 on 2020-04-16, hence the added copyright has 2020 as the initial copyright year. >>>> You used the wrong license header - makefiles don't use the classpath exception. >>> JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which also have classpath exception. quick grep showed that make directory has 794 files which has '"Classpath" exception', out of 850 which contain 'GNU General Public License version 2' string. And although I agree that makefiles shouldn't have the classpath exception, I'd prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up to build team to decide how it's best to handle. >>>> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie > wrote: >>>> >>>> A few comments: >>>> >>>> This seems like code copied from elsewhere: >>>> 57 # This evaluation is expensive and should only be done if this target was >>>> 58 # explicitly called. >>>> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >>>> I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). >>> right, JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which has this comment, I don't know enough details to say if it's expensive or not, yet if you insist and there is a consensus within build team, I can remove the comment. >>>> >>>> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >>>> Please use space after comma. >>> this again was preexisting in JtregNativeJdk.gmk, added the space nevertheless. >>>> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > wrote: >>>> >>>> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). >>> the lines were updated to define BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher > From igor.ignatyev at oracle.com Wed Jun 17 03:39:22 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 16 Jun 2020 20:39:22 -0700 Subject: RFR(T) [15] : 8247716 : JVM_RegisterWhiteBoxMethods checks wrong classloader Message-ID: <81982F2D-5DA5-485A-85BF-AC96DE6005BA@oracle.com> http://cr.openjdk.java.net/~iignatyev//8247716/webrev.00 > 1 line changed: 0 ins; 0 del; 1 mod; Hi all, could you please review this one-liner fix for JVM_RegisterWhiteBoxMethods method? from JBS: > JVM_RegisterWhiteBoxMethods is supposed to check that sun.hotspot.WhiteBox class is loaded by the boot classloader, yet it actually checks that java.lang.Class class is. note: this had no impact, as NativeLookup::lookup_style would use JVM_RegisterWhiteBoxMethods as sun.hotspot.WhiteBox::registerNatives only if the class was load by the boot loader. testing: tier1 JBS: https://bugs.openjdk.java.net/browse/JDK-8247716 webrev: http://cr.openjdk.java.net/~iignatyev//8247716/webrev.00 Thanks, -- Igor From john.r.rose at oracle.com Wed Jun 17 04:06:40 2020 From: john.r.rose at oracle.com (John Rose) Date: Tue, 16 Jun 2020 21:06:40 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On Jun 15, 2020, at 2:58 PM, Mandy Chung wrote: > > Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 Good work. Reviewed. ? John From igor.ignatyev at oracle.com Wed Jun 17 04:49:20 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 16 Jun 2020 21:49:20 -0700 Subject: RFR(T) [15] : 8247725 : move two tests for whitebox from test/hotspot/jtreg/sanity to test/lib-test Message-ID: http://cr.openjdk.java.net/~iignatyev/8247725/webrev.00/ > 0 lines changed: 0 ins; 0 del; 0 mod; Hi all, could you please review this zero-liner which moves MismatchedWhiteBox/WhiteBox.java and WBApi.java tests from test/hotspot/jtreg test suite to test/lib-test test suite? JBS: https://bugs.openjdk.java.net/browse/JDK-8247725 webrev: http://cr.openjdk.java.net/~iignatyev/8247725/webrev.00/ $ hg mv test/hotspot/jtreg/sanity/MismatchedWhiteBox test/lib-test/sun/hotspot/whitebox/ $ hg mv test/hotspot/jtreg/sanity/WBApi.java test/lib-test/sun/hotspot/whitebox/ testing: test/lib-test tests Thanks, -- Igor From sgehwolf at redhat.com Wed Jun 17 09:13:31 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 17 Jun 2020 11:13:31 +0200 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> Message-ID: <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: > On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: > > > Here would be my suggestion > > > (hopefully this one isn't too restrictive again): > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ > > > > Hi Severin, this worked nicely . > > Thanks for providing this . > > Great. If somebody else is willing to OK this, I'll push it. Could I have a second review of this, please? I believe it's the best we can do for this test failure Matthias is seeing. Thanks, Severin From matthias.baesken at sap.com Wed Jun 17 10:26:17 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 17 Jun 2020 10:26:17 +0000 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> Message-ID: Hello, I guess with working mails from hotspot-dev for all, we would maybe get more reviews ... Best regards, Matthias -----Original Message----- From: Severin Gehwolf Sent: Mittwoch, 17. Juni 2020 11:14 To: Baesken, Matthias ; jiefu(??) ; David Holmes ; 'hotspot-dev at openjdk.java.net' Cc: Bob Vandette Subject: Re: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: > On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: > > > Here would be my suggestion > > > (hopefully this one isn't too restrictive again): > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ > > > > Hi Severin, this worked nicely . > > Thanks for providing this . > > Great. If somebody else is willing to OK this, I'll push it. Could I have a second review of this, please? I believe it's the best we can do for this test failure Matthias is seeing. Thanks, Severin From thomas.stuefe at gmail.com Wed Jun 17 10:37:10 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 17 Jun 2020 12:37:10 +0200 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> Message-ID: This looks okay to me. Mailing lists not working really hits us where it hurts. ..Thomas On Wed, Jun 17, 2020 at 12:26 PM Baesken, Matthias wrote: > Hello, > I guess with working mails from hotspot-dev for all, we would maybe get > more reviews ... > > Best regards, Matthias > > -----Original Message----- > From: Severin Gehwolf > Sent: Mittwoch, 17. Juni 2020 11:14 > To: Baesken, Matthias ; jiefu(??) < > jiefu at tencent.com>; David Holmes ; ' > hotspot-dev at openjdk.java.net' > Cc: Bob Vandette > Subject: Re: [PING] RFR [XS]: 8244500: jtreg test error in > test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java > > On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: > > On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: > > > > Here would be my suggestion > > > > (hopefully this one isn't too restrictive again): > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ > > > > > > Hi Severin, this worked nicely . > > > Thanks for providing this . > > > > Great. If somebody else is willing to OK this, I'll push it. > > Could I have a second review of this, please? I believe it's the best > we can do for this test failure Matthias is seeing. > > Thanks, > Severin > > From vladimir.x.ivanov at oracle.com Wed Jun 17 11:13:53 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 17 Jun 2020 14:13:53 +0300 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: Looks good! I welcome and fully support such hardening of final field handling for new language features. Minor comment on naming: "trusted final field" is an JVM implementation detail and I'd prefer to keep it internal to JVM. (I hope it'll eventually go away and will be replaced with a more disciplined approach.) Probably, a better alternative is a name that clearly communicates that the JVM expects/allows modifications of a final field. Or, at least, that it's *the JVM* which "trusts" the contents of a final field after its initialization is over (vmTrustedFinal/isVMTrustedFinalField). But I'm fine with keeping the patch as is. Best regards, Vladimir Ivanov On 16.06.2020 00:58, Mandy Chung wrote: > This patch is joint contribution from Christoph Dreis [1] and me. > > Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 > > This proposes to make final fields in records notmodifiable via > reflection.? Field::set throws IAE if a Field is not modifiable. > Thecurrent specification specifies the following Fields not modifiable: > - static final fields in any class > - final fields in a hidden class > > The spec of Field::set is changed to specify that records are not > modifiable via reflection. > ?Noe that no change in Field::setAccessible(true), i.e. it will succeed > to allow existing frameworks to have read access to final fields in > records.? Just no write access. > > VarHandle does not support write access if it's a static final field or > an instance final field. > > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), then > all bets are off.? This should be re-visited when we consider "final is > truly final" for all classes. > > Make final fields in records not modifiable via reflection enables JIT > optimization as these final fields are effectively truly final. > > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` or > `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. > > I see this change gives a good opportunity to engage the maintainers of > the serialization frameworks and work together to support new features > including records, inline classes and the new serialization mechanism > and which I think it is worth the investment. > > This is a low risk enhancement.? I'd like to request approval for a late > enhancement in JDK 15.? It extends the pre-existing code path with > refactoring the hidden classes to prepare for new kinds of classes with > trusted final fields.? The change is straight-forward. > > Can this wait to integrate in JDK 16? > > ? It's important to get this enhancement in when record is a preview > feature that we can get feedback and give time to 3rd-party > serialization frameworks to look into adding the support for records. If > we delayed this change in 16 and records exit preview, it would be bad > for frameworks if they verify that they support records in 15 but fail > in 16.? OTOH the risk of this patch is low. > > Performance Impact > > I addressed the performance concern I raised earlier myself.? For > reflection, VM creates the reflective Field objects and fills in > MemberName when resolving a member.? VM will tag if this > Field/MemberName is trusted final field.? I think this is a cleaner > approach rather than in each place to check for final static and final > fields in hidden or record class to determine if it has write access or > not. > > `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) > Unsafe has been allowing access of static final fields of any classes > but isTrustedFinalField is not limited to instance fields (2) Unsafe > disallows access to all fields in a hidden class (not limited to trusted > final fields).? So it follows the precedence and simply checks if the > declaring class is a record. `Class::isRecord` calls > `Class::getSuperclass` to check if it's a subtype of `Record`.? As > `Class::getSuperclass` is intrinsified, the call on isRecord on a normal > class is fast. Christoph has contributed the microbenchmarks that > confirm that no performance regression. > > Thanks > Mandy > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html > From bob.vandette at oracle.com Wed Jun 17 13:09:51 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 17 Jun 2020 09:09:51 -0400 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> Message-ID: <73C921E8-03A1-4CFA-BCA8-1FC8C46C2171@oracle.com> Should we just accept the kernel error message rather than add all the additional value checking? "out.shouldContain("Your kernel does not support swap limit capabilities?); Why isn?t the try block around the check for .shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: [1-9][0-9]+?); ? Why doesn?t this call fail as well?? Have you tried this test on a system with cgrouopv1 and v2 that doesn?t have swap enabled? I?m concerned that we?ll be back to fixing this when some other configuration combination fails. Bob. > On Jun 17, 2020, at 5:13 AM, Severin Gehwolf wrote: > > On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: >> On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: >>>> Here would be my suggestion >>>> (hopefully this one isn't too restrictive again): >>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ >>> >>> Hi Severin, this worked nicely . >>> Thanks for providing this . >> >> Great. If somebody else is willing to OK this, I'll push it. > > Could I have a second review of this, please? I believe it's the best > we can do for this test failure Matthias is seeing. > > Thanks, > Severin > From stefan.karlsson at oracle.com Wed Jun 17 14:16:24 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 17 Jun 2020 16:16:24 +0200 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes Message-ID: Hi all, Please review and/or comment on this patch to create leaner and more versatile GrowableArray classes. https://cr.openjdk.java.net/~stefank/8247755/webrev.01 https://bugs.openjdk.java.net/browse/JDK-8247755 This is a proposal to rewrite the GrowableArray classes, with two main goals: 1) Allow a GrowableArray instance to be allocated on the stack or embedded in other objects. This removes one of the two indirections when operating on the array elements. 2) Make the GrowableArray instances smaller, so that there's no extra overhead when they are embedded in other objects. This will enable us to use GrowableArray effectively in ZGC, without having to rely on our own implementations. See follow-up patch: https://cr.openjdk.java.net/~stefank/8247759/webrev.01 Some background about the GrowableArray: GrowableArray is split into two memory parts. One for the instance of the GrowableArray, and a second, separate memory part for the elements. This memory can come from these places: a) Resource area in the thread b) CHeap (malloc) c) Arena (malloced arena) d) Stack e) Embedded in other object From the C++ compiler's point-of-view, (a)-(e) can be used to allocate the GrowableArray, and (a)-(c) can be used to allocate the elements. However, there are runtime asserts that restricts the combinations. One of the restrictions is that if either of the GrowableArray instance memory, or the elements memory, comes from the CHeap, then the other must also come from the CHeap. I've found two reasons why we want to be restrictive with our CHeap allocated elements: - if the GrowableArray instance is allocated in (a) or (c), then the destructor will never run and we get a memory leak because the elements are not deallocated. However, for stack allocated instances the destructor will be run, and the memory will be handed back. For embedded instances it all depends on how the parent instance was allocated. - There's no proper support for copying/moving of the elements, just a simple pointer copy. So, if a copy would be done, the source GrowableArray would still have a pointer to the elements, and if the source were destructed (maybe because it was stack allocated and went out-of-scope) then the memory of the destination GrowableArray will be deallocated. This isn't a problem with instances in (a) or (c), since their destructors won't be run. Proposal for Goal 1: - Allow stack allocated and embedded GrowableArrays to have CHeap allocated elements. - Add an assert in a copy constructor and assignment operator that checks that the GrowableArray elements are not CHeap allocated. The rest of the patch mainly deals with Goal 2: The current fields of GrowableArrays, in product builds, are: ???int _len; ???int _max; ???Arena* _arena; ???MEMFLAGS _memflags; ???E* _data; and the resulting is 32 bytes: 4 + 4 + 8 + 8 + 8. The _memflags size is due to alignment requirements for the _data field. - By combining _arena and _memflags, this is reduced to 24 bytes. - By having a specialized CHeap version with the MEMFLAGS encoded as a template parameter, we can get rid of both _memflags and _arena fields, and reduce this to 16 bytes. Things to note about the patch: - GrowableArrayMetadata: I've fused _arena and _memflags into one uintptr_t field, and encapsulated all bookkeeping about where and how the memory is allocated for the GrowableArray. The hope is to simplify the code a bit, and also to move some of the checking out into a non-template class, so that the code can be put in the .cpp file. - GrowableArrayNestingCheck: Simplify and encapsulate resource area nesting checks. - I'm using CRTP because I don't want a vtable in the class. This means that the super class can cast to derived type, responsible for allocation and deallocation: template class GrowableArray : public GrowableArrayWithAllocator > and then: void GrowableArrayWithAllocator::grow(int j) { ... E* newData = static_cast(this)->allocate(); - Since I'm chainging so much of the structure of the code, I've also made some stylistic changes to make the code more consistent. - Moved the CompareClosure to iterator.hpp, where many of our other closures are located. - Moved _sort_Fn, since it was already used by non-GrowableArray code - I've added a GrowableArrayCHeap class to be able to mostly side-step the complication of the GrowableArray elements allocation, and to minimize the instance size. - The metaspaceShared.cpp was used to show how GrowableArrayCHeap can be used. It's not strictly necessary, and could be reverted. -? The arguments.cpp change shows a usage of GrowableArrayView as a way to only expose an interface that don't allow the array to be grown. - The arguments.cpp change also shows stack allocated GrowableArrayCHeap, that automatically gets deallocated when the function returns, and the cleanups we can do because of this. - The hashtable.* changes show an embedded GrowableArrayCHeap. - The SA agent. I didn't bother renaming the GenericGrowableArray file. I can do that upon request. - I added gtest for GrowableArray. There was not a single unit test of GrowableArray before. The creates all possible memory allocation combinations and run limited set of sanity checks. It also checks that the different memory allocation combinations return the correct allocation types when queried. I've added negative tests to show that GrowableArrays with CHeap allocated elements assert when being copied. Testing: will run at least tier1-3, but most likely more tiers on Linux x64. Suggestions on what to test welcome. Thanks, StefanK From daniel.daugherty at oracle.com Wed Jun 17 14:28:10 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 17 Jun 2020 10:28:10 -0400 Subject: RFR(T) [15] : 8247716 : JVM_RegisterWhiteBoxMethods checks wrong classloader In-Reply-To: <81982F2D-5DA5-485A-85BF-AC96DE6005BA@oracle.com> References: <81982F2D-5DA5-485A-85BF-AC96DE6005BA@oracle.com> Message-ID: <5022bca4-f096-2032-306c-effec17692d1@oracle.com> On 6/16/20 11:39 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8247716/webrev.00 src/hotspot/share/prims/whitebox.cpp ??? No comments. Thumbs up and I agree that this is a trivial fix. Dan >> 1 line changed: 0 ins; 0 del; 1 mod; > > Hi all, > > could you please review this one-liner fix for JVM_RegisterWhiteBoxMethods method? > from JBS: >> JVM_RegisterWhiteBoxMethods is supposed to check that sun.hotspot.WhiteBox class is loaded by the boot classloader, yet it actually checks that java.lang.Class class is. > note: this had no impact, as NativeLookup::lookup_style would use JVM_RegisterWhiteBoxMethods as sun.hotspot.WhiteBox::registerNatives only if the class was load by the boot loader. > > testing: tier1 > JBS: https://bugs.openjdk.java.net/browse/JDK-8247716 > webrev: http://cr.openjdk.java.net/~iignatyev//8247716/webrev.00 > > Thanks, > -- Igor From sgehwolf at redhat.com Wed Jun 17 14:29:33 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 17 Jun 2020 16:29:33 +0200 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: <73C921E8-03A1-4CFA-BCA8-1FC8C46C2171@oracle.com> References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> <73C921E8-03A1-4CFA-BCA8-1FC8C46C2171@oracle.com> Message-ID: On Wed, 2020-06-17 at 09:09 -0400, Bob Vandette wrote: > Should we just accept the kernel error message rather than add all the additional value checking? > > "out.shouldContain("Your kernel does not support swap limit capabilities?); I'd prefer to keep the other checks as well. The bug explains why that is and if we see a config which fails this as well, something is wrong. Hopefully that makes some sense. > Why isn?t the try block around the check for .shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: [1-9][0-9]+?); Why should it? For most set ups this catch block is never reached. The only setup we are aware of which reaches this is the system Matthias runs tests on. I've never heard of a system where this fails as well. > Why doesn?t this call fail as well?? Because the system values are being used for OperatingSystemMXBean.getTotalSwapSpaceSize() when: Metrics.getMemoryLimit() > 0 && Metrics.getMemoryAndSwapLimit() == -1 > Have you tried this test on a system with cgrouopv1 and v2 that doesn?t have swap enabled? I don't think so. I don't have access to such a system. How do you mean, though? A system which has swap disabled on the system level or a container without swap? I believe the former is exactly the case of Matthias' system. But I don't know. The latter should also be covered via a test Jie added a while back. > I?m concerned that we?ll be back to fixing this when some other configuration combination fails. I understand but find it unlikely. Those are systems which are going to be hard to detect correctly and do-the-right-thing. As mentioned in the bug there is no way to distinguish between unlimited swap + a memory limit from kernel-doesn't-support-swap-limits + a memory limit. Note that https://bugs.openjdk.java.net/browse/JDK-8236617 "fixed" this for the old implementation. Unfortunately, it doesn't seem like it was investigated why a 0 limit was being returned. That was when the old implementation was still in place (prior cgroups v2 support). In the old impl, when files were missing 0 was returned for some cases. In fact, the fix of JDK-8236617 with the old impl would have wrongly returned 0 for getTotalSwapSpaceSize() for a system which got configured with unlimited swap size (--memory-swap=-1). In the new impl, this returns physical swap size which seems better. On that note, the code of JDK-8236617 seems unreachable now. Thanks, Severin > Bob. > > > > On Jun 17, 2020, at 5:13 AM, Severin Gehwolf wrote: > > > > On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: > > > On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: > > > > > Here would be my suggestion > > > > > (hopefully this one isn't too restrictive again): > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ > > > > > > > > Hi Severin, this worked nicely . > > > > Thanks for providing this . > > > > > > Great. If somebody else is willing to OK this, I'll push it. > > > > Could I have a second review of this, please? I believe it's the best > > we can do for this test failure Matthias is seeing. > > > > Thanks, > > Severin > > From daniel.daugherty at oracle.com Wed Jun 17 14:32:32 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 17 Jun 2020 10:32:32 -0400 Subject: RFR(T) [15] : 8247725 : move two tests for whitebox from test/hotspot/jtreg/sanity to test/lib-test In-Reply-To: References: Message-ID: On 6/17/20 12:49 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8247725/webrev.00/ No comments. The webrev clearly shows the renames. Thumbs up and I agree that this is a trivial change. Dan >> 0 lines changed: 0 ins; 0 del; 0 mod; > Hi all, > > could you please review this zero-liner which moves MismatchedWhiteBox/WhiteBox.java and WBApi.java tests from test/hotspot/jtreg test suite to test/lib-test test suite? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247725 > webrev: http://cr.openjdk.java.net/~iignatyev/8247725/webrev.00/ > $ hg mv test/hotspot/jtreg/sanity/MismatchedWhiteBox test/lib-test/sun/hotspot/whitebox/ > $ hg mv test/hotspot/jtreg/sanity/WBApi.java test/lib-test/sun/hotspot/whitebox/ > testing: test/lib-test tests > > > Thanks, > -- Igor > > From igor.ignatyev at oracle.com Wed Jun 17 14:43:07 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Jun 2020 07:43:07 -0700 Subject: RFR(T) [15] : 8247716 : JVM_RegisterWhiteBoxMethods checks wrong classloader In-Reply-To: <5022bca4-f096-2032-306c-effec17692d1@oracle.com> References: <81982F2D-5DA5-485A-85BF-AC96DE6005BA@oracle.com> <5022bca4-f096-2032-306c-effec17692d1@oracle.com> Message-ID: Hi Dan, thanks for your review, pushed. -- Igor > On Jun 17, 2020, at 7:28 AM, Daniel D. Daugherty wrote: > > On 6/16/20 11:39 PM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8247716/webrev.00 > > src/hotspot/share/prims/whitebox.cpp > No comments. > > Thumbs up and I agree that this is a trivial fix. > > Dan > > >>> 1 line changed: 0 ins; 0 del; 1 mod; >> >> Hi all, >> >> could you please review this one-liner fix for JVM_RegisterWhiteBoxMethods method? >> from JBS: >>> JVM_RegisterWhiteBoxMethods is supposed to check that sun.hotspot.WhiteBox class is loaded by the boot classloader, yet it actually checks that java.lang.Class class is. >> note: this had no impact, as NativeLookup::lookup_style would use JVM_RegisterWhiteBoxMethods as sun.hotspot.WhiteBox::registerNatives only if the class was load by the boot loader. >> >> testing: tier1 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8247716 >> webrev: http://cr.openjdk.java.net/~iignatyev//8247716/webrev.00 >> >> Thanks, >> -- Igor > From igor.ignatyev at oracle.com Wed Jun 17 14:43:28 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Jun 2020 07:43:28 -0700 Subject: RFR(T) [15] : 8247725 : move two tests for whitebox from test/hotspot/jtreg/sanity to test/lib-test In-Reply-To: References: Message-ID: Hi Dan, thanks for your review, pushed. -- Igor > On Jun 17, 2020, at 7:32 AM, Daniel D. Daugherty wrote: > > On 6/17/20 12:49 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8247725/webrev.00/ > > No comments. The webrev clearly shows the renames. > > Thumbs up and I agree that this is a trivial change. > > Dan > > >>> 0 lines changed: 0 ins; 0 del; 0 mod; >> Hi all, >> >> could you please review this zero-liner which moves MismatchedWhiteBox/WhiteBox.java and WBApi.java tests from test/hotspot/jtreg test suite to test/lib-test test suite? >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8247725 >> webrev: http://cr.openjdk.java.net/~iignatyev/8247725/webrev.00/ >> $ hg mv test/hotspot/jtreg/sanity/MismatchedWhiteBox test/lib-test/sun/hotspot/whitebox/ >> $ hg mv test/hotspot/jtreg/sanity/WBApi.java test/lib-test/sun/hotspot/whitebox/ >> testing: test/lib-test tests >> >> >> Thanks, >> -- Igor >> >> > From shade at redhat.com Wed Jun 17 15:11:21 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Jun 2020 17:11:21 +0200 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On 6/15/20 11:58 PM, Mandy Chung wrote: > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. Note this would break otherwise innocuous introspection for records, for example dumping the VM layout with JOL: http://openjdk.java.net/projects/code-tools/jol/ Amusingly, with the rest of Unsafe available the barn is still full open. Here's the sketch for the workaround: get the field list with getDeclaredFields, optionally guess the VM layout (the rules are not that hard, and JOL does it routinely), instantiate a Record with whatever arguments, poke the test patterns with Unsafe.put*, read them back from record components to verify. It would get me the same objectFieldOffset in a round-about way. And speaking from JOL maintainer standpoint, that one would be very tempting to do, because it would not depend on whatever protection shenanigans a particular JDK release tries to enforce. > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` or > `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. Are we absolutely sure that what ObjectInputStream.readRecord() does fits the 3rd party serialization libraries? Does it work for them performance-wise? (Are we even sure about that for JDK itself?) Because if it is not, 3rd party lib maintainers would proceed to hacking in the "objectFieldOffset workaround" and we would get the cobra-effect-like strengthening of dependency on Unsafe quirks. > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), then > all bets are off. It is fun to consider JNI to be more powerful than Unsafe. This seems backwards. The intent to break Unsafe users might be defensible, but this power oddity is still quite odd. -- Thanks, -Aleksey From mandy.chung at oracle.com Wed Jun 17 16:50:44 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Jun 2020 09:50:44 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On 6/17/20 8:11 AM, Aleksey Shipilev wrote: > On 6/15/20 11:58 PM, Mandy Chung wrote: >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > Note this would break otherwise innocuous introspection for records, for example dumping the VM > layout with JOL: > http://openjdk.java.net/projects/code-tools/jol/ To me, JOL needs a supported API in the serviceability area to introspect a field layout.? JOL will not work for inline types without adding new APIs. sun.misc.Unsafe is an unsupported API.?? My proposal for sun.misc.Unsafe (in fact jdk.unsupported module) is to keep the support for legacy use but not necessarily for new features. > > Amusingly, with the rest of Unsafe available the barn is still full open. Here's the sketch for the > workaround: get the field list with getDeclaredFields, optionally guess the VM layout (the rules are > not that hard, and JOL does it routinely), instantiate a Record with whatever arguments, poke the > test patterns with Unsafe.put*, read them back from record components to verify. It would get me the > same objectFieldOffset in a round-about way. I'm aware of these workarounds in the wild.?? As sun.misc.Unsafe is JDK unsupported API, this patch does not attempt to implement a complete solution but adds an obvious notification informing the frameworks that `sun.misc.Unsafe::objectFieldOffset` and `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. Unsafe is unsafe!?? Use it at your own risk. > And speaking from JOL maintainer standpoint, that one would be very tempting to do, because it would > not depend on whatever protection shenanigans a particular JDK release tries to enforce. > >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` or >> `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via its >> canonical constructor as done by the Java serialization. > Are we absolutely sure that what ObjectInputStream.readRecord() does fits the 3rd party > serialization libraries? Does it work for them performance-wise? There should be performance improvement opportunity (see Peter Levart's good work - JDK-8247532 [1]) > (Are we even sure about that for JDK itself?) Java serialization support for records use its canonical constructor.? (I'm not sure what you tried to point out by this). > Because if it is not, 3rd party lib maintainers would proceed to hacking in the > "objectFieldOffset workaround" and we would get the cobra-effect-like strengthening of dependency on > Unsafe quirks. This is exactly why we request this for 15 so that 3rd-party lib maintainers can prototype and send feedback.? We love to understand any stumbling block and work together to resolve any issue before records exit preview. > >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static and >> instance fields).? But JNI has superpower and the current spec already >> allows to modify the value of a final static field even after it's >> constant folded (via JNI SetField and SetStaticField), then >> all bets are off. > It is fun to consider JNI to be more powerful than Unsafe. This seems backwards. The intent to break > Unsafe users might be defensible, but this power oddity is still quite odd. > I think you misread the message.? There is no claim whether JNI or Unsafe is more powerful. JNI is a supported API.?? The above explains why I propose no spec change to make in JNI.? OTOH jdk.unsupported is unsupported but has more love by frameworks to avoid writing in native.? Adding a check in some sun.misc.Unsafe APIs (even it can be hacked to workaround it) is a clear notification what it does not intend to support. Hope this helps. Mandy [1] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067223.html From mandy.chung at oracle.com Wed Jun 17 17:26:11 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Jun 2020 10:26:11 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <45d5c949-5cd0-a9ee-b747-17fd10c683fe@oracle.com> Hi Vladimir, Thanks for the review and feedback. FWIW.? The spec of java.lang.reflect.AccessibleObject and Field specifies which final fields can be modified or not.? It use the *non-modifiable* term and JVM can rely on. I will keep the patch as is.? I'm happy to clean this up when a more disciplined approach is coming. Mandy On 6/17/20 4:13 AM, Vladimir Ivanov wrote: > Looks good! > > I welcome and fully support such hardening of final field handling for > new language features. > > Minor comment on naming: "trusted final field" is an JVM > implementation detail and I'd prefer to keep it internal to JVM. (I > hope it'll eventually go away and will be replaced with a more > disciplined approach.) > > Probably, a better alternative is a name that clearly communicates > that the JVM expects/allows modifications of a final field. Or, at > least, that it's *the JVM* which "trusts" the contents of a final > field after its initialization is over > (vmTrustedFinal/isVMTrustedFinalField). > > But I'm fine with keeping the patch as is. > > Best regards, > Vladimir Ivanov > > On 16.06.2020 00:58, Mandy Chung wrote: >> This patch is joint contribution from Christoph Dreis [1] and me. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >> >> This proposes to make final fields in records notmodifiable via >> reflection.? Field::set throws IAE if a Field is not modifiable. >> Thecurrent specification specifies the following Fields not modifiable: >> - static final fields in any class >> - final fields in a hidden class >> >> The spec of Field::set is changed to specify that records are not >> modifiable via reflection. >> ??Noe that no change in Field::setAccessible(true), i.e. it will >> succeed to allow existing frameworks to have read access to final >> fields in records.? Just no write access. >> >> VarHandle does not support write access if it's a static final field >> or an instance final field. >> >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >> >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static >> and instance fields).? But JNI has superpower and the current spec >> already allows to modify the value of a final static field even after >> it's constant folded (via JNI SetField and >> SetStaticField), then all bets are off.? This should be >> re-visited when we consider "final is truly final" for all classes. >> >> Make final fields in records not modifiable via reflection enables >> JIT optimization as these final fields are effectively truly final. >> >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` >> or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via >> its canonical constructor as done by the Java serialization. >> >> I see this change gives a good opportunity to engage the maintainers >> of the serialization frameworks and work together to support new >> features including records, inline classes and the new serialization >> mechanism and which I think it is worth the investment. >> >> This is a low risk enhancement.? I'd like to request approval for a >> late enhancement in JDK 15.? It extends the pre-existing code path >> with refactoring the hidden classes to prepare for new kinds of >> classes with trusted final fields.? The change is straight-forward. >> >> Can this wait to integrate in JDK 16? >> >> ?? It's important to get this enhancement in when record is a preview >> feature that we can get feedback and give time to 3rd-party >> serialization frameworks to look into adding the support for records. >> If we delayed this change in 16 and records exit preview, it would be >> bad for frameworks if they verify that they support records in 15 but >> fail in 16.? OTOH the risk of this patch is low. >> >> Performance Impact >> >> I addressed the performance concern I raised earlier myself. For >> reflection, VM creates the reflective Field objects and fills in >> MemberName when resolving a member.? VM will tag if this >> Field/MemberName is trusted final field.? I think this is a cleaner >> approach rather than in each place to check for final static and >> final fields in hidden or record class to determine if it has write >> access or not. >> >> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >> Unsafe has been allowing access of static final fields of any classes >> but isTrustedFinalField is not limited to instance fields (2) Unsafe >> disallows access to all fields in a hidden class (not limited to >> trusted final fields).? So it follows the precedence and simply >> checks if the declaring class is a record. `Class::isRecord` calls >> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >> `Class::getSuperclass` is intrinsified, the call on isRecord on a >> normal class is fast. Christoph has contributed the microbenchmarks >> that confirm that no performance regression. >> >> Thanks >> Mandy >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html >> From coleen.phillimore at oracle.com Wed Jun 17 21:25:25 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 17 Jun 2020 17:25:25 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage Message-ID: Summary: Remove JVMTI oops_do calls from JVMTI and GCs Tested with tier1-3, also built shenandoah to verify shenandoah changes. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8247808 Thanks, Coleen From david.holmes at oracle.com Wed Jun 17 23:49:49 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Jun 2020 09:49:49 +1000 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: References: Message-ID: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> Hi Coleen, On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: > Summary: Remove JVMTI oops_do calls from JVMTI and GCs > > Tested with tier1-3, also built shenandoah to verify shenandoah changes. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8247808 This is a nice cleanup and simplification of the code for working with OopStorage! So LGTM. One query ... I'm assuming that the processing previously done in JvmtiExport::oops_do is now done by OopStorageSet::vm_global()->oops_do. In most cases I can see the call to OopStorageSet::vm_global()->oops_do in the same vicinity as the call to JvmtiExport::oops_do, but not all i.e. ZRootsIterator::oops_do and ShenandoahSerialRoots::oops_do. Tracking through it seems that for those GCs the VM global roots are processed concurrently, whereas currently JVMTI roots are not. Does that make any potential difference? Thanks, David ----- > Thanks, > Coleen From coleen.phillimore at oracle.com Thu Jun 18 01:09:33 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 17 Jun 2020 21:09:33 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> Message-ID: On 6/17/20 7:49 PM, David Holmes wrote: > Hi Coleen, > > On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >> >> Tested with tier1-3, also built shenandoah to verify shenandoah changes. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8247808 > > This is a nice cleanup and simplification of the code for working with > OopStorage! So LGTM. Thanks, David. > > One query ... I'm assuming that the processing previously done in > JvmtiExport::oops_do is now done by > OopStorageSet::vm_global()->oops_do. In most cases I can see the call > to OopStorageSet::vm_global()->oops_do in the same vicinity as the > call to JvmtiExport::oops_do, but not all i.e. ZRootsIterator::oops_do > and ShenandoahSerialRoots::oops_do. Tracking through it seems that for > those GCs the VM global roots are processed concurrently, whereas > currently JVMTI roots are not. Does that make any potential difference? ZGC and Shenandoah are better because when the vm_global() roots grow, they'll be processed faster.? Because accessing oops in OopStorage uses resolve() which uses the Access API, any oops will be marked or fixed when accessed if the GC hasn't yet gotten to this oop in it's concurrent processing. Kim noticed that G1 and ParallelGC should be processing these roots in parallel (with many threads, since OopStorage has that support) and he's going to or has filed a bug to fix it.? As we add more things to OopStorage (see upcoming RFRs), this will become important. Thanks, Coleen > > Thanks, > David > ----- > >> Thanks, >> Coleen From serguei.spitsyn at oracle.com Thu Jun 18 01:46:16 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 17 Jun 2020 18:46:16 -0700 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: References: Message-ID: <0a3648ec-27be-7911-b7b3-cff48f55f793@oracle.com> Hi Coleen, Nice simplification! It looks good to me. I assume you will run the nsk.jvmti tests. Thanks, Serguei On 6/17/20 14:25, coleen.phillimore at oracle.com wrote: > Summary: Remove JVMTI oops_do calls from JVMTI and GCs > > Tested with tier1-3, also built shenandoah to verify shenandoah changes. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8247808 > > Thanks, > Coleen From serguei.spitsyn at oracle.com Thu Jun 18 02:34:34 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 17 Jun 2020 19:34:34 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <18a21b47-9815-270d-3859-7d908d3fc165@oracle.com> Hi Mandy, This looks good from the Serviceability point of view. > No change is made in JNI.? JNI could be considered to disallow modification of > final fields in records and hidden classes (static and instance fields). > But JNI has superpower and the current spec already allows to modify > the value of a final static field even after it's constant folded > (via JNI SetField and SetStaticField), then all bets are off. > This should be re-visited when we consider "final is truly final" for all classes. This can potentially impact the JDWP agent as it uses the JNI to set fields values. Please, see ObjectReference#setValue: https://docs.oracle.com/en/java/javase/14/docs/api/jdk.jdi/com/sun/jdi/ObjectReference.html#setValue(com.sun.jdi.Field,com.sun.jdi.Value) Thanks, Serguei On 6/15/20 14:58, Mandy Chung wrote: > This patch is joint contribution from Christoph Dreis [1] and me. > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 > > This proposes to make final fields in records notmodifiable via > reflection.? Field::set throws IAE if a Field is not modifiable. > Thecurrent specification specifies the following Fields not modifiable: > - static final fields in any class > - final fields in a hidden class > > The spec of Field::set is changed to specify that records are not > modifiable via reflection. > ?Noe that no change in Field::setAccessible(true), i.e. it will > succeed to allow existing frameworks to have read access to final > fields in records.? Just no write access. > > VarHandle does not support write access if it's a static final field > or an instance final field. > > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), > then all bets are off.? This should be re-visited when we consider > "final is truly final" for all classes. > > Make final fields in records not modifiable via reflection enables JIT > optimization as these final fields are effectively truly final. > > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` > or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. > > I see this change gives a good opportunity to engage the maintainers > of the serialization frameworks and work together to support new > features including records, inline classes and the new serialization > mechanism and which I think it is worth the investment. > > This is a low risk enhancement.? I'd like to request approval for a > late enhancement in JDK 15.? It extends the pre-existing code path > with refactoring the hidden classes to prepare for new kinds of > classes with trusted final fields.? The change is straight-forward. > > Can this wait to integrate in JDK 16? > > ? It's important to get this enhancement in when record is a preview > feature that we can get feedback and give time to 3rd-party > serialization frameworks to look into adding the support for records.? > If we delayed this change in 16 and records exit preview, it would be > bad for frameworks if they verify that they support records in 15 but > fail in 16.? OTOH the risk of this patch is low. > > Performance Impact > > I addressed the performance concern I raised earlier myself.? For > reflection, VM creates the reflective Field objects and fills in > MemberName when resolving a member.? VM will tag if this > Field/MemberName is trusted final field.? I think this is a cleaner > approach rather than in each place to check for final static and final > fields in hidden or record class to determine if it has write access > or not. > > `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) > Unsafe has been allowing access of static final fields of any classes > but isTrustedFinalField is not limited to instance fields (2) Unsafe > disallows access to all fields in a hidden class (not limited to > trusted final fields).? So it follows the precedence and simply checks > if the declaring class is a record. `Class::isRecord` calls > `Class::getSuperclass` to check if it's a subtype of `Record`.? As > `Class::getSuperclass` is intrinsified, the call on isRecord on a > normal class is fast. Christoph has contributed the microbenchmarks > that confirm that no performance regression. > > Thanks > Mandy > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html From mandy.chung at oracle.com Thu Jun 18 02:42:13 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Jun 2020 19:42:13 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <18a21b47-9815-270d-3859-7d908d3fc165@oracle.com> References: <18a21b47-9815-270d-3859-7d908d3fc165@oracle.com> Message-ID: <1a95d2f6-e070-aced-aeeb-3e699bd75ad4@oracle.com> On 6/17/20 7:34 PM, serguei.spitsyn at oracle.com wrote: > Hi Mandy, > > This looks good from the Serviceability point of view. > > > No change is made in JNI.? JNI could be considered to disallow > modification of > > final fields in records and hidden classes (static and instance > fields). > > But JNI has superpower and the current spec already allows to modify > > the value of a final static field even after it's constant folded > > (via JNI SetField and SetStaticField), then all bets are > off. > > This should be re-visited when we consider "final is truly final" > for all classes. > > This can potentially impact the JDWP agent as it uses the JNI to set > fields values. > Please, see ObjectReference#setValue: > https://docs.oracle.com/en/java/javase/14/docs/api/jdk.jdi/com/sun/jdi/ObjectReference.html#setValue(com.sun.jdi.Field,com.sun.jdi.Value) > > Thanks for pointing out this API.?? I will file a RFE to follow this up. Mandy > Thanks, > Serguei > > > On 6/15/20 14:58, Mandy Chung wrote: >> This patch is joint contribution from Christoph Dreis [1] and me. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >> >> This proposes to make final fields in records notmodifiable via >> reflection.? Field::set throws IAE if a Field is not modifiable. >> Thecurrent specification specifies the following Fields not modifiable: >> - static final fields in any class >> - final fields in a hidden class >> >> The spec of Field::set is changed to specify that records are not >> modifiable via reflection. >> ?Noe that no change in Field::setAccessible(true), i.e. it will >> succeed to allow existing frameworks to have read access to final >> fields in records.? Just no write access. >> >> VarHandle does not support write access if it's a static final field >> or an instance final field. >> >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >> >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static >> and instance fields).? But JNI has superpower and the current spec >> already allows to modify the value of a final static field even after >> it's constant folded (via JNI SetField and >> SetStaticField), then all bets are off.? This should be >> re-visited when we consider "final is truly final" for all classes. >> >> Make final fields in records not modifiable via reflection enables >> JIT optimization as these final fields are effectively truly final. >> >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` >> or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via >> its canonical constructor as done by the Java serialization. >> >> I see this change gives a good opportunity to engage the maintainers >> of the serialization frameworks and work together to support new >> features including records, inline classes and the new serialization >> mechanism and which I think it is worth the investment. >> >> This is a low risk enhancement.? I'd like to request approval for a >> late enhancement in JDK 15.? It extends the pre-existing code path >> with refactoring the hidden classes to prepare for new kinds of >> classes with trusted final fields.? The change is straight-forward. >> >> Can this wait to integrate in JDK 16? >> >> ? It's important to get this enhancement in when record is a preview >> feature that we can get feedback and give time to 3rd-party >> serialization frameworks to look into adding the support for >> records.? If we delayed this change in 16 and records exit preview, >> it would be bad for frameworks if they verify that they support >> records in 15 but fail in 16.? OTOH the risk of this patch is low. >> >> Performance Impact >> >> I addressed the performance concern I raised earlier myself. For >> reflection, VM creates the reflective Field objects and fills in >> MemberName when resolving a member.? VM will tag if this >> Field/MemberName is trusted final field.? I think this is a cleaner >> approach rather than in each place to check for final static and >> final fields in hidden or record class to determine if it has write >> access or not. >> >> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >> Unsafe has been allowing access of static final fields of any classes >> but isTrustedFinalField is not limited to instance fields (2) Unsafe >> disallows access to all fields in a hidden class (not limited to >> trusted final fields).? So it follows the precedence and simply >> checks if the declaring class is a record. `Class::isRecord` calls >> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >> `Class::getSuperclass` is intrinsified, the call on isRecord on a >> normal class is fast. Christoph has contributed the microbenchmarks >> that confirm that no performance regression. >> >> Thanks >> Mandy >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html > From ioi.lam at oracle.com Thu Jun 18 06:02:39 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 17 Jun 2020 23:02:39 -0700 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> Message-ID: <3f3ee0a9-013b-76af-52ff-905d63098825@oracle.com> Hi Coleen, This looks good to me. G1GCPhaseTimes::phase_name and enum GCParPhases should really be defined using an X-macro, but that's a different RFE. Thanks - Ioi On 6/17/20 6:09 PM, coleen.phillimore at oracle.com wrote: > > > On 6/17/20 7:49 PM, David Holmes wrote: >> Hi Coleen, >> >> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>> >>> Tested with tier1-3, also built shenandoah to verify shenandoah >>> changes. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8247808 >> >> This is a nice cleanup and simplification of the code for working >> with OopStorage! So LGTM. > > Thanks, David. >> >> One query ... I'm assuming that the processing previously done in >> JvmtiExport::oops_do is now done by >> OopStorageSet::vm_global()->oops_do. In most cases I can see the call >> to OopStorageSet::vm_global()->oops_do in the same vicinity as the >> call to JvmtiExport::oops_do, but not all i.e. >> ZRootsIterator::oops_do and ShenandoahSerialRoots::oops_do. Tracking >> through it seems that for those GCs the VM global roots are processed >> concurrently, whereas currently JVMTI roots are not. Does that make >> any potential difference? > > ZGC and Shenandoah are better because when the vm_global() roots grow, > they'll be processed faster.? Because accessing oops in OopStorage > uses resolve() which uses the Access API, any oops will be marked or > fixed when accessed if the GC hasn't yet gotten to this oop in it's > concurrent processing. > > Kim noticed that G1 and ParallelGC should be processing these roots in > parallel (with many threads, since OopStorage has that support) and > he's going to or has filed a bug to fix it.? As we add more things to > OopStorage (see upcoming RFRs), this will become important. > > Thanks, > Coleen > >> >> Thanks, >> David >> ----- >> >>> Thanks, >>> Coleen > From kim.barrett at oracle.com Thu Jun 18 06:34:57 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 18 Jun 2020 02:34:57 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> <5EE5AB71.20505@oracle.com> Message-ID: > On Jun 15, 2020, at 7:41 AM, Kim Barrett wrote: > >> On Jun 14, 2020, at 12:45 AM, Philip Race wrote: >> >> Kim, >> >> >> Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. >> Please rename the JEP. After some off-list discussions I have a better idea of what Phil was asking for and why. In response I've changed the JEP, replacing a few occurrences of "HotSpot" with "the JDK", as described below. All other occurrences of "HotSpot" have been left as-is. Title: JEP 347: Adopt C++14 Language Features in HotSpot => JEP 347: Adopt C++14 Language Features in the JDK Summary: Allow the use of selected C++14 language features in the HotSpot C++ source code. => Allow the use of selected C++14 language features in the JDK C++ source code. Goals: The purpose of this JEP is to formally allow C++ source code changes within HotSpot to take advantage of some C++14 standard language features. => The purpose of this JEP is to formally allow C++ source code changes within the JDK to take advantage of some C++14 standard language features. From stefan.karlsson at oracle.com Thu Jun 18 07:25:37 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 18 Jun 2020 09:25:37 +0200 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: References: Message-ID: <94da70e9-2c61-a97d-e31a-6e0ec4478faa@oracle.com> Hi Coleen, On 2020-06-17 23:25, coleen.phillimore at oracle.com wrote: > Summary: Remove JVMTI oops_do calls from JVMTI and GCs > > Tested with tier1-3, also built shenandoah to verify shenandoah changes. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev https://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev/src/hotspot/share/prims/jvmtiImpl.cpp.udiff.html JvmtiBreakpoint::~JvmtiBreakpoint() { - if (_class_holder != NULL) { - NativeAccess<>::oop_store(_class_holder, (oop)NULL); - OopStorageSet::vm_global()->release(_class_holder); + if (_class_holder.resolve() != NULL) { + _class_holder.release(); } } Could this be changed to peek() / release() instead? The resolve() call is going to keep the object alive until next for ZGC marking cycle. The rest looks OK. Below are some comments about things that I find odd and non-obvious from reading the code, and may be potentials for cleanups to make it easier for the next to understand the code: The above code assumes that as soon as OopHandle::create has been called, we won't store NULL into the _obj pointer. If someone does, then we would leak the memory. OopHandle has a function ptr_raw, that allows someone to clear the _obj pointer. I have to assume that this function isn't used in this code. --- 214 void JvmtiBreakpoint::copy(JvmtiBreakpoint& bp) { 215 _method = bp._method; 216 _bci = bp._bci; 217 _class_holder = OopHandle::create(bp._class_holder.resolve()); 218 } This one looks odd, because the _class_holder is overwritten without releasing the old OopHandle. This is currently OK, because copy is only called from clone, which just created a new JvmtiBreakpoint: ? GrowableElement *clone()??????? { ??? JvmtiBreakpoint *bp = new JvmtiBreakpoint(); ??? bp->copy(*this); ??? return bp; ? } ?I think this would have been much more obvious if copy/clone were a copy constructor. With that said, it looks like we now have two JvmtiBreakpoints with the same OopHandle contents. So, OopHandle::release will be called twice. Now that works because release clears the oop value: inline void OopHandle::release() { ? // Clear the OopHandle first ? NativeAccess<>::oop_store(_obj, (oop)NULL); ? OopStorageSet::vm_global()->release(_obj); } and the resolve() != NULL check will prevent the OopHandle from being released twice: + if (_class_holder.resolve() != NULL) { + _class_holder.release(); } StefanK > bug link https://bugs.openjdk.java.net/browse/JDK-8247808 > > Thanks, > Coleen From thomas.schatzl at oracle.com Thu Jun 18 07:58:00 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 18 Jun 2020 09:58:00 +0200 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> Message-ID: Hi, On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: > > > On 6/17/20 7:49 PM, David Holmes wrote: >> Hi Coleen, >> >> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>> >>> Tested with tier1-3, also built shenandoah to verify shenandoah changes. >>> [...] > > Kim noticed that G1 and ParallelGC should be processing these roots in > parallel (with many threads, since OopStorage has that support) and he's > going to or has filed a bug to fix it.? As we add more things to > OopStorage (see upcoming RFRs), this will become important. > I do not know which exact roots you want to move into OopStorage, but I would like to mention this concern: with moving everything into a single OopStorage (i.e. vm_globals in this case), I am worried that every time important information about the source for these gets lost. Which makes it hard to understand from where these oops came from when there is a performance problem in the "VM Globals" bucket. This may not apply to JVMTI oops, but others may occasionally have a significant amount of oops where it would be very interesting to know from where a particular slowdown comes from. So I would prefer keep some accounting here. Thanks, Thomas From christian.hagedorn at oracle.com Thu Jun 18 10:36:19 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Thu, 18 Jun 2020 12:36:19 +0200 Subject: [16] RFR(S): 8247218: Add default constructor to VectorSet to use Thread::current()->resource_area() as arena by default Message-ID: <55011fe1-2307-ad66-ea39-60a3b9c00682@oracle.com> Hi Please review the following cleanup: https://bugs.openjdk.java.net/browse/JDK-8247218 http://cr.openjdk.java.net/~chagedorn/8247218/webrev.00/ By adding an additional default constructor in VectorSet (as also present in Node_List, Node_Stack etc.), we can replace the often found pattern Arena* a = Thread::current()->resource_area(); VectorSet visited(a); by a simple VectorSet visited; I additionally cleaned up some other places where this pattern was found, for example for Node_List etc. Thank you! Best regards, Christian From coleen.phillimore at oracle.com Thu Jun 18 11:39:34 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 18 Jun 2020 07:39:34 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <0a3648ec-27be-7911-b7b3-cff48f55f793@oracle.com> References: <0a3648ec-27be-7911-b7b3-cff48f55f793@oracle.com> Message-ID: On 6/17/20 9:46 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > Nice simplification! > It looks good to me. > I assume you will run the nsk.jvmti tests. Thanks Serguei.? The nsk jvmti tests are run with tier3, I believe but I'll run them on the command line to verify. thanks! Coleen > > Thanks, > Serguei > > > On 6/17/20 14:25, coleen.phillimore at oracle.com wrote: >> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >> >> Tested with tier1-3, also built shenandoah to verify shenandoah changes. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8247808 >> >> Thanks, >> Coleen > From shade at redhat.com Thu Jun 18 12:43:20 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 18 Jun 2020 14:43:20 +0200 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On 6/17/20 6:50 PM, Mandy Chung wrote: > On 6/17/20 8:11 AM, Aleksey Shipilev wrote: >> On 6/15/20 11:58 PM, Mandy Chung wrote: >> Amusingly, with the rest of Unsafe available the barn is still full open. Here's the sketch for the >> workaround: get the field list with getDeclaredFields, optionally guess the VM layout (the rules are >> not that hard, and JOL does it routinely), instantiate a Record with whatever arguments, poke the >> test patterns with Unsafe.put*, read them back from record components to verify. It would get me the >> same objectFieldOffset in a round-about way. > > I'm aware of these workarounds in the wild.?? As sun.misc.Unsafe is JDK unsupported API, this patch > does not attempt to implement a complete solution but adds an obvious notification informing the > frameworks that `sun.misc.Unsafe::objectFieldOffset` and `sun.misc.Unsafe::staticField{Offset/Base}` > not to support records. I understand the intent. My point is that intent is mistimed during this time. I understand this tripwire needs to be put while Records are still in preview. My point is that it cannot be put in before Records serialization story has the preview-proven answer. The intent also looks rather opaque. The intent is to notify maintainers, fine. I am one of the maintainers (although not of serialization library, but still heavy-Unsafe one), so hear me out. I came up with the workaround above in about 15 minutes after someone pointed out the exception to me. Have it crossed my mind that maybe JDK developers want some help here? No. It looks like another impediment that should be solved on the spot. Does that exception communicate any intent at all? The message is generic. There is no comment. How would you expect to push maintainers to think along the "we should be collaborating to find a proper way to do this", instead of "this is set in stone; let me hack this around too"? This is the actionable bit: At very least the exception message should say something about the intent. And maybe even the comment in Unsafe.java should point to the discussion about this intent and maybe even provide the breadcrumbs to follow, e.g. to ObjectInputStream.readRecord(). >> Are we absolutely sure that what ObjectInputStream.readRecord() does fits the 3rd party >> serialization libraries? Does it work for them performance-wise? > > There should be performance improvement opportunity (see Peter Levart's good work - JDK-8247532 [1]) > >> (Are we even sure about that for JDK itself?) > > Java serialization support for records use its canonical constructor.? (I'm not sure what you tried > to point out by this). Let me try again. The existence of JDK-8247532 solidifies my concern: the JDK code itself had not yet figured out how to do Record serialization fast! Now put yourself in the shoes of the 3rd party lib maintainer: Unsafe is forbidden, JDK code itself is slow. Asking 3rd party lib maintainer to find whatever intricate incantation they should use to get decent performance -- while JDK itself is still working on it! -- puts them into weird position, from where the *sane* way out is to hack around the Unsafe restriction. I mean, it is as enticing as hacking the JDK code itself, with the added benefit of working across the JDK releases. This is time and again the core of Unsafe discussion, which boils down to a very simple request: if JDK developers take away some Unsafe APIs, 3rd party developers need to have the known good replacement for it. If that does not happen, 3rd party developers would invest in doing more Unsafe hacks. Record serialization seems to fall into the same trap. To reiterate: the non-Unsafe code should be proven as the viable alternative before breaking Unsafe. Otherwise, we only deepen the dependency on Unsafe. >> Because if it is not, 3rd party lib maintainers would proceed to hacking in the >> "objectFieldOffset workaround" and we would get the cobra-effect-like strengthening of dependency on >> Unsafe quirks. > > This is exactly why we request this for 15 so that 3rd-party lib maintainers can prototype and send > feedback.? We love to understand any stumbling block and work together to resolve any issue before > records exit preview. Again, JDK-8247532 is the writing on the wall: we don't need 3rd party developers to tell if Record serialization works fast in 15 -- we already know it does not. >> It is fun to consider JNI to be more powerful than Unsafe. This seems backwards. The intent to break >> Unsafe users might be defensible, but this power oddity is still quite odd. > > I think you misread the message.? There is no claim whether JNI or Unsafe is more powerful.? I don't think I did. Disallowing Unsafe to do something that JNI is allowed to do is making Unsafe less powerful. This was nothing to do with what API is supported or not. -- Thanks, -Aleksey From chris.hegarty at oracle.com Thu Jun 18 13:09:32 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 18 Jun 2020 14:09:32 +0100 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <4B268BF1-1DFD-4DCC-AEA3-7F2C7B81DC91@oracle.com> > On 18 Jun 2020, at 13:43, Aleksey Shipilev wrote: > >> ... > > Again, JDK-8247532 is the writing on the wall: we don't need 3rd party developers to tell if Record > serialization works fast in 15 -- we already know it does not. I disagree. JDK-8247532 is under review and well on its way towards JDK 15 (yes, during RDP 1). My reading of Peter?s benchmark result is that Record deserialization *does work fast*. What am I missing. -Chris. From bob.vandette at oracle.com Thu Jun 18 16:09:01 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 18 Jun 2020 12:09:01 -0400 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> <73C921E8-03A1-4CFA-BCA8-1FC8C46C2171@oracle.com> Message-ID: I fired up my cgroupv2 configured system and confirmed that getMemoryAndSwapLimit will indeed return -1 if the kernel does not have swap accounting enabled so I?m ok with your change. The memory.swap.* files are not present in the /sys/fs/cgroup file system which causes -1 to be returned. I agree that the fix for JDK-8236617 is no longer needed. Sorry for the delay, Bob. > On Jun 17, 2020, at 10:29 AM, Severin Gehwolf wrote: > > On Wed, 2020-06-17 at 09:09 -0400, Bob Vandette wrote: >> Should we just accept the kernel error message rather than add all the additional value checking? >> >> "out.shouldContain("Your kernel does not support swap limit capabilities?); > > I'd prefer to keep the other checks as well. The bug explains why that > is and if we see a config which fails this as well, something is wrong. > Hopefully that makes some sense. > >> Why isn?t the try block around the check for .shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: [1-9][0-9]+?); > > Why should it? For most set ups this catch block is never reached. The > only setup we are aware of which reaches this is the system Matthias > runs tests on. I've never heard of a system where this fails as well. > >> Why doesn?t this call fail as well?? > > Because the system values are being used for > OperatingSystemMXBean.getTotalSwapSpaceSize() when: > Metrics.getMemoryLimit() > 0 && Metrics.getMemoryAndSwapLimit() == -1 > >> Have you tried this test on a system with cgrouopv1 and v2 that doesn?t have swap enabled? > > I don't think so. I don't have access to such a system. > > How do you mean, though? A system which has swap disabled on the system > level or a container without swap? I believe the former is exactly the > case of Matthias' system. But I don't know. The latter should also be > covered via a test Jie added a while back. > >> I?m concerned that we?ll be back to fixing this when some other configuration combination fails. > > I understand but find it unlikely. Those are systems which are going to > be hard to detect correctly and do-the-right-thing. As mentioned in the > bug there is no way to distinguish between unlimited swap + a memory > limit from kernel-doesn't-support-swap-limits + a memory limit. > > Note that https://bugs.openjdk.java.net/browse/JDK-8236617 "fixed" this > for the old implementation. Unfortunately, it doesn't seem like it was > investigated why a 0 limit was being returned. That was when the old > implementation was still in place (prior cgroups v2 support). In the > old impl, when files were missing 0 was returned for some cases. In > fact, the fix of JDK-8236617 with the old impl would have wrongly > returned 0 for getTotalSwapSpaceSize() for a system which got > configured with unlimited swap size (--memory-swap=-1). In the new > impl, this returns physical swap size which seems better. > > On that note, the code of JDK-8236617 seems unreachable now. > > Thanks, > Severin > > >> Bob. >> >> >>> On Jun 17, 2020, at 5:13 AM, Severin Gehwolf wrote: >>> >>> On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: >>>> On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: >>>>>> Here would be my suggestion >>>>>> (hopefully this one isn't too restrictive again): >>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ >>>>> >>>>> Hi Severin, this worked nicely . >>>>> Thanks for providing this . >>>> >>>> Great. If somebody else is willing to OK this, I'll push it. >>> >>> Could I have a second review of this, please? I believe it's the best >>> we can do for this test failure Matthias is seeing. >>> >>> Thanks, >>> Severin >>> > From sgehwolf at redhat.com Thu Jun 18 16:57:21 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 18 Jun 2020 18:57:21 +0200 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> <73C921E8-03A1-4CFA-BCA8-1FC8C46C2171@oracle.com> Message-ID: <6368205a60ab894b2d4f24fb04a838d3454a149b.camel@redhat.com> On Thu, 2020-06-18 at 12:09 -0400, Bob Vandette wrote: > I fired up my cgroupv2 configured system and confirmed that getMemoryAndSwapLimit will indeed return -1 > if the kernel does not have swap accounting enabled so I?m ok with your change. > > The memory.swap.* files are not present in the /sys/fs/cgroup file system which causes -1 to be returned. Thank you Bob! > I agree that the fix for JDK-8236617 is no longer needed. OK. I've filed another bug to track this: https://bugs.openjdk.java.net/browse/JDK-8247863 > Sorry for the delay, No worries at all. Thanks, Severin > Bob. > > > > On Jun 17, 2020, at 10:29 AM, Severin Gehwolf wrote: > > > > On Wed, 2020-06-17 at 09:09 -0400, Bob Vandette wrote: > > > Should we just accept the kernel error message rather than add all the additional value checking? > > > > > > "out.shouldContain("Your kernel does not support swap limit capabilities?); > > > > I'd prefer to keep the other checks as well. The bug explains why that > > is and if we see a config which fails this as well, something is wrong. > > Hopefully that makes some sense. > > > > > Why isn?t the try block around the check for .shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: [1-9][0-9]+?); > > > > Why should it? For most set ups this catch block is never reached. The > > only setup we are aware of which reaches this is the system Matthias > > runs tests on. I've never heard of a system where this fails as well. > > > > > Why doesn?t this call fail as well?? > > > > Because the system values are being used for > > OperatingSystemMXBean.getTotalSwapSpaceSize() when: > > Metrics.getMemoryLimit() > 0 && Metrics.getMemoryAndSwapLimit() == -1 > > > > > Have you tried this test on a system with cgrouopv1 and v2 that doesn?t have swap enabled? > > > > I don't think so. I don't have access to such a system. > > > > How do you mean, though? A system which has swap disabled on the system > > level or a container without swap? I believe the former is exactly the > > case of Matthias' system. But I don't know. The latter should also be > > covered via a test Jie added a while back. > > > > > I?m concerned that we?ll be back to fixing this when some other configuration combination fails. > > > > I understand but find it unlikely. Those are systems which are going to > > be hard to detect correctly and do-the-right-thing. As mentioned in the > > bug there is no way to distinguish between unlimited swap + a memory > > limit from kernel-doesn't-support-swap-limits + a memory limit. > > > > Note that https://bugs.openjdk.java.net/browse/JDK-8236617 "fixed" this > > for the old implementation. Unfortunately, it doesn't seem like it was > > investigated why a 0 limit was being returned. That was when the old > > implementation was still in place (prior cgroups v2 support). In the > > old impl, when files were missing 0 was returned for some cases. In > > fact, the fix of JDK-8236617 with the old impl would have wrongly > > returned 0 for getTotalSwapSpaceSize() for a system which got > > configured with unlimited swap size (--memory-swap=-1). In the new > > impl, this returns physical swap size which seems better. > > > > On that note, the code of JDK-8236617 seems unreachable now. > > > > Thanks, > > Severin > > > > > > > Bob. > > > > > > > > > > On Jun 17, 2020, at 5:13 AM, Severin Gehwolf wrote: > > > > > > > > On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: > > > > > On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: > > > > > > > Here would be my suggestion > > > > > > > (hopefully this one isn't too restrictive again): > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ > > > > > > > > > > > > Hi Severin, this worked nicely . > > > > > > Thanks for providing this . > > > > > > > > > > Great. If somebody else is willing to OK this, I'll push it. > > > > > > > > Could I have a second review of this, please? I believe it's the best > > > > we can do for this test failure Matthias is seeing. > > > > > > > > Thanks, > > > > Severin > > > > From shade at redhat.com Thu Jun 18 17:10:08 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 18 Jun 2020 19:10:08 +0200 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <4B268BF1-1DFD-4DCC-AEA3-7F2C7B81DC91@oracle.com> References: <4B268BF1-1DFD-4DCC-AEA3-7F2C7B81DC91@oracle.com> Message-ID: <029fdb77-660b-4f59-7ee7-bc7020b5e49f@redhat.com> On 6/18/20 3:09 PM, Chris Hegarty wrote: >> On 18 Jun 2020, at 13:43, Aleksey Shipilev wrote: >> Again, JDK-8247532 is the writing on the wall: we don't need 3rd party developers to tell if Record >> serialization works fast in 15 -- we already know it does not. > > I disagree. JDK-8247532 is under review and well on its way towards JDK 15 (yes, during RDP 1). > My reading of Peter?s benchmark result is that Record deserialization *does work fast*. What am > I missing. JDK-8247532 is the evidence that Records serialization performance story is not clear. Even if we disregard that after JDK-8247532 [1] Records are still 8% slower, the _existence_ of JDK-8247532 indicates there are performance problems in the area. That evidence now needs to be compensated by much more evidence to the contrary. (Yes, I contracted a lot of Bayesian thinking from my statistician wife...) (Here were several paragraphs of further thoughts, but I realized it basically repeats what I said before.) -- Thanks, -Aleksey [1] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067223.html RecordSerializationBench.deserializeClasses: 31.049 ? 0.235 us/op RecordSerializationBench.deserializeRecords: 33.588 ? 0.394 us/op From bob.vandette at oracle.com Thu Jun 18 18:21:36 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 18 Jun 2020 14:21:36 -0400 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: <6368205a60ab894b2d4f24fb04a838d3454a149b.camel@redhat.com> References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> <73C921E8-03A1-4CFA-BCA8-1FC8C46C2171@oracle.com> <6368205a60ab894b2d4f24fb04a838d3454a149b.camel@redhat.com> Message-ID: <61A22292-D003-418A-B12A-4CC91B5605D5@oracle.com> Actually, I still think we might have an issue with your change on cgroupv2 systems when using podman. I just realized that we need the exact error string to match and I?m not seeing the same one. The error message you are matching may be coming from docker. There are a few other issues prohibiting us from fully testing on podman but it?s definitely a goal to get there. I wonder if you should skip the string check? I don?t even think we can guarantee this error message would occur on all Linux systems with podman since this one has the OCI string in it (Oracle Cloud Infrastructure). # podman run --memory 128m --memory-swap 300m -it ubuntu bash Error: writing file `memory.swap.max`: Permission denied: OCI runtime permission denied error Bob. > On Jun 18, 2020, at 12:57 PM, Severin Gehwolf wrote: > > On Thu, 2020-06-18 at 12:09 -0400, Bob Vandette wrote: >> I fired up my cgroupv2 configured system and confirmed that getMemoryAndSwapLimit will indeed return -1 >> if the kernel does not have swap accounting enabled so I?m ok with your change. >> >> The memory.swap.* files are not present in the /sys/fs/cgroup file system which causes -1 to be returned. > > Thank you Bob! > >> I agree that the fix for JDK-8236617 is no longer needed. > > OK. I've filed another bug to track this: > https://bugs.openjdk.java.net/browse/JDK-8247863 > >> Sorry for the delay, > > No worries at all. > > Thanks, > Severin > >> Bob. >> >> >>> On Jun 17, 2020, at 10:29 AM, Severin Gehwolf wrote: >>> >>> On Wed, 2020-06-17 at 09:09 -0400, Bob Vandette wrote: >>>> Should we just accept the kernel error message rather than add all the additional value checking? >>>> >>>> "out.shouldContain("Your kernel does not support swap limit capabilities?); >>> >>> I'd prefer to keep the other checks as well. The bug explains why that >>> is and if we see a config which fails this as well, something is wrong. >>> Hopefully that makes some sense. >>> >>>> Why isn?t the try block around the check for .shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: [1-9][0-9]+?); >>> >>> Why should it? For most set ups this catch block is never reached. The >>> only setup we are aware of which reaches this is the system Matthias >>> runs tests on. I've never heard of a system where this fails as well. >>> >>>> Why doesn?t this call fail as well?? >>> >>> Because the system values are being used for >>> OperatingSystemMXBean.getTotalSwapSpaceSize() when: >>> Metrics.getMemoryLimit() > 0 && Metrics.getMemoryAndSwapLimit() == -1 >>> >>>> Have you tried this test on a system with cgrouopv1 and v2 that doesn?t have swap enabled? >>> >>> I don't think so. I don't have access to such a system. >>> >>> How do you mean, though? A system which has swap disabled on the system >>> level or a container without swap? I believe the former is exactly the >>> case of Matthias' system. But I don't know. The latter should also be >>> covered via a test Jie added a while back. >>> >>>> I?m concerned that we?ll be back to fixing this when some other configuration combination fails. >>> >>> I understand but find it unlikely. Those are systems which are going to >>> be hard to detect correctly and do-the-right-thing. As mentioned in the >>> bug there is no way to distinguish between unlimited swap + a memory >>> limit from kernel-doesn't-support-swap-limits + a memory limit. >>> >>> Note that https://bugs.openjdk.java.net/browse/JDK-8236617 "fixed" this >>> for the old implementation. Unfortunately, it doesn't seem like it was >>> investigated why a 0 limit was being returned. That was when the old >>> implementation was still in place (prior cgroups v2 support). In the >>> old impl, when files were missing 0 was returned for some cases. In >>> fact, the fix of JDK-8236617 with the old impl would have wrongly >>> returned 0 for getTotalSwapSpaceSize() for a system which got >>> configured with unlimited swap size (--memory-swap=-1). In the new >>> impl, this returns physical swap size which seems better. >>> >>> On that note, the code of JDK-8236617 seems unreachable now. >>> >>> Thanks, >>> Severin >>> >>> >>>> Bob. >>>> >>>> >>>>> On Jun 17, 2020, at 5:13 AM, Severin Gehwolf wrote: >>>>> >>>>> On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: >>>>>> On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: >>>>>>>> Here would be my suggestion >>>>>>>> (hopefully this one isn't too restrictive again): >>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ >>>>>>> >>>>>>> Hi Severin, this worked nicely . >>>>>>> Thanks for providing this . >>>>>> >>>>>> Great. If somebody else is willing to OK this, I'll push it. >>>>> >>>>> Could I have a second review of this, please? I believe it's the best >>>>> we can do for this test failure Matthias is seeing. >>>>> >>>>> Thanks, >>>>> Severin >>>>> > From sgehwolf at redhat.com Thu Jun 18 18:48:30 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 18 Jun 2020 20:48:30 +0200 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: <61A22292-D003-418A-B12A-4CC91B5605D5@oracle.com> References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> <73C921E8-03A1-4CFA-BCA8-1FC8C46C2171@oracle.com> <6368205a60ab894b2d4f24fb04a838d3454a149b.camel@redhat.com> <61A22292-D003-418A-B12A-4CC91B5605D5@oracle.com> Message-ID: <761b35dba5f8ba6d732c5abf487c54e35ebec361.camel@redhat.com> On Thu, 2020-06-18 at 14:21 -0400, Bob Vandette wrote: > Actually, I still think we might have an issue with your change on cgroupv2 systems when using podman. > I just realized that we need the exact error string to match and I?m not seeing the same one. The error > message you are matching may be coming from docker. There are a few other issues prohibiting us from > fully testing on podman but it?s definitely a goal to get there. I wonder if you should skip the string check? > I don?t even think we can guarantee this error message would occur on all Linux systems with podman since > this one has the OCI string in it (Oracle Cloud Infrastructure). > > # podman run --memory 128m --memory-swap 300m -it ubuntu bash > Error: writing file `memory.swap.max`: Permission denied: OCI runtime permission denied error This version omits the check for the precise error message. Would this be OK? http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/04/webrev Thanks, Severin > Bob. > > > On Jun 18, 2020, at 12:57 PM, Severin Gehwolf wrote: > > > > On Thu, 2020-06-18 at 12:09 -0400, Bob Vandette wrote: > > > I fired up my cgroupv2 configured system and confirmed that getMemoryAndSwapLimit will indeed return -1 > > > if the kernel does not have swap accounting enabled so I?m ok with your change. > > > > > > The memory.swap.* files are not present in the /sys/fs/cgroup file system which causes -1 to be returned. > > > > Thank you Bob! > > > > > I agree that the fix for JDK-8236617 is no longer needed. > > > > OK. I've filed another bug to track this: > > https://bugs.openjdk.java.net/browse/JDK-8247863 > > > > > Sorry for the delay, > > > > No worries at all. > > > > Thanks, > > Severin > > > > > Bob. > > > > > > > > > > On Jun 17, 2020, at 10:29 AM, Severin Gehwolf wrote: > > > > > > > > On Wed, 2020-06-17 at 09:09 -0400, Bob Vandette wrote: > > > > > Should we just accept the kernel error message rather than add all the additional value checking? > > > > > > > > > > "out.shouldContain("Your kernel does not support swap limit capabilities?); > > > > > > > > I'd prefer to keep the other checks as well. The bug explains why that > > > > is and if we see a config which fails this as well, something is wrong. > > > > Hopefully that makes some sense. > > > > > > > > > Why isn?t the try block around the check for .shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: [1-9][0-9]+?); > > > > > > > > Why should it? For most set ups this catch block is never reached. The > > > > only setup we are aware of which reaches this is the system Matthias > > > > runs tests on. I've never heard of a system where this fails as well. > > > > > > > > > Why doesn?t this call fail as well?? > > > > > > > > Because the system values are being used for > > > > OperatingSystemMXBean.getTotalSwapSpaceSize() when: > > > > Metrics.getMemoryLimit() > 0 && Metrics.getMemoryAndSwapLimit() == -1 > > > > > > > > > Have you tried this test on a system with cgrouopv1 and v2 that doesn?t have swap enabled? > > > > > > > > I don't think so. I don't have access to such a system. > > > > > > > > How do you mean, though? A system which has swap disabled on the system > > > > level or a container without swap? I believe the former is exactly the > > > > case of Matthias' system. But I don't know. The latter should also be > > > > covered via a test Jie added a while back. > > > > > > > > > I?m concerned that we?ll be back to fixing this when some other configuration combination fails. > > > > > > > > I understand but find it unlikely. Those are systems which are going to > > > > be hard to detect correctly and do-the-right-thing. As mentioned in the > > > > bug there is no way to distinguish between unlimited swap + a memory > > > > limit from kernel-doesn't-support-swap-limits + a memory limit. > > > > > > > > Note that https://bugs.openjdk.java.net/browse/JDK-8236617 "fixed" this > > > > for the old implementation. Unfortunately, it doesn't seem like it was > > > > investigated why a 0 limit was being returned. That was when the old > > > > implementation was still in place (prior cgroups v2 support). In the > > > > old impl, when files were missing 0 was returned for some cases. In > > > > fact, the fix of JDK-8236617 with the old impl would have wrongly > > > > returned 0 for getTotalSwapSpaceSize() for a system which got > > > > configured with unlimited swap size (--memory-swap=-1). In the new > > > > impl, this returns physical swap size which seems better. > > > > > > > > On that note, the code of JDK-8236617 seems unreachable now. > > > > > > > > Thanks, > > > > Severin > > > > > > > > > > > > > Bob. > > > > > > > > > > > > > > > > On Jun 17, 2020, at 5:13 AM, Severin Gehwolf wrote: > > > > > > > > > > > > On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: > > > > > > > On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: > > > > > > > > > Here would be my suggestion > > > > > > > > > (hopefully this one isn't too restrictive again): > > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ > > > > > > > > > > > > > > > > Hi Severin, this worked nicely . > > > > > > > > Thanks for providing this . > > > > > > > > > > > > > > Great. If somebody else is willing to OK this, I'll push it. > > > > > > > > > > > > Could I have a second review of this, please? I believe it's the best > > > > > > we can do for this test failure Matthias is seeing. > > > > > > > > > > > > Thanks, > > > > > > Severin > > > > > > From bob.vandette at oracle.com Thu Jun 18 18:51:31 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 18 Jun 2020 14:51:31 -0400 Subject: [PING] RFR [XS]: 8244500: jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java In-Reply-To: <761b35dba5f8ba6d732c5abf487c54e35ebec361.camel@redhat.com> References: <7201E4D5-96AA-47C1-B0B7-A19E8A0F198B@tencent.com> <51519874123ae26a4d3ebf4863d80e682c13b989.camel@redhat.com> <0228a9ffd9bb316995dc95f4c06538ae7c21c285.camel@redhat.com> <4d09d097612dda9213f1d36f995f85bfb4887f0d.camel@redhat.com> <454bd1fc45dcce5ce7b4ce55b38a350b38928768.camel@redhat.com> <73C921E8-03A1-4CFA-BCA8-1FC8C46C2171@oracle.com> <6368205a60ab894b2d4f24fb04a838d3454a149b.camel@redhat.com> <61A22292-D003-418A-B12A-4CC91B5605D5@oracle.com> <761b35dba5f8ba6d732c5abf487c54e35ebec361.camel@redhat.com> Message-ID: > On Jun 18, 2020, at 2:48 PM, Severin Gehwolf wrote: > > On Thu, 2020-06-18 at 14:21 -0400, Bob Vandette wrote: >> Actually, I still think we might have an issue with your change on cgroupv2 systems when using podman. >> I just realized that we need the exact error string to match and I?m not seeing the same one. The error >> message you are matching may be coming from docker. There are a few other issues prohibiting us from >> fully testing on podman but it?s definitely a goal to get there. I wonder if you should skip the string check? >> I don?t even think we can guarantee this error message would occur on all Linux systems with podman since >> this one has the OCI string in it (Oracle Cloud Infrastructure). >> >> # podman run --memory 128m --memory-swap 300m -it ubuntu bash >> Error: writing file `memory.swap.max`: Permission denied: OCI runtime permission denied error > > This version omits the check for the precise error message. Would this > be OK? Yes, that works. Bob. > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/04/webrev > > Thanks, > Severin > > > >> Bob. >> >>> On Jun 18, 2020, at 12:57 PM, Severin Gehwolf wrote: >>> >>> On Thu, 2020-06-18 at 12:09 -0400, Bob Vandette wrote: >>>> I fired up my cgroupv2 configured system and confirmed that getMemoryAndSwapLimit will indeed return -1 >>>> if the kernel does not have swap accounting enabled so I?m ok with your change. >>>> >>>> The memory.swap.* files are not present in the /sys/fs/cgroup file system which causes -1 to be returned. >>> >>> Thank you Bob! >>> >>>> I agree that the fix for JDK-8236617 is no longer needed. >>> >>> OK. I've filed another bug to track this: >>> https://bugs.openjdk.java.net/browse/JDK-8247863 >>> >>>> Sorry for the delay, >>> >>> No worries at all. >>> >>> Thanks, >>> Severin >>> >>>> Bob. >>>> >>>> >>>>> On Jun 17, 2020, at 10:29 AM, Severin Gehwolf wrote: >>>>> >>>>> On Wed, 2020-06-17 at 09:09 -0400, Bob Vandette wrote: >>>>>> Should we just accept the kernel error message rather than add all the additional value checking? >>>>>> >>>>>> "out.shouldContain("Your kernel does not support swap limit capabilities?); >>>>> >>>>> I'd prefer to keep the other checks as well. The bug explains why that >>>>> is and if we see a config which fails this as well, something is wrong. >>>>> Hopefully that makes some sense. >>>>> >>>>>> Why isn?t the try block around the check for .shouldMatch("OperatingSystemMXBean\\.getFreeSwapSpaceSize: [1-9][0-9]+?); >>>>> >>>>> Why should it? For most set ups this catch block is never reached. The >>>>> only setup we are aware of which reaches this is the system Matthias >>>>> runs tests on. I've never heard of a system where this fails as well. >>>>> >>>>>> Why doesn?t this call fail as well?? >>>>> >>>>> Because the system values are being used for >>>>> OperatingSystemMXBean.getTotalSwapSpaceSize() when: >>>>> Metrics.getMemoryLimit() > 0 && Metrics.getMemoryAndSwapLimit() == -1 >>>>> >>>>>> Have you tried this test on a system with cgrouopv1 and v2 that doesn?t have swap enabled? >>>>> >>>>> I don't think so. I don't have access to such a system. >>>>> >>>>> How do you mean, though? A system which has swap disabled on the system >>>>> level or a container without swap? I believe the former is exactly the >>>>> case of Matthias' system. But I don't know. The latter should also be >>>>> covered via a test Jie added a while back. >>>>> >>>>>> I?m concerned that we?ll be back to fixing this when some other configuration combination fails. >>>>> >>>>> I understand but find it unlikely. Those are systems which are going to >>>>> be hard to detect correctly and do-the-right-thing. As mentioned in the >>>>> bug there is no way to distinguish between unlimited swap + a memory >>>>> limit from kernel-doesn't-support-swap-limits + a memory limit. >>>>> >>>>> Note that https://bugs.openjdk.java.net/browse/JDK-8236617 "fixed" this >>>>> for the old implementation. Unfortunately, it doesn't seem like it was >>>>> investigated why a 0 limit was being returned. That was when the old >>>>> implementation was still in place (prior cgroups v2 support). In the >>>>> old impl, when files were missing 0 was returned for some cases. In >>>>> fact, the fix of JDK-8236617 with the old impl would have wrongly >>>>> returned 0 for getTotalSwapSpaceSize() for a system which got >>>>> configured with unlimited swap size (--memory-swap=-1). In the new >>>>> impl, this returns physical swap size which seems better. >>>>> >>>>> On that note, the code of JDK-8236617 seems unreachable now. >>>>> >>>>> Thanks, >>>>> Severin >>>>> >>>>> >>>>>> Bob. >>>>>> >>>>>> >>>>>>> On Jun 17, 2020, at 5:13 AM, Severin Gehwolf wrote: >>>>>>> >>>>>>> On Fri, 2020-06-05 at 16:07 +0200, Severin Gehwolf wrote: >>>>>>>> On Fri, 2020-06-05 at 10:30 +0000, Baesken, Matthias wrote: >>>>>>>>>> Here would be my suggestion >>>>>>>>>> (hopefully this one isn't too restrictive again): >>>>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ >>>>>>>>> >>>>>>>>> Hi Severin, this worked nicely . >>>>>>>>> Thanks for providing this . >>>>>>>> >>>>>>>> Great. If somebody else is willing to OK this, I'll push it. >>>>>>> >>>>>>> Could I have a second review of this, please? I believe it's the best >>>>>>> we can do for this test failure Matthias is seeing. >>>>>>> >>>>>>> Thanks, >>>>>>> Severin >>>>>>> > From mandy.chung at oracle.com Thu Jun 18 19:29:34 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Jun 2020 12:29:34 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On 6/18/20 5:43 AM, Aleksey Shipilev wrote: > On 6/17/20 6:50 PM, Mandy Chung wrote: >> On 6/17/20 8:11 AM, Aleksey Shipilev wrote: >>> On 6/15/20 11:58 PM, Mandy Chung wrote: >>> Amusingly, with the rest of Unsafe available the barn is still full open. Here's the sketch for the >>> workaround: get the field list with getDeclaredFields, optionally guess the VM layout (the rules are >>> not that hard, and JOL does it routinely), instantiate a Record with whatever arguments, poke the >>> test patterns with Unsafe.put*, read them back from record components to verify. It would get me the >>> same objectFieldOffset in a round-about way. >> I'm aware of these workarounds in the wild.?? As sun.misc.Unsafe is JDK unsupported API, this patch >> does not attempt to implement a complete solution but adds an obvious notification informing the >> frameworks that `sun.misc.Unsafe::objectFieldOffset` and `sun.misc.Unsafe::staticField{Offset/Base}` >> not to support records. > I understand the intent. My point is that intent is mistimed during this time. I understand this > tripwire needs to be put while Records are still in preview. My point is that it cannot be put in > before Records serialization story has the preview-proven answer. > > The intent also looks rather opaque. The intent is to notify maintainers, fine. I am one of the > maintainers (although not of serialization library, but still heavy-Unsafe one), so hear me out. I > came up with the workaround above in about 15 minutes after someone pointed out the exception to me. > Have it crossed my mind that maybe JDK developers want some help here? No. It looks like another > impediment that should be solved on the spot. Does that exception communicate any intent at all? The > message is generic. There is no comment. How would you expect to push maintainers to think along the > "we should be collaborating to find a proper way to do this", instead of "this is set in stone; let > me hack this around too"? Records is in second preview (JEP 384).? As clearly stated in JEP 12 and [1],? introducing a feature as a preview feature enables developers to provide feedbacks before it becoems permanent in a future Java SE Platform. We should encourage developers to engage and provide feedbacks and have the discussion during preview. For JOL, have you raised an enhancement request to the serviceability team? Inspecting on an object layout would be useful and I suggest you file a RFE if you have done so. [1] https://docs.oracle.com/en/java/javase/14/language/preview-language-and-vm-features.html > This is the actionable bit: > > At very least the exception message should say something about the intent. And maybe even the > comment in Unsafe.java should point to the discussion about this intent and maybe even provide the > breadcrumbs to follow, e.g. to ObjectInputStream.readRecord(). To hit this exception, the app must be running with --enable-preview.?? There are documentation about preview feature and warning messages etc.?? To avoid duplicating the info, the best I could do may be to add "preview feature" in the exception message? Chris Hegarty will definitely want to help out w.r.t. the serialization frameworks in supporting records. Mandy From coleen.phillimore at oracle.com Thu Jun 18 19:39:01 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 18 Jun 2020 15:39:01 -0400 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: References: Message-ID: <3f87b59e-4c54-7f5f-c06e-d909078c478f@oracle.com> http://cr.openjdk.java.net/~stefank/8247755/webrev.01/src/hotspot/share/utilities/growableArray.cpp.html 92 #endif nit, can you add // ASSERT here http://cr.openjdk.java.net/~stefank/8247755/webrev.01/src/hotspot/share/utilities/growableArray.hpp.html 302 template 303 class GrowableArrayWithAllocator : public GrowableArrayView { So all the functions in this class need to be able to call grow(), with allocator 'S' right?? Can template parameter 'S' be "ALLOC" so it's easy to find? The early tiers run the SA tests so I think if it's not broken your SA changes are sufficient. Second webrev. http://cr.openjdk.java.net/~stefank/8247759/webrev.01/src/hotspot/share/gc/z/zArray.hpp.udiff.html I looked at the diffs and they seemed fine (don't know the code). This is a nice result of this rewrite. Coleen On 6/17/20 10:16 AM, Stefan Karlsson wrote: > Hi all, > > Please review and/or comment on this patch to create leaner and more > versatile GrowableArray classes. > > https://cr.openjdk.java.net/~stefank/8247755/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8247755 > > This is a proposal to rewrite the GrowableArray classes, with two main > goals: > > 1) Allow a GrowableArray instance to be allocated on the stack or > embedded in other objects. This removes one of the two indirections > when operating on the array elements. > > 2) Make the GrowableArray instances smaller, so that there's no extra > overhead when they are embedded in other objects. > > This will enable us to use GrowableArray effectively in ZGC, without > having to rely on our own implementations. See follow-up patch: > https://cr.openjdk.java.net/~stefank/8247759/webrev.01 > > Some background about the GrowableArray: > > GrowableArray is split into two memory parts. One for the instance of > the GrowableArray, and a second, separate memory part for the > elements. This memory can come from these places: > > a) Resource area in the thread > b) CHeap (malloc) > c) Arena (malloced arena) > d) Stack > e) Embedded in other object > > From the C++ compiler's point-of-view, (a)-(e) can be used to allocate > the GrowableArray, and (a)-(c) can be used to allocate the elements. > However, there are runtime asserts that restricts the combinations. > > One of the restrictions is that if either of the GrowableArray > instance memory, or the elements memory, comes from the CHeap, then > the other must also come from the CHeap. > > I've found two reasons why we want to be restrictive with our CHeap > allocated elements: > > - if the GrowableArray instance is allocated in (a) or (c), then the > destructor will never run and we get a memory leak because the > elements are not deallocated. > > However, for stack allocated instances the destructor will be run, and > the memory will be handed back. For embedded instances it all depends > on how the parent instance was allocated. > > - There's no proper support for copying/moving of the elements, just a > simple pointer copy. So, if a copy would be done, the source > GrowableArray would still have a pointer to the elements, and if the > source were destructed (maybe because it was stack allocated and went > out-of-scope) then the memory of the destination GrowableArray will be > deallocated. This isn't a problem with instances in (a) or (c), since > their destructors won't be run. > > Proposal for Goal 1: > > - Allow stack allocated and embedded GrowableArrays to have CHeap > allocated elements. > - Add an assert in a copy constructor and assignment operator that > checks that the GrowableArray elements are not CHeap allocated. > > The rest of the patch mainly deals with Goal 2: > > The current fields of GrowableArrays, in product builds, are: > ???int _len; > ???int _max; > ???Arena* _arena; > ???MEMFLAGS _memflags; > ???E* _data; > > and the resulting is 32 bytes: 4 + 4 + 8 + 8 + 8. The _memflags size > is due to alignment requirements for the _data field. > > - By combining _arena and _memflags, this is reduced to 24 bytes. > - By having a specialized CHeap version with the MEMFLAGS encoded as a > template parameter, we can get rid of both _memflags and _arena > fields, and reduce this to 16 bytes. > > Things to note about the patch: > > - GrowableArrayMetadata: I've fused _arena and _memflags into one > uintptr_t field, and encapsulated all bookkeeping about where and how > the memory is allocated for the GrowableArray. The hope is to simplify > the code a bit, and also to move some of the checking out into a > non-template class, so that the code can be put in the .cpp file. > > - GrowableArrayNestingCheck: Simplify and encapsulate resource area > nesting checks. > > - I'm using CRTP because I don't want a vtable in the class. This > means that the super class can cast to derived type, responsible for > allocation and deallocation: > > template class GrowableArray : public > GrowableArrayWithAllocator > > > and then: > > void GrowableArrayWithAllocator::grow(int j) { > ... > ? E* newData = static_cast(this)->allocate(); > > - Since I'm chainging so much of the structure of the code, I've also > made some stylistic changes to make the code more consistent. > > - Moved the CompareClosure to iterator.hpp, where many of our other > closures are located. > > - Moved _sort_Fn, since it was already used by non-GrowableArray code > > - I've added a GrowableArrayCHeap class to be able to mostly side-step > the complication of the GrowableArray elements allocation, and to > minimize the instance size. > > - The metaspaceShared.cpp was used to show how GrowableArrayCHeap can > be used. It's not strictly necessary, and could be reverted. > > -? The arguments.cpp change shows a usage of GrowableArrayView as a > way to only expose an interface that don't allow the array to be grown. > > - The arguments.cpp change also shows stack allocated > GrowableArrayCHeap, that automatically gets deallocated when the > function returns, and the cleanups we can do because of this. > > - The hashtable.* changes show an embedded GrowableArrayCHeap. > > - The SA agent. I didn't bother renaming the GenericGrowableArray > file. I can do that upon request. > > - I added gtest for GrowableArray. There was not a single unit test of > GrowableArray before. The creates all possible memory allocation > combinations and run limited set of sanity checks. It also checks that > the different memory allocation combinations return the correct > allocation types when queried. I've added negative tests to show that > GrowableArrays with CHeap allocated elements assert when being copied. > > Testing: will run at least tier1-3, but most likely more tiers on > Linux x64. Suggestions on what to test welcome. > > Thanks, > StefanK From coleen.phillimore at oracle.com Thu Jun 18 19:43:17 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 18 Jun 2020 15:43:17 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <3f3ee0a9-013b-76af-52ff-905d63098825@oracle.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <3f3ee0a9-013b-76af-52ff-905d63098825@oracle.com> Message-ID: Thanks Ioi. On 6/18/20 2:02 AM, Ioi Lam wrote: > Hi Coleen, > > This looks good to me. > > G1GCPhaseTimes::phase_name and enum GCParPhases should really be > defined using an X-macro, but that's a different RFE. Yes, someone who works on G1 more can think about that. thanks, Coleen > > Thanks > - Ioi > > On 6/17/20 6:09 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 6/17/20 7:49 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>> >>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>> changes. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8247808 >>> >>> This is a nice cleanup and simplification of the code for working >>> with OopStorage! So LGTM. >> >> Thanks, David. >>> >>> One query ... I'm assuming that the processing previously done in >>> JvmtiExport::oops_do is now done by >>> OopStorageSet::vm_global()->oops_do. In most cases I can see the >>> call to OopStorageSet::vm_global()->oops_do in the same vicinity as >>> the call to JvmtiExport::oops_do, but not all i.e. >>> ZRootsIterator::oops_do and ShenandoahSerialRoots::oops_do. Tracking >>> through it seems that for those GCs the VM global roots are >>> processed concurrently, whereas currently JVMTI roots are not. Does >>> that make any potential difference? >> >> ZGC and Shenandoah are better because when the vm_global() roots >> grow, they'll be processed faster.? Because accessing oops in >> OopStorage uses resolve() which uses the Access API, any oops will be >> marked or fixed when accessed if the GC hasn't yet gotten to this oop >> in it's concurrent processing. >> >> Kim noticed that G1 and ParallelGC should be processing these roots >> in parallel (with many threads, since OopStorage has that support) >> and he's going to or has filed a bug to fix it.? As we add more >> things to OopStorage (see upcoming RFRs), this will become important. >> >> Thanks, >> Coleen >> >>> >>> Thanks, >>> David >>> ----- >>> >>>> Thanks, >>>> Coleen >> > From coleen.phillimore at oracle.com Thu Jun 18 19:48:11 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 18 Jun 2020 15:48:11 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> Message-ID: <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> On 6/18/20 3:58 AM, Thomas Schatzl wrote: > Hi, > > On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >> >> >> On 6/17/20 7:49 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>> >>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>> changes. >>>> > [...] >> >> Kim noticed that G1 and ParallelGC should be processing these roots >> in parallel (with many threads, since OopStorage has that support) >> and he's going to or has filed a bug to fix it.? As we add more >> things to OopStorage (see upcoming RFRs), this will become important. >> > > I do not know which exact roots you want to move into OopStorage, but > I would like to mention this concern: with moving everything into a > single OopStorage (i.e. vm_globals in this case), I am worried that > every time important information about the source for these gets lost. > > Which makes it hard to understand from where these oops came from when > there is a performance problem in the "VM Globals" bucket. Hi Thomas, I understand this concern.? On the GC list there is a discussion about having the ability to create different strong OopStorages, changing the OopStorage code to process these roots and report statistics in parallel (and/or concurrent), and not having to cascade the code through all the GCs. I'm going to hold this change until this discussion is complete and move the JVMTI and services/management oops_do oops into a different OopStorage that can make use of this.? Then you'll have your statistics and we won't have classes needing traversal with oops_do. Thanks, Coleen > > This may not apply to JVMTI oops, but others may occasionally have a > significant amount of oops where it would be very interesting to know > from where a particular slowdown comes from. > > So I would prefer keep some accounting here. > > Thanks, > ? Thomas From coleen.phillimore at oracle.com Thu Jun 18 19:54:59 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 18 Jun 2020 15:54:59 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <94da70e9-2c61-a97d-e31a-6e0ec4478faa@oracle.com> References: <94da70e9-2c61-a97d-e31a-6e0ec4478faa@oracle.com> Message-ID: <07a7aef1-61f2-aaee-aede-e76fa456e014@oracle.com> On 6/18/20 3:25 AM, Stefan Karlsson wrote: > Hi Coleen, > > On 2020-06-17 23:25, coleen.phillimore at oracle.com wrote: >> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >> >> Tested with tier1-3, also built shenandoah to verify shenandoah changes. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev > > https://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev/src/hotspot/share/prims/jvmtiImpl.cpp.udiff.html > > > ?JvmtiBreakpoint::~JvmtiBreakpoint() { > - if (_class_holder != NULL) { > - NativeAccess<>::oop_store(_class_holder, (oop)NULL); > - OopStorageSet::vm_global()->release(_class_holder); > + if (_class_holder.resolve() != NULL) { > + _class_holder.release(); > ?? } > ?} > > Could this be changed to peek() / release() instead? The resolve() > call is going to keep the object alive until next for ZGC marking cycle. Yes, makes sense. Fixed. > > The rest looks OK. > > Below are some comments about things that I find odd and non-obvious > from reading the code, and may be potentials for cleanups to make it > easier for the next to understand the code: > > The above code assumes that as soon as OopHandle::create has been > called, we won't store NULL into the _obj pointer. If someone does, > then we would leak the memory. OopHandle has a function ptr_raw, that > allows someone to clear the _obj pointer. I have to assume that this > function isn't used in this code. > > --- > > ?214 void JvmtiBreakpoint::copy(JvmtiBreakpoint& bp) { > ?215?? _method?? = bp._method; > ?216?? _bci????? = bp._bci; > 217 _class_holder = OopHandle::create(bp._class_holder.resolve()); > ?218 } > > This one looks odd, because the _class_holder is overwritten without > releasing the old OopHandle. This is currently OK, because copy is > only called from clone, which just created a new JvmtiBreakpoint: > > ? GrowableElement *clone()??????? { > ??? JvmtiBreakpoint *bp = new JvmtiBreakpoint(); > ??? bp->copy(*this); > ??? return bp; > ? } > > ?I think this would have been much more obvious if copy/clone were a > copy constructor. Yes, this would make more sense.? I don't know why this was implemented as clone. > > With that said, it looks like we now have two JvmtiBreakpoints with > the same OopHandle contents. So, OopHandle::release will be called > twice. Now that works because release clears the oop value: > > inline void OopHandle::release() { > ? // Clear the OopHandle first > ? NativeAccess<>::oop_store(_obj, (oop)NULL); > ? OopStorageSet::vm_global()->release(_obj); > } > > and the resolve() != NULL check will prevent the OopHandle from being > released twice: > > + if (_class_holder.resolve() != NULL) { > + _class_holder.release(); > ?? } The release is called on the original JvmtiBreakpoint which has one OopHandle, and it's also called on the copy which has another, so release isn't called twice on the same OopHandle. That said, I had to walk through the code this morning and make sure that release is called on the copy of the JvmtiBreakpoint (it's called in remove() after the breakpoint is cleared.? The entire _bps array is not deleted). Thanks, Coleen > > StefanK > >> bug link https://bugs.openjdk.java.net/browse/JDK-8247808 >> >> Thanks, >> Coleen > From mandy.chung at oracle.com Thu Jun 18 21:12:15 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Jun 2020 14:12:15 -0700 Subject: Review Request JDK-8244090: public lookup should find public members of public exported types Message-ID: <090cff25-2f48-d55f-721d-38acda80029b@oracle.com> Prior to JDK-8173978, publicLookup().in(C.class) produces a Lookup object on C with PUBLIC access which can be used to look up unconditionally exported public types from the module of C. Such lookup can only look up this C class defined by loader 1 but not another class named "C" defined by loader 2. JDK-8173978 adds the support for cross-module teleporting. publicLookup().in(C.class) was changed to produce a public Lookup i.e. with UNCONDITIONAL access.? A public lookup should be able to look up public members of any unconditionally exported public types including a class named "C" loaded by different loaders.? It reveals a bug in VM resolution for Lookup API that adds the loader constraints with java.lang.Object as the accessor that constraints a public lookup to load one type named C but any more. The lookup class of a public lookup is irrelevant to the lookup context.? Type consistency on P/Q/R live Class objects from the given MethodType (MT)is ensured at the following: 1. P/Q/R are consistent from the loader of the declaring class of the resolved member (D's loader) 2. P/Q/R are consistent w.r.t. the invoking class and the caller's MT at invocation time (the method handle carries the caller's MT that will be verified). The loader constraints added for public lookup is not necessary and adds undesirable constraints.? The proposed fixis to skip adding loader constraints if the lookup mode is either TRUSTED or UNCONDITIONAL. Webrev: http://cr.openjdk.java.net/~mchung/jdk16/webrevs/8244090/webrev.00/index.html Thanks Mandy From stefan.karlsson at oracle.com Thu Jun 18 21:20:18 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 18 Jun 2020 23:20:18 +0200 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <3f87b59e-4c54-7f5f-c06e-d909078c478f@oracle.com> References: <3f87b59e-4c54-7f5f-c06e-d909078c478f@oracle.com> Message-ID: <7e673a2d-fde3-5451-779d-5384d1b103b1@oracle.com> Hi Coleen, On 2020-06-18 21:39, coleen.phillimore at oracle.com wrote: > > http://cr.openjdk.java.net/~stefank/8247755/webrev.01/src/hotspot/share/utilities/growableArray.cpp.html > > > ? 92 #endif > > nit, can you add // ASSERT here Sure. > > > > http://cr.openjdk.java.net/~stefank/8247755/webrev.01/src/hotspot/share/utilities/growableArray.hpp.html > > > 302 template > ?303 class GrowableArrayWithAllocator : public GrowableArrayView { > > > So all the functions in this class need to be able to call grow(), > with allocator 'S' right? Yes. > ? Can template parameter 'S' be "ALLOC" so it's easy to find? I chose the name S to match that the class that S represents *must* be a subclass. I can rename it to ALLOC, but then the name would loose that connection. I'd like to here more opinions before changing this. > > The early tiers run the SA tests so I think if it's not broken your SA > changes are sufficient. OK. I'll make sure that they pass. > > > Second webrev. > > http://cr.openjdk.java.net/~stefank/8247759/webrev.01/src/hotspot/share/gc/z/zArray.hpp.udiff.html > > > I looked at the diffs and they seemed fine (don't know the code). This > is a nice result of this rewrite. Yes. At least I think so. This was the code that made me start investigating this: https://cr.openjdk.java.net/~stefank/8247759/webrev.01/src/hotspot/share/gc/z/zPhysicalMemory.cpp.udiff.html Thanks for looking at the code, StefanK > > Coleen > > On 6/17/20 10:16 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review and/or comment on this patch to create leaner and more >> versatile GrowableArray classes. >> >> https://cr.openjdk.java.net/~stefank/8247755/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8247755 >> >> This is a proposal to rewrite the GrowableArray classes, with two >> main goals: >> >> 1) Allow a GrowableArray instance to be allocated on the stack or >> embedded in other objects. This removes one of the two indirections >> when operating on the array elements. >> >> 2) Make the GrowableArray instances smaller, so that there's no extra >> overhead when they are embedded in other objects. >> >> This will enable us to use GrowableArray effectively in ZGC, without >> having to rely on our own implementations. See follow-up patch: >> https://cr.openjdk.java.net/~stefank/8247759/webrev.01 >> >> Some background about the GrowableArray: >> >> GrowableArray is split into two memory parts. One for the instance of >> the GrowableArray, and a second, separate memory part for the >> elements. This memory can come from these places: >> >> a) Resource area in the thread >> b) CHeap (malloc) >> c) Arena (malloced arena) >> d) Stack >> e) Embedded in other object >> >> From the C++ compiler's point-of-view, (a)-(e) can be used to >> allocate the GrowableArray, and (a)-(c) can be used to allocate the >> elements. However, there are runtime asserts that restricts the >> combinations. >> >> One of the restrictions is that if either of the GrowableArray >> instance memory, or the elements memory, comes from the CHeap, then >> the other must also come from the CHeap. >> >> I've found two reasons why we want to be restrictive with our CHeap >> allocated elements: >> >> - if the GrowableArray instance is allocated in (a) or (c), then the >> destructor will never run and we get a memory leak because the >> elements are not deallocated. >> >> However, for stack allocated instances the destructor will be run, >> and the memory will be handed back. For embedded instances it all >> depends on how the parent instance was allocated. >> >> - There's no proper support for copying/moving of the elements, just >> a simple pointer copy. So, if a copy would be done, the source >> GrowableArray would still have a pointer to the elements, and if the >> source were destructed (maybe because it was stack allocated and went >> out-of-scope) then the memory of the destination GrowableArray will >> be deallocated. This isn't a problem with instances in (a) or (c), >> since their destructors won't be run. >> >> Proposal for Goal 1: >> >> - Allow stack allocated and embedded GrowableArrays to have CHeap >> allocated elements. >> - Add an assert in a copy constructor and assignment operator that >> checks that the GrowableArray elements are not CHeap allocated. >> >> The rest of the patch mainly deals with Goal 2: >> >> The current fields of GrowableArrays, in product builds, are: >> ???int _len; >> ???int _max; >> ???Arena* _arena; >> ???MEMFLAGS _memflags; >> ???E* _data; >> >> and the resulting is 32 bytes: 4 + 4 + 8 + 8 + 8. The _memflags size >> is due to alignment requirements for the _data field. >> >> - By combining _arena and _memflags, this is reduced to 24 bytes. >> - By having a specialized CHeap version with the MEMFLAGS encoded as >> a template parameter, we can get rid of both _memflags and _arena >> fields, and reduce this to 16 bytes. >> >> Things to note about the patch: >> >> - GrowableArrayMetadata: I've fused _arena and _memflags into one >> uintptr_t field, and encapsulated all bookkeeping about where and how >> the memory is allocated for the GrowableArray. The hope is to >> simplify the code a bit, and also to move some of the checking out >> into a non-template class, so that the code can be put in the .cpp file. >> >> - GrowableArrayNestingCheck: Simplify and encapsulate resource area >> nesting checks. >> >> - I'm using CRTP because I don't want a vtable in the class. This >> means that the super class can cast to derived type, responsible for >> allocation and deallocation: >> >> template class GrowableArray : public >> GrowableArrayWithAllocator > >> >> and then: >> >> void GrowableArrayWithAllocator::grow(int j) { >> ... >> ? E* newData = static_cast(this)->allocate(); >> >> - Since I'm chainging so much of the structure of the code, I've also >> made some stylistic changes to make the code more consistent. >> >> - Moved the CompareClosure to iterator.hpp, where many of our other >> closures are located. >> >> - Moved _sort_Fn, since it was already used by non-GrowableArray code >> >> - I've added a GrowableArrayCHeap class to be able to mostly >> side-step the complication of the GrowableArray elements allocation, >> and to minimize the instance size. >> >> - The metaspaceShared.cpp was used to show how GrowableArrayCHeap can >> be used. It's not strictly necessary, and could be reverted. >> >> -? The arguments.cpp change shows a usage of GrowableArrayView as a >> way to only expose an interface that don't allow the array to be grown. >> >> - The arguments.cpp change also shows stack allocated >> GrowableArrayCHeap, that automatically gets deallocated when the >> function returns, and the cleanups we can do because of this. >> >> - The hashtable.* changes show an embedded GrowableArrayCHeap. >> >> - The SA agent. I didn't bother renaming the GenericGrowableArray >> file. I can do that upon request. >> >> - I added gtest for GrowableArray. There was not a single unit test >> of GrowableArray before. The creates all possible memory allocation >> combinations and run limited set of sanity checks. It also checks >> that the different memory allocation combinations return the correct >> allocation types when queried. I've added negative tests to show that >> GrowableArrays with CHeap allocated elements assert when being copied. >> >> Testing: will run at least tier1-3, but most likely more tiers on >> Linux x64. Suggestions on what to test welcome. >> >> Thanks, >> StefanK > From stefan.karlsson at oracle.com Thu Jun 18 22:09:28 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 19 Jun 2020 00:09:28 +0200 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages Message-ID: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> Hi all, Please review this patch to rework WeakHandle and OopHandle to dynamically support different OopStorages. https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8247879 Today WeakHandle has a template parameter and specialization of get_storage() to get hold of the backing OopStorage. Coleen, Erik? and I talked about different ways to rewrite that, so that we wouldn't have to write specializations for all weak oop storages, and so that we don't have to carry around the type information about the weak oop storage. Instead of writing: WeakHandle w = WeakHandle::create(obj) we could instead write: WeakHandle w(OopStorageSet::vm_weak(), obj) OopHandle is today hard-coded to use OopStorageSet::vm_global(). The proposal is to change OopHandle to look like WeakHandle, and allow more than one OopStorage. Something that might will become important going forward, when we are going to move more things into separate OopStorages. So, instead of writing: OopHandle h = OopHandle::create(obj) one will have to specify the OopStorage explicitly: OopHandle h(OopStorageSet::vm_global(), obj) This change has a slight drawback that the OopStorage used for the allocation of the handle, needs to be provided when the handle is released. To make this safer, so that you don't use strong OopStorages when allocating WeakHandles, or weak OopStorages when allocating OopHandles, I've created two subclasses to OopStorage: StrongOopStorage and WeakOopStorage. They are just two tiny wrappers around OopStorage, to allow us ensure correct usage during compile time. The oopStorageSet.hpp files seems to be intentionally written to *not* know about the OopStorage implementation. To do proper casting of the OopStorage*s I need to include the definition of StrongOopStorage and WeakOopStorage. I've therefore moved all OopStorageSet::() getters out to an inline header. This adds a lot of new includes. If we don't think that it's important to inline these, then I can move it over to the cpp file instead. Thanks, StefanK From vladimir.kozlov at oracle.com Thu Jun 18 22:35:01 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 18 Jun 2020 15:35:01 -0700 Subject: [16] RFR(S): 8247218: Add default constructor to VectorSet to use Thread::current()->resource_area() as arena by default In-Reply-To: <55011fe1-2307-ad66-ea39-60a3b9c00682@oracle.com> References: <55011fe1-2307-ad66-ea39-60a3b9c00682@oracle.com> Message-ID: Looks good. Note for others, VectorSet(Arena*) version is left because compiler uses it with an other arena (comp_arena). resource_area is used per compilation phase or even smaller scopes. comp_arena is used per one compilation scope. Thanks, Vladimir On 6/18/20 3:36 AM, Christian Hagedorn wrote: > Hi > > Please review the following cleanup: > https://bugs.openjdk.java.net/browse/JDK-8247218 > http://cr.openjdk.java.net/~chagedorn/8247218/webrev.00/ > > By adding an additional default constructor in VectorSet (as also present in Node_List, Node_Stack etc.), we can replace > the often found pattern > > ? Arena* a = Thread::current()->resource_area(); > ? VectorSet visited(a); > > by a simple > > ? VectorSet visited; > > I additionally cleaned up some other places where this pattern was found, for example for Node_List etc. > > Thank you! > > Best regards, > Christian From kim.barrett at oracle.com Fri Jun 19 03:36:44 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 18 Jun 2020 23:36:44 -0400 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <7e673a2d-fde3-5451-779d-5384d1b103b1@oracle.com> References: <3f87b59e-4c54-7f5f-c06e-d909078c478f@oracle.com> <7e673a2d-fde3-5451-779d-5384d1b103b1@oracle.com> Message-ID: <00E78825-2E69-469E-B4DB-B920AB97E5C0@oracle.com> > On Jun 18, 2020, at 5:20 PM, Stefan Karlsson wrote: > On 2020-06-18 21:39, coleen.phillimore at oracle.com wrote: >> So all the functions in this class need to be able to call grow(), with allocator 'S' right? > > Yes. > >> Can template parameter 'S' be "ALLOC" so it's easy to find? > > I chose the name S to match that the class that S represents *must* be a subclass. I can rename it to ALLOC, but then the name would loose that connection. I'd like to here more opinions before changing this. ?S? doesn?t say a whole lot. A convention I?ve seen is to name such a CRTP template parameter ?Derived?. Though I?m not sure CRTP is the way to go here; I?m still working on reviewing though, so possibly more on that later. From christian.hagedorn at oracle.com Fri Jun 19 06:49:34 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Fri, 19 Jun 2020 08:49:34 +0200 Subject: [16] RFR(S): 8247218: Add default constructor to VectorSet to use Thread::current()->resource_area() as arena by default In-Reply-To: References: <55011fe1-2307-ad66-ea39-60a3b9c00682@oracle.com> Message-ID: Thank you Vladimir for your review and the clarification! Best regards, Christian On 19.06.20 00:35, Vladimir Kozlov wrote: > Looks good. > > Note for others, VectorSet(Arena*) version is left because compiler uses > it with an other arena (comp_arena). > > resource_area is used per compilation phase or even smaller scopes. > comp_arena is used per one compilation scope. > > Thanks, > Vladimir > > On 6/18/20 3:36 AM, Christian Hagedorn wrote: >> Hi >> >> Please review the following cleanup: >> https://bugs.openjdk.java.net/browse/JDK-8247218 >> http://cr.openjdk.java.net/~chagedorn/8247218/webrev.00/ >> >> By adding an additional default constructor in VectorSet (as also >> present in Node_List, Node_Stack etc.), we can replace the often found >> pattern >> >> ?? Arena* a = Thread::current()->resource_area(); >> ?? VectorSet visited(a); >> >> by a simple >> >> ?? VectorSet visited; >> >> I additionally cleaned up some other places where this pattern was >> found, for example for Node_List etc. >> >> Thank you! >> >> Best regards, >> Christian From coleen.phillimore at oracle.com Fri Jun 19 15:56:14 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 19 Jun 2020 11:56:14 -0400 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> Message-ID: <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/src/hotspot/share/classfile/classLoaderData.hpp.udiff.html + WeakHandle _holder; // The oop that determines lifetime of this class loader + OopHandle _class_loader; // The instance of java/lang/ClassLoader associated with + // this ClassLoaderData Can you line the member names up? On 6/18/20 6:09 PM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to rework WeakHandle and OopHandle to > dynamically support different OopStorages. > > https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8247879 > > Today WeakHandle has a template parameter and specialization of > get_storage() to get hold of the backing OopStorage. Coleen, Erik? and > I talked about different ways to rewrite that, so that we wouldn't > have to write specializations for all weak oop storages, and so that > we don't have to carry around the type information about the weak oop > storage. > > Instead of writing: > WeakHandle w = WeakHandle::create(obj) > > we could instead write: > WeakHandle w(OopStorageSet::vm_weak(), obj) > > OopHandle is today hard-coded to use OopStorageSet::vm_global(). The > proposal is to change OopHandle to look like WeakHandle, and allow > more than one OopStorage. Something that might will become important > going forward, when we are going to move more things into separate > OopStorages. > > So, instead of writing: > OopHandle h = OopHandle::create(obj) > > one will have to specify the OopStorage explicitly: > OopHandle h(OopStorageSet::vm_global(), obj) > > This change has a slight drawback that the OopStorage used for the > allocation of the handle, needs to be provided when the handle is > released. > > To make this safer, so that you don't use strong OopStorages when > allocating WeakHandles, or weak OopStorages when allocating > OopHandles, I've created two subclasses to OopStorage: > StrongOopStorage and WeakOopStorage. They are just two tiny wrappers > around OopStorage, to allow us ensure correct usage during compile > time. The oopStorageSet.hpp files seems to be intentionally written to > *not* know about the OopStorage implementation. To do proper casting > of the OopStorage*s I need to include the definition of > StrongOopStorage and WeakOopStorage. I've therefore moved all > OopStorageSet::() getters out to an inline > header. This adds a lot of new includes. If we don't think that it's > important to inline these, then I can move it over to the cpp file > instead. I don't like these includes.? In the places where we allocate and release OopHandle/WeakHandles, where we need to refer to these storages, performance isn't important to avoid a call.? I don't know about all of the GC places. I also don't see a large benefit to type safety that these new classes provide, to be honest, since there aren't many call sites where we create and release these handles.? We're more likely to mix different strong oop storages (allocate from one and release from another), once we have more than one strong OopStorage. OopStorage::release has a check that the memory is in that storage (I hope), so there already should be a runtime check. I'll defer to whatever Kim thinks though.? This doesn't seem worth having to me though. Otherwise, the changes look good. thanks, Coleen > > Thanks, > StefanK > From peter.levart at gmail.com Sun Jun 21 22:32:15 2020 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Jun 2020 00:32:15 +0200 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <029fdb77-660b-4f59-7ee7-bc7020b5e49f@redhat.com> References: <4B268BF1-1DFD-4DCC-AEA3-7F2C7B81DC91@oracle.com> <029fdb77-660b-4f59-7ee7-bc7020b5e49f@redhat.com> Message-ID: <2cc3ff95-4352-1a52-e395-ef90d2338891@gmail.com> Hi Aleksey, On 6/18/20 7:10 PM, Aleksey Shipilev wrote: > On 6/18/20 3:09 PM, Chris Hegarty wrote: >>> On 18 Jun 2020, at 13:43, Aleksey Shipilev wrote: >>> Again, JDK-8247532 is the writing on the wall: we don't need 3rd party developers to tell if Record >>> serialization works fast in 15 -- we already know it does not. >> I disagree. JDK-8247532 is under review and well on its way towards JDK 15 (yes, during RDP 1). >> My reading of Peter?s benchmark result is that Record deserialization *does work fast*. What am >> I missing. > JDK-8247532 is the evidence that Records serialization performance story is not clear. > > Even if we disregard that after JDK-8247532 [1] Records are still 8% slower, the _existence_ of > JDK-8247532 indicates there are performance problems in the area. That evidence now needs to be > compensated by much more evidence to the contrary. (Yes, I contracted a lot of Bayesian thinking > from my statistician wife...) > > (Here were several paragraphs of further thoughts, but I realized it basically repeats what I said > before.) Let me just express my thoughts on the subject of preventing writes to final fields in records in connection to deserialization. In short, I don't think any serialization framework would need to have direct write access to record fields in order to be performant. Serialization frameworks need write access to final fields in classical classes mainly because fields are generally encapsulated. If classes don't implement special API (constructors, methods, annotations ...) to interface with serialization framework (like for example Jackson annotations, etc.) then serialization framework can only interface such classes on the level of fields. Now records are very constrained classes. They have an always present API that can be used to interface with serialization frameworks. For each field, they poses a constructor parameter with the same name and type and are arranged so that the fields are set from the constructor parameters. So instead of allocating a "zero" instance and setting fields, one can just invoke a canonical constructor which does the same. Such invocation is equally performant as allocating instance and setting fields from outside the constructor. What I did in JDK-8247532 was that I just optimized the code that transforms the bytes read from stream into values for arguments of the canonical constructor using method handle transformations. That part was slow and not the fact that records are deserialized using their canonical constructor. Every serialization framework that works on the level of fields has to do the transformation of stream encoded data to field values - how it does that is what differs it from the rest - the final mechanism how those values are stored into object fields is not what makes it unique or performant. There is also a desire that record constructor(s) are the only way to create instances of records such that any validation logic in the constructor(s) can't get bypassed. Not even via deserialization of "forged" streams. That way we can have records that are safer without scarifying performance. And BTW, according to latest JDK-8247532 webrev.06, Java deserialization of records is not slower but sometimes even faster than deserialization of equivalent classical classes. Regards, Peter From kim.barrett at oracle.com Mon Jun 22 07:25:55 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 22 Jun 2020 03:25:55 -0400 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: References: Message-ID: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> > On Jun 17, 2020, at 10:16 AM, Stefan Karlsson wrote: > > Hi all, > > Please review and/or comment on this patch to create leaner and more versatile GrowableArray classes. > > https://cr.openjdk.java.net/~stefank/8247755/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8247755 > > This is a proposal to rewrite the GrowableArray classes, with two main goals: > > 1) Allow a GrowableArray instance to be allocated on the stack or embedded in other objects. This removes one of the two indirections when operating on the array elements. > > 2) Make the GrowableArray instances smaller, so that there's no extra overhead when they are embedded in other objects. Those are good goals. In earlier email I wondered whether CRTP was the best approach for this. Never mind, I'm okay with this approach. (I wondered whether the different non-CRTP approach taken by BitMap would serve here, but I think it doesn't. BitMap doesn't have operations that implicitly perform allocations; all allocations are externally requested.) ------------------------------------------------------------------------------ src/hotspot/share/utilities/globalDefinitions.hpp 435 // Need the correct linkage to call qsort without warnings 436 extern "C" { 437 typedef int (*_sort_Fn)(const void *, const void *); 438 } Rather than messing with function pointer types, why not use Quicksort::sort from utilities/quickSort.hpp? ------------------------------------------------------------------------------ src/hotspot/share/memory/metaspaceShared.cpp 459 static GrowableArrayCHeap* _extra_interned_strings = NULL; ... 462 _extra_interned_strings = new GrowableArrayCHeap(10000); [pre-existing] I think this vector is only needed during the preload_and_dump operation that calls read_extra_data, and can then be discarded. It gets allocated by read_extra_data and put in a global variable and left there as a leak, though maybe we don't care. I was going to suggest it might be an on-stack Resource array over some relevant scope in the caller and passed to read_extra_data , but that won't work because of the ResourceMark in the read_extra_data loop. But with the proposed changes to GrowableArray it could be an on-stack CHeap array. ------------------------------------------------------------------------------ src/hotspot/share/utilities/growableArray.hpp 115 class GrowableArrayView : public GrowableArrayBase { I don't like the name "View" for this concept. Common practice elsewhere (including the Standard) is for "view" types to be an adaptor over some other type, providing a limited range or otherwise adjusted access. (Our BitMapView does that.) GrowableArrayData? I'm not sure how often this class name gets used by clients, so it might not matter that it's a little bit long or clumsy / clunky. ------------------------------------------------------------------------------ src/hotspot/share/utilities/growableArray.hpp Destructors for all but the most-derived classes should be non-public. Probably also the constructors. That is, all of GrowableArrayBase, GrowableArrayView, and GrowableArrayWithAllocator should have protected constructors and destructors. ------------------------------------------------------------------------------ src/hotspot/share/utilities/growableArray.hpp [pre-existing] GrowableArray's use of mtNone to indicate resource allocation seems like an abuse of that memory type. That memory type has a description string of "Unknown", nothing to do with resources. But it's considered an invalid type by MamTracker/MemReporter/MallocSite, and it seems to mean something else in the VirtualMemoryTracker, like "uninitialized". aarch64 pf (debugger print frame) passes mtNone to NEW_C_HEAP_OBJ, which seems like it shouldn't work, given the assert in MallocSite. But why have constructors with optional MEMFLAGS at all? And why use mtNone that way? Constructors without MEMFLAGS parameter are resource allocating, constructors with required MEMFLAGS parameter are CHeap allocating. (Though the random status of mtNone noted above will make that one probably not actually usable.) (That would require some small adjustments to GrowableArrayMetadata.) And if mtNone is going to be used that way, then GrowableArrayCHeap should static assert that it's flag value is not mtNone. ------------------------------------------------------------------------------ src/hotspot/share/utilities/growableArray.hpp 586 public: 587 // Where are we going to allocate memory? 588 bool on_C_heap() { return _metadata.on_C_heap(); } 589 bool on_stack () { return _metadata.on_stack(); } 590 bool on_arena () { return _metadata.on_arena(); } These were previously protected; why make them public? ------------------------------------------------------------------------------ From kim.barrett at oracle.com Mon Jun 22 07:52:17 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 22 Jun 2020 03:52:17 -0400 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <7DE4D1D2-ECA8-4980-B665-781BB0E36B2F@oracle.com> References: <4611e386-2d3f-efaa-fdce-920c7d3d5b85@oracle.com> <7DE4D1D2-ECA8-4980-B665-781BB0E36B2F@oracle.com> Message-ID: <480C1195-A280-4859-B108-2F12C7950FB1@oracle.com> > On Jun 14, 2020, at 4:51 AM, Kim Barrett wrote: > >> On Jun 10, 2020, at 8:46 AM, Leo Korinth wrote: >> >> [?] >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8247213 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8247213/0/part1 >> http://cr.openjdk.java.net/~lkorinth/8247213/0/part2 >> http://cr.openjdk.java.net/~lkorinth/8247213/0/full > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 1011 template > 1012 static AtomicValue* cast_to_atomic_ptr(T* nonatomic) { > > I would really like to not have this. I'd like to see how far we can > get without it. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 940 class AtomicValue { > > Should it be default constructible? If so, should the default > constructor guarantee value initialization? I think having a trivial > default constructor is needed to avoid UB in some uses. See discussion > of G1BOT's _offset_array below. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1BlockOffsetTable.cpp > 45 _offset_array = cast_to_atomic_ptr((u_char*) bot_reserved.start()); > > This is currently the only use of cast_to_atomic_ptr. It leads to UB > since the (non-trivial) constructor for AtomicValue is never called > for any of the objects in the array. I think the call to > memset_with_concurrent_readers is also UB. I *think* having a trivial > default constructor solves the UB problem, but that isn't possible > until C++11. > > Rather than just casting to a pointer to AtomicValue<>, this could > iterate over the array memory, doing placement new construction of the > elements. If we gave AtomicValue<> a non-trivial but empty default > constructor (so not initializing the value), then such an iteration > should be optimized away. We'd still have the UB from memset_with_cr, > though that could use an iteration with the one-arg constructor instead. > Or ignore the technical UB and just keep using memset_with_cr. > > And that gets rid of the one use (so far) of cast_to_atomic_ptr. > > The problem, of course, is that having a trivial/empty default > constructor means one could unintentionally construct without > initialization. I think a private trivial default constructor would > deal with that, though again that requires C++11. Responding to myself about the above, here?s a way to get value initialization by the default constructor but have another constructor that does no initialization, and should be compiled to nothing. // Put this somewhere convenient (not! in AtomicValue class). struct NoInitFlag {}; const NoInitFlag NO_INIT; Add these constructors: AtomicValue() : _val() {} // Value initialization. AtomicValue(NoInitFlag) {} // Intentionally implicit. > src/hotspot/share/gc/g1/heapRegionManager.cpp > 663 for (uint i = 0; i < _n_regions; ++i) { > 664 new (&_claims[i]) AtomicValue(Unclaimed); > 665 } > > Even if AtomicValue has a trivial default destructor (as discussed > above) to avoid UB when using memset, this change is an improvement. > The old memset code only works because Unclaimed == 0. Also related, s/destructor/constructor/ From stefan.karlsson at oracle.com Mon Jun 22 09:01:15 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 22 Jun 2020 11:01:15 +0200 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> Message-ID: <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> On 2020-06-22 09:25, Kim Barrett wrote: >> On Jun 17, 2020, at 10:16 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review and/or comment on this patch to create leaner and more versatile GrowableArray classes. >> >> https://cr.openjdk.java.net/~stefank/8247755/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8247755 >> >> This is a proposal to rewrite the GrowableArray classes, with two main goals: >> >> 1) Allow a GrowableArray instance to be allocated on the stack or embedded in other objects. This removes one of the two indirections when operating on the array elements. >> >> 2) Make the GrowableArray instances smaller, so that there's no extra overhead when they are embedded in other objects. > > Those are good goals. > > In earlier email I wondered whether CRTP was the best approach for > this. Never mind, I'm okay with this approach. (I wondered whether the > different non-CRTP approach taken by BitMap would serve here, but I > think it doesn't. BitMap doesn't have operations that implicitly > perform allocations; all allocations are externally requested.) > > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/globalDefinitions.hpp > 435 // Need the correct linkage to call qsort without warnings > 436 extern "C" { > 437 typedef int (*_sort_Fn)(const void *, const void *); > 438 } > > Rather than messing with function pointer types, why not use > Quicksort::sort from utilities/quickSort.hpp? > I didn't change this code, only moved it. This was done to reduce the clutter in growableArray.hpp. You don't suggest that I change this in this RFR, right? > ------------------------------------------------------------------------------ > src/hotspot/share/memory/metaspaceShared.cpp > 459 static GrowableArrayCHeap* _extra_interned_strings = NULL; > ... > 462 _extra_interned_strings = new GrowableArrayCHeap(10000); > > [pre-existing] > > I think this vector is only needed during the preload_and_dump > operation that calls read_extra_data, and can then be discarded. It > gets allocated by read_extra_data and put in a global variable and > left there as a leak, though maybe we don't care. > > I was going to suggest it might be an on-stack Resource array over > some relevant scope in the caller and passed to read_extra_data , but > that won't work because of the ResourceMark in the read_extra_data > loop. But with the proposed changes to GrowableArray it could be an > on-stack CHeap array. > Yes. I mainly wanted to use this as an example of a CHeap/CHeap-allocated GrowableArray. I also think it would make sense to change this to n on-stack CHeap array. > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/growableArray.hpp > 115 class GrowableArrayView : public GrowableArrayBase { > > I don't like the name "View" for this concept. Common practice > elsewhere (including the Standard) is for "view" types to be an > adaptor over some other type, providing a limited range or otherwise > adjusted access. (Our BitMapView does that.) > > GrowableArrayData? > > I'm not sure how often this class name gets used by clients, so it > might not matter that it's a little bit long or clumsy / clunky > My thinking was the just like the with the BitMapView, the backing storage is managed externally: // The BitMapView is used when the backing storage is managed externally. class BitMapView : public BitMap { public: BitMapView() : BitMap(NULL, 0) {} BitMapView(bm_word_t* map, idx_t size_in_bits) : BitMap(map, size_in_bits) {} }; I don't think GrowableArrayData is a better name, so if I need to change this, I'm going to need some decent suggestions. I talked briefly with Kim, but we couldn't immediately find a name that both liked. Suggestions? > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/growableArray.hpp > > Destructors for all but the most-derived classes should be non-public. > Probably also the constructors. That is, all of GrowableArrayBase, > GrowableArrayView, and GrowableArrayWithAllocator should have > protected constructors and destructors. > OK. For GrowableArrayivew, I had an idea that someone *might* want to do: int _array[10]; ... GrowableArrayView view(array, 10, 10); do_something(view); but I'll change this to protected. > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/growableArray.hpp > > [pre-existing] > > GrowableArray's use of mtNone to indicate resource allocation seems > like an abuse of that memory type. That memory type has a description > string of "Unknown", nothing to do with resources. But it's considered > an invalid type by MamTracker/MemReporter/MallocSite, and it seems to > mean something else in the VirtualMemoryTracker, like "uninitialized". > I thinks we view this differently. My view is that memflags belong to the CHeap allocations, but is only valid if != mtNone. Setting it to mtNone doesn't imply that resource allocations are marked with mtNone. It just means that we shouldn't care about the memflags value when allocating resource memory. Just like this function shows: 570 static E* allocate(int max, MEMFLAGS memflags) { 571 if (memflags != mtNone) { 572 return (E*)GrowableArrayCHeapAllocator::allocate(max, sizeof(E), memflags); 573 } 574 575 return (E*)GrowableArrayResourceAllocator::allocate(max, sizeof(E)); 576 } and this: 495 uintptr_t bits(MEMFLAGS memflags) const { 496 if (memflags == mtNone) { 497 // Stack allocation 498 return 0; 499 } 500 501 // CHeap allocation 502 return (uintptr_t(memflags) << 1) | 1; 503 } (Probably should have turned that the other way around to have the same code layout) W.r.t. VirtualMemoryTracker, I think that's odd and probably should be cleaned up. > aarch64 pf (debugger print frame) passes mtNone to NEW_C_HEAP_OBJ, > which seems like it shouldn't work, given the assert in MallocSite. > I'll see if I can provoke this and will create a JBS entry if I do. > But why have constructors with optional MEMFLAGS at all? And why use > mtNone that way? Constructors without MEMFLAGS parameter are resource > allocating, constructors with required MEMFLAGS parameter are CHeap > allocating. (Though the random status of mtNone noted above will make > that one probably not actually usable.) (That would require some small > adjustments to GrowableArrayMetadata.) I think you've lost me here. Are you proposing that I split this up into more constructors. Are you suggesting that I change this: GrowableArray(int initial_max = 2, MEMFLAGS memflags = mtNone) : GrowableArrayWithAllocator >( allocate(initial_max, memflags), initial_max), _metadata(memflags) { init_checks(); } GrowableArray(int initial_max, int initial_len, const E& filler, MEMFLAGS memflags = mtNone) : GrowableArrayWithAllocator >( allocate(initial_max, memflags), initial_max, initial_len, filler), _metadata(memflags) { init_checks(); } to (say): GrowableArray(int initial_max = 2) : GrowableArrayWithAllocator >( allocate(initial_max), initial_max), _metadata() { init_checks(); } GrowableArray(int initial_max, MEMFLAGS memflags) : GrowableArrayWithAllocator >( allocate(initial_max, memflags), initial_max), _metadata(memflags) { init_checks(); } GrowableArray(int initial_max, int initial_len, const E& filler) : GrowableArrayWithAllocator >( allocate(initial_max), initial_max, initial_len, filler), _metadata() { init_checks(); } GrowableArray(int initial_max, int initial_len, const E& filler, MEMFLAGS memflags) : GrowableArrayWithAllocator >( allocate(initial_max, memflags), initial_max, initial_len, filler), _metadata(memflags) { init_checks(); } and adjust allocate(...) and GrowableArrayMetadata(...) accordingly? > > And if mtNone is going to be used that way, then GrowableArrayCHeap > should static assert that it's flag value is not mtNone. > Sure. > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/growableArray.hpp > 586 public: > 587 // Where are we going to allocate memory? > 588 bool on_C_heap() { return _metadata.on_C_heap(); } > 589 bool on_stack () { return _metadata.on_stack(); } > 590 bool on_arena () { return _metadata.on_arena(); } > > These were previously protected; why make them public? > Mainly to be usable by the tests. I can add a friend test class and revert this. > ------------------------------------------------------------------------------ > Thanks for reviewing this, StefanK From vkempik at azul.com Mon Jun 22 09:43:42 2020 From: vkempik at azul.com (Vladimir Kempik) Date: Mon, 22 Jun 2020 09:43:42 +0000 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: <79dadca8-c34b-6923-9c6c-58df3c17220d@oracle.com> References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> <5fabaaee-dc10-0fe5-823a-72295d932559@oracle.com> <79dadca8-c34b-6923-9c6c-58df3c17220d@oracle.com> Message-ID: <2B369869-E5E9-4929-8F3D-F02A829872A3@azul.com> Hello The result of the test varies from run to run, it can give +-5%. So you need average of multiple runs to have some useful result Thanks. > 8 ???? 2020 ?., ? 21:18, gerard ziemski ???????(?): > > hi Vladimir, > > My testing show no problem with the "-O2" flag, however, using just "-Os" (i.e. dropping "-O1" from JvmOverrideFiles.gmk") shows about the same performance as adding "-O2", and in fact it is slightly fastest for the TestRandom use case: > > -Os -O1 (the current default for unsafe.cpp) takes 196s > > -Os (the default for most hotspot files) takes 158s > > -O2 takes 161s > > -O3 takes 161s > > Can you reproduce my findings? > > > cheers > > On 6/3/20 2:59 PM, gerard ziemski wrote: >> hi Vladimir, I will take a look to see if I can help review this... >> >> >> On 6/3/20 12:38 PM, Vladimir Kempik wrote: >>> Hello >>> >>> Can somebody please review this simple change ? >>> >>> Thanks >>> >>>> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >>>> >>>> Adding hotspot-runtime-dev >>>> >>>>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik ???????(?): >>>>> >>>>> >>>>> Hello >>>>> Please review a fix for JDK-8243470 >>>>> >>>>> Long time ago as part of JEP284: New HotSpot Build System this fix was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>>>> At that time it was decided to lower optimisation level for unsafe.cpp from O2 to O1 only for clang on Macosx. >>>>> >>>>> I suppose it was done due to issues in Set/Get helper functions where too optimistic optimisations were eliminating some null pointer checks. it was probably a clang bug. >>>>> That issue could be checked with test jdk/test/sun/misc/CopyMemory.java. >>>>> >>>>> I believe that workaround (going from O2 to O1) produced this issue - https://bugs.openjdk.java.net/browse/JDK-8234963 (Thread.getStackTrace is slow with clang). >>>>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>>>> >>>>> Here I propose the patch which eliminates that workaround for clang 8+. >>>>> >>>>> I have tested clang versions 8/9/9.1/10, all of them showed good results: >>>>> 1) CopyMemory test passes fine on 11/14/15. >>>>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. jdk15: no new failures in tck. >>>>> 3) The testRandom "benchmark" from 8234963 shows great improvements on my machine, going down from ~200 seconds to ~150 seconds (the newer clang the better result). For comparision, gcc built libjvm for jdk11 shows ~130 seconds on my machine. >>>>> >>>>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>>>> >>>>> getStackTrace benchmark: http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>>>> >>>>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>>>> >>>>> Thanks, Vladimir >> From vkempik at azul.com Mon Jun 22 10:39:32 2020 From: vkempik at azul.com (Vladimir Kempik) Date: Mon, 22 Jun 2020 10:39:32 +0000 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> Message-ID: <2BF5F539-4DB1-477D-8AEC-D56AA78EA723@azul.com> Hello Thanks for looking into this Magnus. Here is updated version of webrev - http://cr.openjdk.java.net/~vkempik/8243470/webrev.01/ effectively it does exactly the same as previous version, I have removed the version check for clang. Thanks, Vladimir > 4 ???? 2020 ?., ? 16:04, Magnus Ihse Bursie ???????(?): > > On 2020-06-03 20:07, Magnus Ihse Bursie wrote: >> Thanks for cc:ing us. >> >> This is not the correct way to check for compiler versions. Nor is it the correct place. >> >> I don't have time for a full reply tonight, but will return with a better reply tomorrow. > Ok, I have now looked more into this. > > You are apparently want to check for Xcode version, not clang version. This is not the same. Note that clang can be used on Linux as well. Apple is basically dropping the normal clang versioning and replacing it with their own, messed-up version which somewhat resembles the corresponding Xcode version, but is not identical. Horray. :-/ (For those interested, have a look at https://gist.github.com/yamaya/2924292.) > > Anyway. We do not support Xcode prior to 8. You seem to have tested from 8 and up, so this patch should be applied unconditionally. That is, just remove the -O1 special thing. > > /Magnus >> >> /Magnus >> >> On 2020-06-03 19:47, Daniel D. Daugherty wrote: >>> Adding build-dev at ... since this is a build system change. >>> >>> As for the right HotSpot team, I'm not sure who should be making >>> the call on changing the way unsafe.cpp is built for macOSX. My guess >>> when I moved the bug to hotspot/runtime was that someone on the Runtime >>> team would know, but... >>> >>> Dan >>> >>> >>> >>> On 6/3/20 1:38 PM, Vladimir Kempik wrote: >>>> Hello >>>> >>>> Can somebody please review this simple change ? >>>> >>>> Thanks >>>> >>>>> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >>>>> >>>>> Adding hotspot-runtime-dev >>>>> >>>>>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik ???????(?): >>>>>> >>>>>> >>>>>> Hello >>>>>> Please review a fix for JDK-8243470 >>>>>> >>>>>> Long time ago as part of JEP284: New HotSpot Build System this fix was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>>>>> At that time it was decided to lower optimisation level for unsafe.cpp from O2 to O1 only for clang on Macosx. >>>>>> >>>>>> I suppose it was done due to issues in Set/Get helper functions where too optimistic optimisations were eliminating some null pointer checks. it was probably a clang bug. >>>>>> That issue could be checked with test jdk/test/sun/misc/CopyMemory.java. >>>>>> >>>>>> I believe that workaround (going from O2 to O1) produced this issue - https://bugs.openjdk.java.net/browse/JDK-8234963 (Thread.getStackTrace is slow with clang). >>>>>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>>>>> >>>>>> Here I propose the patch which eliminates that workaround for clang 8+. >>>>>> >>>>>> I have tested clang versions 8/9/9.1/10, all of them showed good results: >>>>>> 1) CopyMemory test passes fine on 11/14/15. >>>>>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. jdk15: no new failures in tck. >>>>>> 3) The testRandom "benchmark" from 8234963 shows great improvements on my machine, going down from ~200 seconds to ~150 seconds (the newer clang the better result). For comparision, gcc built libjvm for jdk11 shows ~130 seconds on my machine. >>>>>> >>>>>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>>>>> >>>>>> getStackTrace benchmark: http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>>>>> >>>>>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>>>>> >>>>>> Thanks, Vladimir >>> >> From stefan.karlsson at oracle.com Mon Jun 22 11:16:08 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 22 Jun 2020 13:16:08 +0200 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> Message-ID: <10b46a39-83fb-69c6-7a9d-7abdc849e720@oracle.com> On 2020-06-19 17:56, coleen.phillimore at oracle.com wrote: > > https://cr.openjdk.java.net/~stefank/8247879/webrev.01/src/hotspot/share/classfile/classLoaderData.hpp.udiff.html > > > + WeakHandle _holder; // The oop that determines lifetime of this class > loader > + OopHandle _class_loader; // The instance of java/lang/ClassLoader > associated with > + // this ClassLoaderData > > > Can you line the member names up? > If you want. They were not aligned, and none of the other members are alined, so the request is a bit unexpected. StefanK > > On 6/18/20 6:09 PM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to rework WeakHandle and OopHandle to >> dynamically support different OopStorages. >> >> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247879 >> >> Today WeakHandle has a template parameter and specialization of >> get_storage() to get hold of the backing OopStorage. Coleen, Erik? and >> I talked about different ways to rewrite that, so that we wouldn't >> have to write specializations for all weak oop storages, and so that >> we don't have to carry around the type information about the weak oop >> storage. >> >> Instead of writing: >> WeakHandle w = WeakHandle::create(obj) >> >> we could instead write: >> WeakHandle w(OopStorageSet::vm_weak(), obj) >> >> OopHandle is today hard-coded to use OopStorageSet::vm_global(). The >> proposal is to change OopHandle to look like WeakHandle, and allow >> more than one OopStorage. Something that might will become important >> going forward, when we are going to move more things into separate >> OopStorages. >> >> So, instead of writing: >> OopHandle h = OopHandle::create(obj) >> >> one will have to specify the OopStorage explicitly: >> OopHandle h(OopStorageSet::vm_global(), obj) >> >> This change has a slight drawback that the OopStorage used for the >> allocation of the handle, needs to be provided when the handle is >> released. >> >> To make this safer, so that you don't use strong OopStorages when >> allocating WeakHandles, or weak OopStorages when allocating >> OopHandles, I've created two subclasses to OopStorage: >> StrongOopStorage and WeakOopStorage. They are just two tiny wrappers >> around OopStorage, to allow us ensure correct usage during compile >> time. The oopStorageSet.hpp files seems to be intentionally written to >> *not* know about the OopStorage implementation. To do proper casting >> of the OopStorage*s I need to include the definition of >> StrongOopStorage and WeakOopStorage. I've therefore moved all >> OopStorageSet::() getters out to an inline >> header. This adds a lot of new includes. If we don't think that it's >> important to inline these, then I can move it over to the cpp file >> instead. > > I don't like these includes.? In the places where we allocate and > release OopHandle/WeakHandles, where we need to refer to these storages, > performance isn't important to avoid a call.? I don't know about all of > the GC places. > > I also don't see a large benefit to type safety that these new classes > provide, to be honest, since there aren't many call sites where we > create and release these handles.? We're more likely to mix different > strong oop storages (allocate from one and release from another), once > we have more than one strong OopStorage. OopStorage::release has a check > that the memory is in that storage (I hope), so there already should be > a runtime check. > > I'll defer to whatever Kim thinks though.? This doesn't seem worth > having to me though. > > Otherwise, the changes look good. > > thanks, > Coleen >> >> Thanks, >> StefanK >> > From stefan.karlsson at oracle.com Mon Jun 22 12:12:59 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 22 Jun 2020 14:12:59 +0200 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> Message-ID: <64ba7392-c728-9865-a597-3c97d7b3ee0c@oracle.com> On 2020-06-19 17:56, coleen.phillimore at oracle.com wrote: > > https://cr.openjdk.java.net/~stefank/8247879/webrev.01/src/hotspot/share/classfile/classLoaderData.hpp.udiff.html > > > + WeakHandle _holder; // The oop that determines lifetime of this class > loader > + OopHandle _class_loader; // The instance of java/lang/ClassLoader > associated with > + // this ClassLoaderData > > > Can you line the member names up? > > > On 6/18/20 6:09 PM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to rework WeakHandle and OopHandle to >> dynamically support different OopStorages. >> >> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8247879 >> >> Today WeakHandle has a template parameter and specialization of >> get_storage() to get hold of the backing OopStorage. Coleen, Erik? and >> I talked about different ways to rewrite that, so that we wouldn't >> have to write specializations for all weak oop storages, and so that >> we don't have to carry around the type information about the weak oop >> storage. >> >> Instead of writing: >> WeakHandle w = WeakHandle::create(obj) >> >> we could instead write: >> WeakHandle w(OopStorageSet::vm_weak(), obj) >> >> OopHandle is today hard-coded to use OopStorageSet::vm_global(). The >> proposal is to change OopHandle to look like WeakHandle, and allow >> more than one OopStorage. Something that might will become important >> going forward, when we are going to move more things into separate >> OopStorages. >> >> So, instead of writing: >> OopHandle h = OopHandle::create(obj) >> >> one will have to specify the OopStorage explicitly: >> OopHandle h(OopStorageSet::vm_global(), obj) >> >> This change has a slight drawback that the OopStorage used for the >> allocation of the handle, needs to be provided when the handle is >> released. >> >> To make this safer, so that you don't use strong OopStorages when >> allocating WeakHandles, or weak OopStorages when allocating >> OopHandles, I've created two subclasses to OopStorage: >> StrongOopStorage and WeakOopStorage. They are just two tiny wrappers >> around OopStorage, to allow us ensure correct usage during compile >> time. The oopStorageSet.hpp files seems to be intentionally written to >> *not* know about the OopStorage implementation. To do proper casting >> of the OopStorage*s I need to include the definition of >> StrongOopStorage and WeakOopStorage. I've therefore moved all >> OopStorageSet::() getters out to an inline >> header. This adds a lot of new includes. If we don't think that it's >> important to inline these, then I can move it over to the cpp file >> instead. > > I don't like these includes.? In the places where we allocate and > release OopHandle/WeakHandles, where we need to refer to these storages, > performance isn't important to avoid a call.? I don't know about all of > the GC places. > > I also don't see a large benefit to type safety that these new classes > provide, to be honest, since there aren't many call sites where we > create and release these handles.? We're more likely to mix different > strong oop storages (allocate from one and release from another), once > we have more than one strong OopStorage. OopStorage::release has a check > that the memory is in that storage (I hope), so there already should be > a runtime check. > > I'll defer to whatever Kim thinks though.? This doesn't seem worth > having to me though. > > Otherwise, the changes look good. I reverted all the Strong/Weak designations. Is this OK?: https://cr.openjdk.java.net/~stefank/8247879/webrev.02 StefanK > > thanks, > Coleen >> >> Thanks, >> StefanK >> > From coleen.phillimore at oracle.com Mon Jun 22 13:13:29 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 22 Jun 2020 09:13:29 -0400 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> Message-ID: <4d2b4b16-b36f-8b1c-4492-8571d8d1cdce@oracle.com> I have a couple of comments to the comments. snip. ------------------------------------------------------------------------------ >> src/hotspot/share/memory/metaspaceShared.cpp >> ? 459 static GrowableArrayCHeap* >> _extra_interned_strings = NULL; >> ... >> ? 462?? _extra_interned_strings = new GrowableArrayCHeap> mtClassShared>(10000); >> >> [pre-existing] >> >> I think this vector is only needed during the preload_and_dump >> operation that calls read_extra_data, and can then be discarded. It >> gets allocated by read_extra_data and put in a global variable and >> left there as a leak, though maybe we don't care. >> >> I was going to suggest it might be an on-stack Resource array over >> some relevant scope in the caller and passed to read_extra_data , but >> that won't work because of the ResourceMark in the read_extra_data >> loop.? But with the proposed changes to GrowableArray it could be an >> on-stack CHeap array. >> > > Yes. I mainly wanted to use this as an example of a > CHeap/CHeap-allocated GrowableArray. I also think it would make sense > to change this to n on-stack CHeap array. Can you file another RFE to fix this? > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/utilities/growableArray.hpp >> ? 115 class GrowableArrayView : public GrowableArrayBase { >> >> I don't like the name "View" for this concept. Common practice >> elsewhere (including the Standard) is for "view" types to be an >> adaptor over some other type, providing a limited range or otherwise >> adjusted access.? (Our BitMapView does that.) >> >> GrowableArrayData? >> >> I'm not sure how often this class name gets used by clients, so it >> might not matter that it's a little bit long or clumsy / clunky > > > My thinking was the just like the with the BitMapView, the backing > storage is managed externally: > > // The BitMapView is used when the backing storage is managed externally. > class BitMapView : public BitMap { > ?public: > ? BitMapView() : BitMap(NULL, 0) {} > ? BitMapView(bm_word_t* map, idx_t size_in_bits) : BitMap(map, > size_in_bits) {} > }; > > I don't think GrowableArrayData is a better name, so if I need to > change this, I'm going to need some decent suggestions. I talked > briefly with Kim, but we couldn't immediately find a name that both > liked. Suggestions? The name "View" looks weird to me but since this class shouldn't be used in outside code, this doesn't matter to me. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/utilities/growableArray.hpp >> >> Destructors for all but the most-derived classes should be non-public. >> Probably also the constructors.? That is, all of GrowableArrayBase, >> GrowableArrayView, and GrowableArrayWithAllocator should have >> protected constructors and destructors. >> > > OK. For GrowableArrayivew, I had an idea that someone *might* want to do: > > int _array[10]; > ... > GrowableArrayView view(array, 10, 10); > do_something(view); > > but I'll change this to protected. > > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/utilities/growableArray.hpp >> >> [pre-existing] >> >> GrowableArray's use of mtNone to indicate resource allocation seems >> like an abuse of that memory type. That memory type has a description >> string of "Unknown", nothing to do with resources. But it's considered >> an invalid type by MamTracker/MemReporter/MallocSite, and it seems to >> mean something else in the VirtualMemoryTracker, like "uninitialized". >> > > I thinks we view this differently. My view is that memflags belong to > the CHeap allocations, but is only valid if != mtNone. Setting it to > mtNone doesn't imply that resource allocations are marked with mtNone. > It just means that we shouldn't care about the memflags value when > allocating resource memory. > > Just like this function shows: > ?570?? static E* allocate(int max, MEMFLAGS memflags) { > ?571???? if (memflags != mtNone) { > ?572?????? return (E*)GrowableArrayCHeapAllocator::allocate(max, > sizeof(E), memflags); > ?573???? } > ?574 > ?575???? return (E*)GrowableArrayResourceAllocator::allocate(max, > sizeof(E)); > ?576?? } > > and this: > ?495?? uintptr_t bits(MEMFLAGS memflags) const { > ?496???? if (memflags == mtNone) { > ?497?????? // Stack allocation > ?498?????? return 0; > ?499???? } > ?500 > ?501???? // CHeap allocation > ?502???? return (uintptr_t(memflags) << 1) | 1; > ?503?? } > > (Probably should have turned that the other way around to have the > same code layout) > > W.r.t. VirtualMemoryTracker, I think that's odd and probably should be > cleaned up. > >> aarch64 pf (debugger print frame) passes mtNone to NEW_C_HEAP_OBJ, >> which seems like it shouldn't work, given the assert in MallocSite. >> > > I'll see if I can provoke this and will create a JBS entry if I do. > >> But why have constructors with optional MEMFLAGS at all?? And why use >> mtNone that way?? Constructors without MEMFLAGS parameter are resource >> allocating, constructors with required MEMFLAGS parameter are CHeap >> allocating.? (Though the random status of mtNone noted above will make >> that one probably not actually usable.)? (That would require some small >> adjustments to GrowableArrayMetadata.) > > I think you've lost me here. Are you proposing that I split this up > into more constructors. Are you suggesting that I change this: > > ?? GrowableArray(int initial_max = 2, MEMFLAGS memflags = mtNone) : > ?????? GrowableArrayWithAllocator >( > ?????????? allocate(initial_max, memflags), > ?????????? initial_max), > ?????? _metadata(memflags) { > ???? init_checks(); > ?? } > > ?? GrowableArray(int initial_max, int initial_len, const E& filler, > MEMFLAGS memflags = mtNone) : > ?????? GrowableArrayWithAllocator >( > ?????????? allocate(initial_max, memflags), > ?????????? initial_max, initial_len, filler), > ?????? _metadata(memflags) { > ???? init_checks(); > ?? } > > to (say): > ?? GrowableArray(int initial_max = 2) : > ?????? GrowableArrayWithAllocator >( > ?????????? allocate(initial_max), > ?????????? initial_max), > ?????? _metadata() { > ???? init_checks(); > ?? } > > ?? GrowableArray(int initial_max, MEMFLAGS memflags) : > ?????? GrowableArrayWithAllocator >( > ?????????? allocate(initial_max, memflags), > ?????????? initial_max), > ?????? _metadata(memflags) { > ???? init_checks(); > ?? } > > ?? GrowableArray(int initial_max, int initial_len, const E& filler) : > ?????? GrowableArrayWithAllocator >( > ?????????? allocate(initial_max), > ?????????? initial_max, initial_len, filler), > ?????? _metadata() { > ???? init_checks(); > ?? } > > ?? GrowableArray(int initial_max, int initial_len, const E& filler, > MEMFLAGS memflags) : > ?????? GrowableArrayWithAllocator >( > ?????????? allocate(initial_max, memflags), > ?????????? initial_max, initial_len, filler), > ?????? _metadata(memflags) { > ???? init_checks(); > ?? } > > and adjust allocate(...) and GrowableArrayMetadata(...) accordingly? No, please don't do this.? Passing mtNone as a default parameter for resource allocated GrowableArrays and !mtNone if not, seemed like a nice compact change, with obvious meaning. Thanks, Coleen > >> >> And if mtNone is going to be used that way, then GrowableArrayCHeap >> should static assert that it's flag value is not mtNone. >> > > Sure. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/utilities/growableArray.hpp >> ? 586 public: >> ? 587?? // Where are we going to allocate memory? >> ? 588?? bool on_C_heap() { return _metadata.on_C_heap(); } >> ? 589?? bool on_stack () { return _metadata.on_stack(); } >> ? 590?? bool on_arena () { return _metadata.on_arena(); } >> >> These were previously protected; why make them public? >> > > Mainly to be usable by the tests. I can add a friend test class and > revert this. > >> ------------------------------------------------------------------------------ >> >> > > Thanks for reviewing this, > StefanK From coleen.phillimore at oracle.com Mon Jun 22 13:15:25 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 22 Jun 2020 09:15:25 -0400 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <10b46a39-83fb-69c6-7a9d-7abdc849e720@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> <10b46a39-83fb-69c6-7a9d-7abdc849e720@oracle.com> Message-ID: <4ede5af2-ddb9-bce5-790f-f09ac64d2ca4@oracle.com> On 6/22/20 7:16 AM, Stefan Karlsson wrote: > On 2020-06-19 17:56, coleen.phillimore at oracle.com wrote: >> >> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/src/hotspot/share/classfile/classLoaderData.hpp.udiff.html >> >> >> + WeakHandle _holder; // The oop that determines lifetime of this >> class loader >> + OopHandle _class_loader; // The instance of java/lang/ClassLoader >> associated with >> + // this ClassLoaderData >> >> >> Can you line the member names up? >> > > If you want. They were not aligned, and none of the other members are > alined, so the request is a bit unexpected. It's just that these two fields are next to each other and off by one, which was visually distracting to me.? Since there are comments separating groups of fields, aligning all of them seems unnecessary for visibility.? Just these two. thanks, Coleen > > StefanK > >> >> On 6/18/20 6:09 PM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to rework WeakHandle and OopHandle to >>> dynamically support different OopStorages. >>> >>> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8247879 >>> >>> Today WeakHandle has a template parameter and specialization of >>> get_storage() to get hold of the backing OopStorage. Coleen, Erik? >>> and I talked about different ways to rewrite that, so that we >>> wouldn't have to write specializations for all weak oop storages, >>> and so that we don't have to carry around the type information about >>> the weak oop storage. >>> >>> Instead of writing: >>> WeakHandle w = WeakHandle::create(obj) >>> >>> we could instead write: >>> WeakHandle w(OopStorageSet::vm_weak(), obj) >>> >>> OopHandle is today hard-coded to use OopStorageSet::vm_global(). The >>> proposal is to change OopHandle to look like WeakHandle, and allow >>> more than one OopStorage. Something that might will become important >>> going forward, when we are going to move more things into separate >>> OopStorages. >>> >>> So, instead of writing: >>> OopHandle h = OopHandle::create(obj) >>> >>> one will have to specify the OopStorage explicitly: >>> OopHandle h(OopStorageSet::vm_global(), obj) >>> >>> This change has a slight drawback that the OopStorage used for the >>> allocation of the handle, needs to be provided when the handle is >>> released. >>> >>> To make this safer, so that you don't use strong OopStorages when >>> allocating WeakHandles, or weak OopStorages when allocating >>> OopHandles, I've created two subclasses to OopStorage: >>> StrongOopStorage and WeakOopStorage. They are just two tiny wrappers >>> around OopStorage, to allow us ensure correct usage during compile >>> time. The oopStorageSet.hpp files seems to be intentionally written >>> to *not* know about the OopStorage implementation. To do proper >>> casting of the OopStorage*s I need to include the definition of >>> StrongOopStorage and WeakOopStorage. I've therefore moved all >>> OopStorageSet::() getters out to an inline >>> header. This adds a lot of new includes. If we don't think that it's >>> important to inline these, then I can move it over to the cpp file >>> instead. >> >> I don't like these includes.? In the places where we allocate and >> release OopHandle/WeakHandles, where we need to refer to these >> storages, performance isn't important to avoid a call.? I don't know >> about all of the GC places. >> >> I also don't see a large benefit to type safety that these new >> classes provide, to be honest, since there aren't many call sites >> where we create and release these handles.? We're more likely to mix >> different strong oop storages (allocate from one and release from >> another), once we have more than one strong OopStorage. >> OopStorage::release has a check that the memory is in that storage (I >> hope), so there already should be a runtime check. >> >> I'll defer to whatever Kim thinks though.? This doesn't seem worth >> having to me though. >> >> Otherwise, the changes look good. >> >> thanks, >> Coleen >>> >>> Thanks, >>> StefanK >>> >> From coleen.phillimore at oracle.com Mon Jun 22 13:18:44 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 22 Jun 2020 09:18:44 -0400 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <64ba7392-c728-9865-a597-3c97d7b3ee0c@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> <64ba7392-c728-9865-a597-3c97d7b3ee0c@oracle.com> Message-ID: On 6/22/20 8:12 AM, Stefan Karlsson wrote: > > On 2020-06-19 17:56, coleen.phillimore at oracle.com wrote: >> >> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/src/hotspot/share/classfile/classLoaderData.hpp.udiff.html >> >> >> + WeakHandle _holder; // The oop that determines lifetime of this >> class loader >> + OopHandle _class_loader; // The instance of java/lang/ClassLoader >> associated with >> + // this ClassLoaderData >> >> >> Can you line the member names up? >> >> >> On 6/18/20 6:09 PM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to rework WeakHandle and OopHandle to >>> dynamically support different OopStorages. >>> >>> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8247879 >>> >>> Today WeakHandle has a template parameter and specialization of >>> get_storage() to get hold of the backing OopStorage. Coleen, Erik? >>> and I talked about different ways to rewrite that, so that we >>> wouldn't have to write specializations for all weak oop storages, >>> and so that we don't have to carry around the type information about >>> the weak oop storage. >>> >>> Instead of writing: >>> WeakHandle w = WeakHandle::create(obj) >>> >>> we could instead write: >>> WeakHandle w(OopStorageSet::vm_weak(), obj) >>> >>> OopHandle is today hard-coded to use OopStorageSet::vm_global(). The >>> proposal is to change OopHandle to look like WeakHandle, and allow >>> more than one OopStorage. Something that might will become important >>> going forward, when we are going to move more things into separate >>> OopStorages. >>> >>> So, instead of writing: >>> OopHandle h = OopHandle::create(obj) >>> >>> one will have to specify the OopStorage explicitly: >>> OopHandle h(OopStorageSet::vm_global(), obj) >>> >>> This change has a slight drawback that the OopStorage used for the >>> allocation of the handle, needs to be provided when the handle is >>> released. >>> >>> To make this safer, so that you don't use strong OopStorages when >>> allocating WeakHandles, or weak OopStorages when allocating >>> OopHandles, I've created two subclasses to OopStorage: >>> StrongOopStorage and WeakOopStorage. They are just two tiny wrappers >>> around OopStorage, to allow us ensure correct usage during compile >>> time. The oopStorageSet.hpp files seems to be intentionally written >>> to *not* know about the OopStorage implementation. To do proper >>> casting of the OopStorage*s I need to include the definition of >>> StrongOopStorage and WeakOopStorage. I've therefore moved all >>> OopStorageSet::() getters out to an inline >>> header. This adds a lot of new includes. If we don't think that it's >>> important to inline these, then I can move it over to the cpp file >>> instead. >> >> I don't like these includes.? In the places where we allocate and >> release OopHandle/WeakHandles, where we need to refer to these >> storages, performance isn't important to avoid a call.? I don't know >> about all of the GC places. >> >> I also don't see a large benefit to type safety that these new >> classes provide, to be honest, since there aren't many call sites >> where we create and release these handles.? We're more likely to mix >> different strong oop storages (allocate from one and release from >> another), once we have more than one strong OopStorage. >> OopStorage::release has a check that the memory is in that storage (I >> hope), so there already should be a runtime check. >> >> I'll defer to whatever Kim thinks though.? This doesn't seem worth >> having to me though. >> >> Otherwise, the changes look good. > > I reverted all the Strong/Weak designations. Is this OK?: > https://cr.openjdk.java.net/~stefank/8247879/webrev.02 This looks really good to me, and will be nicely extensible for more global OopStorages. Thanks, Coleen > > StefanK > >> >> thanks, >> Coleen >>> >>> Thanks, >>> StefanK >>> >> From stefan.karlsson at oracle.com Mon Jun 22 13:46:33 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 22 Jun 2020 15:46:33 +0200 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <4d2b4b16-b36f-8b1c-4492-8571d8d1cdce@oracle.com> References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> <4d2b4b16-b36f-8b1c-4492-8571d8d1cdce@oracle.com> Message-ID: <5cc11db5-eafc-e89e-6274-d0c09eff641f@oracle.com> On 2020-06-22 15:13, coleen.phillimore at oracle.com wrote: > > > I have a couple of comments to the comments. > > snip. > ------------------------------------------------------------------------------ > >>> src/hotspot/share/memory/metaspaceShared.cpp >>> ? 459 static GrowableArrayCHeap* >>> _extra_interned_strings = NULL; >>> ... >>> ? 462?? _extra_interned_strings = new GrowableArrayCHeap>> mtClassShared>(10000); >>> >>> [pre-existing] >>> >>> I think this vector is only needed during the preload_and_dump >>> operation that calls read_extra_data, and can then be discarded. It >>> gets allocated by read_extra_data and put in a global variable and >>> left there as a leak, though maybe we don't care. >>> >>> I was going to suggest it might be an on-stack Resource array over >>> some relevant scope in the caller and passed to read_extra_data , but >>> that won't work because of the ResourceMark in the read_extra_data >>> loop.? But with the proposed changes to GrowableArray it could be an >>> on-stack CHeap array. >>> >> >> Yes. I mainly wanted to use this as an example of a >> CHeap/CHeap-allocated GrowableArray. I also think it would make sense >> to change this to n on-stack CHeap array. > > Can you file another RFE to fix this? Done: https://bugs.openjdk.java.net/browse/JDK-8248007 >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/utilities/growableArray.hpp >>> ? 115 class GrowableArrayView : public GrowableArrayBase { >>> >>> I don't like the name "View" for this concept. Common practice >>> elsewhere (including the Standard) is for "view" types to be an >>> adaptor over some other type, providing a limited range or otherwise >>> adjusted access.? (Our BitMapView does that.) >>> >>> GrowableArrayData? >>> >>> I'm not sure how often this class name gets used by clients, so it >>> might not matter that it's a little bit long or clumsy / clunky > >> >> My thinking was the just like the with the BitMapView, the backing >> storage is managed externally: >> >> // The BitMapView is used when the backing storage is managed externally. >> class BitMapView : public BitMap { >> ?public: >> ? BitMapView() : BitMap(NULL, 0) {} >> ? BitMapView(bm_word_t* map, idx_t size_in_bits) : BitMap(map, >> size_in_bits) {} >> }; >> >> I don't think GrowableArrayData is a better name, so if I need to >> change this, I'm going to need some decent suggestions. I talked >> briefly with Kim, but we couldn't immediately find a name that both >> liked. Suggestions? > > The name "View" looks weird to me but since this class shouldn't be used > in outside code, this doesn't matter to me. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/utilities/growableArray.hpp >>> >>> Destructors for all but the most-derived classes should be non-public. >>> Probably also the constructors.? That is, all of GrowableArrayBase, >>> GrowableArrayView, and GrowableArrayWithAllocator should have >>> protected constructors and destructors. >>> >> >> OK. For GrowableArrayivew, I had an idea that someone *might* want to do: >> >> int _array[10]; >> ... >> GrowableArrayView view(array, 10, 10); >> do_something(view); >> >> but I'll change this to protected. >> >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/utilities/growableArray.hpp >>> >>> [pre-existing] >>> >>> GrowableArray's use of mtNone to indicate resource allocation seems >>> like an abuse of that memory type. That memory type has a description >>> string of "Unknown", nothing to do with resources. But it's considered >>> an invalid type by MamTracker/MemReporter/MallocSite, and it seems to >>> mean something else in the VirtualMemoryTracker, like "uninitialized". >>> >> >> I thinks we view this differently. My view is that memflags belong to >> the CHeap allocations, but is only valid if != mtNone. Setting it to >> mtNone doesn't imply that resource allocations are marked with mtNone. >> It just means that we shouldn't care about the memflags value when >> allocating resource memory. >> >> Just like this function shows: >> ?570?? static E* allocate(int max, MEMFLAGS memflags) { >> ?571???? if (memflags != mtNone) { >> ?572?????? return (E*)GrowableArrayCHeapAllocator::allocate(max, >> sizeof(E), memflags); >> ?573???? } >> ?574 >> ?575???? return (E*)GrowableArrayResourceAllocator::allocate(max, >> sizeof(E)); >> ?576?? } >> >> and this: >> ?495?? uintptr_t bits(MEMFLAGS memflags) const { >> ?496???? if (memflags == mtNone) { >> ?497?????? // Stack allocation >> ?498?????? return 0; >> ?499???? } >> ?500 >> ?501???? // CHeap allocation >> ?502???? return (uintptr_t(memflags) << 1) | 1; >> ?503?? } >> >> (Probably should have turned that the other way around to have the >> same code layout) >> >> W.r.t. VirtualMemoryTracker, I think that's odd and probably should be >> cleaned up. >> >>> aarch64 pf (debugger print frame) passes mtNone to NEW_C_HEAP_OBJ, >>> which seems like it shouldn't work, given the assert in MallocSite. >>> >> >> I'll see if I can provoke this and will create a JBS entry if I do. >> >>> But why have constructors with optional MEMFLAGS at all?? And why use >>> mtNone that way?? Constructors without MEMFLAGS parameter are resource >>> allocating, constructors with required MEMFLAGS parameter are CHeap >>> allocating.? (Though the random status of mtNone noted above will make >>> that one probably not actually usable.)? (That would require some small >>> adjustments to GrowableArrayMetadata.) >> >> I think you've lost me here. Are you proposing that I split this up >> into more constructors. Are you suggesting that I change this: >> >> ?? GrowableArray(int initial_max = 2, MEMFLAGS memflags = mtNone) : >> ?????? GrowableArrayWithAllocator >( >> ?????????? allocate(initial_max, memflags), >> ?????????? initial_max), >> ?????? _metadata(memflags) { >> ???? init_checks(); >> ?? } >> >> ?? GrowableArray(int initial_max, int initial_len, const E& filler, >> MEMFLAGS memflags = mtNone) : >> ?????? GrowableArrayWithAllocator >( >> ?????????? allocate(initial_max, memflags), >> ?????????? initial_max, initial_len, filler), >> ?????? _metadata(memflags) { >> ???? init_checks(); >> ?? } >> >> to (say): >> ?? GrowableArray(int initial_max = 2) : >> ?????? GrowableArrayWithAllocator >( >> ?????????? allocate(initial_max), >> ?????????? initial_max), >> ?????? _metadata() { >> ???? init_checks(); >> ?? } >> >> ?? GrowableArray(int initial_max, MEMFLAGS memflags) : >> ?????? GrowableArrayWithAllocator >( >> ?????????? allocate(initial_max, memflags), >> ?????????? initial_max), >> ?????? _metadata(memflags) { >> ???? init_checks(); >> ?? } >> >> ?? GrowableArray(int initial_max, int initial_len, const E& filler) : >> ?????? GrowableArrayWithAllocator >( >> ?????????? allocate(initial_max), >> ?????????? initial_max, initial_len, filler), >> ?????? _metadata() { >> ???? init_checks(); >> ?? } >> >> ?? GrowableArray(int initial_max, int initial_len, const E& filler, >> MEMFLAGS memflags) : >> ?????? GrowableArrayWithAllocator >( >> ?????????? allocate(initial_max, memflags), >> ?????????? initial_max, initial_len, filler), >> ?????? _metadata(memflags) { >> ???? init_checks(); >> ?? } >> >> and adjust allocate(...) and GrowableArrayMetadata(...) accordingly? > > No, please don't do this.? Passing mtNone as a default parameter for > resource allocated GrowableArrays and !mtNone if not, seemed like a nice > compact change, with obvious meaning. I don't know how strong Kim's objection to this is. I'll wait for him to comment on it. StefanK > > Thanks, > Coleen >> >>> >>> And if mtNone is going to be used that way, then GrowableArrayCHeap >>> should static assert that it's flag value is not mtNone. >>> >> >> Sure. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/utilities/growableArray.hpp >>> ? 586 public: >>> ? 587?? // Where are we going to allocate memory? >>> ? 588?? bool on_C_heap() { return _metadata.on_C_heap(); } >>> ? 589?? bool on_stack () { return _metadata.on_stack(); } >>> ? 590?? bool on_arena () { return _metadata.on_arena(); } >>> >>> These were previously protected; why make them public? >>> >> >> Mainly to be usable by the tests. I can add a friend test class and >> revert this. >> >>> ------------------------------------------------------------------------------ >>> >>> >> >> Thanks for reviewing this, >> StefanK > From erik.osterlund at oracle.com Mon Jun 22 15:05:00 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 22 Jun 2020 17:05:00 +0200 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <64ba7392-c728-9865-a597-3c97d7b3ee0c@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> <64ba7392-c728-9865-a597-3c97d7b3ee0c@oracle.com> Message-ID: <38c9c981-089e-82f3-d1fe-efcbc641e31b@oracle.com> Hi Stefan, I really like the direction of this work. Tiny nit: In oopHandle.hpp maybe include oopsHierarchy.hpp instead of forward declaring class oop; Because in product builds it won't be a class, it will be an oopDesc*, so not sure how happy compilers are with that forward declaration not matching the type. Otherwise, looks good to me. Thanks, /Erik On 2020-06-22 14:12, Stefan Karlsson wrote: > > On 2020-06-19 17:56, coleen.phillimore at oracle.com wrote: >> >> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/src/hotspot/share/classfile/classLoaderData.hpp.udiff.html >> >> >> + WeakHandle _holder; // The oop that determines lifetime of this >> class loader >> + OopHandle _class_loader; // The instance of java/lang/ClassLoader >> associated with >> + // this ClassLoaderData >> >> >> Can you line the member names up? >> >> >> On 6/18/20 6:09 PM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to rework WeakHandle and OopHandle to >>> dynamically support different OopStorages. >>> >>> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8247879 >>> >>> Today WeakHandle has a template parameter and specialization of >>> get_storage() to get hold of the backing OopStorage. Coleen, Erik? >>> and I talked about different ways to rewrite that, so that we >>> wouldn't have to write specializations for all weak oop storages, >>> and so that we don't have to carry around the type information about >>> the weak oop storage. >>> >>> Instead of writing: >>> WeakHandle w = WeakHandle::create(obj) >>> >>> we could instead write: >>> WeakHandle w(OopStorageSet::vm_weak(), obj) >>> >>> OopHandle is today hard-coded to use OopStorageSet::vm_global(). The >>> proposal is to change OopHandle to look like WeakHandle, and allow >>> more than one OopStorage. Something that might will become important >>> going forward, when we are going to move more things into separate >>> OopStorages. >>> >>> So, instead of writing: >>> OopHandle h = OopHandle::create(obj) >>> >>> one will have to specify the OopStorage explicitly: >>> OopHandle h(OopStorageSet::vm_global(), obj) >>> >>> This change has a slight drawback that the OopStorage used for the >>> allocation of the handle, needs to be provided when the handle is >>> released. >>> >>> To make this safer, so that you don't use strong OopStorages when >>> allocating WeakHandles, or weak OopStorages when allocating >>> OopHandles, I've created two subclasses to OopStorage: >>> StrongOopStorage and WeakOopStorage. They are just two tiny wrappers >>> around OopStorage, to allow us ensure correct usage during compile >>> time. The oopStorageSet.hpp files seems to be intentionally written >>> to *not* know about the OopStorage implementation. To do proper >>> casting of the OopStorage*s I need to include the definition of >>> StrongOopStorage and WeakOopStorage. I've therefore moved all >>> OopStorageSet::() getters out to an inline >>> header. This adds a lot of new includes. If we don't think that it's >>> important to inline these, then I can move it over to the cpp file >>> instead. >> >> I don't like these includes.? In the places where we allocate and >> release OopHandle/WeakHandles, where we need to refer to these >> storages, performance isn't important to avoid a call.? I don't know >> about all of the GC places. >> >> I also don't see a large benefit to type safety that these new >> classes provide, to be honest, since there aren't many call sites >> where we create and release these handles.? We're more likely to mix >> different strong oop storages (allocate from one and release from >> another), once we have more than one strong OopStorage. >> OopStorage::release has a check that the memory is in that storage (I >> hope), so there already should be a runtime check. >> >> I'll defer to whatever Kim thinks though.? This doesn't seem worth >> having to me though. >> >> Otherwise, the changes look good. > > I reverted all the Strong/Weak designations. Is this OK?: > https://cr.openjdk.java.net/~stefank/8247879/webrev.02 > > StefanK > >> >> thanks, >> Coleen >>> >>> Thanks, >>> StefanK >>> >> From erik.joelsson at oracle.com Mon Jun 22 15:34:49 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 22 Jun 2020 08:34:49 -0700 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: <2BF5F539-4DB1-477D-8AEC-D56AA78EA723@azul.com> References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> <2BF5F539-4DB1-477D-8AEC-D56AA78EA723@azul.com> Message-ID: Looks good. /Erik On 2020-06-22 03:39, Vladimir Kempik wrote: > Hello > Thanks for looking into this Magnus. > > Here is updated version of webrev - http://cr.openjdk.java.net/~vkempik/8243470/webrev.01/ > effectively it does exactly the same as previous version, I have removed the version check for clang. > > > Thanks, Vladimir > >> 4 ???? 2020 ?., ? 16:04, Magnus Ihse Bursie ???????(?): >> >> On 2020-06-03 20:07, Magnus Ihse Bursie wrote: >>> Thanks for cc:ing us. >>> >>> This is not the correct way to check for compiler versions. Nor is it the correct place. >>> >>> I don't have time for a full reply tonight, but will return with a better reply tomorrow. >> Ok, I have now looked more into this. >> >> You are apparently want to check for Xcode version, not clang version. This is not the same. Note that clang can be used on Linux as well. Apple is basically dropping the normal clang versioning and replacing it with their own, messed-up version which somewhat resembles the corresponding Xcode version, but is not identical. Horray. :-/ (For those interested, have a look at https://gist.github.com/yamaya/2924292.) >> >> Anyway. We do not support Xcode prior to 8. You seem to have tested from 8 and up, so this patch should be applied unconditionally. That is, just remove the -O1 special thing. >> >> /Magnus >>> /Magnus >>> >>> On 2020-06-03 19:47, Daniel D. Daugherty wrote: >>>> Adding build-dev at ... since this is a build system change. >>>> >>>> As for the right HotSpot team, I'm not sure who should be making >>>> the call on changing the way unsafe.cpp is built for macOSX. My guess >>>> when I moved the bug to hotspot/runtime was that someone on the Runtime >>>> team would know, but... >>>> >>>> Dan >>>> >>>> >>>> >>>> On 6/3/20 1:38 PM, Vladimir Kempik wrote: >>>>> Hello >>>>> >>>>> Can somebody please review this simple change ? >>>>> >>>>> Thanks >>>>> >>>>>> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >>>>>> >>>>>> Adding hotspot-runtime-dev >>>>>> >>>>>>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik ???????(?): >>>>>>> >>>>>>> >>>>>>> Hello >>>>>>> Please review a fix for JDK-8243470 >>>>>>> >>>>>>> Long time ago as part of JEP284: New HotSpot Build System this fix was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>>>>>> At that time it was decided to lower optimisation level for unsafe.cpp from O2 to O1 only for clang on Macosx. >>>>>>> >>>>>>> I suppose it was done due to issues in Set/Get helper functions where too optimistic optimisations were eliminating some null pointer checks. it was probably a clang bug. >>>>>>> That issue could be checked with test jdk/test/sun/misc/CopyMemory.java. >>>>>>> >>>>>>> I believe that workaround (going from O2 to O1) produced this issue - https://bugs.openjdk.java.net/browse/JDK-8234963 (Thread.getStackTrace is slow with clang). >>>>>>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>>>>>> >>>>>>> Here I propose the patch which eliminates that workaround for clang 8+. >>>>>>> >>>>>>> I have tested clang versions 8/9/9.1/10, all of them showed good results: >>>>>>> 1) CopyMemory test passes fine on 11/14/15. >>>>>>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. jdk15: no new failures in tck. >>>>>>> 3) The testRandom "benchmark" from 8234963 shows great improvements on my machine, going down from ~200 seconds to ~150 seconds (the newer clang the better result). For comparision, gcc built libjvm for jdk11 shows ~130 seconds on my machine. >>>>>>> >>>>>>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>>>>>> >>>>>>> getStackTrace benchmark: http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>>>>>> >>>>>>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>>>>>> >>>>>>> Thanks, Vladimir From magnus.ihse.bursie at oracle.com Mon Jun 22 18:11:57 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 22 Jun 2020 20:11:57 +0200 Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp In-Reply-To: <2BF5F539-4DB1-477D-8AEC-D56AA78EA723@azul.com> References: <7EA2B268-4267-4219-B257-1EC46808C82C@azul.com> <2BF5F539-4DB1-477D-8AEC-D56AA78EA723@azul.com> Message-ID: Looks good to me. /Magnus > 22 juni 2020 kl. 12:39 skrev Vladimir Kempik : > > Hello > Thanks for looking into this Magnus. > > Here is updated version of webrev - http://cr.openjdk.java.net/~vkempik/8243470/webrev.01/ > effectively it does exactly the same as previous version, I have removed the version check for clang. > > > Thanks, Vladimir > >> 4 ???? 2020 ?., ? 16:04, Magnus Ihse Bursie ???????(?): >> >>> On 2020-06-03 20:07, Magnus Ihse Bursie wrote: >>> Thanks for cc:ing us. >>> >>> This is not the correct way to check for compiler versions. Nor is it the correct place. >>> >>> I don't have time for a full reply tonight, but will return with a better reply tomorrow. >> Ok, I have now looked more into this. >> >> You are apparently want to check for Xcode version, not clang version. This is not the same. Note that clang can be used on Linux as well. Apple is basically dropping the normal clang versioning and replacing it with their own, messed-up version which somewhat resembles the corresponding Xcode version, but is not identical. Horray. :-/ (For those interested, have a look at https://gist.github.com/yamaya/2924292.) >> >> Anyway. We do not support Xcode prior to 8. You seem to have tested from 8 and up, so this patch should be applied unconditionally. That is, just remove the -O1 special thing. >> >> /Magnus >>> >>> /Magnus >>> >>>> On 2020-06-03 19:47, Daniel D. Daugherty wrote: >>>> Adding build-dev at ... since this is a build system change. >>>> >>>> As for the right HotSpot team, I'm not sure who should be making >>>> the call on changing the way unsafe.cpp is built for macOSX. My guess >>>> when I moved the bug to hotspot/runtime was that someone on the Runtime >>>> team would know, but... >>>> >>>> Dan >>>> >>>> >>>> >>>>> On 6/3/20 1:38 PM, Vladimir Kempik wrote: >>>>> Hello >>>>> >>>>> Can somebody please review this simple change ? >>>>> >>>>> Thanks >>>>> >>>>>> 6 ??? 2020 ?., ? 12:43, Vladimir Kempik ???????(?): >>>>>> >>>>>> Adding hotspot-runtime-dev >>>>>> >>>>>>> 23 ???. 2020 ?., ? 18:26, Vladimir Kempik ???????(?): >>>>>>> >>>>>>> >>>>>>> Hello >>>>>>> Please review a fix for JDK-8243470 >>>>>>> >>>>>>> Long time ago as part of JEP284: New HotSpot Build System this fix was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666 >>>>>>> At that time it was decided to lower optimisation level for unsafe.cpp from O2 to O1 only for clang on Macosx. >>>>>>> >>>>>>> I suppose it was done due to issues in Set/Get helper functions where too optimistic optimisations were eliminating some null pointer checks. it was probably a clang bug. >>>>>>> That issue could be checked with test jdk/test/sun/misc/CopyMemory.java. >>>>>>> >>>>>>> I believe that workaround (going from O2 to O1) produced this issue - https://bugs.openjdk.java.net/browse/JDK-8234963 (Thread.getStackTrace is slow with clang). >>>>>>> JDK-8234963 can only be seen on mac with libjvm compiled by clang. >>>>>>> >>>>>>> Here I propose the patch which eliminates that workaround for clang 8+. >>>>>>> >>>>>>> I have tested clang versions 8/9/9.1/10, all of them showed good results: >>>>>>> 1) CopyMemory test passes fine on 11/14/15. >>>>>>> 2) jdk11/jdk14 passed tck. Regression testing were good as well. jdk15: no new failures in tck. >>>>>>> 3) The testRandom "benchmark" from 8234963 shows great improvements on my machine, going down from ~200 seconds to ~150 seconds (the newer clang the better result). For comparision, gcc built libjvm for jdk11 shows ~130 seconds on my machine. >>>>>>> >>>>>>> The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/ >>>>>>> >>>>>>> getStackTrace benchmark: http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java >>>>>>> >>>>>>> The bug: https://bugs.openjdk.java.net/browse/JDK-8243470 >>>>>>> >>>>>>> Thanks, Vladimir >>>> >>> > From stefan.karlsson at oracle.com Mon Jun 22 20:31:29 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 22 Jun 2020 22:31:29 +0200 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <38c9c981-089e-82f3-d1fe-efcbc641e31b@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> <64ba7392-c728-9865-a597-3c97d7b3ee0c@oracle.com> <38c9c981-089e-82f3-d1fe-efcbc641e31b@oracle.com> Message-ID: <02c10886-cbac-799a-601f-f7a8ccf36ee1@oracle.com> On 2020-06-22 17:05, Erik ?sterlund wrote: > Hi Stefan, > > I really like the direction of this work. Thanks. We've heard some offline concerns voiced about need to provide an OopStorage to the release(...) function. Given that, I'm going to wait with this patch until that has been discussed/resolved. Tiny nit: > In oopHandle.hpp maybe include oopsHierarchy.hpp instead of forward > declaring class oop; Because in product builds it won't be a class, it > will be an oopDesc*, so not sure how happy compilers are with that > forward declaration not matching the type. Thanks. I fixed this in another patch, but it should of course be in this patch. I also found this pre-existing oddity that we might want to cleanup in this patch: class WeakHandle { public: private: > > Otherwise, looks good to me. Thanks, StefanK > > Thanks, > /Erik > > On 2020-06-22 14:12, Stefan Karlsson wrote: >> >> On 2020-06-19 17:56, coleen.phillimore at oracle.com wrote: >>> >>> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/src/hotspot/share/classfile/classLoaderData.hpp.udiff.html >>> >>> >>> + WeakHandle _holder; // The oop that determines lifetime of this >>> class loader >>> + OopHandle _class_loader; // The instance of java/lang/ClassLoader >>> associated with >>> + // this ClassLoaderData >>> >>> >>> Can you line the member names up? >>> >>> >>> On 6/18/20 6:09 PM, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to rework WeakHandle and OopHandle to >>>> dynamically support different OopStorages. >>>> >>>> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8247879 >>>> >>>> Today WeakHandle has a template parameter and specialization of >>>> get_storage() to get hold of the backing OopStorage. Coleen, Erik? >>>> and I talked about different ways to rewrite that, so that we >>>> wouldn't have to write specializations for all weak oop storages, >>>> and so that we don't have to carry around the type information about >>>> the weak oop storage. >>>> >>>> Instead of writing: >>>> WeakHandle w = WeakHandle::create(obj) >>>> >>>> we could instead write: >>>> WeakHandle w(OopStorageSet::vm_weak(), obj) >>>> >>>> OopHandle is today hard-coded to use OopStorageSet::vm_global(). The >>>> proposal is to change OopHandle to look like WeakHandle, and allow >>>> more than one OopStorage. Something that might will become important >>>> going forward, when we are going to move more things into separate >>>> OopStorages. >>>> >>>> So, instead of writing: >>>> OopHandle h = OopHandle::create(obj) >>>> >>>> one will have to specify the OopStorage explicitly: >>>> OopHandle h(OopStorageSet::vm_global(), obj) >>>> >>>> This change has a slight drawback that the OopStorage used for the >>>> allocation of the handle, needs to be provided when the handle is >>>> released. >>>> >>>> To make this safer, so that you don't use strong OopStorages when >>>> allocating WeakHandles, or weak OopStorages when allocating >>>> OopHandles, I've created two subclasses to OopStorage: >>>> StrongOopStorage and WeakOopStorage. They are just two tiny wrappers >>>> around OopStorage, to allow us ensure correct usage during compile >>>> time. The oopStorageSet.hpp files seems to be intentionally written >>>> to *not* know about the OopStorage implementation. To do proper >>>> casting of the OopStorage*s I need to include the definition of >>>> StrongOopStorage and WeakOopStorage. I've therefore moved all >>>> OopStorageSet::() getters out to an inline >>>> header. This adds a lot of new includes. If we don't think that it's >>>> important to inline these, then I can move it over to the cpp file >>>> instead. >>> >>> I don't like these includes.? In the places where we allocate and >>> release OopHandle/WeakHandles, where we need to refer to these >>> storages, performance isn't important to avoid a call.? I don't know >>> about all of the GC places. >>> >>> I also don't see a large benefit to type safety that these new >>> classes provide, to be honest, since there aren't many call sites >>> where we create and release these handles.? We're more likely to mix >>> different strong oop storages (allocate from one and release from >>> another), once we have more than one strong OopStorage. >>> OopStorage::release has a check that the memory is in that storage (I >>> hope), so there already should be a runtime check. >>> >>> I'll defer to whatever Kim thinks though.? This doesn't seem worth >>> having to me though. >>> >>> Otherwise, the changes look good. >> >> I reverted all the Strong/Weak designations. Is this OK?: >> https://cr.openjdk.java.net/~stefank/8247879/webrev.02 >> >> StefanK >> >>> >>> thanks, >>> Coleen >>>> >>>> Thanks, >>>> StefanK >>>> >>> > From coleen.phillimore at oracle.com Mon Jun 22 22:20:53 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 22 Jun 2020 18:20:53 -0400 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <02c10886-cbac-799a-601f-f7a8ccf36ee1@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> <64ba7392-c728-9865-a597-3c97d7b3ee0c@oracle.com> <38c9c981-089e-82f3-d1fe-efcbc641e31b@oracle.com> <02c10886-cbac-799a-601f-f7a8ccf36ee1@oracle.com> Message-ID: <01432ab1-1c4d-f081-f4ab-816023817dfe@oracle.com> On 6/22/20 4:31 PM, Stefan Karlsson wrote: > > > On 2020-06-22 17:05, Erik ?sterlund wrote: >> Hi Stefan, >> >> I really like the direction of this work. > > Thanks. We've heard some offline concerns voiced about need to provide > an OopStorage to the release(...) function. Given that, I'm going to > wait with this patch until that has been discussed/resolved. > > ?Tiny nit: >> In oopHandle.hpp maybe include oopsHierarchy.hpp instead of forward >> declaring class oop; Because in product builds it won't be a class, >> it will be an oopDesc*, so not sure how happy compilers are with that >> forward declaration not matching the type. > I've changed my JVMTI change to use OopHandles based on your patch and the forward declaration of oop in oopHandle.hpp, does cause a build error for one of the targets, which might use precompiled headers: /workspace/open/src/hotspot/share/oops/oopHandle.hpp:28:7: error: using typedef-name 'oop' after 'class' [2020-06-22T18:02:13,683Z] 28 | class oop; [2020-06-22T18:02:13,683Z] | ^~~ /workspace/open/src/hotspot/share/oops/oopsHierarchy.hpp:45:43: note: 'oop' has a previous declaration here [2020-06-22T18:02:13,686Z] 45 | typedef class oopDesc* oop; [2020-06-22T18:02:13,687Z] | ^~~ Coleen > Thanks. I fixed this in another patch, but it should of course be in > this patch. I also found this pre-existing oddity that we might want > to cleanup in this patch: > > ?class WeakHandle { > ? public: > ? private: > > >> >> Otherwise, looks good to me. > > Thanks, > StefanK > >> >> Thanks, >> /Erik >> >> On 2020-06-22 14:12, Stefan Karlsson wrote: >>> >>> On 2020-06-19 17:56, coleen.phillimore at oracle.com wrote: >>>> >>>> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/src/hotspot/share/classfile/classLoaderData.hpp.udiff.html >>>> >>>> >>>> + WeakHandle _holder; // The oop that determines lifetime of this >>>> class loader >>>> + OopHandle _class_loader; // The instance of java/lang/ClassLoader >>>> associated with >>>> + // this ClassLoaderData >>>> >>>> >>>> Can you line the member names up? >>>> >>>> >>>> On 6/18/20 6:09 PM, Stefan Karlsson wrote: >>>>> Hi all, >>>>> >>>>> Please review this patch to rework WeakHandle and OopHandle to >>>>> dynamically support different OopStorages. >>>>> >>>>> https://cr.openjdk.java.net/~stefank/8247879/webrev.01/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8247879 >>>>> >>>>> Today WeakHandle has a template parameter and specialization of >>>>> get_storage() to get hold of the backing OopStorage. Coleen, Erik? >>>>> and I talked about different ways to rewrite that, so that we >>>>> wouldn't have to write specializations for all weak oop storages, >>>>> and so that we don't have to carry around the type information >>>>> about the weak oop storage. >>>>> >>>>> Instead of writing: >>>>> WeakHandle w = WeakHandle::create(obj) >>>>> >>>>> we could instead write: >>>>> WeakHandle w(OopStorageSet::vm_weak(), obj) >>>>> >>>>> OopHandle is today hard-coded to use OopStorageSet::vm_global(). >>>>> The proposal is to change OopHandle to look like WeakHandle, and >>>>> allow more than one OopStorage. Something that might will become >>>>> important going forward, when we are going to move more things >>>>> into separate OopStorages. >>>>> >>>>> So, instead of writing: >>>>> OopHandle h = OopHandle::create(obj) >>>>> >>>>> one will have to specify the OopStorage explicitly: >>>>> OopHandle h(OopStorageSet::vm_global(), obj) >>>>> >>>>> This change has a slight drawback that the OopStorage used for the >>>>> allocation of the handle, needs to be provided when the handle is >>>>> released. >>>>> >>>>> To make this safer, so that you don't use strong OopStorages when >>>>> allocating WeakHandles, or weak OopStorages when allocating >>>>> OopHandles, I've created two subclasses to OopStorage: >>>>> StrongOopStorage and WeakOopStorage. They are just two tiny >>>>> wrappers around OopStorage, to allow us ensure correct usage >>>>> during compile time. The oopStorageSet.hpp files seems to be >>>>> intentionally written to *not* know about the OopStorage >>>>> implementation. To do proper casting of the OopStorage*s I need to >>>>> include the definition of StrongOopStorage and WeakOopStorage. >>>>> I've therefore moved all OopStorageSet::() >>>>> getters out to an inline header. This adds a lot of new includes. >>>>> If we don't think that it's important to inline these, then I can >>>>> move it over to the cpp file instead. >>>> >>>> I don't like these includes.? In the places where we allocate and >>>> release OopHandle/WeakHandles, where we need to refer to these >>>> storages, performance isn't important to avoid a call.? I don't >>>> know about all of the GC places. >>>> >>>> I also don't see a large benefit to type safety that these new >>>> classes provide, to be honest, since there aren't many call sites >>>> where we create and release these handles.? We're more likely to >>>> mix different strong oop storages (allocate from one and release >>>> from another), once we have more than one strong OopStorage. >>>> OopStorage::release has a check that the memory is in that storage >>>> (I hope), so there already should be a runtime check. >>>> >>>> I'll defer to whatever Kim thinks though.? This doesn't seem worth >>>> having to me though. >>>> >>>> Otherwise, the changes look good. >>> >>> I reverted all the Strong/Weak designations. Is this OK?: >>> https://cr.openjdk.java.net/~stefank/8247879/webrev.02 >>> >>> StefanK >>> >>>> >>>> thanks, >>>> Coleen >>>>> >>>>> Thanks, >>>>> StefanK >>>>> >>>> >> From stefan.karlsson at oracle.com Tue Jun 23 07:14:33 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 23 Jun 2020 07:14:33 +0000 (UTC) Subject: 8248130: AArch64: pf() uses mtNone instead of mtInternal Message-ID: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> Hi all, Please review this trivial patch to change the NMT flag used by one of the debugging functions on AArch64. https://cr.openjdk.java.net/~stefank/8248130/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8248130 If pf() is used when NMT is turned on we hit this assert: MallocSite* MallocSiteTable::lookup_or_add(const NativeCallStack& key, size_t* bucket_idx, size_t* pos_idx, MEMFLAGS flags) { assert(flags != mtNone, "Should have a real memory type"); The patch simply changes the used flag to mtInternal. Thanks, StefanK From thomas.schatzl at oracle.com Tue Jun 23 08:42:11 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 23 Jun 2020 10:42:11 +0200 Subject: 8248130: AArch64: pf() uses mtNone instead of mtInternal In-Reply-To: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> References: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> Message-ID: <342da471-7a09-2ceb-04ab-894c25ae3be2@oracle.com> Hi, On 23.06.20 09:14, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to change the NMT flag used by one of > the debugging functions on AArch64. > > https://cr.openjdk.java.net/~stefank/8248130/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8248130 > > If pf() is used when NMT is turned on we hit this assert: > MallocSite* MallocSiteTable::lookup_or_add(const NativeCallStack& key, > size_t* bucket_idx, > ? size_t* pos_idx, MEMFLAGS flags) { > ? assert(flags != mtNone, "Should have a real memory type"); > > The patch simply changes the used flag to mtInternal. > looks good and trivial. Thomas From kim.barrett at oracle.com Tue Jun 23 08:46:06 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Jun 2020 04:46:06 -0400 Subject: 8248130: AArch64: pf() uses mtNone instead of mtInternal In-Reply-To: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> References: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> Message-ID: <35DF9BB9-2003-41BF-84D1-F4EA5FC4BA64@oracle.com> > On Jun 23, 2020, at 3:14 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this trivial patch to change the NMT flag used by one of the debugging functions on AArch64. > > https://cr.openjdk.java.net/~stefank/8248130/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8248130 > > If pf() is used when NMT is turned on we hit this assert: > MallocSite* MallocSiteTable::lookup_or_add(const NativeCallStack& key, size_t* bucket_idx, > size_t* pos_idx, MEMFLAGS flags) { > assert(flags != mtNone, "Should have a real memory type"); > > The patch simply changes the used flag to mtInternal. > > Thanks, > StefanK Good, and trivial. Thanks for cleaning this up. From stefan.karlsson at oracle.com Tue Jun 23 09:26:57 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 23 Jun 2020 11:26:57 +0200 Subject: 8248130: AArch64: pf() uses mtNone instead of mtInternal In-Reply-To: <35DF9BB9-2003-41BF-84D1-F4EA5FC4BA64@oracle.com> References: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> <35DF9BB9-2003-41BF-84D1-F4EA5FC4BA64@oracle.com> Message-ID: <6bf3a301-8904-3610-12ae-a425cb3873bc@oracle.com> Thanks for reviewing. StefanK On 2020-06-23 10:46, Kim Barrett wrote: >> On Jun 23, 2020, at 3:14 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this trivial patch to change the NMT flag used by one of the debugging functions on AArch64. >> >> https://cr.openjdk.java.net/~stefank/8248130/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8248130 >> >> If pf() is used when NMT is turned on we hit this assert: >> MallocSite* MallocSiteTable::lookup_or_add(const NativeCallStack& key, size_t* bucket_idx, >> size_t* pos_idx, MEMFLAGS flags) { >> assert(flags != mtNone, "Should have a real memory type"); >> >> The patch simply changes the used flag to mtInternal. >> >> Thanks, >> StefanK > > Good, and trivial. Thanks for cleaning this up. > > From kim.barrett at oracle.com Tue Jun 23 09:35:34 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Jun 2020 05:35:34 -0400 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> Message-ID: > On Jun 22, 2020, at 5:01 AM, Stefan Karlsson wrote: > > On 2020-06-22 09:25, Kim Barrett wrote: >> src/hotspot/share/utilities/globalDefinitions.hpp >> 435 // Need the correct linkage to call qsort without warnings >> 436 extern "C" { >> 437 typedef int (*_sort_Fn)(const void *, const void *); >> 438 } >> Rather than messing with function pointer types, why not use >> Quicksort::sort from utilities/quickSort.hpp? > > I didn't change this code, only moved it. This was done to reduce the clutter in growableArray.hpp. You don't suggest that I change this in this RFR, right? I missed that this was moved from growableArray.hpp, having been in one of the largish deleted chunks there. So there's a bit of a mess here, none of it related to your change. For example, ./share/oops/instanceKlass.cpp: // _sort_Fn is defined in growableArray.hpp. And worse, the semantics of extern "C" includes things like possibly using different calling conventions. So casting a C++ function to _sort_Fn seems technically broken, though apparently happens to work on supported platforms. (In which case it would seem to effectively be a nop.) All of that could be eliminated by using Quicksort::sort (or std::sort, but I don't want to get into that here). I've filed a bug: https://bugs.openjdk.java.net/browse/JDK-8248137 >> src/hotspot/share/memory/metaspaceShared.cpp >> 459 static GrowableArrayCHeap* _extra_interned_strings = NULL; >> ... >> 462 _extra_interned_strings = new GrowableArrayCHeap(10000); >> [pre-existing] >> I think this vector is only needed during the preload_and_dump >> operation that calls read_extra_data, and can then be discarded. [?] > > Yes. I mainly wanted to use this as an example of a CHeap/CHeap-allocated GrowableArray. I also think it would make sense to change this to n on-stack CHeap array. OK. (Did I see a bug for cleaning this up go by?) >> src/hotspot/share/utilities/growableArray.hpp >> 115 class GrowableArrayView : public GrowableArrayBase { >> I don't like the name "View" for this concept. Common practice >> elsewhere (including the Standard) is for "view" types to be an >> adaptor over some other type, providing a limited range or otherwise >> adjusted access. (Our BitMapView does that.) >> GrowableArrayData? >> I'm not sure how often this class name gets used by clients, so it >> might not matter that it's a little bit long or clumsy / clunky > > > My thinking was the just like the with the BitMapView, the backing storage is managed externally: > > // The BitMapView is used when the backing storage is managed externally. > class BitMapView : public BitMap { > public: > BitMapView() : BitMap(NULL, 0) {} > BitMapView(bm_word_t* map, idx_t size_in_bits) : BitMap(map, size_in_bits) {} > }; > > I don't think GrowableArrayData is a better name, so if I need to change this, I'm going to need some decent suggestions. I talked briefly with Kim, but we couldn't immediately find a name that both liked. Suggestions? I haven?t come up with anything better than we discussed. >> src/hotspot/share/utilities/growableArray.hpp >> Destructors for all but the most-derived classes should be non-public. >> Probably also the constructors. That is, all of GrowableArrayBase, >> GrowableArrayView, and GrowableArrayWithAllocator should have >> protected constructors and destructors. > > OK. For GrowableArrayivew, I had an idea that someone *might* want to do: > > int _array[10]; > ... > GrowableArrayView view(array, 10, 10); > do_something(view); > > but I'll change this to protected. Classes that are used as both a base class and a concrete class can cause problems. For example, allowing accidental slicing, which leads to UB (which may or may not be detected in some way). >> src/hotspot/share/utilities/growableArray.hpp >> [pre-existing] >> GrowableArray's use of mtNone to indicate resource allocation seems >> like an abuse of that memory type. That memory type has a description >> string of "Unknown", nothing to do with resources. But it's considered >> an invalid type by MamTracker/MemReporter/MallocSite, and it seems to >> mean something else in the VirtualMemoryTracker, like "uninitialized". > > I thinks we view this differently. My view is that memflags belong to the CHeap allocations, but is only valid if != mtNone. Setting it to mtNone doesn't imply that resource allocations are marked with mtNone. It just means that we shouldn't care about the memflags value when allocating resource memory. I?m OK with an interpretation that mtNone is considered an invalid memflags everywhere. There may be some places that ought to be treating it that way but aren?t. The aarch64 pf function is one; you filed a bug and have a fix for that one - thanks. >> But why have constructors with optional MEMFLAGS at all? And why use >> mtNone that way? Constructors without MEMFLAGS parameter are resource >> allocating, constructors with required MEMFLAGS parameter are CHeap >> allocating. (Though the random status of mtNone noted above will make >> that one probably not actually usable.) (That would require some small >> adjustments to GrowableArrayMetadata.) > > I think you've lost me here. Are you proposing that I split this up into more constructors. Are you suggesting that I change this: > > [?] > and adjust allocate(...) and GrowableArrayMetadata(...) accordingly? Not relevant is mtNone signals ?invalid?. >> src/hotspot/share/utilities/growableArray.hpp >> 586 public: >> 587 // Where are we going to allocate memory? >> 588 bool on_C_heap() { return _metadata.on_C_heap(); } >> 589 bool on_stack () { return _metadata.on_stack(); } >> 590 bool on_arena () { return _metadata.on_arena(); } >> These were previously protected; why make them public? > > Mainly to be usable by the tests. I can add a friend test class and revert this. OK. I think I?d prefer these not be in the public API; I wouldn?t want someone writing code that was conditional on any of them? Your call. From kim.barrett at oracle.com Tue Jun 23 09:39:57 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Jun 2020 05:39:57 -0400 Subject: RFR: 8247879: Rework WeakHandle and OopHandle to dynamically support different OopStorages In-Reply-To: <02c10886-cbac-799a-601f-f7a8ccf36ee1@oracle.com> References: <1a28aad1-c4b6-f55e-f546-a87bd3f1819e@oracle.com> <0629ac1c-0a91-2f7f-40b9-a9c42986c3f4@oracle.com> <64ba7392-c728-9865-a597-3c97d7b3ee0c@oracle.com> <38c9c981-089e-82f3-d1fe-efcbc641e31b@oracle.com> <02c10886-cbac-799a-601f-f7a8ccf36ee1@oracle.com> Message-ID: <0AE6F101-F749-47D6-AC22-50415A033267@oracle.com> > On Jun 22, 2020, at 4:31 PM, Stefan Karlsson wrote: > > On 2020-06-22 17:05, Erik ?sterlund wrote: >> Hi Stefan, >> I really like the direction of this work. > > Thanks. We've heard some offline concerns voiced about need to provide an OopStorage to the release(...) function. Given that, I'm going to wait with this patch until that has been discussed/resolved. I had some alternative ideas that ended up not panning out, so I dropped the concerns I?d raised offline. From david.holmes at oracle.com Tue Jun 23 09:50:47 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 Jun 2020 19:50:47 +1000 Subject: 8248130: AArch64: pf() uses mtNone instead of mtInternal In-Reply-To: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> References: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> Message-ID: Looks good and trivial. Thanks, David On 23/06/2020 5:14 pm, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to change the NMT flag used by one of > the debugging functions on AArch64. > > https://cr.openjdk.java.net/~stefank/8248130/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8248130 > > If pf() is used when NMT is turned on we hit this assert: > MallocSite* MallocSiteTable::lookup_or_add(const NativeCallStack& key, > size_t* bucket_idx, > ? size_t* pos_idx, MEMFLAGS flags) { > ? assert(flags != mtNone, "Should have a real memory type"); > > The patch simply changes the used flag to mtInternal. > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue Jun 23 10:02:25 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 23 Jun 2020 12:02:25 +0200 Subject: 8248130: AArch64: pf() uses mtNone instead of mtInternal In-Reply-To: References: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> Message-ID: Thanks, David. StefanK On 2020-06-23 11:50, David Holmes wrote: > Looks good and trivial. > > Thanks, > David > > On 23/06/2020 5:14 pm, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to change the NMT flag used by one of >> the debugging functions on AArch64. >> >> https://cr.openjdk.java.net/~stefank/8248130/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8248130 >> >> If pf() is used when NMT is turned on we hit this assert: >> MallocSite* MallocSiteTable::lookup_or_add(const NativeCallStack& key, >> size_t* bucket_idx, >> ?? size_t* pos_idx, MEMFLAGS flags) { >> ?? assert(flags != mtNone, "Should have a real memory type"); >> >> The patch simply changes the used flag to mtInternal. >> >> Thanks, >> StefanK From stefan.karlsson at oracle.com Tue Jun 23 10:23:38 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 23 Jun 2020 12:23:38 +0200 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> Message-ID: <3aae3413-b0cd-bf28-c931-031d24fbf28b@oracle.com> Latest patches: https://cr.openjdk.java.net/~stefank/8247755/webrev.02.delta/ https://cr.openjdk.java.net/~stefank/8247755/webrev.02/ I haven't changed: - GrowableArrayView to something else - S to ALLOC or Derived. I'm leaning towards Derived. Some comments inlined: On 2020-06-23 11:35, Kim Barrett wrote: >> On Jun 22, 2020, at 5:01 AM, Stefan Karlsson wrote: >> >> On 2020-06-22 09:25, Kim Barrett wrote: >>> src/hotspot/share/utilities/globalDefinitions.hpp >>> 435 // Need the correct linkage to call qsort without warnings >>> 436 extern "C" { >>> 437 typedef int (*_sort_Fn)(const void *, const void *); >>> 438 } >>> Rather than messing with function pointer types, why not use >>> Quicksort::sort from utilities/quickSort.hpp? >> >> I didn't change this code, only moved it. This was done to reduce the clutter in growableArray.hpp. You don't suggest that I change this in this RFR, right? > > I missed that this was moved from growableArray.hpp, having been in > one of the largish deleted chunks there. > > So there's a bit of a mess here, none of it related to your change. > For example, > > ./share/oops/instanceKlass.cpp: // _sort_Fn is defined in growableArray.hpp. > > And worse, the semantics of extern "C" includes things like possibly > using different calling conventions. So casting a C++ function to > _sort_Fn seems technically broken, though apparently happens to work > on supported platforms. (In which case it would seem to effectively > be a nop.) > > All of that could be eliminated by using Quicksort::sort (or > std::sort, but I don't want to get into that here). > > I've filed a bug: > https://bugs.openjdk.java.net/browse/JDK-8248137 Thanks. > >>> src/hotspot/share/memory/metaspaceShared.cpp >>> 459 static GrowableArrayCHeap* _extra_interned_strings = NULL; >>> ... >>> 462 _extra_interned_strings = new GrowableArrayCHeap(10000); >>> [pre-existing] >>> I think this vector is only needed during the preload_and_dump >>> operation that calls read_extra_data, and can then be discarded. [?] >> >> Yes. I mainly wanted to use this as an example of a CHeap/CHeap-allocated GrowableArray. I also think it would make sense to change this to n on-stack CHeap array. > > OK. (Did I see a bug for cleaning this up go by?) I created this: https://bugs.openjdk.java.net/browse/JDK-8248007 > >>> src/hotspot/share/utilities/growableArray.hpp >>> 115 class GrowableArrayView : public GrowableArrayBase { >>> I don't like the name "View" for this concept. Common practice >>> elsewhere (including the Standard) is for "view" types to be an >>> adaptor over some other type, providing a limited range or otherwise >>> adjusted access. (Our BitMapView does that.) >>> GrowableArrayData? >>> I'm not sure how often this class name gets used by clients, so it >>> might not matter that it's a little bit long or clumsy / clunky > >> >> My thinking was the just like the with the BitMapView, the backing storage is managed externally: >> >> // The BitMapView is used when the backing storage is managed externally. >> class BitMapView : public BitMap { >> public: >> BitMapView() : BitMap(NULL, 0) {} >> BitMapView(bm_word_t* map, idx_t size_in_bits) : BitMap(map, size_in_bits) {} >> }; >> >> I don't think GrowableArrayData is a better name, so if I need to change this, I'm going to need some decent suggestions. I talked briefly with Kim, but we couldn't immediately find a name that both liked. Suggestions? > > I haven?t come up with anything better than we discussed. > Then the current proposal is to keep this as GrowableArrayView. It's also an easy fix to deal with as a follow-up RFE. >>> src/hotspot/share/utilities/growableArray.hpp >>> Destructors for all but the most-derived classes should be non-public. >>> Probably also the constructors. That is, all of GrowableArrayBase, >>> GrowableArrayView, and GrowableArrayWithAllocator should have >>> protected constructors and destructors. >> >> OK. For GrowableArrayivew, I had an idea that someone *might* want to do: >> >> int _array[10]; >> ... >> GrowableArrayView view(array, 10, 10); >> do_something(view); >> >> but I'll change this to protected. > > Classes that are used as both a base class and a concrete class can cause problems. > For example, allowing accidental slicing, which leads to UB (which may or may not > be detected in some way). Noted. > >>> src/hotspot/share/utilities/growableArray.hpp >>> [pre-existing] >>> GrowableArray's use of mtNone to indicate resource allocation seems >>> like an abuse of that memory type. That memory type has a description >>> string of "Unknown", nothing to do with resources. But it's considered >>> an invalid type by MamTracker/MemReporter/MallocSite, and it seems to >>> mean something else in the VirtualMemoryTracker, like "uninitialized". >> >> I thinks we view this differently. My view is that memflags belong to the CHeap allocations, but is only valid if != mtNone. Setting it to mtNone doesn't imply that resource allocations are marked with mtNone. It just means that we shouldn't care about the memflags value when allocating resource memory. > > I?m OK with an interpretation that mtNone is considered an invalid memflags everywhere. > There may be some places that ought to be treating it that way but aren?t. The aarch64 pf > function is one; you filed a bug and have a fix for that one - thanks. For the record: https://bugs.openjdk.java.net/browse/JDK-8248130 I haven't created a cleanup task for VirtualMemoryTracker oddity. I might get to it later. > >>> But why have constructors with optional MEMFLAGS at all? And why use >>> mtNone that way? Constructors without MEMFLAGS parameter are resource >>> allocating, constructors with required MEMFLAGS parameter are CHeap >>> allocating. (Though the random status of mtNone noted above will make >>> that one probably not actually usable.) (That would require some small >>> adjustments to GrowableArrayMetadata.) >> >> I think you've lost me here. Are you proposing that I split this up into more constructors. Are you suggesting that I change this: >> >> [?] >> and adjust allocate(...) and GrowableArrayMetadata(...) accordingly? > > Not relevant is mtNone signals ?invalid?. OK. > >>> src/hotspot/share/utilities/growableArray.hpp >>> 586 public: >>> 587 // Where are we going to allocate memory? >>> 588 bool on_C_heap() { return _metadata.on_C_heap(); } >>> 589 bool on_stack () { return _metadata.on_stack(); } >>> 590 bool on_arena () { return _metadata.on_arena(); } >>> These were previously protected; why make them public? >> >> Mainly to be usable by the tests. I can add a friend test class and revert this. > > OK. I think I?d prefer these not be in the public API; I wouldn?t want someone > writing code that was conditional on any of them? Your call. I changed them to be private. StefanK > > From stefan.karlsson at oracle.com Tue Jun 23 11:41:57 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 23 Jun 2020 13:41:57 +0200 Subject: 8248130: AArch64: pf() uses mtNone instead of mtInternal In-Reply-To: <342da471-7a09-2ceb-04ab-894c25ae3be2@oracle.com> References: <1cd70fc6-df03-7a3e-f35b-2cd060a55547@oracle.com> <342da471-7a09-2ceb-04ab-894c25ae3be2@oracle.com> Message-ID: <1420a377-58af-78f6-31f1-2b32afe90f1b@oracle.com> Thanks, Thomas. StefanK On 2020-06-23 10:42, Thomas Schatzl wrote: > Hi, > > On 23.06.20 09:14, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to change the NMT flag used by one of >> the debugging functions on AArch64. >> >> https://cr.openjdk.java.net/~stefank/8248130/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8248130 >> >> If pf() is used when NMT is turned on we hit this assert: >> MallocSite* MallocSiteTable::lookup_or_add(const NativeCallStack& key, >> size_t* bucket_idx, >> ?? size_t* pos_idx, MEMFLAGS flags) { >> ?? assert(flags != mtNone, "Should have a real memory type"); >> >> The patch simply changes the used flag to mtInternal. >> > > ? looks good and trivial. > > Thomas From leo.korinth at oracle.com Tue Jun 23 13:06:38 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 23 Jun 2020 15:06:38 +0200 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <7DE4D1D2-ECA8-4980-B665-781BB0E36B2F@oracle.com> References: <4611e386-2d3f-efaa-fdce-920c7d3d5b85@oracle.com> <7DE4D1D2-ECA8-4980-B665-781BB0E36B2F@oracle.com> Message-ID: <1e6280a0-0fc1-d035-9050-81a47a99d84f@oracle.com> On 14/06/2020 10:51, Kim Barrett wrote: > > I think having a class for atomic members and variables is good, using > the compiler to help keep us honest about what we're doing. So I > generally like this change, despite the number of comments below. > > First a few general comments, then a bunch of detailed comments on > specific changes. > > ------------------------------------------------------------------------------ > > I kind of wish this had been broken up into smaller chunks. Looking > at a large number of changes involving atomic operations all in one go > is pretty tiring. > I could have broken it down in smaller chunks, sorry. I wanted to show it as a whole. > ------------------------------------------------------------------------------ > > I hope that this will not be a G1-specific usage, nor even a > GC-specific usage, but rather will eventually be used throughout > HotSpot. If not, then I'm not sure this change should be made; I > wouldn't like having the differing styles of volatile members in some > code and AtomicValue in other code (other than as a temporary state as > changes are incrementally made). (That doesn't mean that I think all > atomic operations should involve AtomicValue, and the static Atomic > functions are no longer needed. I think there are some places where > forcing things into the former mold is problematic and the Atomic > functions will still have direct uses.) I have a change waiting for zgc as well. I want feedback on this review before sending that change out. I am planning to fix this for parallel and serial as well, but I do not plan to do anything outside the "gc" folder. > > ------------------------------------------------------------------------------ > > The name of the class has generated some discussion. (Full disclosure: > I suggested AtomicValue.) "AtomicValue" follows the style guide of > using CamelCase for type names. But there are lots of > counter-examples. "Atomic" is a bit strange, since then we'd have a > class that is both a thin wrapper with member functions and an > AllStatic with similarly named static member functions taking an > additional explicit argument. The alternative would be to give this > class the "Atomic" name and rename the existing "Atomic", but that > would be pretty disruptive. atomic is short and aligns nicely with > Atomic, but names that differ only in case may be considered a > problem. Brevity doesn't seem super critical here; the type name is > mostly used as the type for data members and global variables, and > rarely appears in function signatures. The name is okay with me, I am also open for another name (it is hard to get something everyone likes). > ------------------------------------------------------------------------------ > > There are a number of places where plain arithmetic was being > performed on relevant variables (mostly pre/post increment/decrement). > I think most of these were intentionally not using Atomic arithmetic, > because they are in single-threaded (mutex or safepoint) contexts. > > This change currently transforms those all into atomic inc/add/&etc. > I've been waffling about this, and have noted occurrences in the > detailed comments below. There is a (probably often negligable, though > maybe not always) performance cost to that transformation, but that's > not what concerns me. When I see an atomic operation, I expect there's > something interesting happening and ordering or atomicity is > important. Using atomic operations unnecessarily breaks that > expectation, making the code less clear. That doesn't mean we should > use pre/post increment/decrement operators and such for "non-atomic" > uses though; we could have other names that are more explicit. (Note > that atomic arithmetic operations with relaxed memory order are not > the same thing as non-atomic arithmetic operations.) Its a trade-off, when you see the atomic operations you might (wrongly) believe the atomicity being important, but if you do not see the atomic operations you get the urge to prove that the code is only executed in serial phases. If speed is a concern you can always load it into a local variable, do calculations and store it back in the AtomicValue. > > Also note that during review I found one place where I think the use > of non-atomic arithmetic may have been a mistake, in which case the > change to atomic arithmetic would be correct and necessary. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 42 #include "utilities/debug.hpp" > > Out of order. Will fix. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 434 template > 435 struct Atomic::StoreImpl< > 436 D, T, > 437 PlatformOp, > 438 typename EnableIf<(IsIntegral::value || IsRegisteredEnum::value) && (IsIntegral::value || IsRegisteredEnum::value)>::type> > > ... > 442 D value = new_value; > > Allowing implicit conversions here was discussed and rejected during > the templatization of Atomic. While I think allowing some conversions > here would be good, I think this is much too loose. I think a change > of this type should be dealt with separately. > > Also, the line length for the EnableIf line seems rather excessive to me. I will revert this change, and add casts/1u etc to make everything compile. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 943 AtomicValue(const AtomicValue& a); // = delete > > I think copy-assign should also be forbidden. Why not use NONCOPYABLE? Will fix. > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 946 STATIC_ASSERT(sizeof(AtomicValue) == sizeof(D)); // ensure the same memory layout > > I think there are more requirements than same-size. > But maybe they fall out of usage, since the Atomic operations will > provide the additional requirements. If you have a suggestion, I will look at it. > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 949 inline void inc(atomic_memory_order order=memory_order_conservative) { > > I think there should be whitespace around "=". Will fix. > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 940 class AtomicValue { > > I think I'd like the order of the load/store operations rearranged: > > load, load_acquire > store, release_store, release_store_fence > > I also think I'd like inc/dec near add/sub.2 Will try that. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 1007 template > 1008 static AtomicValue* cast_to_atomic_ptr(T* nonatomic); > > Declared but not defined. Refactoring artifact, will remove, thanks for finding it. > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 1011 template > 1012 static AtomicValue* cast_to_atomic_ptr(T* nonatomic) { > > I would really like to not have this. I'd like to see how far we can > get without it. I try to use it as little as possible. > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 940 class AtomicValue { > > Should it be default constructible? If so, should the default > constructor guarantee value initialization? I think having a trivial > default constructor is needed to avoid UB in some uses. See discussion > of G1BOT's _offset_array below. I prefer not having it default constructable if I do not have to. Better to force the user to set the value IMO, but if it creates UB I will have to think again > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/atomic.hpp > 957 inline D load() const { > > 961 template > 962 inline void store(T store_value) { > > I wonder if load_relaxed and relaxed_store or something like that > might be better names? I am convinced that following the names of the AllStatic Atomic is best. I have no opinion about that name though, but if that is to be changed, it is better to do it as a separate change IMO. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1BlockOffsetTable.cpp > 45 _offset_array = cast_to_atomic_ptr((u_char*) bot_reserved.start()); > > This is currently the only use of cast_to_atomic_ptr. It leads to UB > since the (non-trivial) constructor for AtomicValue is never called > for any of the objects in the array. I think the call to > memset_with_concurrent_readers is also UB. I *think* having a trivial > default constructor solves the UB problem, but that isn't possible > until C++11. I will have to look further into this... > > Rather than just casting to a pointer to AtomicValue<>, this could > iterate over the array memory, doing placement new construction of the > elements. If we gave AtomicValue<> a non-trivial but empty default > constructor (so not initializing the value), then such an iteration > should be optimized away. We'd still have the UB from memset_with_cr, > though that could use an iteration with the one-arg constructor instead. > Or ignore the technical UB and just keep using memset_with_cr. > > And that gets rid of the one use (so far) of cast_to_atomic_ptr. But it is useful to have until all code is fixed, and fixing all code at the same time is hard. I do not want to have UB, so I have to look further into this > > The problem, of course, is that having a trivial/empty default > constructor means one could unintentionally construct without > initialization. I think a private trivial default constructor would > deal with that, though again that requires C++11. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CodeCacheRemSet.cpp > 39 AtomicValue G1CodeRootSetTable::_purge_list(NULL); > > Extra space between type and name. Will fix. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 2023 _old_marking_cycles_started.load() == _old_marking_cycles_completed.load() + 1, > > Mis-indented. Will fix. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 2027 _old_marking_cycles_started.inc(); > > This was _old_marking_cycles_started++. The membars introduced by > using inc() aren't needed here. It is sufficient here to use > _old_marking_cycles_started.store(_old_marking_cycles_started.load() + 1) > This is the only write, and it's in the vmthread in a safepoint. > > Similarly in increment_old_marking_cycles_completed, except here we > have the only write serialized by holding G1OldGCCount_lock. > > I also think _old_marking_cycles_started might not need any special > handling at all. > > The additional membars don't really matter for performance, since > these operations are very rare. But they catch they eye and make one > wonder what is being ordered here and why. Maybe some commentary > would suffice? I kind of feel the opposite. If I see an atomically incremented variable, it feels fine (although maybe not as fast as possible), if I see a store(load() + 1) it feels unsafe and that it could be a potential bug, and I must check that the code is only used in serial contexts. Not sure what to prefer here. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 4677 void G1CollectedHeap::increase_used(size_t bytes) { > 4681 void G1CollectedHeap::decrease_used(size_t bytes) { > > I think these functions are mutexed, so the additional membars from > using AtomicValue<>::add/sub aren't needed. Do you think there is a performance problem? Otherwise I would like to keep all operations atomic. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectionSet.cpp > 129 _collection_set_regions[_collection_set_cur_length.fetch_and_add((size_t)1)] = hr->hrm_index(); > > I think unnecessary membars. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectionSet.cpp > 191 size_t len = _collection_set_cur_length.load(); > 192 OrderAccess::loadload(); > > This could just be a load_acquire(). > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1CollectionSet.cpp > 332 _collection_set_cur_length.inc(); > > I think unnecessary membars. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 179 _chunks_in_chunk_list.inc(); > 199 _chunks_in_chunk_list.dec(); > > I think unnecessary membars. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1ConcurrentMark.hpp > 466 AtomicValue* _top_at_rebuild_starts; > > [pre-existing] > Not sure, but I think this doesn't need to be atomic / didn't need to > have a volatile qualifier in there. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1FullGCAdjustTask.hpp > 42 AtomicValue _references_done; // Atomic counter / bool > > Member name no longer aligned with others adjacent. > Will fix. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1ParallelCleaning.hpp > 32 AtomicValue _cleaning_claimed; > > [pre-existing] > This looks like it ought to be bool rather than int, and change values > from 1/0 to true/false. > I rather fix this as a separate issue: https://bugs.openjdk.java.net/browse/JDK-8248149 >------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1RegionsOnNodes.cpp > 44 _count_per_node[node_index].inc(); > > Previously not atomic; was that a bug? If not a bug, then not clear > why these array elements were volatile. > I do not know. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1SurvivorRegions.cpp > 67 _used_bytes.add(used_bytes); > > I think unnecessary membars. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionManager.cpp > 663 for (uint i = 0; i < _n_regions; ++i) { > 664 new (&_claims[i]) AtomicValue(Unclaimed); > 665 } > > Even if AtomicValue has a trivial default destructor (as discussed > above) to avoid UB when using memset, this change is an improvement. > The old memset code only works because Unclaimed == 0. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionRemSet.cpp > 54 fl = _free_list.load(); > > `fl = res;` instead? > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionType.hpp > 94 AtomicValue _tag; > > It's not immediately obvious why _tag needs to be atomic. Unless I've > missed something, there don't appear to be anything other than relaxed > load/store operations. It's not even obvious that those might occur > concurrently. > > ------------------------------------------------------------------------------ > As a general reply to the questions about unnecessary membars and atomicity: I think it is good to just change everything to use atomics even if it is too conservative. I believe this change to be quite big (as you noted), and you thought I should break it down into smaller parts. Why not simply do the changes, most of which are quite conservative, let me take a look at the UB issues that you found, and then later after this change is committed anyone can remove atomicity if it bothers them? For me it is not obvious when I can use volatile (therefore this change). I know it is used to force the compiler not to optimise away load and stores, but whether I can --- portably --- rely on (even) load and stores not to word tear is beyond my knowledge. Therefore, I think it is better to be conservative, and fix performance problems if they appear. I did not notice any visible performance degradations. Also, if we do smaller optimization changes from something conservative it is easier to revert them, rather than to revert this big change or to do small fixes on it. Thanks, Leo From kim.barrett at oracle.com Tue Jun 23 13:42:04 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Jun 2020 09:42:04 -0400 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <3aae3413-b0cd-bf28-c931-031d24fbf28b@oracle.com> References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> <3aae3413-b0cd-bf28-c931-031d24fbf28b@oracle.com> Message-ID: <9A2932A2-FB76-44C6-89FD-D490A66747EC@oracle.com> > On Jun 23, 2020, at 6:23 AM, Stefan Karlsson wrote: > > Latest patches: > https://cr.openjdk.java.net/~stefank/8247755/webrev.02.delta/ > https://cr.openjdk.java.net/~stefank/8247755/webrev.02/ > > I haven't changed: > - GrowableArrayView to something else > - S to ALLOC or Derived. I'm leaning towards Derived. Lacking a really better alternative to offer, I?ll drop the objection to GrowableArrayView. I?m okay with Derived, obviously. webrev.02 doesn't seem to have fully addressed this comment from my first review message: src/hotspot/share/utilities/growableArray.hpp ----- Destructors for all but the most-derived classes should be non-public. Probably also the constructors. That is, all of GrowableArrayBase, GrowableArrayView, and GrowableArrayWithAllocator should have protected constructors and destructors. ----- The constructors are now protected, but these all have default (public) destructors, and it's the destructor that generally prevents inadvertent slicing, especially since we have defaults for copy and assign; if either of those get used, the lack of an accessible destructor will generally trigger a compile-time error. From christian.hagedorn at oracle.com Tue Jun 23 13:58:19 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Tue, 23 Jun 2020 15:58:19 +0200 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache Message-ID: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> Hi Please review the following patch: https://bugs.openjdk.java.net/browse/JDK-8245128 http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ The assert is often hit when enabling the Kitchensink instrumentation module which triggers a lot of class redefinitions. The problem looks similar to the one fixed in JDK-8225681 [1] for the other a) MT-unsafe assert. We could be dealing with an old method which we should also exclude in the second b) MT-unsafe assert (JDK-8225681 fixed it only for a)). A nice description of the problem is found in the comment [2] by Erik ?. Applying this fix, the assert is not hit anymore with repeated Kitchensink runs with the instrumentation module enabled. Thank you! Best regards, Christian [1] https://bugs.openjdk.java.net/browse/JDK-8225681 [2] https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 From stefan.karlsson at oracle.com Tue Jun 23 14:07:31 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 23 Jun 2020 16:07:31 +0200 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <9A2932A2-FB76-44C6-89FD-D490A66747EC@oracle.com> References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> <3aae3413-b0cd-bf28-c931-031d24fbf28b@oracle.com> <9A2932A2-FB76-44C6-89FD-D490A66747EC@oracle.com> Message-ID: This hopefully addresses all your items below: https://cr.openjdk.java.net/~stefank/8247755/webrev.03.delta/ https://cr.openjdk.java.net/~stefank/8247755/webrev.03/ Thanks, StefanK On 2020-06-23 15:42, Kim Barrett wrote: >> On Jun 23, 2020, at 6:23 AM, Stefan Karlsson wrote: >> >> Latest patches: >> https://cr.openjdk.java.net/~stefank/8247755/webrev.02.delta/ >> https://cr.openjdk.java.net/~stefank/8247755/webrev.02/ >> >> I haven't changed: >> - GrowableArrayView to something else >> - S to ALLOC or Derived. I'm leaning towards Derived. > > Lacking a really better alternative to offer, I?ll drop the objection to GrowableArrayView. > > I?m okay with Derived, obviously. > > webrev.02 doesn't seem to have fully addressed this comment from my > first review message: > > src/hotspot/share/utilities/growableArray.hpp > > ----- > Destructors for all but the most-derived classes should be non-public. > Probably also the constructors. That is, all of GrowableArrayBase, > GrowableArrayView, and GrowableArrayWithAllocator should have > protected constructors and destructors. > ----- > > The constructors are now protected, but these all have default > (public) destructors, and it's the destructor that generally prevents > inadvertent slicing, especially since we have defaults for copy and > assign; if either of those get used, the lack of an accessible destructor > will generally trigger a compile-time error. > From kim.barrett at oracle.com Tue Jun 23 14:31:24 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Jun 2020 10:31:24 -0400 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> <3aae3413-b0cd-bf28-c931-031d24fbf28b@oracle.com> <9A2932A2-FB76-44C6-89FD-D490A66747EC@oracle.com> Message-ID: > On Jun 23, 2020, at 10:07 AM, Stefan Karlsson wrote: > > This hopefully addresses all your items below: > https://cr.openjdk.java.net/~stefank/8247755/webrev.03.delta/ > https://cr.openjdk.java.net/~stefank/8247755/webrev.03/ This change caused me to notice a different problem. ~GrowableArrayCHeap should call clear_and_deallocate(). Also, I previously missed that allocate and deallocate are public in GrowableArray &etc. This is presumably so they can be called by the CRTP'ed base class. Rather than exposing implementation details in the public API, it's common practice with CRTP for the derived class to befriend the base class. Or there are more complicated idioms. https://stackoverflow.com/questions/8523762/crtp-with-protected-derived-member https://accu.org/index.php/journals/296 From stefan.karlsson at oracle.com Tue Jun 23 14:47:16 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 23 Jun 2020 16:47:16 +0200 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> <3aae3413-b0cd-bf28-c931-031d24fbf28b@oracle.com> <9A2932A2-FB76-44C6-89FD-D490A66747EC@oracle.com> Message-ID: <5992e183-1602-abca-385f-05fb53c9cc77@oracle.com> Thanks for catching this! Updated webrevs: https://cr.openjdk.java.net/~stefank/8247755/webrev.04.delta https://cr.openjdk.java.net/~stefank/8247755/webrev.04 StefanK On 2020-06-23 16:31, Kim Barrett wrote: >> On Jun 23, 2020, at 10:07 AM, Stefan Karlsson wrote: >> >> This hopefully addresses all your items below: >> https://cr.openjdk.java.net/~stefank/8247755/webrev.03.delta/ >> https://cr.openjdk.java.net/~stefank/8247755/webrev.03/ > > This change caused me to notice a different problem. > ~GrowableArrayCHeap should call clear_and_deallocate(). > > Also, I previously missed that allocate and deallocate are public in > GrowableArray &etc. This is presumably so they can be called by the > CRTP'ed base class. Rather than exposing implementation details in the > public API, it's common practice with CRTP for the derived class to > befriend the base class. Or there are more complicated idioms. > > https://stackoverflow.com/questions/8523762/crtp-with-protected-derived-member > https://accu.org/index.php/journals/296 > > From erik.osterlund at oracle.com Tue Jun 23 15:09:55 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 23 Jun 2020 17:09:55 +0200 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> Message-ID: Hi Christian, Looks good. Thanks, /Erik On 2020-06-23 15:58, Christian Hagedorn wrote: > Hi > > Please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8245128 > http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ > > The assert is often hit when enabling the Kitchensink instrumentation > module which triggers a lot of class redefinitions. The problem looks > similar to the one fixed in JDK-8225681 [1] for the other a) MT-unsafe > assert. We could be dealing with an old method which we should also > exclude in the second b) MT-unsafe assert (JDK-8225681 fixed it only > for a)). A nice description of the problem is found in the comment [2] > by Erik ?. > > Applying this fix, the assert is not hit anymore with repeated > Kitchensink runs with the instrumentation module enabled. > > Thank you! > > Best regards, > Christian > > > [1] https://bugs.openjdk.java.net/browse/JDK-8225681 > [2] > https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 From kim.barrett at oracle.com Tue Jun 23 15:10:47 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Jun 2020 11:10:47 -0400 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <5992e183-1602-abca-385f-05fb53c9cc77@oracle.com> References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> <3aae3413-b0cd-bf28-c931-031d24fbf28b@oracle.com> <9A2932A2-FB76-44C6-89FD-D490A66747EC@oracle.com> <5992e183-1602-abca-385f-05fb53c9cc77@oracle.com> Message-ID: <175DC7D2-CFB2-4FFA-A637-8FDA6D52EBBC@oracle.com> > On Jun 23, 2020, at 10:47 AM, Stefan Karlsson wrote: > > Thanks for catching this! Updated webrevs: > https://cr.openjdk.java.net/~stefank/8247755/webrev.04.delta > https://cr.openjdk.java.net/~stefank/8247755/webrev.04 Looks good. > > StefanK > > On 2020-06-23 16:31, Kim Barrett wrote: >>> On Jun 23, 2020, at 10:07 AM, Stefan Karlsson wrote: >>> >>> This hopefully addresses all your items below: >>> https://cr.openjdk.java.net/~stefank/8247755/webrev.03.delta/ >>> https://cr.openjdk.java.net/~stefank/8247755/webrev.03/ >> This change caused me to notice a different problem. >> ~GrowableArrayCHeap should call clear_and_deallocate(). >> Also, I previously missed that allocate and deallocate are public in >> GrowableArray &etc. This is presumably so they can be called by the >> CRTP'ed base class. Rather than exposing implementation details in the >> public API, it's common practice with CRTP for the derived class to >> befriend the base class. Or there are more complicated idioms. >> https://stackoverflow.com/questions/8523762/crtp-with-protected-derived-member >> https://accu.org/index.php/journals/296 From christian.hagedorn at oracle.com Tue Jun 23 15:16:49 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Tue, 23 Jun 2020 17:16:49 +0200 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> Message-ID: Hi Erik Thank you for your review! Best regards, Christian On 23.06.20 17:09, Erik ?sterlund wrote: > Hi Christian, > > Looks good. > > Thanks, > /Erik > > On 2020-06-23 15:58, Christian Hagedorn wrote: >> Hi >> >> Please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8245128 >> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >> >> The assert is often hit when enabling the Kitchensink instrumentation >> module which triggers a lot of class redefinitions. The problem looks >> similar to the one fixed in JDK-8225681 [1] for the other a) MT-unsafe >> assert. We could be dealing with an old method which we should also >> exclude in the second b) MT-unsafe assert (JDK-8225681 fixed it only >> for a)). A nice description of the problem is found in the comment [2] >> by Erik ?. >> >> Applying this fix, the assert is not hit anymore with repeated >> Kitchensink runs with the instrumentation module enabled. >> >> Thank you! >> >> Best regards, >> Christian >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >> [2] >> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >> > From stefan.karlsson at oracle.com Tue Jun 23 15:21:19 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 23 Jun 2020 17:21:19 +0200 Subject: RFR: 8247755: Leaner and more versatile GrowableArray classes In-Reply-To: <175DC7D2-CFB2-4FFA-A637-8FDA6D52EBBC@oracle.com> References: <07D7E8EE-2DBB-4AB1-8741-DF066889F8FD@oracle.com> <86fa8801-e49d-432f-84b7-1a3908b4e43b@oracle.com> <3aae3413-b0cd-bf28-c931-031d24fbf28b@oracle.com> <9A2932A2-FB76-44C6-89FD-D490A66747EC@oracle.com> <5992e183-1602-abca-385f-05fb53c9cc77@oracle.com> <175DC7D2-CFB2-4FFA-A637-8FDA6D52EBBC@oracle.com> Message-ID: <543dfc6b-445c-70f1-4fda-efa3655669ff@oracle.com> Thanks, Kim. StefanK On 2020-06-23 17:10, Kim Barrett wrote: >> On Jun 23, 2020, at 10:47 AM, Stefan Karlsson wrote: >> >> Thanks for catching this! Updated webrevs: >> https://cr.openjdk.java.net/~stefank/8247755/webrev.04.delta >> https://cr.openjdk.java.net/~stefank/8247755/webrev.04 > > Looks good. > >> >> StefanK >> >> On 2020-06-23 16:31, Kim Barrett wrote: >>>> On Jun 23, 2020, at 10:07 AM, Stefan Karlsson wrote: >>>> >>>> This hopefully addresses all your items below: >>>> https://cr.openjdk.java.net/~stefank/8247755/webrev.03.delta/ >>>> https://cr.openjdk.java.net/~stefank/8247755/webrev.03/ >>> This change caused me to notice a different problem. >>> ~GrowableArrayCHeap should call clear_and_deallocate(). >>> Also, I previously missed that allocate and deallocate are public in >>> GrowableArray &etc. This is presumably so they can be called by the >>> CRTP'ed base class. Rather than exposing implementation details in the >>> public API, it's common practice with CRTP for the derived class to >>> befriend the base class. Or there are more complicated idioms. >>> https://stackoverflow.com/questions/8523762/crtp-with-protected-derived-member >>> https://accu.org/index.php/journals/296 > > From vladimir.kozlov at oracle.com Tue Jun 23 16:31:17 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 23 Jun 2020 09:31:17 -0700 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> Message-ID: <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> Good. Thanks, Vladimir On 6/23/20 6:58 AM, Christian Hagedorn wrote: > Hi > > Please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8245128 > http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ > > The assert is often hit when enabling the Kitchensink instrumentation module which triggers a lot of class > redefinitions. The problem looks similar to the one fixed in JDK-8225681 [1] for the other a) MT-unsafe assert. We could > be dealing with an old method which we should also exclude in the second b) MT-unsafe assert (JDK-8225681 fixed it only > for a)). A nice description of the problem is found in the comment [2] by Erik ?. > > Applying this fix, the assert is not hit anymore with repeated Kitchensink runs with the instrumentation module enabled. > > Thank you! > > Best regards, > Christian > > > [1] https://bugs.openjdk.java.net/browse/JDK-8225681 > [2] > https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 > From magnus.ihse.bursie at oracle.com Tue Jun 23 16:53:57 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 23 Jun 2020 18:53:57 +0200 Subject: RFR: JDK-8247573 gtest/GTestWrapper.java is not helpful if gtest framework is missing In-Reply-To: <0eac549c-bbbe-e8af-db34-85aa743b0c1b@oracle.com> References: <544a72bd-bb45-7772-78b6-86380abdc603@oracle.com> <0eac549c-bbbe-e8af-db34-85aa743b0c1b@oracle.com> Message-ID: <593647c6-86d3-deaf-0e1e-ae2ad983ca72@oracle.com> On 2020-06-23 17:05, Erik Joelsson wrote: > Looks good, but that was the worst way of posting a patch I've seen to > date. The mail you quoted looked awful, yes! :-( I tried a new way of formatting the mail so the patch and log should be fixed space. Apparently it failed horribly. Also, I realized that since the fix is technically in hotspot code, I should have a hotspot review on this as well. cc:ing them. Here is the original mail again, sans the formatting: ------- If you run make test TEST=jtreg:gtest/GTestWrapper.java but had not built using the gtest framework, you would just get a result like this: STDERR: java.lang.Error: TESTBUG: the library has not been found in /home/shade/trunks/jdk-jdk/build/linux-x86_64-server-fastdebug/images/test/hotspot/jtreg/native ??? at GTestWrapper.main(GTestWrapper.java:60) ??? at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ??? at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) ??? at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ??? at java.base/java.lang.reflect.Method.invoke(Method.java:564) ??? at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) ??? at java.base/java.lang.Thread.run(Thread.java:832) JavaTest Message: Test threw exception: java.lang.Error JavaTest Message: shutting down test This is not very helpful in analyzing the problem. I have added a helpful suggestion to the exception. Bug: https://bugs.openjdk.java.net/browse/JDK-8247573 Patch inline: diff --git a/test/hotspot/jtreg/gtest/GTestWrapper.java b/test/hotspot/jtreg/gtest/GTestWrapper.java --- a/test/hotspot/jtreg/gtest/GTestWrapper.java +++ b/test/hotspot/jtreg/gtest/GTestWrapper.java @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2016, 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -57,7 +57,7 @@ ????????????????????????????? .resolve(jvmVariantDir); ???????? } ???????? if (!path.toFile().exists()) { -??????????? throw new Error("TESTBUG: the library has not been found in " + nativePath); +??????????? throw new Error("TESTBUG: the library has not been found in " + nativePath + ". Did you forget to use --with-gtest to configure?"); ???????? } ???????? Path execPath = path.resolve("gtestLauncher" + (Platform.isWindows() ? ".exe" : "")); /Magnus > > /Erik > > On 2020-06-23 07:37, Magnus Ihse Bursie wrote: >> If you run |make test TEST=jtreg:gtest/GTestWrapper.java| but had not >> built using the gtest framework, you would just get a result like this: >> >> |STDERR: java.lang.Error: TESTBUG: the library has not been found in >> /home/shade/trunks/jdk-jdk/build/linux-x86_64-server-fastdebug/images/test/hotspot/jtreg/native >> at GTestWrapper.main(GTestWrapper.java:60) at >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) at >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) >> at >> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:564) at >> com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) >> at java.base/java.lang.Thread.run(Thread.java:832) JavaTest Message: >> Test threw exception: java.lang.Error JavaTest Message: shutting down >> test | >> >> This is not very helpful in analyzing the problem. I have added a >> helpful suggestion to the exception. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8247573 >> Patch inline: >> >> |diff --git a/test/hotspot/jtreg/gtest/GTestWrapper.java >> b/test/hotspot/jtreg/gtest/GTestWrapper.java --- >> a/test/hotspot/jtreg/gtest/GTestWrapper.java +++ >> b/test/hotspot/jtreg/gtest/GTestWrapper.java @@ -1,5 +1,5 @@ /* - * >> Copyright (c) 2016, 2019, Oracle and/or its affiliates. All rights >> reserved. + * Copyright (c) 2016, 2020, Oracle and/or its affiliates. >> All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR >> THIS FILE HEADER. * * This code is free software; you can >> redistribute it and/or modify it @@ -57,7 +57,7 @@ >> .resolve(jvmVariantDir); } if (!path.toFile().exists()) { - throw new >> Error("TESTBUG: the library has not been found in " + nativePath); + >> throw new Error("TESTBUG: the library has not been found in " + >> nativePath + ". Did you forget to use --with-gtest to configure?"); } >> Path execPath = path.resolve("gtestLauncher" + (Platform.isWindows() >> ? ".exe" : "")); | >> >> /Magnus >> >> ? From erik.joelsson at oracle.com Tue Jun 23 16:57:33 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 23 Jun 2020 09:57:33 -0700 Subject: RFR: JDK-8247573 gtest/GTestWrapper.java is not helpful if gtest framework is missing In-Reply-To: <593647c6-86d3-deaf-0e1e-ae2ad983ca72@oracle.com> References: <544a72bd-bb45-7772-78b6-86380abdc603@oracle.com> <0eac549c-bbbe-e8af-db34-85aa743b0c1b@oracle.com> <593647c6-86d3-deaf-0e1e-ae2ad983ca72@oracle.com> Message-ID: <13247367-8fa3-2058-d21e-ac7d55a5fa3f@oracle.com> That looks much better! :) /Erik On 2020-06-23 09:53, Magnus Ihse Bursie wrote: > On 2020-06-23 17:05, Erik Joelsson wrote: >> Looks good, but that was the worst way of posting a patch I've seen >> to date. > The mail you quoted looked awful, yes! :-( I tried a new way of > formatting the mail so the patch and log should be fixed space. > Apparently it failed horribly. > > Also, I realized that since the fix is technically in hotspot code, I > should have a hotspot review on this as well. cc:ing them. > > Here is the original mail again, sans the formatting: > ------- > > If you run make test TEST=jtreg:gtest/GTestWrapper.java but had not > built using the gtest framework, you would just get a result like this: > > STDERR: > java.lang.Error: TESTBUG: the library has not been found in > /home/shade/trunks/jdk-jdk/build/linux-x86_64-server-fastdebug/images/test/hotspot/jtreg/native > ??? at GTestWrapper.main(GTestWrapper.java:60) > ??? at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > ??? at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) > ??? at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ??? at java.base/java.lang.reflect.Method.invoke(Method.java:564) > ??? at > com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) > ??? at java.base/java.lang.Thread.run(Thread.java:832) > > JavaTest Message: Test threw exception: java.lang.Error > JavaTest Message: shutting down test > > This is not very helpful in analyzing the problem. I have added a > helpful suggestion to the exception. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8247573 > Patch inline: > > diff --git a/test/hotspot/jtreg/gtest/GTestWrapper.java > b/test/hotspot/jtreg/gtest/GTestWrapper.java > --- a/test/hotspot/jtreg/gtest/GTestWrapper.java > +++ b/test/hotspot/jtreg/gtest/GTestWrapper.java > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2016, 2019, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -57,7 +57,7 @@ > ????????????????????????????? .resolve(jvmVariantDir); > ???????? } > ???????? if (!path.toFile().exists()) { > -??????????? throw new Error("TESTBUG: the library has not been found > in " + nativePath); > +??????????? throw new Error("TESTBUG: the library has not been found > in " + nativePath + ". Did you forget to use --with-gtest to > configure?"); > ???????? } > > ???????? Path execPath = path.resolve("gtestLauncher" + > (Platform.isWindows() ? ".exe" : "")); > > /Magnus >> >> /Erik >> >> On 2020-06-23 07:37, Magnus Ihse Bursie wrote: >>> If you run |make test TEST=jtreg:gtest/GTestWrapper.java| but had >>> not built using the gtest framework, you would just get a result >>> like this: >>> >>> |STDERR: java.lang.Error: TESTBUG: the library has not been found in >>> /home/shade/trunks/jdk-jdk/build/linux-x86_64-server-fastdebug/images/test/hotspot/jtreg/native >>> at GTestWrapper.main(GTestWrapper.java:60) at >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >>> Method) at >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) >>> at >>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.base/java.lang.reflect.Method.invoke(Method.java:564) at >>> com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) >>> at java.base/java.lang.Thread.run(Thread.java:832) JavaTest Message: >>> Test threw exception: java.lang.Error JavaTest Message: shutting >>> down test | >>> >>> This is not very helpful in analyzing the problem. I have added a >>> helpful suggestion to the exception. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8247573 >>> Patch inline: >>> >>> |diff --git a/test/hotspot/jtreg/gtest/GTestWrapper.java >>> b/test/hotspot/jtreg/gtest/GTestWrapper.java --- >>> a/test/hotspot/jtreg/gtest/GTestWrapper.java +++ >>> b/test/hotspot/jtreg/gtest/GTestWrapper.java @@ -1,5 +1,5 @@ /* - * >>> Copyright (c) 2016, 2019, Oracle and/or its affiliates. All rights >>> reserved. + * Copyright (c) 2016, 2020, Oracle and/or its >>> affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT >>> NOTICES OR THIS FILE HEADER. * * This code is free software; you can >>> redistribute it and/or modify it @@ -57,7 +57,7 @@ >>> .resolve(jvmVariantDir); } if (!path.toFile().exists()) { - throw >>> new Error("TESTBUG: the library has not been found in " + >>> nativePath); + throw new Error("TESTBUG: the library has not been >>> found in " + nativePath + ". Did you forget to use --with-gtest to >>> configure?"); } Path execPath = path.resolve("gtestLauncher" + >>> (Platform.isWindows() ? ".exe" : "")); | >>> >>> /Magnus >>> >>> ? > From igor.ignatyev at oracle.com Tue Jun 23 19:15:39 2020 From: igor.ignatyev at oracle.com (igor.ignatyev at oracle.com) Date: Tue, 23 Jun 2020 12:15:39 -0700 Subject: RFR: JDK-8247573 gtest/GTestWrapper.java is not helpful if gtest framework is missing In-Reply-To: <13247367-8fa3-2058-d21e-ac7d55a5fa3f@oracle.com> References: <13247367-8fa3-2058-d21e-ac7d55a5fa3f@oracle.com> Message-ID: <582683BB-948F-42D0-A564-3C6A96C14ADD@oracle.com> LGTM ? Igor > On Jun 23, 2020, at 11:13 AM, Erik Joelsson wrote: > > ?That looks much better! :) > > /Erik > >> On 2020-06-23 09:53, Magnus Ihse Bursie wrote: >>> On 2020-06-23 17:05, Erik Joelsson wrote: >>> Looks good, but that was the worst way of posting a patch I've seen to date. >> The mail you quoted looked awful, yes! :-( I tried a new way of formatting the mail so the patch and log should be fixed space. Apparently it failed horribly. >> >> Also, I realized that since the fix is technically in hotspot code, I should have a hotspot review on this as well. cc:ing them. >> >> Here is the original mail again, sans the formatting: >> ------- >> >> If you run make test TEST=jtreg:gtest/GTestWrapper.java but had not built using the gtest framework, you would just get a result like this: >> >> STDERR: >> java.lang.Error: TESTBUG: the library has not been found in /home/shade/trunks/jdk-jdk/build/linux-x86_64-server-fastdebug/images/test/hotspot/jtreg/native >> at GTestWrapper.main(GTestWrapper.java:60) >> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) >> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:564) >> at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) >> at java.base/java.lang.Thread.run(Thread.java:832) >> >> JavaTest Message: Test threw exception: java.lang.Error >> JavaTest Message: shutting down test >> >> This is not very helpful in analyzing the problem. I have added a helpful suggestion to the exception. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8247573 >> Patch inline: >> >> diff --git a/test/hotspot/jtreg/gtest/GTestWrapper.java b/test/hotspot/jtreg/gtest/GTestWrapper.java >> --- a/test/hotspot/jtreg/gtest/GTestWrapper.java >> +++ b/test/hotspot/jtreg/gtest/GTestWrapper.java >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 2016, 2019, Oracle and/or its affiliates. All rights reserved. >> + * Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> @@ -57,7 +57,7 @@ >> .resolve(jvmVariantDir); >> } >> if (!path.toFile().exists()) { >> - throw new Error("TESTBUG: the library has not been found in " + nativePath); >> + throw new Error("TESTBUG: the library has not been found in " + nativePath + ". Did you forget to use --with-gtest to configure?"); >> } >> >> Path execPath = path.resolve("gtestLauncher" + (Platform.isWindows() ? ".exe" : "")); >> >> /Magnus >>> >>> /Erik >>> >>> On 2020-06-23 07:37, Magnus Ihse Bursie wrote: >>>> If you run |make test TEST=jtreg:gtest/GTestWrapper.java| but had not built using the gtest framework, you would just get a result like this: >>>> >>>> |STDERR: java.lang.Error: TESTBUG: the library has not been found in /home/shade/trunks/jdk-jdk/build/linux-x86_64-server-fastdebug/images/test/hotspot/jtreg/native at GTestWrapper.main(GTestWrapper.java:60) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298) at java.base/java.lang.Thread.run(Thread.java:832) JavaTest Message: Test threw exception: java.lang.Error JavaTest Message: shutting down test | >>>> >>>> This is not very helpful in analyzing the problem. I have added a helpful suggestion to the exception. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8247573 >>>> Patch inline: >>>> >>>> |diff --git a/test/hotspot/jtreg/gtest/GTestWrapper.java b/test/hotspot/jtreg/gtest/GTestWrapper.java --- a/test/hotspot/jtreg/gtest/GTestWrapper.java +++ b/test/hotspot/jtreg/gtest/GTestWrapper.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2016, 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -57,7 +57,7 @@ .resolve(jvmVariantDir); } if (!path.toFile().exists()) { - throw new Error("TESTBUG: the library has not been found in " + nativePath); + throw new Error("TESTBUG: the library has not been found in " + nativePath + ". Did you forget to use --with-gtest to configure?"); } Path execPath = path.resolve("gtestLauncher" + (Platform.isWindows() ? ".exe" : "")); | >>>> >>>> /Magnus >>>> >>>> ? >> From kim.barrett at oracle.com Tue Jun 23 19:32:59 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Jun 2020 15:32:59 -0400 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption Message-ID: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> Please review / comment on this proposed update to the HotSpot Style Guide. This is part of the work for JDK-8208089: JEP 347: Adopt C++14 Language Features in the JDK. This update includes additions for initial C++14 adoption. It has also been significantly refactored and modified from the current wiki text. There have long been complaints that what's on the wiki hasn't kept up with actual usage. I've also added more explanation and rationale for some parts, based on (sometimes repeated) discussions in various forums. (Note that I might not always entirely agree with some choices, but I've tried to fairly and accurately record what I think is the rationale. Feel free to offer corrections.) For convenience of review, especially for incremental updates based on feedback, I'm publishing the updated document as a webrev. Once reviewed, the finished text can be used to update the wiki, unless we decide it should be homed somewhere else. We don't currently have a process for making updates to this document (one reason for it falling out of date). This update includes a proposal for such a process, but that process may need approvals from elsewhere, and there's the whole bootstrapping question. We'll see how this goes; maybe there won't be any complaints and it's done. That's likely wishful thinking. To see the html-formatted version you'll probably need to download and view it locally; cr.openjdk.java.net doesn't seem to like serving it from the webrev. CR: https://bugs.openjdk.java.net/browse/JDK-8247976 Webrev: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.00/ From coleen.phillimore at oracle.com Tue Jun 23 20:04:46 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 23 Jun 2020 16:04:46 -0400 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> Message-ID: <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> I know less about this code than Erik and Vladimir but would you need to also test for this condition in the b) case? 754 !old_method->method_holder()->is_loader_alive() || You probably won't get as much racy behavior from class unloading though. Otherwise, the change makes sense. thanks, Coleen On 6/23/20 12:31 PM, Vladimir Kozlov wrote: > Good. > > Thanks, > Vladimir > > On 6/23/20 6:58 AM, Christian Hagedorn wrote: >> Hi >> >> Please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8245128 >> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >> >> The assert is often hit when enabling the Kitchensink instrumentation >> module which triggers a lot of class redefinitions. The problem looks >> similar to the one fixed in JDK-8225681 [1] for the other a) >> MT-unsafe assert. We could be dealing with an old method which we >> should also exclude in the second b) MT-unsafe assert (JDK-8225681 >> fixed it only for a)). A nice description of the problem is found in >> the comment [2] by Erik ?. >> >> Applying this fix, the assert is not hit anymore with repeated >> Kitchensink runs with the instrumentation module enabled. >> >> Thank you! >> >> Best regards, >> Christian >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >> [2] >> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >> From ioi.lam at oracle.com Tue Jun 23 20:10:31 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 23 Jun 2020 13:10:31 -0700 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary Message-ID: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8248170 http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are unnecessarily included in some popular headers. This introduces unnecessary dependencies and slows down compile time. slowdebug: total number of files compiled: 264606 -> 260588 = -1.5% total number of lines compiled: 76073364 -> 74477347 = -2.1% size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 bytes = -0.5% (I got the statistics by parsing the *.d files in the build directory). Thanks - Ioi From vladimir.kozlov at oracle.com Tue Jun 23 21:43:32 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 23 Jun 2020 14:43:32 -0700 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> Message-ID: Looks good. Did you test a build without PCH? Thanks, Vladimir On 6/23/20 1:10 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8248170 > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ > > {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are unnecessarily included in some popular headers. This > introduces unnecessary dependencies and slows down compile time. > > slowdebug: > total number of files compiled: 264606 -> 260588 = -1.5% > total number of lines compiled: 76073364 -> 74477347 = -2.1% > size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 bytes = -0.5% > > (I got the statistics by parsing the *.d files in the build directory). > > Thanks > - Ioi From ioi.lam at oracle.com Tue Jun 23 22:25:08 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 23 Jun 2020 15:25:08 -0700 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> Message-ID: <596cef58-9493-a3ee-867c-b9ca89154699@oracle.com> Yes, I test locally with PCH. I also did all 58 different builds in the mach5 CI, many of those are not using PCH. I even fixed a problem in shenandoah :-) Updated webrev in-place (the only change from earlier is in compile.cpp and shenandoahSupport.cpp). http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ Thanks - Ioi On 6/23/20 2:43 PM, Vladimir Kozlov wrote: > Looks good. Did you test a build without PCH? > > Thanks, > Vladimir > > On 6/23/20 1:10 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8248170 >> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ >> >> >> {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are >> unnecessarily included in some popular headers. This introduces >> unnecessary dependencies and slows down compile time. >> >> slowdebug: >> total number of files compiled: 264606 -> 260588 = -1.5% >> total number of lines compiled: 76073364 -> 74477347 = -2.1% >> size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 >> bytes = -0.5% >> >> (I got the statistics by parsing the *.d files in the build directory). >> >> Thanks >> - Ioi From vladimir.kozlov at oracle.com Tue Jun 23 22:58:16 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 23 Jun 2020 15:58:16 -0700 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <596cef58-9493-a3ee-867c-b9ca89154699@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> <596cef58-9493-a3ee-867c-b9ca89154699@oracle.com> Message-ID: <8023b1a1-f203-6783-0960-c09dea449a67@oracle.com> Good. Thanks, Vladimir On 6/23/20 3:25 PM, Ioi Lam wrote: > Yes, I test locally with PCH. I also did all 58 different builds in the mach5 CI, many of those are not using PCH. I > even fixed a problem in shenandoah :-) > > Updated webrev in-place (the only change from earlier is in compile.cpp and shenandoahSupport.cpp). > > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ > > Thanks > - Ioi > > On 6/23/20 2:43 PM, Vladimir Kozlov wrote: >> Looks good. Did you test a build without PCH? >> >> Thanks, >> Vladimir >> >> On 6/23/20 1:10 PM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8248170 >>> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ >>> >>> {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are unnecessarily included in some popular headers. >>> This introduces unnecessary dependencies and slows down compile time. >>> >>> slowdebug: >>> total number of files compiled: 264606 -> 260588 = -1.5% >>> total number of lines compiled: 76073364 -> 74477347 = -2.1% >>> size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 bytes = -0.5% >>> >>> (I got the statistics by parsing the *.d files in the build directory). >>> >>> Thanks >>> - Ioi > From john.r.rose at oracle.com Wed Jun 24 01:57:52 2020 From: john.r.rose at oracle.com (John Rose) Date: Tue, 23 Jun 2020 18:57:52 -0700 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> Message-ID: <95B6BF1F-CF49-4024-86C0-F697F4FD4822@oracle.com> On Jun 23, 2020, at 12:32 PM, Kim Barrett wrote: > > Please review / comment on this proposed update to the HotSpot Style > Guide. This is part of the work for JDK-8208089: JEP 347: Adopt C++14 > Language Features in the JDK. I read it all. It is an excellent and timely refresh of the existing document. (?Which I remember writing first drafts of many years ago!) The terms RAII and SFINAE need parenthetical definitions and/or links to their definitions. Though I?ve been doing C++ most of my life, those acronyms are not in my primary vocabulary, and I?m surely not the only one. ODR already has a parenthetical definition, so it?s good to go. I noted various deletions and additions, and honestly find nothing at all that bothers me. It feels like very careful work. I say ship it! (Crossing my fingers?) ? John From david.holmes at oracle.com Wed Jun 24 04:08:33 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Jun 2020 14:08:33 +1000 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> Message-ID: <6e6e3289-2c91-7019-36de-71548ab4d450@oracle.com> Hi Ioi, On 24/06/2020 6:10 am, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8248170 > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ > > > {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are > unnecessarily included in some popular headers. This introduces > unnecessary dependencies and slows down compile time. Based on your non-PCH local builds? I have no issue with removing unnecessary includes but I see far more added includes in your changes than I see removals, so I'm a little confused by that. I also have to wonder how on earth you concluded that these particular includes were the cause of your problem ?? The change seems harmless so I don't object. Thanks, David > slowdebug: > total number of files compiled: 264606 -> 260588 = -1.5% > total number of lines compiled: 76073364 -> 74477347 = -2.1% > size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 bytes > = -0.5% > > (I got the statistics by parsing the *.d files in the build directory). > > Thanks > - Ioi From ioi.lam at oracle.com Wed Jun 24 06:16:20 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 23 Jun 2020 23:16:20 -0700 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <6e6e3289-2c91-7019-36de-71548ab4d450@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> <6e6e3289-2c91-7019-36de-71548ab4d450@oracle.com> Message-ID: <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> On 6/23/20 9:08 PM, David Holmes wrote: > Hi Ioi, > > On 24/06/2020 6:10 am, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8248170 >> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ >> >> >> {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are >> unnecessarily included in some popular headers. This introduces >> unnecessary dependencies and slows down compile time. > > Based on your non-PCH local builds? > > I have no issue with removing unnecessary includes but I see far more > added includes in your changes than I see removals, so I'm a little > confused by that. I also have to wonder how on earth you concluded > that these particular includes were the cause of your problem ?? > I ran a script on my local non-PCH build and got the following output. I scan the file from the bottom to look for things to fix. http://cr.openjdk.java.net/~iklam/jdk16/misc/8248170-headers.txt The reasons for adding new includes: the fix has 2 parts (a) Remove unnecessary headers included by popular files such as thread.hpp (b) The above causes some sources files to break, because they were missing includes. These files could compile before my patch because the missing dependency was transitively included by thread.hpp, etc. Now I need to add the missing includes. E.g., os_windows.cpp has this 2342???? CodeBlob* cb = CodeCache::find_blob(pc); ... so it needs to include codeCache.hpp, but that was transitively included by vmOperations.hpp. ============= I updated the patch based on Coleen's off-line feedback: http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02/ http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02.delta/ Thanks - Ioi > The change seems harmless so I don't object. > > Thanks, > David > >> slowdebug: >> total number of files compiled: 264606 -> 260588 = -1.5% >> total number of lines compiled: 76073364 -> 74477347 = -2.1% >> size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 >> bytes = -0.5% >> >> (I got the statistics by parsing the *.d files in the build directory). >> >> Thanks >> - Ioi From erik.helin at oracle.com Wed Jun 24 06:29:05 2020 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 24 Jun 2020 08:29:05 +0200 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> Message-ID: On 6/23/20 9:32 PM, Kim Barrett wrote > For convenience of review, especially for incremental updates based on > feedback, I'm publishing the updated document as a webrev. Thanks, appreciated! > Once reviewed, the finished text can be used to update the wiki, unless we > decide it should be homed somewhere else. Should this document perhaps go into the OpenJDK guide [0] instead of being on the wiki [1]? Since the OpenJDK guide resides in a repository [2], that would allow for updates (and reviews/discussions) using our standard contribution processes. I personally find it easier to keep up with changes to the guide (by subscribing to guide-dev [3]) compared to following the wiki change history. Thanks, Erik [0]: https://openjdk.java.net/projects/guide/ [1]: https://wiki.openjdk.java.net/display/HotSpot/StyleGuide [2]: https://github.com/openjdk/guide [3]: https://mail.openjdk.java.net/mailman/listinfo/guide-dev From stefan.karlsson at oracle.com Wed Jun 24 07:24:56 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 24 Jun 2020 09:24:56 +0200 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> Message-ID: Hi Kim, Thanks for working on updating this document. I think this mostly looks good, but do have a few comments. * Macro names usually use upper-case with words separated by a single underscore. However, there are a significant number of lowercase macros. Adding to that set is discouraged. This is new line from the old style guide. Personally I find upper-case macros very noise and grabs a lot of the attention of the code. For most macros that's OK, but if a macro is prevalent in the source code, like assert, guarantee, log_info, offset_of, maybe even the recently discussed ENABLE_IF/enable_if, then I think it makes sense to *NOT* use upper-case names. * Use functions from globalDefinitions.hpp when performing bitwise operations on integers. Do not code directly as C operators, unless they are extremely simple. (Examples:|round_to|,|is_power_of_2|,|exact_log2|.) Over the years, we've seen problems with the intptr_t bit manipulation functions in globalDefinitions.hpp. We have been slowly fixing these problems and moving some of them away into other headers like align.hpp and powerOfTwo.hpp. The bit-manipulation functions left in globalDefinitions.hpp still use intptr_t and I don't think we should encourage people to using them before they have been rewritten, and fully tested with all integer types. round_to has been removed in favor of the align_up function. * Put a member function|FooBar::bang|into the same file that defined|FooBar|, or its associated *.cpp file. I don't think we should promote putting implementation in our header files. * Implement classes as if expecting rough usage by clients. Check for incorrect usage of a class using|assert(...)|,|guarantee(...)|,|ShouldNotReachHere()|and comments wherever needed. Performance is almost never a reason to omit asserts. I know that the Compiler team had a push to replace ShouldNotReachHere() with fatal("descriptive message"). It would be good to have a common guidance w.r.t. that. Could be done as a follow-up task. * When you must defactor to optimize, preserve as much structure as possible. If you must hand-inline some name, label the local copy with the original name. Is defactor a real word? Could this be rewritten to skip that word? * Use One-True-Brace-Style. The opening brace for a function or class is normally at the end of the line; it is sometimes moved to the beginning of the next line for emphasis. Substatements are enclosed in braces, even if there is only a single statement. Extremely simple one-line statements may drop braces around a substatement. Why not just skip the last sentence? People interpret extremely simple one-liners differently and this sentence just adds a loop-hole for those that like to write if-statements without braces. This could be considered an extremely simple one-liner: ? int append(const E& elem) { ??? if (this->_len == this->_max) grow(this->_len); ??? int idx = this->_len++; ??? this->_data[idx] = elem; ??? return idx; ? } but I don't think we should be writing code like this anymore. * Use extra parentheses in expressions whenever operator precedence seems doubtful. Always use parentheses in shift/mask expressions (|<<|,|&|,|||,|~|). Don?t add whitespace immediately inside parentheses. Isn't it a bit odd to mention ~ here? It precedes almost all operators, so either they are needed or they are not. There's not really a style choice. * Type names and global names should use mixed-case with the first letter of each word capitalized (|FooBar|). Embedded abbreviations in otherwise mixed-case names are usually capitalized entirely rather than being treated as a single word with only the initial letter capitalized, e.g. ?HTML? rather than ?Html?. Could this be split into two separate bullets? I think the mixed-case guide makes sense, but I'm not a fan when it's used dogmatically. I would much rather allow us to write atomic instead of the longer more noisier AtomicValue. And I know that this rule is used to argument for the latter. * Constant names may be upper-case or mixed-case, according to historical necessity. (Note: There are many examples of constants with lowercase names.) The "according to historical necessity" makes it unclear if there's a preferred way for new code. If there's no preferred way out of the two, maybe drop that part? * Use|bool|for boolean values. Do not use ints or pointers as (implicit) booleans with|&&|,||||,|if|,|while|. Instead, compare explicitly, i.e.|if (x != 0)|or|if (ptr != nullptr)|, etc. Do not use declarations in/condition/forms, i.e. don?t use|if (T v = value) { ... }|. The last part is a new addition to the style guide. We use it for some iterators: for (JavaThreadIteratorWithHandle jtiwh; JavaThread *thr = jtiwh.next(); ) { this is useful to limit the scope of the instance. I don't think we should discourage this in the style guide. - "Some features are discussed their own subsection" Missing an "in"? StefanK On 2020-06-23 21:32, Kim Barrett wrote: > Please review / comment on this proposed update to the HotSpot Style > Guide. This is part of the work for JDK-8208089: JEP 347: Adopt C++14 > Language Features in the JDK. > > This update includes additions for initial C++14 adoption. It has also > been significantly refactored and modified from the current wiki text. > There have long been complaints that what's on the wiki hasn't kept up > with actual usage. I've also added more explanation and rationale for > some parts, based on (sometimes repeated) discussions in various > forums. (Note that I might not always entirely agree with some > choices, but I've tried to fairly and accurately record what I think > is the rationale. Feel free to offer corrections.) > > For convenience of review, especially for incremental updates based on > feedback, I'm publishing the updated document as a webrev. Once > reviewed, the finished text can be used to update the wiki, unless we > decide it should be homed somewhere else. > > We don't currently have a process for making updates to this document > (one reason for it falling out of date). This update includes a > proposal for such a process, but that process may need approvals from > elsewhere, and there's the whole bootstrapping question. We'll see how > this goes; maybe there won't be any complaints and it's done. That's > likely wishful thinking. > > To see the html-formatted version you'll probably need to download and > view it locally; cr.openjdk.java.net doesn't seem to like serving it > from the webrev. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8247976 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.00/ > > From stefan.karlsson at oracle.com Wed Jun 24 07:35:33 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 24 Jun 2020 09:35:33 +0200 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> <6e6e3289-2c91-7019-36de-71548ab4d450@oracle.com> <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> Message-ID: Looks good (if it compiles)! StefanK On 2020-06-24 08:16, Ioi Lam wrote: > > > On 6/23/20 9:08 PM, David Holmes wrote: >> Hi Ioi, >> >> On 24/06/2020 6:10 am, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8248170 >>> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ >>> >>> >>> {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are >>> unnecessarily included in some popular headers. This introduces >>> unnecessary dependencies and slows down compile time. >> >> Based on your non-PCH local builds? >> >> I have no issue with removing unnecessary includes but I see far more >> added includes in your changes than I see removals, so I'm a little >> confused by that. I also have to wonder how on earth you concluded >> that these particular includes were the cause of your problem ?? >> > I ran a script on my local non-PCH build and got the following output. > I scan the file from the bottom to look for things to fix. > > http://cr.openjdk.java.net/~iklam/jdk16/misc/8248170-headers.txt > > The reasons for adding new includes: the fix has 2 parts > > (a) Remove unnecessary headers included by popular files such as > thread.hpp > (b) The above causes some sources files to break, because they were > missing includes. These files could compile before my patch because > the missing dependency was transitively included by thread.hpp, etc. > Now I need to add the missing includes. > > E.g., os_windows.cpp has this > > 2342???? CodeBlob* cb = CodeCache::find_blob(pc); > > ... so it needs to include codeCache.hpp, but that was transitively > included by vmOperations.hpp. > > ============= > > I updated the patch based on Coleen's off-line feedback: > > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02.delta/ > > > > Thanks > - Ioi > >> The change seems harmless so I don't object. >> >> Thanks, >> David >> >>> slowdebug: >>> total number of files compiled: 264606 -> 260588 = -1.5% >>> total number of lines compiled: 76073364 -> 74477347 = -2.1% >>> size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 >>> bytes = -0.5% >>> >>> (I got the statistics by parsing the *.d files in the build directory). >>> >>> Thanks >>> - Ioi > From david.holmes at oracle.com Wed Jun 24 09:41:15 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Jun 2020 19:41:15 +1000 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> <6e6e3289-2c91-7019-36de-71548ab4d450@oracle.com> <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> Message-ID: <3ac53f65-16d0-f0e5-c278-eb1742432bcb@oracle.com> On 24/06/2020 4:16 pm, Ioi Lam wrote: > > > On 6/23/20 9:08 PM, David Holmes wrote: >> Hi Ioi, >> >> On 24/06/2020 6:10 am, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8248170 >>> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ >>> >>> >>> {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are >>> unnecessarily included in some popular headers. This introduces >>> unnecessary dependencies and slows down compile time. >> >> Based on your non-PCH local builds? >> >> I have no issue with removing unnecessary includes but I see far more >> added includes in your changes than I see removals, so I'm a little >> confused by that. I also have to wonder how on earth you concluded >> that these particular includes were the cause of your problem ?? >> > I ran a script on my local non-PCH build and got the following output. I > scan the file from the bottom to look for things to fix. Interesting! Thanks for sharing. > http://cr.openjdk.java.net/~iklam/jdk16/misc/8248170-headers.txt > > The reasons for adding new includes: the fix has 2 parts > > (a) Remove unnecessary headers included by popular files such as thread.hpp > (b) The above causes some sources files to break, because they were > missing includes. These files could compile before my patch because the > missing dependency was transitively included by thread.hpp, etc. Now I > need to add the missing includes. > > E.g., os_windows.cpp has this > > 2342???? CodeBlob* cb = CodeCache::find_blob(pc); > > ... so it needs to include codeCache.hpp, but that was transitively > included by vmOperations.hpp. Okay. Just seemed odd to see more added than taken away. > ============= > > I updated the patch based on Coleen's off-line feedback: > > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02.delta/ Appears to me that c1_CodeStubs.hpp includes c1/c1_Runtime1.hpp, and each of the c1_CodeStubs_.cpp files also include it. That seems redundant. Thanks, David ----- > > > Thanks > - Ioi > >> The change seems harmless so I don't object. >> >> Thanks, >> David >> >>> slowdebug: >>> total number of files compiled: 264606 -> 260588 = -1.5% >>> total number of lines compiled: 76073364 -> 74477347 = -2.1% >>> size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 >>> bytes = -0.5% >>> >>> (I got the statistics by parsing the *.d files in the build directory). >>> >>> Thanks >>> - Ioi > From magnus.ihse.bursie at oracle.com Wed Jun 24 11:01:55 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 24 Jun 2020 13:01:55 +0200 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> <5EE5AB71.20505@oracle.com> Message-ID: On 2020-06-18 08:34, Kim Barrett wrote: >> On Jun 15, 2020, at 7:41 AM, Kim Barrett wrote: >> >>> On Jun 14, 2020, at 12:45 AM, Philip Race wrote: >>> >>> Kim, >>> >>> >>> Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. >>> Please rename the JEP. > After some off-list discussions I have a better idea of what Phil was > asking for and why. In response I've changed the JEP, replacing a few > occurrences of "HotSpot" with "the JDK", as described below. All > other occurrences of "HotSpot" have been left as-is. > > Title: > JEP 347: Adopt C++14 Language Features in HotSpot > => > JEP 347: Adopt C++14 Language Features in the JDK > > Summary: > Allow the use of selected C++14 language features in the HotSpot C++ source code. > => > Allow the use of selected C++14 language features in the JDK C++ source code. > > Goals: > The purpose of this JEP is to formally allow C++ source code changes within HotSpot to take advantage of some C++14 standard language features. > => > The purpose of this JEP is to formally allow C++ source code changes within the JDK to take advantage of some C++14 standard language features. > This all looks good to me. /Magnus From leo.korinth at oracle.com Wed Jun 24 11:07:18 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 24 Jun 2020 13:07:18 +0200 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> <0250fab1-c743-7077-0485-2d188609cb42@oracle.com> Message-ID: <582f6154-97d1-e8f6-779d-2f5ef5e489fc@oracle.com> On 15/06/2020 06:00, David Holmes wrote: > Hi Leo, > > Thanks for undertaking this effort! > > Some higher-level comments at this stage just looking at the > introduction of AtomicValue. > > As Kim has pointed out it may not always be appropriate to change all > fields subject to atomic operations to be converted to AtomicValue. > There is also the issue of fields accessed in a mix of locked and > lock-free mechanisms (particularly in the runtime where we have locked > writes and lock-free reads in a number of places, though sometimes > combined with atomic updates). If such fields are converted to > AtomicValue then I think we may still need a way to perform some raw > operations on them; or else we chose to leave them as currently defined > and apply Atomic ops directly when needed. (There is a parallel in the > Java APIs where we can declare a field as an AtomicX variable, or else > we can declare the field simply as X, and use an AtomicFieldUpdater to > perform the atomic operations.) Yes, I am not trying to remove usage of the AllStatic Atomic API, I am trying to remove the usage of volatile. Where performance is more important than type safety (or if it feels awkward), just continue use the Atomic API (see part2 of the change), but I see /no/ need for using volatile. No Atomic operations need the volatile qualifier, and (if I understand correctly) volatile is generally not what you want as it does not guarantee enough from the language for much anything outside writing to IO mapped devices. However when I started to remove volatile in gc code that /mostly/ works as documentation, I was afraid of changing the behaviour of the code. Using AtomicValue is a great way to do refactoring code and conserve behaviour. The two part conversion (part1 and part2) of my change shows how AtomicValue is a great way to remove volatile (part1), and in part2 revert to use the AllStatic Atomic API in certain places without forgetting converting implicit volatile load and stores or continuing to use non-atomic operations such as ++ on variables (now not volatile) by mistake. > > I like to imagine AtomicValue as a library-based alternative to a true > "atomic" keyword in a language, but I'm torn between conciseness of > expression and clarity of action. If we had an atomic keyword then: > > atomic int x = 0; > ... > x = 3;? // this is an atomic store > x++;??? // this is an atomic post-increment > > which would then argue for: > > AtomicValue x = 0; > ... > x = 3;? // this is an atomic store > x++;??? // this is an atomic post-increment > > rather than the explicit > > x.store(3);? // this is an atomic store > x.inc();???? // this is an atomic post-increment > > in particular having explicit load() and store() operations makes for > very unwieldy expressions IMHO. > > That said I know some people prefer it if atomic operations stand out > very clearly. Many also dislike operator overloading (although I personally like it, I think I am in a minority), if nothing else, I believe it hard to get through review. I /do/ like that the current verbose explicit way looks almost the same as the Atomic API we already have. Implicit load and stores also would "look" different from what we have today. I am unsure about what I really prefer, and keeping the look seems good and my thought was that it would offend the least... > > So I think we need to get a general consensus on what shape AtomicValue > should take and exactly how we propose to use it (or not) across all of > hotspot. > > And perhaps, as Kim alluded, the addition of AtomicValue should be split > out from its application to any particular subsystem. I do not want to push others to use it all over hotspot. Maybe not have a policy, but use it if it makes sense? Maybe just move it out of atomics.hpp as Coleen suggest is enough? My thinking is that we should reduce usage of volatile all over hotspot (regardless of AtomicValue); I do not want AtomicValue to be a forced way though. Thanks, Leo > Thanks, > David > ----- > > On 12/06/2020 1:51 am, Leo Korinth wrote: >>>> Hi, could I have a review for this change that adds AtomicValue<> to >>>> atomic.hpp and uses it in G1? >>>> >>>> I am adding an AtomicValue<> to atomic.hpp. This is an opaque type >>>> (the "value" is private) and that protect against non-atomic >>>> operations being used by mistake. AtomicValue methods are proxied to >>>> the corresponding (all-static) Atomic:: methods. All operations are >>>> explicitly atomic and in a type safe manner with semantics defined >>>> in enum atomic_memory_order >>>> (memory_order_conservative by default). >>>> >>>> Instead of using field variables as volatile, I change them to be of >>>> AtomicValue<> type. No need to verify that += (for example) will >>>> result in an atomic instruction on all supported compilers. >>>> >>>> I have some open questions regarding the exported fields in >>>> vmStructs_g1.hpp. Today, volatile fields are sometimes "exported" as >>>> volatile (and sometimes not, and I guess this is by mistake). I >>>> choose to export them all as non-volatile. From what I can see the >>>> volatile specific part only casts to void* (only documentation?). >>>> Java code is unchanged and still access them as the unwrapped values >>>> (static assert in atomic.hpp guarantees that memory layout is the >>>> same for T and AtomicValue). I think this is okay, but would like >>>> feedback on all this. >>>> >>>> The change is presented as a two part change. The first part changes >>>> all volatile to AtomicValue, the other part removes the AtomicValue >>>> part on non-field accesses. By doing it two part I will not forget >>>> to transform some operations by mistake. >>>> >>>> Copyright years will be updated when all other changes are approved. >>>> >>>> How about pushing this after 15 is branched off and thus have it for >>>> 16? >>>> >>>> Enhancement: >>>> https://bugs.openjdk.java.net/browse/JDK-8247213 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~lkorinth/8247213/0/part1 >>>> http://cr.openjdk.java.net/~lkorinth/8247213/0/part2 >>>> http://cr.openjdk.java.net/~lkorinth/8247213/0/full >>>> >>>> Testing: >>>> ?? tier1-3 >>>> >>>> Thanks, >>>> Leo >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Thanks, >>>>>> Leo From coleen.phillimore at oracle.com Wed Jun 24 12:45:14 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 24 Jun 2020 08:45:14 -0400 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> <6e6e3289-2c91-7019-36de-71548ab4d450@oracle.com> <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> Message-ID: Hi Ioi,? This looks good to me now. Thanks, Coleen On 6/24/20 2:16 AM, Ioi Lam wrote: > > > On 6/23/20 9:08 PM, David Holmes wrote: >> Hi Ioi, >> >> On 24/06/2020 6:10 am, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8248170 >>> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ >>> >>> >>> {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are >>> unnecessarily included in some popular headers. This introduces >>> unnecessary dependencies and slows down compile time. >> >> Based on your non-PCH local builds? >> >> I have no issue with removing unnecessary includes but I see far more >> added includes in your changes than I see removals, so I'm a little >> confused by that. I also have to wonder how on earth you concluded >> that these particular includes were the cause of your problem ?? >> > I ran a script on my local non-PCH build and got the following output. > I scan the file from the bottom to look for things to fix. > > http://cr.openjdk.java.net/~iklam/jdk16/misc/8248170-headers.txt > > The reasons for adding new includes: the fix has 2 parts > > (a) Remove unnecessary headers included by popular files such as > thread.hpp > (b) The above causes some sources files to break, because they were > missing includes. These files could compile before my patch because > the missing dependency was transitively included by thread.hpp, etc. > Now I need to add the missing includes. > > E.g., os_windows.cpp has this > > 2342???? CodeBlob* cb = CodeCache::find_blob(pc); > > ... so it needs to include codeCache.hpp, but that was transitively > included by vmOperations.hpp. > > ============= > > I updated the patch based on Coleen's off-line feedback: > > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02.delta/ > > > > Thanks > - Ioi > >> The change seems harmless so I don't object. >> >> Thanks, >> David >> >>> slowdebug: >>> total number of files compiled: 264606 -> 260588 = -1.5% >>> total number of lines compiled: 76073364 -> 74477347 = -2.1% >>> size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 >>> bytes = -0.5% >>> >>> (I got the statistics by parsing the *.d files in the build directory). >>> >>> Thanks >>> - Ioi > From coleen.phillimore at oracle.com Wed Jun 24 12:49:48 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 24 Jun 2020 08:49:48 -0400 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <3ac53f65-16d0-f0e5-c278-eb1742432bcb@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> <6e6e3289-2c91-7019-36de-71548ab4d450@oracle.com> <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> <3ac53f65-16d0-f0e5-c278-eb1742432bcb@oracle.com> Message-ID: <3cc47685-5ad3-33e6-5311-cd4ac9bf0a3b@oracle.com> On 6/24/20 5:41 AM, David Holmes wrote: > On 24/06/2020 4:16 pm, Ioi Lam wrote: >> >> >> On 6/23/20 9:08 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> On 24/06/2020 6:10 am, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8248170 >>>> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ >>>> >>>> >>>> {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are >>>> unnecessarily included in some popular headers. This introduces >>>> unnecessary dependencies and slows down compile time. >>> >>> Based on your non-PCH local builds? >>> >>> I have no issue with removing unnecessary includes but I see far >>> more added includes in your changes than I see removals, so I'm a >>> little confused by that. I also have to wonder how on earth you >>> concluded that these particular includes were the cause of your >>> problem ?? >>> >> I ran a script on my local non-PCH build and got the following >> output. I scan the file from the bottom to look for things to fix. > > Interesting! Thanks for sharing. > >> http://cr.openjdk.java.net/~iklam/jdk16/misc/8248170-headers.txt >> >> The reasons for adding new includes: the fix has 2 parts >> >> (a) Remove unnecessary headers included by popular files such as >> thread.hpp >> (b) The above causes some sources files to break, because they were >> missing includes. These files could compile before my patch because >> the missing dependency was transitively included by thread.hpp, etc. >> Now I need to add the missing includes. >> >> E.g., os_windows.cpp has this >> >> 2342???? CodeBlob* cb = CodeCache::find_blob(pc); >> >> ... so it needs to include codeCache.hpp, but that was transitively >> included by vmOperations.hpp. > > Okay. Just seemed odd to see more added than taken away. > >> ============= >> >> I updated the patch based on Coleen's off-line feedback: >> >> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02/ >> >> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02.delta/ > > > Appears to me that c1_CodeStubs.hpp includes c1/c1_Runtime1.hpp, and > each of the c1_CodeStubs_.cpp files also include it. That seems > redundant. I don't immediately see where c1_CodeStubs.hpp needs c1_Runtime1.hpp, so maybe it doesn't need it, and the #include can be pushed down to the .cpp files that might not already transitively include it.?? If it's a small change, you can include it with this, but honestly, there could be many iterations of cleaning up our includes, so it can be done in a future round also. Thanks, Coleen > > Thanks, > David > ----- > >> >> >> Thanks >> - Ioi >> >>> The change seems harmless so I don't object. >>> >>> Thanks, >>> David >>> >>>> slowdebug: >>>> total number of files compiled: 264606 -> 260588 = -1.5% >>>> total number of lines compiled: 76073364 -> 74477347 = -2.1% >>>> size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 >>>> bytes = -0.5% >>>> >>>> (I got the statistics by parsing the *.d files in the build >>>> directory). >>>> >>>> Thanks >>>> - Ioi >> From christian.hagedorn at oracle.com Wed Jun 24 13:17:26 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Wed, 24 Jun 2020 15:17:26 +0200 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> Message-ID: <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> Hi Coleen, hi Vladimir Thank you for your reviews! On 23.06.20 22:04, coleen.phillimore at oracle.com wrote: > I know less about this code than Erik and Vladimir but would you need to > also test for this condition in the b) case? > > ?754????????? !old_method->method_holder()->is_loader_alive() || > > > You probably won't get as much racy behavior from class unloading though. That's a valid concern. I discussed it with Erik ?. and he think we need to add that together with old_method == NULL like in the a) case. Here is a detailed explanation from Erik: "Concurrent class unloading clears the method, so it can be NULL. There is a narrow race when we are concurrently unloading nmethods, while others are loaded. The scenario is that we have an optimized virtual call, materialized kind of like a static call, assuming that there is only a single implementor. When the single implementor is unloaded, the caller nmethod will get deoptimized when the GC gets to it. But before that happens, we can have calls going through that call site. If before the call, a second implementor gets loaded, I think it is possible that the new loaded class gets installed as still the only subclass (single implementor) of the abstract thing. Then, I don't think the class loading triggers CHA deoptimization of the caller nmethod. So I think then you can call through that nmethod to an alternative callee nmethod, due to a race between class unloading, class loading, compilation and calling an optimized virtual call." I updated this in a new webrev accordingly: http://cr.openjdk.java.net/~chagedorn/8245128/webrev.01/ Best regards, Christian > Otherwise, the change makes sense. > thanks, > Coleen > > On 6/23/20 12:31 PM, Vladimir Kozlov wrote: >> Good. >> >> Thanks, >> Vladimir >> >> On 6/23/20 6:58 AM, Christian Hagedorn wrote: >>> Hi >>> >>> Please review the following patch: >>> https://bugs.openjdk.java.net/browse/JDK-8245128 >>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >>> >>> The assert is often hit when enabling the Kitchensink instrumentation >>> module which triggers a lot of class redefinitions. The problem looks >>> similar to the one fixed in JDK-8225681 [1] for the other a) >>> MT-unsafe assert. We could be dealing with an old method which we >>> should also exclude in the second b) MT-unsafe assert (JDK-8225681 >>> fixed it only for a)). A nice description of the problem is found in >>> the comment [2] by Erik ?. >>> >>> Applying this fix, the assert is not hit anymore with repeated >>> Kitchensink runs with the instrumentation module enabled. >>> >>> Thank you! >>> >>> Best regards, >>> Christian >>> >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >>> [2] >>> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >>> > From erik.osterlund at oracle.com Wed Jun 24 14:32:33 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 24 Jun 2020 16:32:33 +0200 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> Message-ID: <6f6c964e-9991-8c31-3946-4e5f094a76ab@oracle.com> Hi Christian, Looks good. Thanks, /Erik On 2020-06-24 15:17, Christian Hagedorn wrote: > Hi Coleen, hi Vladimir > > Thank you for your reviews! > > On 23.06.20 22:04, coleen.phillimore at oracle.com wrote: >> I know less about this code than Erik and Vladimir but would you need >> to also test for this condition in the b) case? >> >> ??754 !old_method->method_holder()->is_loader_alive() || >> >> >> You probably won't get as much racy behavior from class unloading >> though. > > That's a valid concern. I discussed it with Erik ?. and he think we > need to add that together with old_method == NULL like in the a) case. > Here is a detailed explanation from Erik: > > "Concurrent class unloading clears the method, so it can be NULL. > There is a narrow race when we are concurrently unloading nmethods, > while others are loaded. The scenario is that we have an optimized > virtual call, materialized kind of like a static call, assuming that > there is only a single implementor. When the single implementor is > unloaded, the caller nmethod will get deoptimized when the GC gets to > it. But before that happens, we can have calls going through that call > site. If before the call, a second implementor gets loaded, I think it > is possible that the new loaded class gets installed as still the only > subclass (single implementor) of the abstract thing. Then, I don't > think the class loading triggers CHA deoptimization of the caller > nmethod. So I think then you can call through that nmethod to an > alternative callee nmethod, due to a race between class unloading, > class loading, compilation and calling an optimized virtual call." > > I updated this in a new webrev accordingly: > http://cr.openjdk.java.net/~chagedorn/8245128/webrev.01/ > > Best regards, > Christian > > >> Otherwise, the change makes sense. >> thanks, >> Coleen >> >> On 6/23/20 12:31 PM, Vladimir Kozlov wrote: >>> Good. >>> >>> Thanks, >>> Vladimir >>> >>> On 6/23/20 6:58 AM, Christian Hagedorn wrote: >>>> Hi >>>> >>>> Please review the following patch: >>>> https://bugs.openjdk.java.net/browse/JDK-8245128 >>>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >>>> >>>> The assert is often hit when enabling the Kitchensink >>>> instrumentation module which triggers a lot of class redefinitions. >>>> The problem looks similar to the one fixed in JDK-8225681 [1] for >>>> the other a) MT-unsafe assert. We could be dealing with an old >>>> method which we should also exclude in the second b) MT-unsafe >>>> assert (JDK-8225681 fixed it only for a)). A nice description of >>>> the problem is found in the comment [2] by Erik ?. >>>> >>>> Applying this fix, the assert is not hit anymore with repeated >>>> Kitchensink runs with the instrumentation module enabled. >>>> >>>> Thank you! >>>> >>>> Best regards, >>>> Christian >>>> >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >>>> [2] >>>> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >>>> >> From kim.barrett at oracle.com Wed Jun 24 14:44:03 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 24 Jun 2020 10:44:03 -0400 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> Message-ID: <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> > On Jun 24, 2020, at 3:24 AM, Stefan Karlsson wrote: > > Hi Kim, > > Thanks for working on updating this document. I think this mostly looks good, but do have a few comments. Thanks for looking at it. Some of your comments are, I think, suggestions for substantive style changes. I tried not to make too many of those in this revision, other than as related to features from the new Standards. And at least some of your suggestions seem like they would be worthy of their own (possibly significant) discussion. (I did quietly drop "Be sparing with templates." and "Introduce new templates only by Act of Congress.". I also quietly dropped mention of +Verbose, since we should be using UL.) I'm hoping this refresh will be part of making this a living document that we'll change to reflect our needs and actual usage. So I'd like to defer such points [Note: Stefan and I discussed this offline, and I think we're in agreement on most of the points below.] > * > > Macro names usually use upper-case with words separated by a single > underscore. However, there are a significant number of lowercase > macros. Adding to that set is discouraged. > > This is new line from the old style guide. Personally I find upper-case macros very noise and grabs a lot of the attention of the code. For most macros that's OK, but if a macro is prevalent in the source code, like assert, guarantee, log_info, offset_of, maybe even the recently discussed ENABLE_IF/enable_if, then I think it makes sense to *NOT* use upper-case names. This *is* a place where I went beyond editorial cleanup of the existing text. I will take this out and bring it up for discussion later. > * > > Use functions from globalDefinitions.hpp when performing bitwise > operations on integers. Do not code directly as C operators, unless > they are extremely simple. > (Examples:|round_to|,|is_power_of_2|,|exact_log2|.) > > Over the years, we've seen problems with the intptr_t bit manipulation functions in globalDefinitions.hpp. We have been slowly fixing these problems and moving some of them away into other headers like align.hpp and powerOfTwo.hpp. The bit-manipulation functions left in globalDefinitions.hpp still use intptr_t and I don't think we should encourage people to using them before they have been rewritten, and fully tested with all integer types. This is directly from the original document. I understand the intent, even though the execution has been lacking, so didn't want to quietly remove it. I agree that the current set of bit manipulation functions in globalDefinitions.hpp have problems. Besides the types being used, they also aren't constexpr, making them useless in many common contexts. I think revisiting this area with new language features in hand would be worth considering. > round_to has been removed in favor of the align_up function. Changed to align_up and also added ?and related files? after globalDefinitions.hpp, since we?ve been moving some of these elsewhere. > * > > Put a member function|FooBar::bang|into the same file that > defined|FooBar|, or its associated *.cpp file. > > I don't think we should promote putting implementation in our header files. And we also have: * Do not put non-trivial function implementations in .hpp files. If the implementation depends on other .hpp files, put it in a .cpp or a .inline.hpp file. I've changed it to "associated *.inline.hpp or *.cpp"? But I don't think the trend toward lots of .inline.hpp files is necessarily good either, especially since I think we man not be using them properly. (We had a discussion about this some time ago.) But that can be a discussion for later. > * > > Implement classes as if expecting rough usage by clients. Check for > incorrect usage of a class > using|assert(...)|,|guarantee(...)|,|ShouldNotReachHere()|and > comments wherever needed. Performance is almost never a reason to > omit asserts. > > I know that the Compiler team had a push to replace ShouldNotReachHere() with fatal("descriptive message"). It would be good to have a common guidance w.r.t. that. Could be done as a follow-up task. Yes, that should be a followup. > * > > When you must defactor to optimize, preserve as much structure as > possible. If you must hand-inline some name, label the local copy > with the original name. > > Is defactor a real word? Could this be rewritten to skip that word? I think "defactor" is a word, and this was copied directly from the original text. I'd rather not try to rewrite it at this time. > * > > Use One-True-Brace-Style. The opening brace for a function or class > is normally at the end of the line; it is sometimes moved to the > beginning of the next line for emphasis. Substatements are enclosed > in braces, even if there is only a single statement. Extremely > simple one-line statements may drop braces around a substatement. > > Why not just skip the last sentence? People interpret extremely simple one-liners differently and this sentence just adds a loop-hole for those that like to write if-statements without braces. This could be considered an extremely simple one-liner: > > int append(const E& elem) { > if (this->_len == this->_max) grow(this->_len); > int idx = this->_len++; > this->_data[idx] = elem; > return idx; > } > > but I don't think we should be writing code like this anymore. That's a rephrasing of the original, which said: Use braces around substatements. (Relaxable for extremely simple substatements on the same line.) I think this should be a followup. > * > > Use extra parentheses in expressions whenever operator precedence > seems doubtful. Always use parentheses in shift/mask expressions > (|<<|,|&|,|||,|~|). Don?t add whitespace immediately inside parentheses. > > Isn't it a bit odd to mention ~ here? It precedes almost all operators, so either they are needed or they are not. There's not really a style choice. Good point. That was in the original text, but you are right that it's useless here. Removed. > * > > Type names and global names should use mixed-case with the first > letter of each word capitalized (|FooBar|). Embedded abbreviations > in otherwise mixed-case names are usually capitalized entirely > rather than being treated as a single word with only the initial > letter capitalized, e.g. ?HTML? rather than ?Html?. > > Could this be split into two separate bullets? Split. The all-caps style for abbreviation is an addition, not present in the original text. I added it based on observation and a discussion I observed in a review thread a while ago. > I think the mixed-case guide makes sense, but I'm not a fan when it's used dogmatically. I would much rather allow us to write atomic instead of the longer more noisier AtomicValue. And I know that this rule is used to argument for the latter. I think this should be a followup. > * > > Constant names may be upper-case or mixed-case, according to > historical necessity. (Note: There are many examples of constants > with lowercase names.) > > The "according to historical necessity" makes it unclear if there's a preferred way for new code. If there's no preferred way out of the two, maybe drop that part? That phrasing is from the original text. I *think* it probably means one should follow the nearby style, and acknowledges that there's been significant variation on this point. This is something I'd like to see be more prescriptive; I feel like I never know how to name constants in such a way that I won't get review feedback requesting something different. > * > > Use|bool|for boolean values. Do not use ints or pointers as > (implicit) booleans with|&&|,||||,|if|,|while|. Instead, compare > explicitly, i.e.|if (x != 0)|or|if (ptr != nullptr)|, etc. Do not > use declarations in/condition/forms, i.e. don?t use|if (T v = value) > { ... }|. > > The last part is a new addition to the style guide. We use it for some iterators: > > for (JavaThreadIteratorWithHandle jtiwh; JavaThread *thr = jtiwh.next(); ) { > > this is useful to limit the scope of the instance. I don't think we should discourage this in the style guide. I think that's not a new addition, just making explicit something which is already implicit. Using that feature is clearly using an integer or pointer as a boolean, or using a boolean conversion operator (and without C++11 explicit conversions those are either implicit (nearly always a bad idea) or using the safe-bool idiom, which is a pain to write and looks really strange to someone not already familiar with it). So I think permitting it would be a substantive change. I made it explicit to call attention to it, as I'd like to see this changed. While I tried to avoid making substantive changes, that doesn't mean there aren't some leading questions. > - "Some features are discussed their own subsection" > > Missing an "in?? Fixed. I also have a couple of changes suggested by John Rose offline, as well as a couple of nomenclature corrections that I?d previously missed. I?m going to wait a bit before sending an update, to see what other comments might come in. From stefan.karlsson at oracle.com Wed Jun 24 14:47:12 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 24 Jun 2020 16:47:12 +0200 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> Message-ID: <34dc1d5d-ef2a-4ed4-4a3c-5fba20d532b1@oracle.com> Sounds good to me. Thanks, StefanK On 2020-06-24 16:44, Kim Barrett wrote: >> On Jun 24, 2020, at 3:24 AM, Stefan Karlsson wrote: >> >> Hi Kim, >> >> Thanks for working on updating this document. I think this mostly looks good, but do have a few comments. > Thanks for looking at it. > > Some of your comments are, I think, suggestions for substantive style > changes. I tried not to make too many of those in this revision, other > than as related to features from the new Standards. And at least some > of your suggestions seem like they would be worthy of their own > (possibly significant) discussion. > > (I did quietly drop "Be sparing with templates." and "Introduce new > templates only by Act of Congress.". I also quietly dropped mention of > +Verbose, since we should be using UL.) > > I'm hoping this refresh will be part of making this a living document > that we'll change to reflect our needs and actual usage. So I'd like > to defer such points > > [Note: Stefan and I discussed this offline, and I think we're in > agreement on most of the points below.] > >> * >> >> Macro names usually use upper-case with words separated by a single >> underscore. However, there are a significant number of lowercase >> macros. Adding to that set is discouraged. >> >> This is new line from the old style guide. Personally I find upper-case macros very noise and grabs a lot of the attention of the code. For most macros that's OK, but if a macro is prevalent in the source code, like assert, guarantee, log_info, offset_of, maybe even the recently discussed ENABLE_IF/enable_if, then I think it makes sense to *NOT* use upper-case names. > This *is* a place where I went beyond editorial cleanup of the > existing text. I will take this out and bring it up for discussion > later. > >> * >> >> Use functions from globalDefinitions.hpp when performing bitwise >> operations on integers. Do not code directly as C operators, unless >> they are extremely simple. >> (Examples:|round_to|,|is_power_of_2|,|exact_log2|.) >> >> Over the years, we've seen problems with the intptr_t bit manipulation functions in globalDefinitions.hpp. We have been slowly fixing these problems and moving some of them away into other headers like align.hpp and powerOfTwo.hpp. The bit-manipulation functions left in globalDefinitions.hpp still use intptr_t and I don't think we should encourage people to using them before they have been rewritten, and fully tested with all integer types. > This is directly from the original document. I understand the intent, > even though the execution has been lacking, so didn't want to quietly > remove it. I agree that the current set of bit manipulation functions > in globalDefinitions.hpp have problems. Besides the types being used, > they also aren't constexpr, making them useless in many common contexts. > > I think revisiting this area with new language features in hand would > be worth considering. > >> round_to has been removed in favor of the align_up function. > Changed to align_up and also added ?and related files? after globalDefinitions.hpp, > since we?ve been moving some of these elsewhere. > >> * >> >> Put a member function|FooBar::bang|into the same file that >> defined|FooBar|, or its associated *.cpp file. >> >> I don't think we should promote putting implementation in our header files. > And we also have: > > * Do not put non-trivial function implementations in .hpp files. If the > implementation depends on other .hpp files, put it in a .cpp or a > .inline.hpp file. > > I've changed it to "associated *.inline.hpp or *.cpp"? > > But I don't think the trend toward lots of .inline.hpp files is > necessarily good either, especially since I think we man not be using > them properly. (We had a discussion about this some time ago.) But > that can be a discussion for later. > >> * >> >> Implement classes as if expecting rough usage by clients. Check for >> incorrect usage of a class >> using|assert(...)|,|guarantee(...)|,|ShouldNotReachHere()|and >> comments wherever needed. Performance is almost never a reason to >> omit asserts. >> >> I know that the Compiler team had a push to replace ShouldNotReachHere() with fatal("descriptive message"). It would be good to have a common guidance w.r.t. that. Could be done as a follow-up task. > Yes, that should be a followup. > >> * >> >> When you must defactor to optimize, preserve as much structure as >> possible. If you must hand-inline some name, label the local copy >> with the original name. >> >> Is defactor a real word? Could this be rewritten to skip that word? > I think "defactor" is a word, and this was copied directly from the > original text. I'd rather not try to rewrite it at this time. > >> * >> >> Use One-True-Brace-Style. The opening brace for a function or class >> is normally at the end of the line; it is sometimes moved to the >> beginning of the next line for emphasis. Substatements are enclosed >> in braces, even if there is only a single statement. Extremely >> simple one-line statements may drop braces around a substatement. >> >> Why not just skip the last sentence? People interpret extremely simple one-liners differently and this sentence just adds a loop-hole for those that like to write if-statements without braces. This could be considered an extremely simple one-liner: >> >> int append(const E& elem) { >> if (this->_len == this->_max) grow(this->_len); >> int idx = this->_len++; >> this->_data[idx] = elem; >> return idx; >> } >> >> but I don't think we should be writing code like this anymore. > That's a rephrasing of the original, which said: > > Use braces around substatements. (Relaxable for extremely simple > substatements on the same line.) > > I think this should be a followup. > >> * >> >> Use extra parentheses in expressions whenever operator precedence >> seems doubtful. Always use parentheses in shift/mask expressions >> (|<<|,|&|,|||,|~|). Don?t add whitespace immediately inside parentheses. >> >> Isn't it a bit odd to mention ~ here? It precedes almost all operators, so either they are needed or they are not. There's not really a style choice. > Good point. That was in the original text, but you are right that > it's useless here. Removed. > >> * >> >> Type names and global names should use mixed-case with the first >> letter of each word capitalized (|FooBar|). Embedded abbreviations >> in otherwise mixed-case names are usually capitalized entirely >> rather than being treated as a single word with only the initial >> letter capitalized, e.g. ?HTML? rather than ?Html?. >> >> Could this be split into two separate bullets? > Split. The all-caps style for abbreviation is an addition, not > present in the original text. I added it based on observation and a > discussion I observed in a review thread a while ago. > >> I think the mixed-case guide makes sense, but I'm not a fan when it's used dogmatically. I would much rather allow us to write atomic instead of the longer more noisier AtomicValue. And I know that this rule is used to argument for the latter. > I think this should be a followup. > >> * >> >> Constant names may be upper-case or mixed-case, according to >> historical necessity. (Note: There are many examples of constants >> with lowercase names.) >> >> The "according to historical necessity" makes it unclear if there's a preferred way for new code. If there's no preferred way out of the two, maybe drop that part? > That phrasing is from the original text. I *think* it probably means > one should follow the nearby style, and acknowledges that there's been > significant variation on this point. > > This is something I'd like to see be more prescriptive; I feel like I > never know how to name constants in such a way that I won't get review > feedback requesting something different. > >> * >> >> Use|bool|for boolean values. Do not use ints or pointers as >> (implicit) booleans with|&&|,||||,|if|,|while|. Instead, compare >> explicitly, i.e.|if (x != 0)|or|if (ptr != nullptr)|, etc. Do not >> use declarations in/condition/forms, i.e. don?t use|if (T v = value) >> { ... }|. >> >> The last part is a new addition to the style guide. We use it for some iterators: >> >> for (JavaThreadIteratorWithHandle jtiwh; JavaThread *thr = jtiwh.next(); ) { >> >> this is useful to limit the scope of the instance. I don't think we should discourage this in the style guide. > I think that's not a new addition, just making explicit something > which is already implicit. Using that feature is clearly using an > integer or pointer as a boolean, or using a boolean conversion > operator (and without C++11 explicit conversions those are either > implicit (nearly always a bad idea) or using the safe-bool idiom, > which is a pain to write and looks really strange to someone not > already familiar with it). > > So I think permitting it would be a substantive change. > > I made it explicit to call attention to it, as I'd like to see this > changed. While I tried to avoid making substantive changes, that > doesn't mean there aren't some leading questions. > >> - "Some features are discussed their own subsection" >> >> Missing an "in?? > Fixed. > > I also have a couple of changes suggested by John Rose offline, as well > as a couple of nomenclature corrections that I?d previously missed. I?m > going to wait a bit before sending an update, to see what other comments > might come in. > From kim.barrett at oracle.com Wed Jun 24 15:01:25 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 24 Jun 2020 11:01:25 -0400 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: <95B6BF1F-CF49-4024-86C0-F697F4FD4822@oracle.com> References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <95B6BF1F-CF49-4024-86C0-F697F4FD4822@oracle.com> Message-ID: > On Jun 23, 2020, at 9:57 PM, John Rose wrote: > > On Jun 23, 2020, at 12:32 PM, Kim Barrett wrote: >> >> Please review / comment on this proposed update to the HotSpot Style >> Guide. This is part of the work for JDK-8208089: JEP 347: Adopt C++14 >> Language Features in the JDK. > > I read it all. It is an excellent and timely refresh of the existing document. > (?Which I remember writing first drafts of many years ago!) > > The terms RAII and SFINAE need parenthetical definitions and/or links > to their definitions. Though I?ve been doing C++ most of my life, those > acronyms are not in my primary vocabulary, and I?m surely not the only one. > ODR already has a parenthetical definition, so it?s good to go. I?ve added links to relevant references for these acronyms: ADL, ODR, RAII, RTTI, SFINAE. I think that?s currently all of them. > I noted various deletions and additions, and honestly find nothing at > all that bothers me. It feels like very careful work. > > I say ship it! (Crossing my fingers?) > > ? John Thanks. From coleen.phillimore at oracle.com Wed Jun 24 15:43:53 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 24 Jun 2020 11:43:53 -0400 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> Message-ID: <26df478c-a252-a7ef-3441-2893c64b3139@oracle.com> 01 looks good. Thanks for checking out the unloading case. Coleen On 6/24/20 9:17 AM, Christian Hagedorn wrote: > Hi Coleen, hi Vladimir > > Thank you for your reviews! > > On 23.06.20 22:04, coleen.phillimore at oracle.com wrote: >> I know less about this code than Erik and Vladimir but would you need >> to also test for this condition in the b) case? >> >> ??754 !old_method->method_holder()->is_loader_alive() || >> >> >> You probably won't get as much racy behavior from class unloading >> though. > > That's a valid concern. I discussed it with Erik ?. and he think we > need to add that together with old_method == NULL like in the a) case. > Here is a detailed explanation from Erik: > > "Concurrent class unloading clears the method, so it can be NULL. > There is a narrow race when we are concurrently unloading nmethods, > while others are loaded. The scenario is that we have an optimized > virtual call, materialized kind of like a static call, assuming that > there is only a single implementor. When the single implementor is > unloaded, the caller nmethod will get deoptimized when the GC gets to > it. But before that happens, we can have calls going through that call > site. If before the call, a second implementor gets loaded, I think it > is possible that the new loaded class gets installed as still the only > subclass (single implementor) of the abstract thing. Then, I don't > think the class loading triggers CHA deoptimization of the caller > nmethod. So I think then you can call through that nmethod to an > alternative callee nmethod, due to a race between class unloading, > class loading, compilation and calling an optimized virtual call." > > I updated this in a new webrev accordingly: > http://cr.openjdk.java.net/~chagedorn/8245128/webrev.01/ > > Best regards, > Christian > > >> Otherwise, the change makes sense. >> thanks, >> Coleen >> >> On 6/23/20 12:31 PM, Vladimir Kozlov wrote: >>> Good. >>> >>> Thanks, >>> Vladimir >>> >>> On 6/23/20 6:58 AM, Christian Hagedorn wrote: >>>> Hi >>>> >>>> Please review the following patch: >>>> https://bugs.openjdk.java.net/browse/JDK-8245128 >>>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >>>> >>>> The assert is often hit when enabling the Kitchensink >>>> instrumentation module which triggers a lot of class redefinitions. >>>> The problem looks similar to the one fixed in JDK-8225681 [1] for >>>> the other a) MT-unsafe assert. We could be dealing with an old >>>> method which we should also exclude in the second b) MT-unsafe >>>> assert (JDK-8225681 fixed it only for a)). A nice description of >>>> the problem is found in the comment [2] by Erik ?. >>>> >>>> Applying this fix, the assert is not hit anymore with repeated >>>> Kitchensink runs with the instrumentation module enabled. >>>> >>>> Thank you! >>>> >>>> Best regards, >>>> Christian >>>> >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >>>> [2] >>>> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >>>> >> From vladimir.kozlov at oracle.com Wed Jun 24 15:55:44 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 24 Jun 2020 08:55:44 -0700 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <6f6c964e-9991-8c31-3946-4e5f094a76ab@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> <6f6c964e-9991-8c31-3946-4e5f094a76ab@oracle.com> Message-ID: <004a4ac2-125f-0d63-b173-1b8742be9701@oracle.com> +1 thanks, Vladimir On 6/24/20 7:32 AM, Erik ?sterlund wrote: > Hi Christian, > > Looks good. > > Thanks, > /Erik > > On 2020-06-24 15:17, Christian Hagedorn wrote: >> Hi Coleen, hi Vladimir >> >> Thank you for your reviews! >> >> On 23.06.20 22:04, coleen.phillimore at oracle.com wrote: >>> I know less about this code than Erik and Vladimir but would you need to also test for this condition in the b) case? >>> >>> ??754 !old_method->method_holder()->is_loader_alive() || >>> >>> >>> You probably won't get as much racy behavior from class unloading though. >> >> That's a valid concern. I discussed it with Erik ?. and he think we need to add that together with old_method == NULL >> like in the a) case. Here is a detailed explanation from Erik: >> >> "Concurrent class unloading clears the method, so it can be NULL. There is a narrow race when we are concurrently >> unloading nmethods, while others are loaded. The scenario is that we have an optimized virtual call, materialized kind >> of like a static call, assuming that there is only a single implementor. When the single implementor is unloaded, the >> caller nmethod will get deoptimized when the GC gets to it. But before that happens, we can have calls going through >> that call site. If before the call, a second implementor gets loaded, I think it is possible that the new loaded class >> gets installed as still the only subclass (single implementor) of the abstract thing. Then, I don't think the class >> loading triggers CHA deoptimization of the caller nmethod. So I think then you can call through that nmethod to an >> alternative callee nmethod, due to a race between class unloading, class loading, compilation and calling an optimized >> virtual call." >> >> I updated this in a new webrev accordingly: >> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.01/ >> >> Best regards, >> Christian >> >> >>> Otherwise, the change makes sense. >>> thanks, >>> Coleen >>> >>> On 6/23/20 12:31 PM, Vladimir Kozlov wrote: >>>> Good. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 6/23/20 6:58 AM, Christian Hagedorn wrote: >>>>> Hi >>>>> >>>>> Please review the following patch: >>>>> https://bugs.openjdk.java.net/browse/JDK-8245128 >>>>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >>>>> >>>>> The assert is often hit when enabling the Kitchensink instrumentation module which triggers a lot of class >>>>> redefinitions. The problem looks similar to the one fixed in JDK-8225681 [1] for the other a) MT-unsafe assert. We >>>>> could be dealing with an old method which we should also exclude in the second b) MT-unsafe assert (JDK-8225681 >>>>> fixed it only for a)). A nice description of the problem is found in the comment [2] by Erik ?. >>>>> >>>>> Applying this fix, the assert is not hit anymore with repeated Kitchensink runs with the instrumentation module >>>>> enabled. >>>>> >>>>> Thank you! >>>>> >>>>> Best regards, >>>>> Christian >>>>> >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >>>>> [2] >>>>> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >>>>> >>> > From ioi.lam at oracle.com Wed Jun 24 18:33:12 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 24 Jun 2020 11:33:12 -0700 Subject: RFR [16] 8248170 Excessive include of compiledMethod, codeCache, javaClasses and systemDictionary In-Reply-To: <3cc47685-5ad3-33e6-5311-cd4ac9bf0a3b@oracle.com> References: <3e9763fc-bb3b-6ac7-1969-63f456396d19@oracle.com> <6e6e3289-2c91-7019-36de-71548ab4d450@oracle.com> <0fd1dc4a-d092-9621-203e-4b4c07c4d1f2@oracle.com> <3ac53f65-16d0-f0e5-c278-eb1742432bcb@oracle.com> <3cc47685-5ad3-33e6-5311-cd4ac9bf0a3b@oracle.com> Message-ID: <9bd2729c-a4a2-ac1d-2fc5-b276e63d3f00@oracle.com> On 6/24/20 5:49 AM, coleen.phillimore at oracle.com wrote: > > > On 6/24/20 5:41 AM, David Holmes wrote: >> On 24/06/2020 4:16 pm, Ioi Lam wrote: >>> >>> >>> On 6/23/20 9:08 PM, David Holmes wrote: >>>> Hi Ioi, >>>> >>>> On 24/06/2020 6:10 am, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8248170 >>>>> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v01/ >>>>> >>>>> >>>>> {compiledMethod, codeCache, javaClasses, systemDictionary}.hpp are >>>>> unnecessarily included in some popular headers. This introduces >>>>> unnecessary dependencies and slows down compile time. >>>> >>>> Based on your non-PCH local builds? >>>> >>>> I have no issue with removing unnecessary includes but I see far >>>> more added includes in your changes than I see removals, so I'm a >>>> little confused by that. I also have to wonder how on earth you >>>> concluded that these particular includes were the cause of your >>>> problem ?? >>>> >>> I ran a script on my local non-PCH build and got the following >>> output. I scan the file from the bottom to look for things to fix. >> >> Interesting! Thanks for sharing. >> >>> http://cr.openjdk.java.net/~iklam/jdk16/misc/8248170-headers.txt >>> >>> The reasons for adding new includes: the fix has 2 parts >>> >>> (a) Remove unnecessary headers included by popular files such as >>> thread.hpp >>> (b) The above causes some sources files to break, because they were >>> missing includes. These files could compile before my patch because >>> the missing dependency was transitively included by thread.hpp, etc. >>> Now I need to add the missing includes. >>> >>> E.g., os_windows.cpp has this >>> >>> 2342???? CodeBlob* cb = CodeCache::find_blob(pc); >>> >>> ... so it needs to include codeCache.hpp, but that was transitively >>> included by vmOperations.hpp. >> >> Okay. Just seemed odd to see more added than taken away. >> >>> ============= >>> >>> I updated the patch based on Coleen's off-line feedback: >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02/ >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8248170-fix-includes-compiledMethod-etc.v02.delta/ >> >> >> >> Appears to me that c1_CodeStubs.hpp includes c1/c1_Runtime1.hpp, and >> each of the c1_CodeStubs_.cpp files also include it. That seems >> redundant. > > I don't immediately see where c1_CodeStubs.hpp needs c1_Runtime1.hpp, > so maybe it doesn't need it, and the #include can be pushed down to > the .cpp files that might not already transitively include it.?? If > it's a small change, you can include it with this, but honestly, there > could be many iterations of cleaning up our includes, so it can be > done in a future round also. > Hi Coleen, thanks for the review. With the current patch, c1_CodeStubs.hpp is included only 15 times, so even if it has unnecessary includes, fixing them would not gain much. I'll leave it alone for now. Thanks - Ioi > Thanks, > Coleen >> >> Thanks, >> David >> ----- >> >>> >>> >>> Thanks >>> - Ioi >>> >>>> The change seems harmless so I don't object. >>>> >>>> Thanks, >>>> David >>>> >>>>> slowdebug: >>>>> total number of files compiled: 264606 -> 260588 = -1.5% >>>>> total number of lines compiled: 76073364 -> 74477347 = -2.1% >>>>> size of libjvm.so (internal symbols): 234180296 bytes -> 233129904 >>>>> bytes = -0.5% >>>>> >>>>> (I got the statistics by parsing the *.d files in the build >>>>> directory). >>>>> >>>>> Thanks >>>>> - Ioi >>> > From jesper.wilhelmsson at oracle.com Wed Jun 24 19:01:49 2020 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 24 Jun 2020 21:01:49 +0200 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> Message-ID: I agree that the Developers' Guide would be a good place for this document. There is already a discussion on guide-dev about including the Java coding style guide and there are plans to add style guides for our Markdown and Makefiles as well. If we can reduce the number of places one has to look to find this kind of information I think everyone will benefit in the end. Thanks, /Jesper > On 24 Jun 2020, at 08:29, Erik Helin wrote: > > On 6/23/20 9:32 PM, Kim Barrett wrote >> For convenience of review, especially for incremental updates based on >> feedback, I'm publishing the updated document as a webrev. > > Thanks, appreciated! > >> Once reviewed, the finished text can be used to update the wiki, unless we >> decide it should be homed somewhere else. > > Should this document perhaps go into the OpenJDK guide [0] instead of being on the wiki [1]? Since the OpenJDK guide resides in a repository [2], that would allow for updates (and reviews/discussions) using our standard contribution processes. > > I personally find it easier to keep up with changes to the guide (by subscribing to guide-dev [3]) compared to following the wiki change history. > > Thanks, > Erik > > [0]: https://openjdk.java.net/projects/guide/ > [1]: https://wiki.openjdk.java.net/display/HotSpot/StyleGuide > [2]: https://github.com/openjdk/guide > [3]: https://mail.openjdk.java.net/mailman/listinfo/guide-dev From sandhya.viswanathan at intel.com Wed Jun 24 19:03:52 2020 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Wed, 24 Jun 2020 19:03:52 +0000 Subject: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes In-Reply-To: References: <275eb57c-51c0-675e-c32a-91b198023559@redhat.com> <719F9169-ABC4-408E-B732-F1BD9A84337F@oracle.com> <9a13f5df-d946-579d-4282-917dc7338dc8@redhat.com> <09BC0693-80E0-4F87-855E-0B38A6F5EFA2@oracle.com> <668e500e-f621-5a2c-a41e-f73536880f73@redhat.com> <1909fa9d-98bb-c2fb-45d8-540247d1ca8b@redhat.com> Message-ID: Hi Andrew/Yang, We couldn?t propose Vector API to target in time for JDK 15 and hoping to do so early in JDK 16 timeframe. The implementation reviews on other components have made good progress. We have so far ok to PPT from (runtime, shared compiler changes, x86 backend). Java API implementation review is in progress. I wanted to check with you both if we have a go ahead from aarch64 backed point of view. Best Regards, Sandhya -----Original Message----- From: hotspot-compiler-dev On Behalf Of Yang Zhang Sent: Tuesday, May 26, 2020 7:59 PM To: Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes > But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? The new instructions can be classified as: 1. Instructions that can be matched with NEON instructions directly. MulVB and SqrtVF have been merged into jdk master already. The patch of AbsV is in review [1]. 2. Instructions that Jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. May I have a new patch for these? 3. Panama/Vector API specific instructions Such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. Regards Yang [1] https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-May/008861.html -----Original Message----- From: Andrew Haley Sent: Tuesday, May 26, 2020 4:25 PM To: Yang Zhang ; Paul Sandoz Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; nd Subject: Re: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes On 25/05/2020 09:26, Yang Zhang wrote: > In jdk master, what we need to do is that writing m4 file for existing > vector instructions and placed them to a new file aarch64_neon.ad. > If no question, I will do it right away. I'm not entirely sure that such a change is necessary now. In particular, reorganizing the existing vector instructions is IMO excessive, but I admit that it might be an improvement. But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? It'd help if this was possible. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From gnu.andrew at redhat.com Wed Jun 24 20:25:30 2020 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 24 Jun 2020 21:25:30 +0100 Subject: RFR[15] JDK-8243114: Implement montgomery{Multiply, Square}intrinsics on Windows In-Reply-To: <73153f4a-184c-6848-1ca2-436377536cc4@redhat.com> References: <725e8e5a-71b5-7d6e-4d3a-7c799a52ab56@redhat.com> <2442a046-6640-f759-6330-e971405f0c78@oracle.com> <6cb701cd-5652-6b9d-8051-b37adb3ce63f@oracle.com> <44d1daf2-7124-9b4b-b039-d35d1f8e96c6@oracle.com> <101b5758-3660-d2e4-e20a-ffc362907b45@redhat.com> <73153f4a-184c-6848-1ca2-436377536cc4@redhat.com> Message-ID: <4d99c712-d991-5dea-78c8-9915e6286177@redhat.com> On 22/06/2020 10:00, Andrew Haley wrote: > On 18/06/2020 19:33, Martin Balao wrote: snip... >> * L3724 >> * The last argument of 'sub' has type 'int', while in the not-Windows >> variant is a long. Can we align this? > > We should do that, yes. Better it be long everywhere. > Patch looks good to me too. Simon, if you are happy with changing the final argument of sub to long on the Windows side, I'll sponsor this patch, make that minor change and push: https://cr.openjdk.java.net/~stooke/webrevs/jdk-8243114-jdk/01/01/jdk.patch to jdk/jdk Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From serguei.spitsyn at oracle.com Thu Jun 25 02:58:01 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 24 Jun 2020 19:58:01 -0700 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> Message-ID: <1ef866ec-6207-b6fe-8463-bec9a0a0b982@oracle.com> Hi Christian, The fix looks good to me. Nice catch from Coleen. Thanks, Serguei On 6/24/20 06:17, Christian Hagedorn wrote: > Hi Coleen, hi Vladimir > > Thank you for your reviews! > > On 23.06.20 22:04, coleen.phillimore at oracle.com wrote: >> I know less about this code than Erik and Vladimir but would you need >> to also test for this condition in the b) case? >> >> ??754 !old_method->method_holder()->is_loader_alive() || >> >> >> You probably won't get as much racy behavior from class unloading >> though. > > That's a valid concern. I discussed it with Erik ?. and he think we > need to add that together with old_method == NULL like in the a) case. > Here is a detailed explanation from Erik: > > "Concurrent class unloading clears the method, so it can be NULL. > There is a narrow race when we are concurrently unloading nmethods, > while others are loaded. The scenario is that we have an optimized > virtual call, materialized kind of like a static call, assuming that > there is only a single implementor. When the single implementor is > unloaded, the caller nmethod will get deoptimized when the GC gets to > it. But before that happens, we can have calls going through that call > site. If before the call, a second implementor gets loaded, I think it > is possible that the new loaded class gets installed as still the only > subclass (single implementor) of the abstract thing. Then, I don't > think the class loading triggers CHA deoptimization of the caller > nmethod. So I think then you can call through that nmethod to an > alternative callee nmethod, due to a race between class unloading, > class loading, compilation and calling an optimized virtual call." > > I updated this in a new webrev accordingly: > http://cr.openjdk.java.net/~chagedorn/8245128/webrev.01/ > > Best regards, > Christian > > >> Otherwise, the change makes sense. >> thanks, >> Coleen >> >> On 6/23/20 12:31 PM, Vladimir Kozlov wrote: >>> Good. >>> >>> Thanks, >>> Vladimir >>> >>> On 6/23/20 6:58 AM, Christian Hagedorn wrote: >>>> Hi >>>> >>>> Please review the following patch: >>>> https://bugs.openjdk.java.net/browse/JDK-8245128 >>>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >>>> >>>> The assert is often hit when enabling the Kitchensink >>>> instrumentation module which triggers a lot of class redefinitions. >>>> The problem looks similar to the one fixed in JDK-8225681 [1] for >>>> the other a) MT-unsafe assert. We could be dealing with an old >>>> method which we should also exclude in the second b) MT-unsafe >>>> assert (JDK-8225681 fixed it only for a)). A nice description of >>>> the problem is found in the comment [2] by Erik ?. >>>> >>>> Applying this fix, the assert is not hit anymore with repeated >>>> Kitchensink runs with the instrumentation module enabled. >>>> >>>> Thank you! >>>> >>>> Best regards, >>>> Christian >>>> >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >>>> [2] >>>> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >>>> >> From kim.barrett at oracle.com Thu Jun 25 03:50:05 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 24 Jun 2020 23:50:05 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> <5EE5AB71.20505@oracle.com> Message-ID: <3FB6F2CB-1085-4F1B-BFE5-7554F2637004@oracle.com> > On Jun 24, 2020, at 7:01 AM, Magnus Ihse Bursie wrote: > > On 2020-06-18 08:34, Kim Barrett wrote: >>> On Jun 15, 2020, at 7:41 AM, Kim Barrett wrote: >>> >>>> On Jun 14, 2020, at 12:45 AM, Philip Race wrote: >>>> >>>> Kim, >>>> >>>> >>>> Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. >>>> Please rename the JEP. >> After some off-list discussions I have a better idea of what Phil was >> asking for and why. In response I've changed the JEP, replacing a few >> occurrences of "HotSpot" with "the JDK", as described below. All >> other occurrences of "HotSpot" have been left as-is. >> >> Title: >> JEP 347: Adopt C++14 Language Features in HotSpot >> => >> JEP 347: Adopt C++14 Language Features in the JDK >> >> Summary: >> Allow the use of selected C++14 language features in the HotSpot C++ source code. >> => >> Allow the use of selected C++14 language features in the JDK C++ source code. >> >> Goals: >> The purpose of this JEP is to formally allow C++ source code changes within HotSpot to take advantage of some C++14 standard language features. >> => >> The purpose of this JEP is to formally allow C++ source code changes within the JDK to take advantage of some C++14 standard language features. >> > This all looks good to me. > > /Magnus Thanks. From christian.hagedorn at oracle.com Thu Jun 25 06:55:01 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Thu, 25 Jun 2020 08:55:01 +0200 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <004a4ac2-125f-0d63-b173-1b8742be9701@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> <6f6c964e-9991-8c31-3946-4e5f094a76ab@oracle.com> <004a4ac2-125f-0d63-b173-1b8742be9701@oracle.com> Message-ID: Thank you Erik and Vladimir for reviewing it again! Best regards, Christian On 24.06.20 17:55, Vladimir Kozlov wrote: > +1 > > thanks, > Vladimir > > On 6/24/20 7:32 AM, Erik ?sterlund wrote: >> Hi Christian, >> >> Looks good. >> >> Thanks, >> /Erik >> >> On 2020-06-24 15:17, Christian Hagedorn wrote: >>> Hi Coleen, hi Vladimir >>> >>> Thank you for your reviews! >>> >>> On 23.06.20 22:04, coleen.phillimore at oracle.com wrote: >>>> I know less about this code than Erik and Vladimir but would you >>>> need to also test for this condition in the b) case? >>>> >>>> ??754 !old_method->method_holder()->is_loader_alive() || >>>> >>>> >>>> You probably won't get as much racy behavior from class unloading >>>> though. >>> >>> That's a valid concern. I discussed it with Erik ?. and he think we >>> need to add that together with old_method == NULL like in the a) >>> case. Here is a detailed explanation from Erik: >>> >>> "Concurrent class unloading clears the method, so it can be NULL. >>> There is a narrow race when we are concurrently unloading nmethods, >>> while others are loaded. The scenario is that we have an optimized >>> virtual call, materialized kind of like a static call, assuming that >>> there is only a single implementor. When the single implementor is >>> unloaded, the caller nmethod will get deoptimized when the GC gets to >>> it. But before that happens, we can have calls going through that >>> call site. If before the call, a second implementor gets loaded, I >>> think it is possible that the new loaded class gets installed as >>> still the only subclass (single implementor) of the abstract thing. >>> Then, I don't think the class loading triggers CHA deoptimization of >>> the caller nmethod. So I think then you can call through that nmethod >>> to an alternative callee nmethod, due to a race between class >>> unloading, class loading, compilation and calling an optimized >>> virtual call." >>> >>> I updated this in a new webrev accordingly: >>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.01/ >>> >>> Best regards, >>> Christian >>> >>> >>>> Otherwise, the change makes sense. >>>> thanks, >>>> Coleen >>>> >>>> On 6/23/20 12:31 PM, Vladimir Kozlov wrote: >>>>> Good. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 6/23/20 6:58 AM, Christian Hagedorn wrote: >>>>>> Hi >>>>>> >>>>>> Please review the following patch: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8245128 >>>>>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >>>>>> >>>>>> The assert is often hit when enabling the Kitchensink >>>>>> instrumentation module which triggers a lot of class >>>>>> redefinitions. The problem looks similar to the one fixed in >>>>>> JDK-8225681 [1] for the other a) MT-unsafe assert. We could be >>>>>> dealing with an old method which we should also exclude in the >>>>>> second b) MT-unsafe assert (JDK-8225681 fixed it only for a)). A >>>>>> nice description of the problem is found in the comment [2] by >>>>>> Erik ?. >>>>>> >>>>>> Applying this fix, the assert is not hit anymore with repeated >>>>>> Kitchensink runs with the instrumentation module enabled. >>>>>> >>>>>> Thank you! >>>>>> >>>>>> Best regards, >>>>>> Christian >>>>>> >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >>>>>> [2] >>>>>> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >>>>>> >>>> >> From christian.hagedorn at oracle.com Thu Jun 25 06:55:31 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Thu, 25 Jun 2020 08:55:31 +0200 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <26df478c-a252-a7ef-3441-2893c64b3139@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> <26df478c-a252-a7ef-3441-2893c64b3139@oracle.com> Message-ID: <4df9b211-5406-09d4-dbcd-8688ba2e015f@oracle.com> Thank you Coleen for your review! Best regards, Christian On 24.06.20 17:43, coleen.phillimore at oracle.com wrote: > > 01 looks good. > Thanks for checking out the unloading case. > > Coleen > > On 6/24/20 9:17 AM, Christian Hagedorn wrote: >> Hi Coleen, hi Vladimir >> >> Thank you for your reviews! >> >> On 23.06.20 22:04, coleen.phillimore at oracle.com wrote: >>> I know less about this code than Erik and Vladimir but would you need >>> to also test for this condition in the b) case? >>> >>> ??754 !old_method->method_holder()->is_loader_alive() || >>> >>> >>> You probably won't get as much racy behavior from class unloading >>> though. >> >> That's a valid concern. I discussed it with Erik ?. and he think we >> need to add that together with old_method == NULL like in the a) case. >> Here is a detailed explanation from Erik: >> >> "Concurrent class unloading clears the method, so it can be NULL. >> There is a narrow race when we are concurrently unloading nmethods, >> while others are loaded. The scenario is that we have an optimized >> virtual call, materialized kind of like a static call, assuming that >> there is only a single implementor. When the single implementor is >> unloaded, the caller nmethod will get deoptimized when the GC gets to >> it. But before that happens, we can have calls going through that call >> site. If before the call, a second implementor gets loaded, I think it >> is possible that the new loaded class gets installed as still the only >> subclass (single implementor) of the abstract thing. Then, I don't >> think the class loading triggers CHA deoptimization of the caller >> nmethod. So I think then you can call through that nmethod to an >> alternative callee nmethod, due to a race between class unloading, >> class loading, compilation and calling an optimized virtual call." >> >> I updated this in a new webrev accordingly: >> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.01/ >> >> Best regards, >> Christian >> >> >>> Otherwise, the change makes sense. >>> thanks, >>> Coleen >>> >>> On 6/23/20 12:31 PM, Vladimir Kozlov wrote: >>>> Good. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 6/23/20 6:58 AM, Christian Hagedorn wrote: >>>>> Hi >>>>> >>>>> Please review the following patch: >>>>> https://bugs.openjdk.java.net/browse/JDK-8245128 >>>>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >>>>> >>>>> The assert is often hit when enabling the Kitchensink >>>>> instrumentation module which triggers a lot of class redefinitions. >>>>> The problem looks similar to the one fixed in JDK-8225681 [1] for >>>>> the other a) MT-unsafe assert. We could be dealing with an old >>>>> method which we should also exclude in the second b) MT-unsafe >>>>> assert (JDK-8225681 fixed it only for a)). A nice description of >>>>> the problem is found in the comment [2] by Erik ?. >>>>> >>>>> Applying this fix, the assert is not hit anymore with repeated >>>>> Kitchensink runs with the instrumentation module enabled. >>>>> >>>>> Thank you! >>>>> >>>>> Best regards, >>>>> Christian >>>>> >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >>>>> [2] >>>>> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >>>>> >>> > From christian.hagedorn at oracle.com Thu Jun 25 06:56:20 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Thu, 25 Jun 2020 08:56:20 +0200 Subject: [15] RFR(XS): 8245128: Kitchensink fails with: assert(destination == (address)-1 || destination == entry) failed: b) MT-unsafe modification of inline cache In-Reply-To: <1ef866ec-6207-b6fe-8463-bec9a0a0b982@oracle.com> References: <81746ca5-6199-8ecd-7820-fd19ee5f1b95@oracle.com> <60127ab5-990d-ba70-38c4-5363c14d4c6c@oracle.com> <19dacd15-0e5a-462a-d819-bfc23a309169@oracle.com> <1b97e88a-2f87-8b8c-5db5-0a486c4e7a78@oracle.com> <1ef866ec-6207-b6fe-8463-bec9a0a0b982@oracle.com> Message-ID: Thank you Serguei for your review! Best regards, Christian On 25.06.20 04:58, serguei.spitsyn at oracle.com wrote: > Hi Christian, > > The fix looks good to me. > Nice catch from Coleen. > > Thanks, > Serguei > > > On 6/24/20 06:17, Christian Hagedorn wrote: >> Hi Coleen, hi Vladimir >> >> Thank you for your reviews! >> >> On 23.06.20 22:04, coleen.phillimore at oracle.com wrote: >>> I know less about this code than Erik and Vladimir but would you need >>> to also test for this condition in the b) case? >>> >>> ??754 !old_method->method_holder()->is_loader_alive() || >>> >>> >>> You probably won't get as much racy behavior from class unloading >>> though. >> >> That's a valid concern. I discussed it with Erik ?. and he think we >> need to add that together with old_method == NULL like in the a) case. >> Here is a detailed explanation from Erik: >> >> "Concurrent class unloading clears the method, so it can be NULL. >> There is a narrow race when we are concurrently unloading nmethods, >> while others are loaded. The scenario is that we have an optimized >> virtual call, materialized kind of like a static call, assuming that >> there is only a single implementor. When the single implementor is >> unloaded, the caller nmethod will get deoptimized when the GC gets to >> it. But before that happens, we can have calls going through that call >> site. If before the call, a second implementor gets loaded, I think it >> is possible that the new loaded class gets installed as still the only >> subclass (single implementor) of the abstract thing. Then, I don't >> think the class loading triggers CHA deoptimization of the caller >> nmethod. So I think then you can call through that nmethod to an >> alternative callee nmethod, due to a race between class unloading, >> class loading, compilation and calling an optimized virtual call." >> >> I updated this in a new webrev accordingly: >> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.01/ >> >> Best regards, >> Christian >> >> >>> Otherwise, the change makes sense. >>> thanks, >>> Coleen >>> >>> On 6/23/20 12:31 PM, Vladimir Kozlov wrote: >>>> Good. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 6/23/20 6:58 AM, Christian Hagedorn wrote: >>>>> Hi >>>>> >>>>> Please review the following patch: >>>>> https://bugs.openjdk.java.net/browse/JDK-8245128 >>>>> http://cr.openjdk.java.net/~chagedorn/8245128/webrev.00/ >>>>> >>>>> The assert is often hit when enabling the Kitchensink >>>>> instrumentation module which triggers a lot of class redefinitions. >>>>> The problem looks similar to the one fixed in JDK-8225681 [1] for >>>>> the other a) MT-unsafe assert. We could be dealing with an old >>>>> method which we should also exclude in the second b) MT-unsafe >>>>> assert (JDK-8225681 fixed it only for a)). A nice description of >>>>> the problem is found in the comment [2] by Erik ?. >>>>> >>>>> Applying this fix, the assert is not hit anymore with repeated >>>>> Kitchensink runs with the instrumentation module enabled. >>>>> >>>>> Thank you! >>>>> >>>>> Best regards, >>>>> Christian >>>>> >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225681 >>>>> [2] >>>>> https://bugs.openjdk.java.net/browse/JDK-8225681?focusedCommentId=14278441&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14278441 >>>>> >>> > From kim.barrett at oracle.com Thu Jun 25 09:55:23 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Jun 2020 05:55:23 -0400 Subject: RFR: 8247747: Remove macro definition of "inline" in stack.hpp Message-ID: Please review this change to the Stack utility class, removing the definition of "inline" as a macro with an empty expansion. I also made a couple micro-optimizations to the push and pop functions, and made pop_segment NOINLINE, being consistent with the similar slow path push_segment. Making this change didn't produce any measurable difference in the performance tests I ran. This suggests the overflow part of the task queues is either not *too* heavily used, or maybe the effect of this is being drowned by other costs. CR: https://bugs.openjdk.java.net/browse/JDK-8247747 Webrev: https://cr.openjdk.java.net/~kbarrett/8247747/open.00/ Testing: mach5 tier1 From kim.barrett at oracle.com Thu Jun 25 10:39:16 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Jun 2020 06:39:16 -0400 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <582f6154-97d1-e8f6-779d-2f5ef5e489fc@oracle.com> References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> <0250fab1-c743-7077-0485-2d188609cb42@oracle.com> <582f6154-97d1-e8f6-779d-2f5ef5e489fc@oracle.com> Message-ID: <3F62BDA3-7D84-4226-839B-A37687D47918@oracle.com> > On Jun 24, 2020, at 7:07 AM, Leo Korinth wrote: > > > > On 15/06/2020 06:00, David Holmes wrote: >> Hi Leo, >> Thanks for undertaking this effort! >> Some higher-level comments at this stage just looking at the introduction of AtomicValue. >> As Kim has pointed out it may not always be appropriate to change all fields subject to atomic operations to be converted to AtomicValue. There is also the issue of fields accessed in a mix of locked and lock-free mechanisms (particularly in the runtime where we have locked writes and lock-free reads in a number of places, though sometimes combined with atomic updates). If such fields are converted to AtomicValue then I think we may still need a way to perform some raw operations on them; or else we chose to leave them as currently defined and apply Atomic ops directly when needed. (There is a parallel in the Java APIs where we can declare a field as an AtomicX variable, or else we can declare the field simply as X, and use an AtomicFieldUpdater to perform the atomic operations.) > > Yes, I am not trying to remove usage of the AllStatic Atomic API, I am trying to remove the usage of volatile. Where performance is more important than type safety (or if it feels awkward), just continue use the Atomic API (see part2 of the change), but I see /no/ need for using volatile. No Atomic operations need the volatile qualifier, and (if I understand correctly) volatile is generally not what you want as it does not guarantee enough from the language for much anything outside writing to IO mapped devices. I don't think the primary point of this change should be "eliminating volatile". There are places where we're not going to be able to use this new class that still need "volatile-ish" treatment. So it should have the effect of (significantly) reducing the use of volatile for what is effectively documentation, but I don't see elimination being possible. While the standard doesn't make any guarantees about word tearing for volatile accesses, there are platform guarantees for word-size or below and properly aligned and not weird memory cases (in other words, any case interesting to us). And indeed, there are lots of hardware access APIs that would be completely broken otherwise. And we need to make assumptions along those lines about volatile in order to implement the Atomic class's APIs, since we're not generally using "real" atomics provided by the compiler and language. (And for reasons discussed elsewhere, that seems unlikely to change.) My view of this change is that it uses the compiler to help keep us honest about what we're doing, making atomic operations explicit (such as always needing load/store operations), as well as making "non-atomic" operations on otherwise atomic variables explicit. For example, not having increment and decrement operators means we can't unintentionally perform operations that aren't what we intend. > However when I started to remove volatile in gc code that /mostly/ works as documentation, I was afraid of changing the behaviour of the code. Using AtomicValue is a great way to do refactoring code and conserve behaviour. The two part conversion (part1 and part2) of my change shows how AtomicValue is a great way to remove volatile (part1), and in part2 revert to use the AllStatic Atomic API in certain places without forgetting converting implicit volatile load and stores or continuing to use non-atomic operations such as ++ on variables (now not volatile) by mistake. > >> I like to imagine AtomicValue as a library-based alternative to a true "atomic" keyword in a language, but I'm torn between conciseness of expression and clarity of action. If we had an atomic keyword then: >> atomic int x = 0; >> ... >> x = 3; // this is an atomic store >> x++; // this is an atomic post-increment >> which would then argue for: >> AtomicValue x = 0; >> ... >> x = 3; // this is an atomic store >> x++; // this is an atomic post-increment >> rather than the explicit >> x.store(3); // this is an atomic store >> x.inc(); // this is an atomic post-increment >> in particular having explicit load() and store() operations makes for very unwieldy expressions IMHO. > >> That said I know some people prefer it if atomic operations stand out very clearly. > > Many also dislike operator overloading (although I personally like it, I think I am in a minority), if nothing else, I believe it hard to get through review. I /do/ like that the current verbose explicit way looks almost the same as the Atomic API we already have. Implicit load and stores also would "look" different from what we have today. I am unsure about what I really prefer, and keeping the look seems good and my thought was that it would offend the least? I think operator overloading has benefits in some places, but I think this is not one of those places. If we were going to overload operators here I would suggest using the strongest ordering constraints available, rather than the weakest, which is effectively what we have today. I think having the syntactically most convenient form be the least safe (as in the current state) is not a great idea. And I'm definitely in the camp that prefers atomic operations to stand out. >> So I think we need to get a general consensus on what shape AtomicValue should take and exactly how we propose to use it (or not) across all of hotspot. > > >> And perhaps, as Kim alluded, the addition of AtomicValue should be split out from its application to any particular subsystem. > > I do not want to push others to use it all over hotspot. Maybe not have a policy, but use it if it makes sense? Maybe just move it out of atomics.hpp as Coleen suggest is enough? > > My thinking is that we should reduce usage of volatile all over hotspot (regardless of AtomicValue); I do not want AtomicValue to be a forced way though. I think that at this point a substantial fraction (perhaps even the majority) of the remaining uses of volatile outside GC code are exactly the use-case for this new class. And I think the remainder are mostly not removable. (I say "mostly" because there may be some that actually don't require any special treatment. Though I think the last of the volatile member qualifiers has been eliminated, and I haven't found any remaining volatile parameters, which never made sense.) I think it would be bad to have a bifurcation of GC code dealing with atomics (mostly) one way and the rest of HotSpot using a different approach. (Even if we could always criply agree on where the boundary is.) From lois.foltan at oracle.com Thu Jun 25 16:15:02 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 25 Jun 2020 12:15:02 -0400 Subject: RFR: 8247747: Remove macro definition of "inline" in stack.hpp In-Reply-To: References: Message-ID: Looks good Kim!? I like your changes to the push and pop functions. Lois On 6/25/2020 5:55 AM, Kim Barrett wrote: > Please review this change to the Stack utility class, removing the > definition of "inline" as a macro with an empty expansion. I also > made a couple micro-optimizations to the push and pop functions, and > made pop_segment NOINLINE, being consistent with the similar slow path > push_segment. > > Making this change didn't produce any measurable difference in the > performance tests I ran. This suggests the overflow part of the task > queues is either not *too* heavily used, or maybe the effect of this > is being drowned by other costs. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8247747 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8247747/open.00/ > > Testing: > mach5 tier1 > From thomas.schatzl at oracle.com Thu Jun 25 16:19:56 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Jun 2020 18:19:56 +0200 Subject: [Urgent] RFR (XXS): 8248329: JDK-8244603 breaks windows build Message-ID: <57325be8-2d05-101e-4968-365f722ad4b0@oracle.com> Hi all, can I have reviews for this small change that fixes the Windows-build JDK-8244603 broke. (It's also a bug for the other platforms but the compilers do not complain; it would have been fixed in an upcoming change as when I noticed that bug, I fixed it in a patch further up in my mq not part of this push. This is why it slipped through). CR: https://bugs.openjdk.java.net/browse/JDK-8248329 Webrev: http://cr.openjdk.java.net/~tschatzl/8248329/webrev/ Testing: tier-1 passed Sorry for the issue, Thomas From sundararajan.athijegannathan at oracle.com Thu Jun 25 16:33:21 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Thu, 25 Jun 2020 22:03:21 +0530 Subject: [Urgent] RFR (XXS): 8248329: JDK-8244603 breaks windows build In-Reply-To: <57325be8-2d05-101e-4968-365f722ad4b0@oracle.com> References: <57325be8-2d05-101e-4968-365f722ad4b0@oracle.com> Message-ID: Looks good. -Sundar On 25/06/20 9:49 pm, Thomas Schatzl wrote: > Hi all, > > ? can I have reviews for this small change that fixes the > Windows-build JDK-8244603 broke. > > (It's also a bug for the other platforms but the compilers do not > complain; it would have been fixed in an upcoming change as when I > noticed that bug, I fixed it in a patch further up in my mq not part > of this push. This is why it slipped through). > > CR: > https://bugs.openjdk.java.net/browse/JDK-8248329 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8248329/webrev/ > Testing: > tier-1 passed > > Sorry for the issue, > ? Thomas From daniel.daugherty at oracle.com Thu Jun 25 16:56:24 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 25 Jun 2020 12:56:24 -0400 Subject: [Urgent] RFR (XXS): 8248329: JDK-8244603 breaks windows build In-Reply-To: <57325be8-2d05-101e-4968-365f722ad4b0@oracle.com> References: <57325be8-2d05-101e-4968-365f722ad4b0@oracle.com> Message-ID: <59c6401c-b719-c568-007f-b9c177c8b136@oracle.com> On 6/25/20 12:19 PM, Thomas Schatzl wrote: > Hi all, > > ? can I have reviews for this small change that fixes the > Windows-build JDK-8244603 broke. > > (It's also a bug for the other platforms but the compilers do not > complain; it would have been fixed in an upcoming change as when I > noticed that bug, I fixed it in a patch further up in my mq not part > of this push. This is why it slipped through). > > CR: > https://bugs.openjdk.java.net/browse/JDK-8248329 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8248329/webrev/ src/hotspot/share/gc/g1/g1Policy.cpp ??? No comments. Thumbs up! I consider this a trivial change so you do not have to wait for 24 hours to push. Dan > Testing: > tier-1 passed > > Sorry for the issue, > ? Thomas From thomas.schatzl at oracle.com Thu Jun 25 17:00:03 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Jun 2020 19:00:03 +0200 Subject: [Urgent] RFR (XXS): 8248329: JDK-8244603 breaks windows build In-Reply-To: <59c6401c-b719-c568-007f-b9c177c8b136@oracle.com> References: <57325be8-2d05-101e-4968-365f722ad4b0@oracle.com> <59c6401c-b719-c568-007f-b9c177c8b136@oracle.com> Message-ID: Hi Dan, Sundar, thanks for your reviews (Strangely I have only received Dan's response yet, but not Sundar's nor the original message). Pushed. Thanks, Thomas On 25.06.20 18:56, Daniel D. Daugherty wrote: > On 6/25/20 12:19 PM, Thomas Schatzl wrote: >> Hi all, >> >> ? can I have reviews for this small change that fixes the >> Windows-build JDK-8244603 broke. >> >> (It's also a bug for the other platforms but the compilers do not >> complain; it would have been fixed in an upcoming change as when I >> noticed that bug, I fixed it in a patch further up in my mq not part >> of this push. This is why it slipped through). >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8248329 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8248329/webrev/ > > src/hotspot/share/gc/g1/g1Policy.cpp > ??? No comments. > > Thumbs up! I consider this a trivial change so you do not have to wait > for 24 hours to push. > > Dan > > > >> Testing: >> tier-1 passed >> >> Sorry for the issue, >> ? Thomas > From thomas.schatzl at oracle.com Thu Jun 25 17:28:01 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Jun 2020 19:28:01 +0200 Subject: RFR: 8247747: Remove macro definition of "inline" in stack.hpp In-Reply-To: References: Message-ID: <7a39d965-248c-5602-a33d-865def43b2f1@oracle.com> Hi, On 25.06.20 11:55, Kim Barrett wrote: > Please review this change to the Stack utility class, removing the > definition of "inline" as a macro with an empty expansion. I also > made a couple micro-optimizations to the push and pop functions, and > made pop_segment NOINLINE, being consistent with the similar slow path > push_segment. > > Making this change didn't produce any measurable difference in the > performance tests I ran. This suggests the overflow part of the task > queues is either not *too* heavily used, or maybe the effect of this > is being drowned by other costs. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8247747 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8247747/open.00/ > > Testing: > mach5 tier1 > lgtm. Thomas From kim.barrett at oracle.com Thu Jun 25 18:50:59 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Jun 2020 14:50:59 -0400 Subject: RFR: 8247747: Remove macro definition of "inline" in stack.hpp In-Reply-To: References: Message-ID: > On Jun 25, 2020, at 12:15 PM, Lois Foltan wrote: > > Looks good Kim! I like your changes to the push and pop functions. > Lois Thanks. > On 6/25/2020 5:55 AM, Kim Barrett wrote: >> Please review this change to the Stack utility class, removing the >> definition of "inline" as a macro with an empty expansion. I also >> made a couple micro-optimizations to the push and pop functions, and >> made pop_segment NOINLINE, being consistent with the similar slow path >> push_segment. >> >> Making this change didn't produce any measurable difference in the >> performance tests I ran. This suggests the overflow part of the task >> queues is either not *too* heavily used, or maybe the effect of this >> is being drowned by other costs. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8247747 >> >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8247747/open.00/ >> >> Testing: >> mach5 tier1 From kim.barrett at oracle.com Thu Jun 25 18:51:10 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Jun 2020 14:51:10 -0400 Subject: RFR: 8247747: Remove macro definition of "inline" in stack.hpp In-Reply-To: <7a39d965-248c-5602-a33d-865def43b2f1@oracle.com> References: <7a39d965-248c-5602-a33d-865def43b2f1@oracle.com> Message-ID: > On Jun 25, 2020, at 1:28 PM, Thomas Schatzl wrote: > > Hi, > > On 25.06.20 11:55, Kim Barrett wrote: >> Please review this change to the Stack utility class, removing the >> definition of "inline" as a macro with an empty expansion. I also >> made a couple micro-optimizations to the push and pop functions, and >> made pop_segment NOINLINE, being consistent with the similar slow path >> push_segment. >> Making this change didn't produce any measurable difference in the >> performance tests I ran. This suggests the overflow part of the task >> queues is either not *too* heavily used, or maybe the effect of this >> is being drowned by other costs. >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8247747 >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8247747/open.00/ >> Testing: >> mach5 tier1 > > lgtm. > > Thomas Thanks. From leo.korinth at oracle.com Thu Jun 25 18:59:54 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 25 Jun 2020 20:59:54 +0200 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <598ae299-13c6-5713-74ea-2be5285c2c7c@oracle.com> References: <4611e386-2d3f-efaa-fdce-920c7d3d5b85@oracle.com> <7DE4D1D2-ECA8-4980-B665-781BB0E36B2F@oracle.com> <598ae299-13c6-5713-74ea-2be5285c2c7c@oracle.com> Message-ID: On 15/06/2020 14:39, coleen.phillimore at oracle.com wrote: > > I like the concept of this.? I really want the compiler to do to the > work to verify that if something is interestingly concurrent, we > shouldn't use plain loads and stores of it. > > The name AtomicValue<> seems as good as any.? Can AtomicValue be > declared in its own header file: runtime/atomicValue.hpp please? Yes, I think that is a good idea, and when I move it and revert changes in atomics.hpp that Kim disliked, atomics will be untouched I think. I will incorporate this in the next version, thanks! > > On 6/14/20 4:51 AM, Kim Barrett wrote: >>> On Jun 10, 2020, at 8:46 AM, Leo Korinth wrote: >>> >>> Hi, could I have a review for this change that adds AtomicValue<> to >>> atomic.hpp and uses it in G1? >>> >>> I am adding an AtomicValue<> to atomic.hpp. This is an opaque type >>> (the "value" is private) and that protect against non-atomic >>> operations being used by mistake. AtomicValue methods are proxied to >>> the corresponding (all-static) Atomic:: methods. All operations are >>> explicitly atomic and in a type safe manner with semantics defined in >>> enum atomic_memory_order >>> (memory_order_conservative by default). >>> >>> Instead of using field variables as volatile, I change them to be of >>> AtomicValue<> type. No need to verify that += (for example) will >>> result in an atomic instruction on all supported compilers. >>> >>> I have some open questions regarding the exported fields in >>> vmStructs_g1.hpp. Today, volatile fields are sometimes "exported" as >>> volatile (and sometimes not, and I guess this is by mistake). I >>> choose to export them all as non-volatile. From what I can see the >>> volatile specific part only casts to void* (only documentation?). >>> Java code is unchanged and still access them as the unwrapped values >>> (static assert in atomic.hpp guarantees that memory layout is the >>> same for T and AtomicValue). I think this is okay, but would like >>> feedback on all this. > > The SA doesn't really need anything to be volatile. I think they are > there to get vmStructs to compile for volatile fields but don't have any > meaning for SA since it is used to look at core files or unresponsive > VMs.? I don't see the void* cast that you are referring to in the > macros, which doesn't mean it's not there.? The SA seems to read these > values as their underlying types so if you don't get errors in > test/hotspot/jtreg/serviceability/sa, the changes are fine.? Hopefully > you won't have to add AtomicValue to SA. My changes seem to work if they do not fail in any higher tier (>3). The cast I was talking about is the (void*) cast in GENERATE_STATIC_PTR_VOLATILE_VM_STRUCT_ENTRY in vmStructs.hpp. The macro seems redundant; it was used inconsistently. Volatiles fields sometimes uses volatile_nonstatic_field() and sometimes nonstatic_field() --- if the latter, passing volatile as part of the type argument. Maybe sometime there were plans to do something more than (pointlessly?) cast to void*? Or maybe I am just reading the code badly... Thanks, Leo > > thanks, > Coleen > From vladimir.kozlov at oracle.com Thu Jun 25 22:36:18 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 25 Jun 2020 15:36:18 -0700 Subject: [16] RFR(XS) 8248347: windows build broken by JDK-8243114 Message-ID: <7d677d07-2a3d-5922-7d0b-7f2d43e2174a@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8248347 I used ULLONG_MAX instead of -1ULL [1]. I ran tier1. Tests still run on MacOS. Linux and Win are finished. Builds passed on all our supported x64 OSs: Linux, Win, MacOS. Tier1 runs MontgomeryMultiplyTest.java test - it passed on Win. Thanks, Vladimir [1] fix diff -r 315169f1f73a src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp --- a/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp @@ -3770,7 +3770,7 @@ julong t0 = 0, t1 = 0, t2 = 0; // Triple-precision accumulator int i; - assert(inv * n[0] == -1ULL, "broken inverse in Montgomery multiply"); + assert(inv * n[0] == ULLONG_MAX, "broken inverse in Montgomery multiply"); for (i = 0; i < len; i++) { int j; @@ -3812,7 +3812,7 @@ julong t0 = 0, t1 = 0, t2 = 0; // Triple-precision accumulator int i; - assert(inv * n[0] == -1ULL, "broken inverse in Montgomery square"); + assert(inv * n[0] == ULLONG_MAX, "broken inverse in Montgomery square"); for (i = 0; i < len; i++) { int j; From matthias.baesken at sap.com Fri Jun 26 07:06:47 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 26 Jun 2020 07:06:47 +0000 Subject: RFR [XS]: 8248334: hs build errors on ppc64 and s390x platforms Message-ID: Hello, please review this small patch that fixes the ppc64(le) and s390x builds . ( They started to break since the 24th June, looks like some inclusions were missing after recent HS changes ) Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8248334 http://cr.openjdk.java.net/~mbaesken/webrevs/8248334.0/ Thanks, Matthias From david.holmes at oracle.com Fri Jun 26 07:56:17 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 26 Jun 2020 17:56:17 +1000 Subject: RFR [XS]: 8248334: hs build errors on ppc64 and s390x platforms In-Reply-To: References: Message-ID: <894df124-c84e-1634-7593-d2f74cd34f3c@oracle.com> Hi Matthias, That all seems fine to me. David On 26/06/2020 5:06 pm, Baesken, Matthias wrote: > Hello, please review this small patch that fixes the ppc64(le) and s390x builds . > ( They started to break since the 24th June, looks like some inclusions were missing after recent HS changes ) > > > Bug/webrev : > https://bugs.openjdk.java.net/browse/JDK-8248334 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8248334.0/ > > > Thanks, Matthias > > > From goetz.lindenmaier at sap.com Fri Jun 26 08:55:15 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 26 Jun 2020 08:55:15 +0000 Subject: RFR [XS]: 8248334: hs build errors on ppc64 and s390x platforms Message-ID: Hi, I didn't get the RFR mails. Probably the known hotspot-dev Mailing List Problem? But I saw the RFR in the pipermail archive. I had a look at the change. It looks good, and I consider it trivial. Best regards, Goetz. From martin.doerr at sap.com Fri Jun 26 09:14:03 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 26 Jun 2020 09:14:03 +0000 Subject: RFR [XS]: 8248334: hs build errors on ppc64 and s390x platforms In-Reply-To: <894df124-c84e-1634-7593-d2f74cd34f3c@oracle.com> References: , <894df124-c84e-1634-7593-d2f74cd34f3c@oracle.com> Message-ID: <9AC10923-7D7B-4D9B-8AA6-3A46258938C7@sap.com> Hi Matthias, Looks good. Thanks for fixing it. Best regards, Martin > Am 26.06.2020 um 09:59 schrieb David Holmes : > > ?Hi Matthias, > > That all seems fine to me. > > David > >> On 26/06/2020 5:06 pm, Baesken, Matthias wrote: >> Hello, please review this small patch that fixes the ppc64(le) and s390x builds . >> ( They started to break since the 24th June, looks like some inclusions were missing after recent HS changes ) >> Bug/webrev : >> https://bugs.openjdk.java.net/browse/JDK-8248334 >> http://cr.openjdk.java.net/~mbaesken/webrevs/8248334.0/ >> Thanks, Matthias From matthias.baesken at sap.com Fri Jun 26 10:09:53 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 26 Jun 2020 10:09:53 +0000 Subject: RFR [XS]: 8248334: hs build errors on ppc64 and s390x platforms In-Reply-To: <9AC10923-7D7B-4D9B-8AA6-3A46258938C7@sap.com> References: , <894df124-c84e-1634-7593-d2f74cd34f3c@oracle.com> <9AC10923-7D7B-4D9B-8AA6-3A46258938C7@sap.com> Message-ID: HI David and Martin, thanks for the reviews ! > >Hi Matthias, > >Looks good. Thanks for fixing it. > >Best regards, >Martin >> >> ?Hi Matthias, >> >> That all seems fine to me. >> >> David >> From kim.barrett at oracle.com Fri Jun 26 10:21:28 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Jun 2020 06:21:28 -0400 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> Message-ID: I've made updates for comments from John and Stefan. I also made a couple of terminology fixes; this document is about C++, not Java, and should use terminology accordingly. full: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01/ incr: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01.inc/ From stefan.karlsson at oracle.com Fri Jun 26 11:50:47 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 26 Jun 2020 13:50:47 +0200 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> Message-ID: Looks good. One potential cleanup. I see that you added a sentence with the spelling 'Hotspot'. I think you've done some editorial changes to original document and replaced some 'Hotspot' occurrences with 'HotSpot'. Maybe just change all occurrences to say HotSpot? Thanks, StefanK On 2020-06-26 12:21, Kim Barrett wrote: > I've made updates for comments from John and Stefan. I also made a > couple of terminology fixes; this document is about C++, not Java, > and should use terminology accordingly. > > full: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01.inc/ > From kim.barrett at oracle.com Fri Jun 26 11:58:46 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Jun 2020 07:58:46 -0400 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> Message-ID: > On Jun 26, 2020, at 7:50 AM, Stefan Karlsson wrote: > > Looks good. > > One potential cleanup. I see that you added a sentence with the spelling 'Hotspot'. I think you've done some editorial changes to original document and replaced some 'Hotspot' occurrences with 'HotSpot'. Maybe just change all occurrences to say HotSpot? Drat! As far as I can tell, HotSpot is the proper form, so I canonicalized to that. (The original document contains both forms.) Looks like I just messed up a couple, including the new one you noticed. Fixed locally. From kim.barrett at oracle.com Fri Jun 26 12:07:22 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Jun 2020 08:07:22 -0400 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> Message-ID: <729129AC-757F-4042-BA3B-64858343B680@oracle.com> > On Jun 24, 2020, at 3:01 PM, Jesper Wilhelmsson wrote: > > I agree that the Developers' Guide would be a good place for this document. There is already a discussion on guide-dev about including the Java coding style guide and there are plans to add style guides for our Markdown and Makefiles as well. > > If we can reduce the number of places one has to look to find this kind of information I think everyone will benefit in the end. That's an interesting idea. The Developers' Guide is a new Project and I haven't had much time to pay attention to it. Also, some of the early discussion and the Charter suggested to me it wasn't the place for this kind of document. For example, "It contains tutorial-style text that provides step-by-step directions..." and I would in no way describe the HotSpot Style Guide that way. But I've now found other discussion in the other direction. I think hosting this document somewhere other than the current wiki page would be beneficial for a number of reasons. One concern I have with hosting in the Developers' Guide is that the new HotSpot Style Guide is at least somewhat version specific; substantial chunks of it do not apply to earlier JDK releases (unless someone does the hard work of backporting C++14 support and forcing required compiler versions forward and such). Any idea how that might be addressed? I'm also not sure I understand how "ownership" and decisions around updates work for the Developers' Guide. There's never been a clear mechanism for updating the HotSpot Style Guide, so the new document contains a proposal for such. But that might be incompatible with the Project's processes. In particular, I think it's important that the HotSpot Style Guide "belongs to" the HotSpot Group. From kubota.yuji at gmail.com Fri Jun 26 13:10:07 2020 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Fri, 26 Jun 2020 22:10:07 +0900 Subject: RFR(L): JDK-8229517: Support for optional asynchronous/buffered logging In-Reply-To: <49CE3CD3-37D9-458E-9B8A-B12F3A478EBF@amazon.com> References: <49CE3CD3-37D9-458E-9B8A-B12F3A478EBF@amazon.com> Message-ID: Hi, I'm not a reviewer/committer, but I would appreciate it if I could receive how it is in the situation of this patch. Since our environment sometimes faced long STW caused by filesystem delay with long GC loggings at safepoint, your patch is very interesting. Especially, as "vmoperation" logging itself was long-tailed the stopped time count was not aggregated can be occurred. Your patch will help this case. Thanks, Yuji On Tue, Mar 10, 2020 at 4:06 PM Liu, Xin wrote: > > Hi, David, > > Thanks for commenting the RFR. > > Indeed, LogAsyncFlusher needs to wait async loggers to release the RW mutex, but please note that LogAsyncFlusher releases the RW mutex before it traps into filesystem writing. It's not a blocker( refer to LogAsyncFlusher::task). > Ie. The current implementation is blocking-free but not lock-free. I mentioned it in early email. I don't want to make thing complicated in early stage. Lock-free FIFO requires extensive testing. > > For a FIFO queue, there's a broad design space for it. If we only choose selective logging sites and offload them to the async buffer, it won't be contended situation. If it turns out to be a contented lock, we could change it to spinlock, priority lock or even upgrade the FIFO to lock-free implementation. > > LogAsyncFlusher is singleton. Its constructor is private. Currently, I manually initialize and delete the only instance of it. It's very subtle because there's timing issue alongside of the initialization JVM. > If we enables C++11 in the future, we can have Meyer's singleton idiom. It will be more elegant because the instance can be lazy-initialized. > > Speaking of tagset, I think it's advantage of Unified logging framework. The framework separates frond-end and backend, interface and implementation. I love it and make use of it. > A concrete class called 'LogTagSet' represents a set of tags. They are all singleton and created by LogTagSetMapping. How to emit log messages is defined in LogTagSet. They are even configurable in the runtime. > and are just two set of compatible interfaces. Under the hook, they shared the same LogTagset infrastructure. > > Yes, developers have to choose go conventional log or asynclog in build-time. I think developers have insights of their code which may cause long latency from logging. > In order to have drop-in replacement api, I didn't go the direction that provide a new tag for async buffer. So far, tags are all component names of hotspot. > > Thanks, > --lx > > > ?On 3/8/20, 10:34 PM, "David Holmes" wrote: > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > Hi, > > On 5/03/2020 8:04 pm, Liu, Xin wrote: > > Hi, hotspot reviewers, > > > > Could you take a look at my webrev? > > Bug: https://bugs.openjdk.java.net/browse/JDK-8229517 > > Webrev: https://cr.openjdk.java.net/~xliu/8229517/01/webrev/ > > > > Previously, we had discussion at runtime-dev. https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038211.html > > I would like to continue it because I redo my implementation on the tip of jdk and have made it a general solution. > > Not a review as such but a couple of comments/queries. > > The async logging is done as a periodic task that will be executed the > Watcher thread. This task requires a mutex that is also used by the > other threads doing the log operations. So potentially the watcher > thread could experience a significant delay waiting for the mutex. > > I'm not clear what the actual lifecycle of the LogAsyncFlusher instance > is. I would have expected a single instance initialized at VM startup > and cleaned up at VM exit. The code gives the appearance that this can > actually be much more dynamic with multiple instances coming and going > through the lifetime of the VM. Deletion of the instance is particularly > problematic due to the Mutex, which has to be guaranteed not to be > accessed when it gets deleted. > > > This webrev provides on apar with . Even though it doesn?t touch any existing logging calls, it?s very possible to offload any callsite of logging to the asynchronous buffer. > > My rationale is that we only need to redirect some time-critical logs to asynchronous buffer to dodge unexpected filesystem delay. Eg. g1gc has some loggings at safepoint. We mitigate those long-tailed GC latencies rooted by filesystem writing to gclog. > > So this isn't an opt-in facility at runtime, but rather something that > has to be decided a-priori by the developer. > > How does this interact with the synchronous logging that may be used for > a given set of log tags? > > Thanks, > David > ----- > > > I got feedback from my our internal users. They have a concern about my approach of dynamic memory. Indeed, I gave up ringbuffer design in previous version and turn to use a dynamic allocated linkedlist. If writing to filesystem stalls infinitely, JVM may implode because of memory usage. I plan to solve it by adding a memory quota to the FIFO of logAsyncFlusher.hpp. It will start dropping early messages of the FIFO if it has consumed too many free memories. Another potential improvement is changing the FIFO from mutex protected to lock-free. It make sense if we redirect high volume of logging messages to it and mutex becomes hot. > > > > Thanks, > > --lx > > > > > > > > From david.holmes at oracle.com Fri Jun 26 13:54:26 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 26 Jun 2020 23:54:26 +1000 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> Message-ID: <3724b7b5-c7a0-63f0-f598-8b0ed4e37696@oracle.com> Hi Kim, On 26/06/2020 8:21 pm, Kim Barrett wrote: > I've made updates for comments from John and Stefan. I also made a > couple of terminology fixes; this document is about C++, not Java, > and should use terminology accordingly. > > full: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01.inc/ I looked at the incremental. I like the x-ref to the cpp-reference docs for things like RAII and ADL and SFINAE, and it is nice that in a browser the acronyms expand when you hover over the abbreviation, but I would like to see them always expanded in text, so that if it were printed then you can see what they mean. These are normal grammatical writing rules: you introduce something in full, show the abbreviation, and there after can use the abbreviation in place of the full text. E.g. 419 #### RTTI (Runtime Type Information) 420 421 Do not use [RTTI][4] . RTTI is disabled by the build configuration for some should be: 419 #### Runtime Type Information 420 421 Do not use [Runtime Type Information][4](RTTI) . RTTI is disabled by the build configuration for some --- 483 anonoymous namespace symbols, so can't set breakpoints in them &etc. Typo: anonoymous Typo: &etc. Bad link: https://firefox-source-docs.mozilla.org/tools/lint/coding-style/coding_style_cpp.html (I'm not sure how relevant those links in the namespace section are these days but that's a different issue.) Thanks, David From aph at redhat.com Fri Jun 26 15:40:05 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 26 Jun 2020 16:40:05 +0100 Subject: RFR: 8248403: AArch64: Remove uses of kernel integer types Message-ID: <801cd876-2d3a-69c9-b139-905fe21cd52f@redhat.com> In several places the AArch64 back end uses integer types from the kernel such as u_int32_t rather than the standard C type uint64_t. These should be removed. http://cr.openjdk.java.net/~aph/8248403/ -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Fri Jun 26 15:42:39 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 26 Jun 2020 16:42:39 +0100 Subject: RFR: AArch64: Remove uses of long and unsigned long ints Message-ID: <5d1b2d34-e55d-b0c6-28c2-9c8f9b7b9048@redhat.com> All of the uses of long and unsigned long int should be replaced with more appropriate types. Where signed and unsigned 64-bit integers are required, use int64_t. Where pointers are required, use intptr_t. Where Java longs are required, use jlong. http://cr.openjdk.java.net/~aph/8248414/ -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Fri Jun 26 15:43:36 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 26 Jun 2020 16:43:36 +0100 Subject: RFR: 8248414: AArch64: Remove uses of long and unsigned long ints Message-ID: <46226794-abac-3248-85c7-f0e995f5f1eb@redhat.com> All of the uses of long and unsigned long int should be replaced with more appropriate types. Where signed and unsigned 64-bit integers are required, use int64_t. Where pointers are required, use intptr_t. Where Java longs are required, use jlong. http://cr.openjdk.java.net/~aph/8248414/ -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From kim.barrett at oracle.com Sat Jun 27 08:01:42 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 27 Jun 2020 04:01:42 -0400 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: <3724b7b5-c7a0-63f0-f598-8b0ed4e37696@oracle.com> References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> <3724b7b5-c7a0-63f0-f598-8b0ed4e37696@oracle.com> Message-ID: > On Jun 26, 2020, at 9:54 AM, David Holmes wrote: > > Hi Kim, > > On 26/06/2020 8:21 pm, Kim Barrett wrote: >> I've made updates for comments from John and Stefan. I also made a >> couple of terminology fixes; this document is about C++, not Java, >> and should use terminology accordingly. >> full: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01/ >> incr: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01.inc/ > > I looked at the incremental. I like the x-ref to the cpp-reference docs for things like RAII and ADL and SFINAE, and it is nice that in a browser the acronyms expand when you hover over the abbreviation, but I would like to see them always expanded in text, so that if it were printed then you can see what they mean. These are normal grammatical writing rules: you introduce something in full, show the abbreviation, and there after can use the abbreviation in place of the full text. E.g. > > 419 #### RTTI (Runtime Type Information) > 420 > 421 Do not use [RTTI][4] . RTTI is disabled by the build configuration for some > > should be: > > 419 #### Runtime Type Information > 420 > 421 Do not use [Runtime Type Information][4](RTTI) . RTTI is disabled by the build configuration for some Done, I think. > 483 anonoymous namespace symbols, so can't set breakpoints in them &etc. > > Typo: anonoymous > Typo: &etc. Done. I also reran spell checking. (Apparently haven?t done that for a while.) > Bad link: https://firefox-source-docs.mozilla.org/tools/lint/coding-style/coding_style_cpp.html Fixed, as well as another mozilla link that needed updating. > (I'm not sure how relevant those links in the namespace section are these days but that's a different issue.) The last time I checked carefully (a couple years ago) the discussed problems all still seemed to exist. A quick spot check today suggests there might have been improvements since then. But I?m going to leave anonymous namespaces in the out column (that was implicitly more or less the previous status quo, in that there are almost no uses in our code) and let someone who wants to propose using them do the relevant research. New webrevs: full: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.02/ incr: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.02.inc/ From jesper.wilhelmsson at oracle.com Sat Jun 27 08:37:05 2020 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Sat, 27 Jun 2020 10:37:05 +0200 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: <729129AC-757F-4042-BA3B-64858343B680@oracle.com> References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <729129AC-757F-4042-BA3B-64858343B680@oracle.com> Message-ID: > On 26 Jun 2020, at 14:07, Kim Barrett wrote: > >> On Jun 24, 2020, at 3:01 PM, Jesper Wilhelmsson wrote: >> >> I agree that the Developers' Guide would be a good place for this document. There is already a discussion on guide-dev about including the Java coding style guide and there are plans to add style guides for our Markdown and Makefiles as well. >> >> If we can reduce the number of places one has to look to find this kind of information I think everyone will benefit in the end. > > That's an interesting idea. The Developers' Guide is a new Project and > I haven't had much time to pay attention to it. Also, some of the > early discussion and the Charter suggested to me it wasn't the place > for this kind of document. For example, "It contains tutorial-style > text that provides step-by-step directions..." and I would in no way > describe the HotSpot Style Guide that way. But I've now found other > discussion in the other direction. Personally I think the coding style guides would fit perfectly in the Developers' Guide. We'll see what happens with the Java style guide, but regardless I think we can have the other language style guides in there (C/C++, Makefiles, Markdown) since these won't attract the same attention from people outside the community. > I think hosting this document somewhere other than the current wiki > page would be beneficial for a number of reasons. > > One concern I have with hosting in the Developers' Guide is that the > new HotSpot Style Guide is at least somewhat version specific; > substantial chunks of it do not apply to earlier JDK releases (unless > someone does the hard work of backporting C++14 support and forcing > required compiler versions forward and such). Any idea how that might > be addressed? I don't see that as a problem as long as the style guide clearly state that it's focused on C++14 and its applicability to older C++ versions may vary. (or something like that) If someone has the time at some point in the future we can add some C++14 label to places where the guide specifies something that is only applicable to C++14. > I'm also not sure I understand how "ownership" and decisions around > updates work for the Developers' Guide. There's never been a clear > mechanism for updating the HotSpot Style Guide, so the new document > contains a proposal for such. But that might be incompatible with the > Project's processes. In particular, I think it's important that the > HotSpot Style Guide "belongs to" the HotSpot Group. Right now we aren't that many Reviewers, but I recon the principle should be the same as for the JDK, the changes should be reviewed by someone that "belongs" to that area. So any changes to the C++ style guide should be reviewed by a set of people from the HotSpot Group - I know who to ping ;-) Hth, /Jesper From david.holmes at oracle.com Mon Jun 29 00:08:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 29 Jun 2020 10:08:52 +1000 Subject: RFC: 8247976: Update HotSpot Style Guide for C++14 adoption In-Reply-To: References: <8973C296-B60B-4FAB-90F1-2474F4F19B23@oracle.com> <1CFC36DF-5493-4389-B5A9-FD3A4F3146C5@oracle.com> <3724b7b5-c7a0-63f0-f598-8b0ed4e37696@oracle.com> Message-ID: <0f4bb5cf-9a88-95fc-f049-ce69e8a18af9@oracle.com> Hi Kim, Updates look good. One minor nit, titles should not use abbreviations that have not yet been defined e.g. #### RTTI (Runtime Type Information) should just be #### Runtime Type Information (you don't introduce abbreviations in headings/titles either). And: #### Expression SFINAE also needs fixing - though simply expanding SFINAE doesn't really make sense as it seems to be being used as a verb, which seems odd in itself. I don't know how to rewrite that title in a meaningful way. I suppose one way around this would be to say: #### Expression "SFINAE" though even that is a stretch. Thanks, David ----- On 27/06/2020 6:01 pm, Kim Barrett wrote: >> On Jun 26, 2020, at 9:54 AM, David Holmes wrote: >> >> Hi Kim, >> >> On 26/06/2020 8:21 pm, Kim Barrett wrote: >>> I've made updates for comments from John and Stefan. I also made a >>> couple of terminology fixes; this document is about C++, not Java, >>> and should use terminology accordingly. >>> full: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01/ >>> incr: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.01.inc/ >> >> I looked at the incremental. I like the x-ref to the cpp-reference docs for things like RAII and ADL and SFINAE, and it is nice that in a browser the acronyms expand when you hover over the abbreviation, but I would like to see them always expanded in text, so that if it were printed then you can see what they mean. These are normal grammatical writing rules: you introduce something in full, show the abbreviation, and there after can use the abbreviation in place of the full text. E.g. >> >> 419 #### RTTI (Runtime Type Information) >> 420 >> 421 Do not use [RTTI][4] . RTTI is disabled by the build configuration for some >> >> should be: >> >> 419 #### Runtime Type Information >> 420 >> 421 Do not use [Runtime Type Information][4](RTTI) . RTTI is disabled by the build configuration for some > > Done, I think. > >> 483 anonoymous namespace symbols, so can't set breakpoints in them &etc. >> >> Typo: anonoymous >> Typo: &etc. > > Done. I also reran spell checking. (Apparently haven?t done that for a while.) > >> Bad link: https://firefox-source-docs.mozilla.org/tools/lint/coding-style/coding_style_cpp.html > > Fixed, as well as another mozilla link that needed updating. > >> (I'm not sure how relevant those links in the namespace section are these days but that's a different issue.) > > The last time I checked carefully (a couple years ago) the discussed problems > all still seemed to exist. A quick spot check today suggests there might have been > improvements since then. But I?m going to leave anonymous namespaces in the > out column (that was implicitly more or less the previous status quo, in that there > are almost no uses in our code) and let someone who wants to propose using them > do the relevant research. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.02/ > incr: https://cr.openjdk.java.net/~kbarrett/8247976/hotspot-style.02.inc/ > From david.holmes at oracle.com Mon Jun 29 02:44:34 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 29 Jun 2020 12:44:34 +1000 Subject: RFR: 8247213: G1: Reduce usage of volatile in favour of Atomic operations In-Reply-To: <3F62BDA3-7D84-4226-839B-A37687D47918@oracle.com> References: <27810355-8ddc-ee42-4bd6-569988080f72@oracle.com> <0250fab1-c743-7077-0485-2d188609cb42@oracle.com> <582f6154-97d1-e8f6-779d-2f5ef5e489fc@oracle.com> <3F62BDA3-7D84-4226-839B-A37687D47918@oracle.com> Message-ID: Hi Leo, I am in general support of what Kim states below. I also want to see this adopted (in a suitable form) for use across the Hotspot code base in a consistent way. I can live without the convenience of operator overloading for AtomicValue. :) Cheers, David ----- On 25/06/2020 8:39 pm, Kim Barrett wrote: >> On Jun 24, 2020, at 7:07 AM, Leo Korinth wrote: >> >> >> >> On 15/06/2020 06:00, David Holmes wrote: >>> Hi Leo, >>> Thanks for undertaking this effort! >>> Some higher-level comments at this stage just looking at the introduction of AtomicValue. >>> As Kim has pointed out it may not always be appropriate to change all fields subject to atomic operations to be converted to AtomicValue. There is also the issue of fields accessed in a mix of locked and lock-free mechanisms (particularly in the runtime where we have locked writes and lock-free reads in a number of places, though sometimes combined with atomic updates). If such fields are converted to AtomicValue then I think we may still need a way to perform some raw operations on them; or else we chose to leave them as currently defined and apply Atomic ops directly when needed. (There is a parallel in the Java APIs where we can declare a field as an AtomicX variable, or else we can declare the field simply as X, and use an AtomicFieldUpdater to perform the atomic operations.) >> >> Yes, I am not trying to remove usage of the AllStatic Atomic API, I am trying to remove the usage of volatile. Where performance is more important than type safety (or if it feels awkward), just continue use the Atomic API (see part2 of the change), but I see /no/ need for using volatile. No Atomic operations need the volatile qualifier, and (if I understand correctly) volatile is generally not what you want as it does not guarantee enough from the language for much anything outside writing to IO mapped devices. > > I don't think the primary point of this change should be "eliminating > volatile". There are places where we're not going to be able to use > this new class that still need "volatile-ish" treatment. So it should > have the effect of (significantly) reducing the use of volatile for > what is effectively documentation, but I don't see elimination being > possible. > > While the standard doesn't make any guarantees about word tearing for > volatile accesses, there are platform guarantees for word-size or > below and properly aligned and not weird memory cases (in other words, > any case interesting to us). And indeed, there are lots of hardware > access APIs that would be completely broken otherwise. > > And we need to make assumptions along those lines about volatile in > order to implement the Atomic class's APIs, since we're not generally > using "real" atomics provided by the compiler and language. (And for > reasons discussed elsewhere, that seems unlikely to change.) > > My view of this change is that it uses the compiler to help keep us > honest about what we're doing, making atomic operations explicit (such > as always needing load/store operations), as well as making > "non-atomic" operations on otherwise atomic variables explicit. For > example, not having increment and decrement operators means we can't > unintentionally perform operations that aren't what we intend. > >> However when I started to remove volatile in gc code that /mostly/ works as documentation, I was afraid of changing the behaviour of the code. Using AtomicValue is a great way to do refactoring code and conserve behaviour. The two part conversion (part1 and part2) of my change shows how AtomicValue is a great way to remove volatile (part1), and in part2 revert to use the AllStatic Atomic API in certain places without forgetting converting implicit volatile load and stores or continuing to use non-atomic operations such as ++ on variables (now not volatile) by mistake. >> >>> I like to imagine AtomicValue as a library-based alternative to a true "atomic" keyword in a language, but I'm torn between conciseness of expression and clarity of action. If we had an atomic keyword then: >>> atomic int x = 0; >>> ... >>> x = 3; // this is an atomic store >>> x++; // this is an atomic post-increment >>> which would then argue for: >>> AtomicValue x = 0; >>> ... >>> x = 3; // this is an atomic store >>> x++; // this is an atomic post-increment >>> rather than the explicit >>> x.store(3); // this is an atomic store >>> x.inc(); // this is an atomic post-increment >>> in particular having explicit load() and store() operations makes for very unwieldy expressions IMHO. > >>> That said I know some people prefer it if atomic operations stand out very clearly. >> >> Many also dislike operator overloading (although I personally like it, I think I am in a minority), if nothing else, I believe it hard to get through review. I /do/ like that the current verbose explicit way looks almost the same as the Atomic API we already have. Implicit load and stores also would "look" different from what we have today. I am unsure about what I really prefer, and keeping the look seems good and my thought was that it would offend the least? > > I think operator overloading has benefits in some places, but I think > this is not one of those places. If we were going to overload > operators here I would suggest using the strongest ordering > constraints available, rather than the weakest, which is effectively > what we have today. I think having the syntactically most convenient > form be the least safe (as in the current state) is not a great idea. > And I'm definitely in the camp that prefers atomic operations to stand > out. > >>> So I think we need to get a general consensus on what shape AtomicValue should take and exactly how we propose to use it (or not) across all of hotspot. >>> >>> And perhaps, as Kim alluded, the addition of AtomicValue should be split out from its application to any particular subsystem. >> >> I do not want to push others to use it all over hotspot. Maybe not have a policy, but use it if it makes sense? Maybe just move it out of atomics.hpp as Coleen suggest is enough? >> >> My thinking is that we should reduce usage of volatile all over hotspot (regardless of AtomicValue); I do not want AtomicValue to be a forced way though. > > I think that at this point a substantial fraction (perhaps even the > majority) of the remaining uses of volatile outside GC code are > exactly the use-case for this new class. And I think the remainder are > mostly not removable. (I say "mostly" because there may be some that > actually don't require any special treatment. Though I think the last > of the volatile member qualifiers has been eliminated, and I haven't > found any remaining volatile parameters, which never made sense.) > > I think it would be bad to have a bifurcation of GC code dealing with > atomics (mostly) one way and the rest of HotSpot using a different > approach. (Even if we could always criply agree on where the boundary > is.) > From tobias.hartmann at oracle.com Mon Jun 29 07:48:09 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 29 Jun 2020 09:48:09 +0200 Subject: [16] RFR(S): 8247218: Add default constructor to VectorSet to use Thread::current()->resource_area() as arena by default In-Reply-To: <55011fe1-2307-ad66-ea39-60a3b9c00682@oracle.com> References: <55011fe1-2307-ad66-ea39-60a3b9c00682@oracle.com> Message-ID: <9a50c7f3-94d0-ad44-e26b-234ef014eba7@oracle.com> Hi Christian, Nice cleanup, looks good to too! Best regards, Tobias On 18.06.20 12:36, Christian Hagedorn wrote: > Hi > > Please review the following cleanup: > https://bugs.openjdk.java.net/browse/JDK-8247218 > http://cr.openjdk.java.net/~chagedorn/8247218/webrev.00/ > > By adding an additional default constructor in VectorSet (as also present in Node_List, Node_Stack > etc.), we can replace the often found pattern > > ? Arena* a = Thread::current()->resource_area(); > ? VectorSet visited(a); > > by a simple > > ? VectorSet visited; > > I additionally cleaned up some other places where this pattern was found, for example for Node_List > etc. > > Thank you! > > Best regards, > Christian From Yang.Zhang at arm.com Mon Jun 29 07:48:30 2020 From: Yang.Zhang at arm.com (Yang Zhang) Date: Mon, 29 Jun 2020 07:48:30 +0000 Subject: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes In-Reply-To: References: <275eb57c-51c0-675e-c32a-91b198023559@redhat.com> <719F9169-ABC4-408E-B732-F1BD9A84337F@oracle.com> <9a13f5df-d946-579d-4282-917dc7338dc8@redhat.com> <09BC0693-80E0-4F87-855E-0B38A6F5EFA2@oracle.com> <668e500e-f621-5a2c-a41e-f73536880f73@redhat.com> <1909fa9d-98bb-c2fb-45d8-540247d1ca8b@redhat.com> Message-ID: Hi Andrew, 1. Instructions that can be matched with NEON instructions directly. MulVB, SqrtVF and AbsV have been merged into jdk master already. 2. Instructions that jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. 3. Panama/Vector API specific instructions such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. I will put 2 and 3 in a new ad file aarch64_neon.ad. I will also update aarch64_asmtest.py and macroassemler.cpp. When the patch is ready, I will send it again. Hi Sandhya, Could you please help to manual merge panama vectorIntrinsics/vector-unstable to jdk master? So that I can update this patch based on latest jdk master. Regards Yang -----Original Message----- From: Viswanathan, Sandhya Sent: Thursday, June 25, 2020 3:04 AM To: Yang Zhang ; Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes Hi Andrew/Yang, We couldn?t propose Vector API to target in time for JDK 15 and hoping to do so early in JDK 16 timeframe. The implementation reviews on other components have made good progress. We have so far ok to PPT from (runtime, shared compiler changes, x86 backend). Java API implementation review is in progress. I wanted to check with you both if we have a go ahead from aarch64 backed point of view. Best Regards, Sandhya -----Original Message----- From: hotspot-compiler-dev On Behalf Of Yang Zhang Sent: Tuesday, May 26, 2020 7:59 PM To: Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes > But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? The new instructions can be classified as: 1. Instructions that can be matched with NEON instructions directly. MulVB and SqrtVF have been merged into jdk master already. The patch of AbsV is in review [1]. 2. Instructions that Jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. May I have a new patch for these? 3. Panama/Vector API specific instructions Such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. Regards Yang [1] https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-May/008861.html -----Original Message----- From: Andrew Haley Sent: Tuesday, May 26, 2020 4:25 PM To: Yang Zhang ; Paul Sandoz Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; nd Subject: Re: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes On 25/05/2020 09:26, Yang Zhang wrote: > In jdk master, what we need to do is that writing m4 file for existing > vector instructions and placed them to a new file aarch64_neon.ad. > If no question, I will do it right away. I'm not entirely sure that such a change is necessary now. In particular, reorganizing the existing vector instructions is IMO excessive, but I admit that it might be an improvement. But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? It'd help if this was possible. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From christian.hagedorn at oracle.com Mon Jun 29 08:10:57 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Mon, 29 Jun 2020 10:10:57 +0200 Subject: [16] RFR(S): 8247218: Add default constructor to VectorSet to use Thread::current()->resource_area() as arena by default In-Reply-To: <9a50c7f3-94d0-ad44-e26b-234ef014eba7@oracle.com> References: <55011fe1-2307-ad66-ea39-60a3b9c00682@oracle.com> <9a50c7f3-94d0-ad44-e26b-234ef014eba7@oracle.com> Message-ID: <1f6dd318-ee84-5d0d-daaf-6a4ea8ccc36f@oracle.com> Thank you Tobias for your review! Best regards, Christian On 29.06.20 09:48, Tobias Hartmann wrote: > Hi Christian, > > Nice cleanup, looks good to too! > > Best regards, > Tobias > > On 18.06.20 12:36, Christian Hagedorn wrote: >> Hi >> >> Please review the following cleanup: >> https://bugs.openjdk.java.net/browse/JDK-8247218 >> http://cr.openjdk.java.net/~chagedorn/8247218/webrev.00/ >> >> By adding an additional default constructor in VectorSet (as also present in Node_List, Node_Stack >> etc.), we can replace the often found pattern >> >> ? Arena* a = Thread::current()->resource_area(); >> ? VectorSet visited(a); >> >> by a simple >> >> ? VectorSet visited; >> >> I additionally cleaned up some other places where this pattern was found, for example for Node_List >> etc. >> >> Thank you! >> >> Best regards, >> Christian From aph at redhat.com Mon Jun 29 16:10:06 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 29 Jun 2020 17:10:06 +0100 Subject: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes In-Reply-To: References: <275eb57c-51c0-675e-c32a-91b198023559@redhat.com> <719F9169-ABC4-408E-B732-F1BD9A84337F@oracle.com> <9a13f5df-d946-579d-4282-917dc7338dc8@redhat.com> <09BC0693-80E0-4F87-855E-0B38A6F5EFA2@oracle.com> <668e500e-f621-5a2c-a41e-f73536880f73@redhat.com> <1909fa9d-98bb-c2fb-45d8-540247d1ca8b@redhat.com> Message-ID: <2acbcc99-8dd4-b8f1-5982-1d439953c416@redhat.com> On 29/06/2020 08:48, Yang Zhang wrote: > 1. Instructions that can be matched with NEON instructions directly. > MulVB, SqrtVF and AbsV have been merged into jdk master already. > > 2. Instructions that jdk master has middle end support for, but they cannot be matched with NEON instructions directly. > Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. > > 3. Panama/Vector API specific instructions such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. > These instructions cannot be moved into jdk master first because there isn't middle-end support. > > I will put 2 and 3 in a new ad file aarch64_neon.ad. I will also update aarch64_asmtest.py and macroassemler.cpp. When the patch is ready, I will send it again. Thank you *very* much for your hard work. Appreciated! -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From magnus.ihse.bursie at oracle.com Tue Jun 30 10:08:50 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 30 Jun 2020 12:08:50 +0200 Subject: RFR: JDK-8248548 Use DISABLED_WARNINGS for globally disabled warnings on Visual Studio in Hotspot Message-ID: Currently hotspot/share/utilities/globalDefinitions_visCPP.hpp contains a lot of #pragma warning( disable : ...). All these globally disabled warnings should move to the make files instead. I also cleaned out some versions checks that are no longer relevant for the range of supported versions of Microsoft Visual Studio. Bug: https://bugs.openjdk.java.net/browse/JDK-8248548 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8248548-use-DISABLED_WARNINGS_microsoft-for-hotspot/webrev.01 /Magnus From kim.barrett at oracle.com Tue Jun 30 10:25:32 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 30 Jun 2020 06:25:32 -0400 Subject: RFR: 8248545: Remove unneeded warning suppression of MSVC++ 4521/4522 Message-ID: <83C1C1C5-5871-44AD-BAD9-7E91EE76D1C6@oracle.com> Please review this change that removes the conditional suppression of the MSVC++ warnings 4521 and 4522. These warnings no longer get triggered by the oop class. 4521: 'class' : multiple copy constructors specified 4522: 'class' : multiple assignment operators specified These were triggered by the existence of both normal and volatile qualified copy constructors and assignment operators for the class oop, which is only defined as a class when CHECK_UNHANDLED_OOPS is true. CR: https://bugs.openjdk.java.net/browse/JDK-8248545 Webrev: https://cr.openjdk.java.net/~kbarrett/8248545/open.00/ Testing: Build all Oracle supported Windows variants, including forcing -DCHECK_UNHANDLED_OOPS in a fastdebug build. (The build system currently never enables CHECK_UNHANDLED_OOPS for Windows; I changed that for testing.) From kim.barrett at oracle.com Tue Jun 30 10:32:52 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 30 Jun 2020 06:32:52 -0400 Subject: RFR: JDK-8248548 Use DISABLED_WARNINGS for globally disabled warnings on Visual Studio in Hotspot In-Reply-To: References: Message-ID: <66E00E1F-0AC6-42FB-A514-90B8037114C1@oracle.com> > On Jun 30, 2020, at 6:08 AM, Magnus Ihse Bursie wrote: > > Currently hotspot/share/utilities/globalDefinitions_visCPP.hpp contains a lot of #pragma warning( disable : ...). > > All these globally disabled warnings should move to the make files instead. > > I also cleaned out some versions checks that are no longer relevant for the range of supported versions of Microsoft Visual Studio. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8248548 > WebRev: http://cr.openjdk.java.net/~ihse/JDK-8248548-use-DISABLED_WARNINGS_microsoft-for-hotspot/webrev.01 > > /Magnus I think it would be nice to have a comment block describing each of the disabled warnings. But thanks for sorting them, unlike the globalDefinitions code. (Even better would be to have comments describing why we?re disabling them, but we don?t do that for any other platform either.) From magnus.ihse.bursie at oracle.com Tue Jun 30 10:48:26 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 30 Jun 2020 12:48:26 +0200 Subject: RFR: JDK-8248548 Use DISABLED_WARNINGS for globally disabled warnings on Visual Studio in Hotspot In-Reply-To: <66E00E1F-0AC6-42FB-A514-90B8037114C1@oracle.com> References: <66E00E1F-0AC6-42FB-A514-90B8037114C1@oracle.com> Message-ID: On 2020-06-30 12:32, Kim Barrett wrote: >> On Jun 30, 2020, at 6:08 AM, Magnus Ihse Bursie wrote: >> >> Currently hotspot/share/utilities/globalDefinitions_visCPP.hpp contains a lot of #pragma warning( disable : ...). >> >> All these globally disabled warnings should move to the make files instead. >> >> I also cleaned out some versions checks that are no longer relevant for the range of supported versions of Microsoft Visual Studio. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8248548 >> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8248548-use-DISABLED_WARNINGS_microsoft-for-hotspot/webrev.01 >> >> /Magnus > I think it would be nice to have a comment block describing each of the disabled warnings. > But thanks for sorting them, unlike the globalDefinitions code. > > (Even better would be to have comments describing why we?re disabling them, but we don?t do > that for any other platform either.) > We used to have that for solaris. It ended up as a long document inside the code, hiding the actual code. I agree that it would be good to have a rationale on why we disable warnings, and that it should be documented. But I think the source code is the wrong location for that. Sure, non-descriptive warnings designations like microsoft has is typically more in need of documentation than the gcc-style, but the important thing is *why* it is disabled, not *what* is disabled. So I'd argue that we should not pollute the source code with lines upon lines of warning messages, but if anything, we should instead point to an external location? where we can provide not only an explanation of what the warnings does, but a rationale for disabling it. /Magnus From kim.barrett at oracle.com Tue Jun 30 10:54:58 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 30 Jun 2020 06:54:58 -0400 Subject: RFR: JDK-8248548 Use DISABLED_WARNINGS for globally disabled warnings on Visual Studio in Hotspot In-Reply-To: References: <66E00E1F-0AC6-42FB-A514-90B8037114C1@oracle.com> Message-ID: <878BDDAA-90D0-4306-8CAD-96548638DE1F@oracle.com> > On Jun 30, 2020, at 6:48 AM, Magnus Ihse Bursie wrote: > > On 2020-06-30 12:32, Kim Barrett wrote: >>> On Jun 30, 2020, at 6:08 AM, Magnus Ihse Bursie wrote: >>> >>> Currently hotspot/share/utilities/globalDefinitions_visCPP.hpp contains a lot of #pragma warning( disable : ...). >>> >>> All these globally disabled warnings should move to the make files instead. >>> >>> I also cleaned out some versions checks that are no longer relevant for the range of supported versions of Microsoft Visual Studio. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8248548 >>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8248548-use-DISABLED_WARNINGS_microsoft-for-hotspot/webrev.01 >>> >>> /Magnus >> I think it would be nice to have a comment block describing each of the disabled warnings. >> But thanks for sorting them, unlike the globalDefinitions code. >> >> (Even better would be to have comments describing why we?re disabling them, but we don?t do >> that for any other platform either.) >> > We used to have that for solaris. It ended up as a long document inside the code, hiding the actual code. I agree that it would be good to have a rationale on why we disable warnings, and that it should be documented. But I think the source code is the wrong location for that. Sure, non-descriptive warnings designations like microsoft has is typically more in need of documentation than the gcc-style, but the important thing is *why* it is disabled, not *what* is disabled. > > So I'd argue that we should not pollute the source code with lines upon lines of warning messages, but if anything, we should instead point to an external location where we can provide not only an explanation of what the warnings does, but a rationale for disabling it. > > /Magnus I can understand that. Maybe a job for the HotSpot Style Guide. Oh wait, what am I saying. Change looks good. From thomas.schatzl at oracle.com Tue Jun 30 11:49:53 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 30 Jun 2020 13:49:53 +0200 Subject: RFR: 8248545: Remove unneeded warning suppression of MSVC++ 4521/4522 In-Reply-To: <83C1C1C5-5871-44AD-BAD9-7E91EE76D1C6@oracle.com> References: <83C1C1C5-5871-44AD-BAD9-7E91EE76D1C6@oracle.com> Message-ID: <764725a7-b0ec-b396-6c50-282f7b4fb5fb@oracle.com> Hi, On 30.06.20 12:25, Kim Barrett wrote: > Please review this change that removes the conditional suppression of > the MSVC++ warnings 4521 and 4522. These warnings no longer get > triggered by the oop class > > 4521: 'class' : multiple copy constructors specified > 4522: 'class' : multiple assignment operators specified > > These were triggered by the existence of both normal and volatile > qualified copy constructors and assignment operators for the class > oop, which is only defined as a class when CHECK_UNHANDLED_OOPS is > true. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8248545 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8248545/open.00/ > > Testing: > Build all Oracle supported Windows variants, including forcing > -DCHECK_UNHANDLED_OOPS in a fastdebug build. (The build system > currently never enables CHECK_UNHANDLED_OOPS for Windows; I changed > that for testing.) > looks good and trivial. Thomas From erik.joelsson at oracle.com Tue Jun 30 14:05:44 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 30 Jun 2020 07:05:44 -0700 Subject: RFR: JDK-8248548 Use DISABLED_WARNINGS for globally disabled warnings on Visual Studio in Hotspot In-Reply-To: References: Message-ID: <49933a13-8a00-2e20-3f97-111a70f6a7b9@oracle.com> Looks good. /Erik On 2020-06-30 03:08, Magnus Ihse Bursie wrote: > Currently hotspot/share/utilities/globalDefinitions_visCPP.hpp > contains a lot of #pragma warning( disable : ...). > > All these globally disabled warnings should move to the make files > instead. > > I also cleaned out some versions checks that are no longer relevant > for the range of supported versions of Microsoft Visual Studio. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8248548 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8248548-use-DISABLED_WARNINGS_microsoft-for-hotspot/webrev.01 > > /Magnus From magnus.ihse.bursie at oracle.com Tue Jun 30 15:18:43 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 30 Jun 2020 17:18:43 +0200 Subject: RFR: 8248545: Remove unneeded warning suppression of MSVC++ 4521/4522 In-Reply-To: <83C1C1C5-5871-44AD-BAD9-7E91EE76D1C6@oracle.com> References: <83C1C1C5-5871-44AD-BAD9-7E91EE76D1C6@oracle.com> Message-ID: <7f3bbbe1-1043-1317-e5da-ffb4d4d2ed8b@oracle.com> On 2020-06-30 12:25, Kim Barrett wrote: > Please review this change that removes the conditional suppression of > the MSVC++ warnings 4521 and 4522. These warnings no longer get > triggered by the oop class. > > 4521: 'class' : multiple copy constructors specified > 4522: 'class' : multiple assignment operators specified > > These were triggered by the existence of both normal and volatile > qualified copy constructors and assignment operators for the class > oop, which is only defined as a class when CHECK_UNHANDLED_OOPS is > true. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8248545 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8248545/open.00/ Looks good. Thanks for the cleaning up! /Magnus > > Testing: > Build all Oracle supported Windows variants, including forcing > -DCHECK_UNHANDLED_OOPS in a fastdebug build. (The build system > currently never enables CHECK_UNHANDLED_OOPS for Windows; I changed > that for testing.) > From sandhya.viswanathan at intel.com Tue Jun 30 18:56:34 2020 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Tue, 30 Jun 2020 18:56:34 +0000 Subject: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes In-Reply-To: References: <275eb57c-51c0-675e-c32a-91b198023559@redhat.com> <719F9169-ABC4-408E-B732-F1BD9A84337F@oracle.com> <9a13f5df-d946-579d-4282-917dc7338dc8@redhat.com> <09BC0693-80E0-4F87-855E-0B38A6F5EFA2@oracle.com> <668e500e-f621-5a2c-a41e-f73536880f73@redhat.com> <1909fa9d-98bb-c2fb-45d8-540247d1ca8b@redhat.com> Message-ID: Hi Yang, I have merged vectorIntrinsics with changes from panama/default. Hope this helps. Best Regards, Sandhya -----Original Message----- From: Yang Zhang Sent: Monday, June 29, 2020 12:49 AM To: Viswanathan, Sandhya ; Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes Hi Andrew, 1. Instructions that can be matched with NEON instructions directly. MulVB, SqrtVF and AbsV have been merged into jdk master already. 2. Instructions that jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. 3. Panama/Vector API specific instructions such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. I will put 2 and 3 in a new ad file aarch64_neon.ad. I will also update aarch64_asmtest.py and macroassemler.cpp. When the patch is ready, I will send it again. Hi Sandhya, Could you please help to manual merge panama vectorIntrinsics/vector-unstable to jdk master? So that I can update this patch based on latest jdk master. Regards Yang -----Original Message----- From: Viswanathan, Sandhya Sent: Thursday, June 25, 2020 3:04 AM To: Yang Zhang ; Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes Hi Andrew/Yang, We couldn?t propose Vector API to target in time for JDK 15 and hoping to do so early in JDK 16 timeframe. The implementation reviews on other components have made good progress. We have so far ok to PPT from (runtime, shared compiler changes, x86 backend). Java API implementation review is in progress. I wanted to check with you both if we have a go ahead from aarch64 backed point of view. Best Regards, Sandhya -----Original Message----- From: hotspot-compiler-dev On Behalf Of Yang Zhang Sent: Tuesday, May 26, 2020 7:59 PM To: Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes > But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? The new instructions can be classified as: 1. Instructions that can be matched with NEON instructions directly. MulVB and SqrtVF have been merged into jdk master already. The patch of AbsV is in review [1]. 2. Instructions that Jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. May I have a new patch for these? 3. Panama/Vector API specific instructions Such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. Regards Yang [1] https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-May/008861.html -----Original Message----- From: Andrew Haley Sent: Tuesday, May 26, 2020 4:25 PM To: Yang Zhang ; Paul Sandoz Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; nd Subject: Re: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes On 25/05/2020 09:26, Yang Zhang wrote: > In jdk master, what we need to do is that writing m4 file for existing > vector instructions and placed them to a new file aarch64_neon.ad. > If no question, I will do it right away. I'm not entirely sure that such a change is necessary now. In particular, reorganizing the existing vector instructions is IMO excessive, but I admit that it might be an improvement. But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? It'd help if this was possible. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671