From kim.barrett at oracle.com Mon Oct 1 03:21:30 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 30 Sep 2018 23:21:30 -0400 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> Message-ID: > On Sep 30, 2018, at 4:51 PM, David Holmes wrote: > > Hi Kim, Thanks for looking at this. >> (2) Remove call to _dyld_bind_fully_image_containing_address, which >> seems to have been deprecated since MacOSX 10.5. Instead use the >> recommended replacement "dlopen(RTLD_NOW)". However, it might be we >> don't need to do anything here anymore. I ran tier1-5 test without >> the dlopen without any problems. But I'm not familiar with the >> original problem, so not sure that's convincing. Perhaps another RFE >> to determine whether this code can be deleted? > > The use of _dyld_bind_fully_image_containing_address, as per the comment is a workaround to force full binding of symbols in the library. I don't know where the "callbacks" referred to in the comment come from or why they may be unaligned - this seems to be day 1 code for the OS X port from 2008: > > http://hg.openjdk.java.net/macosx-port/macosx-port/hotspot/rev/be94a5de4d32#l75.4001 > > I've cc'd the macosx-port-dev mailing list in case someone still subscribed there might know the answer. Yes, I?d noticed it was day-1 OS X port code. Thanks for cc?ing the macosx-port-dev list; I?d not thought of that. > If you don't have unaligned symbols then the workaround is not needed. In that case the replacement dlopen is also not needed (and it isn't needed to actually dlopen anything as you are opening the current library!). So it is not surprising that removing the dlopen causes no problem. > > With the dlopen in place however the code looks very strange and there's no guarantee it actually engages the workaround of the original function. The current library may have been previously only dlopen?ed with RTLD_LAZY. Reopening with RTLD_NOW forces resolution of all undefined symbols in that shared object, which appears to be the purpose of the deprecated function. The following dlclose is to decrement the refcount increment associated with that dlopen, and does nothing else since the library in question was already loaded and still has references. > So I'd say this either has to be removed completely or else left alone. The warning message is: _dyld_bind_fully_image_containing_address((const void *) &os::init); note: '_dyld_bind_fully_image_containing_address' has been explicitly marked deprecated here extern bool _dyld_bind_fully_image_containing_address(const void* address) __IOS_UNAVAILABLE __BRIDGEOS_UNAVAILABLE __OSX_DEPRECATED(10.1, 10.5, "dlopen(RTLD_NOW)"); The arguments in __OSX_DEPRECATED(10.1, 10.5. "dlopen(RTLD_NOW)") indicate it was introduced in 10.1, deprecated in 10.5, and the replacement to use instead. So I?m just using the recommended replacement (which was described in brief in the warning message, rather than working code). I?m reluctant to remove this completely without a better understanding of what it is there for and why it is no longer needed, if indeed that?s the case. The lack of problems when running tests is suggestive but not definitive. From david.holmes at oracle.com Mon Oct 1 06:27:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Oct 2018 16:27:32 +1000 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> Message-ID: <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> Hi Kim, On 1/10/2018 1:21 PM, Kim Barrett wrote: >> On Sep 30, 2018, at 4:51 PM, David Holmes wrote: >> >> Hi Kim, > > Thanks for looking at this. > >>> (2) Remove call to _dyld_bind_fully_image_containing_address, which >>> seems to have been deprecated since MacOSX 10.5. Instead use the >>> recommended replacement "dlopen(RTLD_NOW)". However, it might be we >>> don't need to do anything here anymore. I ran tier1-5 test without >>> the dlopen without any problems. But I'm not familiar with the >>> original problem, so not sure that's convincing. Perhaps another RFE >>> to determine whether this code can be deleted? >> >> The use of _dyld_bind_fully_image_containing_address, as per the comment is a workaround to force full binding of symbols in the library. I don't know where the "callbacks" referred to in the comment come from or why they may be unaligned - this seems to be day 1 code for the OS X port from 2008: >> >> http://hg.openjdk.java.net/macosx-port/macosx-port/hotspot/rev/be94a5de4d32#l75.4001 >> >> I've cc'd the macosx-port-dev mailing list in case someone still subscribed there might know the answer. > > Yes, I?d noticed it was day-1 OS X port code. Thanks for cc?ing the macosx-port-dev list; I?d not thought of that. > >> If you don't have unaligned symbols then the workaround is not needed. In that case the replacement dlopen is also not needed (and it isn't needed to actually dlopen anything as you are opening the current library!). So it is not surprising that removing the dlopen causes no problem. >> >> With the dlopen in place however the code looks very strange and there's no guarantee it actually engages the workaround of the original function. > > The current library may have been previously only dlopen?ed with RTLD_LAZY. Reopening with RTLD_NOW > forces resolution of all undefined symbols in that shared object, which appears to be the purpose of the > deprecated function. The current library is libjvm and it's already opened in the appropriate way: libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); and it seems to have always been opened this way. So I can't see how this can fix the problem the workaround purports to fix?? http://hg.openjdk.java.net/macosx-port/macosx-port/hotspot/rev/be94a5de4d32#l54.758 > > The following dlclose is to decrement the refcount increment associated with that dlopen, and does nothing > else since the library in question was already loaded and still has references. > >> So I'd say this either has to be removed completely or else left alone. > > The warning message is: > > _dyld_bind_fully_image_containing_address((const void *) &os::init); > note: '_dyld_bind_fully_image_containing_address' has been explicitly marked deprecated here > extern bool _dyld_bind_fully_image_containing_address(const void* address) __IOS_UNAVAILABLE __BRIDGEOS_UNAVAILABLE __OSX_DEPRECATED(10.1, 10.5, "dlopen(RTLD_NOW)"); > > The arguments in __OSX_DEPRECATED(10.1, 10.5. "dlopen(RTLD_NOW)") > indicate it was introduced in 10.1, deprecated in 10.5, and the replacement to use instead. > So I?m just using the recommended replacement (which was described in brief in the > warning message, rather than working code). > > I?m reluctant to remove this completely without a better understanding of what it is there for and why > it is no longer needed, if indeed that?s the case. The lack of problems when running tests is suggestive > but not definitive. I'd be happier knowing the exact details here as well, but if the new code doesn't implement the workaround then it's no better than completely removing it. And as we seem to have been always already using the new code then ... Cheers, David From shade at redhat.com Mon Oct 1 07:37:24 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 1 Oct 2018 09:37:24 +0200 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> Message-ID: <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> On 09/28/2018 10:46 PM, Markus Gronlund wrote: > I am ok with this from a JFR perspective. Thanks! > -----Original Message----- > From: Aleksey Shipilev > Sent: den 28 september 2018 13:25 > To: hotspot-dev at openjdk.java.net > Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8211239 > > Fix: > http://cr.openjdk.java.net/~shade/8211239/webrev.02/ Any other Reviewers? Would like to unbreak jdk/jdk :) -Aleksey From volker.simonis at gmail.com Mon Oct 1 07:39:48 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 1 Oct 2018 09:39:48 +0200 Subject: RFR(S): 8211145: [ppc] [s390]: Build fails due to -Werror=switch (introduced with JDK-8211029) In-Reply-To: References: <3C71CFD0-3764-464F-A12F-FD71BE44658E@sap.com> <6629adb8-f0d5-1f9d-15b5-3159c67e9c3e@redhat.com> <6D26E4F8-51E0-4171-B644-ED92C5830D82@sap.com> <1D9C7321-3E84-40AB-87CD-1BA6012B768C@sap.com> <37f0b889-3d9a-cba1-f51c-76ba56d7d4e2@redhat.com> Message-ID: Hi Lutz, the change looks good. Thanks for cleaning this up, Volker On Fri, Sep 28, 2018 at 12:30 PM Aleksey Shipilev wrote: > > On 09/27/2018 04:28 PM, Schmidt, Lutz wrote: > > re break vs. ShouldNotReachHere(), I tried to change semantics as little as possible. After > > discussion with colleagues, we concluded that ShouldNotReachHere() is the better choice. Code was > > modified accordingly. Your concerns re. coding style are reflected in the new webrev as well. It > > can be found here: http://cr.openjdk.java.net/~lucy/webrevs/8211145.02/ > > Looks good! Ship it. > > -Aleksey > From david.holmes at oracle.com Mon Oct 1 07:49:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Oct 2018 17:49:30 +1000 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> Message-ID: Hi Aleksey, On 1/10/2018 5:37 PM, Aleksey Shipilev wrote: > On 09/28/2018 10:46 PM, Markus Gronlund wrote: >> I am ok with this from a JFR perspective. > > Thanks! > >> -----Original Message----- >> From: Aleksey Shipilev >> Sent: den 28 september 2018 13:25 >> To: hotspot-dev at openjdk.java.net >> Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8211239 >> >> Fix: >> http://cr.openjdk.java.net/~shade/8211239/webrev.02/ > > Any other Reviewers? Would like to unbreak jdk/jdk :) Isn't printEmptyType redundant with this change? 480 out.write("template "); Why is this now a template class? I don't see T being used. The refactoring to handle the "empty" case within the same logic seems reasonable, though it's not immediately obvious to me where the signature mismatch was introduced previously or how the refactor prevents that ? Thanks, David > -Aleksey > From shade at redhat.com Mon Oct 1 08:05:04 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 1 Oct 2018 10:05:04 +0200 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> Message-ID: <3c47386e-40ee-6661-ade7-b0e776f4408f@redhat.com> On 10/01/2018 09:49 AM, David Holmes wrote: >> Any other Reviewers? Would like to unbreak jdk/jdk :) > > Isn't printEmptyType redundant with this change? Right! Removed here: http://cr.openjdk.java.net/~shade/8211239/webrev.03/ > 480???????????? out.write("template "); > > Why is this now a template class? I don't see T being used. "Empty" JfrEvent now matches "full" JfrEvent from jfrEvent.hpp: template class JfrEvent { private: jlong _start_time; jlong _end_time; bool _started; ... The "empty" subclasses use the template parameter here: out.write("class Event" + event.name + " : public JfrEvent"); ...and here: out.write(" using JfrEvent::commit; // else commit() is hidden by overloaded versions in this class"); > The refactoring to handle the "empty" case within the same logic seems reasonable, though it's not > immediately obvious to me where the signature mismatch was introduced previously or how the refactor > prevents that ? I think the mismatch between full and empty events was always there, I see it up to the original commit of JFR event generator. But it was not observed until we started to use the multi-parameter commit() methods in JDK-8196341: http://hg.openjdk.java.net/jdk/jdk/rev/5f931e3e7a63 -- when it broke the non-JFR build. The surprises like that is why I think we are better generating the same methods with/without body on full/empty event paths. Thanks, -Aleksey From david.holmes at oracle.com Mon Oct 1 09:45:06 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Oct 2018 19:45:06 +1000 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: <3c47386e-40ee-6661-ade7-b0e776f4408f@redhat.com> References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> <3c47386e-40ee-6661-ade7-b0e776f4408f@redhat.com> Message-ID: <1cecae6d-ac4c-0bb4-c820-ea7352860b4a@oracle.com> On 1/10/2018 6:05 PM, Aleksey Shipilev wrote: > On 10/01/2018 09:49 AM, David Holmes wrote: >>> Any other Reviewers? Would like to unbreak jdk/jdk :) >> >> Isn't printEmptyType redundant with this change? > > Right! Removed here: > http://cr.openjdk.java.net/~shade/8211239/webrev.03/ > >> 480???????????? out.write("template "); >> >> Why is this now a template class? I don't see T being used. > > "Empty" JfrEvent now matches "full" JfrEvent from jfrEvent.hpp: > > template > class JfrEvent { > private: > jlong _start_time; > jlong _end_time; > bool _started; > ... I'm confused. You stated in the RFR: "JFR events generator produces two versions for each event: "full" and "empty", and the version is then selected by INCLUDE_JFR macro during build time." but really it only generates one version and #includes the .hpp with the "real version". I'm at a loss to understand why it is done this way as the two versions are not connected in any way and so rely on manual intervention to be kept in sync AFAICS. > The "empty" subclasses use the template parameter here: > > out.write("class Event" + event.name + " : public JfrEvent"); > > ...and here: > > out.write(" using JfrEvent::commit; // else commit() is hidden by > overloaded versions in this class"); I see. This is very difficult to follow. > >> The refactoring to handle the "empty" case within the same logic seems reasonable, though it's not >> immediately obvious to me where the signature mismatch was introduced previously or how the refactor >> prevents that ? > > I think the mismatch between full and empty events was always there, I see it up to the original > commit of JFR event generator. But it was not observed until we started to use the multi-parameter > commit() methods in JDK-8196341: http://hg.openjdk.java.net/jdk/jdk/rev/5f931e3e7a63 -- when it > broke the non-JFR build. The surprises like that is why I think we are better generating the same > methods with/without body on full/empty event paths. This seems error prone to me. I'm guessing we keep far more "stubbed out" JFR code in a non-JFR build than might naively be assumed. Thanks, David > Thanks, > -Aleksey > From david.holmes at oracle.com Mon Oct 1 09:51:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Oct 2018 19:51:36 +1000 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: <1cecae6d-ac4c-0bb4-c820-ea7352860b4a@oracle.com> References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> <3c47386e-40ee-6661-ade7-b0e776f4408f@redhat.com> <1cecae6d-ac4c-0bb4-c820-ea7352860b4a@oracle.com> Message-ID: Trying to gather my thoughts ... On 1/10/2018 7:45 PM, David Holmes wrote: > On 1/10/2018 6:05 PM, Aleksey Shipilev wrote: >> On 10/01/2018 09:49 AM, David Holmes wrote: >>>> Any other Reviewers? Would like to unbreak jdk/jdk :) >>> >>> Isn't printEmptyType redundant with this change? >> >> Right! Removed here: >> ?? http://cr.openjdk.java.net/~shade/8211239/webrev.03/ >> >>> 480???????????? out.write("template "); >>> >>> Why is this now a template class? I don't see T being used. >> >> "Empty" JfrEvent now matches "full" JfrEvent from jfrEvent.hpp: >> >> template >> class JfrEvent { >> ? private: >> ?? jlong _start_time; >> ?? jlong _end_time; >> ?? bool _started; >> ?? ... > > I'm confused. You stated in the RFR: > > "JFR events generator produces two versions for each event: "full" and > "empty", and the version is then selected by INCLUDE_JFR macro during > build time." > > but really it only generates one version and #includes the .hpp with the > "real version". Or is it that the top-level JfrEvent exists in the .hpp but otherwise all the subclasses are generated. If so I'd probably find it much clearer if the actual jfrEvent.hpp file contained both versions and the appropriate ifdefs. At least that way the two versions are in the same file and there is less likelihood of them getting out of sync. David > I'm at a loss to understand why it is done this way as the two versions > are not connected in any way and so rely on manual intervention to be > kept in sync AFAICS. > >> The "empty" subclasses use the template parameter here: >> >> ?? out.write("class Event" + event.name + " : public JfrEvent> event.name + ">"); >> >> ...and here: >> >> ?? out.write("? using JfrEvent::commit; // >> else commit() is hidden by >> overloaded versions in this class"); > > I see. This is very difficult to follow. > >> >>> The refactoring to handle the "empty" case within the same logic >>> seems reasonable, though it's not >>> immediately obvious to me where the signature mismatch was introduced >>> previously or how the refactor >>> prevents that ? >> >> I think the mismatch between full and empty events was always there, I >> see it up to the original >> commit of JFR event generator. But it was not observed until we >> started to use the multi-parameter >> commit() methods in JDK-8196341: >> http://hg.openjdk.java.net/jdk/jdk/rev/5f931e3e7a63 -- when it >> broke the non-JFR build. The surprises like that is why I think we are >> better generating the same >> methods with/without body on full/empty event paths. > > This seems error prone to me. I'm guessing we keep far more "stubbed > out" JFR code in a non-JFR build than might naively be assumed. > > Thanks, > David > >> Thanks, >> -Aleksey >> From shade at redhat.com Mon Oct 1 09:59:06 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 1 Oct 2018 11:59:06 +0200 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: <1cecae6d-ac4c-0bb4-c820-ea7352860b4a@oracle.com> References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> <3c47386e-40ee-6661-ade7-b0e776f4408f@redhat.com> <1cecae6d-ac4c-0bb4-c820-ea7352860b4a@oracle.com> Message-ID: <53fcd563-41c0-5fac-6e79-dcfbcbb487bb@redhat.com> On 10/01/2018 11:45 AM, David Holmes wrote: > I'm confused. You stated in the RFR: > > "JFR events generator produces two versions for each event: "full" and "empty", and the version is > then selected by INCLUDE_JFR macro during build time." > > but really it only generates one version and #includes the .hpp with the "real version". Note there is JfrEvent superclass, and there are individual subclasses for the actual events. My statement is about the actual events, which are generated in two versions: $ cat ./build/linux-x86_64-server-fastdebug/hotspot/variant-server/gensrc/jfrfiles/jfrEventClasses.hpp | nl | grep -C 2 "class EventCompilerStatistics" 6825 }; 6826 class EventCompilerStatistics : public JfrEvent 6827 { 6828 private: -- 9896 }; 9897 class EventCompilerStatistics : public JfrEvent 9898 { 9899 public: These individual subclasses give us grief right now. The addition of template to empty JfrEvent is the minor thing that makes both full and empty events agree signature-wise. Granted, the JfrEvent superclass can be moved somewhere else for consistency, but that seems to be a more intrusive change than this. >>> The refactoring to handle the "empty" case within the same logic seems reasonable, though it's not >>> immediately obvious to me where the signature mismatch was introduced previously or how the refactor >>> prevents that ? >> >> I think the mismatch between full and empty events was always there, I see it up to the original >> commit of JFR event generator. But it was not observed until we started to use the multi-parameter >> commit() methods in JDK-8196341: http://hg.openjdk.java.net/jdk/jdk/rev/5f931e3e7a63 -- when it >> broke the non-JFR build. The surprises like that is why I think we are better generating the same >> methods with/without body on full/empty event paths. > > This seems error prone to me. I'm guessing we keep far more "stubbed out" JFR code in a non-JFR > build than might naively be assumed. Automatic generation of same-signature events does look much less error-prone to me, hence the RFR. I would not be audacious to claim this is ideal, but at least it solves the current build breakage, and makes the whole thing incrementally better. -Aleksey From david.holmes at oracle.com Mon Oct 1 11:07:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Oct 2018 21:07:39 +1000 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: <53fcd563-41c0-5fac-6e79-dcfbcbb487bb@redhat.com> References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> <3c47386e-40ee-6661-ade7-b0e776f4408f@redhat.com> <1cecae6d-ac4c-0bb4-c820-ea7352860b4a@oracle.com> <53fcd563-41c0-5fac-6e79-dcfbcbb487bb@redhat.com> Message-ID: <7a71320d-7b42-0cf4-a886-3322b5d15aaa@oracle.com> Okay I see how this fits together now. Reviewed. Thanks, David On 1/10/2018 7:59 PM, Aleksey Shipilev wrote: > On 10/01/2018 11:45 AM, David Holmes wrote: >> I'm confused. You stated in the RFR: >> >> "JFR events generator produces two versions for each event: "full" and "empty", and the version is >> then selected by INCLUDE_JFR macro during build time." >> >> but really it only generates one version and #includes the .hpp with the "real version". > > Note there is JfrEvent superclass, and there are individual subclasses for the actual events. My > statement is about the actual events, which are generated in two versions: > > $ cat > ./build/linux-x86_64-server-fastdebug/hotspot/variant-server/gensrc/jfrfiles/jfrEventClasses.hpp | > nl | grep -C 2 "class EventCompilerStatistics" > 6825 }; > > 6826 class EventCompilerStatistics : public JfrEvent > 6827 { > 6828 private: > -- > 9896 }; > > 9897 class EventCompilerStatistics : public JfrEvent > 9898 { > 9899 public: > > These individual subclasses give us grief right now. The addition of template to empty > JfrEvent is the minor thing that makes both full and empty events agree signature-wise. Granted, the > JfrEvent superclass can be moved somewhere else for consistency, but that seems to be a more > intrusive change than this. > >>>> The refactoring to handle the "empty" case within the same logic seems reasonable, though it's not >>>> immediately obvious to me where the signature mismatch was introduced previously or how the refactor >>>> prevents that ? >>> >>> I think the mismatch between full and empty events was always there, I see it up to the original >>> commit of JFR event generator. But it was not observed until we started to use the multi-parameter >>> commit() methods in JDK-8196341: http://hg.openjdk.java.net/jdk/jdk/rev/5f931e3e7a63 -- when it >>> broke the non-JFR build. The surprises like that is why I think we are better generating the same >>> methods with/without body on full/empty event paths. >> >> This seems error prone to me. I'm guessing we keep far more "stubbed out" JFR code in a non-JFR >> build than might naively be assumed. > > Automatic generation of same-signature events does look much less error-prone to me, hence the RFR. > I would not be audacious to claim this is ideal, but at least it solves the current build breakage, > and makes the whole thing incrementally better. > > -Aleksey > From rkennke at redhat.com Mon Oct 1 11:18:28 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 1 Oct 2018 13:18:28 +0200 Subject: RFR: JDK-8211241: Missing obj equals in TemplateTable::fast_aldc In-Reply-To: <2f9e2610-68ff-2da6-6a71-2398381a601a@oracle.com> References: <2f9e2610-68ff-2da6-6a71-2398381a601a@oracle.com> Message-ID: <5e77a79d-ec30-9af2-b34b-2fb50735b967@redhat.com> Hi Coleen, thanks for looking at this. I've combed the interpreter code for similar occurances, but haven't found any. Extra verification in cmpptr could be done but it's probably not worth.? It'll not be a small effort... Roman > This looks good. Did you search for other cmpptr() to see if they should > be cmpoop?? Wonder if you can do a negative verify_oop in cmpptr.? > Probably not. > > Lots of people look at the interpreter code, but they're on > hotspot-runtime. > > Thanks, > Coleen > > > On 9/27/18 4:33 PM, Roman Kennke wrote: >> Included hotspot-runtime-dev because it's an interpreter bug, excluded >> hotspot-compiler dev, and also here's all the links: >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8211241 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8211241/webrev.00/ >> >> It's never been quite clear to me where to put interpreter bugs. >> hotspot/runtime in Jira and hotspot-runtime-dev on mailing lists? Is >> that right? >> >> Thanks, Roman >> >> >> >>> TemplateTable::fast_aldc compares the just-loaded reference with >>> Universe::the_null_sentinel. If it really is that null-sentinel, we may >>> get a false negative (with GCs like Shenandoah that allow both >>> from-space and to-space copies of an object to be around), and thus skip >>> NULL-ing the ref. In other words, it would allow to get >>> the-null-sentinel out into the wild as oop which can cause subtle and >>> not-so-subtle bugs. >>> >>> Fix is easy, call cmpoop() which re-routes through GC-interface for GCs >>> that need it: >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8211241/webrev.00/ >>> >>> Testing: hotspot/jtreg:tier1 >>> >>> Ok? >>> >>> Roman >>> > From shade at redhat.com Mon Oct 1 11:30:46 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 1 Oct 2018 13:30:46 +0200 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: <7a71320d-7b42-0cf4-a886-3322b5d15aaa@oracle.com> References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> <3c47386e-40ee-6661-ade7-b0e776f4408f@redhat.com> <1cecae6d-ac4c-0bb4-c820-ea7352860b4a@oracle.com> <53fcd563-41c0-5fac-6e79-dcfbcbb487bb@redhat.com> <7a71320d-7b42-0cf4-a886-3322b5d15aaa@oracle.com> Message-ID: <0dc07abe-1c4d-90aa-e30d-7eb45110b8f7@redhat.com> On 10/01/2018 01:07 PM, David Holmes wrote: > Okay I see how this fits together now. > > Reviewed. Thanks! jdk-submit is unresponsive. Okay if I push this with only this testing? Testing: Linux/x86_64 +jfr/-jfr build, Linux/x86_64 run-test jdk/jfr -Aleksey From lutz.schmidt at sap.com Mon Oct 1 12:38:03 2018 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Mon, 1 Oct 2018 12:38:03 +0000 Subject: RFR(S): 8211145: [ppc] [s390]: Build fails due to -Werror=switch (introduced with JDK-8211029) In-Reply-To: References: <3C71CFD0-3764-464F-A12F-FD71BE44658E@sap.com> <6629adb8-f0d5-1f9d-15b5-3159c67e9c3e@redhat.com> <6D26E4F8-51E0-4171-B644-ED92C5830D82@sap.com> <1D9C7321-3E84-40AB-87CD-1BA6012B768C@sap.com> <37f0b889-3d9a-cba1-f51c-76ba56d7d4e2@redhat.com> Message-ID: <20D28443-4DAC-45A9-BEDA-502627A0E605@sap.com> Thank you, Aleksey and Volker! Pushed. Regards, Lutz ?On 01.10.18, 09:39, "Volker Simonis" wrote: Hi Lutz, the change looks good. Thanks for cleaning this up, Volker On Fri, Sep 28, 2018 at 12:30 PM Aleksey Shipilev wrote: > > On 09/27/2018 04:28 PM, Schmidt, Lutz wrote: > > re break vs. ShouldNotReachHere(), I tried to change semantics as little as possible. After > > discussion with colleagues, we concluded that ShouldNotReachHere() is the better choice. Code was > > modified accordingly. Your concerns re. coding style are reflected in the new webrev as well. It > > can be found here: http://cr.openjdk.java.net/~lucy/webrevs/8211145.02/ > > Looks good! Ship it. > > -Aleksey > From coleen.phillimore at oracle.com Mon Oct 1 13:34:53 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 1 Oct 2018 09:34:53 -0400 Subject: RFR: JDK-8211241: Missing obj equals in TemplateTable::fast_aldc In-Reply-To: <5e77a79d-ec30-9af2-b34b-2fb50735b967@redhat.com> References: <2f9e2610-68ff-2da6-6a71-2398381a601a@oracle.com> <5e77a79d-ec30-9af2-b34b-2fb50735b967@redhat.com> Message-ID: On 10/1/18 7:18 AM, Roman Kennke wrote: > Hi Coleen, > > thanks for looking at this. > > I've combed the interpreter code for similar occurances, but haven't > found any. > > Extra verification in cmpptr could be done but it's probably not worth.? > It'll not be a small effort... I agree. Thanks, Coleen > > Roman > > >> This looks good. Did you search for other cmpptr() to see if they should >> be cmpoop?? Wonder if you can do a negative verify_oop in cmpptr. >> Probably not. >> >> Lots of people look at the interpreter code, but they're on >> hotspot-runtime. >> >> Thanks, >> Coleen >> >> >> On 9/27/18 4:33 PM, Roman Kennke wrote: >>> Included hotspot-runtime-dev because it's an interpreter bug, excluded >>> hotspot-compiler dev, and also here's all the links: >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8211241 >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/JDK-8211241/webrev.00/ >>> >>> It's never been quite clear to me where to put interpreter bugs. >>> hotspot/runtime in Jira and hotspot-runtime-dev on mailing lists? Is >>> that right? >>> >>> Thanks, Roman >>> >>> >>> >>>> TemplateTable::fast_aldc compares the just-loaded reference with >>>> Universe::the_null_sentinel. If it really is that null-sentinel, we may >>>> get a false negative (with GCs like Shenandoah that allow both >>>> from-space and to-space copies of an object to be around), and thus skip >>>> NULL-ing the ref. In other words, it would allow to get >>>> the-null-sentinel out into the wild as oop which can cause subtle and >>>> not-so-subtle bugs. >>>> >>>> Fix is easy, call cmpoop() which re-routes through GC-interface for GCs >>>> that need it: >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211241/webrev.00/ >>>> >>>> Testing: hotspot/jtreg:tier1 >>>> >>>> Ok? >>>> >>>> Roman >>>> From shade at redhat.com Mon Oct 1 14:40:43 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 1 Oct 2018 16:40:43 +0200 Subject: RFR (S) 8211239: Build fails without JFR: empty JFR events signatures mismatch In-Reply-To: <0dc07abe-1c4d-90aa-e30d-7eb45110b8f7@redhat.com> References: <1a971dfb-e2f4-441d-8a5d-4aaa1ae266c1@default> <940e52c6-6495-e802-0944-5e1fbb370877@redhat.com> <3c47386e-40ee-6661-ade7-b0e776f4408f@redhat.com> <1cecae6d-ac4c-0bb4-c820-ea7352860b4a@oracle.com> <53fcd563-41c0-5fac-6e79-dcfbcbb487bb@redhat.com> <7a71320d-7b42-0cf4-a886-3322b5d15aaa@oracle.com> <0dc07abe-1c4d-90aa-e30d-7eb45110b8f7@redhat.com> Message-ID: <8c9e0229-16c1-c2fc-b2ad-00875da4cc95@redhat.com> On 10/01/2018 01:30 PM, Aleksey Shipilev wrote: > On 10/01/2018 01:07 PM, David Holmes wrote: >> Okay I see how this fits together now. >> >> Reviewed. > > Thanks! jdk-submit is unresponsive. Okay if I push this with only this testing? > Testing: Linux/x86_64 +jfr/-jfr build, Linux/x86_64 run-test jdk/jfr Nevermind, jdk-submit just returned with PASSED. Pushing it now. -Aleksey From aph at redhat.com Mon Oct 1 16:21:29 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 1 Oct 2018 17:21:29 +0100 Subject: RFR: AArch64: Fix another build failure after JDK-8211029 Message-ID: GCC 6.3 has more warnings, and wants more fixes. This fix is from Pengfei Li's patch. http://cr.openjdk.java.net/~aph/8211333/ -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From shade at redhat.com Mon Oct 1 16:25:08 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 1 Oct 2018 18:25:08 +0200 Subject: RFR: AArch64: Fix another build failure after JDK-8211029 In-Reply-To: References: Message-ID: <8838a352-7fa9-02f9-a83f-dd86bc228c34@redhat.com> On 10/01/2018 06:21 PM, Andrew Haley wrote: > GCC 6.3 has more warnings, and wants more fixes. This fix is from > Pengfei Li's patch. > > http://cr.openjdk.java.net/~aph/8211333/ Looks good and trivial to me. -Aleksey From kim.barrett at oracle.com Mon Oct 1 18:35:10 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 1 Oct 2018 14:35:10 -0400 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> Message-ID: > On Oct 1, 2018, at 2:27 AM, David Holmes wrote: > > On 1/10/2018 1:21 PM, Kim Barrett wrote: >> The current library may have been previously only dlopen?ed with RTLD_LAZY. Reopening with RTLD_NOW >> forces resolution of all undefined symbols in that shared object, which appears to be the purpose of the >> deprecated function. > > The current library is libjvm and it's already opened in the appropriate way: > > libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); > > and it seems to have always been opened this way. So I can't see how this can fix the problem the workaround purports to fix?? > > http://hg.openjdk.java.net/macosx-port/macosx-port/hotspot/rev/be94a5de4d32#l54.758 The current version of LoadJavaVM (for Mac) contains: #ifndef STATIC_BUILD libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); #else libjvm = dlopen(NULL, RTLD_FIRST); #endif where STATIC_BUILD is controlled by the --enable-static-builds configure option. The --enable-static-builds option was added in JDK 9 by 8136556: Add the ability to perform static builds of MacOSX x64 binaries. So it is possible for libjava to be opened without RTLD_NOW. I think the proposed change should work correctly in that case. The dladdr should find the current object, which we'll open by name (rather than using NULL, but that should be okay), using RTLD_NOW, forcing resolution of all symbols. I'm not sure there can be any relevant unresolved symbols on this path, but then, as said before, I don't understand the point of this workaround. >> I?m reluctant to remove this completely without a better understanding of what it is there for and why >> it is no longer needed, if indeed that?s the case. The lack of problems when running tests is suggestive >> but not definitive. > > I'd be happier knowing the exact details here as well, but if the new code doesn't implement the workaround then it's no better than completely removing it. And as we seem to have been always already using the new code then ? I thought this part of the change would be straight-forward. An alternative would be to leave the old deprecated call in place, locally disable the deprecation warning, and file an RFE for someone more knowledgable in this area and with Mac development to look at. From thomas.schatzl at oracle.com Mon Oct 1 19:44:24 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 1 Oct 2018 12:44:24 -0700 (PDT) Subject: RFR: AArch64: Fix another build failure after JDK-8211029 Message-ID: Hi, looks good. Thomas ----- Original Message ----- From: aph at redhat.com To: hotspot-dev at openjdk.java.net Sent: Monday, October 1, 2018 6:21:55 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: RFR: AArch64: Fix another build failure after JDK-8211029 GCC 6.3 has more warnings, and wants more fixes. This fix is from Pengfei Li's patch. http://cr.openjdk.java.net/~aph/8211333/ -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Mon Oct 1 20:37:19 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 1 Oct 2018 16:37:19 -0400 Subject: CFV: New hotspot Group Member: Mikael Vidstedt Message-ID: I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to Membership in the hotspot Group Mikael is managing the hotspot team at Oracle and has been working on JVMs (jrockit and hotspot) for almost 20 years, touching on everything from JIT compilers, operating system and CPU ports, JVMTI, a microkernel for Java, and more recently Project Panama. He is also the project lead for OpenJDK Project Portola. Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. Only current Members of the Members Group [1] are eligible to vote on this nomination.? Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Coleen Phillimore [1] http://openjdk.java.net/census#hotspot [2] http://openjdk.java.net/groups/#member-vote From coleen.phillimore at oracle.com Mon Oct 1 20:38:37 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 1 Oct 2018 16:38:37 -0400 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <34678f20-9cec-813a-0686-c5a2ff5b2316@oracle.com> Vote: yes On 10/1/18 4:37 PM, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to > Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on > JVMs (jrockit and hotspot) for almost 20 years, touching on everything > from JIT compilers, operating system and CPU ports, JVMTI, a > microkernel for Java, and more recently Project Panama. He is also the > project lead for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on > this nomination.? Votes must be cast in the open by replying to this > mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote From igor.ignatyev at oracle.com Mon Oct 1 20:44:40 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 1 Oct 2018 13:44:40 -0700 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <34604459-04A4-4FC0-85F9-8D5F7B75CE38@oracle.com> Vote: yes -- Igor > On Oct 1, 2018, at 1:37 PM, coleen.phillimore at oracle.com wrote: > > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to Membership in the hotspot Group From daniel.daugherty at oracle.com Mon Oct 1 21:04:08 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 1 Oct 2018 17:04:08 -0400 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <33ac337d-457f-9444-7e66-ff7ae5dfa219@oracle.com> Vote: yes Dan On 10/1/18 4:37 PM, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to > Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on > JVMs (jrockit and hotspot) for almost 20 years, touching on everything > from JIT compilers, operating system and CPU ports, JVMTI, a > microkernel for Java, and more recently Project Panama. He is also the > project lead for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on > this nomination.? Votes must be cast in the open by replying to this > mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote > From thomas.schatzl at oracle.com Mon Oct 1 21:11:13 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 01 Oct 2018 23:11:13 +0200 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <7f7b498a75af4e59c9401c850943302e14b70740.camel@oracle.com> Vote: yes On Mon, 2018-10-01 at 16:37 -0400, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to > Membership in the hotspot Group > > [...] From jesper.wilhelmsson at oracle.com Mon Oct 1 22:39:41 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Tue, 2 Oct 2018 00:39:41 +0200 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: Vote: Yes /Jesper > On 1 Oct 2018, at 22:37, coleen.phillimore at oracle.com wrote: > > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on JVMs (jrockit and hotspot) for almost 20 years, touching on everything from JIT compilers, operating system and CPU ports, JVMTI, a microkernel for Java, and more recently Project Panama. He is also the project lead for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote From david.holmes at oracle.com Mon Oct 1 23:10:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 2 Oct 2018 09:10:14 +1000 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> Message-ID: <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> On 2/10/2018 4:35 AM, Kim Barrett wrote: >> On Oct 1, 2018, at 2:27 AM, David Holmes wrote: >> >> On 1/10/2018 1:21 PM, Kim Barrett wrote: >>> The current library may have been previously only dlopen?ed with RTLD_LAZY. Reopening with RTLD_NOW >>> forces resolution of all undefined symbols in that shared object, which appears to be the purpose of the >>> deprecated function. >> >> The current library is libjvm and it's already opened in the appropriate way: >> >> libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); >> >> and it seems to have always been opened this way. So I can't see how this can fix the problem the workaround purports to fix?? >> >> http://hg.openjdk.java.net/macosx-port/macosx-port/hotspot/rev/be94a5de4d32#l54.758 > > The current version of LoadJavaVM (for Mac) contains: > > #ifndef STATIC_BUILD > libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); > #else > libjvm = dlopen(NULL, RTLD_FIRST); > #endif > > where STATIC_BUILD is controlled by the --enable-static-builds > configure option. The --enable-static-builds option was added in JDK 9 > by 8136556: Add the ability to perform static builds of MacOSX x64 > binaries. Which "current version" is this? It is not what is in jdk/jdk repo, which has no STATIC_BUILD support in this area. ?? Although we may have not used RTLD_NOW with static builds (does this discussions re symbol resolution even make sense with a static build?) in some version of the JDK, my point is that we have been using RTLD_NOW for as long as the workaround has been in place. That strongly suggests to me that use of RTLD_NOW is not a solution to the problem the workaround was introduced for. > So it is possible for libjava to be opened without RTLD_NOW. I think > the proposed change should work correctly in that case. The dladdr > should find the current object, which we'll open by name (rather than > using NULL, but that should be okay), using RTLD_NOW, forcing > resolution of all symbols. I'm not sure there can be any relevant > unresolved symbols on this path, but then, as said before, I don't > understand the point of this workaround. > >>> I?m reluctant to remove this completely without a better understanding of what it is there for and why >>> it is no longer needed, if indeed that?s the case. The lack of problems when running tests is suggestive >>> but not definitive. >> >> I'd be happier knowing the exact details here as well, but if the new code doesn't implement the workaround then it's no better than completely removing it. And as we seem to have been always already using the new code then ? > > I thought this part of the change would be straight-forward. An > alternative would be to leave the old deprecated call in place, > locally disable the deprecation warning, and file an RFE for someone > more knowledgable in this area and with Mac development to look at. Sorry for the obstructions on this but changing the code without knowing it even implements the same workaround is just wrong to me. I strongly suspect the code is not needed at all. Perhaps the old per-file flag setting can be restored. By all means file a RFE but I fear there is noone who would know the details of this any more. :( David From ChrisPhi at LGonQn.Org Mon Oct 1 23:52:15 2018 From: ChrisPhi at LGonQn.Org (Chris Phillips) Date: Mon, 1 Oct 2018 19:52:15 -0400 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: Vote: yes Chris On 01/10/18 04:37 PM, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to > Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on > JVMs (jrockit and hotspot) for almost 20 years, touching on everything > from JIT compilers, operating system and CPU ports, JVMTI, a microkernel > for Java, and more recently Project Panama. He is also the project lead > for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on > this nomination.? Votes must be cast in the open by replying to this > mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote > > > From david.holmes at oracle.com Tue Oct 2 00:40:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 2 Oct 2018 10:40:53 +1000 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <201e4ea2-b3be-56c8-cccb-95999e77d4fc@oracle.com> Vote: yes David On 2/10/2018 6:37 AM, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to > Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on > JVMs (jrockit and hotspot) for almost 20 years, touching on everything > from JIT compilers, operating system and CPU ports, JVMTI, a microkernel > for Java, and more recently Project Panama. He is also the project lead > for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on > this nomination.? Votes must be cast in the open by replying to this > mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote From kim.barrett at oracle.com Tue Oct 2 01:14:39 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 1 Oct 2018 21:14:39 -0400 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: vote: yes > On Oct 1, 2018, at 4:37 PM, coleen.phillimore at oracle.com wrote: > > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on JVMs (jrockit and hotspot) for almost 20 years, touching on everything from JIT compilers, operating system and CPU ports, JVMTI, a microkernel for Java, and more recently Project Panama. He is also the project lead for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote From vladimir.kozlov at oracle.com Tue Oct 2 05:52:32 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 1 Oct 2018 22:52:32 -0700 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <41a75fe1-4491-1848-efc3-df1c64f56f28@oracle.com> Vote: yes On 10/1/18 1:37 PM, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on JVMs (jrockit and hotspot) for almost 20 years, > touching on everything from JIT compilers, operating system and CPU ports, JVMTI, a microkernel for Java, and more > recently Project Panama. He is also the project lead for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on this nomination.? Votes must be cast in the open > by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote From tobias.hartmann at oracle.com Tue Oct 2 06:31:27 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 2 Oct 2018 08:31:27 +0200 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <4b7ca3b0-5a1d-5d6a-d8cd-72520491db49@oracle.com> Vote: yes Best regards, Tobias On 01.10.2018 22:37, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on JVMs (jrockit and hotspot) for > almost 20 years, touching on everything from JIT compilers, operating system and CPU ports, JVMTI, a > microkernel for Java, and more recently Project Panama. He is also the project lead for OpenJDK > Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on this nomination.? Votes must > be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote From volker.simonis at gmail.com Tue Oct 2 06:41:11 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 2 Oct 2018 08:41:11 +0200 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: Vote: yes schrieb am Mo. 1. Okt. 2018 um 22:37: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to > Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on > JVMs (jrockit and hotspot) for almost 20 years, touching on everything > from JIT compilers, operating system and CPU ports, JVMTI, a microkernel > for Java, and more recently Project Panama. He is also the project lead > for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote > From mikael.vidstedt at oracle.com Tue Oct 2 07:21:13 2018 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 2 Oct 2018 00:21:13 -0700 Subject: RFR(S): 8211350: Remove jprt support Message-ID: Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. bug: https://bugs.openjdk.java.net/browse/JDK-8211350 webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ * Background (from the issue) The Oracle internal JPRT system was once used to build and test the JDK. It has been superseded and can no longer be used to build/test mainline JDK. The support in the JDK code for running jobs in JPRT should be removed. * About the change The change touches several areas: build/make, hotspot, and jdk. I?m optimistically sending it out as a single webrev anyway. If it helps, I?m expecting the reviews to roughly map like so: build-dev: make related files, jprt.* hotspot-dev: {src,test}/hotspot core-libs-dev: test/jdk - I?m especially interested in hearing if the change to test/jdk/tools/lib/tests/Helper.java (to throw if the jmods directory is missing) is valid Of course I?d welcome reviews across those mappings as well. * Testing tier1 (passed), tier2-3 still running (looking good so far). Cheers, Mikael From matthias.baesken at sap.com Tue Oct 2 07:30:45 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 2 Oct 2018 07:30:45 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file Message-ID: Hello , please review this small enhancement to the hs_err file . Currently the hs_err file contains only limited OS user related information. Just the user name is printed via output of environment variables (at least on Windows with USERNAME - output). The enhanced output on UNIX would contain more information including uid, gid and umask : uid : 1679 (testuser) euid : 1679 (testuser) gid : 25 (testgroup) egid : 25 (testgroup) umask: 0022 (removing ----w--w-) ( Some of the info above could be found currently in error logging output e.g. attachListener_linux.cpp line 362 log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", (and the user name on Windows(-only) is in the env variables section). bug/webrev : ---------------------- https://bugs.openjdk.java.net/browse/JDK-8211326 http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ Thanks, Matthias From david.holmes at oracle.com Tue Oct 2 07:44:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 2 Oct 2018 17:44:05 +1000 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: Message-ID: Hi Mikael, This all looks fine to me. Thanks for cleaning it up! David On 2/10/2018 5:21 PM, Mikael Vidstedt wrote: > > Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. > > bug: https://bugs.openjdk.java.net/browse/JDK-8211350 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ > > * Background (from the issue) > > The Oracle internal JPRT system was once used to build and test the JDK. It has been superseded and can no longer be used to build/test mainline JDK. The support in the JDK code for running jobs in JPRT should be removed. > > > * About the change > > The change touches several areas: build/make, hotspot, and jdk. I?m optimistically sending it out as a single webrev anyway. If it helps, I?m expecting the reviews to roughly map like so: > > build-dev: make related files, jprt.* > hotspot-dev: {src,test}/hotspot > core-libs-dev: test/jdk - I?m especially interested in hearing if the change to test/jdk/tools/lib/tests/Helper.java (to throw if the jmods directory is missing) is valid > > Of course I?d welcome reviews across those mappings as well. > > * Testing > > tier1 (passed), tier2-3 still running (looking good so far). > > Cheers, > Mikael > From david.holmes at oracle.com Tue Oct 2 07:49:06 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 2 Oct 2018 17:49:06 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: Message-ID: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> Hi Matthias, On 2/10/2018 5:30 PM, Baesken, Matthias wrote: > Hello , please review this small enhancement to the hs_err file . > > Currently the hs_err file contains only limited OS user related information. > Just the user name is printed via output of environment variables (at least on Windows with USERNAME - output). > The enhanced output on UNIX would contain more information including uid, gid and umask : > > uid : 1679 (testuser) > euid : 1679 (testuser) > gid : 25 (testgroup) > egid : 25 (testgroup) > > umask: 0022 (removing ----w--w-) Could any of this be considered sensitive information by an end-user? Thanks, David > > ( Some of the info above could be found currently in error logging output e.g. > attachListener_linux.cpp line 362 > log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", > (and the user name on Windows(-only) is in the env variables section). > > > > bug/webrev : > ---------------------- > > https://bugs.openjdk.java.net/browse/JDK-8211326 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ > > > Thanks, Matthias > From thomas.schatzl at oracle.com Tue Oct 2 08:22:03 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Oct 2018 10:22:03 +0200 Subject: [PATCH] JDK-8205051 (UseNUMA memory interleaving vs cpunodebind & localalloc) In-Reply-To: References: <5362f42009cb7caddf43444c322410f3ee364112.camel@oracle.com> Message-ID: Hi Roshan, On Fri, 2018-09-28 at 15:01 +0530, roshan mangal wrote: > Hi Thomas, > > > Hi Roshan, > > > > On Tue, 2018-09-25 at 12:18 +0530, roshan mangal wrote: > > > Hi All, > > > > > > This Patch is for > > > https://bugs.openjdk.java.net/browse/JDK-8205051 > > > > > > Issue: > > > > > > If the JVM isn't allowed to run on all of the nodes (by numactl, > > > cgroups, docker, etc), then a significant fraction of the Java > > > heap will be unusable, causing early GC. > > > > > > Every Thread captures their locality group(lgrp) and allocates > > > memory from that lgrp. > > > > > > lgrp id is same as NUMA node id. Is there a compelling reason to have two identifiers for the same thing? I am just asking, because it is confusing to use both in the same code interchangably. > > > [...] > > > Thread's lgrp > > > Order of Allocation in NUMA node > > > > > > lgrp0 [ numaNode0->numaNode1->numaNode2->numaNode3- > > > > > > > numaNode4->numaNode5->numaNode6->numaNode7 ] > > > lgrp1 [ numaNode1->numaNode0->numaNode2->numaNode3- > > > > > > > numaNode4->numaNode5->numaNode6->numaNode7 ] > > > lgrp2 [ numaNode2->numaNode0->numaNode1->numaNode3- > > > > > > > numaNode4->numaNode5->numaNode6->numaNode7 ] > > > lgrp3 [ numaNode3->numaNode0->numaNode1->numaNode2- > > > > > > > numaNode4->numaNode5->numaNode6->numaNode7 ] > > > lgrp4 [ numaNode4->numaNode5->numaNode6->numaNode7- > > > > > > > numaNode0->numaNode1->numaNode2->numaNode3 ] > > > lgrp5 [ numaNode5->numaNode4->numaNode6->numaNode7- > > > > > > > numaNode0->numaNode1->numaNode2->numaNode3 ] > > > lgrp6 [ numaNode6->numaNode4->numaNode5->numaNode7- > > > > > > > numaNode0->numaNode1->numaNode2->numaNode3 ] > > > lgrp7 [ numaNode7->numaNode4->numaNode5->numaNode6- > > > > > > > numaNode0->numaNode1->numaNode2->numaNode3 ] > > > > I have a question about this: lgrps often have the same distance > > from each other, and this order-of-allocation list seems to be > > deterministic. So in this case nodes with lower lgrp id (but the > > same distance) are preferred to ones with higher lgrp id. > > > > Do you expect some imbalance because of that? If so, couldn't it be > > useful to randomize lgrps with the same distance in this list, and > > regularly change them? > > Yes, I agree there will be an imbalance because of that. > Another option would be to select lgrp based on largest free memory > available. What is your opinion on this ? > > For Example, > node 0 1 2 3 > 0: 10 16 16 16 > 1: 16 10 16 16 > 2: 16 16 10 16 > 3: 16 16 16 10 > > Threads T0,T1,T2 and T3 are running on different numa node with > lgrp0,lgrp1,lgrp2 and lgrp3 respectively. > Allocation within the node has distance 10 but outside is 16. > Suppose lgrp0 gets OOM and memory available at that time is lgrp0(0%) > lgrp1(10%) lgrp2(5%) lgrp3(50%). > If we choose 'lgrp' randomly e.g. lgrp2 over lgrp3 for T0, lgrp2 will > be filled faster and once lgrp2 is full, T2 also need to go other > lgrp with distance 16. > In this scenario, T0 choosing lgrp3 with 50% free memory will be a > better option, which will allow T2 to run for longer time on lgrp2 > having lower memory latency (10). I was more concerned about performance in that case actually: as mentioned, in the current implementation all threads are somewhat likely to access the memory of the same node, the same with this idea of selecting the one with the largest amount of memory. This means that (assuming some very naive hardware implementation), the memory controller on that node will need to handle requests from "all" other nodes, meaning some potential contention. If the accesses were more spread out across nodes, memory accesses would be more spread out across the network, allowing use of higher memory bandwidth as long as possible. That of course is a rather naive model of the interconnect :) I am not so worried about the case you mention: with the use of NUMAAllocationDistanceLimit you extend the time until GC until at most all nodes are full anyway. > Selecting the best possible lgrp dynamically will be a better option > instead of deterministic selection (as in my patch) but that will add > slightly more overhead during memory allocation. > Will that be ok? Since memory allocation is on TLAB basis, and we expect TLABs to be reasonably large, some more time used should not matter. The MutableSpace allocation path is already only used with NUMA enabled. > > > Allocation on NUMA node, which is far from CPU can lead to > > > performance issue. Sometimes triggering GC is a better option > > > than allocating from NUMA node at large distance i.e. high memory > > > latency. > > > > > > For this, I have added option "NumaAllocationDistanceLimit", > > > which will restrict memory allocation from the far nodes. > > > > > > In above system if we set -XX:NumaAllocationDistanceLimit=16. > > > > That makes sense imho, although it is a bit sad that this number is > > specific to the machine. [...] > > > > could you send me a patch as webrev so I can put it on > > cr.openjdk.java.net? (Or maybe sending the patch as attachment > > helps too). It got mangled by your email program, adding many > > linebreaks. > > > > Please find patch as attachment. I created a webrev at http://cr.openjdk.java.net/~tschatzl/8205051/webrev from it. Some comments: - plaese add range constraints for NUMAAllocationDistanceLimit to disallow negative limits. Or are there any reasons to allow negative distance limits? I am aware that int is not the perfect type for this, but otoh the API returns ints for it, so we need to stick to it. Examples can be found in runtime/globals.hpp (look for "range"). - please make sure that the code complies to Hotspot coding style, particularly for the "numaNode" class. Indentation, member, local variable and other naming is off. https://wiki.openjdk.java.net/display/HotSpot/StyleGuide Also look at code e.g. in mutableSpace.hpp. - would it be possible to factor out a "allocate_from_node()" (or "try_allocate_from_node()" method from the MutableSpace::allocate code and use it. Basically the code in the while-loop (now). That would probably make the code a lot more readable. - I am almost sure that the code does not compile on Solaris. Since mutablespace is shared code, it needs to still compile and run there too. Please at least add suitable functional stubs for Solaris (and any other OSes) to be filled out if you can't provide them. That code btw is the one with the mixing of the "NumaNode" and "lgrp" identifiers :( Maybe "numaNode" can be replaced by something more suitable. - some typos in the identifier names: "avaliable" -> "available" > I have run specjbb2015 composite run with "numactl -N 0 program>" on 8 NUMA node system. > it has improved score > composite Max-jOPS: +29% > composite Critical-jOPS: +24% You mean that when bound to a single node, we do not regress that much any more? Are there any changes in performance in the case when all nodes are in use before and after your change with NUMA support? Thanks, Thomas From Alan.Bateman at oracle.com Tue Oct 2 08:34:31 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Oct 2018 09:34:31 +0100 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: Message-ID: On 02/10/2018 08:21, Mikael Vidstedt wrote: > Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. > > bug: https://bugs.openjdk.java.net/browse/JDK-8211350 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ > Does the change to Helper.java and tests that use it break testing of exploded builds? I don't know why it has the comment "JPRT not yet ready" but whether $JAVA_HOME/jmods exists or not isn't connected to JPRT or any build or test system. -Alan From thomas.schatzl at oracle.com Tue Oct 2 08:38:38 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Oct 2018 10:38:38 +0200 Subject: [PATCH] JDK-8205051 (UseNUMA memory interleaving vs cpunodebind & localalloc) In-Reply-To: References: <5362f42009cb7caddf43444c322410f3ee364112.camel@oracle.com> Message-ID: Hi again, some short note: On Tue, 2018-10-02 at 10:22 +0200, Thomas Schatzl wrote: > Hi Roshan, > [...] > - I am almost sure that the code does not compile on Solaris. Since > mutablespace is shared code, it needs to still compile and run there > too. Please at least add suitable functional stubs for Solaris (and > any other OSes) to be filled out if you can't provide them. Actually this is an understatement: the code only compiles on Linux right now ;) Thanks, Thomas From david.holmes at oracle.com Tue Oct 2 09:52:45 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 2 Oct 2018 19:52:45 +1000 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: Message-ID: <3b27a611-3a53-af1e-aa08-4bd81ecdceff@oracle.com> On 2/10/2018 6:34 PM, Alan Bateman wrote: > On 02/10/2018 08:21, Mikael Vidstedt wrote: >> Please review this change which removes support for, and references >> to, the (Oracle internal) JPRT system. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8211350 >> webrev: >> http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ >> > Does the change to Helper.java and tests that use it break testing of > exploded builds? I don't know why it has the comment "JPRT not yet > ready" but whether $JAVA_HOME/jmods exists or not isn't connected to > JPRT or any build or test system. Yes it will break as now an exception will be thrown instead of just cleanly exiting with an error message. I think the Helper changes can be left as-is, just update the comment. David > -Alan From markus.gronlund at oracle.com Tue Oct 2 10:19:21 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 2 Oct 2018 03:19:21 -0700 (PDT) Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <9fd52a3d-b990-4c96-9b33-63a9d20a3d8a@default> Vote: yes Thanks Markus -----Original Message----- From: Coleen Phillimore Sent: den 1 oktober 2018 22:37 To: hotspot-dev developers Subject: CFV: New hotspot Group Member: Mikael Vidstedt I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to Membership in the hotspot Group Mikael is managing the hotspot team at Oracle and has been working on JVMs (jrockit and hotspot) for almost 20 years, touching on everything from JIT compilers, operating system and CPU ports, JVMTI, a microkernel for Java, and more recently Project Panama. He is also the project lead for OpenJDK Project Portola. Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. Only current Members of the Members Group [1] are eligible to vote on this nomination.? Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Coleen Phillimore [1] http://openjdk.java.net/census#hotspot [2] http://openjdk.java.net/groups/#member-vote From shade at redhat.com Tue Oct 2 10:20:44 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 2 Oct 2018 12:20:44 +0200 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <66f6c4b7-c32e-4814-295e-486d9855a49c@redhat.com> Vote: yes -Aleksey On 10/01/2018 10:37 PM, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on JVMs (jrockit and hotspot) for > almost 20 years, touching on everything from JIT compilers, operating system and CPU ports, JVMTI, a > microkernel for Java, and more recently Project Panama. He is also the project lead for OpenJDK > Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on this nomination.? Votes must > be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote From sgehwolf at redhat.com Tue Oct 2 10:33:46 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 02 Oct 2018 12:33:46 +0200 Subject: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and os.arch value on ppc64le cause issues with Java tooling In-Reply-To: References: Message-ID: Hi, Pinging PPC porters. Does this look reasonable to you? Thanks, Severin On Fri, 2018-09-28 at 08:56 -0700, Erik Joelsson wrote: > Build changes look ok to me. > > /Erik > > > On 2018-09-26 04:26, Severin Gehwolf wrote: > > Hi, > > > > Could I please get reviews for this JDK 8 backport which fixes some > > tooling issues on Linux ppc64le? Prior this patch, a ppc64le build > > would report as "ppc64" via os.arch system property which breaks > > tooling such as maven in as much as if some dependency needs native > > libraries it would download BE binaries where it actually should > > download LE binaries. Example for os.arch/java.library.path: > > > > pre: > > $ ./jdk8-pre-ppc64le/bin/java TestProperty > > java.library.path = /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > os.arch = ppc64 > > > > post: > > $ ./jdk8-post-ppc64le/bin/java TestProperty > > java.library.path = /usr/java/packages/lib/ppc64le:/usr/lib64:/lib64:/lib:/usr/lib > > os.arch = ppc64le > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8073139 > > webrevs: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8073139/jdk8/01/ > > > > Including build-dev for build changes. hotspot-dev and ppc-aix-port-dev > > for JDK/hotspot changes. > > > > This backport should only have minor differences to the changes in JDK > > 11. We have been using similar patches in Fedora for months. Thoughts? > > > > Thanks, > > Severin > > > > From ralf.schmelter at sap.com Tue Oct 2 10:37:15 2018 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Tue, 2 Oct 2018 10:37:15 +0000 Subject: RFR(S): : 8211332: Space for stub routines (code_size2) is too small on new Skylake CPUs Message-ID: <96e9bbe2280a49ffb5b16c2c95f069ff@sap.com> Please review this change which increases the space available for stub routines on Windows x86. bug: https://bugs.openjdk.java.net/browse/JDK-8211332 webrev: http://cr.openjdk.java.net/~clanger/webrevs/8211332.0/ My Windows x86 slowdebug build crashed in StubRoutines::initialize1(), since the estimated code size (code_size2 defined in stub_routines_x86.hpp) for the initial stubs was too low. I've increase code_size2 by 500 bytes per step, until the assertions vanished. This seems to be 'caused' the new -XX:UseBASE64Intrinsics flag, which is only turned on if the machine supports the AVX-512VL and AVX-512BW extensions, which is the case for Skylake-SP and Skylake X and later/better CPUs. Best regards, Ralf Schmelter From matthias.baesken at sap.com Tue Oct 2 10:38:04 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 2 Oct 2018 10:38:04 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> Message-ID: Hi David, I think the added info could be seen more or less in line with what currently is reported in hs_err file . For instance you usually see user-names and lots of paths from the system in the hs_err file . In case the umask and gid is seen as more sensitive than that, one could make the output switchable with an XX-flag ; this would have the benefit of making the added output more clear to the user/admin . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Dienstag, 2. Oktober 2018 09:49 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > Hi Matthias, > > On 2/10/2018 5:30 PM, Baesken, Matthias wrote: > > Hello , please review this small enhancement to the hs_err file . > > > > Currently the hs_err file contains only limited OS user related information. > > Just the user name is printed via output of environment variables (at least > on Windows with USERNAME - output). > > The enhanced output on UNIX would contain more information including > uid, gid and umask : > > > > uid : 1679 (testuser) > > euid : 1679 (testuser) > > gid : 25 (testgroup) > > egid : 25 (testgroup) > > > > umask: 0022 (removing ----w--w-) > > Could any of this be considered sensitive information by an end-user? > > Thanks, > David > > > > > ( Some of the info above could be found currently in error logging output > e.g. > > attachListener_linux.cpp line 362 > > log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", > > (and the user name on Windows(-only) is in the env variables section). > > > > > > > > bug/webrev : > > ---------------------- > > > > https://bugs.openjdk.java.net/browse/JDK-8211326 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ > > > > > > Thanks, Matthias > > From goetz.lindenmaier at sap.com Tue Oct 2 10:40:31 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 2 Oct 2018 10:40:31 +0000 Subject: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and os.arch value on ppc64le cause issues with Java tooling In-Reply-To: References: Message-ID: <666987d08f324a0c94208c6cca403214@sap.com> Hi, I'm fine with this. If I remember correctly, this was proposed before but never pushed in the end. Did you test this on ppc64 be, too? Best regards, Goetz. > -----Original Message----- > From: ppc-aix-port-dev On > Behalf Of Severin Gehwolf > Sent: Dienstag, 2. Oktober 2018 12:34 > To: Erik Joelsson ; hotspot-dev dev at openjdk.java.net>; ppc-aix-port-dev dev at openjdk.java.net>; build-dev > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and > os.arch value on ppc64le cause issues with Java tooling > > Hi, > > Pinging PPC porters. Does this look reasonable to you? > > Thanks, > Severin > > > On Fri, 2018-09-28 at 08:56 -0700, Erik Joelsson wrote: > > Build changes look ok to me. > > > > /Erik > > > > > > On 2018-09-26 04:26, Severin Gehwolf wrote: > > > Hi, > > > > > > Could I please get reviews for this JDK 8 backport which fixes some > > > tooling issues on Linux ppc64le? Prior this patch, a ppc64le build > > > would report as "ppc64" via os.arch system property which breaks > > > tooling such as maven in as much as if some dependency needs native > > > libraries it would download BE binaries where it actually should > > > download LE binaries. Example for os.arch/java.library.path: > > > > > > pre: > > > $ ./jdk8-pre-ppc64le/bin/java TestProperty > > > java.library.path = > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > > os.arch = ppc64 > > > > > > post: > > > $ ./jdk8-post-ppc64le/bin/java TestProperty > > > java.library.path = > /usr/java/packages/lib/ppc64le:/usr/lib64:/lib64:/lib:/usr/lib > > > os.arch = ppc64le > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8073139 > > > webrevs: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- > 8073139/jdk8/01/ > > > > > > Including build-dev for build changes. hotspot-dev and ppc-aix-port-dev > > > for JDK/hotspot changes. > > > > > > This backport should only have minor differences to the changes in JDK > > > 11. We have been using similar patches in Fedora for months. Thoughts? > > > > > > Thanks, > > > Severin > > > > > > > From thomas.stuefe at gmail.com Tue Oct 2 10:53:17 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 2 Oct 2018 12:53:17 +0200 Subject: RFR(S): : 8211332: Space for stub routines (code_size2) is too small on new Skylake CPUs In-Reply-To: <96e9bbe2280a49ffb5b16c2c95f069ff@sap.com> References: <96e9bbe2280a49ffb5b16c2c95f069ff@sap.com> Message-ID: This looks good. Thanks for fixing! Best Regards, Thomas On Tue, Oct 2, 2018 at 12:37 PM Schmelter, Ralf wrote: > > Please review this change which increases the space available for stub routines on Windows x86. > > > > bug: https://bugs.openjdk.java.net/browse/JDK-8211332 > > webrev: http://cr.openjdk.java.net/~clanger/webrevs/8211332.0/ > > > > My Windows x86 slowdebug build crashed in StubRoutines::initialize1(), since the estimated code size (code_size2 defined in stub_routines_x86.hpp) for the initial stubs was too low. I've increase code_size2 by 500 bytes per step, until the assertions vanished. > > > > This seems to be 'caused' the new -XX:UseBASE64Intrinsics flag, which is only turned on if the machine supports the AVX-512VL and AVX-512BW extensions, which is the case for Skylake-SP and Skylake X and later/better CPUs. > > > > Best regards, > > Ralf Schmelter From sgehwolf at redhat.com Tue Oct 2 11:09:15 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 02 Oct 2018 13:09:15 +0200 Subject: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and os.arch value on ppc64le cause issues with Java tooling In-Reply-To: <666987d08f324a0c94208c6cca403214@sap.com> References: <666987d08f324a0c94208c6cca403214@sap.com> Message-ID: <994203969d0375a2a53345aa392d30550c505d16.camel@redhat.com> Hi Goetz, On Tue, 2018-10-02 at 10:40 +0000, Lindenmaier, Goetz wrote: > Hi, > > I'm fine with this. Thanks for the review! > If I remember correctly, this was proposed before but > never pushed in the end. Interesting. > Did you test this on ppc64 be, too? I have not. Will do so, though. Thanks, Severin > Best regards, > Goetz. > > > -----Original Message----- > > From: ppc-aix-port-dev On > > Behalf Of Severin Gehwolf > > Sent: Dienstag, 2. Oktober 2018 12:34 > > To: Erik Joelsson ; hotspot-dev > dev at openjdk.java.net>; ppc-aix-port-dev > dev at openjdk.java.net>; build-dev > > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and > > os.arch value on ppc64le cause issues with Java tooling > > > > Hi, > > > > Pinging PPC porters. Does this look reasonable to you? > > > > Thanks, > > Severin > > > > > > On Fri, 2018-09-28 at 08:56 -0700, Erik Joelsson wrote: > > > Build changes look ok to me. > > > > > > /Erik > > > > > > > > > On 2018-09-26 04:26, Severin Gehwolf wrote: > > > > Hi, > > > > > > > > Could I please get reviews for this JDK 8 backport which fixes some > > > > tooling issues on Linux ppc64le? Prior this patch, a ppc64le build > > > > would report as "ppc64" via os.arch system property which breaks > > > > tooling such as maven in as much as if some dependency needs native > > > > libraries it would download BE binaries where it actually should > > > > download LE binaries. Example for os.arch/java.library.path: > > > > > > > > pre: > > > > $ ./jdk8-pre-ppc64le/bin/java TestProperty > > > > java.library.path = > > > > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > > > os.arch = ppc64 > > > > > > > > post: > > > > $ ./jdk8-post-ppc64le/bin/java TestProperty > > > > java.library.path = > > > > /usr/java/packages/lib/ppc64le:/usr/lib64:/lib64:/lib:/usr/lib > > > > os.arch = ppc64le > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8073139 > > > > webrevs: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- > > > > 8073139/jdk8/01/ > > > > > > > > Including build-dev for build changes. hotspot-dev and ppc-aix-port-dev > > > > for JDK/hotspot changes. > > > > > > > > This backport should only have minor differences to the changes in JDK > > > > 11. We have been using similar patches in Fedora for months. Thoughts? > > > > > > > > Thanks, > > > > Severin > > > > > > > > > > > > From goetz.lindenmaier at sap.com Tue Oct 2 12:39:16 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 2 Oct 2018 12:39:16 +0000 Subject: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and os.arch value on ppc64le cause issues with Java tooling In-Reply-To: <994203969d0375a2a53345aa392d30550c505d16.camel@redhat.com> References: <666987d08f324a0c94208c6cca403214@sap.com> <994203969d0375a2a53345aa392d30550c505d16.camel@redhat.com> Message-ID: <6d0a6a512b39469b8afd25ae587c99ba@sap.com> Hi Severin, here for example: http://mail.openjdk.java.net/pipermail/build-dev/2015-July/015370.html While the fix proposed there looks different and the downport was never finished. Best regards, Goetz. > -----Original Message----- > From: Severin Gehwolf > Sent: Dienstag, 2. Oktober 2018 13:09 > To: Lindenmaier, Goetz ; Erik Joelsson > ; hotspot-dev dev at openjdk.java.net>; ppc-aix-port-dev dev at openjdk.java.net>; build-dev > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and > os.arch value on ppc64le cause issues with Java tooling > > Hi Goetz, > > On Tue, 2018-10-02 at 10:40 +0000, Lindenmaier, Goetz wrote: > > Hi, > > > > I'm fine with this. > > Thanks for the review! > > > If I remember correctly, this was proposed before but > > never pushed in the end. > > Interesting. > > > Did you test this on ppc64 be, too? > > I have not. Will do so, though. > > Thanks, > Severin > > > Best regards, > > Goetz. > > > > > -----Original Message----- > > > From: ppc-aix-port-dev > On > > > Behalf Of Severin Gehwolf > > > Sent: Dienstag, 2. Oktober 2018 12:34 > > > To: Erik Joelsson ; hotspot-dev > > dev at openjdk.java.net>; ppc-aix-port-dev > > dev at openjdk.java.net>; build-dev > > > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory > and > > > os.arch value on ppc64le cause issues with Java tooling > > > > > > Hi, > > > > > > Pinging PPC porters. Does this look reasonable to you? > > > > > > Thanks, > > > Severin > > > > > > > > > On Fri, 2018-09-28 at 08:56 -0700, Erik Joelsson wrote: > > > > Build changes look ok to me. > > > > > > > > /Erik > > > > > > > > > > > > On 2018-09-26 04:26, Severin Gehwolf wrote: > > > > > Hi, > > > > > > > > > > Could I please get reviews for this JDK 8 backport which fixes some > > > > > tooling issues on Linux ppc64le? Prior this patch, a ppc64le build > > > > > would report as "ppc64" via os.arch system property which breaks > > > > > tooling such as maven in as much as if some dependency needs > native > > > > > libraries it would download BE binaries where it actually should > > > > > download LE binaries. Example for os.arch/java.library.path: > > > > > > > > > > pre: > > > > > $ ./jdk8-pre-ppc64le/bin/java TestProperty > > > > > java.library.path = > > > > > > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > > > > os.arch = ppc64 > > > > > > > > > > post: > > > > > $ ./jdk8-post-ppc64le/bin/java TestProperty > > > > > java.library.path = > > > > > > /usr/java/packages/lib/ppc64le:/usr/lib64:/lib64:/lib:/usr/lib > > > > > os.arch = ppc64le > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8073139 > > > > > webrevs: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- > > > > > > 8073139/jdk8/01/ > > > > > > > > > > Including build-dev for build changes. hotspot-dev and ppc-aix-port- > dev > > > > > for JDK/hotspot changes. > > > > > > > > > > This backport should only have minor differences to the changes in > JDK > > > > > 11. We have been using similar patches in Fedora for months. > Thoughts? > > > > > > > > > > Thanks, > > > > > Severin > > > > > > > > > > > > > > > > > From sgehwolf at redhat.com Tue Oct 2 12:59:39 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 02 Oct 2018 14:59:39 +0200 Subject: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and os.arch value on ppc64le cause issues with Java tooling In-Reply-To: <6d0a6a512b39469b8afd25ae587c99ba@sap.com> References: <666987d08f324a0c94208c6cca403214@sap.com> <994203969d0375a2a53345aa392d30550c505d16.camel@redhat.com> <6d0a6a512b39469b8afd25ae587c99ba@sap.com> Message-ID: <26f26253dd3bd909ae17e729b6286fce7c86f32c.camel@redhat.com> Hi Goetz, I'm a bit confused :-/ On Tue, 2018-10-02 at 12:39 +0000, Lindenmaier, Goetz wrote: > Hi Severin, > > here for example: http://mail.openjdk.java.net/pipermail/build-dev/2015-July/015370.html As far as I can see that was relating to the JDK-head fix which wasn't available at the time (July vs. pushed in Dec). The original review thread was here: http://mail.openjdk.java.net/pipermail/build-dev/2015-December/016103.html JDK-8073139 has been fixed in JDK 9+ since December 14, 2015. > While the fix proposed there looks different and the downport was never > finished. FWIW, this is a review request for the 8u backport :) Thanks, Severin > Best regards, > Goetz. > > > > > -----Original Message----- > > From: Severin Gehwolf > > Sent: Dienstag, 2. Oktober 2018 13:09 > > To: Lindenmaier, Goetz ; Erik Joelsson > > ; hotspot-dev > dev at openjdk.java.net>; ppc-aix-port-dev > dev at openjdk.java.net>; build-dev > > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and > > os.arch value on ppc64le cause issues with Java tooling > > > > Hi Goetz, > > > > On Tue, 2018-10-02 at 10:40 +0000, Lindenmaier, Goetz wrote: > > > Hi, > > > > > > I'm fine with this. > > > > Thanks for the review! > > > > > If I remember correctly, this was proposed before but > > > never pushed in the end. > > > > Interesting. > > > > > Did you test this on ppc64 be, too? > > > > I have not. Will do so, though. > > > > Thanks, > > Severin > > > > > Best regards, > > > Goetz. > > > > > > > -----Original Message----- > > > > From: ppc-aix-port-dev > > > > On > > > > Behalf Of Severin Gehwolf > > > > Sent: Dienstag, 2. Oktober 2018 12:34 > > > > To: Erik Joelsson ; hotspot-dev > > > dev at openjdk.java.net>; ppc-aix-port-dev > > > dev at openjdk.java.net>; build-dev > > > > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory > > > > and > > > > os.arch value on ppc64le cause issues with Java tooling > > > > > > > > Hi, > > > > > > > > Pinging PPC porters. Does this look reasonable to you? > > > > > > > > Thanks, > > > > Severin > > > > > > > > > > > > On Fri, 2018-09-28 at 08:56 -0700, Erik Joelsson wrote: > > > > > Build changes look ok to me. > > > > > > > > > > /Erik > > > > > > > > > > > > > > > On 2018-09-26 04:26, Severin Gehwolf wrote: > > > > > > Hi, > > > > > > > > > > > > Could I please get reviews for this JDK 8 backport which fixes some > > > > > > tooling issues on Linux ppc64le? Prior this patch, a ppc64le build > > > > > > would report as "ppc64" via os.arch system property which breaks > > > > > > tooling such as maven in as much as if some dependency needs > > > > native > > > > > > libraries it would download BE binaries where it actually should > > > > > > download LE binaries. Example for os.arch/java.library.path: > > > > > > > > > > > > pre: > > > > > > $ ./jdk8-pre-ppc64le/bin/java TestProperty > > > > > > java.library.path = > > > > > > > > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > > > > > os.arch = ppc64 > > > > > > > > > > > > post: > > > > > > $ ./jdk8-post-ppc64le/bin/java TestProperty > > > > > > java.library.path = > > > > > > > > /usr/java/packages/lib/ppc64le:/usr/lib64:/lib64:/lib:/usr/lib > > > > > > os.arch = ppc64le > > > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8073139 > > > > > > webrevs: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- > > > > > > > > 8073139/jdk8/01/ > > > > > > > > > > > > Including build-dev for build changes. hotspot-dev and ppc-aix-port- > > > > dev > > > > > > for JDK/hotspot changes. > > > > > > > > > > > > This backport should only have minor differences to the changes in > > > > JDK > > > > > > 11. We have been using similar patches in Fedora for months. > > > > Thoughts? > > > > > > > > > > > > Thanks, > > > > > > Severin > > > > > > > > > > > > > > > > > > > > > > > > From matthias.baesken at sap.com Tue Oct 2 13:01:06 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 2 Oct 2018 13:01:06 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 Message-ID: Hello, please review change 8211106 . It updates the Windows OS detection code to recognize Windows Server 2019 . For this we have to look at the build number (dwBuildNumber of OSVERSIONINFOEX), https://docs.microsoft.com/en-us/windows/desktop/api/Winnt/ns-winnt-_osversioninfoexa because dwMajorVersion and dwMinorVersion are the same for Windows server 2016 and 2019 . The build number used to compare ( 17677 ) is for Windows Server 2019 preview , most likely the final version will have a higher build number but this is fine for the coding . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8211106 http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ Thanks, Matthias From goetz.lindenmaier at sap.com Tue Oct 2 13:14:49 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 2 Oct 2018 13:14:49 +0000 Subject: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and os.arch value on ppc64le cause issues with Java tooling In-Reply-To: <26f26253dd3bd909ae17e729b6286fce7c86f32c.camel@redhat.com> References: <666987d08f324a0c94208c6cca403214@sap.com> <994203969d0375a2a53345aa392d30550c505d16.camel@redhat.com> <6d0a6a512b39469b8afd25ae587c99ba@sap.com> <26f26253dd3bd909ae17e729b6286fce7c86f32c.camel@redhat.com> Message-ID: <3c5196e76e1043ff820c55e61bed83a1@sap.com> yes, there was the backport review requeset, and there was another one before that, but both never got pushed to 8u. There also were webrevs for the backport, which didn't apply any more after a while. So it's good if someone drives this now, finally :) Best regards, Goetz. > -----Original Message----- > From: Severin Gehwolf > Sent: Dienstag, 2. Oktober 2018 15:00 > To: Lindenmaier, Goetz ; Erik Joelsson > ; hotspot-dev dev at openjdk.java.net>; ppc-aix-port-dev dev at openjdk.java.net>; build-dev > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and > os.arch value on ppc64le cause issues with Java tooling > > Hi Goetz, > > I'm a bit confused :-/ > > On Tue, 2018-10-02 at 12:39 +0000, Lindenmaier, Goetz wrote: > > Hi Severin, > > > > here for example: http://mail.openjdk.java.net/pipermail/build-dev/2015- > July/015370.html > > As far as I can see that was relating to the JDK-head fix which wasn't > available at the time (July vs. pushed in Dec). The original review > thread was here: > http://mail.openjdk.java.net/pipermail/build-dev/2015- > December/016103.html > > JDK-8073139 has been fixed in JDK 9+ since December 14, 2015. > > > While the fix proposed there looks different and the downport was never > > finished. > > FWIW, this is a review request for the 8u backport :) > > Thanks, > Severin > > > Best regards, > > Goetz. > > > > > > > > > -----Original Message----- > > > From: Severin Gehwolf > > > Sent: Dienstag, 2. Oktober 2018 13:09 > > > To: Lindenmaier, Goetz ; Erik Joelsson > > > ; hotspot-dev > > dev at openjdk.java.net>; ppc-aix-port-dev > > dev at openjdk.java.net>; build-dev > > > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory > and > > > os.arch value on ppc64le cause issues with Java tooling > > > > > > Hi Goetz, > > > > > > On Tue, 2018-10-02 at 10:40 +0000, Lindenmaier, Goetz wrote: > > > > Hi, > > > > > > > > I'm fine with this. > > > > > > Thanks for the review! > > > > > > > If I remember correctly, this was proposed before but > > > > never pushed in the end. > > > > > > Interesting. > > > > > > > Did you test this on ppc64 be, too? > > > > > > I have not. Will do so, though. > > > > > > Thanks, > > > Severin > > > > > > > Best regards, > > > > Goetz. > > > > > > > > > -----Original Message----- > > > > > From: ppc-aix-port-dev bounces at openjdk.java.net> > > > > > > On > > > > > Behalf Of Severin Gehwolf > > > > > Sent: Dienstag, 2. Oktober 2018 12:34 > > > > > To: Erik Joelsson ; hotspot-dev > > > > dev at openjdk.java.net>; ppc-aix-port-dev > > > > dev at openjdk.java.net>; build-dev > > > > > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch > directory > > > > > > and > > > > > os.arch value on ppc64le cause issues with Java tooling > > > > > > > > > > Hi, > > > > > > > > > > Pinging PPC porters. Does this look reasonable to you? > > > > > > > > > > Thanks, > > > > > Severin > > > > > > > > > > > > > > > On Fri, 2018-09-28 at 08:56 -0700, Erik Joelsson wrote: > > > > > > Build changes look ok to me. > > > > > > > > > > > > /Erik > > > > > > > > > > > > > > > > > > On 2018-09-26 04:26, Severin Gehwolf wrote: > > > > > > > Hi, > > > > > > > > > > > > > > Could I please get reviews for this JDK 8 backport which fixes > some > > > > > > > tooling issues on Linux ppc64le? Prior this patch, a ppc64le build > > > > > > > would report as "ppc64" via os.arch system property which breaks > > > > > > > tooling such as maven in as much as if some dependency needs > > > > > > native > > > > > > > libraries it would download BE binaries where it actually should > > > > > > > download LE binaries. Example for os.arch/java.library.path: > > > > > > > > > > > > > > pre: > > > > > > > $ ./jdk8-pre-ppc64le/bin/java TestProperty > > > > > > > java.library.path = > > > > > > > > > > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > > > > > > os.arch = ppc64 > > > > > > > > > > > > > > post: > > > > > > > $ ./jdk8-post-ppc64le/bin/java TestProperty > > > > > > > java.library.path = > > > > > > > > > > /usr/java/packages/lib/ppc64le:/usr/lib64:/lib64:/lib:/usr/lib > > > > > > > os.arch = ppc64le > > > > > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8073139 > > > > > > > webrevs: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- > > > > > > > > > > 8073139/jdk8/01/ > > > > > > > > > > > > > > Including build-dev for build changes. hotspot-dev and ppc-aix- > port- > > > > > > dev > > > > > > > for JDK/hotspot changes. > > > > > > > > > > > > > > This backport should only have minor differences to the changes > in > > > > > > JDK > > > > > > > 11. We have been using similar patches in Fedora for months. > > > > > > Thoughts? > > > > > > > > > > > > > > Thanks, > > > > > > > Severin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From bob.vandette at oracle.com Tue Oct 2 14:09:18 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 2 Oct 2018 10:09:18 -0400 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: Message-ID: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Matthias, We don?t usually add logic to the JDK related to unreleased operating system previews. Has the ReleaseID been updated on the system you are testing on? This is the documented way of determining the OS version you are running on? The latest preview release is build 17744. Are you testing on that release? https://techcommunity.microsoft.com/t5/Windows-Server-Insiders/Windows-Server-2019-version-info/td-p/234472 Bob. > On Oct 2, 2018, at 9:01 AM, Baesken, Matthias wrote: > > Hello, please review change 8211106 . > > It updates the Windows OS detection code to recognize Windows Server 2019 . > > For this we have to look at the build number (dwBuildNumber of OSVERSIONINFOEX), > > https://docs.microsoft.com/en-us/windows/desktop/api/Winnt/ns-winnt-_osversioninfoexa > > > because dwMajorVersion and dwMinorVersion are the same for Windows server 2016 and 2019 . > > The build number used to compare ( 17677 ) is for Windows Server 2019 preview , most likely the final version will have a higher build number but this is fine for the coding . > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8211106 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ > > > Thanks, Matthias From matthias.baesken at sap.com Tue Oct 2 15:02:53 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 2 Oct 2018 15:02:53 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: Hi Bob, no problem , we can wait a few days until the final version of Win server 2019 is out, and check the buildNumber again : https://cloudblogs.microsoft.com/windowsserver/2018/09/24/windows-server-2019-announcing-general-availability-in-october/ says : "In October, customers will have access to Windows Server 2019 through all the channels! We will publish a blog post to mark the availability of Windows Server 2019 soon." > > The latest preview release is build 17744. Are you testing on that release? > I was testing on a system with build number 17677 ( Windows Server 2019 preview , seems older than what you are seeing) and 17723 . Best regards, Matthias > -----Original Message----- > From: Bob Vandette > Sent: Dienstag, 2. Oktober 2018 16:09 > To: Baesken, Matthias > Cc: hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > Moldenhauer, Niclas > Subject: Re: RFR : 8211106: [windows] Update OS detection code to > recognize Windows Server 2019 > > Matthias, > > We don?t usually add logic to the JDK related to unreleased operating system > previews. > > Has the ReleaseID been updated on the system you are testing on? > > This is the documented way of determining the OS version you are running > on? > > The latest preview release is build 17744. Are you testing on that release? > > https://techcommunity.microsoft.com/t5/Windows-Server- > Insiders/Windows-Server-2019-version-info/td-p/234472 > > > Bob. > > > > On Oct 2, 2018, at 9:01 AM, Baesken, Matthias > wrote: > > > > Hello, please review change 8211106 . > > > > It updates the Windows OS detection code to recognize Windows Server > 2019 . > > > > For this we have to look at the build number (dwBuildNumber of > OSVERSIONINFOEX), > > > > https://docs.microsoft.com/en-us/windows/desktop/api/Winnt/ns-winnt- > _osversioninfoexa > > > > > > because dwMajorVersion and dwMinorVersion are the same for > Windows server 2016 and 2019 . > > > > The build number used to compare ( 17677 ) is for Windows Server 2019 > preview , most likely the final version will have a higher build number but > this is fine for the coding . > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8211106 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ > > > > > > Thanks, Matthias From mandy.chung at oracle.com Tue Oct 2 15:40:35 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 Oct 2018 08:40:35 -0700 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: Message-ID: <86c65ecd-dca0-6b0d-d144-22ad5a1e73e6@oracle.com> On 10/2/18 12:21 AM, Mikael Vidstedt wrote: > Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. > > bug: https://bugs.openjdk.java.net/browse/JDK-8211350 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ > test/jdk/tools/lib/tests/Helper.java is used by test/jdk/tools/jimage and test/jdk/tools/jlink tests which will be skipped, rather than failing, when running with images where jmods directory is not present (e.g. exploded image). I think Helper class should continue to return null if jmods does not exist.? test/jdk/tools/jimage/JImageTest.java should also be reverted. Otherwise, the clean up looks good. Mandy From erik.joelsson at oracle.com Tue Oct 2 15:53:46 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 2 Oct 2018 08:53:46 -0700 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: Message-ID: <3971e8aa-c8f1-4676-8288-40828b172c90@oracle.com> Build changes look good to me. /Erik On 2018-10-02 00:21, Mikael Vidstedt wrote: > Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. > > bug: https://bugs.openjdk.java.net/browse/JDK-8211350 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ > > * Background (from the issue) > > The Oracle internal JPRT system was once used to build and test the JDK. It has been superseded and can no longer be used to build/test mainline JDK. The support in the JDK code for running jobs in JPRT should be removed. > > > * About the change > > The change touches several areas: build/make, hotspot, and jdk. I?m optimistically sending it out as a single webrev anyway. If it helps, I?m expecting the reviews to roughly map like so: > > build-dev: make related files, jprt.* > hotspot-dev: {src,test}/hotspot > core-libs-dev: test/jdk - I?m especially interested in hearing if the change to test/jdk/tools/lib/tests/Helper.java (to throw if the jmods directory is missing) is valid > > Of course I?d welcome reviews across those mappings as well. > > * Testing > > tier1 (passed), tier2-3 still running (looking good so far). > > Cheers, > Mikael > From vladimir.kozlov at oracle.com Tue Oct 2 18:03:40 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 2 Oct 2018 11:03:40 -0700 Subject: RFR(S): : 8211332: Space for stub routines (code_size2) is too small on new Skylake CPUs In-Reply-To: References: <96e9bbe2280a49ffb5b16c2c95f069ff@sap.com> Message-ID: +1 Thanks, Vladimir On 10/2/18 3:53 AM, Thomas St?fe wrote: > This looks good. Thanks for fixing! > > Best Regards, Thomas > On Tue, Oct 2, 2018 at 12:37 PM Schmelter, Ralf wrote: >> >> Please review this change which increases the space available for stub routines on Windows x86. >> >> >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8211332 >> >> webrev: http://cr.openjdk.java.net/~clanger/webrevs/8211332.0/ >> >> >> >> My Windows x86 slowdebug build crashed in StubRoutines::initialize1(), since the estimated code size (code_size2 defined in stub_routines_x86.hpp) for the initial stubs was too low. I've increase code_size2 by 500 bytes per step, until the assertions vanished. >> >> >> >> This seems to be 'caused' the new -XX:UseBASE64Intrinsics flag, which is only turned on if the machine supports the AVX-512VL and AVX-512BW extensions, which is the case for Skylake-SP and Skylake X and later/better CPUs. >> >> >> >> Best regards, >> >> Ralf Schmelter From mikael.vidstedt at oracle.com Tue Oct 2 18:17:18 2018 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 2 Oct 2018 11:17:18 -0700 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: <86c65ecd-dca0-6b0d-d144-22ad5a1e73e6@oracle.com> References: <86c65ecd-dca0-6b0d-d144-22ad5a1e73e6@oracle.com> Message-ID: Thanks for the reviews. I?ve reverted the changes related to Helper and ?just? adjusted the comments instead. webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01/open/webrev/ incremental (from webrev.00): http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01.incr/open/webrev/ Btw, I notice that "Test not run, NO jmods directory? will be printed twice when jmods is missing - once in Helper::newHelper and once in the methods calling it. In general, the handling of a null return from newHelper could use some clean up, but that is out of scope for this change. Cheers, Mikael > On Oct 2, 2018, at 8:40 AM, Mandy Chung wrote: > > > > On 10/2/18 12:21 AM, Mikael Vidstedt wrote: >> Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8211350 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ >> > > test/jdk/tools/lib/tests/Helper.java is used by test/jdk/tools/jimage and test/jdk/tools/jlink tests which will be skipped, rather than failing, when running with images where jmods directory is not present (e.g. exploded image). > > I think Helper class should continue to return null if jmods does not exist. test/jdk/tools/jimage/JImageTest.java should also be reverted. > > Otherwise, the clean up looks good. > > Mandy From mandy.chung at oracle.com Tue Oct 2 18:18:16 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 Oct 2018 11:18:16 -0700 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: <86c65ecd-dca0-6b0d-d144-22ad5a1e73e6@oracle.com> Message-ID: +1 Mandy On 10/2/18 11:17 AM, Mikael Vidstedt wrote: > > Thanks for the reviews. I?ve reverted the changes related to Helper > and ?just? adjusted the comments instead. > > webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01/open/webrev/ > > incremental (from webrev.00): > http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01.incr/open/webrev/ > > > > Btw, I notice that "Test not run, NO jmods directory? will be printed > twice when jmods is missing - once in Helper::newHelper and once in > the methods calling it. In general, the handling of a null return from > newHelper could use some clean up, but that is out of scope for this > change. > > Cheers, > Mikael From erik.joelsson at oracle.com Tue Oct 2 18:28:01 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 2 Oct 2018 11:28:01 -0700 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: <86c65ecd-dca0-6b0d-d144-22ad5a1e73e6@oracle.com> Message-ID: <7f163563-0c99-d496-95d9-3f0a2ff54ca9@oracle.com> Looks good to me. /Erik On 2018-10-02 11:17, Mikael Vidstedt wrote: > Thanks for the reviews. I?ve reverted the changes related to Helper and ?just? adjusted the comments instead. > > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01/open/webrev/ > incremental (from webrev.00): http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01.incr/open/webrev/ > > > Btw, I notice that "Test not run, NO jmods directory? will be printed twice when jmods is missing - once in Helper::newHelper and once in the methods calling it. In general, the handling of a null return from newHelper could use some clean up, but that is out of scope for this change. > > Cheers, > Mikael > >> On Oct 2, 2018, at 8:40 AM, Mandy Chung wrote: >> >> >> >> On 10/2/18 12:21 AM, Mikael Vidstedt wrote: >>> Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8211350 >>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ >>> >> test/jdk/tools/lib/tests/Helper.java is used by test/jdk/tools/jimage and test/jdk/tools/jlink tests which will be skipped, rather than failing, when running with images where jmods directory is not present (e.g. exploded image). >> >> I think Helper class should continue to return null if jmods does not exist. test/jdk/tools/jimage/JImageTest.java should also be reverted. >> >> Otherwise, the clean up looks good. >> >> Mandy From Alan.Bateman at oracle.com Tue Oct 2 18:28:26 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Oct 2018 19:28:26 +0100 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: <86c65ecd-dca0-6b0d-d144-22ad5a1e73e6@oracle.com> Message-ID: <5ce26e91-9ecc-3adf-e61a-b85eb4349beb@oracle.com> On 02/10/2018 19:17, Mikael Vidstedt wrote: > Thanks for the reviews. I?ve reverted the changes related to Helper and ?just? adjusted the comments instead. > > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01/open/webrev/ > incremental (from webrev.00): http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01.incr/open/webrev/ > > > Btw, I notice that "Test not run, NO jmods directory? will be printed twice when jmods is missing - once in Helper::newHelper and once in the methods calling it. In general, the handling of a null return from newHelper could use some clean up, but that is out of scope for this change. > At some point I think the test for jlink will need to be cleaned up anyway but that is way outside of scope of what you are doing. The updated webrev looks okay to me. -Alan From david.holmes at oracle.com Tue Oct 2 20:06:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Oct 2018 06:06:23 +1000 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: <86c65ecd-dca0-6b0d-d144-22ad5a1e73e6@oracle.com> Message-ID: Looks good. Ship it! David On 3/10/2018 4:17 AM, Mikael Vidstedt wrote: > > Thanks for the reviews. I?ve reverted the changes related to Helper and ?just? adjusted the comments instead. > > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01/open/webrev/ > incremental (from webrev.00): http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.01.incr/open/webrev/ > > > Btw, I notice that "Test not run, NO jmods directory? will be printed twice when jmods is missing - once in Helper::newHelper and once in the methods calling it. In general, the handling of a null return from newHelper could use some clean up, but that is out of scope for this change. > > Cheers, > Mikael > >> On Oct 2, 2018, at 8:40 AM, Mandy Chung wrote: >> >> >> >> On 10/2/18 12:21 AM, Mikael Vidstedt wrote: >>> Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8211350 >>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ >>> >> >> test/jdk/tools/lib/tests/Helper.java is used by test/jdk/tools/jimage and test/jdk/tools/jlink tests which will be skipped, rather than failing, when running with images where jmods directory is not present (e.g. exploded image). >> >> I think Helper class should continue to return null if jmods does not exist. test/jdk/tools/jimage/JImageTest.java should also be reverted. >> >> Otherwise, the clean up looks good. >> >> Mandy > From kim.barrett at oracle.com Tue Oct 2 20:51:40 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Oct 2018 16:51:40 -0400 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> Message-ID: <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> > On Oct 1, 2018, at 7:10 PM, David Holmes wrote: > > On 2/10/2018 4:35 AM, Kim Barrett wrote: >>> On Oct 1, 2018, at 2:27 AM, David Holmes wrote: >>> >>> On 1/10/2018 1:21 PM, Kim Barrett wrote: >>>> The current library may have been previously only dlopen?ed with RTLD_LAZY. Reopening with RTLD_NOW >>>> forces resolution of all undefined symbols in that shared object, which appears to be the purpose of the >>>> deprecated function. >>> >>> The current library is libjvm and it's already opened in the appropriate way: >>> >>> libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); >>> >>> and it seems to have always been opened this way. So I can't see how this can fix the problem the workaround purports to fix?? >>> >>> http://hg.openjdk.java.net/macosx-port/macosx-port/hotspot/rev/be94a5de4d32#l54.758 >> The current version of LoadJavaVM (for Mac) contains: >> #ifndef STATIC_BUILD >> libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); >> #else >> libjvm = dlopen(NULL, RTLD_FIRST); >> #endif >> where STATIC_BUILD is controlled by the --enable-static-builds >> configure option. The --enable-static-builds option was added in JDK 9 >> by 8136556: Add the ability to perform static builds of MacOSX x64 >> binaries. > > Which "current version" is this? It is not what is in jdk/jdk repo, which has no STATIC_BUILD support in this area. ?? I think the MacOSX version of LoadJavaVM is here: ./java.base/macosx/native/libjli/java_md_macosx.m (Note that this appears to be an Obj-C source file.) (And no, I have no idea what build magic goes on to make this work.) This has the STATIC_BUILD stuff. There are also definitions for other platforms. ./java.base/windows/native/libjli/java_md.c // Windows ./java.base/unix/native/libjli/java_md_solinux.c // Solaris/Linux/AIX (Note that java_md_solinux.c is excluded for macosx by the build system; see LIBJLI_EXCLUDE_FILES in CoreLibraries.gmk.) > Although we may have not used RTLD_NOW with static builds (does this discussions re symbol resolution even make sense with a static build?) I think symbol resolution can make sense with a static build, if there are other libraries that are not statically linked. I don't know whether that's the case here. I'm going to try doing a static build without this dlopen code and see what happens. > in some version of the JDK, my point is that we have been using RTLD_NOW for as long as the workaround has been in place. > That strongly suggests to me that use of RTLD_NOW is not a solution to the problem the workaround was introduced for. I think my replacement code is equivalent to the deprecated code. I think the sequence of events may have been something like: (1) dlopen java lib without RTLD_NOW (2) run into problems that led to workaround code (3) run into further problems that led to dlopen java lib with RTLD_NOW but didn't remove the now superfluous workaround (perhaps it was forgotten?) (4) merge >> I thought this part of the change would be straight-forward. An >> alternative would be to leave the old deprecated call in place, >> locally disable the deprecation warning, and file an RFE for someone >> more knowledgable in this area and with Mac development to look at. > > Sorry for the obstructions on this but changing the code without knowing it even implements the same workaround is just wrong to me. I strongly suspect the code is not needed at all. I agree that for the non-STATIC_BUILD case the workaround is superfluous. If a static build without any workaround code works then I think the workaround code should just be removed. Is that okay with you? From jonathan.gibbons at oracle.com Tue Oct 2 21:39:50 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 02 Oct 2018 14:39:50 -0700 Subject: RFR(S): 8211350: Remove jprt support In-Reply-To: References: Message-ID: <5BB3E5A6.8070401@oracle.com> *Chuckle* at the long-obsolete reference in test/langtools/Makefile 292 apt: JTREG_TESTDIRS = tools/apt Otherwise, apart from other overdue cleanup, test/langtools/Makefile looks OK. -- Jon On 10/02/2018 12:21 AM, Mikael Vidstedt wrote: > Please review this change which removes support for, and references to, the (Oracle internal) JPRT system. > > bug: https://bugs.openjdk.java.net/browse/JDK-8211350 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8211350/webrev.00/open/webrev/ > > * Background (from the issue) > > The Oracle internal JPRT system was once used to build and test the JDK. It has been superseded and can no longer be used to build/test mainline JDK. The support in the JDK code for running jobs in JPRT should be removed. > > > * About the change > > The change touches several areas: build/make, hotspot, and jdk. I?m optimistically sending it out as a single webrev anyway. If it helps, I?m expecting the reviews to roughly map like so: > > build-dev: make related files, jprt.* > hotspot-dev: {src,test}/hotspot > core-libs-dev: test/jdk - I?m especially interested in hearing if the change to test/jdk/tools/lib/tests/Helper.java (to throw if the jmods directory is missing) is valid > > Of course I?d welcome reviews across those mappings as well. > > * Testing > > tier1 (passed), tier2-3 still running (looking good so far). > > Cheers, > Mikael > From david.holmes at oracle.com Tue Oct 2 22:10:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Oct 2018 08:10:12 +1000 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> Message-ID: On 3/10/2018 6:51 AM, Kim Barrett wrote: >> On Oct 1, 2018, at 7:10 PM, David Holmes wrote: >> >> On 2/10/2018 4:35 AM, Kim Barrett wrote: >>>> On Oct 1, 2018, at 2:27 AM, David Holmes wrote: >>>> >>>> On 1/10/2018 1:21 PM, Kim Barrett wrote: >>>>> The current library may have been previously only dlopen?ed with RTLD_LAZY. Reopening with RTLD_NOW >>>>> forces resolution of all undefined symbols in that shared object, which appears to be the purpose of the >>>>> deprecated function. >>>> >>>> The current library is libjvm and it's already opened in the appropriate way: >>>> >>>> libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); >>>> >>>> and it seems to have always been opened this way. So I can't see how this can fix the problem the workaround purports to fix?? >>>> >>>> http://hg.openjdk.java.net/macosx-port/macosx-port/hotspot/rev/be94a5de4d32#l54.758 >>> The current version of LoadJavaVM (for Mac) contains: >>> #ifndef STATIC_BUILD >>> libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); >>> #else >>> libjvm = dlopen(NULL, RTLD_FIRST); >>> #endif >>> where STATIC_BUILD is controlled by the --enable-static-builds >>> configure option. The --enable-static-builds option was added in JDK 9 >>> by 8136556: Add the ability to perform static builds of MacOSX x64 >>> binaries. >> >> Which "current version" is this? It is not what is in jdk/jdk repo, which has no STATIC_BUILD support in this area. ?? > > I think the MacOSX version of LoadJavaVM is here: > ./java.base/macosx/native/libjli/java_md_macosx.m > (Note that this appears to be an Obj-C source file.) > (And no, I have no idea what build magic goes on to make this work.) > This has the STATIC_BUILD stuff. Hmmm okay. But I can't find that file in the original mac port, or in 7u at all. In 7u we have jdk/src/macosx/bin/java_md_macosx.c with libjvm = dlopen(jvmpath, RTLD_NOW + RTLD_GLOBAL); and that was created in 2012 when the launcher was refactored under JDK-7124089. The archaeology is getting too hard. :( > There are also definitions for other platforms. > ./java.base/windows/native/libjli/java_md.c // Windows > ./java.base/unix/native/libjli/java_md_solinux.c // Solaris/Linux/AIX > (Note that java_md_solinux.c is excluded for macosx by the build > system; see LIBJLI_EXCLUDE_FILES in CoreLibraries.gmk.) > >> Although we may have not used RTLD_NOW with static builds (does this discussions re symbol resolution even make sense with a static build?) > > I think symbol resolution can make sense with a static build, if there > are other libraries that are not statically linked. I don't know > whether that's the case here. I'm going to try doing a static build > without this dlopen code and see what happens. > >> in some version of the JDK, my point is that we have been using RTLD_NOW for as long as the workaround has been in place. >> That strongly suggests to me that use of RTLD_NOW is not a solution to the problem the workaround was introduced for. > > I think my replacement code is equivalent to the deprecated code. I > think the sequence of events may have been something like: > > (1) dlopen java lib without RTLD_NOW > (2) run into problems that led to workaround code > (3) run into further problems that led to dlopen java lib with RTLD_NOW > but didn't remove the now superfluous workaround (perhaps it was > forgotten?) > (4) merge Perhaps. No way to know. > >>> I thought this part of the change would be straight-forward. An >>> alternative would be to leave the old deprecated call in place, >>> locally disable the deprecation warning, and file an RFE for someone >>> more knowledgable in this area and with Mac development to look at. >> >> Sorry for the obstructions on this but changing the code without knowing it even implements the same workaround is just wrong to me. I strongly suspect the code is not needed at all. > > I agree that for the non-STATIC_BUILD case the workaround is > superfluous. If a static build without any workaround code works then > I think the workaround code should just be removed. Is that okay with > you? Yes but you will need to go through a full round of testing, not just tier 1-3, before being reasonably confident the workaround is not needed. I'd hate to see this break things "in the wild" just because we don't have adequate test coverage. Other opinions welcomed. David > From kim.barrett at oracle.com Tue Oct 2 22:18:38 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Oct 2018 18:18:38 -0400 Subject: RFR: JDK-8211073 Remove -Wno-extra from Hotspot In-Reply-To: References: <671e162b-5580-2a0b-5115-847b8b26c4aa@oracle.com> <47ABAD88-9376-45D3-86DB-16A47B81F061@oracle.com> Message-ID: <55464073-BF13-4D30-9FA7-829778F698E8@oracle.com> > On Sep 29, 2018, at 1:36 PM, Magnus Ihse Bursie wrote: > > On 2018-09-28 23:22, Kim Barrett wrote: > >>> On Sep 24, 2018, at 4:31 PM, Magnus Ihse Bursie wrote: >>> The second warning about the copy constructor is, for what I can tell, a highly valid warning and the code it warned on was indeed broken. As far as I can tell, in a derived copy constructor you should always explicitly initialize the base class. >> I agree the copy constructor warnings are correct and indicate potentially serious bugs. >> These copy constructor changes look correct to me. >> >> The code that is being missed because of this bug is debug-only usage verification. I think >> bad things might happen if we C-heap allocated a ResourceObj and then copied that object. >> Maybe we fortuitously don?t ever do that? > If we had triggered problems we'd probably found out the issue by now, so its likely we're not doing anything that causes this to happen currently. But latent bugs like these are scary, and unnecessary, especially if we can get the compiler's help to avoid them. >> It?s unfortunate that the only way to get these warnings from gcc seems to be via -Wextra. > I agree, it's unfortunate. -Wextra in gcc actually means two things: a bunch of "extra" warnings, that you can turn on or off as a group only by enabling or disabling -Wextra, and a set of separate warnings that this option also turns on by default. The latter is more of a convenience to get a set of warnings that the gcc authors recommend for more in-depth warnings. Since they can be individually disabled, we don't need to care much about them. > > In regard to your previous mail, there has not been much change in the scope of -Wextra. Between gcc 4.8 and 7.3, no changes were made in the first part (the "extra" warnings), and only four more warnings were added to the second part (-Wcast-function-type, -Wimplicit-fallthrough=3, -Wredundant-move and -Wshift-negative-value), all of which can be turned off if we don't want them. > > In general, we try to make the product compile without warnings on common platforms, but as you say, unless you use one of the compilers that are regularly used (at Oracle or elsewhere), there's a risk of triggering new warnings. However, using configure --disable-warnings-as-errors makes the build pass anyway, and is already commonly in use by those building on more exotic platforms or compiler versions. > > I would have preferred to have individual warnings to enable, but gcc has not moved all warnings from -Wextra to individual warnings (new warnings, though, have all been given individual names). Since the warnings, as you agree, can find actual bugs, I think it's worth the hassle to enable -Wextra. (We already do that for all native code in OpenJDK, except hotspot, btw.) gcc8.2 adds -Wcast-function-type, which *might* cause us problems. We play some interesting games with function types in some places, and I don't know if they'll trigger this. We can turn it off explicitly if it is a problem (or perhaps preemptively). I went through all the associated warnings for gcc7.3 and categorized them. Some are just not interesting or relevant to us. I think these include -Wclobbered -Wignored-qualifiers -Wmissing-field-initializers -Wmissing-parameter-type -Wold-style-declaration -Woverride-init Ambiguous virtual bases. Subscripting an array that has been declared register Taking address of a variable declared register Some we are already using: -Wsign-compiler -Wtype-limits -Wuninitialized Some I think we should never enable (so need to explicitly disable if adding -Wextra): -Wempty-body -Wshift-negative-value -Wunused-parameter Perhaps provide some small benefit: -Wunused-but-set-parameter (but does nothing without -Wunused or -Wall) Relational compare of pointer against integer 0 -Wimplicit-fallthrough=3 requires a structured comment or C++17 feature to quiet the warning in code that is using the feature. Provides some benefit. I'm surprised there aren't any warnings from this. Intentional fall-through is a pretty common idiom. Enumerator and non-enumerator as results of conditional expression. Has a well-defined meaning, and used in a number of places. Some of those might be artifacts of some constants being defined using enum members rather than static const members. Problematic because of valid uses and can't be turned off. A base class is not initialized in the copy constructor of a derived class. Indicates a real bug, of which we have 3(?) occurrences, all of which should be fixed. But copy constructors are relatively rare in HotSpot, at least currently. Much of the benefit could be obtained by occasionally doing a build with this enabled and warnings-are-errors disabled and looking for these warnings. I think the benefit we get from detecting future occurrences of the copy constructor bug is not really worth the current cost of "fixing" the occurrences of conditional expressions with enumerators. I might feel differently if a pass were made through those to replace uses of enum with static const members. In summary, I think we shouldn't enable -Wextra. I think the current cost/benefit tradeoff is actually negative. One warning not individually controlled requires ugly workarounds or possibly non-trivial code changes to address, and I really dislike the ugly workarounds. If we do enable it, I think there are a number of specific warnings that ought to be explicitly disabled, either temporarily or permanently. From igor.ignatyev at oracle.com Tue Oct 2 23:26:32 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 2 Oct 2018 16:26:32 -0700 Subject: Test Libraries : Status and Further Steps Message-ID: Dear all, I am pleased to announce that we have finished work on 'Merge jdk and hotspot test libraries' and achieved the big milestone in our work on improving test libraries. Since this project started a long time, I think it's worth it to reiterate why we are doing it, what has been done, and what is planned to be done. So by this email, I hope to bring us all on the same page as well as to initiate discussion on possible further improvements. 1. Background and Past Work The project on merging test libraries started more than three years ago. The test libraries were created long before a consolidated repository, so HotSpot test suite and JDK test suite were not only in different directories but also in separate repositories, and if it was not enough, their developments were happening in different forests. All these were making it really hard to share test libraries and have them synchronized. However, people who work on both hotspot and JDK tests craved to have a consistent test library which covers most common use-cases, this is how several internal forks of test library emerged[1-2]. Obviously, it did not end well: test libraries started to grow differently, and the efforts to keep them in sync became more and more cumbersome. At some point in time, we accepted the forks being different and stopped any attempts to synchronize them and decided to introduce a common library which can be used by all test suites. The common library had been placed in '/test/lib' directory, which was in the top-level repository, and was based on HotSpot's fork (by the right of the "firstborn"). Christian made a huge and very valuable effort[3-4] on switching all HotSpot tests to use this library. Although that significantly increased sanity of HotSpot developers and simplified their life, it did not solve the original problem, as we still had multiple copies of a test library. By the end of JDK 9 time frame, we had three forks of test library: two deprecated in JAXP and JDK repositories and one in the top repository. As the top-level library was based on HotSpot' version, switching other test suites required more work and testing. 8075327[5] was created to switch JAXP and JDK test suites to use the common test library. It appeared that only a few JAXP tests were using JAXP version of the test library, so it was relatively easy and straightforward. JDK test library, on the contrary, was heavily used by the test suite. And as JDK test library had been forked back in 2013, it altered a lot from HotSpot's version: some functionality was missed in one version, some in another, some were implemented very differently, etc. Therefore the patch for JDK was not as easy as a simple change of @library/@build tags and import statements and became a long and tedious process of handling one test library class after another. 8211171[6-7] removed JarUtils and "handled" the last class existing in multiple versions of the common test libraries. 2. Future work Further improvements of the test libraries include handling the rest of JDK and JAXP test library classes; merging logical duplications within the test libraries; modularize the test libraries. Besides these changes, there are other possible enhancements and bug fixes in the test libraries, but they are out of the scope of this project. The following sections explain each of these three improvements in greater details. 2.1. Handling the rest of JDK and JAXP test libraries Currently, neither JDK nor JAXP test library contains classes from the common test library, but these test libraries still have some other classes which, in some cases, should be rearranged to improve the overall experience with the test libraries. One might suggest moving all these classes into the top-level library, but such an action would come at a high price. Placing classes into the top-level test library implies certain difficulties: you must assume that they are used by tests from all test suites, so all changes must be reviewed and tested accordingly, and the changes which break behavioral compatibility must be done with great care. Generally speaking, the further library classes away from tests, the greater risk to break other tests. Leaving classes as-is is another extreme and can also cause some complications. One such complication is a possibility to get a fork of library classes, which will move us back into 2013. Another is a creation of "unintended" test libraries. "Unintended" test library can be created because it is possible to use classes from any other tests (including from another test suite). Although, it is technically possible, expecting someone to test or fix someone else's tests when they change their internal classes would be unreasonable. Thus using classes which are not part of any test libraries and not part of the tests must be discouraged by guidelines and indicated during the review process. Thereby, the rest of the test libraries classes can be split into three categories and placed accordingly: 1) classes which are intended to be used by all test suites are to be moved to the top-level test library; 2) classes which are used by a certain test group and would have very low, if any, usefulness for other test suites. Such classes should be kept in a test library specific for a test suite. 3) classes which are very specific to a few particular tests should be moved closer to the tests which depend on them. I have looked through the remaining test library classes, made an initial assessment of their category, and wrote it done as comments in 8211290[8]. Since I am not an original author of these classes nor their user, I might be completely wrong in my evaluation, so I would really appreciate Core-Libs and JAXP team's opinion on them, and would also love them to check whenever I missed any other test library classes. 2.2. Merging duplications Due to the fact that the top-level test library has been created as a merge of commonly used test library classes, it has several classes which do a similar or the same thing but slightly differently. For example, there are two ways to get a reproducible random generator -- 'jdk.test.lib.Utils.RANDOM_GENERATOR' and 'jdk.test.lib.RandomFactory'. Such duplications have obvious negative impacts on test libraries. All cases of such duplication should be carefully reviewed and "merged" into one entity if possible. Here "merge" doesn't necessarily mean to get a class which provides both a set of APIs and "merge" can be achieved by removing one of the classes accompanied by conversions of the existing tests. Poor documentation and visibility of available test library classes seem to be the key causing factors of such duplications. Therefore this effort also includes documenting public classes of the test libraries and making this documentation easily available for Open JDK developers. 8211289[9] has been created to track internal merges and documentation effort. I made an initial identification of duplications and posted a proposed solution for each found case in the RFE. 2.3. Modularize Analyzing test library classes, one can notice that they are different in their dependencies and/or objectives. Thus it is logical to split them into separate entities and Jigsaw modules are a perfect fit for that. Using Jigsaw modules will allow us to express dependencies more easily and consistently. Although most of the test library depends only on the exported APIs of 'java.base' module, there are classes which have dependencies on other modules as well as classes which use internal packages, so it is logical to use this information to get the first grouping, otherwise, we will needlessly restrict usage of the test library. For example, 'jdk.test.lib.compiler.*' classes are the only classes in the top-level test library which depend on 'java.compiler' module, so it makes sense to place them into a dedicated module and name this module 'test.compiler' to express this dependency. The top-level classes which depend only on 'java.base' module and do not require special handling will be placed into one module which can be named 'test.base'. Since jtreg supports building Jigsaw modules natively, modularization will also guarantee that all classes are built by a simple and straightforward @build tag. Continuing with the example, tests which want to use 'test.base' module will have to have just two directives '@library /test/lib' and '@build test.base/*' to always get all the needed classes properly built. It is important to highlight that although modularization of the test library will make it possible to use a test library as a set of jigsaw modules, it still will be possible to use the test library almost as before without using modules at all, as this is necessary for a certain group of tests, e.g. ones which are strict on the list of available modules. The initial modularization proposal is based on the information extracted by 'jdeps' tool. 8211358[10] provides details on proposed modules, their names, packages/classes to include, and reasoning why a separate module is required. 3. Final Goal After all these improvements are implemented, we will have more organized modularized tests libraries which do not have irritating duplications. Common test libraries classes will be in '/test/lib', test libraries specific for test suites will be in 'lib' directory inside in a test suite. The tests will use these libraries by a few meaningful @library and @build directives, and the test libraries layout will look similar to the following: test |-- lib | |-- test.base | |-- test.compiler | |-- test.jfr | `-- test.whitebox |-- hotspot/lib | |-- test.hotspot.jvmti | `-- test.hotspot.ctw `-- jdk/lib |-- test.jdk.client `-- test.jdk.security 4. Summary Although significant progress has been achieved in this field, there is still a long way ahead of us. We have merged three distinct forks into one common test library available to use in any jtreg test suites. All the existing test suites have been switched to use the classes from this test library. Next steps aim at further improving Open JDK developer experience and make test libraries maintenance easier. 8211290, 8211289, and 8211358 have been filed for each step and contain more details on proposed action plans, I would really like to encourage interested parties to express their opinion in this email thread and/or in the corresponding RFE. Since the adoption of the test library as well as preventing duplication highly depend on awareness people about existing library classes, I would also appreciate people suggesting to use existing classes when appropriate more often during reviews. Thanks, -- Igor [1] https://bugs.openjdk.java.net/browse/JDK-8007142 [2] http://mail.openjdk.java.net/pipermail/serviceability-dev/2013-February/008215.html [3] https://bugs.openjdk.java.net/browse/JDK-8157957 [4] http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-August/024198.html [5] https://bugs.openjdk.java.net/browse/JDK-8075327 [6] https://bugs.openjdk.java.net/browse/JDK-8211171 [7] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-September/055702.html [8] https://bugs.openjdk.java.net/browse/JDK-8211290 [9] https://bugs.openjdk.java.net/browse/JDK-8211289 [10] https://bugs.openjdk.java.net/browse/JDK-8211358 From kim.barrett at oracle.com Tue Oct 2 23:30:45 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Oct 2018 19:30:45 -0400 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> Message-ID: <250FA229-7F95-4FB9-935D-7834A203072F@oracle.com> > On Oct 2, 2018, at 6:10 PM, David Holmes wrote: > The archaeology is getting too hard. :( No kidding! > Yes but you will need to go through a full round of testing, not just tier 1-3, before being reasonably confident the workaround is not needed. I'd hate to see this break things "in the wild" just because we don't have adequate test coverage. > > Other opinions welcomed. I was planning to hit it with tier1-8 and possibly other things if I can find them. JCK tests too; I saw some instructions for running those in mach5 recently. That all assumes that static build actually works at all; it doesn?t look like we test that configuration these days, so who knows what bit rot may have set in. From david.holmes at oracle.com Wed Oct 3 07:54:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Oct 2018 17:54:22 +1000 Subject: RFR (M): 8188764: Obsolete AssumeMP and then remove all support for non-MP builds In-Reply-To: References: <167caf7d-a6b8-e230-d2cf-ee7470780725@bell-sw.com> <25a98e2e-2523-d221-429a-4284938082a9@oracle.com> <321a4c56-dba1-2028-4920-257e542dc83c@oracle.com> Message-ID: <4a3c980d-75c0-c2fe-2ffd-ae806373a02c@oracle.com> These changes have now been pushed. David On 27/09/2018 11:01 PM, David Holmes wrote: > Webrev was updated in place. Second occurrence was renamed. > > Thanks, > David > > On 26/09/2018 9:49 AM, David Holmes wrote: >> Thanks Boris, I'll fix the label problem. >> >> I'm travelling at the moment so won't push till later next week. >> Hopefully you'll have access to the hardware by then. >> >> Thanks, >> David >> >> On 26/09/2018 9:47 AM, Boris Ulasevich wrote: >>> Hi David, >>> >>> ?? For this change the most interesting would be to check it with >>> ARMv5, but I do not have the hardware now. I hope I will get one >>> ARMv5 machine next week. >>> >>> ?? For ARMv7 the change is Ok. Almost Ok. There is a minor build issue: >>> >>> /home/boris/jdk-jdk/src/hotspot/cpu/arm/templateTable_arm.cpp:3724:9: >>> error: redeclaration of ?Label notVolatile? >>> ??? Label notVolatile; >>> ????????? ^~~~~~~~~~~ >>> /home/boris/jdk-jdk/src/hotspot/cpu/arm/templateTable_arm.cpp:3492:9: >>> note: ?Label notVolatile? previously declared here >>> ??? Label notVolatile; >>> ????????? ^~~~~~~~~~~ >>> /home/boris/jdk-jdk/src/hotspot/cpu/arm/templateTable_arm.cpp: In >>> static member function ?static void >>> TemplateTable::fast_storefield(TosState)?: >>> /home/boris/jdk-jdk/src/hotspot/cpu/arm/templateTable_arm.cpp:3885:9: >>> error: redeclaration of ?Label notVolatile? >>> ??? Label notVolatile; >>> ????????? ^~~~~~~~~~~ >>> /home/boris/jdk-jdk/src/hotspot/cpu/arm/templateTable_arm.cpp:3832:9: >>> note: ?Label notVolatile? previously declared here >>> ??? Label notVolatile; >>> ????????? ^~~~~~~~~~~ >>> >>> regards, >>> Boris >>> >>> On 23.09.2018 19:26, David Holmes wrote: >>>> Hi Boris, >>>> >>>> Here is latest webrev with the ARM code updated as well. >>>> >>>> http://cr.openjdk.java.net/~dholmes/8188764/webrev/ >>>> >>>> All is_MP removed except where needed for ARMv5 support, MP specific >>>> instructions, and controlling of biased locking. >>>> >>>> I checked the DMB variants and we only generate for ARMv7 anyway, so >>>> no issue there. >>>> >>>> Would very much appreciate whatever test builds you can do on >>>> different ARM variants Boris! >>>> >>>> Thanks, >>>> David >>>> >>>> On 21/09/2018 9:41 AM, David Holmes wrote: >>>>> Hi Boris, >>>>> >>>>> On 21/09/2018 9:34 AM, Boris Ulasevich wrote: >>>>>> Hi David, >>>>>> >>>>>> On 20.09.2018 18:26, David Holmes wrote: >>>>>>> Hi Boris, >>>>>>> >>>>>>> Thanks for jumping on this. I hope you didn't spend too much time >>>>>>> on it as I had actually started on the ARM code then decided I >>>>>>> didn't need to make the changes, so I'd already gone through >>>>>>> pretty much everything that is needed. My concern is with things >>>>>>> like the change in >>>>>>> >>>>>>> src/hotspot/cpu/arm/assembler_arm_32.hpp >>>>>>> >>>>>>> where you removed the is_MP() check but in fact that needs to >>>>>>> remain because the instruction pldw is not present on variants of >>>>>>> the v7 architecture without the multi-processor extensions: >>>>>> >>>>>> Ok. Good point. Sure, my hasty proposal to get rid of is_MP() >>>>>> check in arm32 codes was wrong. >>>>>> >>>>>> One note. I see there is an ambiguity here: is_MP() is a >>>>>> multiprocessor system flag (returns true when _processor_count != >>>>>> 1), but multi-processor extensions is an optional ARMv7 feature >>>>>> which is not related with processors count, so pldw usage should >>>>>> not be controlled by is_MP() flag. >>>>> >>>>> is_MP can't return true unless there is more than one processor >>>>> (ignoring the pre-initialization case), so if there's more than one >>>>> processor there must be a MP supprting architecture. >>>>> >>>>>>> http://www.keil.com/support/man/docs/armclang_asm/armclang_asm_pge1425899018492.htm >>>>>>> >>>>>>> That makes me wonder whether any of the memory barrier >>>>>>> instructions may also be missing in some v6/v7 variants as well? >>>>>> >>>>>> Nobody complained :) And Reference Manual says DMB/DSB are not >>>>>> optional. >>>>> >>>>> I need to double-check we don't define variants like "dmb sy" that >>>>> are architecture specific. >>>>> >>>>>>> Plus I have to account for uniprocessor ARMv5 so need to see >>>>>>> whether MacroAssembler::membar, for example, must also retain the >>>>>>> is_MP() check. >>>>>> >>>>>> Do you want to delete this check? Processor count check seems >>>>>> quite natural to see if it has sense to generate memory barriers. >>>>> >>>>> That is the general thrust of this change. We assume we always have >>>>> MP and so always issue memory barriers. >>>>> >>>>>> By the way, MacroAssembler::fast_lock have >>>>>> VM_Version::supports_ldrex() check assertion with message >>>>>> "unsupported, yet?". Looks like it is not going to work correctly >>>>>> on multiprocessor ARMv5. >>>>> >>>>> The ARM code does not support multiprocessor ARMv5. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>>> Also in: >>>>>>> >>>>>>> src/hotspot/cpu/arm/stubGenerator_arm.cpp >>>>>>> >>>>>>> you can't remove the !is_MP() check and related code as that is >>>>>>> needed for ARMv5 (uniprocessor) support. >>>>>>> >>>>>>> Similarly in >>>>>>> >>>>>>> src/hotspot/cpu/arm/vm_version_arm_32.cpp >>>>>>> >>>>>>> you'll probably still want to disable biased-locking for ARMv5. >>>>>>> >>>>>>> Cheers, >>>>>>> David >>>>>>> >>>>>>> On 20/09/2018 3:31 AM, Boris Ulasevich wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> ?> I have not updated the about-to-be-removed Oracle ARM port code. >>>>>>>> >>>>>>>> Ok, but we can do the changes in ARM32 codes leaving ARM64 intact. >>>>>>>> I would propose the following change - I think we can either add >>>>>>>> it to your update or to create a separate RFR after your patch >>>>>>>> and JEP-340 commit: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~bulasevich/8188764/webrev.00/ >>>>>>>> >>>>>>>> regards, >>>>>>>> Boris >>>>>>>> >>>>>>>> On 18.09.2018 08:32, David Holmes wrote: >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8188764 >>>>>>>>> Webrev: http://cr.openjdk.java.net/~dholmes/8188764/webrev/ >>>>>>>>> >>>>>>>>> As previously discussed in the RFC thread: >>>>>>>>> >>>>>>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-September/034166.html >>>>>>>>> >>>>>>>>> >>>>>>>>> this change obsoletes the AssumeMP flag and removes the bulk of >>>>>>>>> the logic that is conditional on os::is_MP() as follows: >>>>>>>>> >>>>>>>>> 1. to gate the introduction of MP-specific features, notably >>>>>>>>> memory barriers >>>>>>>>> >>>>>>>>> The is_MP check is removed and we will always issue memory >>>>>>>>> barriers >>>>>>>>> or pre-pend lock prefix etc. >>>>>>>>> >>>>>>>>> 2. to align certain patchable code sequences (method entry, >>>>>>>>> call sites) >>>>>>>>> >>>>>>>>> The is_MP is removed and we always align patchable locations. >>>>>>>>> >>>>>>>>> 3. to gate certain optimizations which are questionable on >>>>>>>>> uniprocessors >>>>>>>>> (see quicken_jni_functions) >>>>>>>>> >>>>>>>>> These are psuedo-memory-barriers where we manufacture a >>>>>>>>> data-dependency >>>>>>>>> instead of inserting mfence/lfence (x86 example). These are >>>>>>>>> treated as >>>>>>>>> #1 and is_MP is removed. >>>>>>>>> >>>>>>>>> 4. to gate optimizations which are desirable on uniprocessors >>>>>>>>> (mutex/monitor short circuits) >>>>>>>>> >>>>>>>>> These are spin controls and so is_MP remains. >>>>>>>>> >>>>>>>>> I have not updated the about-to-be-removed Oracle ARM port code. >>>>>>>>> >>>>>>>>> Testing: >>>>>>>>> ?? - Tiers 1 -3 (mach5) >>>>>>>>> >>>>>>>>> Performance run TBD. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David From kim.barrett at oracle.com Wed Oct 3 19:13:15 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 3 Oct 2018 15:13:15 -0400 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features Message-ID: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> I've submitted a JEP for (1) enabling the use of C++14 Language Features when building the JDK, (2) define a process for deciding and documenting which new features can be used or are forbidden in HotSpot code, (3) provide an initial list of permitted and forbidden new features. https://bugs.openjdk.java.net/browse/JDK-8208089 From mark.reinhold at oracle.com Wed Oct 3 23:55:37 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 03 Oct 2018 16:55:37 -0700 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: <20181003165537.306971476@eggemoggin.niobe.net> 2018/10/3 12:13:15 -0700, kim.barrett at oracle.com: > ... > > https://bugs.openjdk.java.net/browse/JDK-8208089 Or, more readably: https://openjdk.java.net/jeps/8208089 - Mark From goetz.lindenmaier at sap.com Thu Oct 4 06:59:03 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 4 Oct 2018 06:59:03 +0000 Subject: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: <71fb517fba794dc28291cf51afb6e8b9@sap.com> Hi, I appreciate this is handled in a JEP and well communicated! XLc, the compiler we use for AIX, might be a bottleneck here. But we currently have no compiler-constraints by products on our aix port of jdk12 (for jdk11 we must stay with the current compiler xlc 12 which does not support C++11, and jdk11 even should be compilable with aCC by HP for which we won't get new versions!) We will update our compiler for jdk/jdk to the most recent Xlc as soon as possible. To my current knowledge, Xlc 14 was dropped, and xlc 16 is to be released early 2019. It is supposed to support C++ 11, and also some C++ 14 features. Best regards, Goetz. > -----Original Message----- > From: core-libs-dev On Behalf > Of Kim Barrett > Sent: Mittwoch, 3. Oktober 2018 21:13 > To: hotspot-dev developers > Cc: build-dev ; core-libs- > dev at openjdk.java.net > Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features > > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089 From matthias.baesken at sap.com Thu Oct 4 07:31:13 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 4 Oct 2018 07:31:13 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> Message-ID: Hello, my proposal would be to only print uid : 1679 (testuser-name) by default and guard the rest of the info by some XX-flag, any good proposals for the flag-name are appreciated; for example : if (ExtendHsErrorFileByUserRelatedInformation) { // print those too : > > > euid : 1679 (testuser-name) > > > gid : 25 (testgroup) > > > egid : 25 (testgroup) > > > > > > umask: 0022 (removing ----w--w-) } Best regards, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Dienstag, 2. Oktober 2018 12:38 > To: 'David Holmes' ; 'hotspot- > dev at openjdk.java.net' > Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > > Hi David, I think the added info could be seen more or less in line with > what currently is reported in hs_err file . > For instance you usually see user-names and lots of paths from the system > in the hs_err file . > > In case the umask and gid is seen as more sensitive than that, one could > make the output switchable with an XX-flag ; > this would have the benefit of making the added output more clear to the > user/admin . > > Best regards, Matthias > > > > -----Original Message----- > > From: David Holmes > > Sent: Dienstag, 2. Oktober 2018 09:49 > > To: Baesken, Matthias ; 'hotspot- > > dev at openjdk.java.net' > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > Hi Matthias, > > > > On 2/10/2018 5:30 PM, Baesken, Matthias wrote: > > > Hello , please review this small enhancement to the hs_err file . > > > > > > Currently the hs_err file contains only limited OS user related > information. > > > Just the user name is printed via output of environment variables (at > least > > on Windows with USERNAME - output). > > > The enhanced output on UNIX would contain more information including > > uid, gid and umask : > > > > > > uid : 1679 (testuser) > > > euid : 1679 (testuser) > > > gid : 25 (testgroup) > > > egid : 25 (testgroup) > > > > > > umask: 0022 (removing ----w--w-) > > > > Could any of this be considered sensitive information by an end-user? > > > > Thanks, > > David > > > > > > > > ( Some of the info above could be found currently in error logging > output > > e.g. > > > attachListener_linux.cpp line 362 > > > log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", > > > (and the user name on Windows(-only) is in the env variables section). > > > > > > > > > > > > bug/webrev : > > > ---------------------- > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211326 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ > > > > > > > > > Thanks, Matthias > > > From david.holmes at oracle.com Thu Oct 4 07:44:16 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Oct 2018 17:44:16 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> Message-ID: <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Hi Matthias, I'm hoping others will chime in here as I: a) don't know if this information is actually useful for an error log of this kind; b) don't know if the information might be considered sensitive or not; and c) don't think it's worth the effort of adding a flag to control this. Plus the flag is only useful for trying to reproduce an issue; if it's a one-of failure then you've already missed out on the information in the log file. Cheers, David On 4/10/2018 5:31 PM, Baesken, Matthias wrote: > Hello, my proposal would be to only print > > uid : 1679 (testuser-name) > > by default and guard the rest of the info by some XX-flag, any good proposals for the flag-name are appreciated; > for example : > > if (ExtendHsErrorFileByUserRelatedInformation) { > > // print those too : > >>>> euid : 1679 (testuser-name) >>>> gid : 25 (testgroup) >>>> egid : 25 (testgroup) >>>> >>>> umask: 0022 (removing ----w--w-) > > } > > > Best regards, Matthias > > >> -----Original Message----- >> From: Baesken, Matthias >> Sent: Dienstag, 2. Oktober 2018 12:38 >> To: 'David Holmes' ; 'hotspot- >> dev at openjdk.java.net' >> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file >> >> Hi David, I think the added info could be seen more or less in line with >> what currently is reported in hs_err file . >> For instance you usually see user-names and lots of paths from the system >> in the hs_err file . >> >> In case the umask and gid is seen as more sensitive than that, one could >> make the output switchable with an XX-flag ; >> this would have the benefit of making the added output more clear to the >> user/admin . >> >> Best regards, Matthias >> >> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Dienstag, 2. Oktober 2018 09:49 >>> To: Baesken, Matthias ; 'hotspot- >>> dev at openjdk.java.net' >>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >>> >>> Hi Matthias, >>> >>> On 2/10/2018 5:30 PM, Baesken, Matthias wrote: >>>> Hello , please review this small enhancement to the hs_err file . >>>> >>>> Currently the hs_err file contains only limited OS user related >> information. >>>> Just the user name is printed via output of environment variables (at >> least >>> on Windows with USERNAME - output). >>>> The enhanced output on UNIX would contain more information including >>> uid, gid and umask : >>>> >>>> uid : 1679 (testuser) >>>> euid : 1679 (testuser) >>>> gid : 25 (testgroup) >>>> egid : 25 (testgroup) >>>> >>>> umask: 0022 (removing ----w--w-) >>> >>> Could any of this be considered sensitive information by an end-user? >>> >>> Thanks, >>> David >>> >>>> >>>> ( Some of the info above could be found currently in error logging >> output >>> e.g. >>>> attachListener_linux.cpp line 362 >>>> log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", >>>> (and the user name on Windows(-only) is in the env variables section). >>>> >>>> >>>> >>>> bug/webrev : >>>> ---------------------- >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8211326 >>>> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ >>>> >>>> >>>> Thanks, Matthias >>>> From thomas.stuefe at gmail.com Thu Oct 4 08:27:19 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 4 Oct 2018 10:27:19 +0200 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Message-ID: Hi David, On Thu, Oct 4, 2018 at 9:44 AM David Holmes wrote: > > Hi Matthias, > > I'm hoping others will chime in here as I: > > a) don't know if this information is actually useful for an error log of > this kind; > > b) don't know if the information might be considered sensitive or not; and > I have no opinion on (a) and (b). > c) don't think it's worth the effort of adding a flag to control this. > Plus the flag is only useful for trying to reproduce an issue; if it's a > one-of failure then you've already missed out on the information in the > log file. How about a more generic switch to control verbosity of the error report? The way we and you use the error files seem to differ. You seem to prefer them short and snappy and bare any security relevant details (as far as that is even possible in an hs-err file). As was once mentioned in a similar discussion, "OpenJDK hs-err files get posted verbatim in forums and bug reports". We use the hs-err files differently. They are usually handed down to us by our customers thru secure channels, and for us size does not matter much, nor does security relevant information since we have contracts with our customers. That has been a point of contention over and over again in the past. So I wonder whether one, or possibly two, general switches could keep both sides happy. Something like -XX:+ExtendedErrorReports" and possibly "-XX:+ErrorReportsIncludeSensitiveData". Those switches could be, by default, false in the OpenJDK. Any additions we add to error reporting where we cannot find an agreement we could make conditional on one or the other switch. What do you think? ..Thomas > > Cheers, > David > > > On 4/10/2018 5:31 PM, Baesken, Matthias wrote: > > Hello, my proposal would be to only print > > > > uid : 1679 (testuser-name) > > > > by default and guard the rest of the info by some XX-flag, any good proposals for the flag-name are appreciated; > > for example : > > > > if (ExtendHsErrorFileByUserRelatedInformation) { > > > > // print those too : > > > >>>> euid : 1679 (testuser-name) > >>>> gid : 25 (testgroup) > >>>> egid : 25 (testgroup) > >>>> > >>>> umask: 0022 (removing ----w--w-) > > > > } > > > > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: Baesken, Matthias > >> Sent: Dienstag, 2. Oktober 2018 12:38 > >> To: 'David Holmes' ; 'hotspot- > >> dev at openjdk.java.net' > >> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > >> > >> Hi David, I think the added info could be seen more or less in line with > >> what currently is reported in hs_err file . > >> For instance you usually see user-names and lots of paths from the system > >> in the hs_err file . > >> > >> In case the umask and gid is seen as more sensitive than that, one could > >> make the output switchable with an XX-flag ; > >> this would have the benefit of making the added output more clear to the > >> user/admin . > >> > >> Best regards, Matthias > >> > >> > >>> -----Original Message----- > >>> From: David Holmes > >>> Sent: Dienstag, 2. Oktober 2018 09:49 > >>> To: Baesken, Matthias ; 'hotspot- > >>> dev at openjdk.java.net' > >>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > >>> > >>> Hi Matthias, > >>> > >>> On 2/10/2018 5:30 PM, Baesken, Matthias wrote: > >>>> Hello , please review this small enhancement to the hs_err file . > >>>> > >>>> Currently the hs_err file contains only limited OS user related > >> information. > >>>> Just the user name is printed via output of environment variables (at > >> least > >>> on Windows with USERNAME - output). > >>>> The enhanced output on UNIX would contain more information including > >>> uid, gid and umask : > >>>> > >>>> uid : 1679 (testuser) > >>>> euid : 1679 (testuser) > >>>> gid : 25 (testgroup) > >>>> egid : 25 (testgroup) > >>>> > >>>> umask: 0022 (removing ----w--w-) > >>> > >>> Could any of this be considered sensitive information by an end-user? > >>> > >>> Thanks, > >>> David > >>> > >>>> > >>>> ( Some of the info above could be found currently in error logging > >> output > >>> e.g. > >>>> attachListener_linux.cpp line 362 > >>>> log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", > >>>> (and the user name on Windows(-only) is in the env variables section). > >>>> > >>>> > >>>> > >>>> bug/webrev : > >>>> ---------------------- > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8211326 > >>>> > >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ > >>>> > >>>> > >>>> Thanks, Matthias > >>>> From volker.simonis at gmail.com Thu Oct 4 08:35:06 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 4 Oct 2018 11:35:06 +0300 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: Hi Tim, Jeff, Kim has assembled a quite detailed list of new C++ features intended for use in HotSpot/OpenJDK (with links to the corresponding C++ standard) https://bugs.openjdk.java.net/browse/JDK-8208089 Could you please provide a summary of which of these features are already available since which version of xlC for AIX and potentially escalate the implementation of the missing features within IBM?s xlC on AIX team. Thank you and best regards, Volker Kim Barrett schrieb am Mi. 3. Okt. 2018 um 22:13: > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089 > > From martijnverburg at gmail.com Thu Oct 4 08:57:43 2018 From: martijnverburg at gmail.com (Martijn Verburg) Date: Thu, 4 Oct 2018 09:57:43 +0100 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: Hi Kim, I like this initiative. I'm wondering if some of these rules can be easily codified or written into a jcheck style checker (ccheck?) so that Authors can adhere to the conventions and not rely on a Human review to pick out where that convention isn't met. Cheers, Martijn On Wed, 3 Oct 2018 at 20:13, Kim Barrett wrote: > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089 > > From david.holmes at oracle.com Thu Oct 4 09:18:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Oct 2018 19:18:10 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Message-ID: Hi Thomas, On 4/10/2018 6:27 PM, Thomas St?fe wrote: > Hi David, > On Thu, Oct 4, 2018 at 9:44 AM David Holmes wrote: >> >> Hi Matthias, >> >> I'm hoping others will chime in here as I: >> >> a) don't know if this information is actually useful for an error log of >> this kind; >> >> b) don't know if the information might be considered sensitive or not; and >> > > I have no opinion on (a) and (b). That's a shame :) >> c) don't think it's worth the effort of adding a flag to control this. >> Plus the flag is only useful for trying to reproduce an issue; if it's a >> one-of failure then you've already missed out on the information in the >> log file. > > How about a more generic switch to control verbosity of the error report? > > The way we and you use the error files seem to differ. You seem to > prefer them short and snappy and bare any security relevant details > (as far as that is even possible in an hs-err file). As was once > mentioned in a similar discussion, "OpenJDK hs-err files get posted > verbatim in forums and bug reports". > > We use the hs-err files differently. They are usually handed down to > us by our customers thru secure channels, and for us size does not > matter much, nor does security relevant information since we have > contracts with our customers. To be honest I don't deal with hs-err files from other people very much at all. But from what I have dealt with I haven't, to the best of my recollection, encountered any crash investigation where all this extra info would have shed any light. But most crashes I investigate are developer related rather than end user. > That has been a point of contention over and over again in the past. > So I wonder whether one, or possibly two, general switches could keep > both sides happy. Something like -XX:+ExtendedErrorReports" and > possibly "-XX:+ErrorReportsIncludeSensitiveData". > > Those switches could be, by default, false in the OpenJDK. > > Any additions we add to error reporting where we cannot find an > agreement we could make conditional on one or the other switch. > > What do you think? Seems reasonable to propose. Any new flag will require a CSR request. David > ..Thomas > >> >> Cheers, >> David >> >> >> On 4/10/2018 5:31 PM, Baesken, Matthias wrote: >>> Hello, my proposal would be to only print >>> >>> uid : 1679 (testuser-name) >>> >>> by default and guard the rest of the info by some XX-flag, any good proposals for the flag-name are appreciated; >>> for example : >>> >>> if (ExtendHsErrorFileByUserRelatedInformation) { >>> >>> // print those too : >>> >>>>>> euid : 1679 (testuser-name) >>>>>> gid : 25 (testgroup) >>>>>> egid : 25 (testgroup) >>>>>> >>>>>> umask: 0022 (removing ----w--w-) >>> >>> } >>> >>> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: Baesken, Matthias >>>> Sent: Dienstag, 2. Oktober 2018 12:38 >>>> To: 'David Holmes' ; 'hotspot- >>>> dev at openjdk.java.net' >>>> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file >>>> >>>> Hi David, I think the added info could be seen more or less in line with >>>> what currently is reported in hs_err file . >>>> For instance you usually see user-names and lots of paths from the system >>>> in the hs_err file . >>>> >>>> In case the umask and gid is seen as more sensitive than that, one could >>>> make the output switchable with an XX-flag ; >>>> this would have the benefit of making the added output more clear to the >>>> user/admin . >>>> >>>> Best regards, Matthias >>>> >>>> >>>>> -----Original Message----- >>>>> From: David Holmes >>>>> Sent: Dienstag, 2. Oktober 2018 09:49 >>>>> To: Baesken, Matthias ; 'hotspot- >>>>> dev at openjdk.java.net' >>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >>>>> >>>>> Hi Matthias, >>>>> >>>>> On 2/10/2018 5:30 PM, Baesken, Matthias wrote: >>>>>> Hello , please review this small enhancement to the hs_err file . >>>>>> >>>>>> Currently the hs_err file contains only limited OS user related >>>> information. >>>>>> Just the user name is printed via output of environment variables (at >>>> least >>>>> on Windows with USERNAME - output). >>>>>> The enhanced output on UNIX would contain more information including >>>>> uid, gid and umask : >>>>>> >>>>>> uid : 1679 (testuser) >>>>>> euid : 1679 (testuser) >>>>>> gid : 25 (testgroup) >>>>>> egid : 25 (testgroup) >>>>>> >>>>>> umask: 0022 (removing ----w--w-) >>>>> >>>>> Could any of this be considered sensitive information by an end-user? >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> >>>>>> ( Some of the info above could be found currently in error logging >>>> output >>>>> e.g. >>>>>> attachListener_linux.cpp line 362 >>>>>> log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", >>>>>> (and the user name on Windows(-only) is in the env variables section). >>>>>> >>>>>> >>>>>> >>>>>> bug/webrev : >>>>>> ---------------------- >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8211326 >>>>>> >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ >>>>>> >>>>>> >>>>>> Thanks, Matthias >>>>>> From coleen.phillimore at oracle.com Thu Oct 4 11:45:25 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Oct 2018 07:45:25 -0400 Subject: RFR (S) 8209889: RedefineStress tests crash Message-ID: Summary: Create CompileTaskWrapper before potential safepoint open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8209889 MetadataOnStackMark needs to find the CompileTask in the current thread, or else the Method isn't marked as being alive, so if it's old/obsolete, it is freed.? There are checks to disable adding old methods to the compile queue but if the method is redefined after this point, there may be a preexisting bug that we compile an old method.? This change just fixes the crash. Can some compiler person (Martin?) look at where I moved possibly_add_compiler_threads and let me know if that looks okay? There's a comment in the change that says why you need to create the CompileTaskWrapper before that call.? I couldn't find a place to put a NoSafepointVerifier. Tested with the test that failed, which doesn't reproduce the bug except rarely, and test/hotspot/jtreg/compiler tests.? tier1 an 2 in progress. Thanks, Coleen From matthias.baesken at sap.com Thu Oct 4 12:17:09 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 4 Oct 2018 12:17:09 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Message-ID: Hello David and Thomas, regarding (a) - on Windows the user name is already shown in the hs_err, because the value of the environment variable USERNAME is in, so if this is seen as valuable on Windows I do not see any reason why it shouldn?t be added on UNIX as well . The umask - info was a wish by some of our support people, so it is there for good reason too . > > both sides happy. Something like -XX:+ExtendedErrorReports" and > > possibly "-XX:+ErrorReportsIncludeSensitiveData". > > > > Seems reasonable to propose. Any new flag will require a CSR request. > Sure , we can open a CSR request . I think the proposal of Thomas is OK , of course we can discuss what the best flag namings would be . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 4. Oktober 2018 11:18 > To: Thomas St?fe > Cc: Baesken, Matthias ; HotSpot Open Source > Developers > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > Hi Thomas, > > On 4/10/2018 6:27 PM, Thomas St?fe wrote: > > Hi David, > > On Thu, Oct 4, 2018 at 9:44 AM David Holmes > wrote: > >> > >> Hi Matthias, > >> > >> I'm hoping others will chime in here as I: > >> > >> a) don't know if this information is actually useful for an error log of > >> this kind; > >> > >> b) don't know if the information might be considered sensitive or not; and > >> > > > > I have no opinion on (a) and (b). > > That's a shame :) > > >> c) don't think it's worth the effort of adding a flag to control this. > >> Plus the flag is only useful for trying to reproduce an issue; if it's a > >> one-of failure then you've already missed out on the information in the > >> log file. > > > > How about a more generic switch to control verbosity of the error report? > > > > The way we and you use the error files seem to differ. You seem to > > prefer them short and snappy and bare any security relevant details > > (as far as that is even possible in an hs-err file). As was once > > mentioned in a similar discussion, "OpenJDK hs-err files get posted > > verbatim in forums and bug reports". > > > > We use the hs-err files differently. They are usually handed down to > > us by our customers thru secure channels, and for us size does not > > matter much, nor does security relevant information since we have > > contracts with our customers. > > To be honest I don't deal with hs-err files from other people very much > at all. But from what I have dealt with I haven't, to the best of my > recollection, encountered any crash investigation where all this extra > info would have shed any light. But most crashes I investigate are > developer related rather than end user. > > > That has been a point of contention over and over again in the past. > > So I wonder whether one, or possibly two, general switches could keep > > both sides happy. Something like -XX:+ExtendedErrorReports" and > > possibly "-XX:+ErrorReportsIncludeSensitiveData". > > > > Those switches could be, by default, false in the OpenJDK. > > > > Any additions we add to error reporting where we cannot find an > > agreement we could make conditional on one or the other switch. > > > > What do you think? > > Seems reasonable to propose. Any new flag will require a CSR request. > > David > > > ..Thomas > > > >> > >> Cheers, > >> David > >> > >> > >> On 4/10/2018 5:31 PM, Baesken, Matthias wrote: > >>> Hello, my proposal would be to only print > >>> > >>> uid : 1679 (testuser-name) > >>> > >>> by default and guard the rest of the info by some XX-flag, any good > proposals for the flag-name are appreciated; > >>> for example : > >>> > >>> if (ExtendHsErrorFileByUserRelatedInformation) { > >>> > >>> // print those too : > >>> > >>>>>> euid : 1679 (testuser-name) > >>>>>> gid : 25 (testgroup) > >>>>>> egid : 25 (testgroup) > >>>>>> > >>>>>> umask: 0022 (removing ----w--w-) > >>> > >>> } > >>> > >>> > >>> Best regards, Matthias > >>> > >>> > >>>> -----Original Message----- > >>>> From: Baesken, Matthias > >>>> Sent: Dienstag, 2. Oktober 2018 12:38 > >>>> To: 'David Holmes' ; 'hotspot- > >>>> dev at openjdk.java.net' > >>>> Subject: RE: RFR : 8211326 : add OS user related information to hs_err > file > >>>> > >>>> Hi David, I think the added info could be seen more or less in line > with > >>>> what currently is reported in hs_err file . > >>>> For instance you usually see user-names and lots of paths from the > system > >>>> in the hs_err file . > >>>> > >>>> In case the umask and gid is seen as more sensitive than that, one > could > >>>> make the output switchable with an XX-flag ; > >>>> this would have the benefit of making the added output more clear > to the > >>>> user/admin . > >>>> > >>>> Best regards, Matthias > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: David Holmes > >>>>> Sent: Dienstag, 2. Oktober 2018 09:49 > >>>>> To: Baesken, Matthias ; 'hotspot- > >>>>> dev at openjdk.java.net' > >>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err > file > >>>>> > >>>>> Hi Matthias, > >>>>> > >>>>> On 2/10/2018 5:30 PM, Baesken, Matthias wrote: > >>>>>> Hello , please review this small enhancement to the hs_err file . > >>>>>> > >>>>>> Currently the hs_err file contains only limited OS user related > >>>> information. > >>>>>> Just the user name is printed via output of environment variables (at > >>>> least > >>>>> on Windows with USERNAME - output). > >>>>>> The enhanced output on UNIX would contain more information > including > >>>>> uid, gid and umask : > >>>>>> > >>>>>> uid : 1679 (testuser) > >>>>>> euid : 1679 (testuser) > >>>>>> gid : 25 (testgroup) > >>>>>> egid : 25 (testgroup) > >>>>>> > >>>>>> umask: 0022 (removing ----w--w-) > >>>>> > >>>>> Could any of this be considered sensitive information by an end-user? > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> > >>>>>> > >>>>>> ( Some of the info above could be found currently in error logging > >>>> output > >>>>> e.g. > >>>>>> attachListener_linux.cpp line 362 > >>>>>> log_debug(attach)("euid/egid check failed (%d/%d vs > %d/%d)", > >>>>>> (and the user name on Windows(-only) is in the env variables > section). > >>>>>> > >>>>>> > >>>>>> > >>>>>> bug/webrev : > >>>>>> ---------------------- > >>>>>> > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8211326 > >>>>>> > >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ > >>>>>> > >>>>>> > >>>>>> Thanks, Matthias > >>>>>> From magnus.ihse.bursie at oracle.com Thu Oct 4 12:31:43 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 4 Oct 2018 14:31:43 +0200 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: <940757E3-8781-4E7A-B5B9-1F0D3E944E0F@oracle.com> > 4 okt. 2018 kl. 10:57 skrev Martijn Verburg : > > Hi Kim, > > I like this initiative. I'm wondering if some of these rules can be easily > codified or written into a jcheck style checker (ccheck?) so that Authors > can adhere to the conventions and not rely on a Human review to pick out > where that convention isn't met. That's an interesting thought! I googled around a bit, but could find no obvious candidate for doing such a check. In fact, parsing and analyzing C++ seems quite a hard problem, compared to many other languages. (Sad, but not really surprising.) I found two possible routes to explore: cpplint [1], the official Google tool to verify that the Google C++ Style Guide [2] is followed, and Vera++ [3], a framework for creating scripts that can analyze and/or transform C++ code. Neither seem like an ideal solution. The Google tool is hard coded to support the Google rules. It's possible we can fork it and add our own, but it's not clear that this is even possible, much less so that it's a good way forward. Vera++, on the other hand, seems much more likely to be able to provide the tooling needed to write checks that enforce these rules. However, what we have is just a framework, and someone's got to write those rules (in TCL of all languages...). Then again, Vera++ is an old and quite popular tool, so its possible there is already a bunch of predefined rules that we can use as a starting point out there. (I didn't go that far in my analysis). Apart from these two, there appear to be no popular, well-encompassing, open source C++ checker out there, that could possibly be verifying rules like these. :( /Magnus [1] https://github.com/google/styleguide/tree/gh-pages/cpplint [2] https://google.github.io/styleguide/cppguide.html [3] https://bitbucket.org/verateam/vera/wiki/Introduction > > Cheers, > Martijn > > >> On Wed, 3 Oct 2018 at 20:13, Kim Barrett wrote: >> >> I've submitted a JEP for >> >> (1) enabling the use of C++14 Language Features when building the JDK, >> >> (2) define a process for deciding and documenting which new features >> can be used or are forbidden in HotSpot code, >> >> (3) provide an initial list of permitted and forbidden new features. >> >> https://bugs.openjdk.java.net/browse/JDK-8208089 >> >> From sgehwolf at redhat.com Thu Oct 4 13:03:34 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 04 Oct 2018 15:03:34 +0200 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 Message-ID: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> Hi, Please review this forward port for Zero coming from JDK 7 (where Zero is maintained in icedtea) to JDK 12. The update is to use ldrexd instructions on ARMv7 CPUs for atomic_copy64() in Zero: Bug: https://bugs.openjdk.java.net/browse/JDK-8211387 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8211387/webrev.01/ Testing: Zero bootcycle-images build on armv7 where this was satisfied: $ gcc -dM -E - < /dev/null | grep __ARM_ARCH_7A__ #define __ARM_ARCH_7A__ 1 This change was contributed by Andrew Haley. Thoughts? Thanks, Severin From magnus.ihse.bursie at oracle.com Thu Oct 4 13:14:34 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 4 Oct 2018 15:14:34 +0200 Subject: RFR: JDK-8211073 Remove -Wno-extra from Hotspot In-Reply-To: <55464073-BF13-4D30-9FA7-829778F698E8@oracle.com> References: <671e162b-5580-2a0b-5115-847b8b26c4aa@oracle.com> <47ABAD88-9376-45D3-86DB-16A47B81F061@oracle.com> <55464073-BF13-4D30-9FA7-829778F698E8@oracle.com> Message-ID: <1340C14C-265B-4DC7-A00E-0C3BD3B42AD0@oracle.com> Kim, Thank you for your detailed analysis! Comments inline... 3 okt. 2018 kl. 00:18 skrev Kim Barrett : >> On Sep 29, 2018, at 1:36 PM, Magnus Ihse Bursie wrote: >> >> On 2018-09-28 23:22, Kim Barrett wrote: >> >>>> On Sep 24, 2018, at 4:31 PM, Magnus Ihse Bursie wrote: >>>> The second warning about the copy constructor is, for what I can tell, a highly valid warning and the code it warned on was indeed broken. As far as I can tell, in a derived copy constructor you should always explicitly initialize the base class. >>> I agree the copy constructor warnings are correct and indicate potentially serious bugs. >>> These copy constructor changes look correct to me. >>> >>> The code that is being missed because of this bug is debug-only usage verification. I think >>> bad things might happen if we C-heap allocated a ResourceObj and then copied that object. >>> Maybe we fortuitously don?t ever do that? >> If we had triggered problems we'd probably found out the issue by now, so its likely we're not doing anything that causes this to happen currently. But latent bugs like these are scary, and unnecessary, especially if we can get the compiler's help to avoid them. >>> It?s unfortunate that the only way to get these warnings from gcc seems to be via -Wextra. >> I agree, it's unfortunate. -Wextra in gcc actually means two things: a bunch of "extra" warnings, that you can turn on or off as a group only by enabling or disabling -Wextra, and a set of separate warnings that this option also turns on by default. The latter is more of a convenience to get a set of warnings that the gcc authors recommend for more in-depth warnings. Since they can be individually disabled, we don't need to care much about them. >> >> In regard to your previous mail, there has not been much change in the scope of -Wextra. Between gcc 4.8 and 7.3, no changes were made in the first part (the "extra" warnings), and only four more warnings were added to the second part (-Wcast-function-type, -Wimplicit-fallthrough=3, -Wredundant-move and -Wshift-negative-value), all of which can be turned off if we don't want them. >> >> In general, we try to make the product compile without warnings on common platforms, but as you say, unless you use one of the compilers that are regularly used (at Oracle or elsewhere), there's a risk of triggering new warnings. However, using configure --disable-warnings-as-errors makes the build pass anyway, and is already commonly in use by those building on more exotic platforms or compiler versions. >> >> I would have preferred to have individual warnings to enable, but gcc has not moved all warnings from -Wextra to individual warnings (new warnings, though, have all been given individual names). Since the warnings, as you agree, can find actual bugs, I think it's worth the hassle to enable -Wextra. (We already do that for all native code in OpenJDK, except hotspot, btw.) > > gcc8.2 adds -Wcast-function-type, which *might* cause us problems. We > play some interesting games with function types in some places, and I > don't know if they'll trigger this. We can turn it off explicitly if > it is a problem (or perhaps preemptively). > > I went through all the associated warnings for gcc7.3 and categorized > them. Some are just not interesting or relevant to us. I think these > include > > -Wclobbered > -Wignored-qualifiers > -Wmissing-field-initializers > -Wmissing-parameter-type > -Wold-style-declaration > -Woverride-init > Ambiguous virtual bases. > Subscripting an array that has been declared register > Taking address of a variable declared register I agree that some, like clobbered, is not relevant (but I think that also only applies to C code..?). In general, it doesn't hurt though, to turn on warnings that are unlikely to be triggered. I realize it also helps little to nothing, but once again, if this is the price we have to pay for catching some real problems, it's a small price indeed. OTOH, I don't really agree that old-style-declaration is irrelevant. At least in the JDK libraries there are multiple places where non-prototype declarations are made, and apart from the style issue, it's a bug waiting to happen if the function change its signature. I don't remember if this is the case in Hotspot too, but it wouldn't surprise me, if we don't check for it. > Some we are already using: > -Wsign-compiler > -Wtype-limits > -Wuninitialized > > Some I think we should never enable (so need to explicitly disable if > adding -Wextra): > -Wempty-body > -Wshift-negative-value > -Wunused-parameter I agree that unused-parameter is worthless. I think we should disable it globally, if that's not already done. empty-body is mostly good at catching style issues, like double semicolons. It's not a big deal, but it also never hurts to keep up good code quality. I have not seen it make any false positives. shift-negative-values is a bit beyond me, but as I understand it, it warns about a behavior that has changed recently, or will be changed, in the standards, to become undefined. That sounds to me like something that it would be better to act preemptively on. But then again, I might have misunderstood things. If so, we can definitely turn it off. > > Perhaps provide some small benefit: > -Wunused-but-set-parameter (but does nothing without -Wunused or -Wall) > Relational compare of pointer against integer 0 > > -Wimplicit-fallthrough=3 requires a structured comment or C++17 > feature to quiet the warning in code that is using the feature. > Provides some benefit. I'm surprised there aren't any warnings from > this. Intentional fall-through is a pretty common idiom. At level 3, the "structured" comment is just something like a line like this: // fallthrough Most, maybe all, fallthroughs in the hotspot code already does that. I think it's a reasonable requirement to show that the fallthrough is intentional. If you think it's too strong, we can lower it to level 1, which just requires a comment, with whatever text (or even empty, I think). > > Enumerator and non-enumerator as results of conditional expression. > Has a well-defined meaning, and used in a number of places. Some of > those might be artifacts of some constants being defined using enum > members rather than static const members. Problematic because of > valid uses and can't be turned off. Agree, this is the Achilles heel really. It's too bad it can't be turned off. :( The remaining code changes in my patch (apart from the copy constructor) are all due to this. > > A base class is not initialized in the copy constructor of a derived > class. Indicates a real bug, of which we have 3(?) occurrences, all of > which should be fixed. But copy constructors are relatively rare in > HotSpot, at least currently. Much of the benefit could be obtained by > occasionally doing a build with this enabled and warnings-are-errors > disabled and looking for these warnings. Only this is never going to happen. :( In general, I really much prefer for issues that can be detected automatically to be detected automatically. One-shot checks are unlikely to happen, and if done seldom enough will only accumulate so much "cruft" that it's too much work and too boring work for anyone to bother doing. > I think the benefit we get from detecting future occurrences of the > copy constructor bug is not really worth the current cost of "fixing" > the occurrences of conditional expressions with enumerators. I might > feel differently if a pass were made through those to replace uses of > enum with static const members. So I think this is what it really boils down to. Just to summarize/repeat what you said, so I'm sure we understand each other correctly: There's an upside to enabling Wextra; we know already that it found some real bugs, but the real benefit is all future potential bugs it will prohibit, and that's hard to quantify. On the other hand, there's a downside; and that's the conditional enum warnings, whose fixes are ugly. But, if the code was cleaned up, so those enum fixes were no longer ugly, then it would be a net positive to turn on Wextra. I think that's a reasonable solution for now. If we agree on this, I'll open a new bug with just the copy constructor fixes, and a enhancement request for replacing the enums with static const, and making the current bug ("turn on Wextra") dependent on that. I just want to add a final remark about compiler warnings. When I started the current work of normalizing the set of warnings in the entire OpenJDK, I was amazed at what low level of warnings that was used for gcc/clang in hotspot. In all the JDK native libraries, -Wall and -Wextra were on by default. In hotspot, not even -Wall were turned on! I think what have sort of "saved" you is that Microsoft CL was running with /W3, which is roughly equivalent to -Wall -Wextra. So most of the shared code compiled fine when adding -Wall -Wextra, but there was a lot of new warnings in non-Windows code. Of course, /W3 and -Wall -Wextra does not cover exactly the same set of warnings, nor implement similar warning checks identically, hence the current open set of warnings in shared code. I made a long list of warnings that are disabled due to this, but I think it would be a good idea to tackle them one by one. By a quick glance, most of them (but surely not all) seemed to point to real problematic code; if not buggy so at least bad style. > > In summary, I think we shouldn't enable -Wextra. I think the current > cost/benefit tradeoff is actually negative. One warning not > individually controlled requires ugly workarounds or possibly > non-trivial code changes to address, and I really dislike the ugly > workarounds. > > If we do enable it, I think there are a number of specific warnings > that ought to be explicitly disabled, either temporarily or > permanently. Yes, definitely. Had Wextra been *only* a predefined set of other named warnings, we wouldn't have needed to have this discussion. But since it works the way it does, it means that when we turn it on, we have to turn off the included warnings that we do not want. /Magnus From aleksei.voitylov at bell-sw.com Thu Oct 4 13:17:21 2018 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Thu, 4 Oct 2018 16:17:21 +0300 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> Kim, Thank you for posting this. It's an important step to make. I wanted to clarify a couple of points: 1. Since this is infrastructure JEP, is the version of JDK which will undergo such changes going to be some future version or does it apply to past versions as well? I'm concerned with how we can simplify security backports to 8u which (I currently assume) is not subject to this change. 2. Which versions of GCC do you tentatively consider at this point? Non-x86 ports may have a problem upgrading to a specific version of GCC which the shared code will use and may need additional lead time to adjust. Thanks, -Aleksei On 03/10/2018 22:13, Kim Barrett wrote: > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089 > From magnus.ihse.bursie at oracle.com Thu Oct 4 13:27:43 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 4 Oct 2018 15:27:43 +0200 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: <250FA229-7F95-4FB9-935D-7834A203072F@oracle.com> References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> <250FA229-7F95-4FB9-935D-7834A203072F@oracle.com> Message-ID: 3 okt. 2018 kl. 01:30 skrev Kim Barrett : >> On Oct 2, 2018, at 6:10 PM, David Holmes wrote: >> The archaeology is getting too hard. :( > > No kidding! > >> Yes but you will need to go through a full round of testing, not just tier 1-3, before being reasonably confident the workaround is not needed. I'd hate to see this break things "in the wild" just because we don't have adequate test coverage. >> >> Other opinions welcomed. > > I was planning to hit it with tier1-8 and possibly other things if I can find them. > JCK tests too; I saw some instructions for running those in mach5 recently. > > That all assumes that static build actually works at all; it doesn?t look like we > test that configuration these days, so who knows what bit rot may have set in. > No, we don't test static build, and never have. I'd be surprised if it's still possible to build a static build. This was a feature developed for the mobile project. The static build part was merged, somewhat hastily, into mainline, but the rest of the mobile code is still lingering in the mobile project fork. The entire "static build" concept was somewhat artificially injected in some parts of the build code. It is for instance not possible to build all modules with a static build. At the very least you need to build without java.desktop, if I remember correctly. The motivation for the static build is that on iOS, apps are not allowed to load dynamic libraries (for some reason only Apple knows). So a developer that wants to build a Java app needs to have a static library version of the entire JDK to link with into a single executable Java launcher. So in this perspective, the entire use of dlopen in a static build smells funny, and might be just a left-over from static build being more of a hack than a properly supported build option. /Magnus From martin.doerr at sap.com Thu Oct 4 14:34:59 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 4 Oct 2018 14:34:59 +0000 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: References: Message-ID: <68a26e6e6e574f75aec64ede7487a117@sap.com> Hi Coleen, good catch. Looks good where you moved it. I couldn?t find a place for a NoSafepointVerifier, either. Maybe it?d be more clear if we replaced the ?continue? in the NULL case by an else case in the loop so the CompileTaskWrapper is the first thing after the NULL check (see below). Best regards, Martin --- a/src/hotspot/share/compiler/compileBroker.cpp Thu Oct 04 14:03:13 2018 +0200 +++ b/src/hotspot/share/compiler/compileBroker.cpp Thu Oct 04 16:29:45 2018 +0200 @@ -1781,30 +1781,31 @@ return; // Stop this thread. } } - continue; - } - - if (UseDynamicNumberOfCompilerThreads) { - possibly_add_compiler_threads(); - } + } else { + // Assign the task to the current thread. Mark this compilation + // thread as active for the profiler. + // CompileTaskWrapper also keeps the Method* from being deallocated if redefinition + // occurs after fetching the compile task off the queue. + CompileTaskWrapper ctw(task); + nmethodLocker result_handle; // (handle for the nmethod produced by this task) + task->set_code_handle(&result_handle); + methodHandle method(thread, task->method()); - // Assign the task to the current thread. Mark this compilation - // thread as active for the profiler. - CompileTaskWrapper ctw(task); - nmethodLocker result_handle; // (handle for the nmethod produced by this task) - task->set_code_handle(&result_handle); - methodHandle method(thread, task->method()); + // Never compile a method if breakpoints are present in it + if (method()->number_of_breakpoints() == 0) { + // Compile the method. + if ((UseCompiler || AlwaysCompileLoopMethods) && CompileBroker::should_compile_new_jobs()) { + invoke_compiler_on_method(task); + thread->start_idle_timer(); + } else { + // After compilation is disabled, remove remaining methods from queue + method->clear_queued_for_compilation(); + task->set_failure_reason("compilation is disabled"); + } + } - // Never compile a method if breakpoints are present in it - if (method()->number_of_breakpoints() == 0) { - // Compile the method. - if ((UseCompiler || AlwaysCompileLoopMethods) && CompileBroker::should_compile_new_jobs()) { - invoke_compiler_on_method(task); - thread->start_idle_timer(); - } else { - // After compilation is disabled, remove remaining methods from queue - method->clear_queued_for_compilation(); - task->set_failure_reason("compilation is disabled"); + if (UseDynamicNumberOfCompilerThreads) { + possibly_add_compiler_threads(); } } } From: coleen.phillimore at oracle.com Sent: Donnerstag, 4. Oktober 2018 13:45 To: hotspot-dev developers ; Doerr, Martin Subject: RFR (S) 8209889: RedefineStress tests crash Summary: Create CompileTaskWrapper before potential safepoint open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8209889 MetadataOnStackMark needs to find the CompileTask in the current thread, or else the Method isn't marked as being alive, so if it's old/obsolete, it is freed. There are checks to disable adding old methods to the compile queue but if the method is redefined after this point, there may be a preexisting bug that we compile an old method. This change just fixes the crash. Can some compiler person (Martin?) look at where I moved possibly_add_compiler_threads and let me know if that looks okay? There's a comment in the change that says why you need to create the CompileTaskWrapper before that call. I couldn't find a place to put a NoSafepointVerifier. Tested with the test that failed, which doesn't reproduce the bug except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in progress. Thanks, Coleen From sgehwolf at redhat.com Thu Oct 4 14:40:01 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 04 Oct 2018 16:40:01 +0200 Subject: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and os.arch value on ppc64le cause issues with Java tooling In-Reply-To: <666987d08f324a0c94208c6cca403214@sap.com> References: <666987d08f324a0c94208c6cca403214@sap.com> Message-ID: Hi Goetz, On Tue, 2018-10-02 at 10:40 +0000, Lindenmaier, Goetz wrote: [...] > Did you test this on ppc64 be, too? I've tested this patch on PPC64 BE Linux now too. There is no change (as expected): $ ./jdk8-ppc64be-after/bin/java TestProperty os.arch == ppc64 java.library.path == /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib $ ./jdk8-ppc64be-before/bin/java TestProperty os.arch == ppc64 java.library.path == /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib Thanks, Severin > Best regards, > Goetz. > > > -----Original Message----- > > From: ppc-aix-port-dev On > > Behalf Of Severin Gehwolf > > Sent: Dienstag, 2. Oktober 2018 12:34 > > To: Erik Joelsson ; hotspot-dev > dev at openjdk.java.net>; ppc-aix-port-dev > dev at openjdk.java.net>; build-dev > > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and > > os.arch value on ppc64le cause issues with Java tooling > > > > Hi, > > > > Pinging PPC porters. Does this look reasonable to you? > > > > Thanks, > > Severin > > > > > > On Fri, 2018-09-28 at 08:56 -0700, Erik Joelsson wrote: > > > Build changes look ok to me. > > > > > > /Erik > > > > > > > > > On 2018-09-26 04:26, Severin Gehwolf wrote: > > > > Hi, > > > > > > > > Could I please get reviews for this JDK 8 backport which fixes some > > > > tooling issues on Linux ppc64le? Prior this patch, a ppc64le build > > > > would report as "ppc64" via os.arch system property which breaks > > > > tooling such as maven in as much as if some dependency needs native > > > > libraries it would download BE binaries where it actually should > > > > download LE binaries. Example for os.arch/java.library.path: > > > > > > > > pre: > > > > $ ./jdk8-pre-ppc64le/bin/java TestProperty > > > > java.library.path = > > > > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > > > os.arch = ppc64 > > > > > > > > post: > > > > $ ./jdk8-post-ppc64le/bin/java TestProperty > > > > java.library.path = > > > > /usr/java/packages/lib/ppc64le:/usr/lib64:/lib64:/lib:/usr/lib > > > > os.arch = ppc64le > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8073139 > > > > webrevs: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- > > > > 8073139/jdk8/01/ > > > > > > > > Including build-dev for build changes. hotspot-dev and ppc-aix-port-dev > > > > for JDK/hotspot changes. > > > > > > > > This backport should only have minor differences to the changes in JDK > > > > 11. We have been using similar patches in Fedora for months. Thoughts? > > > > > > > > Thanks, > > > > Severin > > > > > > > > > > > > From goetz.lindenmaier at sap.com Thu Oct 4 14:47:47 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 4 Oct 2018 14:47:47 +0000 Subject: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and os.arch value on ppc64le cause issues with Java tooling In-Reply-To: References: <666987d08f324a0c94208c6cca403214@sap.com> Message-ID: Thanks! I'm fine for this to go to 8. Best regards, Goetz. > -----Original Message----- > From: Severin Gehwolf > Sent: Donnerstag, 4. Oktober 2018 16:40 > To: Lindenmaier, Goetz ; Erik Joelsson > ; hotspot-dev dev at openjdk.java.net>; ppc-aix-port-dev dev at openjdk.java.net>; build-dev > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory and > os.arch value on ppc64le cause issues with Java tooling > > Hi Goetz, > > On Tue, 2018-10-02 at 10:40 +0000, Lindenmaier, Goetz wrote: > [...] > > Did you test this on ppc64 be, too? > > I've tested this patch on PPC64 BE Linux now too. There is no change > (as expected): > > $ ./jdk8-ppc64be-after/bin/java TestProperty > os.arch == ppc64 > java.library.path == > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > $ ./jdk8-ppc64be-before/bin/java TestProperty > os.arch == ppc64 > java.library.path == > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > Thanks, > Severin > > > Best regards, > > Goetz. > > > > > -----Original Message----- > > > From: ppc-aix-port-dev > On > > > Behalf Of Severin Gehwolf > > > Sent: Dienstag, 2. Oktober 2018 12:34 > > > To: Erik Joelsson ; hotspot-dev > > dev at openjdk.java.net>; ppc-aix-port-dev > > dev at openjdk.java.net>; build-dev > > > Subject: Re: [PING] [8u] RFR: 8073139: PPC64: User-visible arch directory > and > > > os.arch value on ppc64le cause issues with Java tooling > > > > > > Hi, > > > > > > Pinging PPC porters. Does this look reasonable to you? > > > > > > Thanks, > > > Severin > > > > > > > > > On Fri, 2018-09-28 at 08:56 -0700, Erik Joelsson wrote: > > > > Build changes look ok to me. > > > > > > > > /Erik > > > > > > > > > > > > On 2018-09-26 04:26, Severin Gehwolf wrote: > > > > > Hi, > > > > > > > > > > Could I please get reviews for this JDK 8 backport which fixes some > > > > > tooling issues on Linux ppc64le? Prior this patch, a ppc64le build > > > > > would report as "ppc64" via os.arch system property which breaks > > > > > tooling such as maven in as much as if some dependency needs > native > > > > > libraries it would download BE binaries where it actually should > > > > > download LE binaries. Example for os.arch/java.library.path: > > > > > > > > > > pre: > > > > > $ ./jdk8-pre-ppc64le/bin/java TestProperty > > > > > java.library.path = > > > > > > /usr/java/packages/lib/ppc64:/usr/lib64:/lib64:/lib:/usr/lib > > > > > os.arch = ppc64 > > > > > > > > > > post: > > > > > $ ./jdk8-post-ppc64le/bin/java TestProperty > > > > > java.library.path = > > > > > > /usr/java/packages/lib/ppc64le:/usr/lib64:/lib64:/lib:/usr/lib > > > > > os.arch = ppc64le > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8073139 > > > > > webrevs: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- > > > > > > 8073139/jdk8/01/ > > > > > > > > > > Including build-dev for build changes. hotspot-dev and ppc-aix-port- > dev > > > > > for JDK/hotspot changes. > > > > > > > > > > This backport should only have minor differences to the changes in > JDK > > > > > 11. We have been using similar patches in Fedora for months. > Thoughts? > > > > > > > > > > Thanks, > > > > > Severin > > > > > > > > > > > > > > > > > From coleen.phillimore at oracle.com Thu Oct 4 15:59:55 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Oct 2018 11:59:55 -0400 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: <68a26e6e6e574f75aec64ede7487a117@sap.com> References: <68a26e6e6e574f75aec64ede7487a117@sap.com> Message-ID: Yes, that looks cleaner for control flow.? So the UseDynamicNumberOfCompilerThreads is outside the 'else' then. I'll test that out and resend. Coleen On 10/4/18 10:34 AM, Doerr, Martin wrote: > > Hi Coleen, > > good catch. Looks good where you moved it. > > I couldn?t find a place for a NoSafepointVerifier, either. > > Maybe it?d be more clear if we replaced the ?continue? in the NULL > case by an else case in the loop so the CompileTaskWrapper is the > first thing after the NULL check (see below). > > Best regards, > > Martin > > --- a/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 > 14:03:13 2018 +0200 > > +++ b/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 > 16:29:45 2018 +0200 > > @@ -1781,30 +1781,31 @@ > > ?????????? return; // Stop this thread. > > ???????? } > > ?????? } > > -????? continue; > > -??? } > > - > > -??? if (UseDynamicNumberOfCompilerThreads) { > > -????? possibly_add_compiler_threads(); > > -??? } > > +??? } else { > > +????? // Assign the task to the current thread.? Mark this compilation > > +????? // thread as active for the profiler. > > +????? // CompileTaskWrapper also keeps the Method* from being > deallocated if redefinition > > +????? // occurs after fetching the compile task off the queue. > > +????? CompileTaskWrapper ctw(task); > > +????? nmethodLocker result_handle;? // (handle for the nmethod > produced by this task) > > + task->set_code_handle(&result_handle); > > +????? methodHandle method(thread, task->method()); > > -??? // Assign the task to the current thread. Mark this compilation > > -??? // thread as active for the profiler. > > -??? CompileTaskWrapper ctw(task); > > -??? nmethodLocker result_handle;? // (handle for the nmethod produced > by this task) > > - task->set_code_handle(&result_handle); > > -??? methodHandle method(thread, task->method()); > > +????? // Never compile a method if breakpoints are present in it > > +????? if (method()->number_of_breakpoints() == 0) { > > +??????? // Compile the method. > > +??????? if ((UseCompiler || AlwaysCompileLoopMethods) && > CompileBroker::should_compile_new_jobs()) { > > +????????? invoke_compiler_on_method(task); > > +????????? thread->start_idle_timer(); > > +???? ???} else { > > +????????? // After compilation is disabled, remove remaining methods > from queue > > + method->clear_queued_for_compilation(); > > + task->set_failure_reason("compilation is disabled"); > > +??????? } > > +????? } > > -??? // Never compile a method if breakpoints are present in it > > -??? if (method()->number_of_breakpoints() == 0) { > > -????? // Compile the method. > > -????? if ((UseCompiler || AlwaysCompileLoopMethods) && > CompileBroker::should_compile_new_jobs()) { > > -??????? invoke_compiler_on_method(task); > > -??????? thread->start_idle_timer(); > > -????? } else { > > -??????? // After compilation is disabled, remove remaining methods > from queue > > - method->clear_queued_for_compilation(); > > - task->set_failure_reason("compilation is disabled"); > > +????? if (UseDynamicNumberOfCompilerThreads) { > > +??????? possibly_add_compiler_threads(); > > ?????? } > > ???? } > > ?? } > > *From:*coleen.phillimore at oracle.com > *Sent:* Donnerstag, 4. Oktober 2018 13:45 > *To:* hotspot-dev developers ; Doerr, > Martin > *Subject:* RFR (S) 8209889: RedefineStress tests crash > > Summary: Create CompileTaskWrapper before potential safepoint > > open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev > > bug link https://bugs.openjdk.java.net/browse/JDK-8209889 > > MetadataOnStackMark needs to find the CompileTask in the current > thread, or else the Method isn't marked as being alive, so if it's > old/obsolete, it is freed.? There are checks to disable adding old > methods to the compile queue but if the method is redefined after this > point, there may be a preexisting bug that we compile an old method.? > This change just fixes the crash. > > Can some compiler person (Martin?) look at where I moved > possibly_add_compiler_threads and let me know if that looks okay?? > There's a comment in the change that says why you need to create the > CompileTaskWrapper before that call.? I couldn't find a place to put a > NoSafepointVerifier. > > Tested with the test that failed, which doesn't reproduce the bug > except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in > progress. > > Thanks, > Coleen > From coleen.phillimore at oracle.com Thu Oct 4 16:09:23 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Oct 2018 12:09:23 -0400 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: <68a26e6e6e574f75aec64ede7487a117@sap.com> References: <68a26e6e6e574f75aec64ede7487a117@sap.com> Message-ID: <7b92e8ff-7dea-1130-b768-874faf2794dc@oracle.com> On 10/4/18 10:34 AM, Doerr, Martin wrote: > > Hi Coleen, > > good catch. Looks good where you moved it. > > I couldn?t find a place for a NoSafepointVerifier, either. > > Maybe it?d be more clear if we replaced the ?continue? in the NULL > case by an else case in the loop so the CompileTaskWrapper is the > first thing after the NULL check (see below). > I see, thanks to the magic of patch, the possibly_add_compiler_threads() is inside the 'else' clause where there's a compile task on the queue.? Which makes perfect sense.? It seems like there should be a possibly_remove_compiler_thread() in the 'if' case but I'll leave that to the next person to touch this code. Ok, I'll test this variant, which I also like. thanks, Coleen > Best regards, > > Martin > > --- a/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 > 14:03:13 2018 +0200 > > +++ b/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 > 16:29:45 2018 +0200 > > @@ -1781,30 +1781,31 @@ > > ?????????? return; // Stop this thread. > > ???????? } > > ?????? } > > -????? continue; > > -??? } > > - > > -??? if (UseDynamicNumberOfCompilerThreads) { > > -????? possibly_add_compiler_threads(); > > -??? } > > +??? } else { > > +????? // Assign the task to the current thread.? Mark this compilation > > +????? // thread as active for the profiler. > > +????? // CompileTaskWrapper also keeps the Method* from being > deallocated if redefinition > > +????? // occurs after fetching the compile task off the queue. > > +????? CompileTaskWrapper ctw(task); > > +????? nmethodLocker result_handle;? // (handle for the nmethod > produced by this task) > > + task->set_code_handle(&result_handle); > > +????? methodHandle method(thread, task->method()); > > -??? // Assign the task to the current thread. Mark this compilation > > -??? // thread as active for the profiler. > > -??? CompileTaskWrapper ctw(task); > > -??? nmethodLocker result_handle;? // (handle for the nmethod produced > by this task) > > - task->set_code_handle(&result_handle); > > -??? methodHandle method(thread, task->method()); > > +????? // Never compile a method if breakpoints are present in it > > +????? if (method()->number_of_breakpoints() == 0) { > > +??????? // Compile the method. > > +??????? if ((UseCompiler || AlwaysCompileLoopMethods) && > CompileBroker::should_compile_new_jobs()) { > > +????????? invoke_compiler_on_method(task); > > +????????? thread->start_idle_timer(); > > +???? ???} else { > > +????????? // After compilation is disabled, remove remaining methods > from queue > > + method->clear_queued_for_compilation(); > > + task->set_failure_reason("compilation is disabled"); > > +??????? } > > +????? } > > -??? // Never compile a method if breakpoints are present in it > > -??? if (method()->number_of_breakpoints() == 0) { > > -????? // Compile the method. > > -????? if ((UseCompiler || AlwaysCompileLoopMethods) && > CompileBroker::should_compile_new_jobs()) { > > -??????? invoke_compiler_on_method(task); > > -??????? thread->start_idle_timer(); > > -????? } else { > > -??????? // After compilation is disabled, remove remaining methods > from queue > > - method->clear_queued_for_compilation(); > > - task->set_failure_reason("compilation is disabled"); > > +????? if (UseDynamicNumberOfCompilerThreads) { > > +??????? possibly_add_compiler_threads(); > > ?????? } > > ???? } > > ?? } > > *From:*coleen.phillimore at oracle.com > *Sent:* Donnerstag, 4. Oktober 2018 13:45 > *To:* hotspot-dev developers ; Doerr, > Martin > *Subject:* RFR (S) 8209889: RedefineStress tests crash > > Summary: Create CompileTaskWrapper before potential safepoint > > open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev > > bug link https://bugs.openjdk.java.net/browse/JDK-8209889 > > MetadataOnStackMark needs to find the CompileTask in the current > thread, or else the Method isn't marked as being alive, so if it's > old/obsolete, it is freed.? There are checks to disable adding old > methods to the compile queue but if the method is redefined after this > point, there may be a preexisting bug that we compile an old method.? > This change just fixes the crash. > > Can some compiler person (Martin?) look at where I moved > possibly_add_compiler_threads and let me know if that looks okay?? > There's a comment in the change that says why you need to create the > CompileTaskWrapper before that call.? I couldn't find a place to put a > NoSafepointVerifier. > > Tested with the test that failed, which doesn't reproduce the bug > except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in > progress. > > Thanks, > Coleen > From coleen.phillimore at oracle.com Thu Oct 4 17:57:53 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Oct 2018 13:57:53 -0400 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: <7b92e8ff-7dea-1130-b768-874faf2794dc@oracle.com> References: <68a26e6e6e574f75aec64ede7487a117@sap.com> <7b92e8ff-7dea-1130-b768-874faf2794dc@oracle.com> Message-ID: Martin, Here's your version of the fix, freshly tested. http://cr.openjdk.java.net/~coleenp/8209889.02/webrev/index.html Thanks, Coleen On 10/4/18 12:09 PM, coleen.phillimore at oracle.com wrote: > > > On 10/4/18 10:34 AM, Doerr, Martin wrote: >> >> Hi Coleen, >> >> good catch. Looks good where you moved it. >> >> I couldn?t find a place for a NoSafepointVerifier, either. >> >> Maybe it?d be more clear if we replaced the ?continue? in the NULL >> case by an else case in the loop so the CompileTaskWrapper is the >> first thing after the NULL check (see below). >> > > I see, thanks to the magic of patch, the > possibly_add_compiler_threads() is inside the 'else' clause where > there's a compile task on the queue.? Which makes perfect sense. It > seems like there should be a possibly_remove_compiler_thread() in the > 'if' case but I'll leave that to the next person to touch this code. > > Ok, I'll test this variant, which I also like. > thanks, > Coleen > >> Best regards, >> >> Martin >> >> --- a/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >> 14:03:13 2018 +0200 >> >> +++ b/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >> 16:29:45 2018 +0200 >> >> @@ -1781,30 +1781,31 @@ >> >> ?????????? return; // Stop this thread. >> >> ???????? } >> >> ?????? } >> >> -????? continue; >> >> -??? } >> >> - >> >> -??? if (UseDynamicNumberOfCompilerThreads) { >> >> -????? possibly_add_compiler_threads(); >> >> -??? } >> >> +??? } else { >> >> +????? // Assign the task to the current thread.? Mark this compilation >> >> +????? // thread as active for the profiler. >> >> +????? // CompileTaskWrapper also keeps the Method* from being >> deallocated if redefinition >> >> +????? // occurs after fetching the compile task off the queue. >> >> +????? CompileTaskWrapper ctw(task); >> >> +????? nmethodLocker result_handle;? // (handle for the nmethod >> produced by this task) >> >> + task->set_code_handle(&result_handle); >> >> +????? methodHandle method(thread, task->method()); >> >> -??? // Assign the task to the current thread. Mark this compilation >> >> -??? // thread as active for the profiler. >> >> -??? CompileTaskWrapper ctw(task); >> >> -??? nmethodLocker result_handle;? // (handle for the nmethod >> produced by this task) >> >> - task->set_code_handle(&result_handle); >> >> -??? methodHandle method(thread, task->method()); >> >> +????? // Never compile a method if breakpoints are present in it >> >> +????? if (method()->number_of_breakpoints() == 0) { >> >> +??????? // Compile the method. >> >> +??????? if ((UseCompiler || AlwaysCompileLoopMethods) && >> CompileBroker::should_compile_new_jobs()) { >> >> +????????? invoke_compiler_on_method(task); >> >> +????????? thread->start_idle_timer(); >> >> +???? ???} else { >> >> +????????? // After compilation is disabled, remove remaining methods >> from queue >> >> + method->clear_queued_for_compilation(); >> >> + task->set_failure_reason("compilation is disabled"); >> >> +??????? } >> >> +????? } >> >> -??? // Never compile a method if breakpoints are present in it >> >> -??? if (method()->number_of_breakpoints() == 0) { >> >> -????? // Compile the method. >> >> -????? if ((UseCompiler || AlwaysCompileLoopMethods) && >> CompileBroker::should_compile_new_jobs()) { >> >> -??????? invoke_compiler_on_method(task); >> >> -??????? thread->start_idle_timer(); >> >> -????? } else { >> >> -??????? // After compilation is disabled, remove remaining methods >> from queue >> >> - method->clear_queued_for_compilation(); >> >> - task->set_failure_reason("compilation is disabled"); >> >> +????? if (UseDynamicNumberOfCompilerThreads) { >> >> +??????? possibly_add_compiler_threads(); >> >> ?????? } >> >> ???? } >> >> ?? } >> >> *From:*coleen.phillimore at oracle.com >> *Sent:* Donnerstag, 4. Oktober 2018 13:45 >> *To:* hotspot-dev developers ; Doerr, >> Martin >> *Subject:* RFR (S) 8209889: RedefineStress tests crash >> >> Summary: Create CompileTaskWrapper before potential safepoint >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev >> >> bug link https://bugs.openjdk.java.net/browse/JDK-8209889 >> >> MetadataOnStackMark needs to find the CompileTask in the current >> thread, or else the Method isn't marked as being alive, so if it's >> old/obsolete, it is freed.? There are checks to disable adding old >> methods to the compile queue but if the method is redefined after >> this point, there may be a preexisting bug that we compile an old >> method.? This change just fixes the crash. >> >> Can some compiler person (Martin?) look at where I moved >> possibly_add_compiler_threads and let me know if that looks okay?? >> There's a comment in the change that says why you need to create the >> CompileTaskWrapper before that call.? I couldn't find a place to put >> a NoSafepointVerifier. >> >> Tested with the test that failed, which doesn't reproduce the bug >> except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in >> progress. >> >> Thanks, >> Coleen >> > From dean.long at oracle.com Thu Oct 4 18:00:45 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 4 Oct 2018 11:00:45 -0700 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: References: Message-ID: So having a raw CompileTask* is like having a raw Method*.? I'm tempted to suggest a new CompiledTaskHandle, but it would only be used in one place, so how about changing CompileTaskWrapper so we could do something like this instead: ??? CompileTaskWrapper ctw(queue->get()); ??? CompileTask* task = ctw.task(); CompileTaskWrapper would need to be able to handle a NULL task gracefully. dl On 10/4/18 4:45 AM, coleen.phillimore at oracle.com wrote: > Summary: Create CompileTaskWrapper before potential safepoint > > open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8209889 > > MetadataOnStackMark needs to find the CompileTask in the current > thread, or else the Method isn't marked as being alive, so if it's > old/obsolete, it is freed.? There are checks to disable adding old > methods to the compile queue but if the method is redefined after this > point, there may be a preexisting bug that we compile an old method.? > This change just fixes the crash. > > Can some compiler person (Martin?) look at where I moved > possibly_add_compiler_threads and let me know if that looks okay? > There's a comment in the change that says why you need to create the > CompileTaskWrapper before that call.? I couldn't find a place to put a > NoSafepointVerifier. > > Tested with the test that failed, which doesn't reproduce the bug > except rarely, and test/hotspot/jtreg/compiler tests.? tier1 an 2 in > progress. > > Thanks, > Coleen From thomas.stuefe at gmail.com Thu Oct 4 18:11:17 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 4 Oct 2018 20:11:17 +0200 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Message-ID: On Thu, Oct 4, 2018 at 11:18 AM David Holmes wrote: > > Hi Thomas, > > On 4/10/2018 6:27 PM, Thomas St?fe wrote: > > Hi David, > > On Thu, Oct 4, 2018 at 9:44 AM David Holmes wrote: > >> > >> Hi Matthias, > >> > >> I'm hoping others will chime in here as I: > >> > >> a) don't know if this information is actually useful for an error log of > >> this kind; > >> > >> b) don't know if the information might be considered sensitive or not; and > >> > > > > I have no opinion on (a) and (b). > > That's a shame :) > > >> c) don't think it's worth the effort of adding a flag to control this. > >> Plus the flag is only useful for trying to reproduce an issue; if it's a > >> one-of failure then you've already missed out on the information in the > >> log file. > > > > How about a more generic switch to control verbosity of the error report? > > > > The way we and you use the error files seem to differ. You seem to > > prefer them short and snappy and bare any security relevant details > > (as far as that is even possible in an hs-err file). As was once > > mentioned in a similar discussion, "OpenJDK hs-err files get posted > > verbatim in forums and bug reports". > > > > We use the hs-err files differently. They are usually handed down to > > us by our customers thru secure channels, and for us size does not > > matter much, nor does security relevant information since we have > > contracts with our customers. > > To be honest I don't deal with hs-err files from other people very much > at all. But from what I have dealt with I haven't, to the best of my > recollection, encountered any crash investigation where all this extra > info would have shed any light. But most crashes I investigate are > developer related rather than end user. That may be the difference. We get crash reports from customers, often with little context, no repros, no cores, nothing. So we have to make do with what we have, which is the hs-err files. Therefore we make them as extensive as possible. Our support worked miracles with just these files. It is incredible what you can deduce from them. User information, e.g., can hint towards a mismanaged installation, if the server node does not run under the correct user. I also dimly remember it being useful in analyzing SIGBUS crashes for mapped files on Solaris. So, it turns out I have an opinion :) Well when I say no opinion I mean I can see arguments for both sides, and I feel uncomfortable pressing our way of working upon the OpenJDK code base. > > > That has been a point of contention over and over again in the past. > > So I wonder whether one, or possibly two, general switches could keep > > both sides happy. Something like -XX:+ExtendedErrorReports" and > > possibly "-XX:+ErrorReportsIncludeSensitiveData". > > > > Those switches could be, by default, false in the OpenJDK. > > > > Any additions we add to error reporting where we cannot find an > > agreement we could make conditional on one or the other switch. > > > > What do you think? > > Seems reasonable to propose. Any new flag will require a CSR request. > Okay, thanks! I will file a CSR next week when I am back at the office. ..Thomas > David > > > ..Thomas > > > >> > >> Cheers, > >> David > >> > >> > >> On 4/10/2018 5:31 PM, Baesken, Matthias wrote: > >>> Hello, my proposal would be to only print > >>> > >>> uid : 1679 (testuser-name) > >>> > >>> by default and guard the rest of the info by some XX-flag, any good proposals for the flag-name are appreciated; > >>> for example : > >>> > >>> if (ExtendHsErrorFileByUserRelatedInformation) { > >>> > >>> // print those too : > >>> > >>>>>> euid : 1679 (testuser-name) > >>>>>> gid : 25 (testgroup) > >>>>>> egid : 25 (testgroup) > >>>>>> > >>>>>> umask: 0022 (removing ----w--w-) > >>> > >>> } > >>> > >>> > >>> Best regards, Matthias > >>> > >>> > >>>> -----Original Message----- > >>>> From: Baesken, Matthias > >>>> Sent: Dienstag, 2. Oktober 2018 12:38 > >>>> To: 'David Holmes' ; 'hotspot- > >>>> dev at openjdk.java.net' > >>>> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > >>>> > >>>> Hi David, I think the added info could be seen more or less in line with > >>>> what currently is reported in hs_err file . > >>>> For instance you usually see user-names and lots of paths from the system > >>>> in the hs_err file . > >>>> > >>>> In case the umask and gid is seen as more sensitive than that, one could > >>>> make the output switchable with an XX-flag ; > >>>> this would have the benefit of making the added output more clear to the > >>>> user/admin . > >>>> > >>>> Best regards, Matthias > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: David Holmes > >>>>> Sent: Dienstag, 2. Oktober 2018 09:49 > >>>>> To: Baesken, Matthias ; 'hotspot- > >>>>> dev at openjdk.java.net' > >>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > >>>>> > >>>>> Hi Matthias, > >>>>> > >>>>> On 2/10/2018 5:30 PM, Baesken, Matthias wrote: > >>>>>> Hello , please review this small enhancement to the hs_err file . > >>>>>> > >>>>>> Currently the hs_err file contains only limited OS user related > >>>> information. > >>>>>> Just the user name is printed via output of environment variables (at > >>>> least > >>>>> on Windows with USERNAME - output). > >>>>>> The enhanced output on UNIX would contain more information including > >>>>> uid, gid and umask : > >>>>>> > >>>>>> uid : 1679 (testuser) > >>>>>> euid : 1679 (testuser) > >>>>>> gid : 25 (testgroup) > >>>>>> egid : 25 (testgroup) > >>>>>> > >>>>>> umask: 0022 (removing ----w--w-) > >>>>> > >>>>> Could any of this be considered sensitive information by an end-user? > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> > >>>>>> > >>>>>> ( Some of the info above could be found currently in error logging > >>>> output > >>>>> e.g. > >>>>>> attachListener_linux.cpp line 362 > >>>>>> log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", > >>>>>> (and the user name on Windows(-only) is in the env variables section). > >>>>>> > >>>>>> > >>>>>> > >>>>>> bug/webrev : > >>>>>> ---------------------- > >>>>>> > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8211326 > >>>>>> > >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ > >>>>>> > >>>>>> > >>>>>> Thanks, Matthias > >>>>>> From coleen.phillimore at oracle.com Thu Oct 4 18:28:28 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Oct 2018 14:28:28 -0400 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: References: Message-ID: On 10/4/18 2:00 PM, dean.long at oracle.com wrote: > So having a raw CompileTask* is like having a raw Method*.? I'm > tempted to suggest a new CompiledTaskHandle, but it would only be used > in one place, so how about changing CompileTaskWrapper so we could do > something like this instead: > > ??? CompileTaskWrapper ctw(queue->get()); > ??? CompileTask* task = ctw.task(); I started down this path but since CompileTaskWrapper cannot completely encapsulate CompileTask to make it safe, the code I have I think is just as good.? Also, adding NULL checks seems like it would allow some new behavoir that is now an invariant (ie CompileTask is never NULL). Thanks, Coleen > > CompileTaskWrapper would need to be able to handle a NULL task > gracefully. > > dl > > On 10/4/18 4:45 AM, coleen.phillimore at oracle.com wrote: >> Summary: Create CompileTaskWrapper before potential safepoint >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8209889 >> >> MetadataOnStackMark needs to find the CompileTask in the current >> thread, or else the Method isn't marked as being alive, so if it's >> old/obsolete, it is freed.? There are checks to disable adding old >> methods to the compile queue but if the method is redefined after >> this point, there may be a preexisting bug that we compile an old >> method.? This change just fixes the crash. >> >> Can some compiler person (Martin?) look at where I moved >> possibly_add_compiler_threads and let me know if that looks okay? >> There's a comment in the change that says why you need to create the >> CompileTaskWrapper before that call.? I couldn't find a place to put >> a NoSafepointVerifier. >> >> Tested with the test that failed, which doesn't reproduce the bug >> except rarely, and test/hotspot/jtreg/compiler tests.? tier1 an 2 in >> progress. >> >> Thanks, >> Coleen > From dean.long at oracle.com Thu Oct 4 19:52:29 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 4 Oct 2018 12:52:29 -0700 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: References: Message-ID: <4471ebaf-dd19-4cb3-cc41-610f4d054730@oracle.com> On 10/4/18 11:28 AM, coleen.phillimore at oracle.com wrote: > > > On 10/4/18 2:00 PM, dean.long at oracle.com wrote: >> So having a raw CompileTask* is like having a raw Method*.? I'm >> tempted to suggest a new CompiledTaskHandle, but it would only be >> used in one place, so how about changing CompileTaskWrapper so we >> could do something like this instead: >> >> ??? CompileTaskWrapper ctw(queue->get()); >> ??? CompileTask* task = ctw.task(); > > I started down this path but since CompileTaskWrapper cannot > completely encapsulate CompileTask to make it safe, the code I have I > think is just as good.? Also, adding NULL checks seems like it would > allow some new behavoir that is now an invariant (ie CompileTask is > never NULL). > OK.? Reviewed. dl > Thanks, > Coleen >> >> CompileTaskWrapper would need to be able to handle a NULL task >> gracefully. >> >> dl >> >> On 10/4/18 4:45 AM, coleen.phillimore at oracle.com wrote: >>> Summary: Create CompileTaskWrapper before potential safepoint >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8209889 >>> >>> MetadataOnStackMark needs to find the CompileTask in the current >>> thread, or else the Method isn't marked as being alive, so if it's >>> old/obsolete, it is freed.? There are checks to disable adding old >>> methods to the compile queue but if the method is redefined after >>> this point, there may be a preexisting bug that we compile an old >>> method.? This change just fixes the crash. >>> >>> Can some compiler person (Martin?) look at where I moved >>> possibly_add_compiler_threads and let me know if that looks okay? >>> There's a comment in the change that says why you need to create the >>> CompileTaskWrapper before that call.? I couldn't find a place to put >>> a NoSafepointVerifier. >>> >>> Tested with the test that failed, which doesn't reproduce the bug >>> except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in >>> progress. >>> >>> Thanks, >>> Coleen >> > From kim.barrett at oracle.com Thu Oct 4 20:39:31 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Oct 2018 16:39:31 -0400 Subject: RFR: JDK-8211073 Remove -Wno-extra from Hotspot In-Reply-To: <1340C14C-265B-4DC7-A00E-0C3BD3B42AD0@oracle.com> References: <671e162b-5580-2a0b-5115-847b8b26c4aa@oracle.com> <47ABAD88-9376-45D3-86DB-16A47B81F061@oracle.com> <55464073-BF13-4D30-9FA7-829778F698E8@oracle.com> <1340C14C-265B-4DC7-A00E-0C3BD3B42AD0@oracle.com> Message-ID: > On Oct 4, 2018, at 9:14 AM, Magnus Ihse Bursie wrote: > > 3 okt. 2018 kl. 00:18 skrev Kim Barrett : > >> I went through all the associated warnings for gcc7.3 and categorized >> them. Some are just not interesting or relevant to us. I think these >> include >> >> [?] > > I agree that some, like clobbered, is not relevant (but I think that also only applies to C code..?). In general, it doesn't hurt though, to turn on warnings that are unlikely to be triggered. I realize it also helps little to nothing, but once again, if this is the price we have to pay for catching some real problems, it's a small price indeed. I made this list for completeness of discussion. I think these options should carry no weight and should just be ignored in the decision making process. > OTOH, I don't really agree that old-style-declaration is irrelevant. At least in the JDK libraries there are multiple places where non-prototype declarations are made, and apart from the style issue, it's a bug waiting to happen if the function change its signature. I don't remember if this is the case in Hotspot too, but it wouldn't surprise me, if we don't check for it. Regarding -Wold-style-declaration, remember that we're discussion a change to HotSpot, where this isn't relevant. > empty-body is mostly good at catching style issues, like double semicolons. It's not a big deal, but it also never hurts to keep up good code quality. I have not seen it make any false positives. -Wempty-body doesn't catch empty semi-colons (which are permitted in C++11 and later, and no compiler I know of complains about them in C++98 mode, because they arise too often with conditional compilation and macros and such). There is also a well-known macro idiom involving empty if bodies, e.g. if (!) {} else to avoid dangling else. Since you didn't run into any occurences of this warning, we're probably not using that idiom right now. But there's nothing wrong with it and I've used it myself in the past. It could have been used in some of the UL macros, but a different idiom was used there. (The two are semantically mostly equivalent, the differences being fairly subtle. If I'd written these macros they would probably be using the "if" style.) > shift-negative-values is a bit beyond me, but as I understand it, it warns about a behavior that has changed recently, or will be changed, in the standards, to become undefined. That sounds to me like something that it would be better to act preemptively on. But then again, I might have misunderstood things. If so, we can definitely turn it off. I mis-remembered the situation with -Wshift-negative-value; we probably do want that turned on once we've eliminated uses (and there are some). C89/C++98 didn't specify the behavior in that case. C99/C++11 clarified that it is explicitly UB. >> -Wimplicit-fallthrough=3 requires a structured comment or C++17 >> feature to quiet the warning in code that is using the feature. >> Provides some benefit. I'm surprised there aren't any warnings from >> this. Intentional fall-through is a pretty common idiom. > > At level 3, the "structured" comment is just something like a line like this: > > // fallthrough > > Most, maybe all, fallthroughs in the hotspot code already does that. I think it's a reasonable requirement to show that the fallthrough is intentional. If you think it's too strong, we can lower it to level 1, which just requires a comment, with whatever text (or even empty, I think). I'm concerned that other compilers / checkers might want other comments, and one might end up with a bit of a mess because of that. If Visual Studio (for example) has a similar warning option, we might want to turn that on too. >> I think the benefit we get from detecting future occurrences of the >> copy constructor bug is not really worth the current cost of "fixing" >> the occurrences of conditional expressions with enumerators. I might >> feel differently if a pass were made through those to replace uses of >> enum with static const members. > > So I think this is what it really boils down to. Just to summarize/repeat what you said, so I'm sure we understand each other correctly: > > There's an upside to enabling Wextra; we know already that it found some real bugs, but the real benefit is all future potential bugs it will prohibit, and that's hard to quantify. On the other hand, there's a downside; and that's the conditional enum warnings, whose fixes are ugly. But, if the code was cleaned up, so those enum fixes were no longer ugly, then it would be a net positive to turn on Wextra. > > I think that's a reasonable solution for now. If we agree on this, I'll open a new bug with just the copy constructor fixes, and a enhancement request for replacing the enums with static const, and making the current bug ("turn on Wextra") dependent on that. That seems like a good plan to me. > I just want to add a final remark about compiler warnings. When I started the current work of normalizing the set of warnings in the entire OpenJDK, I was amazed at what low level of warnings that was used for gcc/clang in hotspot. In all the JDK native libraries, -Wall and -Wextra were on by default. In hotspot, not even -Wall were turned on! I think what have sort of "saved" you is that Microsoft CL was running with /W3, which is roughly equivalent to -Wall -Wextra. So most of the shared code compiled fine when adding -Wall -Wextra, but there was a lot of new warnings in non-Windows code. Of course, /W3 and -Wall -Wextra does not cover exactly the same set of warnings, nor implement similar warning checks identically, hence the current open set of warnings in shared code. I mostly agree with this, though the version to version changes in the content of these umbrella warning sets is a concern. > I made a long list of warnings that are disabled due to this, but I think it would be a good idea to tackle them one by one. By a quick glance, most of them (but surely not all) seemed to point to real problematic code; if not buggy so at least bad style. Yes. For example, Mikael V has a patch set to enable -Wunused-variable. It?s unpleasantly large and in some parts intrusive. And there is https://bugs.openjdk.java.net/browse/JDK-8135181 Re-enable '-Wconversion' for GCC 4.3 and later which would also be huge and intrusive. From kim.barrett at oracle.com Thu Oct 4 21:01:03 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Oct 2018 17:01:03 -0400 Subject: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <71fb517fba794dc28291cf51afb6e8b9@sap.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <71fb517fba794dc28291cf51afb6e8b9@sap.com> Message-ID: <2DB05A59-EA8F-46FD-9ECF-CA47DA5E313E@oracle.com> > On Oct 4, 2018, at 2:59 AM, Lindenmaier, Goetz wrote: > > Hi, > > I appreciate this is handled in a JEP and well communicated! > > XLc, the compiler we use for AIX, might be a bottleneck here. > > But we currently have no compiler-constraints by products on > our aix port of jdk12 (for jdk11 we must stay with the current > compiler xlc 12 which does not support C++11, and jdk11 even > should be compilable with aCC by HP for which we won't > get new versions!) > > We will update our compiler for jdk/jdk to the most recent Xlc > as soon as possible. > To my current knowledge, Xlc 14 was dropped, and xlc 16 > is to be released early 2019. It is supposed to support > C++ 11, and also some C++ 14 features. The information I've been able to find mostly discusses Linux support for XLC 16. Any information about "some C++14 features"? I haven't found any detail on that. I am working toward being able to target this for JDK 12, but realize that might not be reachable. Windows is already there, because of VS2017 support. There are a couple of minor patches needed for gcc and clang based builds; Linux-x64 and MacOSX-x64 have had a fair amount of ad hoc testing. We (Oracle) only switched over the Solaris/sparc builds to the necessary compiler version (Oracle Studio 12.6) very recently, and there are some issues still to be dealt with there. And the currently used XLC is just not up to the change. From kim.barrett at oracle.com Thu Oct 4 21:09:57 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Oct 2018 17:09:57 -0400 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> Message-ID: <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> > On Oct 4, 2018, at 9:17 AM, Aleksei Voitylov wrote: > > Kim, > > Thank you for posting this. It's an important step to make. I wanted to clarify a couple of points: > > 1. Since this is infrastructure JEP, is the version of JDK which will undergo such changes going to be some future version or does it apply to past versions as well? I'm concerned with how we can simplify security backports to 8u which (I currently assume) is not subject to this change. Future version (perhaps as soon as JDK 12). I don't think there is any desire to backport this change. And yes, introducing the use of new language features will make backports even more interesting than they already are. That seems unavoidable if we're going to pursue this direction. > 2. Which versions of GCC do you tentatively consider at this point? Non-x86 ports may have a problem upgrading to a specific version of GCC which the shared code will use and may need additional lead time to adjust. I think our (ad hoc) testing has been limited to gcc 7.x. But looking at the gcc language conformance tables and release notes, gcc 5.x might be good enough, and gcc 6.x seems fairly likely sufficient. Of course, older versions are more likely to have bugs in some of these new features. Updating the minimum supported versions for gcc and clang are noted as TBD in the JEP. From coleen.phillimore at oracle.com Thu Oct 4 21:10:40 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 4 Oct 2018 17:10:40 -0400 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: <4471ebaf-dd19-4cb3-cc41-610f4d054730@oracle.com> References: <4471ebaf-dd19-4cb3-cc41-610f4d054730@oracle.com> Message-ID: On 10/4/18 3:52 PM, dean.long at oracle.com wrote: > On 10/4/18 11:28 AM, coleen.phillimore at oracle.com wrote: >> >> >> On 10/4/18 2:00 PM, dean.long at oracle.com wrote: >>> So having a raw CompileTask* is like having a raw Method*.? I'm >>> tempted to suggest a new CompiledTaskHandle, but it would only be >>> used in one place, so how about changing CompileTaskWrapper so we >>> could do something like this instead: >>> >>> ??? CompileTaskWrapper ctw(queue->get()); >>> ??? CompileTask* task = ctw.task(); >> >> I started down this path but since CompileTaskWrapper cannot >> completely encapsulate CompileTask to make it safe, the code I have I >> think is just as good.? Also, adding NULL checks seems like it would >> allow some new behavoir that is now an invariant (ie CompileTask is >> never NULL). >> > > OK.? Reviewed. Thanks, Dean. Coleen > > dl > >> Thanks, >> Coleen >>> >>> CompileTaskWrapper would need to be able to handle a NULL task >>> gracefully. >>> >>> dl >>> >>> On 10/4/18 4:45 AM, coleen.phillimore at oracle.com wrote: >>>> Summary: Create CompileTaskWrapper before potential safepoint >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8209889 >>>> >>>> MetadataOnStackMark needs to find the CompileTask in the current >>>> thread, or else the Method isn't marked as being alive, so if it's >>>> old/obsolete, it is freed.? There are checks to disable adding old >>>> methods to the compile queue but if the method is redefined after >>>> this point, there may be a preexisting bug that we compile an old >>>> method.? This change just fixes the crash. >>>> >>>> Can some compiler person (Martin?) look at where I moved >>>> possibly_add_compiler_threads and let me know if that looks okay? >>>> There's a comment in the change that says why you need to create >>>> the CompileTaskWrapper before that call.? I couldn't find a place >>>> to put a NoSafepointVerifier. >>>> >>>> Tested with the test that failed, which doesn't reproduce the bug >>>> except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in >>>> progress. >>>> >>>> Thanks, >>>> Coleen >>> >> > From kim.barrett at oracle.com Thu Oct 4 21:18:05 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Oct 2018 17:18:05 -0400 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> <250FA229-7F95-4FB9-935D-7834A203072F@oracle.com> Message-ID: <39BB147B-089C-44D6-BD29-3F5B906D9EE9@oracle.com> > On Oct 4, 2018, at 9:27 AM, Magnus Ihse Bursie wrote: >> That all assumes that static build actually works at all; it doesn?t look like we >> test that configuration these days, so who knows what bit rot may have set in. >> > > No, we don't test static build, and never have. I'd be surprised if it's still possible to build a static build. > > This was a feature developed for the mobile project. The static build part was merged, somewhat hastily, into mainline, but the rest of the mobile code is still lingering in the mobile project fork. > > The entire "static build" concept was somewhat artificially injected in some parts of the build code. It is for instance not possible to build all modules with a static build. At the very least you need to build without java.desktop, if I remember correctly. > > The motivation for the static build is that on iOS, apps are not allowed to load dynamic libraries (for some reason only Apple knows). So a developer that wants to build a Java app needs to have a static library version of the entire JDK to link with into a single executable Java launcher. So in this perspective, the entire use of dlopen in a static build smells funny, and might be just a left-over from static build being more of a hack than a properly supported build option. > > /Magnus The "static build" doesn't build anymore. We're all shocked that this untested configuration has bit-rotted. :) LoadJavaVM uses RTLD_NOW in the normal (non-"static build") case, the workaround is superfluous in that case. Given that, I'm going to delete the workaround and file an RFE [1] to fix or remove the currently broken "static build" support, with a comment there referring to this workaround code as possibly being relevant to fixing the static build. [1] https://bugs.openjdk.java.net/browse/JDK-8211732 New webrev: http://cr.openjdk.java.net/~kbarrett/8211296/open.01/ From mikael.vidstedt at oracle.com Thu Oct 4 21:25:06 2018 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 4 Oct 2018 14:25:06 -0700 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: <39BB147B-089C-44D6-BD29-3F5B906D9EE9@oracle.com> References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> <250FA229-7F95-4FB9-935D-7834A203072F@oracle.com> <39BB147B-089C-44D6-BD29-3F5B906D9EE9@oracle.com> Message-ID: > On Oct 4, 2018, at 2:18 PM, Kim Barrett wrote: > >> On Oct 4, 2018, at 9:27 AM, Magnus Ihse Bursie wrote: >>> That all assumes that static build actually works at all; it doesn?t look like we >>> test that configuration these days, so who knows what bit rot may have set in. >>> >> >> No, we don't test static build, and never have. I'd be surprised if it's still possible to build a static build. >> >> This was a feature developed for the mobile project. The static build part was merged, somewhat hastily, into mainline, but the rest of the mobile code is still lingering in the mobile project fork. >> >> The entire "static build" concept was somewhat artificially injected in some parts of the build code. It is for instance not possible to build all modules with a static build. At the very least you need to build without java.desktop, if I remember correctly. >> >> The motivation for the static build is that on iOS, apps are not allowed to load dynamic libraries (for some reason only Apple knows). So a developer that wants to build a Java app needs to have a static library version of the entire JDK to link with into a single executable Java launcher. So in this perspective, the entire use of dlopen in a static build smells funny, and might be just a left-over from static build being more of a hack than a properly supported build option. >> >> /Magnus > > The "static build" doesn't build anymore. We're all shocked that this > untested configuration has bit-rotted. :) > > LoadJavaVM uses RTLD_NOW in the normal (non-"static build") case, the > workaround is superfluous in that case. Given that, I'm going to > delete the workaround and file an RFE [1] to fix or remove the > currently broken "static build" support, with a comment there > referring to this workaround code as possibly being relevant to fixing > the static build. I?m glad to see this gone. Looks good! Cheers, Mikael > > [1] https://bugs.openjdk.java.net/browse/JDK-8211732 > > New webrev: > http://cr.openjdk.java.net/~kbarrett/8211296/open.01/ > From david.holmes at oracle.com Thu Oct 4 21:28:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 5 Oct 2018 07:28:30 +1000 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> Message-ID: <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> Hi Severin, On 4/10/2018 11:03 PM, Severin Gehwolf wrote: > Hi, > > Please review this forward port for Zero coming from JDK 7 (where Zero > is maintained in icedtea) to JDK 12. The update is to use ldrexd > instructions on ARMv7 CPUs for atomic_copy64() in Zero: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8211387 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8211387/webrev.01/ Not sure how zero normally operates but I would have expected to see something more dynamic (as used in the rest of the VM: VM_version::supports_ldrexd()) rather than hardwiring for a v7_A build. Also I thought that if ldrex was not paired with a strex then you should at least issue clrex - as done in generate_atomic_load_long() in stubGenerator_arm.cpp Cheers, David > Testing: Zero bootcycle-images build on armv7 where this was satisfied: > > $ gcc -dM -E - < /dev/null | grep __ARM_ARCH_7A__ > #define __ARM_ARCH_7A__ 1 > > This change was contributed by Andrew Haley. > > Thoughts? > > Thanks, > Severin > From david.holmes at oracle.com Thu Oct 4 21:31:33 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 5 Oct 2018 07:31:33 +1000 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: <39BB147B-089C-44D6-BD29-3F5B906D9EE9@oracle.com> References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> <250FA229-7F95-4FB9-935D-7834A203072F@oracle.com> <39BB147B-089C-44D6-BD29-3F5B906D9EE9@oracle.com> Message-ID: <2dad2275-684a-50f6-d4ef-f20e2c7241b1@oracle.com> Looks good! Thanks for your patience and perseverance! :) David On 5/10/2018 7:18 AM, Kim Barrett wrote: >> On Oct 4, 2018, at 9:27 AM, Magnus Ihse Bursie wrote: >>> That all assumes that static build actually works at all; it doesn?t look like we >>> test that configuration these days, so who knows what bit rot may have set in. >>> >> >> No, we don't test static build, and never have. I'd be surprised if it's still possible to build a static build. >> >> This was a feature developed for the mobile project. The static build part was merged, somewhat hastily, into mainline, but the rest of the mobile code is still lingering in the mobile project fork. >> >> The entire "static build" concept was somewhat artificially injected in some parts of the build code. It is for instance not possible to build all modules with a static build. At the very least you need to build without java.desktop, if I remember correctly. >> >> The motivation for the static build is that on iOS, apps are not allowed to load dynamic libraries (for some reason only Apple knows). So a developer that wants to build a Java app needs to have a static library version of the entire JDK to link with into a single executable Java launcher. So in this perspective, the entire use of dlopen in a static build smells funny, and might be just a left-over from static build being more of a hack than a properly supported build option. >> >> /Magnus > > The "static build" doesn't build anymore. We're all shocked that this > untested configuration has bit-rotted. :) > > LoadJavaVM uses RTLD_NOW in the normal (non-"static build") case, the > workaround is superfluous in that case. Given that, I'm going to > delete the workaround and file an RFE [1] to fix or remove the > currently broken "static build" support, with a comment there > referring to this workaround code as possibly being relevant to fixing > the static build. > > [1] https://bugs.openjdk.java.net/browse/JDK-8211732 > > New webrev: > http://cr.openjdk.java.net/~kbarrett/8211296/open.01/ > From kim.barrett at oracle.com Thu Oct 4 21:53:27 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Oct 2018 17:53:27 -0400 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: <3C557F9A-82B7-4941-850B-EA2D0ED44EDA@oracle.com> > On Oct 4, 2018, at 4:35 AM, Volker Simonis wrote: > > Hi Tim, Jeff, > > Kim has assembled a quite detailed list of new C++ features intended for use in HotSpot/OpenJDK (with links to the corresponding C++ standard) > > https://bugs.openjdk.java.net/browse/JDK-8208089 Note that this list is intended as a starting point. The JEP also describes a process for adding to the list of permitted features, and I anticipate that process will be used. There are several that I personally think would be good to add (perhaps with some limits or restrictions), but that I think may involve more detailed discussion. And other folks may have their favorite must have feature(s) that didn?t seem to me to be needed in this initial seed list. From kim.barrett at oracle.com Thu Oct 4 22:32:41 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Oct 2018 18:32:41 -0400 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> <250FA229-7F95-4FB9-935D-7834A203072F@oracle.com> <39BB147B-089C-44D6-BD29-3F5B906D9EE9@oracle.com> Message-ID: > On Oct 4, 2018, at 5:25 PM, Mikael Vidstedt wrote: >> The "static build" doesn't build anymore. We're all shocked that this >> untested configuration has bit-rotted. :) >> >> LoadJavaVM uses RTLD_NOW in the normal (non-"static build") case, the >> workaround is superfluous in that case. Given that, I'm going to >> delete the workaround and file an RFE [1] to fix or remove the >> currently broken "static build" support, with a comment there >> referring to this workaround code as possibly being relevant to fixing >> the static build. > > I?m glad to see this gone. Looks good! Thanks! > > Cheers, > Mikael > >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8211732 >> >> New webrev: >> http://cr.openjdk.java.net/~kbarrett/8211296/open.01/ From kim.barrett at oracle.com Thu Oct 4 22:33:27 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Oct 2018 18:33:27 -0400 Subject: RFR: 8211296: Remove HotSpot deprecation warning suppression for Mac/clang In-Reply-To: <2dad2275-684a-50f6-d4ef-f20e2c7241b1@oracle.com> References: <050B2C90-F236-4C15-8C32-E54B63C152C6@oracle.com> <4052541c-8f3a-f7e4-a094-11beadd6a60d@oracle.com> <0a2f01f4-608e-0a1c-dd4d-9d2c859524a9@oracle.com> <6BA6A076-FABB-40E4-9BBE-8F10A03A8A9A@oracle.com> <250FA229-7F95-4FB9-935D-7834A203072F@oracle.com> <39BB147B-089C-44D6-BD29-3F5B906D9EE9@oracle.com> <2dad2275-684a-50f6-d4ef-f20e2c7241b1@oracle.com> Message-ID: <724BAC82-812B-40F6-84F9-9327203101B3@oracle.com> > On Oct 4, 2018, at 5:31 PM, David Holmes wrote: > > Looks good! > > Thanks for your patience and perseverance! :) Thanks! And thanks for your patience and perseverance too! > > David From erik.helin at oracle.com Fri Oct 5 07:58:53 2018 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 5 Oct 2018 09:58:53 +0200 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <940757E3-8781-4E7A-B5B9-1F0D3E944E0F@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <940757E3-8781-4E7A-B5B9-1F0D3E944E0F@oracle.com> Message-ID: <8a771c9d-0803-d8c7-20f2-675733d416c4@oracle.com> On 10/4/18 2:31 PM, Magnus Ihse Bursie wrote:>> 4 okt. 2018 kl. 10:57 skrev Martijn Verburg : >> I like this initiative. I'm wondering if some of these rules can be easily >> codified or written into a jcheck style checker (ccheck?) so that Authors >> can adhere to the conventions and not rely on a Human review to pick out >> where that convention isn't met. > > That's an interesting thought! > > I googled around a bit, but could find no obvious candidate for doing such a check. In fact, parsing and analyzing C++ seems quite a hard problem, compared to many other languages. (Sad, but not really surprising.) > > I found two possible routes to explore: cpplint [1], the official Google tool to verify that the Google C++ Style Guide [2] is followed, and Vera++ [3], a framework for creating scripts that can analyze and/or transform C++ code. As usual there is of a third route, this one is "just a matter of programming" :) The (probably) easiest way to enforce usage of only a subset of C++ is to write custom clang-tidy checks [0]. We could most likely write a couple of ASTMatchers [1] to watch out for features we do not want. It used to be very hard to use the clang tooling with HotSpot, but thanks to Robin Westberg's compile-commands patch [2] that just went in, this is no longer an issue. Unfortunately I don't think requiring all contributors to install a custom version of clang-tidy with HotSpot checks is feasible, this is more something you would want to run in a CI setup. Thanks, Erik [0]: https://clang.llvm.org/extra/clang-tidy/#writing-a-clang-tidy-check [1]: http://clang.llvm.org/docs/LibASTMatchers.html [2]: http://hg.openjdk.java.net/jdk/jdk/rev/804792ce736f From goetz.lindenmaier at sap.com Fri Oct 5 08:13:21 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 5 Oct 2018 08:13:21 +0000 Subject: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <2DB05A59-EA8F-46FD-9ECF-CA47DA5E313E@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <71fb517fba794dc28291cf51afb6e8b9@sap.com> <2DB05A59-EA8F-46FD-9ECF-CA47DA5E313E@oracle.com> Message-ID: <22d25e558b2d426f827d3322b9a56d3a@sap.com> Hi Kim, > for XLC 16. Any information about "some C++14 features"? I haven't > found any detail on that. we will user our internal connections to IBM to find out :) Best regards, Goetz. > -----Original Message----- > From: Kim Barrett > Sent: Donnerstag, 4. Oktober 2018 23:01 > To: Lindenmaier, Goetz > Cc: hotspot-dev developers ; build-dev > ; core-libs-dev at openjdk.java.net > Subject: Re: JEP JDK-8208089: Implement C++14 Language Features > > > On Oct 4, 2018, at 2:59 AM, Lindenmaier, Goetz > wrote: > > > > Hi, > > > > I appreciate this is handled in a JEP and well communicated! > > > > XLc, the compiler we use for AIX, might be a bottleneck here. > > > > But we currently have no compiler-constraints by products on > > our aix port of jdk12 (for jdk11 we must stay with the current > > compiler xlc 12 which does not support C++11, and jdk11 even > > should be compilable with aCC by HP for which we won't > > get new versions!) > > > > We will update our compiler for jdk/jdk to the most recent Xlc > > as soon as possible. > > To my current knowledge, Xlc 14 was dropped, and xlc 16 > > is to be released early 2019. It is supposed to support > > C++ 11, and also some C++ 14 features. > > The information I've been able to find mostly discusses Linux support > for XLC 16. Any information about "some C++14 features"? I haven't > found any detail on that. > > I am working toward being able to target this for JDK 12, but realize > that might not be reachable. Windows is already there, because of > VS2017 support. There are a couple of minor patches needed for gcc > and clang based builds; Linux-x64 and MacOSX-x64 have had a fair > amount of ad hoc testing. We (Oracle) only switched over the > Solaris/sparc builds to the necessary compiler version (Oracle Studio > 12.6) very recently, and there are some issues still to be dealt with > there. And the currently used XLC is just not up to the change. From matthias.baesken at sap.com Fri Oct 5 08:29:36 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 5 Oct 2018 08:29:36 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Message-ID: Hi Thomas , looking forward to the CSR , I think we can use then the flags introduced in the CSR in 8211326 . Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Donnerstag, 4. Oktober 2018 20:11 > To: David Holmes > Cc: Baesken, Matthias ; HotSpot Open Source > Developers > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > On Thu, Oct 4, 2018 at 11:18 AM David Holmes > wrote: > > > > Hi Thomas, > > > > On 4/10/2018 6:27 PM, Thomas St?fe wrote: > > > Hi David, > > > On Thu, Oct 4, 2018 at 9:44 AM David Holmes > wrote: > > >> > > >> Hi Matthias, > > >> > > >> I'm hoping others will chime in here as I: > > >> > > >> a) don't know if this information is actually useful for an error log of > > >> this kind; > > >> > > >> b) don't know if the information might be considered sensitive or not; > and > > >> > > > > > > I have no opinion on (a) and (b). > > > > That's a shame :) > > > > >> c) don't think it's worth the effort of adding a flag to control this. > > >> Plus the flag is only useful for trying to reproduce an issue; if it's a > > >> one-of failure then you've already missed out on the information in the > > >> log file. > > > > > > How about a more generic switch to control verbosity of the error report? > > > > > > The way we and you use the error files seem to differ. You seem to > > > prefer them short and snappy and bare any security relevant details > > > (as far as that is even possible in an hs-err file). As was once > > > mentioned in a similar discussion, "OpenJDK hs-err files get posted > > > verbatim in forums and bug reports". > > > > > > We use the hs-err files differently. They are usually handed down to > > > us by our customers thru secure channels, and for us size does not > > > matter much, nor does security relevant information since we have > > > contracts with our customers. > > > > To be honest I don't deal with hs-err files from other people very much > > at all. But from what I have dealt with I haven't, to the best of my > > recollection, encountered any crash investigation where all this extra > > info would have shed any light. But most crashes I investigate are > > developer related rather than end user. > > That may be the difference. > > We get crash reports from customers, often with little context, no > repros, no cores, nothing. So we have to make do with what we have, > which is the hs-err files. Therefore we make them as extensive as > possible. Our support worked miracles with just these files. It is > incredible what you can deduce from them. > > User information, e.g., can hint towards a mismanaged installation, if > the server node does not run under the correct user. I also dimly > remember it being useful in analyzing SIGBUS crashes for mapped files > on Solaris. > > So, it turns out I have an opinion :) Well when I say no opinion I > mean I can see arguments for both sides, and I feel uncomfortable > pressing our way of working upon the OpenJDK code base. > > > > > > That has been a point of contention over and over again in the past. > > > So I wonder whether one, or possibly two, general switches could keep > > > both sides happy. Something like -XX:+ExtendedErrorReports" and > > > possibly "-XX:+ErrorReportsIncludeSensitiveData". > > > > > > Those switches could be, by default, false in the OpenJDK. > > > > > > Any additions we add to error reporting where we cannot find an > > > agreement we could make conditional on one or the other switch. > > > > > > What do you think? > > > > Seems reasonable to propose. Any new flag will require a CSR request. > > > > Okay, thanks! > > I will file a CSR next week when I am back at the office. > > ..Thomas > From martin.doerr at sap.com Fri Oct 5 09:05:31 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 5 Oct 2018 09:05:31 +0000 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: References: <68a26e6e6e574f75aec64ede7487a117@sap.com> <7b92e8ff-7dea-1130-b768-874faf2794dc@oracle.com> Message-ID: <7f40e04ade1c463cad48a056115e624d@sap.com> Hi Coleen, thanks for fixing this issue. I appreciate it. Looks good. Do you think it can get backported to 11u? Best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of coleen.phillimore at oracle.com Sent: Donnerstag, 4. Oktober 2018 19:58 To: hotspot-dev at openjdk.java.net Subject: Re: RFR (S) 8209889: RedefineStress tests crash Martin, Here's your version of the fix, freshly tested. http://cr.openjdk.java.net/~coleenp/8209889.02/webrev/index.html Thanks, Coleen On 10/4/18 12:09 PM, coleen.phillimore at oracle.com wrote: > > > On 10/4/18 10:34 AM, Doerr, Martin wrote: >> >> Hi Coleen, >> >> good catch. Looks good where you moved it. >> >> I couldn?t find a place for a NoSafepointVerifier, either. >> >> Maybe it?d be more clear if we replaced the ?continue? in the NULL >> case by an else case in the loop so the CompileTaskWrapper is the >> first thing after the NULL check (see below). >> > > I see, thanks to the magic of patch, the > possibly_add_compiler_threads() is inside the 'else' clause where > there's a compile task on the queue.? Which makes perfect sense. It > seems like there should be a possibly_remove_compiler_thread() in the > 'if' case but I'll leave that to the next person to touch this code. > > Ok, I'll test this variant, which I also like. > thanks, > Coleen > >> Best regards, >> >> Martin >> >> --- a/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >> 14:03:13 2018 +0200 >> >> +++ b/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >> 16:29:45 2018 +0200 >> >> @@ -1781,30 +1781,31 @@ >> >> ?????????? return; // Stop this thread. >> >> ???????? } >> >> ?????? } >> >> -????? continue; >> >> -??? } >> >> - >> >> -??? if (UseDynamicNumberOfCompilerThreads) { >> >> -????? possibly_add_compiler_threads(); >> >> -??? } >> >> +??? } else { >> >> +????? // Assign the task to the current thread.? Mark this compilation >> >> +????? // thread as active for the profiler. >> >> +????? // CompileTaskWrapper also keeps the Method* from being >> deallocated if redefinition >> >> +????? // occurs after fetching the compile task off the queue. >> >> +????? CompileTaskWrapper ctw(task); >> >> +????? nmethodLocker result_handle;? // (handle for the nmethod >> produced by this task) >> >> + task->set_code_handle(&result_handle); >> >> +????? methodHandle method(thread, task->method()); >> >> -??? // Assign the task to the current thread. Mark this compilation >> >> -??? // thread as active for the profiler. >> >> -??? CompileTaskWrapper ctw(task); >> >> -??? nmethodLocker result_handle;? // (handle for the nmethod >> produced by this task) >> >> - task->set_code_handle(&result_handle); >> >> -??? methodHandle method(thread, task->method()); >> >> +????? // Never compile a method if breakpoints are present in it >> >> +????? if (method()->number_of_breakpoints() == 0) { >> >> +??????? // Compile the method. >> >> +??????? if ((UseCompiler || AlwaysCompileLoopMethods) && >> CompileBroker::should_compile_new_jobs()) { >> >> +????????? invoke_compiler_on_method(task); >> >> +????????? thread->start_idle_timer(); >> >> +???? ???} else { >> >> +????????? // After compilation is disabled, remove remaining methods >> from queue >> >> + method->clear_queued_for_compilation(); >> >> + task->set_failure_reason("compilation is disabled"); >> >> +??????? } >> >> +????? } >> >> -??? // Never compile a method if breakpoints are present in it >> >> -??? if (method()->number_of_breakpoints() == 0) { >> >> -????? // Compile the method. >> >> -????? if ((UseCompiler || AlwaysCompileLoopMethods) && >> CompileBroker::should_compile_new_jobs()) { >> >> -??????? invoke_compiler_on_method(task); >> >> -??????? thread->start_idle_timer(); >> >> -????? } else { >> >> -??????? // After compilation is disabled, remove remaining methods >> from queue >> >> - method->clear_queued_for_compilation(); >> >> - task->set_failure_reason("compilation is disabled"); >> >> +????? if (UseDynamicNumberOfCompilerThreads) { >> >> +??????? possibly_add_compiler_threads(); >> >> ?????? } >> >> ???? } >> >> ?? } >> >> *From:*coleen.phillimore at oracle.com >> *Sent:* Donnerstag, 4. Oktober 2018 13:45 >> *To:* hotspot-dev developers ; Doerr, >> Martin >> *Subject:* RFR (S) 8209889: RedefineStress tests crash >> >> Summary: Create CompileTaskWrapper before potential safepoint >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev >> >> bug link https://bugs.openjdk.java.net/browse/JDK-8209889 >> >> MetadataOnStackMark needs to find the CompileTask in the current >> thread, or else the Method isn't marked as being alive, so if it's >> old/obsolete, it is freed.? There are checks to disable adding old >> methods to the compile queue but if the method is redefined after >> this point, there may be a preexisting bug that we compile an old >> method.? This change just fixes the crash. >> >> Can some compiler person (Martin?) look at where I moved >> possibly_add_compiler_threads and let me know if that looks okay?? >> There's a comment in the change that says why you need to create the >> CompileTaskWrapper before that call.? I couldn't find a place to put >> a NoSafepointVerifier. >> >> Tested with the test that failed, which doesn't reproduce the bug >> except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in >> progress. >> >> Thanks, >> Coleen >> > From sgehwolf at redhat.com Fri Oct 5 13:40:07 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 05 Oct 2018 15:40:07 +0200 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> Message-ID: Hi David, On Fri, 2018-10-05 at 07:28 +1000, David Holmes wrote: > Hi Severin, > > On 4/10/2018 11:03 PM, Severin Gehwolf wrote: > > Hi, > > > > Please review this forward port for Zero coming from JDK 7 (where Zero > > is maintained in icedtea) to JDK 12. The update is to use ldrexd > > instructions on ARMv7 CPUs for atomic_copy64() in Zero: > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8211387 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8211387/webrev.01/ > > Not sure how zero normally operates but I would have expected to see > something more dynamic (as used in the rest of the VM: > VM_version::supports_ldrexd()) rather than hardwiring for a v7_A build. Unfortunately, this isn't as easily done for Zero. It might be running on some arbitrary architecture. Not a specific one which supports different features, like arm. We don't initialize VM_version of Zero with any sort of achitecture specific info for that reason. Since ldrexd is available for ARMv7 and up guarding code being generated at compile time that way seems reasonable to me. > Also I thought that if ldrex was not paired with a strex then you should > at least issue clrex - as done in generate_atomic_load_long() in > stubGenerator_arm.cpp I'll defer to Andrew Haley for this. FWIW, the arm version snippet from generate_atomic_load_long() looks like this: if (!os::is_MP()) { __ ldmia(src, RegisterSet(result_lo, result_hi)); __ bx(LR); } else if (VM_Version::supports_ldrexd()) { __ ldrexd(result_lo, Address(src)); __ clrex(); // FIXME: safe to remove? __ bx(LR); } else { __ stop("Atomic load(jlong) unsupported on this platform"); __ bx(LR); } Note the FIXME. I wonder what's the best course of action here. a) remove clrex() in the arm port or b) add clrex to the Zero atomic assembly too and remove the FIXME. Something else? Thanks, Severin > Cheers, > David > > > Testing: Zero bootcycle-images build on armv7 where this was satisfied: > > > > $ gcc -dM -E - < /dev/null | grep __ARM_ARCH_7A__ > > #define __ARM_ARCH_7A__ 1 > > > > This change was contributed by Andrew Haley. > > > > Thoughts? > > > > Thanks, > > Severin > > From aph at redhat.com Fri Oct 5 14:18:42 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 5 Oct 2018 15:18:42 +0100 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> Message-ID: On 10/05/2018 02:40 PM, Severin Gehwolf wrote: >> Also I thought that if ldrex was not paired with a strex then you should >> at least issue clrex - as done in generate_atomic_load_long() in >> stubGenerator_arm.cpp > > I'll defer to Andrew Haley for this. I'm skeptical that CLREX is necessary. The classic CAS for Arm is MOV R1, #0xFF ; load the ?lock taken? value try LDREX R0, [LockAddr] ; load the lock value CMP R0, #0 ; is the lock free? STREXEQ R1, R0, [LockAddr]; try and claim the lock CMPEQ R0, #0 ; did this succeed? BNE try ; no ? try again. . . ; yes ? we have the lock There's no CLREX, and no-one ever reported any problem. CLREX could be a performance boost because it takes the cache line out of exclusive state back to shared state, reducing later bus traffic. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Fri Oct 5 14:22:48 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 5 Oct 2018 15:22:48 +0100 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> Message-ID: On 10/05/2018 03:18 PM, Andrew Haley wrote: > > There's no CLREX, and no-one ever reported any problem. CLREX could be > a performance boost because it takes the cache line out of exclusive > state back to shared state, reducing later bus traffic. One more thing: LoadExclusive and StoreExclusive must be executed in program order with the same address. CLREX is essential in interrupt code because you might return to a StoreExclusive instruction and in that case the StoreExclusive must fail: the CLREX guarantees that. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Fri Oct 5 14:54:05 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 5 Oct 2018 10:54:05 -0400 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: <7f40e04ade1c463cad48a056115e624d@sap.com> References: <68a26e6e6e574f75aec64ede7487a117@sap.com> <7b92e8ff-7dea-1130-b768-874faf2794dc@oracle.com> <7f40e04ade1c463cad48a056115e624d@sap.com> Message-ID: Hi Martin, Thank you for the code review and suggestion for fix. I've created a backport issue: https://bugs.openjdk.java.net/browse/JDK-8211775 The backport doesn't merge automatically because of the removal of CompilerThreadHintNoPreempt (which doesn't safepoint). http://cr.openjdk.java.net/~coleenp/8211775.01/webrev/ But it seems trivial.?? I'm running mach5 tests on this.?? Can you review this too? thanks, Coleen On 10/5/18 5:05 AM, Doerr, Martin wrote: > Hi Coleen, > > thanks for fixing this issue. I appreciate it. Looks good. > > Do you think it can get backported to 11u? > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of coleen.phillimore at oracle.com > Sent: Donnerstag, 4. Oktober 2018 19:58 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR (S) 8209889: RedefineStress tests crash > > > Martin, Here's your version of the fix, freshly tested. > http://cr.openjdk.java.net/~coleenp/8209889.02/webrev/index.html > Thanks, > Coleen > > On 10/4/18 12:09 PM, coleen.phillimore at oracle.com wrote: >> >> On 10/4/18 10:34 AM, Doerr, Martin wrote: >>> Hi Coleen, >>> >>> good catch. Looks good where you moved it. >>> >>> I couldn?t find a place for a NoSafepointVerifier, either. >>> >>> Maybe it?d be more clear if we replaced the ?continue? in the NULL >>> case by an else case in the loop so the CompileTaskWrapper is the >>> first thing after the NULL check (see below). >>> >> I see, thanks to the magic of patch, the >> possibly_add_compiler_threads() is inside the 'else' clause where >> there's a compile task on the queue.? Which makes perfect sense. It >> seems like there should be a possibly_remove_compiler_thread() in the >> 'if' case but I'll leave that to the next person to touch this code. >> >> Ok, I'll test this variant, which I also like. >> thanks, >> Coleen >> >>> Best regards, >>> >>> Martin >>> >>> --- a/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >>> 14:03:13 2018 +0200 >>> >>> +++ b/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >>> 16:29:45 2018 +0200 >>> >>> @@ -1781,30 +1781,31 @@ >>> >>> ?????????? return; // Stop this thread. >>> >>> ???????? } >>> >>> ?????? } >>> >>> -????? continue; >>> >>> -??? } >>> >>> - >>> >>> -??? if (UseDynamicNumberOfCompilerThreads) { >>> >>> -????? possibly_add_compiler_threads(); >>> >>> -??? } >>> >>> +??? } else { >>> >>> +????? // Assign the task to the current thread.? Mark this compilation >>> >>> +????? // thread as active for the profiler. >>> >>> +????? // CompileTaskWrapper also keeps the Method* from being >>> deallocated if redefinition >>> >>> +????? // occurs after fetching the compile task off the queue. >>> >>> +????? CompileTaskWrapper ctw(task); >>> >>> +????? nmethodLocker result_handle;? // (handle for the nmethod >>> produced by this task) >>> >>> + task->set_code_handle(&result_handle); >>> >>> +????? methodHandle method(thread, task->method()); >>> >>> -??? // Assign the task to the current thread. Mark this compilation >>> >>> -??? // thread as active for the profiler. >>> >>> -??? CompileTaskWrapper ctw(task); >>> >>> -??? nmethodLocker result_handle;? // (handle for the nmethod >>> produced by this task) >>> >>> - task->set_code_handle(&result_handle); >>> >>> -??? methodHandle method(thread, task->method()); >>> >>> +????? // Never compile a method if breakpoints are present in it >>> >>> +????? if (method()->number_of_breakpoints() == 0) { >>> >>> +??????? // Compile the method. >>> >>> +??????? if ((UseCompiler || AlwaysCompileLoopMethods) && >>> CompileBroker::should_compile_new_jobs()) { >>> >>> +????????? invoke_compiler_on_method(task); >>> >>> +????????? thread->start_idle_timer(); >>> >>> +???? ???} else { >>> >>> +????????? // After compilation is disabled, remove remaining methods >>> from queue >>> >>> + method->clear_queued_for_compilation(); >>> >>> + task->set_failure_reason("compilation is disabled"); >>> >>> +??????? } >>> >>> +????? } >>> >>> -??? // Never compile a method if breakpoints are present in it >>> >>> -??? if (method()->number_of_breakpoints() == 0) { >>> >>> -????? // Compile the method. >>> >>> -????? if ((UseCompiler || AlwaysCompileLoopMethods) && >>> CompileBroker::should_compile_new_jobs()) { >>> >>> -??????? invoke_compiler_on_method(task); >>> >>> -??????? thread->start_idle_timer(); >>> >>> -????? } else { >>> >>> -??????? // After compilation is disabled, remove remaining methods >>> from queue >>> >>> - method->clear_queued_for_compilation(); >>> >>> - task->set_failure_reason("compilation is disabled"); >>> >>> +????? if (UseDynamicNumberOfCompilerThreads) { >>> >>> +??????? possibly_add_compiler_threads(); >>> >>> ?????? } >>> >>> ???? } >>> >>> ?? } >>> >>> *From:*coleen.phillimore at oracle.com >>> *Sent:* Donnerstag, 4. Oktober 2018 13:45 >>> *To:* hotspot-dev developers ; Doerr, >>> Martin >>> *Subject:* RFR (S) 8209889: RedefineStress tests crash >>> >>> Summary: Create CompileTaskWrapper before potential safepoint >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev >>> >>> bug link https://bugs.openjdk.java.net/browse/JDK-8209889 >>> >>> MetadataOnStackMark needs to find the CompileTask in the current >>> thread, or else the Method isn't marked as being alive, so if it's >>> old/obsolete, it is freed.? There are checks to disable adding old >>> methods to the compile queue but if the method is redefined after >>> this point, there may be a preexisting bug that we compile an old >>> method.? This change just fixes the crash. >>> >>> Can some compiler person (Martin?) look at where I moved >>> possibly_add_compiler_threads and let me know if that looks okay? >>> There's a comment in the change that says why you need to create the >>> CompileTaskWrapper before that call.? I couldn't find a place to put >>> a NoSafepointVerifier. >>> >>> Tested with the test that failed, which doesn't reproduce the bug >>> except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in >>> progress. >>> >>> Thanks, >>> Coleen >>> From martin.doerr at sap.com Fri Oct 5 14:56:37 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 5 Oct 2018 14:56:37 +0000 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: References: <68a26e6e6e574f75aec64ede7487a117@sap.com> <7b92e8ff-7dea-1130-b768-874faf2794dc@oracle.com> <7f40e04ade1c463cad48a056115e624d@sap.com> Message-ID: <2e5bf818b4d6491f901841a87c19d298@sap.com> Hi Coleen, thank you for the backport. Looks good, too. Best regards, Martin -----Original Message----- From: coleen.phillimore at oracle.com Sent: Freitag, 5. Oktober 2018 16:54 To: Doerr, Martin ; hotspot-dev at openjdk.java.net Subject: Re: RFR (S) 8209889: RedefineStress tests crash Hi Martin, Thank you for the code review and suggestion for fix. I've created a backport issue: https://bugs.openjdk.java.net/browse/JDK-8211775 The backport doesn't merge automatically because of the removal of CompilerThreadHintNoPreempt (which doesn't safepoint). http://cr.openjdk.java.net/~coleenp/8211775.01/webrev/ But it seems trivial.?? I'm running mach5 tests on this.?? Can you review this too? thanks, Coleen On 10/5/18 5:05 AM, Doerr, Martin wrote: > Hi Coleen, > > thanks for fixing this issue. I appreciate it. Looks good. > > Do you think it can get backported to 11u? > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of coleen.phillimore at oracle.com > Sent: Donnerstag, 4. Oktober 2018 19:58 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR (S) 8209889: RedefineStress tests crash > > > Martin, Here's your version of the fix, freshly tested. > http://cr.openjdk.java.net/~coleenp/8209889.02/webrev/index.html > Thanks, > Coleen > > On 10/4/18 12:09 PM, coleen.phillimore at oracle.com wrote: >> >> On 10/4/18 10:34 AM, Doerr, Martin wrote: >>> Hi Coleen, >>> >>> good catch. Looks good where you moved it. >>> >>> I couldn?t find a place for a NoSafepointVerifier, either. >>> >>> Maybe it?d be more clear if we replaced the ?continue? in the NULL >>> case by an else case in the loop so the CompileTaskWrapper is the >>> first thing after the NULL check (see below). >>> >> I see, thanks to the magic of patch, the >> possibly_add_compiler_threads() is inside the 'else' clause where >> there's a compile task on the queue.? Which makes perfect sense. It >> seems like there should be a possibly_remove_compiler_thread() in the >> 'if' case but I'll leave that to the next person to touch this code. >> >> Ok, I'll test this variant, which I also like. >> thanks, >> Coleen >> >>> Best regards, >>> >>> Martin >>> >>> --- a/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >>> 14:03:13 2018 +0200 >>> >>> +++ b/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >>> 16:29:45 2018 +0200 >>> >>> @@ -1781,30 +1781,31 @@ >>> >>> ?????????? return; // Stop this thread. >>> >>> ???????? } >>> >>> ?????? } >>> >>> -????? continue; >>> >>> -??? } >>> >>> - >>> >>> -??? if (UseDynamicNumberOfCompilerThreads) { >>> >>> -????? possibly_add_compiler_threads(); >>> >>> -??? } >>> >>> +??? } else { >>> >>> +????? // Assign the task to the current thread.? Mark this compilation >>> >>> +????? // thread as active for the profiler. >>> >>> +????? // CompileTaskWrapper also keeps the Method* from being >>> deallocated if redefinition >>> >>> +????? // occurs after fetching the compile task off the queue. >>> >>> +????? CompileTaskWrapper ctw(task); >>> >>> +????? nmethodLocker result_handle;? // (handle for the nmethod >>> produced by this task) >>> >>> + task->set_code_handle(&result_handle); >>> >>> +????? methodHandle method(thread, task->method()); >>> >>> -??? // Assign the task to the current thread. Mark this compilation >>> >>> -??? // thread as active for the profiler. >>> >>> -??? CompileTaskWrapper ctw(task); >>> >>> -??? nmethodLocker result_handle;? // (handle for the nmethod >>> produced by this task) >>> >>> - task->set_code_handle(&result_handle); >>> >>> -??? methodHandle method(thread, task->method()); >>> >>> +????? // Never compile a method if breakpoints are present in it >>> >>> +????? if (method()->number_of_breakpoints() == 0) { >>> >>> +??????? // Compile the method. >>> >>> +??????? if ((UseCompiler || AlwaysCompileLoopMethods) && >>> CompileBroker::should_compile_new_jobs()) { >>> >>> +????????? invoke_compiler_on_method(task); >>> >>> +????????? thread->start_idle_timer(); >>> >>> +???? ???} else { >>> >>> +????????? // After compilation is disabled, remove remaining methods >>> from queue >>> >>> + method->clear_queued_for_compilation(); >>> >>> + task->set_failure_reason("compilation is disabled"); >>> >>> +??????? } >>> >>> +????? } >>> >>> -??? // Never compile a method if breakpoints are present in it >>> >>> -??? if (method()->number_of_breakpoints() == 0) { >>> >>> -????? // Compile the method. >>> >>> -????? if ((UseCompiler || AlwaysCompileLoopMethods) && >>> CompileBroker::should_compile_new_jobs()) { >>> >>> -??????? invoke_compiler_on_method(task); >>> >>> -??????? thread->start_idle_timer(); >>> >>> -????? } else { >>> >>> -??????? // After compilation is disabled, remove remaining methods >>> from queue >>> >>> - method->clear_queued_for_compilation(); >>> >>> - task->set_failure_reason("compilation is disabled"); >>> >>> +????? if (UseDynamicNumberOfCompilerThreads) { >>> >>> +??????? possibly_add_compiler_threads(); >>> >>> ?????? } >>> >>> ???? } >>> >>> ?? } >>> >>> *From:*coleen.phillimore at oracle.com >>> *Sent:* Donnerstag, 4. Oktober 2018 13:45 >>> *To:* hotspot-dev developers ; Doerr, >>> Martin >>> *Subject:* RFR (S) 8209889: RedefineStress tests crash >>> >>> Summary: Create CompileTaskWrapper before potential safepoint >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev >>> >>> bug link https://bugs.openjdk.java.net/browse/JDK-8209889 >>> >>> MetadataOnStackMark needs to find the CompileTask in the current >>> thread, or else the Method isn't marked as being alive, so if it's >>> old/obsolete, it is freed.? There are checks to disable adding old >>> methods to the compile queue but if the method is redefined after >>> this point, there may be a preexisting bug that we compile an old >>> method.? This change just fixes the crash. >>> >>> Can some compiler person (Martin?) look at where I moved >>> possibly_add_compiler_threads and let me know if that looks okay? >>> There's a comment in the change that says why you need to create the >>> CompileTaskWrapper before that call.? I couldn't find a place to put >>> a NoSafepointVerifier. >>> >>> Tested with the test that failed, which doesn't reproduce the bug >>> except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in >>> progress. >>> >>> Thanks, >>> Coleen >>> From erik.osterlund at oracle.com Fri Oct 5 15:07:04 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 5 Oct 2018 17:07:04 +0200 Subject: RFR: 8210498: nmethod entry barriers Message-ID: <5BB77E18.6040401@oracle.com> Hi, In order to implement concurrent class unloading for ZGC, a mechanism will be required to arm nmethods in a safepoint such that the first entry into an nmethod after the pause triggers a barrier before continuing. The barrier will be used for: * Patching immediate oops, and keeping phantomly reachable oops alive through the nmethod entry * Catching calls from stale IC caches (due to class unloading) into nmethods that are being unloaded (containing broken oops and metadata), and lazily clean the IC cache and re-resolve the call. This implementation is for x86_64 and is based on the prototyping work of Rickard B?ckman. So big thanks to Rickard, who will co-author of this patch. The way it works is that there is a cmp #offset(r15_thread), simm32_epoch); je #continuation; inserted into the verified entry of nmethods that conditionally calls a stub trampolining into the VM. When the simm32_epoch is not equal to some TLS-local field, then a slow path is taken to run some barrier inside of the VM. By changing this TLS-local field in the safepoint, all verified entries will take the slow path into the VM when run, where the barrier is eventually disarmed by patching the simm32_epoch value of the instruction stream to the current globally correct value. In terms of abstractions, to utilize this mechanism, you need to create your BarierSet with an BarrierSetNMethod helper object, just like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The BarrierSetNMethod base class handles the barrier itself. It has a virtual function, nmethod_entry_barrier, which gets called by the slow paths. So each user of this mechanism inherits from BarrierSetNMethod and puts its barrier in there. The OSR nmethods do not have a verified entry. Therefore, the barrier is run explicitly in the runtime before calling into OSR nmethods to achieve the same effect of trapping new calls into nmethods. The nmethod entry barrier returns true/false depending on whether the called nmethod may be entered or not. This allows lazily cleaning nmethod IC caches during unloading, which invokes the same helper method that is patched into the verified entry when made not entrant. Eventually, I would like to also use this mechanism to remove the nmethod hotness counter maintenance we today perform in safepoint cleanup instead. These nmethod entry barriers can provide better heuristics for whether nmethods are used or not. But that would require buy-in from maintainers of other platforms. So let's take one step at a time. For the curious reader thinking that this might be an expensive operation, it is not. This check in the nmethod verified entry is very predictable and cheap. And because of use of inlining, the checks become more infrequent. In my evaluation, it has gone completely under the radar in benchmarking (running this with my current version of concurrent class unloading in ZGC). Webrev: http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ Bug ID: https://bugs.openjdk.java.net/browse/JDK-8210498 Thanks, /Erik From coleen.phillimore at oracle.com Fri Oct 5 15:08:09 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 5 Oct 2018 11:08:09 -0400 Subject: RFR (S) 8209889: RedefineStress tests crash In-Reply-To: <2e5bf818b4d6491f901841a87c19d298@sap.com> References: <68a26e6e6e574f75aec64ede7487a117@sap.com> <7b92e8ff-7dea-1130-b768-874faf2794dc@oracle.com> <7f40e04ade1c463cad48a056115e624d@sap.com> <2e5bf818b4d6491f901841a87c19d298@sap.com> Message-ID: Thank you! Coleen On 10/5/18 10:56 AM, Doerr, Martin wrote: > Hi Coleen, > > thank you for the backport. Looks good, too. > > Best regards, > Martin > > > -----Original Message----- > From: coleen.phillimore at oracle.com > Sent: Freitag, 5. Oktober 2018 16:54 > To: Doerr, Martin ; hotspot-dev at openjdk.java.net > Subject: Re: RFR (S) 8209889: RedefineStress tests crash > > > Hi Martin, Thank you for the code review and suggestion for fix. > > I've created a backport issue: > > https://bugs.openjdk.java.net/browse/JDK-8211775 > > The backport doesn't merge automatically because of the removal of > CompilerThreadHintNoPreempt (which doesn't safepoint). > > http://cr.openjdk.java.net/~coleenp/8211775.01/webrev/ > > But it seems trivial.?? I'm running mach5 tests on this.?? Can you > review this too? > > thanks, > Coleen > > On 10/5/18 5:05 AM, Doerr, Martin wrote: >> Hi Coleen, >> >> thanks for fixing this issue. I appreciate it. Looks good. >> >> Do you think it can get backported to 11u? >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: hotspot-dev On Behalf Of coleen.phillimore at oracle.com >> Sent: Donnerstag, 4. Oktober 2018 19:58 >> To: hotspot-dev at openjdk.java.net >> Subject: Re: RFR (S) 8209889: RedefineStress tests crash >> >> >> Martin, Here's your version of the fix, freshly tested. >> http://cr.openjdk.java.net/~coleenp/8209889.02/webrev/index.html >> Thanks, >> Coleen >> >> On 10/4/18 12:09 PM, coleen.phillimore at oracle.com wrote: >>> On 10/4/18 10:34 AM, Doerr, Martin wrote: >>>> Hi Coleen, >>>> >>>> good catch. Looks good where you moved it. >>>> >>>> I couldn?t find a place for a NoSafepointVerifier, either. >>>> >>>> Maybe it?d be more clear if we replaced the ?continue? in the NULL >>>> case by an else case in the loop so the CompileTaskWrapper is the >>>> first thing after the NULL check (see below). >>>> >>> I see, thanks to the magic of patch, the >>> possibly_add_compiler_threads() is inside the 'else' clause where >>> there's a compile task on the queue.? Which makes perfect sense. It >>> seems like there should be a possibly_remove_compiler_thread() in the >>> 'if' case but I'll leave that to the next person to touch this code. >>> >>> Ok, I'll test this variant, which I also like. >>> thanks, >>> Coleen >>> >>>> Best regards, >>>> >>>> Martin >>>> >>>> --- a/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >>>> 14:03:13 2018 +0200 >>>> >>>> +++ b/src/hotspot/share/compiler/compileBroker.cpp????? Thu Oct 04 >>>> 16:29:45 2018 +0200 >>>> >>>> @@ -1781,30 +1781,31 @@ >>>> >>>> ?????????? return; // Stop this thread. >>>> >>>> ???????? } >>>> >>>> ?????? } >>>> >>>> -????? continue; >>>> >>>> -??? } >>>> >>>> - >>>> >>>> -??? if (UseDynamicNumberOfCompilerThreads) { >>>> >>>> -????? possibly_add_compiler_threads(); >>>> >>>> -??? } >>>> >>>> +??? } else { >>>> >>>> +????? // Assign the task to the current thread.? Mark this compilation >>>> >>>> +????? // thread as active for the profiler. >>>> >>>> +????? // CompileTaskWrapper also keeps the Method* from being >>>> deallocated if redefinition >>>> >>>> +????? // occurs after fetching the compile task off the queue. >>>> >>>> +????? CompileTaskWrapper ctw(task); >>>> >>>> +????? nmethodLocker result_handle;? // (handle for the nmethod >>>> produced by this task) >>>> >>>> + task->set_code_handle(&result_handle); >>>> >>>> +????? methodHandle method(thread, task->method()); >>>> >>>> -??? // Assign the task to the current thread. Mark this compilation >>>> >>>> -??? // thread as active for the profiler. >>>> >>>> -??? CompileTaskWrapper ctw(task); >>>> >>>> -??? nmethodLocker result_handle;? // (handle for the nmethod >>>> produced by this task) >>>> >>>> - task->set_code_handle(&result_handle); >>>> >>>> -??? methodHandle method(thread, task->method()); >>>> >>>> +????? // Never compile a method if breakpoints are present in it >>>> >>>> +????? if (method()->number_of_breakpoints() == 0) { >>>> >>>> +??????? // Compile the method. >>>> >>>> +??????? if ((UseCompiler || AlwaysCompileLoopMethods) && >>>> CompileBroker::should_compile_new_jobs()) { >>>> >>>> +????????? invoke_compiler_on_method(task); >>>> >>>> +????????? thread->start_idle_timer(); >>>> >>>> +???? ???} else { >>>> >>>> +????????? // After compilation is disabled, remove remaining methods >>>> from queue >>>> >>>> + method->clear_queued_for_compilation(); >>>> >>>> + task->set_failure_reason("compilation is disabled"); >>>> >>>> +??????? } >>>> >>>> +????? } >>>> >>>> -??? // Never compile a method if breakpoints are present in it >>>> >>>> -??? if (method()->number_of_breakpoints() == 0) { >>>> >>>> -????? // Compile the method. >>>> >>>> -????? if ((UseCompiler || AlwaysCompileLoopMethods) && >>>> CompileBroker::should_compile_new_jobs()) { >>>> >>>> -??????? invoke_compiler_on_method(task); >>>> >>>> -??????? thread->start_idle_timer(); >>>> >>>> -????? } else { >>>> >>>> -??????? // After compilation is disabled, remove remaining methods >>>> from queue >>>> >>>> - method->clear_queued_for_compilation(); >>>> >>>> - task->set_failure_reason("compilation is disabled"); >>>> >>>> +????? if (UseDynamicNumberOfCompilerThreads) { >>>> >>>> +??????? possibly_add_compiler_threads(); >>>> >>>> ?????? } >>>> >>>> ???? } >>>> >>>> ?? } >>>> >>>> *From:*coleen.phillimore at oracle.com >>>> *Sent:* Donnerstag, 4. Oktober 2018 13:45 >>>> *To:* hotspot-dev developers ; Doerr, >>>> Martin >>>> *Subject:* RFR (S) 8209889: RedefineStress tests crash >>>> >>>> Summary: Create CompileTaskWrapper before potential safepoint >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8209889.01/webrev >>>> >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8209889 >>>> >>>> MetadataOnStackMark needs to find the CompileTask in the current >>>> thread, or else the Method isn't marked as being alive, so if it's >>>> old/obsolete, it is freed.? There are checks to disable adding old >>>> methods to the compile queue but if the method is redefined after >>>> this point, there may be a preexisting bug that we compile an old >>>> method.? This change just fixes the crash. >>>> >>>> Can some compiler person (Martin?) look at where I moved >>>> possibly_add_compiler_threads and let me know if that looks okay? >>>> There's a comment in the change that says why you need to create the >>>> CompileTaskWrapper before that call.? I couldn't find a place to put >>>> a NoSafepointVerifier. >>>> >>>> Tested with the test that failed, which doesn't reproduce the bug >>>> except rarely, and test/hotspot/jtreg/compiler tests. tier1 an 2 in >>>> progress. >>>> >>>> Thanks, >>>> Coleen >>>> From jiangli.zhou at oracle.com Fri Oct 5 16:57:05 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 5 Oct 2018 09:57:05 -0700 Subject: RFR: JDK-8202951: Implementation of JEPJDK-8204247: Include default CDS (Class Data Sharing) archive in JDK binary In-Reply-To: <0f79551e-5f0a-b7f4-cc72-eb84ed8baa14@oracle.com> References: <8e17729f-9f5b-4d7c-a8cf-e0d3ffeeb694@oracle.com> <0f79551e-5f0a-b7f4-cc72-eb84ed8baa14@oracle.com> Message-ID: <37b2f736-88dc-f523-1551-0a87c3fb1a4b@oracle.com> Dear all, JEP 341, Default CDS Archive has been targeted to JDK 12 (please see Mark's email from yesterday, Re: JEP proposed to target JDK 12: 341: Default CDS Archives). Code review, testing and performance runs are completed for the default CDS archive changes. I've started the final pre-integration test yesterday with all tier1 - tier8. So far most of the tiers have been completed. Just a heads up, I'm planning to integrate the change in the next few hour. Thanks everyone for contributing to this effort!!! Jiangli On 9/6/18 12:45 PM, Jiangli Zhou wrote: > Hi, > > The JEP (http://openjdk.java.net/jeps/341) for the default CDS > archives is now a candidate. Please see Mark's email, 'New candidate > JEP: 341: Default CDS Archives'. Please help test Erik's makefile > patch for your platform if it is not built/tested in openjdk mainline > to prevent any possible breakage on your side. Any comments/feedbacks > on the default CDS archive are highly appreciated! > > ? http://cr.openjdk.java.net/~jiangli/8202951/webrev.02/ > > The above webrev is sync'ed with the lasted jdk/jdk repository today. > > Thanks! > > Jiangli > > On 8/30/18 11:26 AM, Jiangli Zhou wrote: >> Hi Magnus, >> >> Sounds good. Will update the message. >> >> Thanks! >> >> Jiangli >> >> >> On 8/30/18 3:56 AM, Magnus Ihse Bursie wrote: >>> On 2018-08-29 17:45, Erik Joelsson wrote: >>>> >>>> Hello Magnus, >>>> >>>> As the author of the build changes I will answer this. >>>> >>>> On 2018-08-29 04:53, Magnus Ihse Bursie wrote: >>>>> On 2018-08-28 18:25, Erik Joelsson wrote: >>>>>> Build changes look good to me (but should probably get review >>>>>> from someone else). >>>>> >>>>> I'm (as usually) not as happy as Erik. ;-) >>>>> >>>>> In Images.gmk, you have added this rule: >>>>> ????????? $@ -Xshare:dump -Xmx128M -Xms128M $(LOG_INFO) >>>>> >>>>> It took me a while to grasp. You are relying on >>>>> $(JIMAGE_TARGET_FILE) to evaluate to bin/java. But that is only >>>>> incidentally so. This file is just picked arbitrarily to represent >>>>> when the entire image is done, for make. Please use >>>>> $(JRE_IMAGE_DIR)/bin/java instead. >>>>> >>>> I can agree with this part. That was a bit of a hack done in a hurry. >>>>> The logic in jdk-options.m4 is broken. If indeed it is not >>>>> possible to use cds archive with zero, then things will break >>>>> since it will still be built if using --enable-cds-archive! >>>>> >>>>> What you should do is to set default to true unless using cross >>>>> compilation or zero. If the user explicitly sets >>>>> --enable-cds-archive, and it's not possible, a fatal error should >>>>> be generated. >>>>> >>>> Here I'm not as sure. I deliberately let it be possible to override >>>> the default behavior for zero as someone might want to fix that at >>>> some point, you never know. For cross compilation it's just not >>>> possible ever so that's different. Maybe my reasoning is invalid. >>> >>> I see. I still think it would have made more sense, in that case, to >>> set the default to false if using zero, but allow override. But I'm >>> not going to insist on that. >>> >>> However, if the problem with zero is not that it's technically >>> impossible, just that it's not tested, I think the message should be >>> changed from "[no, not possible with JVM variant zero]" to just >>> "[no]" with a comment in the configure script that it's off by >>> default since it's not tested. >>> >>> Apart from that, Jiangli's latest webrev looks good. >>> >>> Jiangli: If you fix it like I suggested, you do not need to respin >>> the webrev. >>> >>> /Magnus >>> >>>> >>>> /Erik >>>>> Apart from this, I'm more on Erik's line. :-) >>>>> >>>>> /Magnus >>>>> >>>>> >>>>> >>>>>> >>>>>> /Erik >>>>>> >>>>>> >>>>>> On 2018-08-27 13:33, Jiangli Zhou wrote: >>>>>>> Please review the implementation for JEP JDK-8204247 >>>>>>> (https://bugs.openjdk.java.net/browse/JDK-8204247). The goal of >>>>>>> the JEP is to include a default CDS archive in JDK 12 binary >>>>>>> distribution (downloadable from http://jdk.java.net/12/). The >>>>>>> default CDS archive is generated using the default classlist >>>>>>> (resides in the lib/ directory) at JDK build time. Any >>>>>>> comments/suggestions are highly appreciated. >>>>>>> >>>>>>> All makefile changes in the webrev are contributed by Erik >>>>>>> Joelsson (many thanks!!). >>>>>>> >>>>>>> This is a combination of efforts from different teams and >>>>>>> individuals. Thanks to everyone who has been involved in the JEP >>>>>>> & implementation discussions, testing and bug fixing! >>>>>>> >>>>>>> ? JEP: https://bugs.openjdk.java.net/browse/JDK-8204247 >>>>>>> ? RFE: https://bugs.openjdk.java.net/browse/JDK-8202951 >>>>>>> ? webrev: http://cr.openjdk.java.net/~jiangli/8202951/webrev.00/ >>>>>>> >>>>>>> Two sanity test cases for the default CDS archive are included >>>>>>> test/hotspot/jtreg/runtime/SharedArchiveFile. They are not >>>>>>> intended for in-depth CDS functional testing, which is already >>>>>>> covered by the existing cds/appcds tests and all tiered tests >>>>>>> executing with the default CDS archive enabled. >>>>>>> >>>>>>> As part of the webrev, >>>>>>> test/jdk/javax/imageio/plugins/png/ItxtUtf8Test.java is also >>>>>>> fixed to use larger java heap (JDK-8209739 >>>>>>> , https://bugs.openjdk.java.net/browse/JDK-8209739). >>>>>>> >>>>>>> Tests executed: >>>>>>> ?- several rounds of tier1 - tier8 via mach5 >>>>>>> ?- JCK lang, api and vm tests via mach5 >>>>>>> >>>>>>> >>>>>>> Thanks! >>>>>>> Calvin, Ioi, Jiangli >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Sat Oct 6 00:18:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 6 Oct 2018 10:18:13 +1000 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> Message-ID: <6c34f766-eb48-cd18-1d0d-c202b0b4052d@oracle.com> On 6/10/2018 12:18 AM, Andrew Haley wrote: > On 10/05/2018 02:40 PM, Severin Gehwolf wrote: >>> Also I thought that if ldrex was not paired with a strex then you should >>> at least issue clrex - as done in generate_atomic_load_long() in >>> stubGenerator_arm.cpp >> >> I'll defer to Andrew Haley for this. > > I'm skeptical that CLREX is necessary. The classic CAS for Arm is > > MOV R1, #0xFF ; load the ?lock taken? value > try LDREX R0, [LockAddr] ; load the lock value > CMP R0, #0 ; is the lock free? > STREXEQ R1, R0, [LockAddr]; try and claim the lock > CMPEQ R0, #0 ; did this succeed? > BNE try ; no ? try again. . . > ; yes ? we have the lock > > There's no CLREX, and no-one ever reported any problem. CLREX could be > a performance boost because it takes the cache line out of exclusive > state back to shared state, reducing later bus traffic. I would see CLREX as a potential performance boost not something needed for correctness. But you're right that failure paths from typical CAS code won't execute the strex and don't include a clrex. David From yumin.qi at gmail.com Sat Oct 6 00:54:29 2018 From: yumin.qi at gmail.com (yumin qi) Date: Fri, 5 Oct 2018 17:54:29 -0700 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: <34678f20-9cec-813a-0686-c5a2ff5b2316@oracle.com> References: <34678f20-9cec-813a-0686-c5a2ff5b2316@oracle.com> Message-ID: Vote: yes On Mon, Oct 1, 2018 at 1:39 PM wrote: > Vote: yes > > On 10/1/18 4:37 PM, coleen.phillimore at oracle.com wrote: > > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to > > Membership in the hotspot Group > > > > Mikael is managing the hotspot team at Oracle and has been working on > > JVMs (jrockit and hotspot) for almost 20 years, touching on everything > > from JIT compilers, operating system and CPU ports, JVMTI, a > > microkernel for Java, and more recently Project Panama. He is also the > > project lead for OpenJDK Project Portola. > > > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > > > Only current Members of the Members Group [1] are eligible to vote on > > this nomination. Votes must be cast in the open by replying to this > > mailing list. For Lazy Consensus voting instructions, see [2]. > > > > Coleen Phillimore > > > > [1] http://openjdk.java.net/census#hotspot > > [2] http://openjdk.java.net/groups/#member-vote > > From aleksei.voitylov at bell-sw.com Sat Oct 6 15:07:52 2018 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Sat, 6 Oct 2018 18:07:52 +0300 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> Message-ID: <10d8f2ea-883e-ec21-4fea-06a52a52fdc6@bell-sw.com> Kim, from an ARM port perspective, the stable GCC version is 4.9. The port compiles with stock GCC 7.3 but a lot of testing is required before moving to GCC 7.3. I agree on the overall direction and we'll commit resources to testing it further, but from an ARM port perspective it may happen JDK 12 is a little too optimistic. GCC x86 is a lot more stable than for ARM32 and AARCH64. -Aleksei On 05/10/2018 00:09, Kim Barrett wrote: >> On Oct 4, 2018, at 9:17 AM, Aleksei Voitylov wrote: >> >> Kim, >> >> Thank you for posting this. It's an important step to make. I wanted to clarify a couple of points: >> >> 1. Since this is infrastructure JEP, is the version of JDK which will undergo such changes going to be some future version or does it apply to past versions as well? I'm concerned with how we can simplify security backports to 8u which (I currently assume) is not subject to this change. > Future version (perhaps as soon as JDK 12). I don't think there is > any desire to backport this change. And yes, introducing the use of > new language features will make backports even more interesting than > they already are. That seems unavoidable if we're going to pursue > this direction. > >> 2. Which versions of GCC do you tentatively consider at this point? Non-x86 ports may have a problem upgrading to a specific version of GCC which the shared code will use and may need additional lead time to adjust. > I think our (ad hoc) testing has been limited to gcc 7.x. But looking > at the gcc language conformance tables and release notes, gcc 5.x > might be good enough, and gcc 6.x seems fairly likely sufficient. Of > course, older versions are more likely to have bugs in some of these > new features. Updating the minimum supported versions for gcc and > clang are noted as TBD in the JEP. > From dean.long at oracle.com Sat Oct 6 20:11:01 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sat, 6 Oct 2018 13:11:01 -0700 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: <6c34f766-eb48-cd18-1d0d-c202b0b4052d@oracle.com> References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> <6c34f766-eb48-cd18-1d0d-c202b0b4052d@oracle.com> Message-ID: <4fa520f3-1341-ade8-79a1-962c0d5d7f5f@oracle.com> On 10/5/18 5:18 PM, David Holmes wrote: > On 6/10/2018 12:18 AM, Andrew Haley wrote: >> On 10/05/2018 02:40 PM, Severin Gehwolf wrote: >>>> Also I thought that if ldrex was not paired with a strex then you >>>> should >>>> at least issue clrex - as done in generate_atomic_load_long() in >>>> stubGenerator_arm.cpp >>> >>> I'll defer to Andrew Haley for this. >> >> I'm skeptical that CLREX is necessary. The classic CAS for Arm is >> >> ???? MOV? R1, #0xFF???????????? ; load the ?lock taken? value >> try LDREX??? R0, [LockAddr]??? ; load the lock value >> ???? CMP R0,? #0??????????????? ; is the lock free? >> ???? STREXEQ? R1, R0, [LockAddr]; try and claim the lock >> ???? CMPEQ R0, #0?????????????? ; did this succeed? >> ???? BNE try??????????????????? ; no ? try again. . . >> ; yes ? we have the lock >> >> There's no CLREX, and no-one ever reported any problem. CLREX could be >> a performance boost because it takes the cache line out of exclusive >> state back to shared state, reducing later bus traffic. > > I would see CLREX as a potential performance boost not something > needed for correctness. But you're right that failure paths from > typical CAS code won't execute the strex and don't include a clrex. > I think it's needed for correctness only at context-switch time.? If you context switch after the LDREX, you don't want the exclusive state to remain for the next thread scheduled. dl > David From david.holmes at oracle.com Sat Oct 6 21:52:20 2018 From: david.holmes at oracle.com (David Holmes) Date: Sun, 7 Oct 2018 07:52:20 +1000 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: <4fa520f3-1341-ade8-79a1-962c0d5d7f5f@oracle.com> References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> <6c34f766-eb48-cd18-1d0d-c202b0b4052d@oracle.com> <4fa520f3-1341-ade8-79a1-962c0d5d7f5f@oracle.com> Message-ID: <6b1c457d-321b-f8a9-4b3a-44c79e0e8987@oracle.com> On 7/10/2018 6:11 AM, dean.long at oracle.com wrote: > On 10/5/18 5:18 PM, David Holmes wrote: >> On 6/10/2018 12:18 AM, Andrew Haley wrote: >>> On 10/05/2018 02:40 PM, Severin Gehwolf wrote: >>>>> Also I thought that if ldrex was not paired with a strex then you >>>>> should >>>>> at least issue clrex - as done in generate_atomic_load_long() in >>>>> stubGenerator_arm.cpp >>>> >>>> I'll defer to Andrew Haley for this. >>> >>> I'm skeptical that CLREX is necessary. The classic CAS for Arm is >>> >>> ???? MOV? R1, #0xFF???????????? ; load the ?lock taken? value >>> try LDREX??? R0, [LockAddr]??? ; load the lock value >>> ???? CMP R0,? #0??????????????? ; is the lock free? >>> ???? STREXEQ? R1, R0, [LockAddr]; try and claim the lock >>> ???? CMPEQ R0, #0?????????????? ; did this succeed? >>> ???? BNE try??????????????????? ; no ? try again. . . >>> ; yes ? we have the lock >>> >>> There's no CLREX, and no-one ever reported any problem. CLREX could be >>> a performance boost because it takes the cache line out of exclusive >>> state back to shared state, reducing later bus traffic. >> >> I would see CLREX as a potential performance boost not something >> needed for correctness. But you're right that failure paths from >> typical CAS code won't execute the strex and don't include a clrex. >> > > I think it's needed for correctness only at context-switch time.? If you > context switch after the LDREX, you don't want the exclusive state to > remain for the next thread scheduled. See also: https://community.arm.com/processors/f/discussions/6572/synchronization-primitives-do-i-need-clrex Seems if CLREX is needed for correctness it is at the OS level. On performance concern see: http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20150921/300951.html Bottom line seems to be: - clrex is not essential - clrex is probably not harmful to performance - clrex may benefit performance David > dl > >> David > From bsrbnd at gmail.com Sat Oct 6 22:05:58 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 7 Oct 2018 00:05:58 +0200 Subject: Primitive boolean array packing Message-ID: Hi, Per JVMS, primitive boolean arrays are currently using 8 bits (one byte) per boolean value, see [1] (baload/bastore). We've see in some threads [2] that 'BitSet' or 'EnumSet' are occasionally preferred solutions as they are using boolean vectors (aka long values) using 8x less memory. But as the spec [1] allows primitive boolean array packing, I made a quick experiment which is available here: http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.00/ This is a partial draft implementing primitive boolean array packing (8x smaller) at machine-level (vs BitSet at class-level): * only x86 interpreter implemented => c1/c2/inlining disabled for methods using baload/bastore (not much performance regression when building the jdk and running the tests) * unsafe access partially implemented * array copy currently not implemented * -XX:+/-BooleanPacking to enable/disable the feature currently not implemented A couple of residual tier1 tests are still failing due to the unfinished implementation. I didn't search much if such experiments have already been accomplished, but I'd like to take the temperature of this feature as completing the implementation represents a significant amount of work. Is this something that is worth exploring? Thanks in advance for any feedback. Regards, Bernard [1] https://docs.oracle.com/javase/specs/jvms/se11/html/jvms-6.html#jvms-6.5.baload [2] http://mail.openjdk.java.net/pipermail/compiler-dev/2018-September/012504.html From shade at redhat.com Sun Oct 7 08:01:47 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sun, 7 Oct 2018 10:01:47 +0200 Subject: Primitive boolean array packing In-Reply-To: References: Message-ID: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> On 10/07/2018 12:05 AM, B. Blaser wrote: > I didn't search much if such experiments have already been > accomplished, but I'd like to take the temperature of this feature as > completing the implementation represents a significant amount of work. > Is this something that is worth exploring? The most problematic part, in my mind, is that JMM requires the absence of word tearing for array element accesses. See here: https://shipilev.net/blog/2014/jmm-pragmatics/#_part_ii_word_tearing With densely packed boolean[], you'd need to convert plain stores to locked/CAS-loops to support this, I think, which raises all sorts of performance questions. Choosing between boolean[] and BitSet is basically choosing between stricter/relaxed concurrency guarantees vs dense/sparse footprint. -Aleksey From aph at redhat.com Sun Oct 7 11:47:45 2018 From: aph at redhat.com (Andrew Haley) Date: Sun, 7 Oct 2018 12:47:45 +0100 Subject: Primitive boolean array packing In-Reply-To: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> Message-ID: <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> On 10/07/2018 09:01 AM, Aleksey Shipilev wrote: > On 10/07/2018 12:05 AM, B. Blaser wrote: >> I didn't search much if such experiments have already been >> accomplished, but I'd like to take the temperature of this feature as >> completing the implementation represents a significant amount of work. >> Is this something that is worth exploring? > > The most problematic part, in my mind, is that JMM requires the absence of word tearing for array > element accesses. See here: https://shipilev.net/blog/2014/jmm-pragmatics/#_part_ii_word_tearing > > With densely packed boolean[], you'd need to convert plain stores to locked/CAS-loops to support > this, I think, which raises all sorts of performance questions. Choosing between boolean[] and > BitSet is basically choosing between stricter/relaxed concurrency guarantees vs dense/sparse footprint. But you get a lot of performance conflicts between cores having to share cache lines anyway. Maybe we should do some performance experiments: we wouldn't need to do all of the C2 work, just write a little C++ and asm code and measure what happens under some plausible conditions. We'd have to try both contended and uncontended situations. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rkennke at redhat.com Sun Oct 7 11:51:38 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 7 Oct 2018 13:51:38 +0200 Subject: Primitive boolean array packing In-Reply-To: <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: >>> I didn't search much if such experiments have already been >>> accomplished, but I'd like to take the temperature of this feature as >>> completing the implementation represents a significant amount of work. >>> Is this something that is worth exploring? >> >> The most problematic part, in my mind, is that JMM requires the absence of word tearing for array >> element accesses. See here: https://shipilev.net/blog/2014/jmm-pragmatics/#_part_ii_word_tearing >> >> With densely packed boolean[], you'd need to convert plain stores to locked/CAS-loops to support >> this, I think, which raises all sorts of performance questions. Choosing between boolean[] and >> BitSet is basically choosing between stricter/relaxed concurrency guarantees vs dense/sparse footprint. > > But you get a lot of performance conflicts between cores having to share > cache lines anyway. Maybe we should do some performance experiments: we > wouldn't need to do all of the C2 work, just write a little C++ and asm > code and measure what happens under some plausible conditions. We'd have > to try both contended and uncontended situations. Are boolean arrays even common enough to make a difference? Roman From bsrbnd at gmail.com Sun Oct 7 12:38:21 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 7 Oct 2018 14:38:21 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: On Sun, 7 Oct 2018 at 13:51, Roman Kennke wrote: > > >>> I didn't search much if such experiments have already been > >>> accomplished, but I'd like to take the temperature of this feature as > >>> completing the implementation represents a significant amount of work. > >>> Is this something that is worth exploring? > >> > >> The most problematic part, in my mind, is that JMM requires the absence of word tearing for array > >> element accesses. See here: https://shipilev.net/blog/2014/jmm-pragmatics/#_part_ii_word_tearing > >> > >> With densely packed boolean[], you'd need to convert plain stores to locked/CAS-loops to support > >> this, I think, which raises all sorts of performance questions. Choosing between boolean[] and > >> BitSet is basically choosing between stricter/relaxed concurrency guarantees vs dense/sparse footprint. > > > > But you get a lot of performance conflicts between cores having to share > > cache lines anyway. Maybe we should do some performance experiments: we > > wouldn't need to do all of the C2 work, just write a little C++ and asm > > code and measure what happens under some plausible conditions. We'd have > > to try both contended and uncontended situations. > > Are boolean arrays even common enough to make a difference? > > Roman Thanks Aleksey, you're absolutely right but a programmer can still disable this feature and use regular boolean arrays if necessary. Nevertheless, if the memory consumption is a priority and boolean packing a necessity, the problem you mentioned has two solutions: 1) either the JVM could lock distinct boolean elements per 8-bit block to preserve synchronized data, in which case some profiling would be necessary as Andrew suggested 2) or the JVM could no more guarantee concurrent access on distinct boolean array elements if packing is enabled delegating the synchronization to the programmer. While the first solution is safer, the second one minimizes the additional performance cost with a better synchronization focusing which maybe addresses Roman's question... Bernard From shade at redhat.com Sun Oct 7 15:13:11 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sun, 7 Oct 2018 17:13:11 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: On 10/07/2018 02:38 PM, B. Blaser wrote: > Thanks Aleksey, you're absolutely right but a programmer can still > disable this feature and use regular boolean arrays if necessary. I fully expect the JVM feature/flag what optionally breaks the Java specification will be flat out rejected. You can have the optional feature that makes the behavior tighter than required by spec, but it is a hard stop to make the behaviors weaker than required by spec. So, the work on packed boolean arrays has to demonstrate good performance with along with work tearing guarantees. On 10/07/2018 01:47 PM, Andrew Haley wrote: > But you get a lot of performance conflicts between cores having to share > cache lines anyway. Maybe we should do some performance experiments: we > wouldn't need to do all of the C2 work, just write a little C++ and asm > code and measure what happens under some plausible conditions. We'd have > to try both contended and uncontended situations. My educated guess from the subword atomic operations over VarHandles: the local latency of uncontended CAS would still be too significant to ignore. Over years, we hoped CAS overhead would become negligible, but it still isn't (see Biased Locking as another example that depends on this). -Aleksey From aph at redhat.com Sun Oct 7 17:51:20 2018 From: aph at redhat.com (Andrew Haley) Date: Sun, 7 Oct 2018 18:51:20 +0100 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: On 10/07/2018 04:13 PM, Aleksey Shipilev wrote: > On 10/07/2018 02:38 PM, B. Blaser wrote: >> Thanks Aleksey, you're absolutely right but a programmer can still >> disable this feature and use regular boolean arrays if necessary. > > I fully expect the JVM feature/flag what optionally breaks the Java > specification will be flat out rejected. It doesn't break the Java specification: Notes The baload instruction is used to load values from both byte and boolean arrays. In Oracle's Java Virtual Machine implementation, boolean arrays - that is, arrays of type T_BOOLEAN (?2.2, ?newarray) - are implemented as arrays of 8-bit values. Other implementations may implement packed boolean arrays; the baload instruction of such implementations must be used to access those arrays. > My educated guess from the subword atomic operations over > VarHandles: the local latency of uncontended CAS would still be too > significant to ignore. Over years, we hoped CAS overhead would > become negligible, but it still isn't Probably so. Numbers would be nice. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rkennke at redhat.com Sun Oct 7 18:29:33 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 7 Oct 2018 20:29:33 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: IIRC, some CPUs even have instructions for directly getting/setting bits, without explicit mask/shift. AArch64 can do that I think. Not sure if it's more efficient though. Roman > On 10/07/2018 04:13 PM, Aleksey Shipilev wrote: >> On 10/07/2018 02:38 PM, B. Blaser wrote: >>> Thanks Aleksey, you're absolutely right but a programmer can still >>> disable this feature and use regular boolean arrays if necessary. >> >> I fully expect the JVM feature/flag what optionally breaks the Java >> specification will be flat out rejected. > > It doesn't break the Java specification: > > Notes > > The baload instruction is used to load values from both byte and > boolean arrays. In Oracle's Java Virtual Machine implementation, > boolean arrays - that is, arrays of type T_BOOLEAN (?2.2, > ?newarray) - are implemented as arrays of 8-bit values. Other > implementations may implement packed boolean arrays; the baload > instruction of such implementations must be used to access those > arrays. > >> My educated guess from the subword atomic operations over >> VarHandles: the local latency of uncontended CAS would still be too >> significant to ignore. Over years, we hoped CAS overhead would >> become negligible, but it still isn't > > Probably so. Numbers would be nice. > From shade at redhat.com Sun Oct 7 19:54:00 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sun, 7 Oct 2018 21:54:00 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> On 10/07/2018 07:51 PM, Andrew Haley wrote: > On 10/07/2018 04:13 PM, Aleksey Shipilev wrote: >> On 10/07/2018 02:38 PM, B. Blaser wrote: >>> Thanks Aleksey, you're absolutely right but a programmer can still >>> disable this feature and use regular boolean arrays if necessary. >> >> I fully expect the JVM feature/flag what optionally breaks the Java >> specification will be flat out rejected. > > It doesn't break the Java specification: > > Notes > > The baload instruction is used to load values from both byte and > boolean arrays. In Oracle's Java Virtual Machine implementation, > boolean arrays - that is, arrays of type T_BOOLEAN (?2.2, > ?newarray) - are implemented as arrays of 8-bit values. Other > implementations may implement packed boolean arrays; the baload > instruction of such implementations must be used to access those > arrays. You need to specify what do you mean by "it". Current prototype sure does break the provisions of JLS 17.6 "Word Tearing": https://docs.oracle.com/javase/specs/jls/se11/html/jls-17.html#jls-17.6 My comment was about generic thing: we cannot break the Java spec even if we have the JVM flag that makes it right again. I thought that what B. was suggesting as the escape hatch. Of course, if you do packed boolean[] with locked/CAS-ed writes or otherwise guarantee the absence of word tearing, it does not break the spec. -Aleksey From per.liden at oracle.com Mon Oct 8 06:13:06 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 8 Oct 2018 08:13:06 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <5BB77E18.6040401@oracle.com> References: <5BB77E18.6040401@oracle.com> Message-ID: <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> On 2018-10-05 17:07, Erik ?sterlund wrote: > Hi, > > In order to implement concurrent class unloading for ZGC, a mechanism > will be required to arm nmethods in a safepoint such that the first > entry into an nmethod after the pause triggers a barrier before continuing. > > The barrier will be used for: > ? * Patching immediate oops, and keeping phantomly reachable oops alive > through the nmethod entry > ? * Catching calls from stale IC caches (due to class unloading) into > nmethods that are being unloaded (containing broken oops and metadata), > and lazily clean the IC cache and re-resolve the call. > > This implementation is for x86_64 and is based on the prototyping work > of Rickard B?ckman. So big thanks to Rickard, who will co-author of this > patch. > The way it works is that there is a cmp #offset(r15_thread), > simm32_epoch); je #continuation; inserted into the verified entry of > nmethods that conditionally calls a stub trampolining into the VM. > When the simm32_epoch is not equal to some TLS-local field, then a slow > path is taken to run some barrier inside of the VM. > By changing this TLS-local field in the safepoint, all verified entries > will take the slow path into the VM when run, where the barrier is > eventually disarmed by patching the simm32_epoch value of the > instruction stream to the current globally correct value. > > In terms of abstractions, to utilize this mechanism, you need to create > your BarierSet with an BarrierSetNMethod helper object, just like the > BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The > BarrierSetNMethod base class handles the barrier itself. It has a > virtual function, nmethod_entry_barrier, which gets called by the slow > paths. So each user of this mechanism inherits from BarrierSetNMethod > and puts its barrier in there. > > The OSR nmethods do not have a verified entry. Therefore, the barrier is > run explicitly in the runtime before calling into OSR nmethods to > achieve the same effect of trapping new calls into nmethods. > > The nmethod entry barrier returns true/false depending on whether the > called nmethod may be entered or not. This allows lazily cleaning > nmethod IC caches during unloading, which invokes the same helper method > that is patched into the verified entry when made not entrant. > > Eventually, I would like to also use this mechanism to remove the > nmethod hotness counter maintenance we today perform in safepoint > cleanup instead. These nmethod entry barriers can provide better > heuristics for whether nmethods are used or not. But that would require > buy-in from maintainers of other platforms. So let's take one step at a > time. > > For the curious reader thinking that this might be an expensive > operation, it is not. This check in the nmethod verified entry is very > predictable and cheap. And because of use of inlining, the checks become > more infrequent. In my evaluation, it has gone completely under the > radar in benchmarking (running this with my current version of > concurrent class unloading in ZGC). > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ I reviewed this before Erik sent it out, so my comments have already been addressed. Just found one additional thing, the BarrierSet constructor now looks like this: 96 BarrierSet(BarrierSetAssembler* barrier_set_assembler, 97 BarrierSetC1* barrier_set_c1, 98 BarrierSetC2* barrier_set_c2, 99 const FakeRtti& fake_rtti, 100 BarrierSetNMethod* barrier_set_nmethod = NULL) : 101 _fake_rtti(fake_rtti), 102 _barrier_set_assembler(barrier_set_assembler), 103 _barrier_set_c1(barrier_set_c1), 104 _barrier_set_c2(barrier_set_c2), 105 _barrier_set_nmethod(barrier_set_nmethod) {} I find it odd that the fake_rtti argument comes in the middle of the barrier_set_* arguments. I can see that you don't want GC barrier sets to have to supply NULL if they don't have a nmethod barrier set, but maybe we can have two constructors instead, like: BarrierSet(BarrierSetAssembler* barrier_set_assembler, BarrierSetC1* barrier_set_c1, BarrierSetC2* barrier_set_c2, const FakeRtti& fake_rtti) : _fake_rtti(fake_rtti), _barrier_set_assembler(barrier_set_assembler), _barrier_set_c1(barrier_set_c1), _barrier_set_c2(barrier_set_c2), _barrier_set_nmethod(NULL) {} BarrierSet(BarrierSetAssembler* barrier_set_assembler, BarrierSetC1* barrier_set_c1, BarrierSetC2* barrier_set_c2, BarrierSetNMethod* barrier_set_nmethod, const FakeRtti& fake_rtti,) : _fake_rtti(fake_rtti), _barrier_set_assembler(barrier_set_assembler), _barrier_set_c1(barrier_set_c1), _barrier_set_c2(barrier_set_c2), _barrier_set_nmethod(barrier_set_nmethod) {} What do you think? Other than that, this looks good to me. cheers, Per > > Bug ID: > https://bugs.openjdk.java.net/browse/JDK-8210498 > > Thanks, > /Erik From per.liden at oracle.com Mon Oct 8 06:14:16 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 8 Oct 2018 08:14:16 +0200 Subject: CFV: New hotspot Group Member: Mikael Vidstedt In-Reply-To: References: Message-ID: <466524bd-10c2-38a6-c9dd-58d39dd45d23@oracle.com> Vote: yes /Per On 2018-10-01 22:37, coleen.phillimore at oracle.com wrote: > I hereby nominate Mikael Vidstedt (OpenJDK user name: mikael) to > Membership in the hotspot Group > > Mikael is managing the hotspot team at Oracle and has been working on > JVMs (jrockit and hotspot) for almost 20 years, touching on everything > from JIT compilers, operating system and CPU ports, JVMTI, a microkernel > for Java, and more recently Project Panama. He is also the project lead > for OpenJDK Project Portola. > > Votes are due by Monday, October 15th, 2018, 16:37 US Eastern time. > > Only current Members of the Members Group [1] are eligible to vote on > this nomination.? Votes must be cast in the open by replying to this > mailing list. For Lazy Consensus voting instructions, see [2]. > > Coleen Phillimore > > [1] http://openjdk.java.net/census#hotspot > [2] http://openjdk.java.net/groups/#member-vote From rkennke at redhat.com Mon Oct 8 06:48:28 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 8 Oct 2018 08:48:28 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> Message-ID: I agree. I wouldn't mind to keep it simple and pass NULL though. I'm not a big fan of default arguments anyway. Roman > On 2018-10-05 17:07, Erik ?sterlund wrote: >> Hi, >> >> In order to implement concurrent class unloading for ZGC, a mechanism >> will be required to arm nmethods in a safepoint such that the first >> entry into an nmethod after the pause triggers a barrier before >> continuing. >> >> The barrier will be used for: >> ?? * Patching immediate oops, and keeping phantomly reachable oops >> alive through the nmethod entry >> ?? * Catching calls from stale IC caches (due to class unloading) into >> nmethods that are being unloaded (containing broken oops and >> metadata), and lazily clean the IC cache and re-resolve the call. >> >> This implementation is for x86_64 and is based on the prototyping work >> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >> this patch. >> The way it works is that there is a cmp #offset(r15_thread), >> simm32_epoch); je #continuation; inserted into the verified entry of >> nmethods that conditionally calls a stub trampolining into the VM. >> When the simm32_epoch is not equal to some TLS-local field, then a >> slow path is taken to run some barrier inside of the VM. >> By changing this TLS-local field in the safepoint, all verified >> entries will take the slow path into the VM when run, where the >> barrier is eventually disarmed by patching the simm32_epoch value of >> the instruction stream to the current globally correct value. >> >> In terms of abstractions, to utilize this mechanism, you need to >> create your BarierSet with an BarrierSetNMethod helper object, just >> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >> BarrierSetNMethod base class handles the barrier itself. It has a >> virtual function, nmethod_entry_barrier, which gets called by the slow >> paths. So each user of this mechanism inherits from BarrierSetNMethod >> and puts its barrier in there. >> >> The OSR nmethods do not have a verified entry. Therefore, the barrier >> is run explicitly in the runtime before calling into OSR nmethods to >> achieve the same effect of trapping new calls into nmethods. >> >> The nmethod entry barrier returns true/false depending on whether the >> called nmethod may be entered or not. This allows lazily cleaning >> nmethod IC caches during unloading, which invokes the same helper >> method that is patched into the verified entry when made not entrant. >> >> Eventually, I would like to also use this mechanism to remove the >> nmethod hotness counter maintenance we today perform in safepoint >> cleanup instead. These nmethod entry barriers can provide better >> heuristics for whether nmethods are used or not. But that would >> require buy-in from maintainers of other platforms. So let's take one >> step at a time. >> >> For the curious reader thinking that this might be an expensive >> operation, it is not. This check in the nmethod verified entry is very >> predictable and cheap. And because of use of inlining, the checks >> become more infrequent. In my evaluation, it has gone completely under >> the radar in benchmarking (running this with my current version of >> concurrent class unloading in ZGC). >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ > > I reviewed this before Erik sent it out, so my comments have already > been addressed. Just found one additional thing, the BarrierSet > constructor now looks like this: > > 96?? BarrierSet(BarrierSetAssembler* barrier_set_assembler, > 97????????????? BarrierSetC1* barrier_set_c1, > 98????????????? BarrierSetC2* barrier_set_c2, > 99????????????? const FakeRtti& fake_rtti, > 100???????????? BarrierSetNMethod* barrier_set_nmethod = NULL) : > 101???? _fake_rtti(fake_rtti), > 102???? _barrier_set_assembler(barrier_set_assembler), > 103???? _barrier_set_c1(barrier_set_c1), > 104???? _barrier_set_c2(barrier_set_c2), > 105???? _barrier_set_nmethod(barrier_set_nmethod) {} > > I find it odd that the fake_rtti argument comes in the middle of the > barrier_set_* arguments. I can see that you don't want GC barrier sets > to have to supply NULL if they don't have a nmethod barrier set, but > maybe we can have two constructors instead, like: > > BarrierSet(BarrierSetAssembler* barrier_set_assembler, > ?????????? BarrierSetC1* barrier_set_c1, > ?????????? BarrierSetC2* barrier_set_c2, > ?????????? const FakeRtti& fake_rtti) : > ??? _fake_rtti(fake_rtti), > ??? _barrier_set_assembler(barrier_set_assembler), > ??? _barrier_set_c1(barrier_set_c1), > ??? _barrier_set_c2(barrier_set_c2), > ??? _barrier_set_nmethod(NULL) {} > > BarrierSet(BarrierSetAssembler* barrier_set_assembler, > ?????????? BarrierSetC1* barrier_set_c1, > ?????????? BarrierSetC2* barrier_set_c2, > ?????????? BarrierSetNMethod* barrier_set_nmethod, > ?????????? const FakeRtti& fake_rtti,) : > ??? _fake_rtti(fake_rtti), > ??? _barrier_set_assembler(barrier_set_assembler), > ??? _barrier_set_c1(barrier_set_c1), > ??? _barrier_set_c2(barrier_set_c2), > ??? _barrier_set_nmethod(barrier_set_nmethod) {} > > What do you think? > > Other than that, this looks good to me. > > cheers, > Per > >> >> Bug ID: >> https://bugs.openjdk.java.net/browse/JDK-8210498 >> >> Thanks, >> /Erik From christoph.langer at sap.com Mon Oct 8 07:04:39 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 8 Oct 2018 07:04:39 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Message-ID: Hi, > I'm hoping others will chime in here as I: > > a) don't know if this information is actually useful for an error log of > this kind; I remember quite a few times when I was working on a customer ticket and found this information very helpful (we've already had this for quite a while in our SAP JVM). > b) don't know if the information might be considered sensitive or not; and I guess this is kind of sensitive information, for sure. However, it's probably not more sensitive than lots of other information you'll find in a hs-err file, e.g. register and memory contents, paths etc. So in my opinion we should not be overly concerned with this point. > c) don't think it's worth the effort of adding a flag to control this. > Plus the flag is only useful for trying to reproduce an issue; if it's a > one-of failure then you've already missed out on the information in the > log file. I'd prefer if we could get this piece of information in unconditionally. But if it's the only way to introduce this behind a flag, then so be it. (We'd probably enable the flag by default for our SapMachine OpenJDK builds that we'll need to support.) Best regards Christoph From erik.osterlund at oracle.com Mon Oct 8 08:19:52 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 8 Oct 2018 10:19:52 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> Message-ID: <5BBB1328.3020503@oracle.com> Hi Per and Roman, Thank you for having a look at this. I went with Roman's suggestion and made this an explicit opt-out instead. But I let ModRef pass in NULL, as none of the barrier sets in that category currently use nmethod entry barriers. When they start to do so, that can be easily changed. Incremental: http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ Full: http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ Thanks, /Erik On 2018-10-08 08:48, Roman Kennke wrote: > I agree. I wouldn't mind to keep it simple and pass NULL though. I'm not > a big fan of default arguments anyway. > > Roman > >> On 2018-10-05 17:07, Erik ?sterlund wrote: >>> Hi, >>> >>> In order to implement concurrent class unloading for ZGC, a mechanism >>> will be required to arm nmethods in a safepoint such that the first >>> entry into an nmethod after the pause triggers a barrier before >>> continuing. >>> >>> The barrier will be used for: >>> * Patching immediate oops, and keeping phantomly reachable oops >>> alive through the nmethod entry >>> * Catching calls from stale IC caches (due to class unloading) into >>> nmethods that are being unloaded (containing broken oops and >>> metadata), and lazily clean the IC cache and re-resolve the call. >>> >>> This implementation is for x86_64 and is based on the prototyping work >>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>> this patch. >>> The way it works is that there is a cmp #offset(r15_thread), >>> simm32_epoch); je #continuation; inserted into the verified entry of >>> nmethods that conditionally calls a stub trampolining into the VM. >>> When the simm32_epoch is not equal to some TLS-local field, then a >>> slow path is taken to run some barrier inside of the VM. >>> By changing this TLS-local field in the safepoint, all verified >>> entries will take the slow path into the VM when run, where the >>> barrier is eventually disarmed by patching the simm32_epoch value of >>> the instruction stream to the current globally correct value. >>> >>> In terms of abstractions, to utilize this mechanism, you need to >>> create your BarierSet with an BarrierSetNMethod helper object, just >>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>> BarrierSetNMethod base class handles the barrier itself. It has a >>> virtual function, nmethod_entry_barrier, which gets called by the slow >>> paths. So each user of this mechanism inherits from BarrierSetNMethod >>> and puts its barrier in there. >>> >>> The OSR nmethods do not have a verified entry. Therefore, the barrier >>> is run explicitly in the runtime before calling into OSR nmethods to >>> achieve the same effect of trapping new calls into nmethods. >>> >>> The nmethod entry barrier returns true/false depending on whether the >>> called nmethod may be entered or not. This allows lazily cleaning >>> nmethod IC caches during unloading, which invokes the same helper >>> method that is patched into the verified entry when made not entrant. >>> >>> Eventually, I would like to also use this mechanism to remove the >>> nmethod hotness counter maintenance we today perform in safepoint >>> cleanup instead. These nmethod entry barriers can provide better >>> heuristics for whether nmethods are used or not. But that would >>> require buy-in from maintainers of other platforms. So let's take one >>> step at a time. >>> >>> For the curious reader thinking that this might be an expensive >>> operation, it is not. This check in the nmethod verified entry is very >>> predictable and cheap. And because of use of inlining, the checks >>> become more infrequent. In my evaluation, it has gone completely under >>> the radar in benchmarking (running this with my current version of >>> concurrent class unloading in ZGC). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >> I reviewed this before Erik sent it out, so my comments have already >> been addressed. Just found one additional thing, the BarrierSet >> constructor now looks like this: >> >> 96 BarrierSet(BarrierSetAssembler* barrier_set_assembler, >> 97 BarrierSetC1* barrier_set_c1, >> 98 BarrierSetC2* barrier_set_c2, >> 99 const FakeRtti& fake_rtti, >> 100 BarrierSetNMethod* barrier_set_nmethod = NULL) : >> 101 _fake_rtti(fake_rtti), >> 102 _barrier_set_assembler(barrier_set_assembler), >> 103 _barrier_set_c1(barrier_set_c1), >> 104 _barrier_set_c2(barrier_set_c2), >> 105 _barrier_set_nmethod(barrier_set_nmethod) {} >> >> I find it odd that the fake_rtti argument comes in the middle of the >> barrier_set_* arguments. I can see that you don't want GC barrier sets >> to have to supply NULL if they don't have a nmethod barrier set, but >> maybe we can have two constructors instead, like: >> >> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >> BarrierSetC1* barrier_set_c1, >> BarrierSetC2* barrier_set_c2, >> const FakeRtti& fake_rtti) : >> _fake_rtti(fake_rtti), >> _barrier_set_assembler(barrier_set_assembler), >> _barrier_set_c1(barrier_set_c1), >> _barrier_set_c2(barrier_set_c2), >> _barrier_set_nmethod(NULL) {} >> >> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >> BarrierSetC1* barrier_set_c1, >> BarrierSetC2* barrier_set_c2, >> BarrierSetNMethod* barrier_set_nmethod, >> const FakeRtti& fake_rtti,) : >> _fake_rtti(fake_rtti), >> _barrier_set_assembler(barrier_set_assembler), >> _barrier_set_c1(barrier_set_c1), >> _barrier_set_c2(barrier_set_c2), >> _barrier_set_nmethod(barrier_set_nmethod) {} >> >> What do you think? >> >> Other than that, this looks good to me. >> >> cheers, >> Per >> >>> Bug ID: >>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>> >>> Thanks, >>> /Erik From bsrbnd at gmail.com Mon Oct 8 08:30:19 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 10:30:19 +0200 Subject: Primitive boolean array packing In-Reply-To: <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Sun, 7 Oct 2018 at 21:54, Aleksey Shipilev wrote: > > On 10/07/2018 07:51 PM, Andrew Haley wrote: > > On 10/07/2018 04:13 PM, Aleksey Shipilev wrote: > >> On 10/07/2018 02:38 PM, B. Blaser wrote: > >>> Thanks Aleksey, you're absolutely right but a programmer can still > >>> disable this feature and use regular boolean arrays if necessary. > >> > >> I fully expect the JVM feature/flag what optionally breaks the Java > >> specification will be flat out rejected. > > > > It doesn't break the Java specification: > > > > Notes > > > > The baload instruction is used to load values from both byte and > > boolean arrays. In Oracle's Java Virtual Machine implementation, > > boolean arrays - that is, arrays of type T_BOOLEAN (?2.2, > > ?newarray) - are implemented as arrays of 8-bit values. Other > > implementations may implement packed boolean arrays; the baload > > instruction of such implementations must be used to access those > > arrays. > > You need to specify what do you mean by "it". Current prototype sure does break the provisions of > JLS 17.6 "Word Tearing": > https://docs.oracle.com/javase/specs/jls/se11/html/jls-17.html#jls-17.6 > > My comment was about generic thing: we cannot break the Java spec even if we have the JVM flag that > makes it right again. I thought that what B. was suggesting as the escape hatch. Of course, if you > do packed boolean[] with locked/CAS-ed writes or otherwise guarantee the absence of word tearing, it > does not break the spec. > > -Aleksey You're undoubtedly right, I was focusing on the JVMS and I missed that point in the JLS... I'll try to put the prototype in sync with *all* the specs. Thanks, Bernard From per.liden at oracle.com Mon Oct 8 08:34:38 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 8 Oct 2018 10:34:38 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <5BBB1328.3020503@oracle.com> References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> <5BBB1328.3020503@oracle.com> Message-ID: <105725e2-55b9-9397-0e27-16316afc25d7@oracle.com> On 2018-10-08 10:19, Erik ?sterlund wrote: > Hi Per and Roman, > > Thank you for having a look at this. > I went with Roman's suggestion and made this an explicit opt-out > instead. But I let ModRef pass in NULL, as none of the barrier sets in > that category currently use nmethod entry barriers. When they start to > do so, that can be easily changed. Sounds good. > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ Looks good! cheers, Per > > Thanks, > /Erik > > On 2018-10-08 08:48, Roman Kennke wrote: >> I agree. I wouldn't mind to keep it simple and pass NULL though. I'm not >> a big fan of default arguments anyway. >> >> Roman >> >>> On 2018-10-05 17:07, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> In order to implement concurrent class unloading for ZGC, a mechanism >>>> will be required to arm nmethods in a safepoint such that the first >>>> entry into an nmethod after the pause triggers a barrier before >>>> continuing. >>>> >>>> The barrier will be used for: >>>> ??? * Patching immediate oops, and keeping phantomly reachable oops >>>> alive through the nmethod entry >>>> ??? * Catching calls from stale IC caches (due to class unloading) into >>>> nmethods that are being unloaded (containing broken oops and >>>> metadata), and lazily clean the IC cache and re-resolve the call. >>>> >>>> This implementation is for x86_64 and is based on the prototyping work >>>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>>> this patch. >>>> The way it works is that there is a cmp #offset(r15_thread), >>>> simm32_epoch); je #continuation; inserted into the verified entry of >>>> nmethods that conditionally calls a stub trampolining into the VM. >>>> When the simm32_epoch is not equal to some TLS-local field, then a >>>> slow path is taken to run some barrier inside of the VM. >>>> By changing this TLS-local field in the safepoint, all verified >>>> entries will take the slow path into the VM when run, where the >>>> barrier is eventually disarmed by patching the simm32_epoch value of >>>> the instruction stream to the current globally correct value. >>>> >>>> In terms of abstractions, to utilize this mechanism, you need to >>>> create your BarierSet with an BarrierSetNMethod helper object, just >>>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>>> BarrierSetNMethod base class handles the barrier itself. It has a >>>> virtual function, nmethod_entry_barrier, which gets called by the slow >>>> paths. So each user of this mechanism inherits from BarrierSetNMethod >>>> and puts its barrier in there. >>>> >>>> The OSR nmethods do not have a verified entry. Therefore, the barrier >>>> is run explicitly in the runtime before calling into OSR nmethods to >>>> achieve the same effect of trapping new calls into nmethods. >>>> >>>> The nmethod entry barrier returns true/false depending on whether the >>>> called nmethod may be entered or not. This allows lazily cleaning >>>> nmethod IC caches during unloading, which invokes the same helper >>>> method that is patched into the verified entry when made not entrant. >>>> >>>> Eventually, I would like to also use this mechanism to remove the >>>> nmethod hotness counter maintenance we today perform in safepoint >>>> cleanup instead. These nmethod entry barriers can provide better >>>> heuristics for whether nmethods are used or not. But that would >>>> require buy-in from maintainers of other platforms. So let's take one >>>> step at a time. >>>> >>>> For the curious reader thinking that this might be an expensive >>>> operation, it is not. This check in the nmethod verified entry is very >>>> predictable and cheap. And because of use of inlining, the checks >>>> become more infrequent. In my evaluation, it has gone completely under >>>> the radar in benchmarking (running this with my current version of >>>> concurrent class unloading in ZGC). >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >>> I reviewed this before Erik sent it out, so my comments have already >>> been addressed. Just found one additional thing, the BarrierSet >>> constructor now looks like this: >>> >>> 96?? BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>> 97????????????? BarrierSetC1* barrier_set_c1, >>> 98????????????? BarrierSetC2* barrier_set_c2, >>> 99????????????? const FakeRtti& fake_rtti, >>> 100???????????? BarrierSetNMethod* barrier_set_nmethod = NULL) : >>> 101???? _fake_rtti(fake_rtti), >>> 102???? _barrier_set_assembler(barrier_set_assembler), >>> 103???? _barrier_set_c1(barrier_set_c1), >>> 104???? _barrier_set_c2(barrier_set_c2), >>> 105???? _barrier_set_nmethod(barrier_set_nmethod) {} >>> >>> I find it odd that the fake_rtti argument comes in the middle of the >>> barrier_set_* arguments. I can see that you don't want GC barrier sets >>> to have to supply NULL if they don't have a nmethod barrier set, but >>> maybe we can have two constructors instead, like: >>> >>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>> ??????????? BarrierSetC1* barrier_set_c1, >>> ??????????? BarrierSetC2* barrier_set_c2, >>> ??????????? const FakeRtti& fake_rtti) : >>> ???? _fake_rtti(fake_rtti), >>> ???? _barrier_set_assembler(barrier_set_assembler), >>> ???? _barrier_set_c1(barrier_set_c1), >>> ???? _barrier_set_c2(barrier_set_c2), >>> ???? _barrier_set_nmethod(NULL) {} >>> >>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>> ??????????? BarrierSetC1* barrier_set_c1, >>> ??????????? BarrierSetC2* barrier_set_c2, >>> ??????????? BarrierSetNMethod* barrier_set_nmethod, >>> ??????????? const FakeRtti& fake_rtti,) : >>> ???? _fake_rtti(fake_rtti), >>> ???? _barrier_set_assembler(barrier_set_assembler), >>> ???? _barrier_set_c1(barrier_set_c1), >>> ???? _barrier_set_c2(barrier_set_c2), >>> ???? _barrier_set_nmethod(barrier_set_nmethod) {} >>> >>> What do you think? >>> >>> Other than that, this looks good to me. >>> >>> cheers, >>> Per >>> >>>> Bug ID: >>>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>>> >>>> Thanks, >>>> /Erik > From erik.osterlund at oracle.com Mon Oct 8 08:35:36 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 8 Oct 2018 10:35:36 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <105725e2-55b9-9397-0e27-16316afc25d7@oracle.com> References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> <5BBB1328.3020503@oracle.com> <105725e2-55b9-9397-0e27-16316afc25d7@oracle.com> Message-ID: <5BBB16D8.9060901@oracle.com> Hi Per, Thanks for the review! /Erik On 2018-10-08 10:34, Per Liden wrote: > > On 2018-10-08 10:19, Erik ?sterlund wrote: >> Hi Per and Roman, >> >> Thank you for having a look at this. >> I went with Roman's suggestion and made this an explicit opt-out >> instead. But I let ModRef pass in NULL, as none of the barrier sets >> in that category currently use nmethod entry barriers. When they >> start to do so, that can be easily changed. > > Sounds good. > >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ >> >> Full: >> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ > > Looks good! > > cheers, > Per > >> >> Thanks, >> /Erik >> >> On 2018-10-08 08:48, Roman Kennke wrote: >>> I agree. I wouldn't mind to keep it simple and pass NULL though. I'm >>> not >>> a big fan of default arguments anyway. >>> >>> Roman >>> >>>> On 2018-10-05 17:07, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> In order to implement concurrent class unloading for ZGC, a mechanism >>>>> will be required to arm nmethods in a safepoint such that the first >>>>> entry into an nmethod after the pause triggers a barrier before >>>>> continuing. >>>>> >>>>> The barrier will be used for: >>>>> * Patching immediate oops, and keeping phantomly reachable oops >>>>> alive through the nmethod entry >>>>> * Catching calls from stale IC caches (due to class unloading) >>>>> into >>>>> nmethods that are being unloaded (containing broken oops and >>>>> metadata), and lazily clean the IC cache and re-resolve the call. >>>>> >>>>> This implementation is for x86_64 and is based on the prototyping >>>>> work >>>>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>>>> this patch. >>>>> The way it works is that there is a cmp #offset(r15_thread), >>>>> simm32_epoch); je #continuation; inserted into the verified entry of >>>>> nmethods that conditionally calls a stub trampolining into the VM. >>>>> When the simm32_epoch is not equal to some TLS-local field, then a >>>>> slow path is taken to run some barrier inside of the VM. >>>>> By changing this TLS-local field in the safepoint, all verified >>>>> entries will take the slow path into the VM when run, where the >>>>> barrier is eventually disarmed by patching the simm32_epoch value of >>>>> the instruction stream to the current globally correct value. >>>>> >>>>> In terms of abstractions, to utilize this mechanism, you need to >>>>> create your BarierSet with an BarrierSetNMethod helper object, just >>>>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>>>> BarrierSetNMethod base class handles the barrier itself. It has a >>>>> virtual function, nmethod_entry_barrier, which gets called by the >>>>> slow >>>>> paths. So each user of this mechanism inherits from BarrierSetNMethod >>>>> and puts its barrier in there. >>>>> >>>>> The OSR nmethods do not have a verified entry. Therefore, the barrier >>>>> is run explicitly in the runtime before calling into OSR nmethods to >>>>> achieve the same effect of trapping new calls into nmethods. >>>>> >>>>> The nmethod entry barrier returns true/false depending on whether the >>>>> called nmethod may be entered or not. This allows lazily cleaning >>>>> nmethod IC caches during unloading, which invokes the same helper >>>>> method that is patched into the verified entry when made not entrant. >>>>> >>>>> Eventually, I would like to also use this mechanism to remove the >>>>> nmethod hotness counter maintenance we today perform in safepoint >>>>> cleanup instead. These nmethod entry barriers can provide better >>>>> heuristics for whether nmethods are used or not. But that would >>>>> require buy-in from maintainers of other platforms. So let's take one >>>>> step at a time. >>>>> >>>>> For the curious reader thinking that this might be an expensive >>>>> operation, it is not. This check in the nmethod verified entry is >>>>> very >>>>> predictable and cheap. And because of use of inlining, the checks >>>>> become more infrequent. In my evaluation, it has gone completely >>>>> under >>>>> the radar in benchmarking (running this with my current version of >>>>> concurrent class unloading in ZGC). >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >>>> I reviewed this before Erik sent it out, so my comments have already >>>> been addressed. Just found one additional thing, the BarrierSet >>>> constructor now looks like this: >>>> >>>> 96 BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>> 97 BarrierSetC1* barrier_set_c1, >>>> 98 BarrierSetC2* barrier_set_c2, >>>> 99 const FakeRtti& fake_rtti, >>>> 100 BarrierSetNMethod* barrier_set_nmethod = NULL) : >>>> 101 _fake_rtti(fake_rtti), >>>> 102 _barrier_set_assembler(barrier_set_assembler), >>>> 103 _barrier_set_c1(barrier_set_c1), >>>> 104 _barrier_set_c2(barrier_set_c2), >>>> 105 _barrier_set_nmethod(barrier_set_nmethod) {} >>>> >>>> I find it odd that the fake_rtti argument comes in the middle of the >>>> barrier_set_* arguments. I can see that you don't want GC barrier sets >>>> to have to supply NULL if they don't have a nmethod barrier set, but >>>> maybe we can have two constructors instead, like: >>>> >>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>> BarrierSetC1* barrier_set_c1, >>>> BarrierSetC2* barrier_set_c2, >>>> const FakeRtti& fake_rtti) : >>>> _fake_rtti(fake_rtti), >>>> _barrier_set_assembler(barrier_set_assembler), >>>> _barrier_set_c1(barrier_set_c1), >>>> _barrier_set_c2(barrier_set_c2), >>>> _barrier_set_nmethod(NULL) {} >>>> >>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>> BarrierSetC1* barrier_set_c1, >>>> BarrierSetC2* barrier_set_c2, >>>> BarrierSetNMethod* barrier_set_nmethod, >>>> const FakeRtti& fake_rtti,) : >>>> _fake_rtti(fake_rtti), >>>> _barrier_set_assembler(barrier_set_assembler), >>>> _barrier_set_c1(barrier_set_c1), >>>> _barrier_set_c2(barrier_set_c2), >>>> _barrier_set_nmethod(barrier_set_nmethod) {} >>>> >>>> What do you think? >>>> >>>> Other than that, this looks good to me. >>>> >>>> cheers, >>>> Per >>>> >>>>> Bug ID: >>>>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>>>> >>>>> Thanks, >>>>> /Erik >> From bsrbnd at gmail.com Mon Oct 8 08:40:19 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 10:40:19 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: On Sun, 7 Oct 2018 at 20:29, Roman Kennke wrote: > > IIRC, some CPUs even have instructions for directly getting/setting > bits, without explicit mask/shift. AArch64 can do that I think. Not sure > if it's more efficient though. > > Roman Sounds great, probably no word-tearing problem and no additional sync would be necessary on this arch. Unfortunately, I don't have the opportunity to work on this hardware, maybe someone could experiment on it? Bernard From aph at redhat.com Mon Oct 8 08:53:17 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 8 Oct 2018 09:53:17 +0100 Subject: Primitive boolean array packing In-Reply-To: <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On 10/07/2018 08:54 PM, Aleksey Shipilev wrote: > You need to specify what do you mean by "it". Current prototype sure > does break the provisions of JLS 17.6 "Word Tearing": > https://docs.oracle.com/javase/specs/jls/se11/html/jls-17.html#jls-17.6 Oh, sure, of course. > My comment was about generic thing: we cannot break the Java spec > even if we have the JVM flag that makes it right again. I thought > that what B. was suggesting as the escape hatch. And I thought it was a switch between bit arrays and byte arrays. > Of course, if you do packed boolean[] with locked/CAS-ed writes or > otherwise guarantee the absence of word tearing, it does not break > the spec. Exactly. it's an interesting idea for large bit vectors, and it could well generate good code. However, it probably makes more sense to implement it as a bit vector class with suitable helper intrinsics. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Mon Oct 8 09:31:27 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 8 Oct 2018 10:31:27 +0100 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <5BB77E18.6040401@oracle.com> References: <5BB77E18.6040401@oracle.com> Message-ID: <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com> On 10/05/2018 04:07 PM, Erik ?sterlund wrote: > This implementation is for x86_64 and is based on the prototyping > work of Rickard B?ckman. So big thanks to Rickard, who will > co-author of this patch. > The way it works is that there is a cmp #offset(r15_thread), > simm32_epoch); je #continuation; inserted into the verified entry of > nmethods that conditionally calls a stub trampolining into the VM. > When the simm32_epoch is not equal to some TLS-local field, then a > slow path is taken to run some barrier inside of the VM. By > changing this TLS-local field in the safepoint, all verified entries > will take the slow path into the VM when run, where the barrier is > eventually disarmed by patching the simm32_epoch value of the > instruction stream to the current globally correct value. OK. And because it's only triggered on a safepoint there is no need for any memory fences. > For the curious reader thinking that this might be an expensive > operation, it is not. This check in the nmethod verified entry is > very predictable and cheap. And because of use of inlining, the > checks become more infrequent. In my evaluation, it has gone > completely under the radar in benchmarking (running this with my > current version of concurrent class unloading in ZGC). I'm unconvinced. This change makes our entry/exit mechanism, already rather heavyweight, even more so. I'm aware of the "below the noise" argument but I don't think it's valid. Efficient systems are composed of thousands of tiny optimizations, each one of which is too small to produce any measurable benefit on its own. But they add up, or rather, they multiply. What's happening here, I suspect, is that branch prediction and speculation logic in the x86 can often do the whole lookahead/speculate cache line read/speculate branch sequence in parallel with whatever else the method needs to do. If so, that's a card you get to play, but only once. Not that this is an argument against the patch: if we need to do it, then we have to. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.stuefe at gmail.com Mon Oct 8 09:43:57 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 8 Oct 2018 11:43:57 +0200 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Message-ID: Hi all, I drafted a CSR: https://bugs.openjdk.java.net/browse/JDK-8211846 Not sure what to do now, should I open a discussion thread on a mailing list? Or just wait for the committee to decide? (I'm doing this too rarely..) Thanks, Thomas On Thu, Oct 4, 2018 at 8:11 PM Thomas St?fe wrote: > > On Thu, Oct 4, 2018 at 11:18 AM David Holmes wrote: > > > > Hi Thomas, > > > > On 4/10/2018 6:27 PM, Thomas St?fe wrote: > > > Hi David, > > > On Thu, Oct 4, 2018 at 9:44 AM David Holmes wrote: > > >> > > >> Hi Matthias, > > >> > > >> I'm hoping others will chime in here as I: > > >> > > >> a) don't know if this information is actually useful for an error log of > > >> this kind; > > >> > > >> b) don't know if the information might be considered sensitive or not; and > > >> > > > > > > I have no opinion on (a) and (b). > > > > That's a shame :) > > > > >> c) don't think it's worth the effort of adding a flag to control this. > > >> Plus the flag is only useful for trying to reproduce an issue; if it's a > > >> one-of failure then you've already missed out on the information in the > > >> log file. > > > > > > How about a more generic switch to control verbosity of the error report? > > > > > > The way we and you use the error files seem to differ. You seem to > > > prefer them short and snappy and bare any security relevant details > > > (as far as that is even possible in an hs-err file). As was once > > > mentioned in a similar discussion, "OpenJDK hs-err files get posted > > > verbatim in forums and bug reports". > > > > > > We use the hs-err files differently. They are usually handed down to > > > us by our customers thru secure channels, and for us size does not > > > matter much, nor does security relevant information since we have > > > contracts with our customers. > > > > To be honest I don't deal with hs-err files from other people very much > > at all. But from what I have dealt with I haven't, to the best of my > > recollection, encountered any crash investigation where all this extra > > info would have shed any light. But most crashes I investigate are > > developer related rather than end user. > > That may be the difference. > > We get crash reports from customers, often with little context, no > repros, no cores, nothing. So we have to make do with what we have, > which is the hs-err files. Therefore we make them as extensive as > possible. Our support worked miracles with just these files. It is > incredible what you can deduce from them. > > User information, e.g., can hint towards a mismanaged installation, if > the server node does not run under the correct user. I also dimly > remember it being useful in analyzing SIGBUS crashes for mapped files > on Solaris. > > So, it turns out I have an opinion :) Well when I say no opinion I > mean I can see arguments for both sides, and I feel uncomfortable > pressing our way of working upon the OpenJDK code base. > > > > > > That has been a point of contention over and over again in the past. > > > So I wonder whether one, or possibly two, general switches could keep > > > both sides happy. Something like -XX:+ExtendedErrorReports" and > > > possibly "-XX:+ErrorReportsIncludeSensitiveData". > > > > > > Those switches could be, by default, false in the OpenJDK. > > > > > > Any additions we add to error reporting where we cannot find an > > > agreement we could make conditional on one or the other switch. > > > > > > What do you think? > > > > Seems reasonable to propose. Any new flag will require a CSR request. > > > > Okay, thanks! > > I will file a CSR next week when I am back at the office. > > ..Thomas > > > David > > > > > ..Thomas > > > > > >> > > >> Cheers, > > >> David > > >> > > >> > > >> On 4/10/2018 5:31 PM, Baesken, Matthias wrote: > > >>> Hello, my proposal would be to only print > > >>> > > >>> uid : 1679 (testuser-name) > > >>> > > >>> by default and guard the rest of the info by some XX-flag, any good proposals for the flag-name are appreciated; > > >>> for example : > > >>> > > >>> if (ExtendHsErrorFileByUserRelatedInformation) { > > >>> > > >>> // print those too : > > >>> > > >>>>>> euid : 1679 (testuser-name) > > >>>>>> gid : 25 (testgroup) > > >>>>>> egid : 25 (testgroup) > > >>>>>> > > >>>>>> umask: 0022 (removing ----w--w-) > > >>> > > >>> } > > >>> > > >>> > > >>> Best regards, Matthias > > >>> > > >>> > > >>>> -----Original Message----- > > >>>> From: Baesken, Matthias > > >>>> Sent: Dienstag, 2. Oktober 2018 12:38 > > >>>> To: 'David Holmes' ; 'hotspot- > > >>>> dev at openjdk.java.net' > > >>>> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > > >>>> > > >>>> Hi David, I think the added info could be seen more or less in line with > > >>>> what currently is reported in hs_err file . > > >>>> For instance you usually see user-names and lots of paths from the system > > >>>> in the hs_err file . > > >>>> > > >>>> In case the umask and gid is seen as more sensitive than that, one could > > >>>> make the output switchable with an XX-flag ; > > >>>> this would have the benefit of making the added output more clear to the > > >>>> user/admin . > > >>>> > > >>>> Best regards, Matthias > > >>>> > > >>>> > > >>>>> -----Original Message----- > > >>>>> From: David Holmes > > >>>>> Sent: Dienstag, 2. Oktober 2018 09:49 > > >>>>> To: Baesken, Matthias ; 'hotspot- > > >>>>> dev at openjdk.java.net' > > >>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > >>>>> > > >>>>> Hi Matthias, > > >>>>> > > >>>>> On 2/10/2018 5:30 PM, Baesken, Matthias wrote: > > >>>>>> Hello , please review this small enhancement to the hs_err file . > > >>>>>> > > >>>>>> Currently the hs_err file contains only limited OS user related > > >>>> information. > > >>>>>> Just the user name is printed via output of environment variables (at > > >>>> least > > >>>>> on Windows with USERNAME - output). > > >>>>>> The enhanced output on UNIX would contain more information including > > >>>>> uid, gid and umask : > > >>>>>> > > >>>>>> uid : 1679 (testuser) > > >>>>>> euid : 1679 (testuser) > > >>>>>> gid : 25 (testgroup) > > >>>>>> egid : 25 (testgroup) > > >>>>>> > > >>>>>> umask: 0022 (removing ----w--w-) > > >>>>> > > >>>>> Could any of this be considered sensitive information by an end-user? > > >>>>> > > >>>>> Thanks, > > >>>>> David > > >>>>> > > >>>>>> > > >>>>>> ( Some of the info above could be found currently in error logging > > >>>> output > > >>>>> e.g. > > >>>>>> attachListener_linux.cpp line 362 > > >>>>>> log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", > > >>>>>> (and the user name on Windows(-only) is in the env variables section). > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> bug/webrev : > > >>>>>> ---------------------- > > >>>>>> > > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8211326 > > >>>>>> > > >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ > > >>>>>> > > >>>>>> > > >>>>>> Thanks, Matthias > > >>>>>> From david.holmes at oracle.com Mon Oct 8 09:50:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 8 Oct 2018 19:50:22 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> Message-ID: <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Hi Thomas, On 8/10/2018 7:43 PM, Thomas St?fe wrote: > Hi all, > > I drafted a CSR: > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > Not sure what to do now, should I open a discussion thread on a > mailing list? Or just wait for the committee to decide? (I'm doing > this too rarely..) I filled in some of the additional info needed. The text needs to be completed and the CSR needs to have at least one reviewer added to it. Then it can be marked "final" and ready for the committee to evaluate. David > Thanks, Thomas > > On Thu, Oct 4, 2018 at 8:11 PM Thomas St?fe wrote: >> >> On Thu, Oct 4, 2018 at 11:18 AM David Holmes wrote: >>> >>> Hi Thomas, >>> >>> On 4/10/2018 6:27 PM, Thomas St?fe wrote: >>>> Hi David, >>>> On Thu, Oct 4, 2018 at 9:44 AM David Holmes wrote: >>>>> >>>>> Hi Matthias, >>>>> >>>>> I'm hoping others will chime in here as I: >>>>> >>>>> a) don't know if this information is actually useful for an error log of >>>>> this kind; >>>>> >>>>> b) don't know if the information might be considered sensitive or not; and >>>>> >>>> >>>> I have no opinion on (a) and (b). >>> >>> That's a shame :) >>> >>>>> c) don't think it's worth the effort of adding a flag to control this. >>>>> Plus the flag is only useful for trying to reproduce an issue; if it's a >>>>> one-of failure then you've already missed out on the information in the >>>>> log file. >>>> >>>> How about a more generic switch to control verbosity of the error report? >>>> >>>> The way we and you use the error files seem to differ. You seem to >>>> prefer them short and snappy and bare any security relevant details >>>> (as far as that is even possible in an hs-err file). As was once >>>> mentioned in a similar discussion, "OpenJDK hs-err files get posted >>>> verbatim in forums and bug reports". >>>> >>>> We use the hs-err files differently. They are usually handed down to >>>> us by our customers thru secure channels, and for us size does not >>>> matter much, nor does security relevant information since we have >>>> contracts with our customers. >>> >>> To be honest I don't deal with hs-err files from other people very much >>> at all. But from what I have dealt with I haven't, to the best of my >>> recollection, encountered any crash investigation where all this extra >>> info would have shed any light. But most crashes I investigate are >>> developer related rather than end user. >> >> That may be the difference. >> >> We get crash reports from customers, often with little context, no >> repros, no cores, nothing. So we have to make do with what we have, >> which is the hs-err files. Therefore we make them as extensive as >> possible. Our support worked miracles with just these files. It is >> incredible what you can deduce from them. >> >> User information, e.g., can hint towards a mismanaged installation, if >> the server node does not run under the correct user. I also dimly >> remember it being useful in analyzing SIGBUS crashes for mapped files >> on Solaris. >> >> So, it turns out I have an opinion :) Well when I say no opinion I >> mean I can see arguments for both sides, and I feel uncomfortable >> pressing our way of working upon the OpenJDK code base. >> >>> >>>> That has been a point of contention over and over again in the past. >>>> So I wonder whether one, or possibly two, general switches could keep >>>> both sides happy. Something like -XX:+ExtendedErrorReports" and >>>> possibly "-XX:+ErrorReportsIncludeSensitiveData". >>>> >>>> Those switches could be, by default, false in the OpenJDK. >>>> >>>> Any additions we add to error reporting where we cannot find an >>>> agreement we could make conditional on one or the other switch. >>>> >>>> What do you think? >>> >>> Seems reasonable to propose. Any new flag will require a CSR request. >>> >> >> Okay, thanks! >> >> I will file a CSR next week when I am back at the office. >> >> ..Thomas >> >>> David >>> >>>> ..Thomas >>>> >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>> >>>>> On 4/10/2018 5:31 PM, Baesken, Matthias wrote: >>>>>> Hello, my proposal would be to only print >>>>>> >>>>>> uid : 1679 (testuser-name) >>>>>> >>>>>> by default and guard the rest of the info by some XX-flag, any good proposals for the flag-name are appreciated; >>>>>> for example : >>>>>> >>>>>> if (ExtendHsErrorFileByUserRelatedInformation) { >>>>>> >>>>>> // print those too : >>>>>> >>>>>>>>> euid : 1679 (testuser-name) >>>>>>>>> gid : 25 (testgroup) >>>>>>>>> egid : 25 (testgroup) >>>>>>>>> >>>>>>>>> umask: 0022 (removing ----w--w-) >>>>>> >>>>>> } >>>>>> >>>>>> >>>>>> Best regards, Matthias >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Baesken, Matthias >>>>>>> Sent: Dienstag, 2. Oktober 2018 12:38 >>>>>>> To: 'David Holmes' ; 'hotspot- >>>>>>> dev at openjdk.java.net' >>>>>>> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file >>>>>>> >>>>>>> Hi David, I think the added info could be seen more or less in line with >>>>>>> what currently is reported in hs_err file . >>>>>>> For instance you usually see user-names and lots of paths from the system >>>>>>> in the hs_err file . >>>>>>> >>>>>>> In case the umask and gid is seen as more sensitive than that, one could >>>>>>> make the output switchable with an XX-flag ; >>>>>>> this would have the benefit of making the added output more clear to the >>>>>>> user/admin . >>>>>>> >>>>>>> Best regards, Matthias >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: David Holmes >>>>>>>> Sent: Dienstag, 2. Oktober 2018 09:49 >>>>>>>> To: Baesken, Matthias ; 'hotspot- >>>>>>>> dev at openjdk.java.net' >>>>>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >>>>>>>> >>>>>>>> Hi Matthias, >>>>>>>> >>>>>>>> On 2/10/2018 5:30 PM, Baesken, Matthias wrote: >>>>>>>>> Hello , please review this small enhancement to the hs_err file . >>>>>>>>> >>>>>>>>> Currently the hs_err file contains only limited OS user related >>>>>>> information. >>>>>>>>> Just the user name is printed via output of environment variables (at >>>>>>> least >>>>>>>> on Windows with USERNAME - output). >>>>>>>>> The enhanced output on UNIX would contain more information including >>>>>>>> uid, gid and umask : >>>>>>>>> >>>>>>>>> uid : 1679 (testuser) >>>>>>>>> euid : 1679 (testuser) >>>>>>>>> gid : 25 (testgroup) >>>>>>>>> egid : 25 (testgroup) >>>>>>>>> >>>>>>>>> umask: 0022 (removing ----w--w-) >>>>>>>> >>>>>>>> Could any of this be considered sensitive information by an end-user? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> >>>>>>>>> ( Some of the info above could be found currently in error logging >>>>>>> output >>>>>>>> e.g. >>>>>>>>> attachListener_linux.cpp line 362 >>>>>>>>> log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", >>>>>>>>> (and the user name on Windows(-only) is in the env variables section). >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> bug/webrev : >>>>>>>>> ---------------------- >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8211326 >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, Matthias >>>>>>>>> From thomas.stuefe at gmail.com Mon Oct 8 10:21:24 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 8 Oct 2018 12:21:24 +0200 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: On Mon, Oct 8, 2018 at 11:50 AM David Holmes wrote: > > Hi Thomas, > > On 8/10/2018 7:43 PM, Thomas St?fe wrote: > > Hi all, > > > > I drafted a CSR: > > > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > > > Not sure what to do now, should I open a discussion thread on a > > mailing list? Or just wait for the committee to decide? (I'm doing > > this too rarely..) > > I filled in some of the additional info needed. The text needs to be > completed and the CSR needs to have at least one reviewer added to it. > Then it can be marked "final" and ready for the committee to evaluate. > Thanks! I filled out what was missing and will wait for reviewers. Thanks, Thomas > David > > > Thanks, Thomas > > > > On Thu, Oct 4, 2018 at 8:11 PM Thomas St?fe wrote: > >> > >> On Thu, Oct 4, 2018 at 11:18 AM David Holmes wrote: > >>> > >>> Hi Thomas, > >>> > >>> On 4/10/2018 6:27 PM, Thomas St?fe wrote: > >>>> Hi David, > >>>> On Thu, Oct 4, 2018 at 9:44 AM David Holmes wrote: > >>>>> > >>>>> Hi Matthias, > >>>>> > >>>>> I'm hoping others will chime in here as I: > >>>>> > >>>>> a) don't know if this information is actually useful for an error log of > >>>>> this kind; > >>>>> > >>>>> b) don't know if the information might be considered sensitive or not; and > >>>>> > >>>> > >>>> I have no opinion on (a) and (b). > >>> > >>> That's a shame :) > >>> > >>>>> c) don't think it's worth the effort of adding a flag to control this. > >>>>> Plus the flag is only useful for trying to reproduce an issue; if it's a > >>>>> one-of failure then you've already missed out on the information in the > >>>>> log file. > >>>> > >>>> How about a more generic switch to control verbosity of the error report? > >>>> > >>>> The way we and you use the error files seem to differ. You seem to > >>>> prefer them short and snappy and bare any security relevant details > >>>> (as far as that is even possible in an hs-err file). As was once > >>>> mentioned in a similar discussion, "OpenJDK hs-err files get posted > >>>> verbatim in forums and bug reports". > >>>> > >>>> We use the hs-err files differently. They are usually handed down to > >>>> us by our customers thru secure channels, and for us size does not > >>>> matter much, nor does security relevant information since we have > >>>> contracts with our customers. > >>> > >>> To be honest I don't deal with hs-err files from other people very much > >>> at all. But from what I have dealt with I haven't, to the best of my > >>> recollection, encountered any crash investigation where all this extra > >>> info would have shed any light. But most crashes I investigate are > >>> developer related rather than end user. > >> > >> That may be the difference. > >> > >> We get crash reports from customers, often with little context, no > >> repros, no cores, nothing. So we have to make do with what we have, > >> which is the hs-err files. Therefore we make them as extensive as > >> possible. Our support worked miracles with just these files. It is > >> incredible what you can deduce from them. > >> > >> User information, e.g., can hint towards a mismanaged installation, if > >> the server node does not run under the correct user. I also dimly > >> remember it being useful in analyzing SIGBUS crashes for mapped files > >> on Solaris. > >> > >> So, it turns out I have an opinion :) Well when I say no opinion I > >> mean I can see arguments for both sides, and I feel uncomfortable > >> pressing our way of working upon the OpenJDK code base. > >> > >>> > >>>> That has been a point of contention over and over again in the past. > >>>> So I wonder whether one, or possibly two, general switches could keep > >>>> both sides happy. Something like -XX:+ExtendedErrorReports" and > >>>> possibly "-XX:+ErrorReportsIncludeSensitiveData". > >>>> > >>>> Those switches could be, by default, false in the OpenJDK. > >>>> > >>>> Any additions we add to error reporting where we cannot find an > >>>> agreement we could make conditional on one or the other switch. > >>>> > >>>> What do you think? > >>> > >>> Seems reasonable to propose. Any new flag will require a CSR request. > >>> > >> > >> Okay, thanks! > >> > >> I will file a CSR next week when I am back at the office. > >> > >> ..Thomas > >> > >>> David > >>> > >>>> ..Thomas > >>>> > >>>>> > >>>>> Cheers, > >>>>> David > >>>>> > >>>>> > >>>>> On 4/10/2018 5:31 PM, Baesken, Matthias wrote: > >>>>>> Hello, my proposal would be to only print > >>>>>> > >>>>>> uid : 1679 (testuser-name) > >>>>>> > >>>>>> by default and guard the rest of the info by some XX-flag, any good proposals for the flag-name are appreciated; > >>>>>> for example : > >>>>>> > >>>>>> if (ExtendHsErrorFileByUserRelatedInformation) { > >>>>>> > >>>>>> // print those too : > >>>>>> > >>>>>>>>> euid : 1679 (testuser-name) > >>>>>>>>> gid : 25 (testgroup) > >>>>>>>>> egid : 25 (testgroup) > >>>>>>>>> > >>>>>>>>> umask: 0022 (removing ----w--w-) > >>>>>> > >>>>>> } > >>>>>> > >>>>>> > >>>>>> Best regards, Matthias > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Baesken, Matthias > >>>>>>> Sent: Dienstag, 2. Oktober 2018 12:38 > >>>>>>> To: 'David Holmes' ; 'hotspot- > >>>>>>> dev at openjdk.java.net' > >>>>>>> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > >>>>>>> > >>>>>>> Hi David, I think the added info could be seen more or less in line with > >>>>>>> what currently is reported in hs_err file . > >>>>>>> For instance you usually see user-names and lots of paths from the system > >>>>>>> in the hs_err file . > >>>>>>> > >>>>>>> In case the umask and gid is seen as more sensitive than that, one could > >>>>>>> make the output switchable with an XX-flag ; > >>>>>>> this would have the benefit of making the added output more clear to the > >>>>>>> user/admin . > >>>>>>> > >>>>>>> Best regards, Matthias > >>>>>>> > >>>>>>> > >>>>>>>> -----Original Message----- > >>>>>>>> From: David Holmes > >>>>>>>> Sent: Dienstag, 2. Oktober 2018 09:49 > >>>>>>>> To: Baesken, Matthias ; 'hotspot- > >>>>>>>> dev at openjdk.java.net' > >>>>>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > >>>>>>>> > >>>>>>>> Hi Matthias, > >>>>>>>> > >>>>>>>> On 2/10/2018 5:30 PM, Baesken, Matthias wrote: > >>>>>>>>> Hello , please review this small enhancement to the hs_err file . > >>>>>>>>> > >>>>>>>>> Currently the hs_err file contains only limited OS user related > >>>>>>> information. > >>>>>>>>> Just the user name is printed via output of environment variables (at > >>>>>>> least > >>>>>>>> on Windows with USERNAME - output). > >>>>>>>>> The enhanced output on UNIX would contain more information including > >>>>>>>> uid, gid and umask : > >>>>>>>>> > >>>>>>>>> uid : 1679 (testuser) > >>>>>>>>> euid : 1679 (testuser) > >>>>>>>>> gid : 25 (testgroup) > >>>>>>>>> egid : 25 (testgroup) > >>>>>>>>> > >>>>>>>>> umask: 0022 (removing ----w--w-) > >>>>>>>> > >>>>>>>> Could any of this be considered sensitive information by an end-user? > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> David > >>>>>>>> > >>>>>>>>> > >>>>>>>>> ( Some of the info above could be found currently in error logging > >>>>>>> output > >>>>>>>> e.g. > >>>>>>>>> attachListener_linux.cpp line 362 > >>>>>>>>> log_debug(attach)("euid/egid check failed (%d/%d vs %d/%d)", > >>>>>>>>> (and the user name on Windows(-only) is in the env variables section). > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> bug/webrev : > >>>>>>>>> ---------------------- > >>>>>>>>> > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8211326 > >>>>>>>>> > >>>>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.0/ > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Thanks, Matthias > >>>>>>>>> From thomas.stuefe at gmail.com Mon Oct 8 10:48:57 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 8 Oct 2018 12:48:57 +0200 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: Hi Kim, is this JEP only about C++14 features or shall we discuss older features too? The reason I am asking is that I would like us to officially endorse namespaces. Not inline namespaces, just plain old namespaces. HotSpot makes very limited use of namespaces. Not really true, we already use them. E.g. in metaspace coding, I used them to keep the global name space clean and to keep internals internal. This was met with positive reviews, and it works on all toolchains, so compiler support should not be a problem. Using namespaces, we could get slowly replace the "AllStatic" classes, which are namespaces in all but name. In contrast to classes, namespaces can be spread over multiple files and compilation units, and allow for cleaner separation of internal and external coding. It also would allow us to get rid the middle-of-header-platform-inclusions: For example, today we have: [os.hpp] class os: AllStatic { .... (platform independent, outward facing os:: functions) #include "os_linux.hpp" >> (Inner class "Linux" with platform specific os functions) ... } Not only is the inclusion in the middle of a class terrifying, it also means the shared, outward facing os:: namespace contains class Linux and lots of platform specific internals. With namespaces one could: [os.hpp] namespace os { .... (platform independent, outward facing os:: functions) .... } [os_linux.hpp] namespace os { namespace Linux { (linux specific os functions) } } I think this is way cleaner, and keeps platform specifics from including files which only care for the shared os interface. -- Note that I would prefer forbidding the "using" directive for callers of namespace functions, but rather force them to spell out the namespace: So, instead of this: using os; jlong m = available_memory(); I would prefer this, which is our current practice with AllStatic childs: jlong m = os::available_memory(); The latter form would keep the code grepable. Best Regards, Thomas On Wed, Oct 3, 2018 at 9:13 PM Kim Barrett wrote: > > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089 > From bsrbnd at gmail.com Mon Oct 8 11:16:28 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 13:16:28 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Mon, 8 Oct 2018 at 10:53, Andrew Haley wrote: > > On 10/07/2018 08:54 PM, Aleksey Shipilev wrote: > > > Of course, if you do packed boolean[] with locked/CAS-ed writes or > > otherwise guarantee the absence of word tearing, it does not break > > the spec. > > Exactly. it's an interesting idea for large bit vectors, and it could > well generate good code. However, it probably makes more sense to > implement it as a bit vector class with suitable helper intrinsics. The current prototype was intended to scrupulously implement what the JVMS explicitly allows for baload/bastore instructions. But helper intrinsics might be good alternatives too, we could try both. Note that the existing BitSet needs external sync (most probably on the whole set) when used in multi-threaded environments. With packed baload/bastore instructions or dedicated intrinsics, the sync would be on smaller 8-bit blocks or no sync at all on some arch as Roman mentioned (AArch64). Bernard From volker.simonis at gmail.com Mon Oct 8 12:43:53 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 8 Oct 2018 14:43:53 +0200 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: On Mon, Oct 8, 2018 at 12:49 PM Thomas St?fe wrote: > > Hi Kim, > > is this JEP only about C++14 features or shall we discuss older > features too? The reason I am asking is that I would like us to > officially endorse namespaces. Not inline namespaces, just plain old > namespaces. > > HotSpot makes very limited use of namespaces. > > Not really true, we already use them. E.g. in metaspace coding, I used > them to keep the global name space clean and to keep internals > internal. This was met with positive reviews, and it works on all > toolchains, so compiler support should not be a problem. Using > namespaces, we could get slowly replace the "AllStatic" classes, which > are namespaces in all but name. In contrast to classes, namespaces can > be spread over multiple files and compilation units, and allow for > cleaner separation of internal and external coding. > > It also would allow us to get rid the middle-of-header-platform-inclusions: > That's overdue since a long time! It would allow us to finally get correct code navigation in IDEs. > For example, today we have: > > [os.hpp] > class os: AllStatic { > .... > (platform independent, outward facing os:: functions) > #include "os_linux.hpp" > >> (Inner class "Linux" with platform specific os functions) > ... > } > > Not only is the inclusion in the middle of a class terrifying, it also > means the shared, outward facing os:: namespace contains class Linux > and lots of platform specific internals. > > With namespaces one could: > > [os.hpp] > namespace os { > .... > (platform independent, outward facing os:: functions) > .... > } > > [os_linux.hpp] > namespace os { > namespace Linux { > (linux specific os functions) > } > } > > I think this is way cleaner, and keeps platform specifics from > including files which only care for the shared os interface. > > -- > > Note that I would prefer forbidding the "using" directive for callers > of namespace functions, but rather force them to spell out the > namespace: > As far as I saw, "using" directives are already on the "Exclude" list in Kim's proposal. > So, instead of this: > > using os; > jlong m = available_memory(); > > I would prefer this, which is our current practice with AllStatic childs: > > jlong m = os::available_memory(); > > The latter form would keep the code grepable. > > Best Regards, Thomas > On Wed, Oct 3, 2018 at 9:13 PM Kim Barrett wrote: > > > > I've submitted a JEP for > > > > (1) enabling the use of C++14 Language Features when building the JDK, > > > > (2) define a process for deciding and documenting which new features > > can be used or are forbidden in HotSpot code, > > > > (3) provide an initial list of permitted and forbidden new features. > > > > https://bugs.openjdk.java.net/browse/JDK-8208089 > > From gunter.haug at sap.com Mon Oct 8 12:47:36 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Mon, 8 Oct 2018 12:47:36 +0000 Subject: [RFR]: [S390] Implement JFR profiling Message-ID: Hi all, can I please have reviews and a sponsor fort the following change: https://bugs.openjdk.java.net/browse/JDK-8211768 http://cr.openjdk.java.net/~ghaug/webrevs/8211768/ The implementation is basically the same as for PPC. It was only adapted for the different use of registers and the slightly different frame layout of s390. The respective tests don't fail anymore on s390 with this patch. Thanks, Gunter From erik.osterlund at oracle.com Mon Oct 8 12:54:35 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 8 Oct 2018 14:54:35 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com> References: <5BB77E18.6040401@oracle.com> <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com> Message-ID: <5BBB538B.5070404@oracle.com> Hi Andrew, On 2018-10-08 11:31, Andrew Haley wrote: > On 10/05/2018 04:07 PM, Erik ?sterlund wrote: >> This implementation is for x86_64 and is based on the prototyping >> work of Rickard B?ckman. So big thanks to Rickard, who will >> co-author of this patch. >> The way it works is that there is a cmp #offset(r15_thread), >> simm32_epoch); je #continuation; inserted into the verified entry of >> nmethods that conditionally calls a stub trampolining into the VM. >> When the simm32_epoch is not equal to some TLS-local field, then a >> slow path is taken to run some barrier inside of the VM. By >> changing this TLS-local field in the safepoint, all verified entries >> will take the slow path into the VM when run, where the barrier is >> eventually disarmed by patching the simm32_epoch value of the >> instruction stream to the current globally correct value. > OK. And because it's only triggered on a safepoint there is no need for > any memory fences. Correct. >> For the curious reader thinking that this might be an expensive >> operation, it is not. This check in the nmethod verified entry is >> very predictable and cheap. And because of use of inlining, the >> checks become more infrequent. In my evaluation, it has gone >> completely under the radar in benchmarking (running this with my >> current version of concurrent class unloading in ZGC). > I'm unconvinced. This change makes our entry/exit mechanism, already > rather heavyweight, even more so. I'm aware of the "below the noise" > argument but I don't think it's valid. Efficient systems are composed > of thousands of tiny optimizations, each one of which is too small to > produce any measurable benefit on its own. But they add up, or rather, > they multiply. > > What's happening here, I suspect, is that branch prediction and > speculation logic in the x86 can often do the whole > lookahead/speculate cache line read/speculate branch sequence in > parallel with whatever else the method needs to do. If so, that's a > card you get to play, but only once. > > Not that this is an argument against the patch: if we need to do it, > then we have to. I do understand your concerns. I did not have a good intuition about the performance characteristics either. So I will start by answering the implicit question whether we need this or not. We do need it badly. We have been considering our options for a few years, and all paths led us here. We do not have a single model for even STW class unloading in ZGC that does not involve nmethod entry barriers. Note that this is also an opt-in mechanism that you may choose to use it or not to use. I would like to convince you though that this may be useful for other GCs than ZGC as well. In fact, you may use it for optimization purposes, and hopefully buy back more performance than you lost. Here are a few possible optimization applications that nmethod entry barriers could enable: 1) Removing nmethod hotness counter maintenance causing a full system stack scanning operation every safepoint, in favour of marking nmethods that are being used in an entry barrier. 2) Patching GC barriers when changing phases. GC barriers like the pre-write barrier of Shenandoah and G1, are only necessary during concurrent marking. By patching the nmethods lazily, you can remove those unnecessary paths from the write barriers. The same argument applies for Shenandoah copy-on-write barriers that we know can not trigger after the relocation phase is done. 3) Other GCs might also want to utilize the concurrent class (and nmethod) unloading mechanisms that we introduce using nmethod entry barriers. Also note that the implementation space of the barrier itself has some flexibility. Rickard's first prototype involved having an unconditional branch patched in over a nop. Since the nop is removed in the frontend, it seemed like the most conservative starting point. But since there was no measurable difference to the conditional branch, that was more favourable in the end, since it hade the additional advantage of not requiring a code cache walk in the safepoint. But if you have a platform where the trade off is not as obvious, both mechanisms could easily be supported. Regarding your hypothesis why this works well, I agree. The very predictable nature of the branch combined with the speculation machinery, is probably very beneficial. And I think that since the introduction of 2 branch ports in the reservation stations, this type of code is handled even better by the hardware. If I manage to persuade you to give this a shot on AArch64, I would be interested in hearing if further optimizations are required. In theory, you might be able to go even further and do VA tricks with thread stacks and bake this check into the stack banging, to both dodge the extra conditional branch, and not require code cache walks in safepoints. But that would have a much higher complexity budget, and would certainly require an observable problem to motivate a matching solution complexity, IMO. Thanks, /Erik From volker.simonis at gmail.com Mon Oct 8 13:22:47 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 8 Oct 2018 15:22:47 +0200 Subject: [RFR]: [S390] Implement JFR profiling In-Reply-To: References: Message-ID: Hi Gunter, thanks for fixing this issue on s390. In general your change looks good. Just some minor comments: thread_linux_s390.cpp ================= + // nothing else to try + return false; + + ucontext_t* uc = (ucontext_t*) ucontext; return true; } I think the last two lines after "return false" are a leftover from the old version and should be removed. Strange you didn't got a compiler warning because of unreachable code. frame_s390.cpp ============ + // reliable. Unfortunately we can only check frame completeness for + // runtime stubs and nmethods. Other generic buffer blobs are more + // problematic so we just assume they are OK. Adapter blobs never have a + // complete frame and are never OK + if (!_cb->is_frame_complete_at(_pc)) { + if (_cb->is_adapter_blob() || _cb->is_runtime_stub()) { In the comment you mention that we can only check completeness for runtime stubs and nmethods but later in the code you only check for the stubs case and not for nmethods. Which is right, the code or the comment? frame_s390.hpp ============ I think from a philosophical point of view it is unfortunate, that the ijava_state_unsafe() is "public" while ijava_state() is "private", but I don't mind if you don't change this any more :) Regards, Volker On Mon, Oct 8, 2018 at 2:48 PM Haug, Gunter wrote: > > Hi all, > > can I please have reviews and a sponsor fort the following change: > > https://bugs.openjdk.java.net/browse/JDK-8211768 > http://cr.openjdk.java.net/~ghaug/webrevs/8211768/ > > The implementation is basically the same as for PPC. It was only adapted for the different use of registers and the slightly different frame layout of s390. The respective tests don't fail anymore on s390 with this patch. > > Thanks, > Gunter > > > > From gunter.haug at sap.com Mon Oct 8 13:45:11 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Mon, 8 Oct 2018 13:45:11 +0000 Subject: [RFR]: [S390] Implement JFR profiling In-Reply-To: References: Message-ID: Thanks a lot, Volker! > I think the last two lines after "return false" are a leftover from > the old version and should be removed. Strange you didn't got a > compiler warning because of unreachable code. You are completely right! I'll remove it. > In the comment you mention that we can only check completeness for > runtime stubs and nmethods but later in the code you only check for > the stubs case and not for nmethods. Which is right, the code or the > comment? The comment is a leftover from PPC, as well. There is no frame complete in compiled methods on s390. I'll remove the comment. > I think from a philosophical point of view it is unfortunate, that the > ijava_state_unsafe() is "public" while ijava_state() is "private", but > I don't mind if you don't change this any more :) I see what you mean. S390 is the only platform where there is a magic number in the i_state in debug builds and this is asserted ijava_state(). I'm with you that the naming is a little unfortunate but I simply head no better idea. If you have one, let me know, I would be happy to incorporate it! Thanks again, Gunter ?On 08.10.18, 15:23, "Volker Simonis" wrote: Hi Gunter, thanks for fixing this issue on s390. In general your change looks good. Just some minor comments: thread_linux_s390.cpp ================= + // nothing else to try + return false; + + ucontext_t* uc = (ucontext_t*) ucontext; return true; } I think the last two lines after "return false" are a leftover from the old version and should be removed. Strange you didn't got a compiler warning because of unreachable code. frame_s390.cpp ============ + // reliable. Unfortunately we can only check frame completeness for + // runtime stubs and nmethods. Other generic buffer blobs are more + // problematic so we just assume they are OK. Adapter blobs never have a + // complete frame and are never OK + if (!_cb->is_frame_complete_at(_pc)) { + if (_cb->is_adapter_blob() || _cb->is_runtime_stub()) { In the comment you mention that we can only check completeness for runtime stubs and nmethods but later in the code you only check for the stubs case and not for nmethods. Which is right, the code or the comment? frame_s390.hpp ============ I think from a philosophical point of view it is unfortunate, that the ijava_state_unsafe() is "public" while ijava_state() is "private", but I don't mind if you don't change this any more :) Regards, Volker On Mon, Oct 8, 2018 at 2:48 PM Haug, Gunter wrote: > > Hi all, > > can I please have reviews and a sponsor fort the following change: > > https://bugs.openjdk.java.net/browse/JDK-8211768 > http://cr.openjdk.java.net/~ghaug/webrevs/8211768/ > > The implementation is basically the same as for PPC. It was only adapted for the different use of registers and the slightly different frame layout of s390. The respective tests don't fail anymore on s390 with this patch. > > Thanks, > Gunter > > > > From martin.doerr at sap.com Mon Oct 8 15:18:33 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 8 Oct 2018 15:18:33 +0000 Subject: [RFR]: [S390] Implement JFR profiling In-Reply-To: References: Message-ID: Hi Gunter, I suggest to call the function "ijava_state_unchecked". In addition to Volker's change requests, the indentation in thread_linux_s390.cpp looks odd (regarding one of the else cases). Thanks, Martin -----Original Message----- From: hotspot-dev On Behalf Of Haug, Gunter Sent: Montag, 8. Oktober 2018 15:45 To: Volker Simonis Cc: HotSpot Open Source Developers Subject: Re: [RFR]: [S390] Implement JFR profiling Thanks a lot, Volker! > I think the last two lines after "return false" are a leftover from > the old version and should be removed. Strange you didn't got a > compiler warning because of unreachable code. You are completely right! I'll remove it. > In the comment you mention that we can only check completeness for > runtime stubs and nmethods but later in the code you only check for > the stubs case and not for nmethods. Which is right, the code or the > comment? The comment is a leftover from PPC, as well. There is no frame complete in compiled methods on s390. I'll remove the comment. > I think from a philosophical point of view it is unfortunate, that the > ijava_state_unsafe() is "public" while ijava_state() is "private", but > I don't mind if you don't change this any more :) I see what you mean. S390 is the only platform where there is a magic number in the i_state in debug builds and this is asserted ijava_state(). I'm with you that the naming is a little unfortunate but I simply head no better idea. If you have one, let me know, I would be happy to incorporate it! Thanks again, Gunter ?On 08.10.18, 15:23, "Volker Simonis" wrote: Hi Gunter, thanks for fixing this issue on s390. In general your change looks good. Just some minor comments: thread_linux_s390.cpp ================= + // nothing else to try + return false; + + ucontext_t* uc = (ucontext_t*) ucontext; return true; } I think the last two lines after "return false" are a leftover from the old version and should be removed. Strange you didn't got a compiler warning because of unreachable code. frame_s390.cpp ============ + // reliable. Unfortunately we can only check frame completeness for + // runtime stubs and nmethods. Other generic buffer blobs are more + // problematic so we just assume they are OK. Adapter blobs never have a + // complete frame and are never OK + if (!_cb->is_frame_complete_at(_pc)) { + if (_cb->is_adapter_blob() || _cb->is_runtime_stub()) { In the comment you mention that we can only check completeness for runtime stubs and nmethods but later in the code you only check for the stubs case and not for nmethods. Which is right, the code or the comment? frame_s390.hpp ============ I think from a philosophical point of view it is unfortunate, that the ijava_state_unsafe() is "public" while ijava_state() is "private", but I don't mind if you don't change this any more :) Regards, Volker On Mon, Oct 8, 2018 at 2:48 PM Haug, Gunter wrote: > > Hi all, > > can I please have reviews and a sponsor fort the following change: > > https://bugs.openjdk.java.net/browse/JDK-8211768 > http://cr.openjdk.java.net/~ghaug/webrevs/8211768/ > > The implementation is basically the same as for PPC. It was only adapted for the different use of registers and the slightly different frame layout of s390. The respective tests don't fail anymore on s390 with this patch. > > Thanks, > Gunter > > > > From aph at redhat.com Mon Oct 8 15:19:26 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 8 Oct 2018 16:19:26 +0100 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: <882cb210-5870-2292-e78f-62857abdc54f@redhat.com> On 10/08/2018 12:16 PM, B. Blaser wrote: > The current prototype was intended to scrupulously implement what the > JVMS explicitly allows for baload/bastore instructions. > But helper intrinsics might be good alternatives too, we could try both. So I tried it with a rough C++/assembly language test, and it doesn't make much sense to use this for default boolean arrays. The update sequence on AArch64 is 0: ldxrb w2, [x0]; and w2, w2, w1 stxrb w3, w2, [x0] cbnz w3, 0b (combined with a bunch of uninteresting shifts and logical operations.) We have to load a byte in exclusive state, AND or OR it as appropriate, then store it, still in exclusive state. That gets us the atomicity we need. 10**9 random set operations on an 8 Mbit array take 0.550s; with a boolean array the time is 0.150s. This is the local overhead of the load/store exclusive. > Note that the existing BitSet needs external sync (most probably on > the whole set) when used in multi-threaded environments. > With packed baload/bastore instructions or dedicated intrinsics, the > sync would be on smaller 8-bit blocks or no sync at all on some arch > as Roman mentioned (AArch64). It might help. The question is how much we need large bit arrays. They're certainly useful for things like Bloom filters. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From gunter.haug at sap.com Mon Oct 8 15:22:24 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Mon, 8 Oct 2018 15:22:24 +0000 Subject: [RFR]: [S390] Implement JFR profiling In-Reply-To: References: Message-ID: Hi Martin, > I suggest to call the function "ijava_state_unchecked". Yes, why not. > In addition to Volker's change requests, the indentation in thread_linux_s390.cpp looks odd (regarding one of the else cases). I'll check. Thanks, Gunter ?On 08.10.18, 17:18, "Doerr, Martin" wrote: Hi Gunter, I suggest to call the function "ijava_state_unchecked". In addition to Volker's change requests, the indentation in thread_linux_s390.cpp looks odd (regarding one of the else cases). Thanks, Martin -----Original Message----- From: hotspot-dev On Behalf Of Haug, Gunter Sent: Montag, 8. Oktober 2018 15:45 To: Volker Simonis Cc: HotSpot Open Source Developers Subject: Re: [RFR]: [S390] Implement JFR profiling Thanks a lot, Volker! > I think the last two lines after "return false" are a leftover from > the old version and should be removed. Strange you didn't got a > compiler warning because of unreachable code. You are completely right! I'll remove it. > In the comment you mention that we can only check completeness for > runtime stubs and nmethods but later in the code you only check for > the stubs case and not for nmethods. Which is right, the code or the > comment? The comment is a leftover from PPC, as well. There is no frame complete in compiled methods on s390. I'll remove the comment. > I think from a philosophical point of view it is unfortunate, that the > ijava_state_unsafe() is "public" while ijava_state() is "private", but > I don't mind if you don't change this any more :) I see what you mean. S390 is the only platform where there is a magic number in the i_state in debug builds and this is asserted ijava_state(). I'm with you that the naming is a little unfortunate but I simply head no better idea. If you have one, let me know, I would be happy to incorporate it! Thanks again, Gunter On 08.10.18, 15:23, "Volker Simonis" wrote: Hi Gunter, thanks for fixing this issue on s390. In general your change looks good. Just some minor comments: thread_linux_s390.cpp ================= + // nothing else to try + return false; + + ucontext_t* uc = (ucontext_t*) ucontext; return true; } I think the last two lines after "return false" are a leftover from the old version and should be removed. Strange you didn't got a compiler warning because of unreachable code. frame_s390.cpp ============ + // reliable. Unfortunately we can only check frame completeness for + // runtime stubs and nmethods. Other generic buffer blobs are more + // problematic so we just assume they are OK. Adapter blobs never have a + // complete frame and are never OK + if (!_cb->is_frame_complete_at(_pc)) { + if (_cb->is_adapter_blob() || _cb->is_runtime_stub()) { In the comment you mention that we can only check completeness for runtime stubs and nmethods but later in the code you only check for the stubs case and not for nmethods. Which is right, the code or the comment? frame_s390.hpp ============ I think from a philosophical point of view it is unfortunate, that the ijava_state_unsafe() is "public" while ijava_state() is "private", but I don't mind if you don't change this any more :) Regards, Volker On Mon, Oct 8, 2018 at 2:48 PM Haug, Gunter wrote: > > Hi all, > > can I please have reviews and a sponsor fort the following change: > > https://bugs.openjdk.java.net/browse/JDK-8211768 > http://cr.openjdk.java.net/~ghaug/webrevs/8211768/ > > The implementation is basically the same as for PPC. It was only adapted for the different use of registers and the slightly different frame layout of s390. The respective tests don't fail anymore on s390 with this patch. > > Thanks, > Gunter > > > > From sgehwolf at redhat.com Mon Oct 8 16:20:52 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 08 Oct 2018 18:20:52 +0200 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: <6b1c457d-321b-f8a9-4b3a-44c79e0e8987@oracle.com> References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> <6c34f766-eb48-cd18-1d0d-c202b0b4052d@oracle.com> <4fa520f3-1341-ade8-79a1-962c0d5d7f5f@oracle.com> <6b1c457d-321b-f8a9-4b3a-44c79e0e8987@oracle.com> Message-ID: Hi! On Sun, 2018-10-07 at 07:52 +1000, David Holmes wrote: > On 7/10/2018 6:11 AM, dean.long at oracle.com wrote: > > On 10/5/18 5:18 PM, David Holmes wrote: > > > On 6/10/2018 12:18 AM, Andrew Haley wrote: > > > > On 10/05/2018 02:40 PM, Severin Gehwolf wrote: > > > > > > Also I thought that if ldrex was not paired with a strex then you > > > > > > should > > > > > > at least issue clrex - as done in generate_atomic_load_long() in > > > > > > stubGenerator_arm.cpp > > > > > > > > > > I'll defer to Andrew Haley for this. > > > > > > > > I'm skeptical that CLREX is necessary. The classic CAS for Arm is > > > > > > > > MOV R1, #0xFF ; load the ?lock taken? value > > > > try LDREX R0, [LockAddr] ; load the lock value > > > > CMP R0, #0 ; is the lock free? > > > > STREXEQ R1, R0, [LockAddr]; try and claim the lock > > > > CMPEQ R0, #0 ; did this succeed? > > > > BNE try ; no ? try again. . . > > > > ; yes ? we have the lock > > > > > > > > There's no CLREX, and no-one ever reported any problem. CLREX could be > > > > a performance boost because it takes the cache line out of exclusive > > > > state back to shared state, reducing later bus traffic. > > > > > > I would see CLREX as a potential performance boost not something > > > needed for correctness. But you're right that failure paths from > > > typical CAS code won't execute the strex and don't include a clrex. > > > > > > > I think it's needed for correctness only at context-switch time. If you > > context switch after the LDREX, you don't want the exclusive state to > > remain for the next thread scheduled. > > See also: > > https://community.arm.com/processors/f/discussions/6572/synchronization-primitives-do-i-need-clrex > > Seems if CLREX is needed for correctness it is at the OS level. > > On performance concern see: > > http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20150921/300951.html > > Bottom line seems to be: > - clrex is not essential > - clrex is probably not harmful to performance > - clrex may benefit performance OK. David, how would you like me to update the webrev then? 2a) http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8211387/webrev.02_a/ 2b) http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8211387/webrev.02_b/ FWIW, the performance gain for Zero is probably negligible, but it probably won't hurt either... Thanks, Severin From maurizio.cimadamore at oracle.com Mon Oct 8 16:30:05 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 8 Oct 2018 17:30:05 +0100 Subject: Primitive boolean array packing In-Reply-To: References: Message-ID: Hi Bernard, what you describe here is also overlapping with Project Panama, which allows you to define a Java wrapper for a packed struct using layout descriptions. If you go down that path, you can, I believe, address the memory footprint issues w/o touching the VM (e.g. by changing the implementation of some of these classes to use Panama structs). Could be a worthwhile experiment to try and do that! Cheers Maurizio On 06/10/18 23:05, B. Blaser wrote: > Hi, > > Per JVMS, primitive boolean arrays are currently using 8 bits (one > byte) per boolean value, see [1] (baload/bastore). > > We've see in some threads [2] that 'BitSet' or 'EnumSet' are > occasionally preferred solutions as they are using boolean vectors > (aka long values) using 8x less memory. > > But as the spec [1] allows primitive boolean array packing, I made a > quick experiment which is available here: > > http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.00/ > > This is a partial draft implementing primitive boolean array packing > (8x smaller) at machine-level (vs BitSet at class-level): > * only x86 interpreter implemented => c1/c2/inlining disabled for > methods using baload/bastore (not much performance regression when > building the jdk and running the tests) > * unsafe access partially implemented > * array copy currently not implemented > * -XX:+/-BooleanPacking to enable/disable the feature currently not implemented > > A couple of residual tier1 tests are still failing due to the > unfinished implementation. > > I didn't search much if such experiments have already been > accomplished, but I'd like to take the temperature of this feature as > completing the implementation represents a significant amount of work. > Is this something that is worth exploring? > > Thanks in advance for any feedback. > > Regards, > Bernard > > > [1] https://docs.oracle.com/javase/specs/jvms/se11/html/jvms-6.html#jvms-6.5.baload > [2] http://mail.openjdk.java.net/pipermail/compiler-dev/2018-September/012504.html From bsrbnd at gmail.com Mon Oct 8 17:18:08 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 19:18:08 +0200 Subject: Primitive boolean array packing In-Reply-To: <882cb210-5870-2292-e78f-62857abdc54f@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> <882cb210-5870-2292-e78f-62857abdc54f@redhat.com> Message-ID: On Mon, 8 Oct 2018 at 17:19, Andrew Haley wrote: > > On 10/08/2018 12:16 PM, B. Blaser wrote: > > The current prototype was intended to scrupulously implement what the > > JVMS explicitly allows for baload/bastore instructions. > > But helper intrinsics might be good alternatives too, we could try both. > > So I tried it with a rough C++/assembly language test, and it doesn't > make much sense to use this for default boolean arrays. The update > sequence on AArch64 is > > 0: ldxrb w2, [x0]; > and w2, w2, w1 > stxrb w3, w2, [x0] > cbnz w3, 0b > > (combined with a bunch of uninteresting shifts and logical > operations.) We have to load a byte in exclusive state, AND or OR it > as appropriate, then store it, still in exclusive state. That gets us > the atomicity we need. > > 10**9 random set operations on an 8 Mbit array take 0.550s; with a > boolean array the time is 0.150s. This is the local overhead of the > load/store exclusive. Reducing memory consumption has a price which doesn't seem too expensive, fortunately. > > Note that the existing BitSet needs external sync (most probably on > > the whole set) when used in multi-threaded environments. > > With packed baload/bastore instructions or dedicated intrinsics, the > > sync would be on smaller 8-bit blocks or no sync at all on some arch > > as Roman mentioned (AArch64). > > It might help. The question is how much we need large bit > arrays. They're certainly useful for things like Bloom filters. As I initially mentioned, this would be useful for large bit arrays or for a large *number* of smaller bit arrays. We are planning to refactor javac flags [1] currently using one long value per symbol [2] which potentially means a huge number of bit arrays (or vectors) of more than 64 elements. Bernard [1] http://mail.openjdk.java.net/pipermail/compiler-dev/2018-September/012504.html [2] http://hg.openjdk.java.net/jdk/jdk/file/d8aebcc2d3ac/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java#l98 > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From bsrbnd at gmail.com Mon Oct 8 17:25:41 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 19:25:41 +0200 Subject: Primitive boolean array packing In-Reply-To: References: Message-ID: On Mon, 8 Oct 2018 at 18:30, Maurizio Cimadamore wrote: > > Hi Bernard, > what you describe here is also overlapping with Project Panama, which > allows you to define a Java wrapper for a packed struct using layout > descriptions. If you go down that path, you can, I believe, address the > memory footprint issues w/o touching the VM (e.g. by changing the > implementation of some of these classes to use Panama structs). Could be > a worthwhile experiment to try and do that! > > Cheers > Maurizio Thanks Maurizio, I'll take a look ;-) Cheers, Bernard From vladimir.kozlov at oracle.com Mon Oct 8 17:57:22 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 8 Oct 2018 10:57:22 -0700 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <5BBB1328.3020503@oracle.com> References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> <5BBB1328.3020503@oracle.com> Message-ID: <0b99da49-b118-1db8-d21c-3213fdfffe36@oracle.com> First, all nmethods have verified entry. MachPrologNode is generated by C2 for all nmethods including OSR (we need stack bang in all cases). OSR does not have *Unverified* entry represented by MachUEPNode. C1 compiled OSR nmethods are different - you may need your changes for it but I am not sure - ask C1 experts. There could be complications if an OSR nmethods is not executed (after barrier check) due to state of stack on OSR entry but it is different issue. I am curious why you are using next 2 lines everywhere? + BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); + bs->nmethod_entry_barrier(this); instead of one call: BarrierSet::barrier_set()->nmethod_entry_barrier(this); The same question about duplicated code in interpreterRuntime.cpp Would be nice if you also update MachPrologNode::format() for case when barrier is used. It is used in PrintOptoAssembly. Thanks, Vladimir On 10/8/18 1:19 AM, Erik ?sterlund wrote: > Hi Per and Roman, > > Thank you for having a look at this. > I went with Roman's suggestion and made this an explicit opt-out instead. But I let ModRef pass in NULL, as none of the > barrier sets in that category currently use nmethod entry barriers. When they start to do so, that can be easily changed. > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ > > Thanks, > /Erik > > On 2018-10-08 08:48, Roman Kennke wrote: >> I agree. I wouldn't mind to keep it simple and pass NULL though. I'm not >> a big fan of default arguments anyway. >> >> Roman >> >>> On 2018-10-05 17:07, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> In order to implement concurrent class unloading for ZGC, a mechanism >>>> will be required to arm nmethods in a safepoint such that the first >>>> entry into an nmethod after the pause triggers a barrier before >>>> continuing. >>>> >>>> The barrier will be used for: >>>> ??? * Patching immediate oops, and keeping phantomly reachable oops >>>> alive through the nmethod entry >>>> ??? * Catching calls from stale IC caches (due to class unloading) into >>>> nmethods that are being unloaded (containing broken oops and >>>> metadata), and lazily clean the IC cache and re-resolve the call. >>>> >>>> This implementation is for x86_64 and is based on the prototyping work >>>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>>> this patch. >>>> The way it works is that there is a cmp #offset(r15_thread), >>>> simm32_epoch); je #continuation; inserted into the verified entry of >>>> nmethods that conditionally calls a stub trampolining into the VM. >>>> When the simm32_epoch is not equal to some TLS-local field, then a >>>> slow path is taken to run some barrier inside of the VM. >>>> By changing this TLS-local field in the safepoint, all verified >>>> entries will take the slow path into the VM when run, where the >>>> barrier is eventually disarmed by patching the simm32_epoch value of >>>> the instruction stream to the current globally correct value. >>>> >>>> In terms of abstractions, to utilize this mechanism, you need to >>>> create your BarierSet with an BarrierSetNMethod helper object, just >>>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>>> BarrierSetNMethod base class handles the barrier itself. It has a >>>> virtual function, nmethod_entry_barrier, which gets called by the slow >>>> paths. So each user of this mechanism inherits from BarrierSetNMethod >>>> and puts its barrier in there. >>>> >>>> The OSR nmethods do not have a verified entry. Therefore, the barrier >>>> is run explicitly in the runtime before calling into OSR nmethods to >>>> achieve the same effect of trapping new calls into nmethods. >>>> >>>> The nmethod entry barrier returns true/false depending on whether the >>>> called nmethod may be entered or not. This allows lazily cleaning >>>> nmethod IC caches during unloading, which invokes the same helper >>>> method that is patched into the verified entry when made not entrant. >>>> >>>> Eventually, I would like to also use this mechanism to remove the >>>> nmethod hotness counter maintenance we today perform in safepoint >>>> cleanup instead. These nmethod entry barriers can provide better >>>> heuristics for whether nmethods are used or not. But that would >>>> require buy-in from maintainers of other platforms. So let's take one >>>> step at a time. >>>> >>>> For the curious reader thinking that this might be an expensive >>>> operation, it is not. This check in the nmethod verified entry is very >>>> predictable and cheap. And because of use of inlining, the checks >>>> become more infrequent. In my evaluation, it has gone completely under >>>> the radar in benchmarking (running this with my current version of >>>> concurrent class unloading in ZGC). >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >>> I reviewed this before Erik sent it out, so my comments have already >>> been addressed. Just found one additional thing, the BarrierSet >>> constructor now looks like this: >>> >>> 96?? BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>> 97????????????? BarrierSetC1* barrier_set_c1, >>> 98????????????? BarrierSetC2* barrier_set_c2, >>> 99????????????? const FakeRtti& fake_rtti, >>> 100???????????? BarrierSetNMethod* barrier_set_nmethod = NULL) : >>> 101???? _fake_rtti(fake_rtti), >>> 102???? _barrier_set_assembler(barrier_set_assembler), >>> 103???? _barrier_set_c1(barrier_set_c1), >>> 104???? _barrier_set_c2(barrier_set_c2), >>> 105???? _barrier_set_nmethod(barrier_set_nmethod) {} >>> >>> I find it odd that the fake_rtti argument comes in the middle of the >>> barrier_set_* arguments. I can see that you don't want GC barrier sets >>> to have to supply NULL if they don't have a nmethod barrier set, but >>> maybe we can have two constructors instead, like: >>> >>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>> ??????????? BarrierSetC1* barrier_set_c1, >>> ??????????? BarrierSetC2* barrier_set_c2, >>> ??????????? const FakeRtti& fake_rtti) : >>> ???? _fake_rtti(fake_rtti), >>> ???? _barrier_set_assembler(barrier_set_assembler), >>> ???? _barrier_set_c1(barrier_set_c1), >>> ???? _barrier_set_c2(barrier_set_c2), >>> ???? _barrier_set_nmethod(NULL) {} >>> >>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>> ??????????? BarrierSetC1* barrier_set_c1, >>> ??????????? BarrierSetC2* barrier_set_c2, >>> ??????????? BarrierSetNMethod* barrier_set_nmethod, >>> ??????????? const FakeRtti& fake_rtti,) : >>> ???? _fake_rtti(fake_rtti), >>> ???? _barrier_set_assembler(barrier_set_assembler), >>> ???? _barrier_set_c1(barrier_set_c1), >>> ???? _barrier_set_c2(barrier_set_c2), >>> ???? _barrier_set_nmethod(barrier_set_nmethod) {} >>> >>> What do you think? >>> >>> Other than that, this looks good to me. >>> >>> cheers, >>> Per >>> >>>> Bug ID: >>>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>>> >>>> Thanks, >>>> /Erik > From kim.barrett at oracle.com Mon Oct 8 18:32:02 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Oct 2018 14:32:02 -0400 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: <2C15D9F7-6403-4FCA-806B-263FEF6F0DBA@oracle.com> > On Oct 8, 2018, at 6:48 AM, Thomas St?fe wrote: > > Hi Kim, > > is this JEP only about C++14 features or shall we discuss older > features too? The reason I am asking is that I would like us to > officially endorse namespaces. Not inline namespaces, just plain old > namespaces. I would like to keep this JEP focused on C++11/14 usage. However, it presently describes a process for recording and updating HotSpot usage. I think that process is pretty close to the de facto process, which is rarely used successfully because it's not written down anywhere. We could extract that part of the JEP out, formalize, discuss, agree, and record it. Then you could propose a change and there would be a process for dealing with the proposal, rather than having it slide into oblivion because we don't know how to proceed. And the JEP could be made a bit smaller because it could just refer to that process. From kim.barrett at oracle.com Mon Oct 8 19:34:04 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Oct 2018 15:34:04 -0400 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <10d8f2ea-883e-ec21-4fea-06a52a52fdc6@bell-sw.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> <10d8f2ea-883e-ec21-4fea-06a52a52fdc6@bell-sw.com> Message-ID: <407CE450-6EED-4955-9B1F-EA564E134D11@oracle.com> > On Oct 6, 2018, at 11:07 AM, Aleksei Voitylov wrote: > > Kim, > > from an ARM port perspective, the stable GCC version is 4.9. The port compiles with stock GCC 7.3 but a lot of testing is required before moving to GCC 7.3. I agree on the overall direction and we'll commit resources to testing it further, but from an ARM port perspective it may happen JDK 12 is a little too optimistic. > > GCC x86 is a lot more stable than for ARM32 and AARCH64. It seems to me that JDK 11 being an LTS might provide an answer to this. JDK 12 support for C++11/14 on arm32/aarch64 might be somewhat "experimental" in that it might require a more recent compiler version that hasn't received as much testing as was done for JDK 11. Similarly, the AIX/ppc platform might not be supported after JDK 11 until an improved version of the relevant compiler becomes available. I don't know if such an approach is acceptable to the community, nor do I know how such a decision might be made. But I think it would be unfortunate if such issues blocked progress in this area. From thomas.stuefe at gmail.com Mon Oct 8 19:42:47 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 8 Oct 2018 21:42:47 +0200 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <2C15D9F7-6403-4FCA-806B-263FEF6F0DBA@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <2C15D9F7-6403-4FCA-806B-263FEF6F0DBA@oracle.com> Message-ID: Hi Kim, On Mon, Oct 8, 2018 at 8:32 PM Kim Barrett wrote: > > > On Oct 8, 2018, at 6:48 AM, Thomas St?fe wrote: > > > > Hi Kim, > > > > is this JEP only about C++14 features or shall we discuss older > > features too? The reason I am asking is that I would like us to > > officially endorse namespaces. Not inline namespaces, just plain old > > namespaces. > > I would like to keep this JEP focused on C++11/14 usage. > > However, it presently describes a process for recording and updating > HotSpot usage. I think that process is pretty close to the de facto > process, which is rarely used successfully because it's not written > down anywhere. > > We could extract that part of the JEP out, formalize, discuss, agree, > and record it. Then you could propose a change and there would be a > process for dealing with the proposal, rather than having it slide > into oblivion because we don't know how to proceed. And the JEP could > be made a bit smaller because it could just refer to that process. > Yes, I would like that. It would be really good to have a process in place to preserve and actually live a consensus about what features we use. We have the Hotspot Style Guide, but we let it become woefully outdated in the face of new developments. I always smile when I read the "Be sparing with templates." remark - that ship has sailed long ago. --- A small remark to the text of your JEP: "As a rule of thumb, permitting features which simplify writing, and especially reading, code should be encouraged." I would like to add to that that simplifying build error analysis should also be a goal. That directly influences programmer productivity. We had some cases in the recent past where we spent a lot of time scratching our heads over walls of template related compiler errors. (not sure how to reach this goal though, since this seems to be a problem inherent in using C++ templates). Best Regards, Thomas From aleksei.voitylov at bell-sw.com Mon Oct 8 20:45:07 2018 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Mon, 8 Oct 2018 23:45:07 +0300 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <407CE450-6EED-4955-9B1F-EA564E134D11@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> <10d8f2ea-883e-ec21-4fea-06a52a52fdc6@bell-sw.com> <407CE450-6EED-4955-9B1F-EA564E134D11@oracle.com> Message-ID: Kim, Let's not underestimate the importance of continuous testing throughout the release cycle. Over the last year alternative platforms and configurations were broken accidentally not once (and I think the number is reaching 50 or something). Suggesting a platform to be "not supported for a release or two" may mean by the time the compiler is good the amount of issues to fix for a platform to regain quality may become a blocker for the next LTS. I really hope this is not the option Oracle is proposing. We both know what and how long it takes to do a thorough OpenJDK compiler upgrade. If the community were made aware of this earlier, I would have definitely started planning for a compiler upgrade for ARM port in JDK 11. With that, I conditionally agree with the proposal provided cooperating ports are given sufficient lead time to upgrade. We started testing ARM with 7.3 and will report on success. -Aleksei On 08/10/2018 22:34, Kim Barrett wrote: >> On Oct 6, 2018, at 11:07 AM, Aleksei Voitylov wrote: >> >> Kim, >> >> from an ARM port perspective, the stable GCC version is 4.9. The port compiles with stock GCC 7.3 but a lot of testing is required before moving to GCC 7.3. I agree on the overall direction and we'll commit resources to testing it further, but from an ARM port perspective it may happen JDK 12 is a little too optimistic. >> >> GCC x86 is a lot more stable than for ARM32 and AARCH64. > It seems to me that JDK 11 being an LTS might provide an answer to this. > > JDK 12 support for C++11/14 on arm32/aarch64 might be somewhat > "experimental" in that it might require a more recent compiler version > that hasn't received as much testing as was done for JDK 11. > > Similarly, the AIX/ppc platform might not be supported after JDK 11 > until an improved version of the relevant compiler becomes available. > > I don't know if such an approach is acceptable to the community, nor > do I know how such a decision might be made. But I think it would be > unfortunate if such issues blocked progress in this area. > From david.holmes at oracle.com Mon Oct 8 21:26:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 9 Oct 2018 07:26:56 +1000 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> <6c34f766-eb48-cd18-1d0d-c202b0b4052d@oracle.com> <4fa520f3-1341-ade8-79a1-962c0d5d7f5f@oracle.com> <6b1c457d-321b-f8a9-4b3a-44c79e0e8987@oracle.com> Message-ID: <51f81db2-6fe2-e2a6-f4af-c8326ea3d1d9@oracle.com> Hi Severin, On 9/10/2018 2:20 AM, Severin Gehwolf wrote: > Hi! > > On Sun, 2018-10-07 at 07:52 +1000, David Holmes wrote: >> On 7/10/2018 6:11 AM, dean.long at oracle.com wrote: >>> On 10/5/18 5:18 PM, David Holmes wrote: >>>> On 6/10/2018 12:18 AM, Andrew Haley wrote: >>>>> On 10/05/2018 02:40 PM, Severin Gehwolf wrote: >>>>>>> Also I thought that if ldrex was not paired with a strex then you >>>>>>> should >>>>>>> at least issue clrex - as done in generate_atomic_load_long() in >>>>>>> stubGenerator_arm.cpp >>>>>> >>>>>> I'll defer to Andrew Haley for this. >>>>> >>>>> I'm skeptical that CLREX is necessary. The classic CAS for Arm is >>>>> >>>>> MOV R1, #0xFF ; load the ?lock taken? value >>>>> try LDREX R0, [LockAddr] ; load the lock value >>>>> CMP R0, #0 ; is the lock free? >>>>> STREXEQ R1, R0, [LockAddr]; try and claim the lock >>>>> CMPEQ R0, #0 ; did this succeed? >>>>> BNE try ; no ? try again. . . >>>>> ; yes ? we have the lock >>>>> >>>>> There's no CLREX, and no-one ever reported any problem. CLREX could be >>>>> a performance boost because it takes the cache line out of exclusive >>>>> state back to shared state, reducing later bus traffic. >>>> >>>> I would see CLREX as a potential performance boost not something >>>> needed for correctness. But you're right that failure paths from >>>> typical CAS code won't execute the strex and don't include a clrex. >>>> >>> >>> I think it's needed for correctness only at context-switch time. If you >>> context switch after the LDREX, you don't want the exclusive state to >>> remain for the next thread scheduled. >> >> See also: >> >> https://community.arm.com/processors/f/discussions/6572/synchronization-primitives-do-i-need-clrex >> >> Seems if CLREX is needed for correctness it is at the OS level. >> >> On performance concern see: >> >> http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20150921/300951.html >> >> Bottom line seems to be: >> - clrex is not essential >> - clrex is probably not harmful to performance >> - clrex may benefit performance > > OK. David, how would you like me to update the webrev then? Based on other responses your original is probably fine, but as you've already gone to the trouble ... > 2a) > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8211387/webrev.02_a/ this one is fine. Thanks, David > 2b) > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8211387/webrev.02_b/ > > FWIW, the performance gain for Zero is probably negligible, but it > probably won't hurt either... > > Thanks, > Severin > From erik.osterlund at oracle.com Tue Oct 9 07:24:54 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 9 Oct 2018 09:24:54 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <0b99da49-b118-1db8-d21c-3213fdfffe36@oracle.com> References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> <5BBB1328.3020503@oracle.com> <0b99da49-b118-1db8-d21c-3213fdfffe36@oracle.com> Message-ID: <7c0fafa2-831b-fdce-2a33-cacb7921395f@oracle.com> Hi Vladimir, Thank you for reviewing this. On 2018-10-08 19:57, Vladimir Kozlov wrote: > First, all nmethods have verified entry. MachPrologNode is generated > by C2 for all nmethods including OSR (we need stack bang in all > cases). OSR does not have *Unverified* entry represented by > MachUEPNode. C1 compiled OSR nmethods are different - you may need > your changes for it but I am not sure - ask C1 experts. > > There could be complications if an OSR nmethods is not executed (after > barrier check) due to state of stack on OSR entry but it is different > issue. I should have been more precise. OSR nmethods have a prologue for building frames, yes. But OSR entries do not have a verified entry offset, in the sense that in e.g. c1_LIRAssembler.cpp, no CodeOffsets::Verified_Entry entry is set for OSR entries (the lir_osr_entry case). It is set only for lir_std_entry. Similarly for C2, while they both have a prologue, there is no CodeOffsets::Verified_Entry set for OSR nmethods. Read for example opto/compile.cpp: ??? if (is_osr_compilation()) { ????? _code_offsets.set_value(CodeOffsets::Verified_Entry, 0); ????? _code_offsets.set_value(CodeOffsets::OSR_Entry, _first_block_size); ??? } else { ????? _code_offsets.set_value(CodeOffsets::Verified_Entry, _first_block_size); ????? _code_offsets.set_value(CodeOffsets::OSR_Entry, 0); ??? } The different treatment of OSR methods for the entry barrier is analogous to the different treatment of OSR methods when making them not entrant. OSR nmethods are made not entrant by detaching them from the Method* (so the runtime won't find them), while non-OSR nmethods are made not entrant by patching a jump to the SharedRuntime::get_handle_wrong_method_stub() into the verified entry. And that handler stub has particular ABI requirements on the call site, that do not match the OSR call site. Similarly for nmethod entry barriers, if the callee is an OSR nmethod, it is much easier to check in the runtime before the call if said nmethod is toast or not, than it is to check when the nmethod has entered, and then figure out how to deoptimize from inside of the nmethod. > I am curious why you are using next 2 lines everywhere? > > +? BarrierSetAssembler* bs = > BarrierSet::barrier_set()->barrier_set_assembler(); > +? bs->nmethod_entry_barrier(this); > > instead of one call: > > BarrierSet::barrier_set()->nmethod_entry_barrier(this); Because we never make straight calls to BarrierSet for inserting barriers. Instead, we use one out of the (currently) 4 helpers that have their own responsibility. The BarrierSetAssembler is responsible for inserting barriers into assembly code. > The same question about duplicated code in interpreterRuntime.cpp Same answer. ;) > Would be nice if you also update MachPrologNode::format() for case > when barrier is used. It is used in PrintOptoAssembly. Yes, that is a good idea. I will adjust the printing code. Thanks, /Erik > Thanks, > Vladimir > > On 10/8/18 1:19 AM, Erik ?sterlund wrote: >> Hi Per and Roman, >> >> Thank you for having a look at this. >> I went with Roman's suggestion and made this an explicit opt-out >> instead. But I let ModRef pass in NULL, as none of the barrier sets >> in that category currently use nmethod entry barriers. When they >> start to do so, that can be easily changed. >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ >> >> Full: >> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ >> >> Thanks, >> /Erik >> >> On 2018-10-08 08:48, Roman Kennke wrote: >>> I agree. I wouldn't mind to keep it simple and pass NULL though. I'm >>> not >>> a big fan of default arguments anyway. >>> >>> Roman >>> >>>> On 2018-10-05 17:07, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> In order to implement concurrent class unloading for ZGC, a mechanism >>>>> will be required to arm nmethods in a safepoint such that the first >>>>> entry into an nmethod after the pause triggers a barrier before >>>>> continuing. >>>>> >>>>> The barrier will be used for: >>>>> ??? * Patching immediate oops, and keeping phantomly reachable oops >>>>> alive through the nmethod entry >>>>> ??? * Catching calls from stale IC caches (due to class unloading) >>>>> into >>>>> nmethods that are being unloaded (containing broken oops and >>>>> metadata), and lazily clean the IC cache and re-resolve the call. >>>>> >>>>> This implementation is for x86_64 and is based on the prototyping >>>>> work >>>>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>>>> this patch. >>>>> The way it works is that there is a cmp #offset(r15_thread), >>>>> simm32_epoch); je #continuation; inserted into the verified entry of >>>>> nmethods that conditionally calls a stub trampolining into the VM. >>>>> When the simm32_epoch is not equal to some TLS-local field, then a >>>>> slow path is taken to run some barrier inside of the VM. >>>>> By changing this TLS-local field in the safepoint, all verified >>>>> entries will take the slow path into the VM when run, where the >>>>> barrier is eventually disarmed by patching the simm32_epoch value of >>>>> the instruction stream to the current globally correct value. >>>>> >>>>> In terms of abstractions, to utilize this mechanism, you need to >>>>> create your BarierSet with an BarrierSetNMethod helper object, just >>>>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>>>> BarrierSetNMethod base class handles the barrier itself. It has a >>>>> virtual function, nmethod_entry_barrier, which gets called by the >>>>> slow >>>>> paths. So each user of this mechanism inherits from BarrierSetNMethod >>>>> and puts its barrier in there. >>>>> >>>>> The OSR nmethods do not have a verified entry. Therefore, the barrier >>>>> is run explicitly in the runtime before calling into OSR nmethods to >>>>> achieve the same effect of trapping new calls into nmethods. >>>>> >>>>> The nmethod entry barrier returns true/false depending on whether the >>>>> called nmethod may be entered or not. This allows lazily cleaning >>>>> nmethod IC caches during unloading, which invokes the same helper >>>>> method that is patched into the verified entry when made not entrant. >>>>> >>>>> Eventually, I would like to also use this mechanism to remove the >>>>> nmethod hotness counter maintenance we today perform in safepoint >>>>> cleanup instead. These nmethod entry barriers can provide better >>>>> heuristics for whether nmethods are used or not. But that would >>>>> require buy-in from maintainers of other platforms. So let's take one >>>>> step at a time. >>>>> >>>>> For the curious reader thinking that this might be an expensive >>>>> operation, it is not. This check in the nmethod verified entry is >>>>> very >>>>> predictable and cheap. And because of use of inlining, the checks >>>>> become more infrequent. In my evaluation, it has gone completely >>>>> under >>>>> the radar in benchmarking (running this with my current version of >>>>> concurrent class unloading in ZGC). >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >>>> I reviewed this before Erik sent it out, so my comments have already >>>> been addressed. Just found one additional thing, the BarrierSet >>>> constructor now looks like this: >>>> >>>> 96?? BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>> 97????????????? BarrierSetC1* barrier_set_c1, >>>> 98????????????? BarrierSetC2* barrier_set_c2, >>>> 99????????????? const FakeRtti& fake_rtti, >>>> 100???????????? BarrierSetNMethod* barrier_set_nmethod = NULL) : >>>> 101???? _fake_rtti(fake_rtti), >>>> 102???? _barrier_set_assembler(barrier_set_assembler), >>>> 103???? _barrier_set_c1(barrier_set_c1), >>>> 104???? _barrier_set_c2(barrier_set_c2), >>>> 105???? _barrier_set_nmethod(barrier_set_nmethod) {} >>>> >>>> I find it odd that the fake_rtti argument comes in the middle of the >>>> barrier_set_* arguments. I can see that you don't want GC barrier sets >>>> to have to supply NULL if they don't have a nmethod barrier set, but >>>> maybe we can have two constructors instead, like: >>>> >>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>> ??????????? BarrierSetC1* barrier_set_c1, >>>> ??????????? BarrierSetC2* barrier_set_c2, >>>> ??????????? const FakeRtti& fake_rtti) : >>>> ???? _fake_rtti(fake_rtti), >>>> ???? _barrier_set_assembler(barrier_set_assembler), >>>> ???? _barrier_set_c1(barrier_set_c1), >>>> ???? _barrier_set_c2(barrier_set_c2), >>>> ???? _barrier_set_nmethod(NULL) {} >>>> >>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>> ??????????? BarrierSetC1* barrier_set_c1, >>>> ??????????? BarrierSetC2* barrier_set_c2, >>>> ??????????? BarrierSetNMethod* barrier_set_nmethod, >>>> ??????????? const FakeRtti& fake_rtti,) : >>>> ???? _fake_rtti(fake_rtti), >>>> ???? _barrier_set_assembler(barrier_set_assembler), >>>> ???? _barrier_set_c1(barrier_set_c1), >>>> ???? _barrier_set_c2(barrier_set_c2), >>>> ???? _barrier_set_nmethod(barrier_set_nmethod) {} >>>> >>>> What do you think? >>>> >>>> Other than that, this looks good to me. >>>> >>>> cheers, >>>> Per >>>> >>>>> Bug ID: >>>>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>>>> >>>>> Thanks, >>>>> /Erik >> From sgehwolf at redhat.com Tue Oct 9 08:26:51 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 09 Oct 2018 10:26:51 +0200 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: <51f81db2-6fe2-e2a6-f4af-c8326ea3d1d9@oracle.com> References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> <6c34f766-eb48-cd18-1d0d-c202b0b4052d@oracle.com> <4fa520f3-1341-ade8-79a1-962c0d5d7f5f@oracle.com> <6b1c457d-321b-f8a9-4b3a-44c79e0e8987@oracle.com> <51f81db2-6fe2-e2a6-f4af-c8326ea3d1d9@oracle.com> Message-ID: <6b8436c73be2300ae50a29bb749de930c6a875a6.camel@redhat.com> On Tue, 2018-10-09 at 07:26 +1000, David Holmes wrote: > > OK. David, how would you like me to update the webrev then? > > Based on other responses your original is probably fine, but as you've > already gone to the trouble ... > > > 2a) > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8211387/webrev.02_a/ > > this one is fine. Thanks, David! Cheers, Severin From aph at redhat.com Tue Oct 9 08:28:27 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 9 Oct 2018 09:28:27 +0100 Subject: RFR(xs): 8211387: [Zero] atomic_copy64: Use ldrexd for atomic reads on ARMv7 In-Reply-To: <6b1c457d-321b-f8a9-4b3a-44c79e0e8987@oracle.com> References: <6bba6407936ae161fb3463b15f30fe1d66b76bd6.camel@redhat.com> <2a5e103e-53f8-199a-be3e-4c0ae3175de0@oracle.com> <6c34f766-eb48-cd18-1d0d-c202b0b4052d@oracle.com> <4fa520f3-1341-ade8-79a1-962c0d5d7f5f@oracle.com> <6b1c457d-321b-f8a9-4b3a-44c79e0e8987@oracle.com> Message-ID: <63748f77-72fc-0191-0ab9-a37425c1c98e@redhat.com> On 10/06/2018 10:52 PM, David Holmes wrote: > Bottom line seems to be: > - clrex is not essential > - clrex is probably not harmful to performance > - clrex may benefit performance clrex may also make performance worse: it's another instruction, after all. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From gunter.haug at sap.com Tue Oct 9 13:08:38 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Tue, 9 Oct 2018 13:08:38 +0000 Subject: [RFR]: [S390] Implement JFR profiling In-Reply-To: References: Message-ID: Hi Volker, Martin! Here is an updated webrev: http://cr.openjdk.java.net/~ghaug/webrevs/8211768.v2/ It contains all your suggestions. Volker, would you be so kind and push it? Thanks, Gunter ?On 08.10.18, 15:23, "Volker Simonis" wrote: Hi Gunter, thanks for fixing this issue on s390. In general your change looks good. Just some minor comments: thread_linux_s390.cpp ================= + // nothing else to try + return false; + + ucontext_t* uc = (ucontext_t*) ucontext; return true; } I think the last two lines after "return false" are a leftover from the old version and should be removed. Strange you didn't got a compiler warning because of unreachable code. frame_s390.cpp ============ + // reliable. Unfortunately we can only check frame completeness for + // runtime stubs and nmethods. Other generic buffer blobs are more + // problematic so we just assume they are OK. Adapter blobs never have a + // complete frame and are never OK + if (!_cb->is_frame_complete_at(_pc)) { + if (_cb->is_adapter_blob() || _cb->is_runtime_stub()) { In the comment you mention that we can only check completeness for runtime stubs and nmethods but later in the code you only check for the stubs case and not for nmethods. Which is right, the code or the comment? frame_s390.hpp ============ I think from a philosophical point of view it is unfortunate, that the ijava_state_unsafe() is "public" while ijava_state() is "private", but I don't mind if you don't change this any more :) Regards, Volker On Mon, Oct 8, 2018 at 2:48 PM Haug, Gunter wrote: > > Hi all, > > can I please have reviews and a sponsor fort the following change: > > https://bugs.openjdk.java.net/browse/JDK-8211768 > http://cr.openjdk.java.net/~ghaug/webrevs/8211768/ > > The implementation is basically the same as for PPC. It was only adapted for the different use of registers and the slightly different frame layout of s390. The respective tests don't fail anymore on s390 with this patch. > > Thanks, > Gunter > > > > From matthias.baesken at sap.com Tue Oct 9 13:51:19 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 9 Oct 2018 13:51:19 +0000 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 Message-ID: Hello, please review this gcc 7 related build fix . When building on Linux with gcc 7.3.1 I run into this compile error : /open_jdk/jdk_just_clone/jdk/src/hotspot/share/opto/parse2.cpp:1349:14: note: 'sprintf' output 2 or more bytes (assuming 2147488583) into a destination of size 30 sprintf(prob_str_buf, "%g", prob); ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ cc1plus: all warnings being treated as errors The issue is that we print a floating point number into a fixed size buffer for output . I adjusted the coding a bit : http://cr.openjdk.java.net/~mbaesken/webrevs/8211929.0/ https://bugs.openjdk.java.net/browse/JDK-8211929 Best regards, Matthias From vladimir.kozlov at oracle.com Tue Oct 9 14:42:03 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 9 Oct 2018 07:42:03 -0700 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <7c0fafa2-831b-fdce-2a33-cacb7921395f@oracle.com> References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> <5BBB1328.3020503@oracle.com> <0b99da49-b118-1db8-d21c-3213fdfffe36@oracle.com> <7c0fafa2-831b-fdce-2a33-cacb7921395f@oracle.com> Message-ID: <23ac864d-44b4-a54f-b2b2-869899553cfd@oracle.com> On 10/9/18 12:24 AM, Erik ?sterlund wrote: > Hi Vladimir, > > Thank you for reviewing this. > > On 2018-10-08 19:57, Vladimir Kozlov wrote: >> First, all nmethods have verified entry. MachPrologNode is generated by C2 for all nmethods including OSR (we need >> stack bang in all cases). OSR does not have *Unverified* entry represented by MachUEPNode. C1 compiled OSR nmethods >> are different - you may need your changes for it but I am not sure - ask C1 experts. >> >> There could be complications if an OSR nmethods is not executed (after barrier check) due to state of stack on OSR >> entry but it is different issue. > > I should have been more precise. OSR nmethods have a prologue for building frames, yes. But OSR entries do not have a > verified entry offset, in the sense that in e.g. c1_LIRAssembler.cpp, no CodeOffsets::Verified_Entry entry is set for > OSR entries (the lir_osr_entry case). It is set only for lir_std_entry. Similarly for C2, while they both have a > prologue, there is no CodeOffsets::Verified_Entry set for OSR nmethods. Read for example opto/compile.cpp: > > ??? if (is_osr_compilation()) { > ????? _code_offsets.set_value(CodeOffsets::Verified_Entry, 0); > ????? _code_offsets.set_value(CodeOffsets::OSR_Entry, _first_block_size); > ??? } else { > ????? _code_offsets.set_value(CodeOffsets::Verified_Entry, _first_block_size); > ????? _code_offsets.set_value(CodeOffsets::OSR_Entry, 0); > ??? } > > The different treatment of OSR methods for the entry barrier is analogous to the different treatment of OSR methods when > making them not entrant. OSR nmethods are made not entrant by detaching them from the Method* (so the runtime won't find > them), while non-OSR nmethods are made not entrant by patching a jump to the > SharedRuntime::get_handle_wrong_method_stub() into the verified entry. And that handler stub has particular ABI > requirements on the call site, that do not match the OSR call site. > > Similarly for nmethod entry barriers, if the callee is an OSR nmethod, it is much easier to check in the runtime before > the call if said nmethod is toast or not, than it is to check when the nmethod has entered, and then figure out how to > deoptimize from inside of the nmethod. Good. Note, I did not argued about changes for OSR nmethods, I was surprise by statement. Your explanation cleared it. > >> I am curious why you are using next 2 lines everywhere? >> >> +? BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); >> +? bs->nmethod_entry_barrier(this); >> >> instead of one call: >> >> BarrierSet::barrier_set()->nmethod_entry_barrier(this); > > Because we never make straight calls to BarrierSet for inserting barriers. Instead, we use one out of the (currently) 4 > helpers that have their own responsibility. The BarrierSetAssembler is responsible for inserting barriers into assembly > code. So it is by GC Interface design? Okay. > >> The same question about duplicated code in interpreterRuntime.cpp > > Same answer. ;) > >> Would be nice if you also update MachPrologNode::format() for case when barrier is used. It is used in PrintOptoAssembly. > > Yes, that is a good idea. I will adjust the printing code. Thanks, Vladimir > > Thanks, > /Erik > >> Thanks, >> Vladimir >> >> On 10/8/18 1:19 AM, Erik ?sterlund wrote: >>> Hi Per and Roman, >>> >>> Thank you for having a look at this. >>> I went with Roman's suggestion and made this an explicit opt-out instead. But I let ModRef pass in NULL, as none of >>> the barrier sets in that category currently use nmethod entry barriers. When they start to do so, that can be easily >>> changed. >>> >>> Incremental: >>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ >>> >>> Full: >>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ >>> >>> Thanks, >>> /Erik >>> >>> On 2018-10-08 08:48, Roman Kennke wrote: >>>> I agree. I wouldn't mind to keep it simple and pass NULL though. I'm not >>>> a big fan of default arguments anyway. >>>> >>>> Roman >>>> >>>>> On 2018-10-05 17:07, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> In order to implement concurrent class unloading for ZGC, a mechanism >>>>>> will be required to arm nmethods in a safepoint such that the first >>>>>> entry into an nmethod after the pause triggers a barrier before >>>>>> continuing. >>>>>> >>>>>> The barrier will be used for: >>>>>> ??? * Patching immediate oops, and keeping phantomly reachable oops >>>>>> alive through the nmethod entry >>>>>> ??? * Catching calls from stale IC caches (due to class unloading) into >>>>>> nmethods that are being unloaded (containing broken oops and >>>>>> metadata), and lazily clean the IC cache and re-resolve the call. >>>>>> >>>>>> This implementation is for x86_64 and is based on the prototyping work >>>>>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>>>>> this patch. >>>>>> The way it works is that there is a cmp #offset(r15_thread), >>>>>> simm32_epoch); je #continuation; inserted into the verified entry of >>>>>> nmethods that conditionally calls a stub trampolining into the VM. >>>>>> When the simm32_epoch is not equal to some TLS-local field, then a >>>>>> slow path is taken to run some barrier inside of the VM. >>>>>> By changing this TLS-local field in the safepoint, all verified >>>>>> entries will take the slow path into the VM when run, where the >>>>>> barrier is eventually disarmed by patching the simm32_epoch value of >>>>>> the instruction stream to the current globally correct value. >>>>>> >>>>>> In terms of abstractions, to utilize this mechanism, you need to >>>>>> create your BarierSet with an BarrierSetNMethod helper object, just >>>>>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>>>>> BarrierSetNMethod base class handles the barrier itself. It has a >>>>>> virtual function, nmethod_entry_barrier, which gets called by the slow >>>>>> paths. So each user of this mechanism inherits from BarrierSetNMethod >>>>>> and puts its barrier in there. >>>>>> >>>>>> The OSR nmethods do not have a verified entry. Therefore, the barrier >>>>>> is run explicitly in the runtime before calling into OSR nmethods to >>>>>> achieve the same effect of trapping new calls into nmethods. >>>>>> >>>>>> The nmethod entry barrier returns true/false depending on whether the >>>>>> called nmethod may be entered or not. This allows lazily cleaning >>>>>> nmethod IC caches during unloading, which invokes the same helper >>>>>> method that is patched into the verified entry when made not entrant. >>>>>> >>>>>> Eventually, I would like to also use this mechanism to remove the >>>>>> nmethod hotness counter maintenance we today perform in safepoint >>>>>> cleanup instead. These nmethod entry barriers can provide better >>>>>> heuristics for whether nmethods are used or not. But that would >>>>>> require buy-in from maintainers of other platforms. So let's take one >>>>>> step at a time. >>>>>> >>>>>> For the curious reader thinking that this might be an expensive >>>>>> operation, it is not. This check in the nmethod verified entry is very >>>>>> predictable and cheap. And because of use of inlining, the checks >>>>>> become more infrequent. In my evaluation, it has gone completely under >>>>>> the radar in benchmarking (running this with my current version of >>>>>> concurrent class unloading in ZGC). >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >>>>> I reviewed this before Erik sent it out, so my comments have already >>>>> been addressed. Just found one additional thing, the BarrierSet >>>>> constructor now looks like this: >>>>> >>>>> 96?? BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>> 97????????????? BarrierSetC1* barrier_set_c1, >>>>> 98????????????? BarrierSetC2* barrier_set_c2, >>>>> 99????????????? const FakeRtti& fake_rtti, >>>>> 100???????????? BarrierSetNMethod* barrier_set_nmethod = NULL) : >>>>> 101???? _fake_rtti(fake_rtti), >>>>> 102???? _barrier_set_assembler(barrier_set_assembler), >>>>> 103???? _barrier_set_c1(barrier_set_c1), >>>>> 104???? _barrier_set_c2(barrier_set_c2), >>>>> 105???? _barrier_set_nmethod(barrier_set_nmethod) {} >>>>> >>>>> I find it odd that the fake_rtti argument comes in the middle of the >>>>> barrier_set_* arguments. I can see that you don't want GC barrier sets >>>>> to have to supply NULL if they don't have a nmethod barrier set, but >>>>> maybe we can have two constructors instead, like: >>>>> >>>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>> ??????????? BarrierSetC1* barrier_set_c1, >>>>> ??????????? BarrierSetC2* barrier_set_c2, >>>>> ??????????? const FakeRtti& fake_rtti) : >>>>> ???? _fake_rtti(fake_rtti), >>>>> ???? _barrier_set_assembler(barrier_set_assembler), >>>>> ???? _barrier_set_c1(barrier_set_c1), >>>>> ???? _barrier_set_c2(barrier_set_c2), >>>>> ???? _barrier_set_nmethod(NULL) {} >>>>> >>>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>> ??????????? BarrierSetC1* barrier_set_c1, >>>>> ??????????? BarrierSetC2* barrier_set_c2, >>>>> ??????????? BarrierSetNMethod* barrier_set_nmethod, >>>>> ??????????? const FakeRtti& fake_rtti,) : >>>>> ???? _fake_rtti(fake_rtti), >>>>> ???? _barrier_set_assembler(barrier_set_assembler), >>>>> ???? _barrier_set_c1(barrier_set_c1), >>>>> ???? _barrier_set_c2(barrier_set_c2), >>>>> ???? _barrier_set_nmethod(barrier_set_nmethod) {} >>>>> >>>>> What do you think? >>>>> >>>>> Other than that, this looks good to me. >>>>> >>>>> cheers, >>>>> Per >>>>> >>>>>> Bug ID: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>> > From matthias.baesken at sap.com Tue Oct 9 14:55:17 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 9 Oct 2018 14:55:17 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Thomas , thanks for filing the CSR . I am fine with the name proposal "-XX:+-ExtensiveErrorReports", but will wait until it is agreed to use this name . Then I'll post another webrev of 8211326 using the flag . Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Montag, 8. Oktober 2018 12:21 > To: David Holmes > Cc: Baesken, Matthias ; Langer, Christoph > ; Volker Simonis ; > HotSpot Open Source Developers > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > On Mon, Oct 8, 2018 at 11:50 AM David Holmes > wrote: > > > > Hi Thomas, > > > > On 8/10/2018 7:43 PM, Thomas St?fe wrote: > > > Hi all, > > > > > > I drafted a CSR: > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > > > > > Not sure what to do now, should I open a discussion thread on a > > > mailing list? Or just wait for the committee to decide? (I'm doing > > > this too rarely..) > > > > I filled in some of the additional info needed. The text needs to be > > completed and the CSR needs to have at least one reviewer added to it. > > Then it can be marked "final" and ready for the committee to evaluate. > > > > Thanks! I filled out what was missing and will wait for reviewers. > > Thanks, Thomas > From vladimir.kozlov at oracle.com Tue Oct 9 15:06:17 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 9 Oct 2018 08:06:17 -0700 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 In-Reply-To: References: Message-ID: <03f8fa68-a758-d956-cfd3-1572b093d3f9@oracle.com> Looks good. Why there is no complain about sprintf at line 2857 which uses %s? : http://hg.openjdk.java.net/jdk/jdk/file/d24b89390f6c/src/hotspot/share/opto/parse2.cpp#l2857 Thanks, Vladimir On 10/9/18 6:51 AM, Baesken, Matthias wrote: > Hello, please review this gcc 7 related build fix . > > > When building on Linux with gcc 7.3.1 I run into this compile error : > > /open_jdk/jdk_just_clone/jdk/src/hotspot/share/opto/parse2.cpp:1349:14: note: 'sprintf' output 2 or more bytes (assuming 2147488583) into a destination of size 30 > sprintf(prob_str_buf, "%g", prob); > ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ > cc1plus: all warnings being treated as errors > > > The issue is that we print a floating point number into a fixed size buffer for output . > > I adjusted the coding a bit : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211929.0/ > > https://bugs.openjdk.java.net/browse/JDK-8211929 > > > > Best regards, Matthias > From erik.osterlund at oracle.com Tue Oct 9 15:52:20 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 9 Oct 2018 17:52:20 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <23ac864d-44b4-a54f-b2b2-869899553cfd@oracle.com> References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> <5BBB1328.3020503@oracle.com> <0b99da49-b118-1db8-d21c-3213fdfffe36@oracle.com> <7c0fafa2-831b-fdce-2a33-cacb7921395f@oracle.com> <23ac864d-44b4-a54f-b2b2-869899553cfd@oracle.com> Message-ID: <5BBCCEB4.6030104@oracle.com> Hi Vladimir, Sounds like we are in agreement. Thanks for looking into this change. I updated the webrev with the added assembly printing for C2 as requested. Incremental: http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01_02/ Full: http://cr.openjdk.java.net/~eosterlund/8210498/webrev.02/ Thanks, /Erik On 2018-10-09 16:42, Vladimir Kozlov wrote: > On 10/9/18 12:24 AM, Erik ?sterlund wrote: >> Hi Vladimir, >> >> Thank you for reviewing this. >> >> On 2018-10-08 19:57, Vladimir Kozlov wrote: >>> First, all nmethods have verified entry. MachPrologNode is generated >>> by C2 for all nmethods including OSR (we need stack bang in all >>> cases). OSR does not have *Unverified* entry represented by >>> MachUEPNode. C1 compiled OSR nmethods are different - you may need >>> your changes for it but I am not sure - ask C1 experts. >>> >>> There could be complications if an OSR nmethods is not executed >>> (after barrier check) due to state of stack on OSR entry but it is >>> different issue. >> >> I should have been more precise. OSR nmethods have a prologue for >> building frames, yes. But OSR entries do not have a verified entry >> offset, in the sense that in e.g. c1_LIRAssembler.cpp, no >> CodeOffsets::Verified_Entry entry is set for OSR entries (the >> lir_osr_entry case). It is set only for lir_std_entry. Similarly for >> C2, while they both have a prologue, there is no >> CodeOffsets::Verified_Entry set for OSR nmethods. Read for example >> opto/compile.cpp: >> >> if (is_osr_compilation()) { >> _code_offsets.set_value(CodeOffsets::Verified_Entry, 0); >> _code_offsets.set_value(CodeOffsets::OSR_Entry, >> _first_block_size); >> } else { >> _code_offsets.set_value(CodeOffsets::Verified_Entry, >> _first_block_size); >> _code_offsets.set_value(CodeOffsets::OSR_Entry, 0); >> } >> >> The different treatment of OSR methods for the entry barrier is >> analogous to the different treatment of OSR methods when making them >> not entrant. OSR nmethods are made not entrant by detaching them from >> the Method* (so the runtime won't find them), while non-OSR nmethods >> are made not entrant by patching a jump to the >> SharedRuntime::get_handle_wrong_method_stub() into the verified >> entry. And that handler stub has particular ABI requirements on the >> call site, that do not match the OSR call site. >> >> Similarly for nmethod entry barriers, if the callee is an OSR >> nmethod, it is much easier to check in the runtime before the call if >> said nmethod is toast or not, than it is to check when the nmethod >> has entered, and then figure out how to deoptimize from inside of the >> nmethod. > > Good. Note, I did not argued about changes for OSR nmethods, I was > surprise by statement. Your explanation cleared it. > >> >>> I am curious why you are using next 2 lines everywhere? >>> >>> + BarrierSetAssembler* bs = >>> BarrierSet::barrier_set()->barrier_set_assembler(); >>> + bs->nmethod_entry_barrier(this); >>> >>> instead of one call: >>> >>> BarrierSet::barrier_set()->nmethod_entry_barrier(this); >> >> Because we never make straight calls to BarrierSet for inserting >> barriers. Instead, we use one out of the (currently) 4 helpers that >> have their own responsibility. The BarrierSetAssembler is responsible >> for inserting barriers into assembly code. > > So it is by GC Interface design? Okay. > >> >>> The same question about duplicated code in interpreterRuntime.cpp >> >> Same answer. ;) >> >>> Would be nice if you also update MachPrologNode::format() for case >>> when barrier is used. It is used in PrintOptoAssembly. >> >> Yes, that is a good idea. I will adjust the printing code. > > Thanks, > Vladimir > >> >> Thanks, >> /Erik >> >>> Thanks, >>> Vladimir >>> >>> On 10/8/18 1:19 AM, Erik ?sterlund wrote: >>>> Hi Per and Roman, >>>> >>>> Thank you for having a look at this. >>>> I went with Roman's suggestion and made this an explicit opt-out >>>> instead. But I let ModRef pass in NULL, as none of the barrier sets >>>> in that category currently use nmethod entry barriers. When they >>>> start to do so, that can be easily changed. >>>> >>>> Incremental: >>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ >>>> >>>> Full: >>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-10-08 08:48, Roman Kennke wrote: >>>>> I agree. I wouldn't mind to keep it simple and pass NULL though. >>>>> I'm not >>>>> a big fan of default arguments anyway. >>>>> >>>>> Roman >>>>> >>>>>> On 2018-10-05 17:07, Erik ?sterlund wrote: >>>>>>> Hi, >>>>>>> >>>>>>> In order to implement concurrent class unloading for ZGC, a >>>>>>> mechanism >>>>>>> will be required to arm nmethods in a safepoint such that the first >>>>>>> entry into an nmethod after the pause triggers a barrier before >>>>>>> continuing. >>>>>>> >>>>>>> The barrier will be used for: >>>>>>> * Patching immediate oops, and keeping phantomly reachable oops >>>>>>> alive through the nmethod entry >>>>>>> * Catching calls from stale IC caches (due to class >>>>>>> unloading) into >>>>>>> nmethods that are being unloaded (containing broken oops and >>>>>>> metadata), and lazily clean the IC cache and re-resolve the call. >>>>>>> >>>>>>> This implementation is for x86_64 and is based on the >>>>>>> prototyping work >>>>>>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>>>>>> this patch. >>>>>>> The way it works is that there is a cmp #offset(r15_thread), >>>>>>> simm32_epoch); je #continuation; inserted into the verified >>>>>>> entry of >>>>>>> nmethods that conditionally calls a stub trampolining into the VM. >>>>>>> When the simm32_epoch is not equal to some TLS-local field, then a >>>>>>> slow path is taken to run some barrier inside of the VM. >>>>>>> By changing this TLS-local field in the safepoint, all verified >>>>>>> entries will take the slow path into the VM when run, where the >>>>>>> barrier is eventually disarmed by patching the simm32_epoch >>>>>>> value of >>>>>>> the instruction stream to the current globally correct value. >>>>>>> >>>>>>> In terms of abstractions, to utilize this mechanism, you need to >>>>>>> create your BarierSet with an BarrierSetNMethod helper object, just >>>>>>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>>>>>> BarrierSetNMethod base class handles the barrier itself. It has a >>>>>>> virtual function, nmethod_entry_barrier, which gets called by >>>>>>> the slow >>>>>>> paths. So each user of this mechanism inherits from >>>>>>> BarrierSetNMethod >>>>>>> and puts its barrier in there. >>>>>>> >>>>>>> The OSR nmethods do not have a verified entry. Therefore, the >>>>>>> barrier >>>>>>> is run explicitly in the runtime before calling into OSR >>>>>>> nmethods to >>>>>>> achieve the same effect of trapping new calls into nmethods. >>>>>>> >>>>>>> The nmethod entry barrier returns true/false depending on >>>>>>> whether the >>>>>>> called nmethod may be entered or not. This allows lazily cleaning >>>>>>> nmethod IC caches during unloading, which invokes the same helper >>>>>>> method that is patched into the verified entry when made not >>>>>>> entrant. >>>>>>> >>>>>>> Eventually, I would like to also use this mechanism to remove the >>>>>>> nmethod hotness counter maintenance we today perform in safepoint >>>>>>> cleanup instead. These nmethod entry barriers can provide better >>>>>>> heuristics for whether nmethods are used or not. But that would >>>>>>> require buy-in from maintainers of other platforms. So let's >>>>>>> take one >>>>>>> step at a time. >>>>>>> >>>>>>> For the curious reader thinking that this might be an expensive >>>>>>> operation, it is not. This check in the nmethod verified entry >>>>>>> is very >>>>>>> predictable and cheap. And because of use of inlining, the checks >>>>>>> become more infrequent. In my evaluation, it has gone completely >>>>>>> under >>>>>>> the radar in benchmarking (running this with my current version of >>>>>>> concurrent class unloading in ZGC). >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >>>>>> I reviewed this before Erik sent it out, so my comments have already >>>>>> been addressed. Just found one additional thing, the BarrierSet >>>>>> constructor now looks like this: >>>>>> >>>>>> 96 BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>> 97 BarrierSetC1* barrier_set_c1, >>>>>> 98 BarrierSetC2* barrier_set_c2, >>>>>> 99 const FakeRtti& fake_rtti, >>>>>> 100 BarrierSetNMethod* barrier_set_nmethod = NULL) : >>>>>> 101 _fake_rtti(fake_rtti), >>>>>> 102 _barrier_set_assembler(barrier_set_assembler), >>>>>> 103 _barrier_set_c1(barrier_set_c1), >>>>>> 104 _barrier_set_c2(barrier_set_c2), >>>>>> 105 _barrier_set_nmethod(barrier_set_nmethod) {} >>>>>> >>>>>> I find it odd that the fake_rtti argument comes in the middle of the >>>>>> barrier_set_* arguments. I can see that you don't want GC barrier >>>>>> sets >>>>>> to have to supply NULL if they don't have a nmethod barrier set, but >>>>>> maybe we can have two constructors instead, like: >>>>>> >>>>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>> BarrierSetC1* barrier_set_c1, >>>>>> BarrierSetC2* barrier_set_c2, >>>>>> const FakeRtti& fake_rtti) : >>>>>> _fake_rtti(fake_rtti), >>>>>> _barrier_set_assembler(barrier_set_assembler), >>>>>> _barrier_set_c1(barrier_set_c1), >>>>>> _barrier_set_c2(barrier_set_c2), >>>>>> _barrier_set_nmethod(NULL) {} >>>>>> >>>>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>> BarrierSetC1* barrier_set_c1, >>>>>> BarrierSetC2* barrier_set_c2, >>>>>> BarrierSetNMethod* barrier_set_nmethod, >>>>>> const FakeRtti& fake_rtti,) : >>>>>> _fake_rtti(fake_rtti), >>>>>> _barrier_set_assembler(barrier_set_assembler), >>>>>> _barrier_set_c1(barrier_set_c1), >>>>>> _barrier_set_c2(barrier_set_c2), >>>>>> _barrier_set_nmethod(barrier_set_nmethod) {} >>>>>> >>>>>> What do you think? >>>>>> >>>>>> Other than that, this looks good to me. >>>>>> >>>>>> cheers, >>>>>> Per >>>>>> >>>>>>> Bug ID: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>> >> From sgehwolf at redhat.com Tue Oct 9 15:59:35 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 09 Oct 2018 17:59:35 +0200 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 In-Reply-To: <03f8fa68-a758-d956-cfd3-1572b093d3f9@oracle.com> References: <03f8fa68-a758-d956-cfd3-1572b093d3f9@oracle.com> Message-ID: <5dda0126bc7d30257c1644214a256e29c860d5dc.camel@redhat.com> On Tue, 2018-10-09 at 08:06 -0700, Vladimir Kozlov wrote: > Why there is no complain about sprintf at line 2857 which uses %s? : > http://hg.openjdk.java.net/jdk/jdk/file/d24b89390f6c/src/hotspot/share/opto/parse2.cpp#l2857 Perhaps because this is non-product code? Thanks, Severin From vladimir.kozlov at oracle.com Tue Oct 9 17:19:09 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 9 Oct 2018 10:19:09 -0700 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: <5BBCCEB4.6030104@oracle.com> References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> <5BBB1328.3020503@oracle.com> <0b99da49-b118-1db8-d21c-3213fdfffe36@oracle.com> <7c0fafa2-831b-fdce-2a33-cacb7921395f@oracle.com> <23ac864d-44b4-a54f-b2b2-869899553cfd@oracle.com> <5BBCCEB4.6030104@oracle.com> Message-ID: Looks good. Thanks, Vladimir On 10/9/18 8:52 AM, Erik ?sterlund wrote: > Hi Vladimir, > > Sounds like we are in agreement. Thanks for looking into this change. > I updated the webrev with the added assembly printing for C2 as requested. > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01_02/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8210498/webrev.02/ > > Thanks, > /Erik > > On 2018-10-09 16:42, Vladimir Kozlov wrote: >> On 10/9/18 12:24 AM, Erik ?sterlund wrote: >>> Hi Vladimir, >>> >>> Thank you for reviewing this. >>> >>> On 2018-10-08 19:57, Vladimir Kozlov wrote: >>>> First, all nmethods have verified entry. MachPrologNode is generated by C2 for all nmethods including OSR (we need >>>> stack bang in all cases). OSR does not have *Unverified* entry represented by MachUEPNode. C1 compiled OSR nmethods >>>> are different - you may need your changes for it but I am not sure - ask C1 experts. >>>> >>>> There could be complications if an OSR nmethods is not executed (after barrier check) due to state of stack on OSR >>>> entry but it is different issue. >>> >>> I should have been more precise. OSR nmethods have a prologue for building frames, yes. But OSR entries do not have a >>> verified entry offset, in the sense that in e.g. c1_LIRAssembler.cpp, no CodeOffsets::Verified_Entry entry is set for >>> OSR entries (the lir_osr_entry case). It is set only for lir_std_entry. Similarly for C2, while they both have a >>> prologue, there is no CodeOffsets::Verified_Entry set for OSR nmethods. Read for example opto/compile.cpp: >>> >>> ???? if (is_osr_compilation()) { >>> ?????? _code_offsets.set_value(CodeOffsets::Verified_Entry, 0); >>> ?????? _code_offsets.set_value(CodeOffsets::OSR_Entry, _first_block_size); >>> ???? } else { >>> ?????? _code_offsets.set_value(CodeOffsets::Verified_Entry, _first_block_size); >>> ?????? _code_offsets.set_value(CodeOffsets::OSR_Entry, 0); >>> ???? } >>> >>> The different treatment of OSR methods for the entry barrier is analogous to the different treatment of OSR methods >>> when making them not entrant. OSR nmethods are made not entrant by detaching them from the Method* (so the runtime >>> won't find them), while non-OSR nmethods are made not entrant by patching a jump to the >>> SharedRuntime::get_handle_wrong_method_stub() into the verified entry. And that handler stub has particular ABI >>> requirements on the call site, that do not match the OSR call site. >>> >>> Similarly for nmethod entry barriers, if the callee is an OSR nmethod, it is much easier to check in the runtime >>> before the call if said nmethod is toast or not, than it is to check when the nmethod has entered, and then figure >>> out how to deoptimize from inside of the nmethod. >> >> Good. Note, I did not argued about changes for OSR nmethods, I was surprise by statement. Your explanation cleared it. >> >>> >>>> I am curious why you are using next 2 lines everywhere? >>>> >>>> +? BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); >>>> +? bs->nmethod_entry_barrier(this); >>>> >>>> instead of one call: >>>> >>>> BarrierSet::barrier_set()->nmethod_entry_barrier(this); >>> >>> Because we never make straight calls to BarrierSet for inserting barriers. Instead, we use one out of the (currently) >>> 4 helpers that have their own responsibility. The BarrierSetAssembler is responsible for inserting barriers into >>> assembly code. >> >> So it is by GC Interface design? Okay. >> >>> >>>> The same question about duplicated code in interpreterRuntime.cpp >>> >>> Same answer. ;) >>> >>>> Would be nice if you also update MachPrologNode::format() for case when barrier is used. It is used in >>>> PrintOptoAssembly. >>> >>> Yes, that is a good idea. I will adjust the printing code. >> >> Thanks, >> Vladimir >> >>> >>> Thanks, >>> /Erik >>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 10/8/18 1:19 AM, Erik ?sterlund wrote: >>>>> Hi Per and Roman, >>>>> >>>>> Thank you for having a look at this. >>>>> I went with Roman's suggestion and made this an explicit opt-out instead. But I let ModRef pass in NULL, as none of >>>>> the barrier sets in that category currently use nmethod entry barriers. When they start to do so, that can be >>>>> easily changed. >>>>> >>>>> Incremental: >>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ >>>>> >>>>> Full: >>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-10-08 08:48, Roman Kennke wrote: >>>>>> I agree. I wouldn't mind to keep it simple and pass NULL though. I'm not >>>>>> a big fan of default arguments anyway. >>>>>> >>>>>> Roman >>>>>> >>>>>>> On 2018-10-05 17:07, Erik ?sterlund wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> In order to implement concurrent class unloading for ZGC, a mechanism >>>>>>>> will be required to arm nmethods in a safepoint such that the first >>>>>>>> entry into an nmethod after the pause triggers a barrier before >>>>>>>> continuing. >>>>>>>> >>>>>>>> The barrier will be used for: >>>>>>>> ??? * Patching immediate oops, and keeping phantomly reachable oops >>>>>>>> alive through the nmethod entry >>>>>>>> ??? * Catching calls from stale IC caches (due to class unloading) into >>>>>>>> nmethods that are being unloaded (containing broken oops and >>>>>>>> metadata), and lazily clean the IC cache and re-resolve the call. >>>>>>>> >>>>>>>> This implementation is for x86_64 and is based on the prototyping work >>>>>>>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>>>>>>> this patch. >>>>>>>> The way it works is that there is a cmp #offset(r15_thread), >>>>>>>> simm32_epoch); je #continuation; inserted into the verified entry of >>>>>>>> nmethods that conditionally calls a stub trampolining into the VM. >>>>>>>> When the simm32_epoch is not equal to some TLS-local field, then a >>>>>>>> slow path is taken to run some barrier inside of the VM. >>>>>>>> By changing this TLS-local field in the safepoint, all verified >>>>>>>> entries will take the slow path into the VM when run, where the >>>>>>>> barrier is eventually disarmed by patching the simm32_epoch value of >>>>>>>> the instruction stream to the current globally correct value. >>>>>>>> >>>>>>>> In terms of abstractions, to utilize this mechanism, you need to >>>>>>>> create your BarierSet with an BarrierSetNMethod helper object, just >>>>>>>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>>>>>>> BarrierSetNMethod base class handles the barrier itself. It has a >>>>>>>> virtual function, nmethod_entry_barrier, which gets called by the slow >>>>>>>> paths. So each user of this mechanism inherits from BarrierSetNMethod >>>>>>>> and puts its barrier in there. >>>>>>>> >>>>>>>> The OSR nmethods do not have a verified entry. Therefore, the barrier >>>>>>>> is run explicitly in the runtime before calling into OSR nmethods to >>>>>>>> achieve the same effect of trapping new calls into nmethods. >>>>>>>> >>>>>>>> The nmethod entry barrier returns true/false depending on whether the >>>>>>>> called nmethod may be entered or not. This allows lazily cleaning >>>>>>>> nmethod IC caches during unloading, which invokes the same helper >>>>>>>> method that is patched into the verified entry when made not entrant. >>>>>>>> >>>>>>>> Eventually, I would like to also use this mechanism to remove the >>>>>>>> nmethod hotness counter maintenance we today perform in safepoint >>>>>>>> cleanup instead. These nmethod entry barriers can provide better >>>>>>>> heuristics for whether nmethods are used or not. But that would >>>>>>>> require buy-in from maintainers of other platforms. So let's take one >>>>>>>> step at a time. >>>>>>>> >>>>>>>> For the curious reader thinking that this might be an expensive >>>>>>>> operation, it is not. This check in the nmethod verified entry is very >>>>>>>> predictable and cheap. And because of use of inlining, the checks >>>>>>>> become more infrequent. In my evaluation, it has gone completely under >>>>>>>> the radar in benchmarking (running this with my current version of >>>>>>>> concurrent class unloading in ZGC). >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >>>>>>> I reviewed this before Erik sent it out, so my comments have already >>>>>>> been addressed. Just found one additional thing, the BarrierSet >>>>>>> constructor now looks like this: >>>>>>> >>>>>>> 96?? BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>>> 97????????????? BarrierSetC1* barrier_set_c1, >>>>>>> 98????????????? BarrierSetC2* barrier_set_c2, >>>>>>> 99????????????? const FakeRtti& fake_rtti, >>>>>>> 100???????????? BarrierSetNMethod* barrier_set_nmethod = NULL) : >>>>>>> 101???? _fake_rtti(fake_rtti), >>>>>>> 102???? _barrier_set_assembler(barrier_set_assembler), >>>>>>> 103???? _barrier_set_c1(barrier_set_c1), >>>>>>> 104???? _barrier_set_c2(barrier_set_c2), >>>>>>> 105???? _barrier_set_nmethod(barrier_set_nmethod) {} >>>>>>> >>>>>>> I find it odd that the fake_rtti argument comes in the middle of the >>>>>>> barrier_set_* arguments. I can see that you don't want GC barrier sets >>>>>>> to have to supply NULL if they don't have a nmethod barrier set, but >>>>>>> maybe we can have two constructors instead, like: >>>>>>> >>>>>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>>> ??????????? BarrierSetC1* barrier_set_c1, >>>>>>> ??????????? BarrierSetC2* barrier_set_c2, >>>>>>> ??????????? const FakeRtti& fake_rtti) : >>>>>>> ???? _fake_rtti(fake_rtti), >>>>>>> ???? _barrier_set_assembler(barrier_set_assembler), >>>>>>> ???? _barrier_set_c1(barrier_set_c1), >>>>>>> ???? _barrier_set_c2(barrier_set_c2), >>>>>>> ???? _barrier_set_nmethod(NULL) {} >>>>>>> >>>>>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>>> ??????????? BarrierSetC1* barrier_set_c1, >>>>>>> ??????????? BarrierSetC2* barrier_set_c2, >>>>>>> ??????????? BarrierSetNMethod* barrier_set_nmethod, >>>>>>> ??????????? const FakeRtti& fake_rtti,) : >>>>>>> ???? _fake_rtti(fake_rtti), >>>>>>> ???? _barrier_set_assembler(barrier_set_assembler), >>>>>>> ???? _barrier_set_c1(barrier_set_c1), >>>>>>> ???? _barrier_set_c2(barrier_set_c2), >>>>>>> ???? _barrier_set_nmethod(barrier_set_nmethod) {} >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> Other than that, this looks good to me. >>>>>>> >>>>>>> cheers, >>>>>>> Per >>>>>>> >>>>>>>> Bug ID: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>> >>> > From igor.ignatyev at oracle.com Tue Oct 9 17:23:55 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Oct 2018 10:23:55 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest Message-ID: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > 450 lines changed: 238 ins; 211 del; 1 mod; Hi all, could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 testing: - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants - build w/ precompiled-headers enabled and disabled PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review Thanks, -- Igor From erik.osterlund at oracle.com Tue Oct 9 18:58:53 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Tue, 9 Oct 2018 20:58:53 +0200 Subject: RFR: 8210498: nmethod entry barriers In-Reply-To: References: <5BB77E18.6040401@oracle.com> <74be6598-5349-69ea-2f78-ea2053b537c4@oracle.com> <5BBB1328.3020503@oracle.com> <0b99da49-b118-1db8-d21c-3213fdfffe36@oracle.com> <7c0fafa2-831b-fdce-2a33-cacb7921395f@oracle.com> <23ac864d-44b4-a54f-b2b2-869899553cfd@oracle.com> <5BBCCEB4.6030104@oracle.com> Message-ID: <7BBAB82F-6C4E-485B-A4CF-C10D164ED17F@oracle.com> Hi Vladimir, Thanks for the review! /Erik > On 9 Oct 2018, at 19:19, Vladimir Kozlov wrote: > > Looks good. > > Thanks, > Vladimir > >> On 10/9/18 8:52 AM, Erik ?sterlund wrote: >> Hi Vladimir, >> Sounds like we are in agreement. Thanks for looking into this change. >> I updated the webrev with the added assembly printing for C2 as requested. >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01_02/ >> Full: >> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.02/ >> Thanks, >> /Erik >>> On 2018-10-09 16:42, Vladimir Kozlov wrote: >>>> On 10/9/18 12:24 AM, Erik ?sterlund wrote: >>>> Hi Vladimir, >>>> >>>> Thank you for reviewing this. >>>> >>>>> On 2018-10-08 19:57, Vladimir Kozlov wrote: >>>>> First, all nmethods have verified entry. MachPrologNode is generated by C2 for all nmethods including OSR (we need stack bang in all cases). OSR does not have *Unverified* entry represented by MachUEPNode. C1 compiled OSR nmethods are different - you may need your changes for it but I am not sure - ask C1 experts. >>>>> >>>>> There could be complications if an OSR nmethods is not executed (after barrier check) due to state of stack on OSR entry but it is different issue. >>>> >>>> I should have been more precise. OSR nmethods have a prologue for building frames, yes. But OSR entries do not have a verified entry offset, in the sense that in e.g. c1_LIRAssembler.cpp, no CodeOffsets::Verified_Entry entry is set for OSR entries (the lir_osr_entry case). It is set only for lir_std_entry. Similarly for C2, while they both have a prologue, there is no CodeOffsets::Verified_Entry set for OSR nmethods. Read for example opto/compile.cpp: >>>> >>>> if (is_osr_compilation()) { >>>> _code_offsets.set_value(CodeOffsets::Verified_Entry, 0); >>>> _code_offsets.set_value(CodeOffsets::OSR_Entry, _first_block_size); >>>> } else { >>>> _code_offsets.set_value(CodeOffsets::Verified_Entry, _first_block_size); >>>> _code_offsets.set_value(CodeOffsets::OSR_Entry, 0); >>>> } >>>> >>>> The different treatment of OSR methods for the entry barrier is analogous to the different treatment of OSR methods when making them not entrant. OSR nmethods are made not entrant by detaching them from the Method* (so the runtime won't find them), while non-OSR nmethods are made not entrant by patching a jump to the SharedRuntime::get_handle_wrong_method_stub() into the verified entry. And that handler stub has particular ABI requirements on the call site, that do not match the OSR call site. >>>> >>>> Similarly for nmethod entry barriers, if the callee is an OSR nmethod, it is much easier to check in the runtime before the call if said nmethod is toast or not, than it is to check when the nmethod has entered, and then figure out how to deoptimize from inside of the nmethod. >>> >>> Good. Note, I did not argued about changes for OSR nmethods, I was surprise by statement. Your explanation cleared it. >>> >>>> >>>>> I am curious why you are using next 2 lines everywhere? >>>>> >>>>> + BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); >>>>> + bs->nmethod_entry_barrier(this); >>>>> >>>>> instead of one call: >>>>> >>>>> BarrierSet::barrier_set()->nmethod_entry_barrier(this); >>>> >>>> Because we never make straight calls to BarrierSet for inserting barriers. Instead, we use one out of the (currently) 4 helpers that have their own responsibility. The BarrierSetAssembler is responsible for inserting barriers into assembly code. >>> >>> So it is by GC Interface design? Okay. >>> >>>> >>>>> The same question about duplicated code in interpreterRuntime.cpp >>>> >>>> Same answer. ;) >>>> >>>>> Would be nice if you also update MachPrologNode::format() for case when barrier is used. It is used in PrintOptoAssembly. >>>> >>>> Yes, that is a good idea. I will adjust the printing code. >>> >>> Thanks, >>> Vladimir >>> >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>>> On 10/8/18 1:19 AM, Erik ?sterlund wrote: >>>>>> Hi Per and Roman, >>>>>> >>>>>> Thank you for having a look at this. >>>>>> I went with Roman's suggestion and made this an explicit opt-out instead. But I let ModRef pass in NULL, as none of the barrier sets in that category currently use nmethod entry barriers. When they start to do so, that can be easily changed. >>>>>> >>>>>> Incremental: >>>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00_01/ >>>>>> >>>>>> Full: >>>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.01/ >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> On 2018-10-08 08:48, Roman Kennke wrote: >>>>>>> I agree. I wouldn't mind to keep it simple and pass NULL though. I'm not >>>>>>> a big fan of default arguments anyway. >>>>>>> >>>>>>> Roman >>>>>>> >>>>>>>>> On 2018-10-05 17:07, Erik ?sterlund wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> In order to implement concurrent class unloading for ZGC, a mechanism >>>>>>>>> will be required to arm nmethods in a safepoint such that the first >>>>>>>>> entry into an nmethod after the pause triggers a barrier before >>>>>>>>> continuing. >>>>>>>>> >>>>>>>>> The barrier will be used for: >>>>>>>>> * Patching immediate oops, and keeping phantomly reachable oops >>>>>>>>> alive through the nmethod entry >>>>>>>>> * Catching calls from stale IC caches (due to class unloading) into >>>>>>>>> nmethods that are being unloaded (containing broken oops and >>>>>>>>> metadata), and lazily clean the IC cache and re-resolve the call. >>>>>>>>> >>>>>>>>> This implementation is for x86_64 and is based on the prototyping work >>>>>>>>> of Rickard B?ckman. So big thanks to Rickard, who will co-author of >>>>>>>>> this patch. >>>>>>>>> The way it works is that there is a cmp #offset(r15_thread), >>>>>>>>> simm32_epoch); je #continuation; inserted into the verified entry of >>>>>>>>> nmethods that conditionally calls a stub trampolining into the VM. >>>>>>>>> When the simm32_epoch is not equal to some TLS-local field, then a >>>>>>>>> slow path is taken to run some barrier inside of the VM. >>>>>>>>> By changing this TLS-local field in the safepoint, all verified >>>>>>>>> entries will take the slow path into the VM when run, where the >>>>>>>>> barrier is eventually disarmed by patching the simm32_epoch value of >>>>>>>>> the instruction stream to the current globally correct value. >>>>>>>>> >>>>>>>>> In terms of abstractions, to utilize this mechanism, you need to >>>>>>>>> create your BarierSet with an BarrierSetNMethod helper object, just >>>>>>>>> like the BarrierSetAssembler, BarrierSetC1 and BarrierSetC2. The >>>>>>>>> BarrierSetNMethod base class handles the barrier itself. It has a >>>>>>>>> virtual function, nmethod_entry_barrier, which gets called by the slow >>>>>>>>> paths. So each user of this mechanism inherits from BarrierSetNMethod >>>>>>>>> and puts its barrier in there. >>>>>>>>> >>>>>>>>> The OSR nmethods do not have a verified entry. Therefore, the barrier >>>>>>>>> is run explicitly in the runtime before calling into OSR nmethods to >>>>>>>>> achieve the same effect of trapping new calls into nmethods. >>>>>>>>> >>>>>>>>> The nmethod entry barrier returns true/false depending on whether the >>>>>>>>> called nmethod may be entered or not. This allows lazily cleaning >>>>>>>>> nmethod IC caches during unloading, which invokes the same helper >>>>>>>>> method that is patched into the verified entry when made not entrant. >>>>>>>>> >>>>>>>>> Eventually, I would like to also use this mechanism to remove the >>>>>>>>> nmethod hotness counter maintenance we today perform in safepoint >>>>>>>>> cleanup instead. These nmethod entry barriers can provide better >>>>>>>>> heuristics for whether nmethods are used or not. But that would >>>>>>>>> require buy-in from maintainers of other platforms. So let's take one >>>>>>>>> step at a time. >>>>>>>>> >>>>>>>>> For the curious reader thinking that this might be an expensive >>>>>>>>> operation, it is not. This check in the nmethod verified entry is very >>>>>>>>> predictable and cheap. And because of use of inlining, the checks >>>>>>>>> become more infrequent. In my evaluation, it has gone completely under >>>>>>>>> the radar in benchmarking (running this with my current version of >>>>>>>>> concurrent class unloading in ZGC). >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8210498/webrev.00/ >>>>>>>> I reviewed this before Erik sent it out, so my comments have already >>>>>>>> been addressed. Just found one additional thing, the BarrierSet >>>>>>>> constructor now looks like this: >>>>>>>> >>>>>>>> 96 BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>>>> 97 BarrierSetC1* barrier_set_c1, >>>>>>>> 98 BarrierSetC2* barrier_set_c2, >>>>>>>> 99 const FakeRtti& fake_rtti, >>>>>>>> 100 BarrierSetNMethod* barrier_set_nmethod = NULL) : >>>>>>>> 101 _fake_rtti(fake_rtti), >>>>>>>> 102 _barrier_set_assembler(barrier_set_assembler), >>>>>>>> 103 _barrier_set_c1(barrier_set_c1), >>>>>>>> 104 _barrier_set_c2(barrier_set_c2), >>>>>>>> 105 _barrier_set_nmethod(barrier_set_nmethod) {} >>>>>>>> >>>>>>>> I find it odd that the fake_rtti argument comes in the middle of the >>>>>>>> barrier_set_* arguments. I can see that you don't want GC barrier sets >>>>>>>> to have to supply NULL if they don't have a nmethod barrier set, but >>>>>>>> maybe we can have two constructors instead, like: >>>>>>>> >>>>>>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>>>> BarrierSetC1* barrier_set_c1, >>>>>>>> BarrierSetC2* barrier_set_c2, >>>>>>>> const FakeRtti& fake_rtti) : >>>>>>>> _fake_rtti(fake_rtti), >>>>>>>> _barrier_set_assembler(barrier_set_assembler), >>>>>>>> _barrier_set_c1(barrier_set_c1), >>>>>>>> _barrier_set_c2(barrier_set_c2), >>>>>>>> _barrier_set_nmethod(NULL) {} >>>>>>>> >>>>>>>> BarrierSet(BarrierSetAssembler* barrier_set_assembler, >>>>>>>> BarrierSetC1* barrier_set_c1, >>>>>>>> BarrierSetC2* barrier_set_c2, >>>>>>>> BarrierSetNMethod* barrier_set_nmethod, >>>>>>>> const FakeRtti& fake_rtti,) : >>>>>>>> _fake_rtti(fake_rtti), >>>>>>>> _barrier_set_assembler(barrier_set_assembler), >>>>>>>> _barrier_set_c1(barrier_set_c1), >>>>>>>> _barrier_set_c2(barrier_set_c2), >>>>>>>> _barrier_set_nmethod(barrier_set_nmethod) {} >>>>>>>> >>>>>>>> What do you think? >>>>>>>> >>>>>>>> Other than that, this looks good to me. >>>>>>>> >>>>>>>> cheers, >>>>>>>> Per >>>>>>>> >>>>>>>>> Bug ID: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8210498 >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> /Erik >>>>>> >>>> From jcbeyler at google.com Tue Oct 9 20:47:03 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 9 Oct 2018 13:47:03 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> Message-ID: Hi Igor, I was looking at this and it looks good to me (not a reviewer of course) but I noticed a few nits: - http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/src/hotspot/share/gc/shared/gcTimer.cpp.udiff.html - Seems like you could remove a few more lines there because now you have empty lines at the end of the file - http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/test/hotspot/gtest/gc/shared/test_gcTimer.cpp.html - Should the copyright year 2001 be here? - Indentation is a bit off: - GCTimerTest has 2 spaces for public and 4 for methods - TimePartitionPhasesIteratorTest has 1 and 2 - Tests afterwards have 2 - The code could be simplified if we removed the TimePartitionPhasesIteratorTest entirely and just put it in an anonymous namespace - hotspot/gtest/utilities/test_bitMap_search.cpp uses anonymous namespaces so gtest does support it Then lines such as: 219 EXPECT_NO_FATAL_FAILURE(TimePartitionPhasesIteratorTest::validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); become: 219 EXPECT_NO_FATAL_FAILURE(validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); Apart from that, it looks good to me :) Jc On Tue, Oct 9, 2018 at 10:24 AM Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > > 450 lines changed: 238 ins; 211 del; 1 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which > converts internal GCTimer_test to gtest? > > webrev: > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 > in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been > sent out for official review > > Thanks, > -- Igor > -- Thanks, Jc From igor.ignatyev at oracle.com Tue Oct 9 21:03:46 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Oct 2018 14:03:46 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> Message-ID: <5D55535E-A01E-42DA-A64A-64BCE828FD30@oracle.com> Hi JC, thanks for reviewing it! > On Oct 9, 2018, at 1:47 PM, JC Beyler wrote: > > Hi Igor, > > I was looking at this and it looks good to me (not a reviewer of course) but I noticed a few nits: > > - http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/src/hotspot/share/gc/shared/gcTimer.cpp.udiff.html > - Seems like you could remove a few more lines there because now you have empty lines at the end of the file fixed > > - http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/test/hotspot/gtest/gc/shared/test_gcTimer.cpp.html > - Should the copyright year 2001 be here? yes, as this code was originally introduced in 2001 as part of gcTimer.cpp > > - Indentation is a bit off: > - GCTimerTest has 2 spaces for public and 4 for methods > - TimePartitionPhasesIteratorTest has 1 and 2 > - Tests afterwards have 2 dough, that's what happens when you copy-paste someone's else code... fixed. now GCTimerTest and TimePartitionPhasesIteratorTest have 1 for public and 2 for methods, all tests have 2. > > - The code could be simplified if we removed the TimePartitionPhasesIteratorTest entirely and just put it in an anonymous namespace > - hotspot/gtest/utilities/test_bitMap_search.cpp uses anonymous namespaces so gtest does support it > Then lines such as: > 219 EXPECT_NO_FATAL_FAILURE(TimePartitionPhasesIteratorTest::validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); > become: > 219 EXPECT_NO_FATAL_FAILURE(validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); unfortunately we can't do it, TimePartitionPhasesIteratorTest (as well as GCTimerTest) is needed b/c it's a friend of TimeInstant (Ticks), so it can use private c-tor TimeInstant(jlong). -- Igor > Apart from that, it looks good to me :) > Jc > > > On Tue, Oct 9, 2018 at 10:24 AM Igor Ignatyev > wrote: > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > > 450 lines changed: 238 ins; 211 del; 1 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? > > webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review > > Thanks, > -- Igor > > > > -- > > Thanks, > Jc From jcbeyler at google.com Tue Oct 9 21:11:43 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 9 Oct 2018 14:11:43 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <5D55535E-A01E-42DA-A64A-64BCE828FD30@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> <5D55535E-A01E-42DA-A64A-64BCE828FD30@oracle.com> Message-ID: Good point on the friend class, looks good to me then :) Jc On Tue, Oct 9, 2018 at 2:03 PM Igor Ignatyev wrote: > Hi JC, > > thanks for reviewing it! > > On Oct 9, 2018, at 1:47 PM, JC Beyler wrote: > > Hi Igor, > > I was looking at this and it looks good to me (not a reviewer of course) > but I noticed a few nits: > > - > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/src/hotspot/share/gc/shared/gcTimer.cpp.udiff.html > - Seems like you could remove a few more lines there because now you > have empty lines at the end of the file > > fixed > > > - > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/test/hotspot/gtest/gc/shared/test_gcTimer.cpp.html > - Should the copyright year 2001 be here? > > yes, as this code was originally introduced in 2001 as part of gcTimer.cpp > > > - Indentation is a bit off: > - GCTimerTest has 2 spaces for public and 4 for methods > - TimePartitionPhasesIteratorTest has 1 and 2 > - Tests afterwards have 2 > > dough, that's what happens when you copy-paste someone's else code... > fixed. now GCTimerTest and TimePartitionPhasesIteratorTest have 1 for > public and 2 for methods, all tests have 2. > > > - The code could be simplified if we removed the TimePartitionPhasesIteratorTest > entirely and just put it in an anonymous namespace > > - hotspot/gtest/utilities/test_bitMap_search.cpp uses anonymous > namespaces so gtest does support it > Then lines such as: > > 219 EXPECT_NO_FATAL_FAILURE(TimePartitionPhasesIteratorTest::validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); > > become: > > 219 EXPECT_NO_FATAL_FAILURE(validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); > > unfortunately we can't do it, TimePartitionPhasesIteratorTest (as well as > GCTimerTest) is needed b/c it's a friend of TimeInstant (Ticks), so it can > use private c-tor TimeInstant(jlong). > > -- Igor > > Apart from that, it looks good to me :) > > Jc > > > > On Tue, Oct 9, 2018 at 10:24 AM Igor Ignatyev > wrote: > >> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >> > 450 lines changed: 238 ins; 211 del; 1 mod; >> >> Hi all, >> >> could you please review this small (and hopefully trivial) patch which >> converts internal GCTimer_test to gtest? >> >> webrev: >> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 >> testing: >> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 >> in product and fastdebug variants >> - build w/ precompiled-headers enabled and disabled >> >> PS the patch has been originally created by Kirill Zh, but hasn't been >> sent out for official review >> >> Thanks, >> -- Igor >> > > > > -- > > Thanks, > Jc > > > -- Thanks, Jc From vladimir.kozlov at oracle.com Tue Oct 9 22:45:56 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 9 Oct 2018 15:45:56 -0700 Subject: RFC: JEP (JDK-8211652) Update JVMCI to support libgraal Message-ID: https://bugs.openjdk.java.net/browse/JDK-8211652 Please review and comment on this newly filed JEP. These JVMCI changes are required for deploying Graal as an AOT compiled shared library libgraal. Future libgraal JEP depends on it and we want JVMCI changes to be pushed first. -- Thanks, Vladimir From david.holmes at oracle.com Wed Oct 10 00:35:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Oct 2018 10:35:50 +1000 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: Message-ID: Hi Luke, Moving this over to hotspot-dev. If it is an issue with the JIT escape analysis then hotspot-compiler-dev would be the place to raise this. But a general finalization issue may hit GC and runtime as well, so I redirected to the broader hotspot-dev. Also note that finalizers can run while methods of a class instance are still in progress [1] "Optimizing transformations of a program can be designed that reduce the number of objects that are reachable to be less than those which would naively be considered reachable." - this is one of the evil things about finalizers. So it may not be a bug, just surprising. Cheers, David [1] https://docs.oracle.com/javase/specs/jls/se11/html/jls-12.html#jls-12.6 On 10/10/2018 10:27 AM, Luke Hutchison wrote: > A user of a library I maintain, ClassGraph, has reported that a finalizer > is being called on a class while methods of the class are still running, > indicating that the class, even though it overrides finalize(), is not > being correctly marked as GlobalEscape, so that escape analysis is trying > to garbage-collect the class too early. Is this mailing list the correct > place to report this bug? > > Here is the Maven rule for ClassGraph (you will also need to add Scala > deps) -- you will need version 4.2.8, since I put a mitigation in place in > version 4.2.9: > > > io.github.classgraph > classgraph > 4.2.8 > > > Here is the Scala code that reproduces the problem 100% of the time: > > import io.github.classgraph.{ClassGraph, ClassInfo} > import scala.collection.JavaConverters._ > import scala.compat.Platform.ConcurrentModificationException > > while (true) { > try { > new ClassGraph().scan() > .getResourcesMatchingPattern(".*".r.pattern) > .asScala > .map(_.getURL) > } catch { > case cme: ConcurrentModificationException => > cme.printStackTrace() > throw cme > } > } > > Then start the JVM with -XX:+DoEscapeAnalysis > > The code new ClassGraph().scan() returns a ScanResult, and then as soon as > ScanResult::getResourcesMatchingPattern is called with the ScanResult as > the invocation target, escape analysis sees that the ScanResult has "gone > out of scope" (despite the fact that the ScanResult is the invocation > target for a currently-running method), and immediately tries to garbage > collect it. This should not happen (1) since an invocation target should > never be marked for garbage collection while its methods are being run, and > (2) since ScanResult overrides Object::finalize, so should have been marked > as "GlobalEscape" by escape analysis. > > The finalizer for the ScanResult class calls ScanResult::close, which > clears the allResources list: > > https://github.com/classgraph/classgraph/blob/b9e8165d34575378697a727888457abb164311b8/src/main/java/io/github/classgraph/ScanResult.java#L940 > > This results in a ConcurrentModificationException since > ScanResult::getResourcesMatchingPattern is still iterating through > allResources: > > https://github.com/classgraph/classgraph/blob/b9e8165d34575378697a727888457abb164311b8/src/main/java/io/github/classgraph/ScanResult.java#L383 > > I read somewhere (and it makes sense) that escape analysis is performed by > the compiler, so maybe this is a Scala bug, but this behavior is triggered > by the JVM switch -XX:+DoEscapeAnalysis , and I couldn't find any > references to escape analysis information in the Java classfile format > spec, so that leads me to think this may be a JVM bug (which is why I am > asking on this list). > > Obviously having a garbage collector run a finalizer run while methods of > the object are still running is a very serious bug. Any pointers as to > where to report this issue would be appreciated. Thanks! > From martinrb at google.com Wed Oct 10 00:51:39 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Oct 2018 17:51:39 -0700 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: Message-ID: Search for calls to Reference.reachabilityFence(this); in the jdk sources. On Tue, Oct 9, 2018 at 5:35 PM, David Holmes wrote: > Hi Luke, > > Moving this over to hotspot-dev. If it is an issue with the JIT escape > analysis then hotspot-compiler-dev would be the place to raise this. But a > general finalization issue may hit GC and runtime as well, so I redirected > to the broader hotspot-dev. > > Also note that finalizers can run while methods of a class instance are > still in progress [1] > > "Optimizing transformations of a program can be designed that reduce the > number of objects that are reachable to be less than those which would > naively be considered reachable." > > - this is one of the evil things about finalizers. So it may not be a > bug, just surprising. > > Cheers, > David > > [1] https://docs.oracle.com/javase/specs/jls/se11/html/jls-12. > html#jls-12.6 > > > On 10/10/2018 10:27 AM, Luke Hutchison wrote: > >> A user of a library I maintain, ClassGraph, has reported that a finalizer >> is being called on a class while methods of the class are still running, >> indicating that the class, even though it overrides finalize(), is not >> being correctly marked as GlobalEscape, so that escape analysis is trying >> to garbage-collect the class too early. Is this mailing list the correct >> place to report this bug? >> >> Here is the Maven rule for ClassGraph (you will also need to add Scala >> deps) -- you will need version 4.2.8, since I put a mitigation in place in >> version 4.2.9: >> >> >> io.github.classgraph >> classgraph >> 4.2.8 >> >> >> Here is the Scala code that reproduces the problem 100% of the time: >> >> import io.github.classgraph.{ClassGraph, ClassInfo} >> import scala.collection.JavaConverters._ >> import scala.compat.Platform.ConcurrentModificationException >> >> while (true) { >> try { >> new ClassGraph().scan() >> .getResourcesMatchingPattern(".*".r.pattern) >> .asScala >> .map(_.getURL) >> } catch { >> case cme: ConcurrentModificationException => >> cme.printStackTrace() >> throw cme >> } >> } >> >> Then start the JVM with -XX:+DoEscapeAnalysis >> >> The code new ClassGraph().scan() returns a ScanResult, and then as soon as >> ScanResult::getResourcesMatchingPattern is called with the ScanResult as >> the invocation target, escape analysis sees that the ScanResult has "gone >> out of scope" (despite the fact that the ScanResult is the invocation >> target for a currently-running method), and immediately tries to garbage >> collect it. This should not happen (1) since an invocation target should >> never be marked for garbage collection while its methods are being run, >> and >> (2) since ScanResult overrides Object::finalize, so should have been >> marked >> as "GlobalEscape" by escape analysis. >> >> The finalizer for the ScanResult class calls ScanResult::close, which >> clears the allResources list: >> >> https://github.com/classgraph/classgraph/blob/b9e8165d345753 >> 78697a727888457abb164311b8/src/main/java/io/github/ >> classgraph/ScanResult.java#L940 >> >> This results in a ConcurrentModificationException since >> ScanResult::getResourcesMatchingPattern is still iterating through >> allResources: >> >> https://github.com/classgraph/classgraph/blob/b9e8165d345753 >> 78697a727888457abb164311b8/src/main/java/io/github/ >> classgraph/ScanResult.java#L383 >> >> I read somewhere (and it makes sense) that escape analysis is performed by >> the compiler, so maybe this is a Scala bug, but this behavior is triggered >> by the JVM switch -XX:+DoEscapeAnalysis , and I couldn't find any >> references to escape analysis information in the Java classfile format >> spec, so that leads me to think this may be a JVM bug (which is why I am >> asking on this list). >> >> Obviously having a garbage collector run a finalizer run while methods of >> the object are still running is a very serious bug. Any pointers as to >> where to report this issue would be appreciated. Thanks! >> >> From vicente.romero at oracle.com Wed Oct 10 01:40:52 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 9 Oct 2018 21:40:52 -0400 Subject: mach5 job name to execute old tonga tests Message-ID: Hi, I think that the old hs-tonga tests were converted to standard open tests and can be executed now with mach5. Does anybody knows what's the mach5 job corresponding to those tests? Thanks, Vicente From david.holmes at oracle.com Wed Oct 10 01:49:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Oct 2018 11:49:08 +1000 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> Message-ID: Hi Igor, Not a review - I fixed the typo in the bug synopsis: Covert -> Convert :) Please ensure you commit with corrected synopsis. Thanks, David On 10/10/2018 3:23 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >> 450 lines changed: 238 ins; 211 del; 1 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? > > webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review > > Thanks, > -- Igor > > From tobias.hartmann at oracle.com Wed Oct 10 06:37:29 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 10 Oct 2018 08:37:29 +0200 Subject: RFR(S): : 8211332: Space for stub routines (code_size2) is too small on new Skylake CPUs In-Reply-To: References: <96e9bbe2280a49ffb5b16c2c95f069ff@sap.com> Message-ID: Hi Ralf, looks good to me as well. I've sponsored your fix: http://hg.openjdk.java.net/jdk/jdk/rev/3ecaae33241a Best regards, Tobias On 02.10.2018 20:03, Vladimir Kozlov wrote: > +1 > > Thanks, > Vladimir > > On 10/2/18 3:53 AM, Thomas St?fe wrote: >> This looks good. Thanks for fixing! >> >> Best Regards, Thomas >> On Tue, Oct 2, 2018 at 12:37 PM Schmelter, Ralf wrote: >>> >>> Please review this change which increases the space available for stub routines on Windows x86. >>> >>> >>> >>> bug: >>> https://bugs.openjdk.java.net/browse/JDK-8211332 >>> >>> webrev: http://cr.openjdk.java.net/~clanger/webrevs/8211332.0/ >>> >>> >>> >>> My Windows x86 slowdebug build crashed in StubRoutines::initialize1(), since the estimated code >>> size (code_size2 defined in stub_routines_x86.hpp) for the initial stubs was too low. I've >>> increase code_size2 by 500 bytes per step, until the assertions vanished. >>> >>> >>> >>> This seems to be 'caused' the new -XX:UseBASE64Intrinsics flag, which is only turned on if the >>> machine supports the AVX-512VL and AVX-512BW extensions, which is the case for Skylake-SP and >>> Skylake X and later/better CPUs. >>> >>> >>> >>> Best regards, >>> >>> Ralf Schmelter From volker.simonis at gmail.com Wed Oct 10 07:31:48 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 10 Oct 2018 09:31:48 +0200 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: Message-ID: Hi Luke, I think the behaviour you see in not related to Escape Analysis at all. It should also occur if you run with -XX:-DoEscapeAnalysis (note that -XX:+DoEscapeAnalysis is the default). I also think that this behaviour doesn't break the Java specification. You have to carefully read the section cited by David. Consider the following small, stand alone program which should simulate your use case quite well: ================================================= import java.util.ArrayList; public class EAFinalizer { private static ArrayList cal = new ArrayList(10_000); static { for (int i = 0; i < 10_000; i++) cal.add(i); } private static ArrayList sink = new ArrayList(); private ArrayList al; public EAFinalizer(ArrayList al) { this.al = new ArrayList(al); } public static void main(String[] args) { while(true) { EAFinalizer eaf = new EAFinalizer(cal); System.err.println("Iterating " + eaf); for (Integer i : eaf.al) { // Create some garbage to trigger GC and finalization sink.add(new int[i.toString().length() * 1_000]); } // System.err.println("Iterated " + eaf); } } @Override protected void finalize() throws Throwable { System.err.println("Finalizing " + this); al.clear(); } } ================================================== Running this program (even with -XX:-DoEscapeAnalysis) will give you: $ java -XX:-DoEscapeAnalysis EAFinalizer Iterating EAFinalizer at 5ecddf8f Iterating EAFinalizer at 3f102e87 Iterating EAFinalizer at 27abe2cd Iterating EAFinalizer at 5f5a92bb Iterating EAFinalizer at 6fdb1f78 Iterating EAFinalizer at 51016012 Iterating EAFinalizer at 29444d75 Iterating EAFinalizer at 2280cdac Finalizing EAFinalizer at 2280cdac Finalizing EAFinalizer at 29444d75 Exception in thread "main" java.util.ConcurrentModificationException at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042) at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996) at EAFinalizer.main(EAFinalizer.java:21) The problem with the program is that in the JIT compiled version of "test()" the EAFinalizer object "eaf" is not reachable in the for loop any more (because the JIT compiler is smart enough to detect that after extracting the AraryList "al" from "eaf", "eaf" is no longer needed). So when a GC is being triggered in the loop because the allocation of the integer array fails, the VM can safely garbage collect the "eaf" object and call its finalizer (notice that within the loop we only reference "eaf"s ArrayList field "al", but not the containing EAFinalizer object). You can easily fix this little toy example by uncommenting the second System.println statement in the "main()" method. This outputs "eaf" and thus keeps it alive. You can also run the program in interpreted mode ("-Xint") in which case you won't see the problem as well. That's because the Interpreter doesn't optimizes as eagerly as the JIT compiler and keeps "eaf" alive for the whole life time of the corresponding method invocation. That's actually what most people naivly expect to be the "guaranteed" life time of an object, but as the specification tells us, that's not required by the standard. If it would, the JIT would miss a whole lot of optimization opportunities. Regards, Volker On Wed, Oct 10, 2018 at 2:36 AM David Holmes wrote: > > Hi Luke, > > Moving this over to hotspot-dev. If it is an issue with the JIT escape > analysis then hotspot-compiler-dev would be the place to raise this. But > a general finalization issue may hit GC and runtime as well, so I > redirected to the broader hotspot-dev. > > Also note that finalizers can run while methods of a class instance are > still in progress [1] > > "Optimizing transformations of a program can be designed that reduce the > number of objects that are reachable to be less than those which would > naively be considered reachable." > > - this is one of the evil things about finalizers. So it may not be a > bug, just surprising. > > Cheers, > David > > [1] https://docs.oracle.com/javase/specs/jls/se11/html/jls-12.html#jls-12.6 > > On 10/10/2018 10:27 AM, Luke Hutchison wrote: > > A user of a library I maintain, ClassGraph, has reported that a finalizer > > is being called on a class while methods of the class are still running, > > indicating that the class, even though it overrides finalize(), is not > > being correctly marked as GlobalEscape, so that escape analysis is trying > > to garbage-collect the class too early. Is this mailing list the correct > > place to report this bug? > > > > Here is the Maven rule for ClassGraph (you will also need to add Scala > > deps) -- you will need version 4.2.8, since I put a mitigation in place in > > version 4.2.9: > > > > > > io.github.classgraph > > classgraph > > 4.2.8 > > > > > > Here is the Scala code that reproduces the problem 100% of the time: > > > > import io.github.classgraph.{ClassGraph, ClassInfo} > > import scala.collection.JavaConverters._ > > import scala.compat.Platform.ConcurrentModificationException > > > > while (true) { > > try { > > new ClassGraph().scan() > > .getResourcesMatchingPattern(".*".r.pattern) > > .asScala > > .map(_.getURL) > > } catch { > > case cme: ConcurrentModificationException => > > cme.printStackTrace() > > throw cme > > } > > } > > > > Then start the JVM with -XX:+DoEscapeAnalysis > > > > The code new ClassGraph().scan() returns a ScanResult, and then as soon as > > ScanResult::getResourcesMatchingPattern is called with the ScanResult as > > the invocation target, escape analysis sees that the ScanResult has "gone > > out of scope" (despite the fact that the ScanResult is the invocation > > target for a currently-running method), and immediately tries to garbage > > collect it. This should not happen (1) since an invocation target should > > never be marked for garbage collection while its methods are being run, and > > (2) since ScanResult overrides Object::finalize, so should have been marked > > as "GlobalEscape" by escape analysis. > > > > The finalizer for the ScanResult class calls ScanResult::close, which > > clears the allResources list: > > > > https://github.com/classgraph/classgraph/blob/b9e8165d34575378697a727888457abb164311b8/src/main/java/io/github/classgraph/ScanResult.java#L940 > > > > This results in a ConcurrentModificationException since > > ScanResult::getResourcesMatchingPattern is still iterating through > > allResources: > > > > https://github.com/classgraph/classgraph/blob/b9e8165d34575378697a727888457abb164311b8/src/main/java/io/github/classgraph/ScanResult.java#L383 > > > > I read somewhere (and it makes sense) that escape analysis is performed by > > the compiler, so maybe this is a Scala bug, but this behavior is triggered > > by the JVM switch -XX:+DoEscapeAnalysis , and I couldn't find any > > references to escape analysis information in the Java classfile format > > spec, so that leads me to think this may be a JVM bug (which is why I am > > asking on this list). > > > > Obviously having a garbage collector run a finalizer run while methods of > > the object are still running is a very serious bug. Any pointers as to > > where to report this issue would be appreciated. Thanks! > > From luke.hutch at gmail.com Wed Oct 10 09:10:01 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Wed, 10 Oct 2018 03:10:01 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: Message-ID: Hi Volker, Thanks for the standalone example, and I agree that your example does not violate the spec, because after the last reference to "eaf" is used in "for (Integer i : eaf.al)", the eaf object is free to be garbage collected. However, this is not an example of the behavior I was illustrating, because the for loop is running within a *static* method, main. I specifically mentioned non-static methods, so that a running method can access "this" at any time (and "this" will never "go out of scope"). My point was that if any *non-static* method of an object instance is currently running (i.e. whenever an object is "currently serving as an invocation target"), that object should never be considered unreachable. I believe that I did not misread the spec, and that this is covered by the wording, "A reachable object is any object that can be accessed in any potential continuing computation from any live thread", since if a non-static method of an object is currently running, that object not only "can be accessed" by a live thread, it *is* currently being accessed by a live thread. In fact, if you minimally modify your code example to put the loop inside a non-static method, this program no longer throws ConcurrentModificationException, illustrating that at least for the example you gave, the compiler and Hotspot are doing the right thing. The following code will iterate until OutOfMemoryError. ================================================= import java.util.ArrayList; public class EAFinalizer { private static ArrayList cal = new ArrayList(10_000); static { for (int i = 0; i < 10_000; i++) cal.add(i); } private static ArrayList sink = new ArrayList(); private ArrayList al; public EAFinalizer(ArrayList al) { this.al = new ArrayList(al); } public static void main(String[] args) { while(true) { EAFinalizer eaf = new EAFinalizer(cal); System.err.println("Iterating " + eaf); eaf.iterate(); // System.err.println("Iterated " + eaf); } } private void iterate() { for (Integer i : al) { // Create some garbage to trigger GC and finalization sink.add(new int[i.toString().length() * 1_000]); } } @Override protected void finalize() throws Throwable { System.err.println("Finalizing " + this); al.clear(); } } ================================================= The reason I didn't give a minimal testcase like the above for the behavior I am seeing is that I have not been able to duplicate this behavior except by running the actual code that was reported to me by the user. Thanks, Luke On Wed, Oct 10, 2018 at 1:32 AM Volker Simonis wrote: > Hi Luke, > > I think the behaviour you see in not related to Escape Analysis at > all. It should also occur if you run with -XX:-DoEscapeAnalysis (note > that -XX:+DoEscapeAnalysis is the default). I also think that this > behaviour doesn't break the Java specification. You have to carefully > read the section cited by David. > > Consider the following small, stand alone program which should > simulate your use case quite well: > > ================================================= > import java.util.ArrayList; > > public class EAFinalizer { > > private static ArrayList cal = new ArrayList(10_000); > static { > for (int i = 0; i < 10_000; i++) cal.add(i); > } > private static ArrayList sink = new ArrayList(); > > private ArrayList al; > > public EAFinalizer(ArrayList al) { > this.al = new ArrayList(al); > } > > public static void main(String[] args) { > while(true) { > EAFinalizer eaf = new EAFinalizer(cal); > System.err.println("Iterating " + eaf); > for (Integer i : eaf.al) { > // Create some garbage to trigger GC and finalization > sink.add(new int[i.toString().length() * 1_000]); > } > // System.err.println("Iterated " + eaf); > } > } > > @Override > protected void finalize() throws Throwable { > System.err.println("Finalizing " + this); > al.clear(); > } > } > ================================================== > > Running this program (even with -XX:-DoEscapeAnalysis) will give you: > > $ java -XX:-DoEscapeAnalysis EAFinalizer > Iterating EAFinalizer at 5ecddf8f > Iterating EAFinalizer at 3f102e87 > Iterating EAFinalizer at 27abe2cd > Iterating EAFinalizer at 5f5a92bb > Iterating EAFinalizer at 6fdb1f78 > Iterating EAFinalizer at 51016012 > Iterating EAFinalizer at 29444d75 > Iterating EAFinalizer at 2280cdac > Finalizing EAFinalizer at 2280cdac > Finalizing EAFinalizer at 29444d75 > Exception in thread "main" java.util.ConcurrentModificationException > at > java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042) > at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996) > at EAFinalizer.main(EAFinalizer.java:21) > > The problem with the program is that in the JIT compiled version of > "test()" the EAFinalizer object "eaf" is not reachable in the for loop > any more (because the JIT compiler is smart enough to detect that > after extracting the AraryList "al" from "eaf", "eaf" is no longer > needed). So when a GC is being triggered in the loop because the > allocation of the integer array fails, the VM can safely garbage > collect the "eaf" object and call its finalizer (notice that within > the loop we only reference "eaf"s ArrayList field "al", but not the > containing EAFinalizer object). > > You can easily fix this little toy example by uncommenting the second > System.println statement in the "main()" method. This outputs "eaf" > and thus keeps it alive. You can also run the program in interpreted > mode ("-Xint") in which case you won't see the problem as well. That's > because the Interpreter doesn't optimizes as eagerly as the JIT > compiler and keeps "eaf" alive for the whole life time of the > corresponding method invocation. That's actually what most people > naivly expect to be the "guaranteed" life time of an object, but as > the specification tells us, that's not required by the standard. If it > would, the JIT would miss a whole lot of optimization opportunities. > > Regards, > Volker > > From aph at redhat.com Wed Oct 10 09:16:13 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 10 Oct 2018 10:16:13 +0100 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: Message-ID: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> On 10/10/2018 10:10 AM, Luke Hutchison wrote: > My point was that if any *non-static* method of an object instance > is currently running (i.e. whenever an object is "currently serving > as an invocation target"), that object should never be considered > unreachable. That's incorrect. > I believe that I did not misread the spec, and that this is covered > by the wording, "A reachable object is any object that can be > accessed in any potential continuing computation from any live > thread", since if a non-static method of an object is currently > running, that object not only "can be accessed" by a live thread, it > *is* currently being accessed by a live thread. No. Believe it or not, a live method does not constitute an access to an object. An object can die and be finalized even before one if its methods is entered. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From david.holmes at oracle.com Wed Oct 10 09:39:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Oct 2018 19:39:03 +1000 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: Message-ID: <9b6e0c25-d359-4256-b476-e72f7ab836bc@oracle.com> On 10/10/2018 7:10 PM, Luke Hutchison wrote: > Hi Volker, > > Thanks for the standalone example, and I agree that your example does not > violate the spec, because after the last reference to "eaf" is used in "for > (Integer i : eaf.al)", the eaf object is free to be garbage collected. > > However, this is not an example of the behavior I was illustrating, because > the for loop is running within a *static* method, main. I specifically > mentioned non-static methods, so that a running method can access "this" at > any time (and "this" will never "go out of scope"). My point was that if > any *non-static* method of an object instance is currently running (i.e. > whenever an object is "currently serving as an invocation target"), that > object should never be considered unreachable. I believe that I did not > misread the spec, and that this is covered by the wording, "A reachable > object is any object that can be accessed in any potential continuing > computation from any live thread", since if a non-static method of an > object is currently running, that object not only "can be accessed" by a > live thread, it *is* currently being accessed by a live thread. This unexpected scenario is exactly what the part of the JLS I referenced is referring to. An executing method need not reference "this" and so the instance can be considered unreachable and hence finalizable. David ----- > In fact, if you minimally modify your code example to put the loop inside a > non-static method, this program no longer throws > ConcurrentModificationException, illustrating that at least for the example > you gave, the compiler and Hotspot are doing the right thing. The following > code will iterate until OutOfMemoryError. > > ================================================= > import java.util.ArrayList; > > public class EAFinalizer { > > private static ArrayList cal = new ArrayList(10_000); > static { > for (int i = 0; i < 10_000; i++) cal.add(i); > } > private static ArrayList sink = new ArrayList(); > > private ArrayList al; > > public EAFinalizer(ArrayList al) { > this.al = new ArrayList(al); > } > > public static void main(String[] args) { > while(true) { > EAFinalizer eaf = new EAFinalizer(cal); > System.err.println("Iterating " + eaf); > eaf.iterate(); > // System.err.println("Iterated " + eaf); > } > } > > private void iterate() { > for (Integer i : al) { > // Create some garbage to trigger GC and finalization > sink.add(new int[i.toString().length() * 1_000]); > } > } > > > @Override > protected void finalize() throws Throwable { > System.err.println("Finalizing " + this); > al.clear(); > } > } > ================================================= > > The reason I didn't give a minimal testcase like the above for the behavior > I am seeing is that I have not been able to duplicate this behavior except > by running the actual code that was reported to me by the user. > > Thanks, > Luke > > > > > On Wed, Oct 10, 2018 at 1:32 AM Volker Simonis > wrote: > >> Hi Luke, >> >> I think the behaviour you see in not related to Escape Analysis at >> all. It should also occur if you run with -XX:-DoEscapeAnalysis (note >> that -XX:+DoEscapeAnalysis is the default). I also think that this >> behaviour doesn't break the Java specification. You have to carefully >> read the section cited by David. >> >> Consider the following small, stand alone program which should >> simulate your use case quite well: >> >> ================================================= >> import java.util.ArrayList; >> >> public class EAFinalizer { >> >> private static ArrayList cal = new ArrayList(10_000); >> static { >> for (int i = 0; i < 10_000; i++) cal.add(i); >> } >> private static ArrayList sink = new ArrayList(); >> >> private ArrayList al; >> >> public EAFinalizer(ArrayList al) { >> this.al = new ArrayList(al); >> } >> >> public static void main(String[] args) { >> while(true) { >> EAFinalizer eaf = new EAFinalizer(cal); >> System.err.println("Iterating " + eaf); >> for (Integer i : eaf.al) { >> // Create some garbage to trigger GC and finalization >> sink.add(new int[i.toString().length() * 1_000]); >> } >> // System.err.println("Iterated " + eaf); >> } >> } >> >> @Override >> protected void finalize() throws Throwable { >> System.err.println("Finalizing " + this); >> al.clear(); >> } >> } >> ================================================== >> >> Running this program (even with -XX:-DoEscapeAnalysis) will give you: >> >> $ java -XX:-DoEscapeAnalysis EAFinalizer >> Iterating EAFinalizer at 5ecddf8f >> Iterating EAFinalizer at 3f102e87 >> Iterating EAFinalizer at 27abe2cd >> Iterating EAFinalizer at 5f5a92bb >> Iterating EAFinalizer at 6fdb1f78 >> Iterating EAFinalizer at 51016012 >> Iterating EAFinalizer at 29444d75 >> Iterating EAFinalizer at 2280cdac >> Finalizing EAFinalizer at 2280cdac >> Finalizing EAFinalizer at 29444d75 >> Exception in thread "main" java.util.ConcurrentModificationException >> at >> java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042) >> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996) >> at EAFinalizer.main(EAFinalizer.java:21) >> >> The problem with the program is that in the JIT compiled version of >> "test()" the EAFinalizer object "eaf" is not reachable in the for loop >> any more (because the JIT compiler is smart enough to detect that >> after extracting the AraryList "al" from "eaf", "eaf" is no longer >> needed). So when a GC is being triggered in the loop because the >> allocation of the integer array fails, the VM can safely garbage >> collect the "eaf" object and call its finalizer (notice that within >> the loop we only reference "eaf"s ArrayList field "al", but not the >> containing EAFinalizer object). >> >> You can easily fix this little toy example by uncommenting the second >> System.println statement in the "main()" method. This outputs "eaf" >> and thus keeps it alive. You can also run the program in interpreted >> mode ("-Xint") in which case you won't see the problem as well. That's >> because the Interpreter doesn't optimizes as eagerly as the JIT >> compiler and keeps "eaf" alive for the whole life time of the >> corresponding method invocation. That's actually what most people >> naivly expect to be the "guaranteed" life time of an object, but as >> the specification tells us, that's not required by the standard. If it >> would, the JIT would miss a whole lot of optimization opportunities. >> >> Regards, >> Volker >> >> From luke.hutch at gmail.com Wed Oct 10 09:39:37 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Wed, 10 Oct 2018 03:39:37 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> Message-ID: On Wed, Oct 10, 2018 at 3:06 AM Andrew Haley wrote: > The problem with using finalizers is that finalizers may be run too > soon (as here,) too late, or never. The "never" possibility really > does happen: sometimes a generational collector moves objects to the > old generation, and as long as there is little memory pressure only > the young generations are collected. You may run out of file handles > long before you run out of memory. I know there are a lot of downsides to finalizers, and I have made it clear in the documentation of my library that they should assign the object concerned in a try-with-resources block. The finalizer is simply there to clean up resources if the user neglects to follow this advice, which is common in Scala, since it doesn't have try-with-resources, so the user has to manually close the object in a "finally" block. I'm not as concerned with the "may never be run" case, since that is really on the users of the library. I'm more concerned about the "may be run while still in scope" case, since that can break things in very surprising ways. So given a class class A { void someMethod() { try { // ... } finally { Reference.reachabilityFence(this); } } } how is it that a call new A().someMethod() does not have a race condition between the "new A()" instance being invoked and reachability analysis realizing that there is a "this" reference in the finally block? Is it always true that there is an overlap between the reference to the invocation target being held and the reference to "this" in the method body being considered as reachable? Or stated as the inverse, is it guaranteed that there will be no gap in time (albeit infinitessimally small) between method invocation and the running of the method in which there will be no reference to the "new A()" object, where the finalizer could still be run? On Wed, Oct 10, 2018 at 3:16 AM Andrew Haley wrote: > > I believe that I did not misread the spec, and that this is covered > > by the wording, "A reachable object is any object that can be > > accessed in any potential continuing computation from any live > > thread", since if a non-static method of an object is currently > > running, that object not only "can be accessed" by a live thread, it > > *is* currently being accessed by a live thread. > > No. Believe it or not, a live method does not constitute an access > to an object. An object can die and be finalized even before one > if its methods is entered. > That is highly surprising. This makes finalizers a lot less reliable and more difficult to use than they could be. Wouldn't it make sense to consider the invocation targets in the stack during liveness analysis, so that out of the cases you gave, "finalizers may be run too soon (as here,) too late, or never", at least the "too soon" case is fixed? From luke.hutch at gmail.com Wed Oct 10 09:48:26 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Wed, 10 Oct 2018 03:48:26 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> Message-ID: On Wed, Oct 10, 2018 at 3:39 AM Luke Hutchison wrote: > So given a class > > class A { > void someMethod() { > try { > // ... > } finally { > Reference.reachabilityFence(this); > } > } > } > (Sorry, that should have a finalizer too..) Another case: given the class class A { A someMethod1() { // ... return this; } A someMethod2() { // ... return this; } public void finalize() { /* ... */ } } will the call new A().someMethod1().someMethod2(); never run the finalizer early, because "this" is returned for method chaining? i.e. does any sort of reference to "this" that cannot be trivially optimized away by the compiler serve the same purpose as Reference.reachabilityFence(this) ? Or will the finalizer never be run while someMethod1() is running, but may be run while someMethod2() is running, since the return value of someMethod2() is being discarded? From aph at redhat.com Wed Oct 10 09:50:51 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 10 Oct 2018 10:50:51 +0100 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> Message-ID: <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> On 10/10/2018 10:39 AM, Luke Hutchison wrote: > > class A { > void someMethod() { > try { > // ... > } finally { > Reference.reachabilityFence(this); > } > } > } > > how is it that a call > > new A().someMethod() > > does not have a race condition between the "new A()" instance being > invoked and reachability analysis realizing that there is a "this" > reference in the finally block? Is it always true that there is an > overlap between the reference to the invocation target being held > and the reference to "this" in the method body being considered as > reachable? You need two threads to have a race condition. I'm not sure what you're trying to ask: the reachabilityFence must run strictly after the body of the try clause, and it the object will be reachable until then. > Or stated as the inverse, is it guaranteed that there will be no gap in > time (albeit infinitessimally small) between method invocation and the > running of the method in which there will be no reference to the "new A()" > object, where the finalizer could still be run? The finalizer cannot be run as long as the object is reachable. The object is reachable until the reachabilityFence. > On Wed, Oct 10, 2018 at 3:16 AM Andrew Haley wrote: > >>> I believe that I did not misread the spec, and that this is covered >>> by the wording, "A reachable object is any object that can be >>> accessed in any potential continuing computation from any live >>> thread", since if a non-static method of an object is currently >>> running, that object not only "can be accessed" by a live thread, it >>> *is* currently being accessed by a live thread. >> >> No. Believe it or not, a live method does not constitute an access >> to an object. An object can die and be finalized even before one >> if its methods is entered. > > That is highly surprising. Yes, it is . You are not the first to be surprised by this. > This makes finalizers a lot less reliable and more difficult to use > than they could be. > > Wouldn't it make sense to consider the invocation targets in the stack > during liveness analysis, so that out of the cases you gave, "finalizers > may be run too soon (as here,) too late, or never", at least the "too soon" > case is fixed? We considered doing that; the discussion is online somewhere. I was in favour of doing what you suggest, but there is some performance impact of preserving liveness for all methods. In the end it was decided that we'd provide reachabilityFence. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Oct 10 09:54:39 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 10 Oct 2018 10:54:39 +0100 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> Message-ID: <0f3adcbe-6783-2b8e-14cb-7ccc91949099@redhat.com> On 10/10/2018 10:48 AM, Luke Hutchison wrote: > Another case: given the class > > class A { > A someMethod1() { > // ... > return this; > } > > A someMethod2() { > // ... > return this; > } > > public void finalize() { /* ... */ } > } > > will the call > > new A().someMethod1().someMethod2(); > > never run the finalizer early, because "this" is returned for method > chaining? It might. If those methods have no visible side effects then they may be elided altogether. The easiest way to think about it is to consider that every method my be inlined, and all references to "this" elided. > i.e. does any sort of reference to "this" that cannot be trivially > optimized away by the compiler serve the same purpose as > Reference.reachabilityFence(this) ? If it were so we would not have had to provide reachabilityFence. > Or will the finalizer never be run while someMethod1() is running, but may > be run while someMethod2() is running, since the return value of > someMethod2() is being discarded? It doesn't help because the bodies of someMethod1 and someMethod2 may be inlined in the caller. The only thing that matters is the last access of a field of Class A. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From luke.hutch at gmail.com Wed Oct 10 10:22:45 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Wed, 10 Oct 2018 04:22:45 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> Message-ID: On Wed, Oct 10, 2018 at 3:50 AM Andrew Haley wrote: > > This makes finalizers a lot less reliable and more difficult to use > > than they could be. > > > > Wouldn't it make sense to consider the invocation targets in the stack > > during liveness analysis, so that out of the cases you gave, "finalizers > > may be run too soon (as here,) too late, or never", at least the "too > soon" > > case is fixed? > > We considered doing that; the discussion is online somewhere. I was in > favour of doing what you suggest, but there is some performance impact > of preserving liveness for all methods. In the end it was decided that > we'd provide reachabilityFence. > The stack already has to be walked for reachability analysis, and there would no doubt be some performance impact for marking invocation targets as reachable, but that already has to be done for a conceivably much larger set of objects than just the number of stack frames, so it seems like the overall impact to performance would be minimal. This is especially true given that only classes that override finalize() would need to be tracked in this way. Was this benchmarked before this decision was made, in order to determine actual rather than theorized performance impact? I don't know what went into the discussion you refer to, but not doing what is arguably "the right thing" in the name of what is probably a tiny performance margin doesn't seem strongly justifiable. I understand your arguments about elision of methods etc., but I really can't believe that a currently running method would not be considered a strong enough reference to "this". Forcing users to put reachabilityFence wrappers (or "synchronized (this)" wrappers) around every single method of every object that contains a finalizer seems like a very poor fix to this problem. It is well known that a lot of things about finalizers are broken. But why not make them a little less broken? You need two threads to have a race condition. I'm not sure what > you're trying to ask: the reachabilityFence must run strictly after > the body of the try clause, and it the object will be reachable until > then. > Right -- one thread is the garbage collector. What I'm asking is with a method invocation "new A().someMethod()", is there conceivably a gap in time between when the runtime realizes that there are no more references to the "A" object in the caller, and when it realizes that there is a fenced reference to "this" in "someMethod"? I'm guessing the answer is no, otherwise the reachabilityFence mechanism would not always work, but I was asking about the exact ordering of the dropping of the invocation target reference, and adding object references within the invoked method to the set of reachable objects. From bsrbnd at gmail.com Wed Oct 10 10:37:28 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 10 Oct 2018 12:37:28 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> <882cb210-5870-2292-e78f-62857abdc54f@redhat.com> Message-ID: On Mon, 8 Oct 2018 at 19:18, B. Blaser wrote: > > On Mon, 8 Oct 2018 at 17:19, Andrew Haley wrote: > > > > On 10/08/2018 12:16 PM, B. Blaser wrote: > > > The current prototype was intended to scrupulously implement what the > > > JVMS explicitly allows for baload/bastore instructions. > > > But helper intrinsics might be good alternatives too, we could try both. > > > > So I tried it with a rough C++/assembly language test, and it doesn't > > make much sense to use this for default boolean arrays. The update > > sequence on AArch64 is > > > > 0: ldxrb w2, [x0]; > > and w2, w2, w1 > > stxrb w3, w2, [x0] > > cbnz w3, 0b > > > > (combined with a bunch of uninteresting shifts and logical > > operations.) We have to load a byte in exclusive state, AND or OR it > > as appropriate, then store it, still in exclusive state. That gets us > > the atomicity we need. > > > > 10**9 random set operations on an 8 Mbit array take 0.550s; with a > > boolean array the time is 0.150s. This is the local overhead of the > > load/store exclusive. > > Reducing memory consumption has a price which doesn't seem too > expensive, fortunately. On x86, I guess we have BTS/BTR (bit set and reset) with the LOCK prefix to assure atomicity. I'll do some experiment and update the prototype accordingly. Bernard From luke.hutch at gmail.com Wed Oct 10 10:37:28 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Wed, 10 Oct 2018 04:37:28 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> Message-ID: > > We considered doing that; the discussion is online somewhere. I was in > favour of doing what you suggest, but there is some performance impact > of preserving liveness for all methods. In the end it was decided that > we'd provide reachabilityFence. Separate question: other than using "synchronized (this)" around every method (including the finalizer), is there a way to emulate reachabilityFence in JDK 7 and 8? For example, is there some sort of standard construct I can put in a finally block that references a field of the object with some sort of a no-op (or low-cost op) that is guaranteed not to be optimized away? From luke.hutch at gmail.com Wed Oct 10 10:56:22 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Wed, 10 Oct 2018 04:56:22 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> Message-ID: On Wed, Oct 10, 2018 at 4:37 AM Luke Hutchison wrote: > Separate question: other than using "synchronized (this)" around every > method (including the finalizer), is there a way to emulate > reachabilityFence in JDK 7 and 8? > For examle, I just tested this, and it seems to prevent the finalizer from being run early (I create a local static instance FENCE of type Fence to try to avoid Hotspot optimizing away the empty method Fence::reachabilityFence) -- but is there a reason this could fail with future Hotspot optimizations?: ========================================= private static class Fence { public void reachabilityFence(Object object) { } } private static Fence FENCE = new Fence(); private Object fencedField; private void fencedMethod() { try { // ... } finally { FENCE.reachabilityFence(this); // Or maybe: FENCE.reachabilityFence(this.fencedField); } } From ralf.schmelter at sap.com Wed Oct 10 11:50:23 2018 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Wed, 10 Oct 2018 11:50:23 +0000 Subject: RFR(S): : 8211332: Space for stub routines (code_size2) is too small on new Skylake CPUs In-Reply-To: References: <96e9bbe2280a49ffb5b16c2c95f069ff@sap.com> Message-ID: <10bf94fd68a841769fe79582a084ca57@sap.com> Hi all, thanks for the reviews und sponsoring the fix. Best regards, Ralf -----Original Message----- From: Tobias Hartmann Sent: Mittwoch, 10. Oktober 2018 08:37 To: Vladimir Kozlov ; Thomas St?fe ; Schmelter, Ralf Cc: HotSpot Open Source Developers Subject: Re: RFR(S): : 8211332: Space for stub routines (code_size2) is too small on new Skylake CPUs Hi Ralf, looks good to me as well. I've sponsored your fix: http://hg.openjdk.java.net/jdk/jdk/rev/3ecaae33241a Best regards, Tobias On 02.10.2018 20:03, Vladimir Kozlov wrote: > +1 > > Thanks, > Vladimir > > On 10/2/18 3:53 AM, Thomas St?fe wrote: >> This looks good. Thanks for fixing! >> >> Best Regards, Thomas >> On Tue, Oct 2, 2018 at 12:37 PM Schmelter, Ralf wrote: >>> >>> Please review this change which increases the space available for stub routines on Windows x86. >>> >>> >>> >>> bug: >>> https://bugs.openjdk.java.net/browse/JDK-8211332 >>> >>> webrev: http://cr.openjdk.java.net/~clanger/webrevs/8211332.0/ >>> >>> >>> >>> My Windows x86 slowdebug build crashed in StubRoutines::initialize1(), since the estimated code >>> size (code_size2 defined in stub_routines_x86.hpp) for the initial stubs was too low. I've >>> increase code_size2 by 500 bytes per step, until the assertions vanished. >>> >>> >>> >>> This seems to be 'caused' the new -XX:UseBASE64Intrinsics flag, which is only turned on if the >>> machine supports the AVX-512VL and AVX-512BW extensions, which is the case for Skylake-SP and >>> Skylake X and later/better CPUs. >>> >>> >>> >>> Best regards, >>> >>> Ralf Schmelter From matthias.baesken at sap.com Wed Oct 10 12:36:02 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 10 Oct 2018 12:36:02 +0000 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 In-Reply-To: <5dda0126bc7d30257c1644214a256e29c860d5dc.camel@redhat.com> References: <03f8fa68-a758-d956-cfd3-1572b093d3f9@oracle.com> <5dda0126bc7d30257c1644214a256e29c860d5dc.camel@redhat.com> Message-ID: Hello Severin and Vladimir , I tried a slowdebug build but the expected error at 2857 sprintf(buffer, "Bytecode %d: %s", bci(), Bytecodes::name(bc())); does not show up . So it seem the float-output I addressed in my webrev is handled in another way than this coding part by gcc . Should I still fix this as well and use snprintf , and post another webrev ? Best regards, Matthias > -----Original Message----- > From: Severin Gehwolf > Sent: Dienstag, 9. Oktober 2018 18:00 > To: Vladimir Kozlov ; Baesken, Matthias > ; 'hotspot-dev at openjdk.java.net' dev at openjdk.java.net> > Subject: Re: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc > 7.3.1 > > On Tue, 2018-10-09 at 08:06 -0700, Vladimir Kozlov wrote: > > Why there is no complain about sprintf at line 2857 which uses %s? : > > > http://hg.openjdk.java.net/jdk/jdk/file/d24b89390f6c/src/hotspot/share/op > to/parse2.cpp#l2857 > > Perhaps because this is non-product code? > > Thanks, > Severin From aph at redhat.com Wed Oct 10 12:36:29 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 10 Oct 2018 13:36:29 +0100 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> Message-ID: <1b3aa942-0d81-fc4f-dbb1-c9d616cf43d9@redhat.com> On 10/10/2018 11:22 AM, Luke Hutchison wrote: > On Wed, Oct 10, 2018 at 3:50 AM Andrew Haley wrote: > >>> This makes finalizers a lot less reliable and more difficult to use >>> than they could be. >>> >>> Wouldn't it make sense to consider the invocation targets in the stack >>> during liveness analysis, so that out of the cases you gave, "finalizers >>> may be run too soon (as here,) too late, or never", at least the "too >> soon" >>> case is fixed? >> >> We considered doing that; the discussion is online somewhere. I was in >> favour of doing what you suggest, but there is some performance impact >> of preserving liveness for all methods. In the end it was decided that >> we'd provide reachabilityFence. > > The stack already has to be walked for reachability analysis, and there > would no doubt be some performance impact for marking invocation targets as > reachable, but that already has to be done for a conceivably much larger > set of objects than just the number of stack frames, so it seems like the > overall impact to performance would be minimal. Not necessarily. A JIT compiler has a great deal of latitude to reorder operations, and if it can see that the only access to an Object O before it dies is to read Field X, it can generate code to read X, cache it somewhere, and delete the last live reference to O. > This is especially true given that only classes that override > finalize() would need to be tracked in this way. Was this > benchmarked before this decision was made, in order to determine > actual rather than theorized performance impact? I don't know. I do know that it would increase the lifetime of temporaries, and thereby increase pressure on the register allocator. It would have been interesting to change the Java specification to do that, but it would be difficult. > I don't know what went into the discussion you refer to, but not > doing what is arguably "the right thing" in the name of what is > probably a tiny performance margin doesn't seem strongly > justifiable. I understand your arguments about elision of methods > etc., but I really can't believe that a currently running method > would not be considered a strong enough reference to "this". Believe it. It is true. We have been dicussing problems with finalization for more than a decade, and almost every possible argument has been made. reachabilityFence was a great step forward. > Forcing users to put reachabilityFence wrappers (or "synchronized > (this)" wrappers) around every single method of every object that > contains a finalizer seems like a very poor fix to this problem. > It is well known that a lot of things about finalizers are > broken. But why not make them a little less broken? Again, because it would increase the lifetime of temporaries. >> You need two threads to have a race condition. I'm not sure what >> you're trying to ask: the reachabilityFence must run strictly after >> the body of the try clause, and it the object will be reachable until >> then. > > Right -- one thread is the garbage collector. > > What I'm asking is with a method invocation "new A().someMethod()", is > there conceivably a gap in time between when the runtime realizes that > there are no more references to the "A" object in the caller, and when it > realizes that there is a fenced reference to "this" in "someMethod"? No, there cannot be. "this" has to be kept alive because it's an input to the reachabilityFence. > I'm guessing the answer is no, otherwise the reachabilityFence > mechanism would not always work, but I was asking about the exact > ordering of the dropping of the invocation target reference, and > adding object references within the invoked method to the set of > reachable objects. "this" is an input to the reachabilityFence, and the fence cannot be moved. It's important to realize that reachabilityFence is magic and is known to the compilers. It's similar to a volatile store of "this" but in some ways even stronger, and with less runtime overhead. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Oct 10 12:38:34 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 10 Oct 2018 13:38:34 +0100 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> Message-ID: On 10/10/2018 11:56 AM, Luke Hutchison wrote: > For examle, I just tested this, and it seems to prevent the finalizer from > being run early (I create a local static instance FENCE of type Fence to > try to avoid Hotspot optimizing away the empty method > Fence::reachabilityFence) -- but is there a reason this could fail with > future Hotspot optimizations?: Absolutely, yes. I'm surprised it works. A volatile store will probably work, but even that isn't guaranteed unless someone reads the result of the store. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.stuefe at gmail.com Wed Oct 10 12:46:03 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 10 Oct 2018 14:46:03 +0200 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 In-Reply-To: References: Message-ID: Hi Matthias, could you please use jio_snprintf, and also specify sizeof(buffer) instead of literally re-specifying the buffer size? Thanks! Thomas On Tue, Oct 9, 2018 at 3:51 PM Baesken, Matthias wrote: > > Hello, please review this gcc 7 related build fix . > > > When building on Linux with gcc 7.3.1 I run into this compile error : > > /open_jdk/jdk_just_clone/jdk/src/hotspot/share/opto/parse2.cpp:1349:14: note: 'sprintf' output 2 or more bytes (assuming 2147488583) into a destination of size 30 > sprintf(prob_str_buf, "%g", prob); > ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ > cc1plus: all warnings being treated as errors > > > The issue is that we print a floating point number into a fixed size buffer for output . > > I adjusted the coding a bit : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211929.0/ > > https://bugs.openjdk.java.net/browse/JDK-8211929 > > > > Best regards, Matthias From sgehwolf at redhat.com Wed Oct 10 12:46:37 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 10 Oct 2018 14:46:37 +0200 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 In-Reply-To: References: <03f8fa68-a758-d956-cfd3-1572b093d3f9@oracle.com> <5dda0126bc7d30257c1644214a256e29c860d5dc.camel@redhat.com> Message-ID: <0e60556ee05a527e25383241e56128575d896b45.camel@redhat.com> Hi Matthias, On Wed, 2018-10-10 at 12:36 +0000, Baesken, Matthias wrote: > Hello Severin and Vladimir , > I tried a slowdebug build but the expected error at > > 2857 sprintf(buffer, "Bytecode %d: %s", bci(), Bytecodes::name(bc())); > > does not show up . > So it seem the float-output I addressed in my webrev is handled in another way than this coding part by gcc . > > Should I still fix this as well and use snprintf , and post another webrev ? That matches what I've seen with JDK-8210647. The optimization level matters for warnings to get produced. libsaproc was previously compiled with -O0 and post-change, JDK-8210836 happened. The reporter had gcc 7.3.0. Apparently it wasn't an issue prior JDK-8210647 when no optimization was being used. Hence, I'm not surprised you don't see the warning in slowdebug. Have you tried fastdebug? I'll leave it to Vladimir to decide whether he wants it changed or not. Thanks, Severin > Best regards, Matthias > > > > > > -----Original Message----- > > From: Severin Gehwolf > > Sent: Dienstag, 9. Oktober 2018 18:00 > > To: Vladimir Kozlov ; Baesken, Matthias > > ; 'hotspot-dev at openjdk.java.net' > dev at openjdk.java.net> > > Subject: Re: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc > > 7.3.1 > > > > On Tue, 2018-10-09 at 08:06 -0700, Vladimir Kozlov wrote: > > > Why there is no complain about sprintf at line 2857 which uses %s? : > > > > > > > http://hg.openjdk.java.net/jdk/jdk/file/d24b89390f6c/src/hotspot/share/op > > to/parse2.cpp#l2857 > > > > Perhaps because this is non-product code? > > > > Thanks, > > Severin > > From david.lloyd at redhat.com Wed Oct 10 12:59:53 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 10 Oct 2018 07:59:53 -0500 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <1b3aa942-0d81-fc4f-dbb1-c9d616cf43d9@redhat.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <1b3aa942-0d81-fc4f-dbb1-c9d616cf43d9@redhat.com> Message-ID: On Wed, Oct 10, 2018 at 7:37 AM Andrew Haley wrote: > On 10/10/2018 11:22 AM, Luke Hutchison wrote: > > I don't know what went into the discussion you refer to, but not > > doing what is arguably "the right thing" in the name of what is > > probably a tiny performance margin doesn't seem strongly > > justifiable. I understand your arguments about elision of methods > > etc., but I really can't believe that a currently running method > > would not be considered a strong enough reference to "this". > > Believe it. It is true. It helps to think of "this" as being simply an invisible zeroth method parameter. If you don't refer to a parameter value in the body of a method, why should the compiler keep it around? -- - DML From luke.hutch at gmail.com Wed Oct 10 13:08:25 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Wed, 10 Oct 2018 07:08:25 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <1b3aa942-0d81-fc4f-dbb1-c9d616cf43d9@redhat.com> Message-ID: OK, I understand the scope of the problem much better now. Thank you all for taking the time to patiently explain all of this, despite the fact that as Andrew pointed out, this has been debated for over a decade. I removed the finalizer from my class, and I'm punting responsibility to the caller to simply call the close() method if they don't want to leak resources. I guess I simply didn't want to believe that finalizers were as broken as they were rumored to be -- and I don't think the full scope of these problems can be fully grasped until there's an issue like this that has to be debugged! From erik.osterlund at oracle.com Wed Oct 10 13:09:41 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 10 Oct 2018 15:09:41 +0200 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> Message-ID: <5BBDFA15.80702@oracle.com> Hi Andrew, The implementation of reachabilityFence was changed recently to become a normal static method, and stop forcing no inlining. No magic and special treatment by HotSpot. I think the analysis made was that the oops would have to be kept alive based on the bytecode level live analysis, as required by deoptimization to work as expected. I was curious if this works with Graal and asked Graal folks about it. Turns out it works the same for Graal as JIT and AoT, due to the need to support deoptimization in those configurations, leading the oops to be kept alive. But with SVM, an actual intrinsic will be added to support this operation, as there is no need for deoptimization in SVM, causing the compiler to go a bit further. Thanks, /Erik On 2018-10-10 14:38, Andrew Haley wrote: > On 10/10/2018 11:56 AM, Luke Hutchison wrote: >> For examle, I just tested this, and it seems to prevent the finalizer from >> being run early (I create a local static instance FENCE of type Fence to >> try to avoid Hotspot optimizing away the empty method >> Fence::reachabilityFence) -- but is there a reason this could fail with >> future Hotspot optimizations?: > Absolutely, yes. I'm surprised it works. > > A volatile store will probably work, but even that isn't guaranteed > unless someone reads the result of the store. > From aph at redhat.com Wed Oct 10 13:45:43 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 10 Oct 2018 14:45:43 +0100 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <5BBDFA15.80702@oracle.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <5BBDFA15.80702@oracle.com> Message-ID: <7008238b-ee6e-95cb-cf43-a5abc322c50f@redhat.com> On 10/10/2018 02:09 PM, Erik ?sterlund wrote: > The implementation of reachabilityFence was changed recently to become a > normal static method, and stop forcing no inlining. No magic and special > treatment by HotSpot. I'm sure it depends on the JIT being used. I'm slightly surprised something so simple works, given that a normal empty method could be trivially inlined and the reference dropped. And we know that does happen, so I wonder why not here? Incidentally, I can't find the review thread for 8199462. > I think the analysis made was that the oops would have to be kept > alive based on the bytecode level live analysis, as required by > deoptimization to work as expected. > I was curious if this works with Graal and asked Graal folks about it. > Turns out it works the same for Graal as JIT and AoT, due to the need to > support deoptimization in those configurations, leading the oops to be > kept alive. But with SVM, an actual intrinsic will be added to support > this operation, as there is no need for deoptimization in SVM, causing > the compiler to go a bit further. Mm, yes. It's not at all nice that the correctness of the Java library relies on something which seems to be a side effect of the design of the JIT. Still, it's up to other JITs to provide their own implementation. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From matthias.baesken at sap.com Wed Oct 10 15:50:38 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 10 Oct 2018 15:50:38 +0000 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 In-Reply-To: <0e60556ee05a527e25383241e56128575d896b45.camel@redhat.com> References: <03f8fa68-a758-d956-cfd3-1572b093d3f9@oracle.com> <5dda0126bc7d30257c1644214a256e29c860d5dc.camel@redhat.com> <0e60556ee05a527e25383241e56128575d896b45.camel@redhat.com> Message-ID: Hi Thomas / Severin, I prepared a second webrev , this uses jio_snprintf and adjusts the other line found by Vladimir as well : http://cr.openjdk.java.net/~mbaesken/webrevs/8211929.1/ > Hence, I'm not surprised you don't see the warning in slowdebug. Have > you tried fastdebug? I have, but do not see a compiler warning . Best regards, Matthias > -----Original Message----- > From: Severin Gehwolf > Sent: Mittwoch, 10. Oktober 2018 14:47 > To: Baesken, Matthias ; Vladimir Kozlov > ; 'hotspot-dev at openjdk.java.net' dev at openjdk.java.net> > Subject: Re: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc > 7.3.1 > > Hi Matthias, > > On Wed, 2018-10-10 at 12:36 +0000, Baesken, Matthias wrote: > > Hello Severin and Vladimir , > > I tried a slowdebug build but the expected error at > > > > 2857 sprintf(buffer, "Bytecode %d: %s", bci(), Bytecodes::name(bc())); > > > > does not show up . > > So it seem the float-output I addressed in my webrev is handled in > another way than this coding part by gcc . > > > > Should I still fix this as well and use snprintf , and post another webrev ? > > That matches what I've seen with JDK-8210647. The optimization level > matters for warnings to get produced. libsaproc was previously compiled > with -O0 and post-change, JDK-8210836 happened. The reporter had gcc > 7.3.0. Apparently it wasn't an issue prior JDK-8210647 when no > optimization was being used. > > Hence, I'm not surprised you don't see the warning in slowdebug. Have > you tried fastdebug? > > I'll leave it to Vladimir to decide whether he wants it changed or not. > > Thanks, > Severin > > > Best regards, Matthias > > > > > > > > > > > -----Original Message----- > > > From: Severin Gehwolf > > > Sent: Dienstag, 9. Oktober 2018 18:00 > > > To: Vladimir Kozlov ; Baesken, Matthias > > > ; 'hotspot-dev at openjdk.java.net' > > > dev at openjdk.java.net> > > > Subject: Re: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with > gcc > > > 7.3.1 > > > > > > On Tue, 2018-10-09 at 08:06 -0700, Vladimir Kozlov wrote: > > > > Why there is no complain about sprintf at line 2857 which uses %s? : > > > > > > > > > > > http://hg.openjdk.java.net/jdk/jdk/file/d24b89390f6c/src/hotspot/share/op > > > to/parse2.cpp#l2857 > > > > > > Perhaps because this is non-product code? > > > > > > Thanks, > > > Severin > > > > From kim.barrett at oracle.com Wed Oct 10 15:53:50 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 10 Oct 2018 11:53:50 -0400 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <719DC264-C858-43D2-9589-178564F67EC2@gmail.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <719DC264-C858-43D2-9589-178564F67EC2@gmail.com> Message-ID: > On Oct 10, 2018, at 10:01 AM, Attila Szegedi wrote: > > I think the general consensus is that finalization is a shunned feature[1]. Weakening optimizations, or really making any tradeoff no matter how small for the purpose of improving finalization ergonomics is not going to happen as there?s no perceived value in doing it. Finalization was deprecated in JDK 9 (*). There's an ongoing (though progressing slowly) process of eliminating its usage in the JDK. Sometime after that it may go away entirely, or perhaps require some sort of explicit opt-in, making it possible to eliminate various GC overheads associated with it. (*) Use PhantomReference, or a helper like java.lang.ref.Cleaner. Though that change the liveness question in this thread. From volker.simonis at gmail.com Wed Oct 10 16:17:03 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 10 Oct 2018 18:17:03 +0200 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <7008238b-ee6e-95cb-cf43-a5abc322c50f@redhat.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <5BBDFA15.80702@oracle.com> <7008238b-ee6e-95cb-cf43-a5abc322c50f@redhat.com> Message-ID: On Wed, Oct 10, 2018 at 5:16 PM Andrew Haley wrote: > > On 10/10/2018 02:09 PM, Erik ?sterlund wrote: > > The implementation of reachabilityFence was changed recently to become a > > normal static method, and stop forcing no inlining. No magic and special > > treatment by HotSpot. > > I'm sure it depends on the JIT being used. I'm slightly surprised > something so simple works, given that a normal empty method could be > trivially inlined and the reference dropped. And we know that does > happen, so I wonder why not here? > > Incidentally, I can't find the review thread for 8199462. The thread was called "[PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup" and the bug was only created at the very end of the discussion. You can find it at: http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-February/thread.html#51221 http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-March/thread.html#51863 Very interesting reading :) > > > I think the analysis made was that the oops would have to be kept > > alive based on the bytecode level live analysis, as required by > > deoptimization to work as expected. > > > I was curious if this works with Graal and asked Graal folks about it. > > Turns out it works the same for Graal as JIT and AoT, due to the need to > > support deoptimization in those configurations, leading the oops to be > > kept alive. But with SVM, an actual intrinsic will be added to support > > this operation, as there is no need for deoptimization in SVM, causing > > the compiler to go a bit further. > > Mm, yes. It's not at all nice that the correctness of the Java library > relies on something which seems to be a side effect of the design of > the JIT. Still, it's up to other JITs to provide their own > implementation. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Oct 10 16:24:11 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 10 Oct 2018 17:24:11 +0100 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <5BBDFA15.80702@oracle.com> <7008238b-ee6e-95cb-cf43-a5abc322c50f@redhat.com> Message-ID: <1ff6575a-daaa-7eb6-8daa-09e748557a5b@redhat.com> On 10/10/2018 05:17 PM, Volker Simonis wrote: > On Wed, Oct 10, 2018 at 5:16 PM Andrew Haley wrote: > >> Incidentally, I can't find the review thread for 8199462. > > The thread was called "[PATCH] Reduce Chance Of Mistakenly Early > Backing Memory Cleanup" and the bug was only created at the very end > of the discussion. You can find it at: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-February/thread.html#51221 > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-March/thread.html#51863 LOL! Why did I not guess? > Very interesting reading :) On Feb 7, 2018, at 7:22 AM, Vladimir Ivanov wrote: > > JIT-compilers in HotSpot aggressively prune dead locals [1], but > they do that based on method bytecode analysis [2] (and not on > optimized IR). So, any usage of a local extends its live range, > even if that usage is eliminated in generated code. It means an oop > in that local will live past its last usage in generated code and > all safepoints (in generated code) till last usage (on bytecode > level) will enumerate the local it is held in. That's just a fscking bug, surely. I'm appalled that we rely on that. Appalled, I tell you! ;-) -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From vladimir.kozlov at oracle.com Wed Oct 10 16:42:31 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 10 Oct 2018 09:42:31 -0700 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 In-Reply-To: References: <03f8fa68-a758-d956-cfd3-1572b093d3f9@oracle.com> <5dda0126bc7d30257c1644214a256e29c860d5dc.camel@redhat.com> <0e60556ee05a527e25383241e56128575d896b45.camel@redhat.com> Message-ID: <7b7ae1da-02ff-7d34-1106-812b8e589c2c@oracle.com> Looks good. Thanks, Vladimir On 10/10/18 8:50 AM, Baesken, Matthias wrote: > Hi Thomas / Severin, I prepared a second webrev , this uses jio_snprintf and adjusts the other line found by Vladimir as well : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211929.1/ > > >> Hence, I'm not surprised you don't see the warning in slowdebug. Have >> you tried fastdebug? > > I have, but do not see a compiler warning . > > Best regards, Matthias > > >> -----Original Message----- >> From: Severin Gehwolf >> Sent: Mittwoch, 10. Oktober 2018 14:47 >> To: Baesken, Matthias ; Vladimir Kozlov >> ; 'hotspot-dev at openjdk.java.net' > dev at openjdk.java.net> >> Subject: Re: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc >> 7.3.1 >> >> Hi Matthias, >> >> On Wed, 2018-10-10 at 12:36 +0000, Baesken, Matthias wrote: >>> Hello Severin and Vladimir , >>> I tried a slowdebug build but the expected error at >>> >>> 2857 sprintf(buffer, "Bytecode %d: %s", bci(), Bytecodes::name(bc())); >>> >>> does not show up . >>> So it seem the float-output I addressed in my webrev is handled in >> another way than this coding part by gcc . >>> >>> Should I still fix this as well and use snprintf , and post another webrev ? >> >> That matches what I've seen with JDK-8210647. The optimization level >> matters for warnings to get produced. libsaproc was previously compiled >> with -O0 and post-change, JDK-8210836 happened. The reporter had gcc >> 7.3.0. Apparently it wasn't an issue prior JDK-8210647 when no >> optimization was being used. >> >> Hence, I'm not surprised you don't see the warning in slowdebug. Have >> you tried fastdebug? >> >> I'll leave it to Vladimir to decide whether he wants it changed or not. >> >> Thanks, >> Severin >> >>> Best regards, Matthias >>> >>> >>> >>> >>>> -----Original Message----- >>>> From: Severin Gehwolf >>>> Sent: Dienstag, 9. Oktober 2018 18:00 >>>> To: Vladimir Kozlov ; Baesken, Matthias >>>> ; 'hotspot-dev at openjdk.java.net' >> >>> dev at openjdk.java.net> >>>> Subject: Re: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with >> gcc >>>> 7.3.1 >>>> >>>> On Tue, 2018-10-09 at 08:06 -0700, Vladimir Kozlov wrote: >>>>> Why there is no complain about sprintf at line 2857 which uses %s? : >>>>> >>>> >>>> >> http://hg.openjdk.java.net/jdk/jdk/file/d24b89390f6c/src/hotspot/share/op >>>> to/parse2.cpp#l2857 >>>> >>>> Perhaps because this is non-product code? >>>> >>>> Thanks, >>>> Severin >>> >>> > From thomas.stuefe at gmail.com Wed Oct 10 16:59:27 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 10 Oct 2018 18:59:27 +0200 Subject: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc 7.3.1 In-Reply-To: References: <03f8fa68-a758-d956-cfd3-1572b093d3f9@oracle.com> <5dda0126bc7d30257c1644214a256e29c860d5dc.camel@redhat.com> <0e60556ee05a527e25383241e56128575d896b45.camel@redhat.com> Message-ID: Looks fine, Matthias. Cheers, Thomas On Wed, Oct 10, 2018 at 5:50 PM Baesken, Matthias wrote: > > Hi Thomas / Severin, I prepared a second webrev , this uses jio_snprintf and adjusts the other line found by Vladimir as well : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211929.1/ > > > > Hence, I'm not surprised you don't see the warning in slowdebug. Have > > you tried fastdebug? > > I have, but do not see a compiler warning . > > Best regards, Matthias > > > > -----Original Message----- > > From: Severin Gehwolf > > Sent: Mittwoch, 10. Oktober 2018 14:47 > > To: Baesken, Matthias ; Vladimir Kozlov > > ; 'hotspot-dev at openjdk.java.net' > dev at openjdk.java.net> > > Subject: Re: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with gcc > > 7.3.1 > > > > Hi Matthias, > > > > On Wed, 2018-10-10 at 12:36 +0000, Baesken, Matthias wrote: > > > Hello Severin and Vladimir , > > > I tried a slowdebug build but the expected error at > > > > > > 2857 sprintf(buffer, "Bytecode %d: %s", bci(), Bytecodes::name(bc())); > > > > > > does not show up . > > > So it seem the float-output I addressed in my webrev is handled in > > another way than this coding part by gcc . > > > > > > Should I still fix this as well and use snprintf , and post another webrev ? > > > > That matches what I've seen with JDK-8210647. The optimization level > > matters for warnings to get produced. libsaproc was previously compiled > > with -O0 and post-change, JDK-8210836 happened. The reporter had gcc > > 7.3.0. Apparently it wasn't an issue prior JDK-8210647 when no > > optimization was being used. > > > > Hence, I'm not surprised you don't see the warning in slowdebug. Have > > you tried fastdebug? > > > > I'll leave it to Vladimir to decide whether he wants it changed or not. > > > > Thanks, > > Severin > > > > > Best regards, Matthias > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: Severin Gehwolf > > > > Sent: Dienstag, 9. Oktober 2018 18:00 > > > > To: Vladimir Kozlov ; Baesken, Matthias > > > > ; 'hotspot-dev at openjdk.java.net' > > > > > dev at openjdk.java.net> > > > > Subject: Re: RFR [XS] : hotspot/share/opto/parse2.cpp compile error with > > gcc > > > > 7.3.1 > > > > > > > > On Tue, 2018-10-09 at 08:06 -0700, Vladimir Kozlov wrote: > > > > > Why there is no complain about sprintf at line 2857 which uses %s? : > > > > > > > > > > > > > > > http://hg.openjdk.java.net/jdk/jdk/file/d24b89390f6c/src/hotspot/share/op > > > > to/parse2.cpp#l2857 > > > > > > > > Perhaps because this is non-product code? > > > > > > > > Thanks, > > > > Severin > > > > > > > From igor.ignatyev at oracle.com Wed Oct 10 18:33:22 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 10 Oct 2018 11:33:22 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> Message-ID: <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> Hi David, thanks for spotting the typo, I've fixed the commit message. still looking for a Reviewer though. Cheers, -- Igor > On Oct 9, 2018, at 6:49 PM, David Holmes wrote: > > Hi Igor, > > Not a review - I fixed the typo in the bug synopsis: Covert -> Convert :) > > Please ensure you commit with corrected synopsis. > > Thanks, > David > > On 10/10/2018 3:23 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >>> 450 lines changed: 238 ins; 211 del; 1 mod; >> Hi all, >> could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? >> webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 >> testing: >> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >> - build w/ precompiled-headers enabled and disabled >> PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review >> Thanks, >> -- Igor >> From luke.hutch at gmail.com Wed Oct 10 19:56:01 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Wed, 10 Oct 2018 13:56:01 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <719DC264-C858-43D2-9589-178564F67EC2@gmail.com> Message-ID: On Wed, Oct 10, 2018 at 9:53 AM Kim Barrett wrote: > Finalization was deprecated in JDK 9 (*). There's an ongoing (though > progressing slowly) process of eliminating its usage in the JDK. > Sometime after that it may go away entirely, or perhaps require some > sort of explicit opt-in, making it possible to eliminate various GC > overheads associated with it. > > (*) Use PhantomReference, or a helper like java.lang.ref.Cleaner. > Though that change the liveness question in this thread. > Yes, I saw both the deprecation of finalization and the introduction of Cleaner -- though there will be a lot of legacy code and legacy runtime environments that are going to stick around for a long time, especially given the problems so many people are having moving to JPMS for non-trivial usecases. What would be wrong with simply disabling inlining altogether for any classes that override finalize()? Inlining seems to be the main sticking point. I would strongly prefer to take a performance hit over having finalization being so broken. Running finalizers late or never is much less of a problem than running them early, since running them late or never *may* lead to resource leaks, but running them early *will almost certainly* break everything that uses finalizers at some point. It would be a much better API contract to state that "if you override finalize(), some code optimizations may not kick in" rather than "if you override finalize(), things will almost certainly break, and you get to keep all the pieces". From david.lloyd at redhat.com Wed Oct 10 20:19:27 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 10 Oct 2018 15:19:27 -0500 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <719DC264-C858-43D2-9589-178564F67EC2@gmail.com> Message-ID: On Wed, Oct 10, 2018 at 2:57 PM Luke Hutchison wrote: > On Wed, Oct 10, 2018 at 9:53 AM Kim Barrett wrote: > > Finalization was deprecated in JDK 9 (*). There's an ongoing (though > > progressing slowly) process of eliminating its usage in the JDK. > > Sometime after that it may go away entirely, or perhaps require some > > sort of explicit opt-in, making it possible to eliminate various GC > > overheads associated with it. > [..] > It would be a much better API contract to state that "if you override > finalize(), some code optimizations may not kick in" rather than "if you > override finalize(), things will almost certainly break, and you get to > keep all the pieces". Unfortunately this doesn't really make a difference: finalize() is broken for reasons beyond the "early" cleanup problem, and the "early" cleanup problem actually also applies equally to PhantomReference- and Cleaner-based solutions, as well as other *Reference types. This email thread may have obscured the fact that the two issues are really unconnected. One thing that always (safely) forces references to stay alive (apart from reachabilityFence(), which is the best solution) is calling either a non-static native method on that object, or otherwise passing the reference to a native method as a parameter or part of the graph that is reachable from a parameter. Since finalizers often deal with native resources, sometimes this prevents the problem from appearing in the first place. Another easy-ish solution is synchronization. If your impactful methods execute under a synchronized block, and your finalize() or cleanup operations also operates under the same block, then premature cleanup cannot happen. Obviously you can't use the object itself in the PhantomReference/Cleaner case otherwise the object will never be enqueued. As Andrew said, you could make your own fence using a volatile field write/read. I think it might be enough to write the field at the fence point and read the field in finalize(), though a JMM ace would have to confirm that. -- - DML From luke.hutch at gmail.com Wed Oct 10 02:26:46 2018 From: luke.hutch at gmail.com (Luke Hutchison) Date: Tue, 9 Oct 2018 20:26:46 -0600 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: Message-ID: On Tue, Oct 9, 2018 at 6:35 PM David Holmes wrote: > Also note that finalizers can run while methods of a class instance are > still in progress [1] > > "Optimizing transformations of a program can be designed that reduce the > number of objects that are reachable to be less than those which would > naively be considered reachable." > > - this is one of the evil things about finalizers. So it may not be a > bug, just surprising. > > [1] > https://docs.oracle.com/javase/specs/jls/se11/html/jls-12.html#jls-12.6 > I would assume this is referring to the *visibility* of a symbol -- i.e. if a symbol is in scope, the object referenced by that symbol is considered reachable -- but aggressive escape analysis could schedule an object for garbage collection after the last usage of the symbol within its defining scope. However, it should never be true that an object should be deemed non-reachable when it is serving as the invocation target for a currently-executing non-static method, since "this" is always reachable in the scope of that non-static method. I think this is what Martin was indicating with this comment: On Tue, Oct 9, 2018 at 6:51 PM Martin Buchholz wrote: > Search for calls to Reference.reachabilityFence(this); in the jdk sources. > In the JLS page you linked, you will also see: > A reachable object is any object that can be accessed in any potential continuing computation from any live thread. > A finalizer-reachable object can be reached from some finalizable object through some chain of references, but not from any live thread. Since a live thread is still executing a method of the ScanResult object when the finalizer is run, it cannot be true that this is simply "not a bug, just suprising". This is the whole intent of classes that override the finalize() method being marked "GlobalEscape", so that escape analysis is disabled for these classes, in order to prevent this exact thing from happening. From kim.barrett at oracle.com Thu Oct 11 04:52:34 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Oct 2018 00:52:34 -0400 Subject: RFR: 8212023: Implicit narrowing in Solaris/sparc initializers Message-ID: <9A12D624-7A9D-4EE9-B6E8-980701B81C1D@oracle.com> Please review this small fix to eliminate some implicit narrowings in initializer lists in Solaris/sparc code. Implicit narrowings in initializer lists are not permitted starting with C++11. In os::dll_load, changed the definition of arch_t to have its member types match those of the underlying values used to initialize them, e.g. unsigned char rather than char. In nativeInst_sparc.cpp, in the various int[] offsets arrays in test functions, cast large unsigned values to int. An alternative would be to change the array element type to uint, but that would require additional modification of the using code in some cases, to account for the change of type. Also note there are no callers of these test functions. CR: https://bugs.openjdk.java.net/browse/JDK-8212023 Webrev: http://cr.openjdk.java.net/~kbarrett/8212023/open.00/ Testing: mach5 tier1 for solaris/sparc only. From david.holmes at oracle.com Thu Oct 11 08:19:55 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 Oct 2018 18:19:55 +1000 Subject: RFR: 8212023: Implicit narrowing in Solaris/sparc initializers In-Reply-To: <9A12D624-7A9D-4EE9-B6E8-980701B81C1D@oracle.com> References: <9A12D624-7A9D-4EE9-B6E8-980701B81C1D@oracle.com> Message-ID: Seems fine. Thanks Kim. David On 11/10/2018 2:52 PM, Kim Barrett wrote: > Please review this small fix to eliminate some implicit narrowings in > initializer lists in Solaris/sparc code. Implicit narrowings in > initializer lists are not permitted starting with C++11. > > In os::dll_load, changed the definition of arch_t to have its member > types match those of the underlying values used to initialize them, > e.g. unsigned char rather than char. > > In nativeInst_sparc.cpp, in the various int[] offsets arrays in test > functions, cast large unsigned values to int. An alternative would be > to change the array element type to uint, but that would require > additional modification of the using code in some cases, to account > for the change of type. Also note there are no callers of these test > functions. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8212023 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8212023/open.00/ > > Testing: > mach5 tier1 for solaris/sparc only. > From thomas.schatzl at oracle.com Thu Oct 11 08:22:39 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 Oct 2018 10:22:39 +0200 Subject: RFR: 8212023: Implicit narrowing in Solaris/sparc initializers In-Reply-To: <9A12D624-7A9D-4EE9-B6E8-980701B81C1D@oracle.com> References: <9A12D624-7A9D-4EE9-B6E8-980701B81C1D@oracle.com> Message-ID: <0adc2ee68c827f323575978af8e143d9576bd68d.camel@oracle.com> Hi, On Thu, 2018-10-11 at 00:52 -0400, Kim Barrett wrote: > Please review this small fix to eliminate some implicit narrowings in > initializer lists in Solaris/sparc code. Implicit narrowings in > initializer lists are not permitted starting with C++11. > > In os::dll_load, changed the definition of arch_t to have its member > types match those of the underlying values used to initialize them, > e.g. unsigned char rather than char. > > In nativeInst_sparc.cpp, in the various int[] offsets arrays in test > functions, cast large unsigned values to int. An alternative would be > to change the array element type to uint, but that would require > additional modification of the using code in some cases, to account > for the change of type. Also note there are no callers of these test > functions. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8212023 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8212023/open.00/ > looks good. Thomas From aph at redhat.com Thu Oct 11 09:29:11 2018 From: aph at redhat.com (Andrew Haley) Date: Thu, 11 Oct 2018 10:29:11 +0100 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <719DC264-C858-43D2-9589-178564F67EC2@gmail.com> Message-ID: <63077d5e-56ad-6f9d-5a05-082bbee7c953@redhat.com> On Wed, Oct 10, 2018 at 9:53 AM Kim Barrett wrote: > Finalization was deprecated in JDK 9 (*). There's an ongoing (though > progressing slowly) process of eliminating its usage in the JDK. > Sometime after that it may go away entirely, or perhaps require some > sort of explicit opt-in, making it possible to eliminate various GC > overheads associated with it. > > (*) Use PhantomReference, or a helper like java.lang.ref.Cleaner. > Though that change the liveness question in this thread. It doesn't matter whether finalization is used or PhantomReferences, or whatever. The problem of "early" collection remains the same on every possible mechanism which relies on the GC to trigger cleanup, so reachabilityFence still has to be used to prevent early collectiona from happening. The existence of PhantomReference does show, however, why treating classes with finalizers in a special way is pointless. Hans Boehm's proposal for an annotation such as @Cleaned on the class makes the most sense, IMO. But we can probably live with reachabilityFence, although it's a horrible name. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From matthias.baesken at sap.com Thu Oct 11 14:47:56 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 11 Oct 2018 14:47:56 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: Hello Bob, > Hi Bob, > no problem , we can wait a few days until the final version of Win server > 2019 is out, and check the buildNumber again : > seems Windows server 2019 is GA now : https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows-server-2019-now-generally-available/ > > We don?t usually add logic to the JDK related to unreleased operating system > > previews. so I think we can process with the review . Still have to find out the buildNumber of the GA version, probably it has been incremented again . Best regards, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Dienstag, 2. Oktober 2018 17:03 > To: Bob Vandette > Cc: hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > Moldenhauer, Niclas > Subject: RE: RFR : 8211106: [windows] Update OS detection code to > recognize Windows Server 2019 > > Hi Bob, > no problem , we can wait a few days until the final version of Win server > 2019 is out, and check the buildNumber again : > > https://cloudblogs.microsoft.com/windowsserver/2018/09/24/windows- > server-2019-announcing-general-availability-in-october/ > > says : > "In October, customers will have access to Windows Server 2019 through all > the channels! > We will publish a blog post to mark the availability of Windows Server 2019 > soon." > > > > > > The latest preview release is build 17744. Are you testing on that release? > > > > I was testing on a system with build number 17677 ( Windows Server 2019 > preview , seems older than what you are seeing) > and 17723 . > > > Best regards, Matthias > > > > -----Original Message----- > > From: Bob Vandette > > Sent: Dienstag, 2. Oktober 2018 16:09 > > To: Baesken, Matthias > > Cc: hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > > Moldenhauer, Niclas > > Subject: Re: RFR : 8211106: [windows] Update OS detection code to > > recognize Windows Server 2019 > > > > Matthias, > > > > We don?t usually add logic to the JDK related to unreleased operating > system > > previews. > > > > Has the ReleaseID been updated on the system you are testing on? > > > > This is the documented way of determining the OS version you are running > > on? > > > > The latest preview release is build 17744. Are you testing on that release? > > > > https://techcommunity.microsoft.com/t5/Windows-Server- > > Insiders/Windows-Server-2019-version-info/td-p/234472 > > > > > > Bob. > > > > > > > On Oct 2, 2018, at 9:01 AM, Baesken, Matthias > > wrote: > > > > > > Hello, please review change 8211106 . > > > > > > It updates the Windows OS detection code to recognize Windows > Server > > 2019 . > > > > > > For this we have to look at the build number (dwBuildNumber of > > OSVERSIONINFOEX), > > > > > > https://docs.microsoft.com/en-us/windows/desktop/api/Winnt/ns- > winnt- > > _osversioninfoexa > > > > > > > > > because dwMajorVersion and dwMinorVersion are the same for > > Windows server 2016 and 2019 . > > > > > > The build number used to compare ( 17677 ) is for Windows Server 2019 > > preview , most likely the final version will have a higher build number but > > this is fine for the coding . > > > > > > > > > Bug/webrev : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211106 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ > > > > > > > > > Thanks, Matthias From volker.simonis at gmail.com Thu Oct 11 17:01:02 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 11 Oct 2018 19:01:02 +0200 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* Message-ID: Hi, the change "CODETOOLS-7902290: introduce error state of @requires properties" [1] which was recently pushed to the codetools repo, breaks all JTreg tests which have a require clause that queries a non final (i.e. no "vm.opt.final") VM option. For example the test "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: test result: Error. Error evaluating expression: name not defined: vm.opt.DeoptimizeALot This is because com.sun.javatest.regtest.config.RegressionContext.get() was changed to return null instead of the string "null" for unknown properties. Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" is indeed not defined (as many other "vm.opt" properties, see below). So it turns out that all the tests which @required a "vm.opt" option did run in a faulty configuration until now. I could for example easily fix "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding the corresponding option in VMProps.java: $ hg diff diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 2018 -0400 +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 2018 +0200 @@ -266,6 +266,8 @@ vmOptFinalFlag(map, "ClassUnloading"); vmOptFinalFlag(map, "UseCompressedOops"); vmOptFinalFlag(map, "EnableJVMCI"); + String value = String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); + map.put("vm.opt.DeoptimizeALot", value); } Following is a list of all "vm.opt" options used in @requires clauses without being defined in VMProps.java: vm.opt.AggressiveOpts vm.opt.ClassUnloading vm.opt.ClassUnloadingWithConcurrentMark vm.opt.DeoptimizeALot vm.opt.DisableExplicitGC vm.opt.ExplicitGCInvokesConcurrent vm.opt.FlightRecorder vm.opt.G1HeapRegionSize vm.opt.Inline vm.opt.MaxGCPauseMillis vm.opt.StartFlightRecording vm.opt.SurvivorAlignmentInBytes vm.opt.TieredCompilation vm.opt.TieredStopAtLevel vm.opt.TieredStopAtLevel==4) vm.opt.TieredStopAtLevel==null vm.opt.UseJVMCICompiler What's the current plan? Do we want to add all these options in VMProps.java? Or should we better add all the -XX VM options together to VMProps such that they can all be freely used within @requires clauses? Thank you and best regards, Volker PS: I've noticed that there's "8209807: improve handling exception in requires.VMProps" [2] to apparently adapt VMProps.java to CODETOOLS-7902290 but I've tried the attached webrev and it doesn't help with the described problem. [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 [2] https://bugs.openjdk.java.net/browse/JDK-8209807 From jonathan.gibbons at oracle.com Thu Oct 11 17:13:20 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 11 Oct 2018 10:13:20 -0700 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: Message-ID: Volker, Thanks for the report.? I'll take a look shortly. -- Jon On 10/11/18 10:01 AM, Volker Simonis wrote: > Hi, > > the change "CODETOOLS-7902290: introduce error state of @requires > properties" [1] which was recently pushed to the codetools repo, > breaks all JTreg tests which have a require clause that queries a non > final (i.e. no "vm.opt.final") VM option. > > For example the test > "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which > uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: > > test result: Error. Error evaluating expression: name not defined: > vm.opt.DeoptimizeALot > > This is because > com.sun.javatest.regtest.config.RegressionContext.get() was changed to > return null instead of the string "null" for unknown properties. > Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" > is indeed not defined (as many other "vm.opt" properties, see below). > So it turns out that all the tests which @required a "vm.opt" option > did run in a faulty configuration until now. > > I could for example easily fix > "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding > the corresponding option in VMProps.java: > > $ hg diff > diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java > --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 2018 -0400 > +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 2018 +0200 > @@ -266,6 +266,8 @@ > vmOptFinalFlag(map, "ClassUnloading"); > vmOptFinalFlag(map, "UseCompressedOops"); > vmOptFinalFlag(map, "EnableJVMCI"); > + String value = String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); > + map.put("vm.opt.DeoptimizeALot", value); > } > > Following is a list of all "vm.opt" options used in @requires clauses > without being defined in VMProps.java: > > vm.opt.AggressiveOpts > vm.opt.ClassUnloading > vm.opt.ClassUnloadingWithConcurrentMark > vm.opt.DeoptimizeALot > vm.opt.DisableExplicitGC > vm.opt.ExplicitGCInvokesConcurrent > vm.opt.FlightRecorder > vm.opt.G1HeapRegionSize > vm.opt.Inline > vm.opt.MaxGCPauseMillis > vm.opt.StartFlightRecording > vm.opt.SurvivorAlignmentInBytes > vm.opt.TieredCompilation > vm.opt.TieredStopAtLevel > vm.opt.TieredStopAtLevel==4) > vm.opt.TieredStopAtLevel==null > vm.opt.UseJVMCICompiler > > What's the current plan? Do we want to add all these options in > VMProps.java? Or should we better add all the -XX VM options together > to VMProps such that they can all be freely used within @requires > clauses? > > Thank you and best regards, > Volker > > PS: I've noticed that there's "8209807: improve handling exception in > requires.VMProps" [2] to apparently adapt VMProps.java to > CODETOOLS-7902290 but I've tried the attached webrev and it doesn't > help with the described problem. > > [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 > [2] https://bugs.openjdk.java.net/browse/JDK-8209807 From jonathan.gibbons at oracle.com Thu Oct 11 17:16:08 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 11 Oct 2018 10:16:08 -0700 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: Message-ID: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Volker, It's a typo/oops on my part.? I'll fix it. -- Jon On 10/11/18 10:13 AM, Jonathan Gibbons wrote: > Volker, > > Thanks for the report.? I'll take a look shortly. > > -- Jon > > > On 10/11/18 10:01 AM, Volker Simonis wrote: >> Hi, >> >> the change "CODETOOLS-7902290: introduce error state of @requires >> properties" [1] which was recently pushed to the codetools repo, >> breaks all JTreg tests which have a require clause that queries a non >> final (i.e. no "vm.opt.final") VM option. >> >> For example the test >> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which >> uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: >> >> test result: Error. Error evaluating expression: name not defined: >> vm.opt.DeoptimizeALot >> >> This is because >> com.sun.javatest.regtest.config.RegressionContext.get() was changed to >> return null instead of the string "null" for unknown properties. >> Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" >> is indeed not defined (as many other "vm.opt" properties, see below). >> So it turns out that all the tests which @required a "vm.opt" option >> did run in a faulty configuration until now. >> >> I could for example easily fix >> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding >> the corresponding option in VMProps.java: >> >> $ hg diff >> diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java >> --- a/test/jtreg-ext/requires/VMProps.java????? Thu Oct 11 10:11:18 >> 2018 -0400 >> +++ b/test/jtreg-ext/requires/VMProps.java????? Thu Oct 11 18:51:37 >> 2018 +0200 >> @@ -266,6 +266,8 @@ >> ????????? vmOptFinalFlag(map, "ClassUnloading"); >> ????????? vmOptFinalFlag(map, "UseCompressedOops"); >> ????????? vmOptFinalFlag(map, "EnableJVMCI"); >> +??????? String value = >> String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); >> +??????? map.put("vm.opt.DeoptimizeALot", value); >> ????? } >> >> Following is a list of all "vm.opt" options used in @requires clauses >> without being defined in VMProps.java: >> >> vm.opt.AggressiveOpts >> vm.opt.ClassUnloading >> vm.opt.ClassUnloadingWithConcurrentMark >> vm.opt.DeoptimizeALot >> vm.opt.DisableExplicitGC >> vm.opt.ExplicitGCInvokesConcurrent >> vm.opt.FlightRecorder >> vm.opt.G1HeapRegionSize >> vm.opt.Inline >> vm.opt.MaxGCPauseMillis >> vm.opt.StartFlightRecording >> vm.opt.SurvivorAlignmentInBytes >> vm.opt.TieredCompilation >> vm.opt.TieredStopAtLevel >> vm.opt.TieredStopAtLevel==4) >> vm.opt.TieredStopAtLevel==null >> vm.opt.UseJVMCICompiler >> >> What's the current plan? Do we want to add all these options in >> VMProps.java? Or should we better add all the -XX VM options together >> to VMProps such that they can all be freely used within @requires >> clauses? >> >> Thank you and best regards, >> Volker >> >> PS: I've noticed that there's "8209807: improve handling exception in >> requires.VMProps" [2] to apparently adapt VMProps.java to >> CODETOOLS-7902290 but I've tried the attached webrev and it doesn't >> help with the described problem. >> >> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 >> [2] https://bugs.openjdk.java.net/browse/JDK-8209807 > From kim.barrett at oracle.com Thu Oct 11 17:24:19 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Oct 2018 13:24:19 -0400 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <63077d5e-56ad-6f9d-5a05-082bbee7c953@redhat.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> <719DC264-C858-43D2-9589-178564F67EC2@gmail.com> <63077d5e-56ad-6f9d-5a05-082bbee7c953@redhat.com> Message-ID: > On Oct 11, 2018, at 5:29 AM, Andrew Haley wrote: > > On Wed, Oct 10, 2018 at 9:53 AM Kim Barrett wrote: >> Finalization was deprecated in JDK 9 (*). There's an ongoing (though >> progressing slowly) process of eliminating its usage in the JDK. >> Sometime after that it may go away entirely, or perhaps require some >> sort of explicit opt-in, making it possible to eliminate various GC >> overheads associated with it. >> >> (*) Use PhantomReference, or a helper like java.lang.ref.Cleaner. >> Though that change the liveness question in this thread. > > It doesn't matter whether finalization is used or PhantomReferences, > or whatever. The problem of "early" collection remains the same on > every possible mechanism which relies on the GC to trigger cleanup, so > reachabilityFence still has to be used to prevent early collectiona > from happening. Correct, and sorry for the ?typo?; that was supposed to be "Though that doesn?t change the liveness question in this thread." From kim.barrett at oracle.com Thu Oct 11 19:23:03 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Oct 2018 15:23:03 -0400 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> <10d8f2ea-883e-ec21-4fea-06a52a52fdc6@bell-sw.com> <407CE450-6EED-4955-9B1F-EA564E134D11@oracle.com> Message-ID: > On Oct 8, 2018, at 4:45 PM, Aleksei Voitylov wrote: > > Kim, > > Let's not underestimate the importance of continuous testing throughout the release cycle. Over the last year alternative platforms and configurations were broken accidentally not once (and I think the number is reaching 50 or something). Suggesting a platform to be "not supported for a release or two" may mean by the time the compiler is good the amount of issues to fix for a platform to regain quality may become a blocker for the next LTS. I really hope this is not the option Oracle is proposing. My impression is that most of these were not toolchain issues per se. Rather, they were broken or incomplete changes in platform-dependent code that weren't tested on these "alternative" platforms before being pushed. Oracle has provided dev-submit so that non-Oracle folks can do some minimal testing on all the platforms that Oracle supports. I know that I would sometimes like to have similarly "easy" testing for various platforms not supported by Oracle. I didn't suggest "no testing" if there is a compiler version that supports the new language standards but has not yet been fully product-qualified by the people who are using it. I think gcc on arm may be in that category. But I think it would be very disappointing if the complete lack of a usable version of some compiler were to completely block us in this area. (Unfortunately, such a lack appears to be the situation for XLC compiler used for the AIX/ppc port.) The proposal is not very aggressive. > We both know what and how long it takes to do a thorough OpenJDK compiler upgrade. If the community were made aware of this earlier, I would have definitely started planning for a compiler upgrade for ARM port in JDK 11. I'm understand that it takes time and effort to do a toolchain upgrade. And turning on support for new language version without changing the toolchain version isn't really all that different. This proposal didn't suddenly appear out of nowhere. There has been wistful discussion about using new language features for a long time, with an understanding that going anywhere with that was difficult because of some popular toolchain deficiencies (notably MSVC++) and the need to upgrade others. There have been ongoing efforts in this direction, e.g. various changes to support building with C++11/14 support enabled. Oracle made toolchain upgrades in JDK 11, in part to support the language standard change. (Unfortunately, the relevant toolchain upgrade JEP was labelled "Confidential", even though a lot of the work was in the open, and some of it was explicitly about dealing with differences arising from turning on C++11/14.) I think JDK 12 for this JEP is reasonable goal at this stage. Of course, that goal might not be achieved, for a variety of reasons. That's why it is not yet in the Targeted state and why there is a formal process for moving to that state; there's still work to be done, and we'll have to see how that work proceeds. > With that, I conditionally agree with the proposal provided cooperating ports are given sufficient lead time to upgrade. We started testing ARM with 7.3 and will report on success. From kim.barrett at oracle.com Thu Oct 11 19:55:49 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Oct 2018 15:55:49 -0400 Subject: RFR: 8212023: Implicit narrowing in Solaris/sparc initializers In-Reply-To: References: <9A12D624-7A9D-4EE9-B6E8-980701B81C1D@oracle.com> Message-ID: > On Oct 11, 2018, at 4:19 AM, David Holmes wrote: > > Seems fine. > > Thanks Kim. > > David Thanks. From kim.barrett at oracle.com Thu Oct 11 19:56:04 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Oct 2018 15:56:04 -0400 Subject: RFR: 8212023: Implicit narrowing in Solaris/sparc initializers In-Reply-To: <0adc2ee68c827f323575978af8e143d9576bd68d.camel@oracle.com> References: <9A12D624-7A9D-4EE9-B6E8-980701B81C1D@oracle.com> <0adc2ee68c827f323575978af8e143d9576bd68d.camel@oracle.com> Message-ID: > On Oct 11, 2018, at 4:22 AM, Thomas Schatzl wrote: > > Hi, > > On Thu, 2018-10-11 at 00:52 -0400, Kim Barrett wrote: >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8212023 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8212023/open.00/ >> > > looks good. > > Thomas Thanks. From volker.simonis at gmail.com Thu Oct 11 20:59:49 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 11 Oct 2018 22:59:49 +0200 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> References: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Message-ID: Jonathan Gibbons schrieb am Do. 11. Okt. 2018 um 19:16: > Volker, > > It's a typo/oops on my part. I'll fix it. > Hi John, thanks for looking at this issue! What do you mean with ? typo?? Returning null instead of the string ?null?? That would ?fix? the corresponding tests in the sense that they would run again, however still with bogus results for all the @requires guards which check the ?vm.opt.*? options. Shouldn?t we fix that as well? Regards, Volker > -- Jon > > > On 10/11/18 10:13 AM, Jonathan Gibbons wrote: > > Volker, > > > > Thanks for the report. I'll take a look shortly. > > > > -- Jon > > > > > > On 10/11/18 10:01 AM, Volker Simonis wrote: > >> Hi, > >> > >> the change "CODETOOLS-7902290: introduce error state of @requires > >> properties" [1] which was recently pushed to the codetools repo, > >> breaks all JTreg tests which have a require clause that queries a non > >> final (i.e. no "vm.opt.final") VM option. > >> > >> For example the test > >> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which > >> uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: > >> > >> test result: Error. Error evaluating expression: name not defined: > >> vm.opt.DeoptimizeALot > >> > >> This is because > >> com.sun.javatest.regtest.config.RegressionContext.get() was changed to > >> return null instead of the string "null" for unknown properties. > >> Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" > >> is indeed not defined (as many other "vm.opt" properties, see below). > >> So it turns out that all the tests which @required a "vm.opt" option > >> did run in a faulty configuration until now. > >> > >> I could for example easily fix > >> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding > >> the corresponding option in VMProps.java: > >> > >> $ hg diff > >> diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java > >> --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 > >> 2018 -0400 > >> +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 > >> 2018 +0200 > >> @@ -266,6 +266,8 @@ > >> vmOptFinalFlag(map, "ClassUnloading"); > >> vmOptFinalFlag(map, "UseCompressedOops"); > >> vmOptFinalFlag(map, "EnableJVMCI"); > >> + String value = > >> String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); > >> + map.put("vm.opt.DeoptimizeALot", value); > >> } > >> > >> Following is a list of all "vm.opt" options used in @requires clauses > >> without being defined in VMProps.java: > >> > >> vm.opt.AggressiveOpts > >> vm.opt.ClassUnloading > >> vm.opt.ClassUnloadingWithConcurrentMark > >> vm.opt.DeoptimizeALot > >> vm.opt.DisableExplicitGC > >> vm.opt.ExplicitGCInvokesConcurrent > >> vm.opt.FlightRecorder > >> vm.opt.G1HeapRegionSize > >> vm.opt.Inline > >> vm.opt.MaxGCPauseMillis > >> vm.opt.StartFlightRecording > >> vm.opt.SurvivorAlignmentInBytes > >> vm.opt.TieredCompilation > >> vm.opt.TieredStopAtLevel > >> vm.opt.TieredStopAtLevel==4) > >> vm.opt.TieredStopAtLevel==null > >> vm.opt.UseJVMCICompiler > >> > >> What's the current plan? Do we want to add all these options in > >> VMProps.java? Or should we better add all the -XX VM options together > >> to VMProps such that they can all be freely used within @requires > >> clauses? > >> > >> Thank you and best regards, > >> Volker > >> > >> PS: I've noticed that there's "8209807: improve handling exception in > >> requires.VMProps" [2] to apparently adapt VMProps.java to > >> CODETOOLS-7902290 but I've tried the attached webrev and it doesn't > >> help with the described problem. > >> > >> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 > >> [2] https://bugs.openjdk.java.net/browse/JDK-8209807 > > > > From igor.ignatyev at oracle.com Thu Oct 11 21:09:30 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 11 Oct 2018 14:09:30 -0700 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Message-ID: Hi Volker, vm.opt.* options are set by jtreg (RegressionContext::processVMOptions), and their value is expected to be "null" if a flag hasn't been passed to JDK under tests via -javaoptions or -vmoptions. I don't think there is something to fix (at least not in jtreg). -- Igor > On Oct 11, 2018, at 1:59 PM, Volker Simonis wrote: > > Jonathan Gibbons schrieb am Do. 11. Okt. 2018 > um 19:16: > >> Volker, >> >> It's a typo/oops on my part. I'll fix it. >> > > Hi John, > > thanks for looking at this issue! > > What do you mean with ? typo?? Returning null instead of the string ?null?? > > That would ?fix? the corresponding tests in the sense that they would run > again, however still with bogus results for all the @requires guards which > check the ?vm.opt.*? options. Shouldn?t we fix that as well? > > Regards, > Volker > > > >> -- Jon >> >> >> On 10/11/18 10:13 AM, Jonathan Gibbons wrote: >>> Volker, >>> >>> Thanks for the report. I'll take a look shortly. >>> >>> -- Jon >>> >>> >>> On 10/11/18 10:01 AM, Volker Simonis wrote: >>>> Hi, >>>> >>>> the change "CODETOOLS-7902290: introduce error state of @requires >>>> properties" [1] which was recently pushed to the codetools repo, >>>> breaks all JTreg tests which have a require clause that queries a non >>>> final (i.e. no "vm.opt.final") VM option. >>>> >>>> For example the test >>>> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which >>>> uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: >>>> >>>> test result: Error. Error evaluating expression: name not defined: >>>> vm.opt.DeoptimizeALot >>>> >>>> This is because >>>> com.sun.javatest.regtest.config.RegressionContext.get() was changed to >>>> return null instead of the string "null" for unknown properties. >>>> Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" >>>> is indeed not defined (as many other "vm.opt" properties, see below). >>>> So it turns out that all the tests which @required a "vm.opt" option >>>> did run in a faulty configuration until now. >>>> >>>> I could for example easily fix >>>> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding >>>> the corresponding option in VMProps.java: >>>> >>>> $ hg diff >>>> diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java >>>> --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 >>>> 2018 -0400 >>>> +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 >>>> 2018 +0200 >>>> @@ -266,6 +266,8 @@ >>>> vmOptFinalFlag(map, "ClassUnloading"); >>>> vmOptFinalFlag(map, "UseCompressedOops"); >>>> vmOptFinalFlag(map, "EnableJVMCI"); >>>> + String value = >>>> String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); >>>> + map.put("vm.opt.DeoptimizeALot", value); >>>> } >>>> >>>> Following is a list of all "vm.opt" options used in @requires clauses >>>> without being defined in VMProps.java: >>>> >>>> vm.opt.AggressiveOpts >>>> vm.opt.ClassUnloading >>>> vm.opt.ClassUnloadingWithConcurrentMark >>>> vm.opt.DeoptimizeALot >>>> vm.opt.DisableExplicitGC >>>> vm.opt.ExplicitGCInvokesConcurrent >>>> vm.opt.FlightRecorder >>>> vm.opt.G1HeapRegionSize >>>> vm.opt.Inline >>>> vm.opt.MaxGCPauseMillis >>>> vm.opt.StartFlightRecording >>>> vm.opt.SurvivorAlignmentInBytes >>>> vm.opt.TieredCompilation >>>> vm.opt.TieredStopAtLevel >>>> vm.opt.TieredStopAtLevel==4) >>>> vm.opt.TieredStopAtLevel==null >>>> vm.opt.UseJVMCICompiler >>>> >>>> What's the current plan? Do we want to add all these options in >>>> VMProps.java? Or should we better add all the -XX VM options together >>>> to VMProps such that they can all be freely used within @requires >>>> clauses? >>>> >>>> Thank you and best regards, >>>> Volker >>>> >>>> PS: I've noticed that there's "8209807: improve handling exception in >>>> requires.VMProps" [2] to apparently adapt VMProps.java to >>>> CODETOOLS-7902290 but I've tried the attached webrev and it doesn't >>>> help with the described problem. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8209807 >>> >> >> From erik.joelsson at oracle.com Thu Oct 11 22:29:38 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 11 Oct 2018 15:29:38 -0700 Subject: RFR: JDK-8212028: Use run-test makefile framework for testing in Oracle's Mach5 Message-ID: <457fabb0-5f30-b4dc-4e98-0853f209f3c3@oracle.com> Hello, (adding serviceability-dev and hotspot-dev for test changes) Bug: https://bugs.openjdk.java.net/browse/JDK-8212028 Webrev: http://cr.openjdk.java.net/~erikj/8212028/webrev.01/index.html (From ihse-runtestprebuilt-branch in jdk-sandbox) In order to fully adopt the new run-test framework, we need to switch over the automated and distributed testing system at Oracle to the new framework. To get this to work, there are number of issues that needed to be fixed. Here follows a brief explanation, see bug for more details. For RunTest.gmk and related makefiles there are a number of minor tweaks to support all the necessary control variables that are currently used for the old test makefiles, as well as correcting some test setup settings. In addition to that, some tests also needed to be modified: Timeouts The current default timeoutFactor in the makefiles is 4. However, the old Mach5 executor overrides that to 10. I don't think it should dabble with such things and leave it to the makefiles, the user, or a specific job definition, so with the new run-test executor, it no longer does. This means many tests now have a much shorter effective timeout. Because of this, we need to increase the timeout on some that are now prone to timing out. I have run tier1-5 a few times to try and find these and added /timeout=300 (which will result in the same effective timeout as before) when specific tests seemed problematic. test/hotspot/jtreg/runtime/appcds/jvmti/InstrumentationTest.java This test spawns a child process and tries to locate it using the attach api, by looking for a unique token in the command line string of the spawned JVM. The problem is that the command line string it gets from the attach api is truncated and the token is last on the command line. This normally works well, but the arguments before it are 3 files, with full absolute paths inside the jtreg work directory. With Mach5 we have pretty deep work directories, and with run-test, we make them even deeper. This unfortunately trips the limit and the test fails. I have fixed this by reordering the arguments to the child process. /Erik From david.holmes at oracle.com Thu Oct 11 22:40:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 12 Oct 2018 08:40:34 +1000 Subject: RFR: JDK-8212028: Use run-test makefile framework for testing in Oracle's Mach5 In-Reply-To: <457fabb0-5f30-b4dc-4e98-0853f209f3c3@oracle.com> References: <457fabb0-5f30-b4dc-4e98-0853f209f3c3@oracle.com> Message-ID: Hi Erik, On 12/10/2018 8:29 AM, Erik Joelsson wrote: > Hello, > > (adding serviceability-dev and hotspot-dev for test changes) > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212028 > > Webrev: http://cr.openjdk.java.net/~erikj/8212028/webrev.01/index.html > (From ihse-runtestprebuilt-branch in jdk-sandbox) > > In order to fully adopt the new run-test framework, we need to switch > over the automated and distributed testing system at Oracle to the new > framework. To get this to work, there are number of issues that needed > to be fixed. Here follows a brief explanation, see bug for more details. > > For RunTest.gmk and related makefiles there are a number of minor tweaks > to support all the necessary control variables that are currently used > for the old test makefiles, as well as correcting some test setup settings. > > In addition to that, some tests also needed to be modified: > > Timeouts > The current default timeoutFactor in the makefiles is 4. However, the > old Mach5 executor overrides that to 10. I don't think it should dabble > with such things and leave it to the makefiles, the user, or a specific > job definition, so with the new run-test executor, it no longer does. > This means many tests now have a much shorter effective timeout. Because > of this, we need to increase the timeout on some that are now prone to > timing out. I have run tier1-5 a few times to try and find these and > added /timeout=300 (which will result in the same effective timeout as > before) when specific tests seemed problematic. This should be fixed in the tier job definitions not the individual tests. We have moved away from putting explicit timeouts on individual tests and instead rely on the framework timeout being set appropriately. David ----- > test/hotspot/jtreg/runtime/appcds/jvmti/InstrumentationTest.java > This test spawns a child process and tries to locate it using the attach > api, by looking for a unique token in the command line string of the > spawned JVM. The problem is that the command line string it gets from > the attach api is truncated and the token is last on the command line. > This normally works well, but the arguments before it are 3 files, with > full absolute paths inside the jtreg work directory. With Mach5 we have > pretty deep work directories, and with run-test, we make them even > deeper. This unfortunately trips the limit and the test fails. I have > fixed this by reordering the arguments to the child process. > > /Erik > From matthias.baesken at sap.com Fri Oct 12 06:57:22 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 12 Oct 2018 06:57:22 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: I got the info that the GA of Windows Server 2019 has build number 17763 . Should I update the webrev to check this build number (would mean we do not detect the preview as Windows Server 2019) , or keep it the way it is ? Current webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ Best regards, Matthias > > Hello Bob, > > > Hi Bob, > > no problem , we can wait a few days until the final version of Win > server > > 2019 is out, and check the buildNumber again : > > > > seems Windows server 2019 is GA now : > > > https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows- > server-2019-now-generally-available/ > > > > We don?t usually add logic to the JDK related to unreleased operating > system > > > previews. > > > so I think we can process with the review . > Still have to find out the buildNumber of the GA version, probably it has been > incremented again . > > > > > > > > > Bug/webrev : > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211106 > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ > > > > > > > > > > > > Thanks, Matthias From magnus.ihse.bursie at oracle.com Fri Oct 12 08:37:07 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 12 Oct 2018 10:37:07 +0200 Subject: RFR: JDK-8212028: Use run-test makefile framework for testing in Oracle's Mach5 In-Reply-To: <457fabb0-5f30-b4dc-4e98-0853f209f3c3@oracle.com> References: <457fabb0-5f30-b4dc-4e98-0853f209f3c3@oracle.com> Message-ID: <965e23c7-531d-2f9e-3c66-1d8c5a7bcdd0@oracle.com> Hi Erik, Thank you for preserving through this, so we finally can move to 100% run-test! On 2018-10-12 00:29, Erik Joelsson wrote: > Hello, > > (adding serviceability-dev and hotspot-dev for test changes) > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212028 > > Webrev: http://cr.openjdk.java.net/~erikj/8212028/webrev.01/index.html > (From ihse-runtestprebuilt-branch in jdk-sandbox) Build changes look good. And I agree with the solution to add longer timeout to individual tests. > > In order to fully adopt the new run-test framework, we need to switch > over the automated and distributed testing system at Oracle to the new > framework. To get this to work, there are number of issues that needed > to be fixed. Here follows a brief explanation, see bug for more details. > > For RunTest.gmk and related makefiles there are a number of minor > tweaks to support all the necessary control variables that are > currently used for the old test makefiles, as well as correcting some > test setup settings. > > In addition to that, some tests also needed to be modified: > > Timeouts > The current default timeoutFactor in the makefiles is 4. However, the > old Mach5 executor overrides that to 10. I don't think it should > dabble with such things and leave it to the makefiles, the user, or a > specific job definition, so with the new run-test executor, it no > longer does. This means many tests now have a much shorter effective > timeout. Because of this, we need to increase the timeout on some that > are now prone to timing out. I have run tier1-5 a few times to try and > find these and added /timeout=300 (which will result in the same > effective timeout as before) when specific tests seemed problematic. > > test/hotspot/jtreg/runtime/appcds/jvmti/InstrumentationTest.java > This test spawns a child process and tries to locate it using the > attach api, by looking for a unique token in the command line string > of the spawned JVM. The problem is that the command line string it > gets from the attach api is truncated and the token is last on the > command line. This normally works well, but the arguments before it > are 3 files, with full absolute paths inside the jtreg work directory. > With Mach5 we have pretty deep work directories, and with run-test, we > make them even deeper. This unfortunately trips the limit and the test > fails. I have fixed this by reordering the arguments to the child > process. ... but I'm not sure I understand this. Is it a command-line length we're hitting? If so, how can reordering the arguments solve anything? I understand why this can preserve the token, but will not one of the paths be cut of instead? /Magnus > > /Erik > From rkennke at redhat.com Fri Oct 12 08:57:25 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 12 Oct 2018 10:57:25 +0200 Subject: RFC: How to deal with/abstract some offset-related Shenandoah changes? Message-ID: <27467623-88df-c77d-20a3-2e2e26a3ecee@redhat.com> Hello, I'd like to get some opinions on how should we deal with some changes that we need in Shenandoah, that really look a bit ugly, but that seem a little out of place in a GC interface too. Let me show you the code: https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html There are 3 blocks there: - The first deals with the aarch64 addressing bits. Only the lower 48bits are used for addressing, and thus the upper 16 are masked away. - the second block adjusts the offset by 8 bytes for Shenandoah when computing the normalized heap-based-offset for narrow-oops. This is needed to get combined decode+read-barrier instruction to work iirc. - the last block allows implicit null-checks on Shenandoah fwd ptr loads (now including the masking from block#1 above). I really am not sure how do deal with this. It does look like it would require a two-way abstraction: 1. CPU specific. i.e. move this method under src/cpu/$CPU and do the right thing there (possibly duplicating for platforms that do common stuff) 2. GC specific to deal with the -8 offset of Shenandoah However, I doubt that it would make the code any more easier to understand/read/etc. And how would the GC abstraction look like? bool BarrierSetAssembler::needs_explicit_null_check(..) ? And if GC says 'false' then don't check any of the other stuff? I'm a bit at a loss here. Any suggestions are appreciated. There is another place that deals with special offsets: https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/ci/ciInstanceKlass.cpp.udiff.html (look at get_canonical_holder() there) Please let me know what you think. Roman From volker.simonis at gmail.com Fri Oct 12 15:15:21 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 12 Oct 2018 17:15:21 +0200 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Message-ID: On Thu, Oct 11, 2018 at 11:09 PM Igor Ignatyev wrote: > > Hi Volker, > > vm.opt.* options are set by jtreg (RegressionContext::processVMOptions), and their value is expected to be "null" if a flag hasn't been passed to JDK under tests via -javaoptions or -vmoptions. I don't think there is something to fix (at least not in jtreg). > Well, if "vm.opt.*" options are indeed only options which are taken from the command line (i.e. "-javaoption/-vmoption"), than there's definitely something we have to fix! Because, as I described before, after CODETOOLS-7902290, all test which check such an option in a @requires clause (negatively or positively) will now fail if that option is not given explicitly on the command line. If "vm.opt.*" options are really only the options given on the command line (please confirm this and please document it somewhere prominently!) I think many tests which use them are broken because they simply check such options like this: @requires vm.opt.TieredCompilation == true This would be wrong and would have to be replaced by: @requires vm.opt.TieredCompilation == null | vm.opt.TieredCompilation == true to also account for the case where -XX:TieredCompilation was not explicitly set on the command line. Notice that in in such a case, the value of "TieredCompilation" in the VM could be either "true" or "false" (depending on the platform and VM build configuration), but it wouldn't be possible to test that by using "vm.opt.TieredCompilation". That said, there are three "vm.opt.final.*" options which are set by VMProps: vmOptFinalFlag(map, "ClassUnloading"); vmOptFinalFlag(map, "UseCompressedOops"); vmOptFinalFlag(map, "EnableJVMCI"); Notice that these options are set to the values of the corresponding flags in the agent VM . For tests which run in "othervm" mode (or which start new VMs with ProcessBuilder or jdk.test.lib.process.ProcessTools), that doesn't necessarily corresponds to the values of these flags in the testee VM. Finally, options specified with "-javaoption/-vmoption" and reflected in the corresponding "vm.opt.*" flags, may be over-ridden by command line options specified for tests which run in "othervm" mode (so checking them with a @requires clause is of limited usefulness as well). All this is quite complex and I think we should *really* document it in a prominent place like the "jtreg: Command Line Options" page [1] AND the "The JDK Test Framework: Tag Language Specification" page [2]. Regards, Volker [1] http://openjdk.java.net/jtreg/command-help.html [2] http://openjdk.java.net/jtreg/tag-spec.html > -- Igor > > > > On Oct 11, 2018, at 1:59 PM, Volker Simonis wrote: > > > > Jonathan Gibbons schrieb am Do. 11. Okt. 2018 > > um 19:16: > > > >> Volker, > >> > >> It's a typo/oops on my part. I'll fix it. > >> > > > > Hi John, > > > > thanks for looking at this issue! > > > > What do you mean with ? typo?? Returning null instead of the string ?null?? > > > > That would ?fix? the corresponding tests in the sense that they would run > > again, however still with bogus results for all the @requires guards which > > check the ?vm.opt.*? options. Shouldn?t we fix that as well? > > > > Regards, > > Volker > > > > > > > >> -- Jon > >> > >> > >> On 10/11/18 10:13 AM, Jonathan Gibbons wrote: > >>> Volker, > >>> > >>> Thanks for the report. I'll take a look shortly. > >>> > >>> -- Jon > >>> > >>> > >>> On 10/11/18 10:01 AM, Volker Simonis wrote: > >>>> Hi, > >>>> > >>>> the change "CODETOOLS-7902290: introduce error state of @requires > >>>> properties" [1] which was recently pushed to the codetools repo, > >>>> breaks all JTreg tests which have a require clause that queries a non > >>>> final (i.e. no "vm.opt.final") VM option. > >>>> > >>>> For example the test > >>>> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which > >>>> uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: > >>>> > >>>> test result: Error. Error evaluating expression: name not defined: > >>>> vm.opt.DeoptimizeALot > >>>> > >>>> This is because > >>>> com.sun.javatest.regtest.config.RegressionContext.get() was changed to > >>>> return null instead of the string "null" for unknown properties. > >>>> Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" > >>>> is indeed not defined (as many other "vm.opt" properties, see below). > >>>> So it turns out that all the tests which @required a "vm.opt" option > >>>> did run in a faulty configuration until now. > >>>> > >>>> I could for example easily fix > >>>> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding > >>>> the corresponding option in VMProps.java: > >>>> > >>>> $ hg diff > >>>> diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java > >>>> --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 > >>>> 2018 -0400 > >>>> +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 > >>>> 2018 +0200 > >>>> @@ -266,6 +266,8 @@ > >>>> vmOptFinalFlag(map, "ClassUnloading"); > >>>> vmOptFinalFlag(map, "UseCompressedOops"); > >>>> vmOptFinalFlag(map, "EnableJVMCI"); > >>>> + String value = > >>>> String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); > >>>> + map.put("vm.opt.DeoptimizeALot", value); > >>>> } > >>>> > >>>> Following is a list of all "vm.opt" options used in @requires clauses > >>>> without being defined in VMProps.java: > >>>> > >>>> vm.opt.AggressiveOpts > >>>> vm.opt.ClassUnloading > >>>> vm.opt.ClassUnloadingWithConcurrentMark > >>>> vm.opt.DeoptimizeALot > >>>> vm.opt.DisableExplicitGC > >>>> vm.opt.ExplicitGCInvokesConcurrent > >>>> vm.opt.FlightRecorder > >>>> vm.opt.G1HeapRegionSize > >>>> vm.opt.Inline > >>>> vm.opt.MaxGCPauseMillis > >>>> vm.opt.StartFlightRecording > >>>> vm.opt.SurvivorAlignmentInBytes > >>>> vm.opt.TieredCompilation > >>>> vm.opt.TieredStopAtLevel > >>>> vm.opt.TieredStopAtLevel==4) > >>>> vm.opt.TieredStopAtLevel==null > >>>> vm.opt.UseJVMCICompiler > >>>> > >>>> What's the current plan? Do we want to add all these options in > >>>> VMProps.java? Or should we better add all the -XX VM options together > >>>> to VMProps such that they can all be freely used within @requires > >>>> clauses? > >>>> > >>>> Thank you and best regards, > >>>> Volker > >>>> > >>>> PS: I've noticed that there's "8209807: improve handling exception in > >>>> requires.VMProps" [2] to apparently adapt VMProps.java to > >>>> CODETOOLS-7902290 but I've tried the attached webrev and it doesn't > >>>> help with the described problem. > >>>> > >>>> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 > >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8209807 > >>> > >> > >> > From christoph.langer at sap.com Fri Oct 12 15:23:33 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 12 Oct 2018 15:23:33 +0000 Subject: RFR : 8211089: windows : print process heaps information in hs_err file In-Reply-To: References: Message-ID: <60a12e13950c434abfe89da7b3bad756@sap.com> Hi Matthias, I think this information is category "nice to have". It should probably guarded by the new option that is proposed by Thomas' CSR draft: https://bugs.openjdk.java.net/browse/JDK-8211846. The implementation looks ok to me. Best regards Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Baesken, Matthias > Sent: Freitag, 28. September 2018 09:41 > To: 'hotspot-dev at openjdk.java.net' > Subject: RE: RFR : 8211089: windows : print process heaps information in > hs_err file > > Ping .... > > Would be great to have some reviews on this ! > > Thanks, Matthias > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Dienstag, 25. September 2018 15:53 > > To: 'hotspot-dev at openjdk.java.net' > > Subject: RFR : 8211089: windows : print process heaps information in hs_err > > file > > > > Hello , please review this small change. > > It adds output of information regarding Windows process heaps (see > > https://docs.microsoft.com/en-us/windows/desktop/api/heapapi/nf- > > heapapi-getprocessheaps ) > > to the hs_error file. > > we had this for some time in our internal JVM and want to bring it to > > OpenJDK as well. > > It is a Windows-only extension. > > > > bug : 8211089: windows : print process heaps information in hs_err file > > https://bugs.openjdk.java.net/browse/JDK-8211089 > > > > > > webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211089.0/ > > > > > > > > extended output is usually small, and looks like this : > > > > > > --------------- S Y S T E M --------------- > > > > OS: Windows 10 , 64 bit Build ...... > > > > .... > > > > 4 Process Heaps: > > Heap ID: 1 0x00000186491a0000 default crt low-fragmentation > > Heap ID: 2 0x0000018649000000 standard > > Heap ID: 3 0x0000018649430000 low-fragmentation > > Heap ID: 4 0x000001864adb0000 low-fragmentation > > > > > > Thanks, Matthias From jonathan.gibbons at oracle.com Fri Oct 12 15:42:13 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 12 Oct 2018 08:42:13 -0700 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Message-ID: <7ea8c652-bcbd-154a-a02c-0a55cc4c6478@oracle.com> On 10/12/18 8:15 AM, Volker Simonis wrote: > On Thu, Oct 11, 2018 at 11:09 PM Igor Ignatyev wrote: >> Hi Volker, >> >> vm.opt.* options are set by jtreg (RegressionContext::processVMOptions), and their value is expected to be "null" if a flag hasn't been passed to JDK under tests via -javaoptions or -vmoptions. I don't think there is something to fix (at least not in jtreg). >> > Well, if "vm.opt.*" options are indeed only options which are taken > from the command line (i.e. "-javaoption/-vmoption"), than there's > definitely something we have to fix! Because, as I described before, > after CODETOOLS-7902290, all test which check such an option in a > @requires clause (negatively or positively) will now fail if that > option is not given explicitly on the command line. > > If "vm.opt.*" options are really only the options given on the command > line (please confirm this and please document it somewhere > prominently!) I think many tests which use them are broken because > they simply check such options like this: > > @requires vm.opt.TieredCompilation == true > > This would be wrong and would have to be replaced by: > > @requires vm.opt.TieredCompilation == null | vm.opt.TieredCompilation == true > > to also account for the case where -XX:TieredCompilation was not > explicitly set on the command line. Notice that in in such a case, the > value of "TieredCompilation" in the VM could be either "true" or > "false" (depending on the platform and VM build configuration), but it > wouldn't be possible to test that by using "vm.opt.TieredCompilation". > > That said, there are three "vm.opt.final.*" options which are set by VMProps: > > vmOptFinalFlag(map, "ClassUnloading"); > vmOptFinalFlag(map, "UseCompressedOops"); > vmOptFinalFlag(map, "EnableJVMCI"); > > Notice that these options are set to the values of the corresponding > flags in the agent VM . For tests which run in "othervm" mode (or > which start new VMs with ProcessBuilder or > jdk.test.lib.process.ProcessTools), that doesn't necessarily > corresponds to the values of these flags in the testee VM. > > Finally, options specified with "-javaoption/-vmoption" and reflected > in the corresponding "vm.opt.*" flags, may be over-ridden by command > line options specified for tests which run in "othervm" mode (so > checking them with a @requires clause is of limited usefulness as > well). > > All this is quite complex and I think we should*really* document it > in a prominent place like the "jtreg: Command Line Options" page [1] > AND the > "The JDK Test Framework: Tag Language Specification" page [2]. > > Regards, > Volker > > [1]http://openjdk.java.net/jtreg/command-help.html > [2]http://openjdk.java.net/jtreg/tag-spec.html > Volker, I will look? at what can be done to improve the documentation, especially with regard to documenting the limitations of the @requires mechanism. That being said, most of the mechanism is provided outside of jtreg, by the extraPropDefns? extension mechanism specified in TEST.ROOT.? It was a deliberate design choice to decouple these classes from jtreg, so that they can evolve faster and separately from jtreg promotions, according to the needs of the test suite. So, without diminishing the need for additional documentation, I'll note that detailed documentation may not belong in the jtreg pages you specified. Finally, you are right to observe the inadequacies of the vm.opt.* mechanism. IMO, it is better to use custom properties defined by the extraPropDefns mechanism that provide the result of analyzing the options, as compared to checking the options themselves. My prime example of this, when I was writing the jtreg support, was the "-server" and "-client" options, which are so-called supported options on all system, but may be no-ops on some platforms. Therefore, it is more important to check the internal settings in the VM than to check the options given to the VM. -- Jon From erik.joelsson at oracle.com Fri Oct 12 16:16:43 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 12 Oct 2018 09:16:43 -0700 Subject: RFR: JDK-8212028: Use run-test makefile framework for testing in Oracle's Mach5 In-Reply-To: <965e23c7-531d-2f9e-3c66-1d8c5a7bcdd0@oracle.com> References: <457fabb0-5f30-b4dc-4e98-0853f209f3c3@oracle.com> <965e23c7-531d-2f9e-3c66-1d8c5a7bcdd0@oracle.com> Message-ID: <02afe047-2c6d-e902-68b4-595ca85cbad4@oracle.com> On 2018-10-12 01:37, Magnus Ihse Bursie wrote: > Hi Erik, > > Thank you for preserving through this, so we finally can move to 100% > run-test! > > On 2018-10-12 00:29, Erik Joelsson wrote: > >> Hello, >> >> (adding serviceability-dev and hotspot-dev for test changes) >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8212028 >> >> Webrev: >> http://cr.openjdk.java.net/~erikj/8212028/webrev.01/index.html (From >> ihse-runtestprebuilt-branch in jdk-sandbox) > Build changes look good. And I agree with the solution to add longer > timeout to individual tests. Thanks! >> >> test/hotspot/jtreg/runtime/appcds/jvmti/InstrumentationTest.java >> This test spawns a child process and tries to locate it using the >> attach api, by looking for a unique token in the command line string >> of the spawned JVM. The problem is that the command line string it >> gets from the attach api is truncated and the token is last on the >> command line. This normally works well, but the arguments before it >> are 3 files, with full absolute paths inside the jtreg work >> directory. With Mach5 we have pretty deep work directories, and with >> run-test, we make them even deeper. This unfortunately trips the >> limit and the test fails. I have fixed this by reordering the >> arguments to the child process. > ... but I'm not sure I understand this. Is it a command-line length > we're hitting? If so, how can reordering the arguments solve anything? > I understand why this can preserve the token, but will not one of the > paths be cut of instead? > I will try to be clearer. The command line is fine for running the child process. The truncation happens when the attach api is used to try to find the child process. It basically calls something that lists all JVMs on system, iterates through them and looks at the "description" field. This field happens to contain the first X characters of the command line so the test looks for the unique token it knows it put there. (I don't know the exact value of X, but could find out). Until now, X was big enough to fit the whole command line, but with the 3 absolute path files in there now growing long enough, the last argument is pushed out of the description field. This means the test process is unable to locate the child process in the list of JVMs. What then happens is the test keeps on looking, taking new snapshots of all JVMs on the system until the test finally times out. By reordering the arguments, the token is less likely to be cut out of the description field. /Erik From magnus.ihse.bursie at oracle.com Fri Oct 12 17:07:08 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 12 Oct 2018 19:07:08 +0200 Subject: RFR: JDK-8212028: Use run-test makefile framework for testing in Oracle's Mach5 In-Reply-To: <02afe047-2c6d-e902-68b4-595ca85cbad4@oracle.com> References: <457fabb0-5f30-b4dc-4e98-0853f209f3c3@oracle.com> <965e23c7-531d-2f9e-3c66-1d8c5a7bcdd0@oracle.com> <02afe047-2c6d-e902-68b4-595ca85cbad4@oracle.com> Message-ID: <84B6AC75-0A3E-489B-B7AF-D60C285AA408@oracle.com> > 12 okt. 2018 kl. 18:16 skrev Erik Joelsson : > > >> On 2018-10-12 01:37, Magnus Ihse Bursie wrote: >> Hi Erik, >> >> Thank you for preserving through this, so we finally can move to 100% run-test! >> >>> On 2018-10-12 00:29, Erik Joelsson wrote: >>> >>> Hello, >>> >>> (adding serviceability-dev and hotspot-dev for test changes) >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8212028 >>> >>> Webrev: http://cr.openjdk.java.net/~erikj/8212028/webrev.01/index.html (From ihse-runtestprebuilt-branch in jdk-sandbox) >> Build changes look good. And I agree with the solution to add longer timeout to individual tests. > Thanks! >>> >>> test/hotspot/jtreg/runtime/appcds/jvmti/InstrumentationTest.java >>> This test spawns a child process and tries to locate it using the attach api, by looking for a unique token in the command line string of the spawned JVM. The problem is that the command line string it gets from the attach api is truncated and the token is last on the command line. This normally works well, but the arguments before it are 3 files, with full absolute paths inside the jtreg work directory. With Mach5 we have pretty deep work directories, and with run-test, we make them even deeper. This unfortunately trips the limit and the test fails. I have fixed this by reordering the arguments to the child process. >> ... but I'm not sure I understand this. Is it a command-line length we're hitting? If so, how can reordering the arguments solve anything? I understand why this can preserve the token, but will not one of the paths be cut of instead? > I will try to be clearer. The command line is fine for running the child process. The truncation happens when the attach api is used to try to find the child process. It basically calls something that lists all JVMs on system, iterates through them and looks at the "description" field. This field happens to contain the first X characters of the command line so the test looks for the unique token it knows it put there. (I don't know the exact value of X, but could find out). Until now, X was big enough to fit the whole command line, but with the 3 absolute path files in there now growing long enough, the last argument is pushed out of the description field. This means the test process is unable to locate the child process in the list of JVMs. What then happens is the test keeps on looking, taking new snapshots of all JVMs on the system until the test finally times out. By reordering the arguments, the token is less likely to be cut out of the description field. > I see. Then your fix makes totally sense. /Magnus > /Erik > From matthias.baesken at sap.com Mon Oct 15 06:54:43 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 15 Oct 2018 06:54:43 +0000 Subject: RFR : 8211089: windows : print process heaps information in hs_err file In-Reply-To: <60a12e13950c434abfe89da7b3bad756@sap.com> References: <60a12e13950c434abfe89da7b3bad756@sap.com> Message-ID: Hi Christoph, thanks for the review . > It should probably guarded by the new option that is proposed by Thomas' CSR draft: > https://bugs.openjdk.java.net/browse/JDK-8211846. Would be fine with me , once the CSR is approved, I can post another webrev , where the flag - guarding is used . Best regards, Matthias > -----Original Message----- > From: Langer, Christoph > Sent: Freitag, 12. Oktober 2018 17:24 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Cc: Stuefe, Thomas > Subject: RE: RFR : 8211089: windows : print process heaps information in > hs_err file > > Hi Matthias, > > I think this information is category "nice to have". It should probably guarded > by the new option that is proposed by Thomas' CSR draft: > https://bugs.openjdk.java.net/browse/JDK-8211846. > > The implementation looks ok to me. > > Best regards > Christoph > > > -----Original Message----- > > From: hotspot-dev On Behalf > Of > > Baesken, Matthias > > Sent: Freitag, 28. September 2018 09:41 > > To: 'hotspot-dev at openjdk.java.net' > > Subject: RE: RFR : 8211089: windows : print process heaps information in > > hs_err file > > > > Ping .... > > > > Would be great to have some reviews on this ! > > > > Thanks, Matthias > > > > > -----Original Message----- > > > From: Baesken, Matthias > > > Sent: Dienstag, 25. September 2018 15:53 > > > To: 'hotspot-dev at openjdk.java.net' > > > Subject: RFR : 8211089: windows : print process heaps information in > hs_err > > > file > > > > > > Hello , please review this small change. > > > It adds output of information regarding Windows process heaps (see > > > https://docs.microsoft.com/en-us/windows/desktop/api/heapapi/nf- > > > heapapi-getprocessheaps ) > > > to the hs_error file. > > > we had this for some time in our internal JVM and want to bring it to > > > OpenJDK as well. > > > It is a Windows-only extension. > > > > > > bug : 8211089: windows : print process heaps information in hs_err file > > > https://bugs.openjdk.java.net/browse/JDK-8211089 > > > > > > > > > webrev : > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211089.0/ > > > > > > > > > > > > extended output is usually small, and looks like this : > > > > > > > > > --------------- S Y S T E M --------------- > > > > > > OS: Windows 10 , 64 bit Build ...... > > > > > > .... > > > > > > 4 Process Heaps: > > > Heap ID: 1 0x00000186491a0000 default crt low-fragmentation > > > Heap ID: 2 0x0000018649000000 standard > > > Heap ID: 3 0x0000018649430000 low-fragmentation > > > Heap ID: 4 0x000001864adb0000 low-fragmentation > > > > > > > > > Thanks, Matthias From aleksei.voitylov at bell-sw.com Mon Oct 15 11:51:24 2018 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Mon, 15 Oct 2018 14:51:24 +0300 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> <10d8f2ea-883e-ec21-4fea-06a52a52fdc6@bell-sw.com> <407CE450-6EED-4955-9B1F-EA564E134D11@oracle.com> Message-ID: Kim, If you were suggesting to just proceed with the change without giving a sufficient lead time for the ports that were willing to upgrade to do so, that sounds very alarming. Meanwhile, our testing has finished and I'm now confident we will be able to switch ARM port over to 7.x in 12 time frame. There are several issues that we still need to diagnose, but this does not look like a blocker. -Aleksei On 11/10/2018 22:23, Kim Barrett wrote: >> On Oct 8, 2018, at 4:45 PM, Aleksei Voitylov wrote: >> >> Kim, >> >> Let's not underestimate the importance of continuous testing throughout the release cycle. Over the last year alternative platforms and configurations were broken accidentally not once (and I think the number is reaching 50 or something). Suggesting a platform to be "not supported for a release or two" may mean by the time the compiler is good the amount of issues to fix for a platform to regain quality may become a blocker for the next LTS. I really hope this is not the option Oracle is proposing. > My impression is that most of these were not toolchain issues per se. > Rather, they were broken or incomplete changes in platform-dependent > code that weren't tested on these "alternative" platforms before being > pushed. Oracle has provided dev-submit so that non-Oracle folks can > do some minimal testing on all the platforms that Oracle supports. I > know that I would sometimes like to have similarly "easy" testing for > various platforms not supported by Oracle. > > I didn't suggest "no testing" if there is a compiler version that > supports the new language standards but has not yet been fully > product-qualified by the people who are using it. I think gcc on arm > may be in that category. But I think it would be very disappointing > if the complete lack of a usable version of some compiler were to > completely block us in this area. (Unfortunately, such a lack appears > to be the situation for XLC compiler used for the AIX/ppc port.) The > proposal is not very aggressive. > >> We both know what and how long it takes to do a thorough OpenJDK compiler upgrade. If the community were made aware of this earlier, I would have definitely started planning for a compiler upgrade for ARM port in JDK 11. > I'm understand that it takes time and effort to do a toolchain > upgrade. And turning on support for new language version without > changing the toolchain version isn't really all that different. > > This proposal didn't suddenly appear out of nowhere. There has been > wistful discussion about using new language features for a long time, > with an understanding that going anywhere with that was difficult > because of some popular toolchain deficiencies (notably MSVC++) and > the need to upgrade others. There have been ongoing efforts in this > direction, e.g. various changes to support building with C++11/14 > support enabled. Oracle made toolchain upgrades in JDK 11, in part to > support the language standard change. (Unfortunately, the relevant > toolchain upgrade JEP was labelled "Confidential", even though a lot > of the work was in the open, and some of it was explicitly about > dealing with differences arising from turning on C++11/14.) > > I think JDK 12 for this JEP is reasonable goal at this stage. Of > course, that goal might not be achieved, for a variety of reasons. > That's why it is not yet in the Targeted state and why there is a > formal process for moving to that state; there's still work to be > done, and we'll have to see how that work proceeds. > >> With that, I conditionally agree with the proposal provided cooperating ports are given sufficient lead time to upgrade. We started testing ARM with 7.3 and will report on success. > From Michael.Rasmussen at roguewave.com Mon Oct 15 15:32:17 2018 From: Michael.Rasmussen at roguewave.com (Michael Rasmussen) Date: Mon, 15 Oct 2018 15:32:17 +0000 Subject: CDS and JVM-TI agent questions and crash Message-ID: Hi I have some questions regarding Class Data Sharing and class file transforming agents: 1. Is it possible to detect during Agent_OnLoad if CDS is used or not? Checking for "sharing" in the "java.vm.info" system property, which seems to be how the tests are detecting it, is not an option, as this property contains bogus information during Agent_OnLoad. 2. Should the registration of a ClassFileHook, especially with can_generate_all_class_hook_events and can_generate_early_class_hook_events capabilities set, disable CDS? 3. Is it possible to explicitly disable CDS from a JVM-TI agent during Agent_OnLoad? The reason for these questions is that we (JRebel) recently became aware of an issue if CDS is used, where transforming some of the boot classes causes the JVM to crash. The simplest example of this is replacing Object's byte in a ClassFileLoadHook with a copy of themselves, see inlined example agent code below. I don't know if there are more issues than that, as that's where we are currently stuck. Running on a 11+28 fastdebug (with classes.jsa file present) it produces the following assert: # Internal Error (.../src/hotspot/share/oops/constantPool.cpp:325), pid=589, tid=590 # assert(on_stack()) failed: should always be set for shared constant pools if running with -Xshare:off (or no classes.jsa file), it works without any issue. Kind regards Michael Rasmussen /* --- snip --- */ #include #include #include void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env, jclass class_being_redefined, jobject loader, const char* name, jobject protection_domain, jint class_data_len, const unsigned char* class_data, jint* new_class_data_len, unsigned char** new_class_data) { if (strcmp("java/lang/Object", name) == 0) { *new_class_data_len = class_data_len; (*jvmti)->Allocate(jvmti, *new_class_data_len, new_class_data); memcpy(*new_class_data, class_data, *new_class_data_len); } } JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void *reserved) { jvmtiEnv *jvmti = NULL; jint rc = (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9); jvmtiCapabilities caps; memset(&caps, 0, sizeof(caps)); caps.can_redefine_classes = 1; caps.can_generate_all_class_hook_events = 1; caps.can_generate_early_vmstart = 1; caps.can_generate_early_class_hook_events = 1; (*jvmti)->AddCapabilities(jvmti, &caps); jvmtiEventCallbacks callbacks; memset(&callbacks, 0, sizeof(callbacks)); callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook; (*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks)); (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL); return 0; } /* --- snap --- */ From volker.simonis at gmail.com Mon Oct 15 18:07:11 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 15 Oct 2018 20:07:11 +0200 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: Message-ID: Just a quick analysis (will look into a fix tomorrow). - When saving a class to the CDS archive, the VM calls ConstantPool::remove_unshareable_info() on its constant pool: void ConstantPool::remove_unshareable_info() { ... // Shared ConstantPools are in the RO region, so the _flags cannot be modified. // The _on_stack flag is used to prevent ConstantPools from deallocation during // class redefinition. Since shared ConstantPools cannot be deallocated anyway, // we always set _on_stack to true to avoid having to change _flags during runtime. _flags |= (_on_stack | _is_shared); This calls sets the '_on_stack' flag on the constant pool for the reasons described in the comment. - When a class is being loaded from the CDS archive, the VM calls ConstantPool::restore_unshareable_info(TRAPS) on its constant pool: // CDS support. Create a new resolved_references array. void ConstantPool::restore_unshareable_info(TRAPS) { assert(is_constantPool(), "ensure C++ vtable is restored"); assert(on_stack(), "should always be set for shared constant pools"); assert(is_shared(), "should always be set for shared constant pools"); assert(_cache != NULL, "constant pool _cache should not be NULL"); This calls asserts for the '_on_stack' flag being set. - When loading a class from the CSD archive withSystemDictionary::load_shared_class(), ConstantPool::restore_unshareable_info(TRAPS) is actually called from InstanceKlass::restore_unshareable_info(TRAPS) after the instanceKlass was loaded: InstanceKlass* SystemDictionary::load_shared_class(InstanceKlass* ik, Handle class_loader, Handle protection_domain, TRAPS) { ... InstanceKlass* new_ik = KlassFactory::check_shared_class_file_load_hook( ik, class_name, class_loader, protection_domain, CHECK_NULL); if (new_ik != NULL) { // The class is changed by CFLH. Return the new class. The shared class is // not used. return new_ik; } // Adjust methods to recover missing data. They need addresses for // interpreter entry points and their default native method address // must be reset. // Updating methods must be done under a lock so multiple // threads don't update these in parallel // // Shared classes are all currently loaded by either the bootstrap or // internal parallel class loaders, so this will never cause a deadlock // on a custom class loader lock. ClassLoaderData* loader_data = ClassLoaderData::class_loader_data(class_loader()); { HandleMark hm(THREAD); Handle lockObject = compute_loader_lock_object(class_loader, THREAD); check_loader_lock_contention(lockObject, THREAD); ObjectLocker ol(lockObject, THREAD, true); // prohibited package check assumes all classes loaded from archive call // restore_unshareable_info which calls ik->set_package() ik->restore_unshareable_info(loader_data, protection_domain, CHECK_NULL); } - However, when a class loaded from the CDS archive is being redefined there's a shortcut in SystemDictionary::load_shared_class() which instantly returns the new, redefined class, without calling InstanceKlass::restore_unshareable_info() on it: InstanceKlass* new_ik = KlassFactory::check_shared_class_file_load_hook( ik, class_name, class_loader, protection_domain, CHECK_NULL); if (new_ik != NULL) { // The class is changed by CFLH. Return the new class. The shared class is // not used. return new_ik; } - For the well known class java.lang.Object the procedure is a little different. It gets loaded early in SystemDictionary::resolve_preloaded_classes() and for some reason we directly call ConstantPool::restore_unshareable_info() right on its constant pool after it was loaded: #if INCLUDE_CDS if (UseSharedSpaces) { resolve_wk_klasses_through(WK_KLASS_ENUM_NAME(Object_klass), scan, CHECK); ... // Initialize the constant pool for the Object_class Object_klass()->constants()->restore_unshareable_info(CHECK); - This fails if java.lang.Object was redefined, because in that case its constant pool won't have its '_on_stack' flag set to 'true'. - I don't think that this extra call to Object_klass()->constants()->restore_unshareable_info() is required any more, because the call to resolve_wk_klasses_through(WK_KLASS_ENUM_NAME(Object_klass)) already loads java.lang.Object through SystemDictionary::load_shared_class() which will call restore_unshareable_info() on it (if it was not redefined). But I'll have to check this more thoroughly... I'll do that and open a bug for this issue tomorrow. Regards, Volker On Mon, Oct 15, 2018 at 5:32 PM Michael Rasmussen wrote: > > Hi > > I have some questions regarding Class Data Sharing and class file transforming agents: > > 1. Is it possible to detect during Agent_OnLoad if CDS is used or not? > Checking for "sharing" in the "java.vm.info" system property, which seems to be how the tests are detecting it, is not an option, as this property contains bogus information during Agent_OnLoad. > > 2. Should the registration of a ClassFileHook, especially with can_generate_all_class_hook_events and can_generate_early_class_hook_events capabilities set, disable CDS? > > 3. Is it possible to explicitly disable CDS from a JVM-TI agent during Agent_OnLoad? > > > The reason for these questions is that we (JRebel) recently became aware of an issue if CDS is used, where transforming some of the boot classes causes the JVM to crash. > The simplest example of this is replacing Object's byte in a ClassFileLoadHook with a copy of themselves, see inlined example agent code below. > I don't know if there are more issues than that, as that's where we are currently stuck. > > Running on a 11+28 fastdebug (with classes.jsa file present) it produces the following assert: > # Internal Error (.../src/hotspot/share/oops/constantPool.cpp:325), pid=589, tid=590 > # assert(on_stack()) failed: should always be set for shared constant pools > > if running with -Xshare:off (or no classes.jsa file), it works without any issue. > > Kind regards > Michael Rasmussen > > /* --- snip --- */ > #include > #include > #include > > void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env, > jclass class_being_redefined, jobject loader, const char* name, jobject protection_domain, > jint class_data_len, const unsigned char* class_data, > jint* new_class_data_len, unsigned char** new_class_data) { > > if (strcmp("java/lang/Object", name) == 0) { > *new_class_data_len = class_data_len; > (*jvmti)->Allocate(jvmti, *new_class_data_len, new_class_data); > memcpy(*new_class_data, class_data, *new_class_data_len); > } > } > > JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void *reserved) { > jvmtiEnv *jvmti = NULL; > jint rc = (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9); > > jvmtiCapabilities caps; > memset(&caps, 0, sizeof(caps)); > > caps.can_redefine_classes = 1; > caps.can_generate_all_class_hook_events = 1; > caps.can_generate_early_vmstart = 1; > caps.can_generate_early_class_hook_events = 1; > > (*jvmti)->AddCapabilities(jvmti, &caps); > > jvmtiEventCallbacks callbacks; > memset(&callbacks, 0, sizeof(callbacks)); > callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook; > (*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks)); > (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL); > > return 0; > } > /* --- snap --- */ From jiangli.zhou at oracle.com Mon Oct 15 18:50:49 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 15 Oct 2018 11:50:49 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: Message-ID: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> Hi Volker, Thanks for taking this! I was thinking about this issue also. We probably should disable sharing by turning off UseSharedSpaces (in KlassFactory::check_shared_class_file_load_hook) if a shared class loaded by boot loader is redefined. Archived sub-classes are invalidated and not used if the super class is different. When shared java.lang.Object is redefined, all archived classes are invalidated. Michael, please see more comments below. On 10/15/18 11:07 AM, Volker Simonis wrote: > Just a quick analysis (will look into a fix tomorrow). > > - When saving a class to the CDS archive, the VM calls > ConstantPool::remove_unshareable_info() on its constant pool: > > void ConstantPool::remove_unshareable_info() { > ... > // Shared ConstantPools are in the RO region, so the _flags cannot > be modified. > // The _on_stack flag is used to prevent ConstantPools from > deallocation during > // class redefinition. Since shared ConstantPools cannot be > deallocated anyway, > // we always set _on_stack to true to avoid having to change _flags > during runtime. > _flags |= (_on_stack | _is_shared); > > This calls sets the '_on_stack' flag on the constant pool for the > reasons described in the comment. > > - When a class is being loaded from the CDS archive, the VM calls > ConstantPool::restore_unshareable_info(TRAPS) on its constant pool: > > // CDS support. Create a new resolved_references array. > void ConstantPool::restore_unshareable_info(TRAPS) { > assert(is_constantPool(), "ensure C++ vtable is restored"); > assert(on_stack(), "should always be set for shared constant pools"); > assert(is_shared(), "should always be set for shared constant pools"); > assert(_cache != NULL, "constant pool _cache should not be NULL"); > > This calls asserts for the '_on_stack' flag being set. > > - When loading a class from the CSD archive > withSystemDictionary::load_shared_class(), > ConstantPool::restore_unshareable_info(TRAPS) is actually called from > InstanceKlass::restore_unshareable_info(TRAPS) after the instanceKlass > was loaded: > > InstanceKlass* SystemDictionary::load_shared_class(InstanceKlass* ik, > Handle class_loader, > Handle > protection_domain, TRAPS) { > ... > InstanceKlass* new_ik = KlassFactory::check_shared_class_file_load_hook( > ik, class_name, class_loader, protection_domain, CHECK_NULL); > if (new_ik != NULL) { > // The class is changed by CFLH. Return the new class. The shared class is > // not used. > return new_ik; > } > > // Adjust methods to recover missing data. They need addresses for > // interpreter entry points and their default native method address > // must be reset. > > // Updating methods must be done under a lock so multiple > // threads don't update these in parallel > // > // Shared classes are all currently loaded by either the bootstrap or > // internal parallel class loaders, so this will never cause a deadlock > // on a custom class loader lock. > > ClassLoaderData* loader_data = > ClassLoaderData::class_loader_data(class_loader()); > { > HandleMark hm(THREAD); > Handle lockObject = compute_loader_lock_object(class_loader, THREAD); > check_loader_lock_contention(lockObject, THREAD); > ObjectLocker ol(lockObject, THREAD, true); > // prohibited package check assumes all classes loaded from archive call > // restore_unshareable_info which calls ik->set_package() > ik->restore_unshareable_info(loader_data, protection_domain, CHECK_NULL); > } > > - However, when a class loaded from the CDS archive is being redefined > there's a shortcut in SystemDictionary::load_shared_class() which > instantly returns the new, redefined class, without calling > InstanceKlass::restore_unshareable_info() on it: > > InstanceKlass* new_ik = KlassFactory::check_shared_class_file_load_hook( > ik, class_name, class_loader, protection_domain, CHECK_NULL); > if (new_ik != NULL) { > // The class is changed by CFLH. Return the new class. The shared class is > // not used. > return new_ik; > } > > - For the well known class java.lang.Object the procedure is a little > different. It gets loaded early in > SystemDictionary::resolve_preloaded_classes() and for some reason we > directly call ConstantPool::restore_unshareable_info() right on its > constant pool after it was loaded: > > #if INCLUDE_CDS > if (UseSharedSpaces) { > resolve_wk_klasses_through(WK_KLASS_ENUM_NAME(Object_klass), scan, CHECK); > ... > // Initialize the constant pool for the Object_class > Object_klass()->constants()->restore_unshareable_info(CHECK); > > - This fails if java.lang.Object was redefined, because in that case > its constant pool won't have its '_on_stack' flag set to 'true'. > > - I don't think that this extra call to > Object_klass()->constants()->restore_unshareable_info() is required > any more, because the call to > resolve_wk_klasses_through(WK_KLASS_ENUM_NAME(Object_klass)) already > loads java.lang.Object through SystemDictionary::load_shared_class() > which will call restore_unshareable_info() on it (if it was not > redefined). But I'll have to check this more thoroughly... > > I'll do that and open a bug for this issue tomorrow. > > Regards, > Volker > > On Mon, Oct 15, 2018 at 5:32 PM Michael Rasmussen > wrote: >> Hi >> >> I have some questions regarding Class Data Sharing and class file transforming agents: >> >> 1. Is it possible to detect during Agent_OnLoad if CDS is used or not? >> Checking for "sharing" in the "java.vm.info" system property, which seems to be how the tests are detecting it, is not an option, as this property contains bogus information during Agent_OnLoad. >> >> 2. Should the registration of a ClassFileHook, especially with can_generate_all_class_hook_events and can_generate_early_class_hook_events capabilities set, disable CDS? >> >> 3. Is it possible to explicitly disable CDS from a JVM-TI agent during Agent_OnLoad? Currently, there is no easy to disable CDS from a JVMTI agent that I can think of. We could provide a property for this. Best regards, Jiangli >> >> >> The reason for these questions is that we (JRebel) recently became aware of an issue if CDS is used, where transforming some of the boot classes causes the JVM to crash. >> The simplest example of this is replacing Object's byte in a ClassFileLoadHook with a copy of themselves, see inlined example agent code below. >> I don't know if there are more issues than that, as that's where we are currently stuck. >> >> Running on a 11+28 fastdebug (with classes.jsa file present) it produces the following assert: >> # Internal Error (.../src/hotspot/share/oops/constantPool.cpp:325), pid=589, tid=590 >> # assert(on_stack()) failed: should always be set for shared constant pools >> >> if running with -Xshare:off (or no classes.jsa file), it works without any issue. >> >> Kind regards >> Michael Rasmussen >> >> /* --- snip --- */ >> #include >> #include >> #include >> >> void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env, >> jclass class_being_redefined, jobject loader, const char* name, jobject protection_domain, >> jint class_data_len, const unsigned char* class_data, >> jint* new_class_data_len, unsigned char** new_class_data) { >> >> if (strcmp("java/lang/Object", name) == 0) { >> *new_class_data_len = class_data_len; >> (*jvmti)->Allocate(jvmti, *new_class_data_len, new_class_data); >> memcpy(*new_class_data, class_data, *new_class_data_len); >> } >> } >> >> JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void *reserved) { >> jvmtiEnv *jvmti = NULL; >> jint rc = (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9); >> >> jvmtiCapabilities caps; >> memset(&caps, 0, sizeof(caps)); >> >> caps.can_redefine_classes = 1; >> caps.can_generate_all_class_hook_events = 1; >> caps.can_generate_early_vmstart = 1; >> caps.can_generate_early_class_hook_events = 1; >> >> (*jvmti)->AddCapabilities(jvmti, &caps); >> >> jvmtiEventCallbacks callbacks; >> memset(&callbacks, 0, sizeof(callbacks)); >> callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook; >> (*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks)); >> (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL); >> >> return 0; >> } >> /* --- snap --- */ From jiangli.zhou at oracle.com Mon Oct 15 19:09:37 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 15 Oct 2018 12:09:37 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> Message-ID: <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> I logged issue in JDK-8212200, https://bugs.openjdk.java.net/browse/JDK-8212200. Thanks, Jiangli On 10/15/18 11:50 AM, Jiangli Zhou wrote: > Hi Volker, > > Thanks for taking this! I was thinking about this issue also. We > probably should disable sharing by turning off UseSharedSpaces (in > KlassFactory::check_shared_class_file_load_hook) if a shared class > loaded by boot loader is redefined. Archived sub-classes are > invalidated and not used if the super class is different. When shared > java.lang.Object is redefined, all archived classes are invalidated. > > Michael, please see more comments below. > > On 10/15/18 11:07 AM, Volker Simonis wrote: >> Just a quick analysis (will look into a fix tomorrow). >> >> - When saving a class to the CDS archive, the VM calls >> ConstantPool::remove_unshareable_info() on its constant pool: >> >> void ConstantPool::remove_unshareable_info() { >> ... >> ?? // Shared ConstantPools are in the RO region, so the _flags cannot >> be modified. >> ?? // The _on_stack flag is used to prevent ConstantPools from >> deallocation during >> ?? // class redefinition. Since shared ConstantPools cannot be >> deallocated anyway, >> ?? // we always set _on_stack to true to avoid having to change _flags >> during runtime. >> ?? _flags |= (_on_stack | _is_shared); >> >> This calls sets the '_on_stack' flag on the constant pool for the >> reasons described in the comment. >> >> - When a class is being loaded from the CDS archive, the VM calls >> ConstantPool::restore_unshareable_info(TRAPS) on its constant pool: >> >> // CDS support. Create a new resolved_references array. >> void ConstantPool::restore_unshareable_info(TRAPS) { >> ?? assert(is_constantPool(), "ensure C++ vtable is restored"); >> ?? assert(on_stack(), "should always be set for shared constant pools"); >> ?? assert(is_shared(), "should always be set for shared constant >> pools"); >> ?? assert(_cache != NULL, "constant pool _cache should not be NULL"); >> >> This calls asserts for the '_on_stack' flag being set. >> >> - When loading a class from the CSD archive >> withSystemDictionary::load_shared_class(), >> ConstantPool::restore_unshareable_info(TRAPS) is actually called from >> InstanceKlass::restore_unshareable_info(TRAPS) after the instanceKlass >> was loaded: >> >> InstanceKlass* SystemDictionary::load_shared_class(InstanceKlass* ik, >> ??????????????????????????????????????????????????? Handle class_loader, >> ??????????????????????????????????????????????????? Handle >> protection_domain, TRAPS) { >> ... >> ???? InstanceKlass* new_ik = >> KlassFactory::check_shared_class_file_load_hook( >> ???????? ik, class_name, class_loader, protection_domain, CHECK_NULL); >> ???? if (new_ik != NULL) { >> ?????? // The class is changed by CFLH. Return the new class. The >> shared class is >> ?????? // not used. >> ?????? return new_ik; >> ???? } >> >> ???? // Adjust methods to recover missing data.? They need addresses for >> ???? // interpreter entry points and their default native method address >> ???? // must be reset. >> >> ???? // Updating methods must be done under a lock so multiple >> ???? // threads don't update these in parallel >> ???? // >> ???? // Shared classes are all currently loaded by either the >> bootstrap or >> ???? // internal parallel class loaders, so this will never cause a >> deadlock >> ???? // on a custom class loader lock. >> >> ???? ClassLoaderData* loader_data = >> ClassLoaderData::class_loader_data(class_loader()); >> ???? { >> ?????? HandleMark hm(THREAD); >> ?????? Handle lockObject = compute_loader_lock_object(class_loader, >> THREAD); >> ?????? check_loader_lock_contention(lockObject, THREAD); >> ?????? ObjectLocker ol(lockObject, THREAD, true); >> ?????? // prohibited package check assumes all classes loaded from >> archive call >> ?????? // restore_unshareable_info which calls ik->set_package() >> ?????? ik->restore_unshareable_info(loader_data, protection_domain, >> CHECK_NULL); >> ???? } >> >> - However, when a class loaded from the CDS archive is being redefined >> there's a shortcut in SystemDictionary::load_shared_class() which >> instantly returns the new, redefined class, without calling >> InstanceKlass::restore_unshareable_info() on it: >> >> ???? InstanceKlass* new_ik = >> KlassFactory::check_shared_class_file_load_hook( >> ???????? ik, class_name, class_loader, protection_domain, CHECK_NULL); >> ???? if (new_ik != NULL) { >> ?????? // The class is changed by CFLH. Return the new class. The >> shared class is >> ?????? // not used. >> ?????? return new_ik; >> ???? } >> >> - For the well known class java.lang.Object the procedure is a little >> different. It gets loaded early in >> SystemDictionary::resolve_preloaded_classes() and for some reason we >> directly call ConstantPool::restore_unshareable_info() right on its >> constant pool after it was loaded: >> >> #if INCLUDE_CDS >> ?? if (UseSharedSpaces) { >> resolve_wk_klasses_through(WK_KLASS_ENUM_NAME(Object_klass), scan, >> CHECK); >> ... >> ???? // Initialize the constant pool for the Object_class >> Object_klass()->constants()->restore_unshareable_info(CHECK); >> >> - This fails if java.lang.Object was redefined, because in that case >> its constant pool won't have its '_on_stack' flag set to 'true'. >> >> - I don't think that this extra call to >> Object_klass()->constants()->restore_unshareable_info() is required >> any more, because the call to >> resolve_wk_klasses_through(WK_KLASS_ENUM_NAME(Object_klass)) already >> loads java.lang.Object through SystemDictionary::load_shared_class() >> which will call restore_unshareable_info() on it (if it was not >> redefined). But I'll have to check this more thoroughly... >> >> I'll do that and open a bug for this issue tomorrow. >> >> Regards, >> Volker >> >> On Mon, Oct 15, 2018 at 5:32 PM Michael Rasmussen >> wrote: >>> Hi >>> >>> I have some questions regarding Class Data Sharing and class file >>> transforming agents: >>> >>> 1. Is it possible to detect during Agent_OnLoad if CDS is used or not? >>> Checking for "sharing" in the "java.vm.info" system property, which >>> seems to be how the tests are detecting it, is not an option, as >>> this property contains bogus information during Agent_OnLoad. >>> >>> 2. Should the registration of a ClassFileHook, especially with >>> can_generate_all_class_hook_events and >>> can_generate_early_class_hook_events capabilities set, disable CDS? >>> >>> 3. Is it possible to explicitly disable CDS from a JVM-TI agent >>> during Agent_OnLoad? > Currently, there is no easy to disable CDS from a JVMTI agent that I > can think of. We could provide a property for this. > > Best regards, > Jiangli >>> >>> >>> The reason for these questions is that we (JRebel) recently became >>> aware of an issue if CDS is used, where transforming some of the >>> boot classes causes the JVM to crash. >>> The simplest example of this is replacing Object's byte in a >>> ClassFileLoadHook with a copy of themselves, see inlined example >>> agent code below. >>> I don't know if there are more issues than that, as that's where we >>> are currently stuck. >>> >>> Running on a 11+28 fastdebug (with classes.jsa file present) it >>> produces the following assert: >>> #? Internal Error (.../src/hotspot/share/oops/constantPool.cpp:325), >>> pid=589, tid=590 >>> #? assert(on_stack()) failed: should always be set for shared >>> constant pools >>> >>> if running with -Xshare:off (or no classes.jsa file), it works >>> without any issue. >>> >>> Kind regards >>> Michael Rasmussen >>> >>> /* --- snip --- */ >>> #include >>> #include >>> #include >>> >>> void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env, >>> ???????? jclass class_being_redefined, jobject loader, const char* >>> name, jobject protection_domain, >>> ???????? jint class_data_len, const unsigned char* class_data, >>> ???????? jint* new_class_data_len, unsigned char** new_class_data) { >>> >>> ???????? if (strcmp("java/lang/Object", name) == 0) { >>> ???????????????? *new_class_data_len = class_data_len; >>> ???????????????? (*jvmti)->Allocate(jvmti, *new_class_data_len, >>> new_class_data); >>> ???????????????? memcpy(*new_class_data, class_data, >>> *new_class_data_len); >>> ???????? } >>> } >>> >>> JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void >>> *reserved) { >>> ???????? jvmtiEnv *jvmti = NULL; >>> ???????? jint rc = (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9); >>> >>> ???????? jvmtiCapabilities caps; >>> ???????? memset(&caps, 0, sizeof(caps)); >>> >>> ???????? caps.can_redefine_classes = 1; >>> ???????? caps.can_generate_all_class_hook_events = 1; >>> ???????? caps.can_generate_early_vmstart = 1; >>> ???????? caps.can_generate_early_class_hook_events = 1; >>> >>> ???????? (*jvmti)->AddCapabilities(jvmti, &caps); >>> >>> ???????? jvmtiEventCallbacks callbacks; >>> ???????? memset(&callbacks, 0, sizeof(callbacks)); >>> ???????? callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook; >>> ???????? (*jvmti)->SetEventCallbacks(jvmti, &callbacks, >>> sizeof(callbacks)); >>> ???????? (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, >>> JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL); >>> >>> ???????? return 0; >>> } >>> /* --- snap --- */ > From szegedia at gmail.com Wed Oct 10 14:00:59 2018 From: szegedia at gmail.com (Attila Szegedi) Date: Wed, 10 Oct 2018 16:00:59 +0200 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> Message-ID: > On 2018. Oct 10., at 11:39, Luke Hutchison wrote: > > So given a class > > class A { > void someMethod() { > try { > // ... > } finally { > Reference.reachabilityFence(this); > } > } > } > > how is it that a call > > new A().someMethod() > > does not have a race condition between the "new A()" instance being invoked > and reachability analysis realizing that there is a "this" reference in the > finally block? Is it always true that there is an overlap between the > reference to the invocation target being held and the reference to "this" > in the method body being considered as reachable? > > Or stated as the inverse, is it guaranteed that there will be no gap in > time (albeit infinitessimally small) between method invocation and the > running of the method in which there will be no reference to the "new A()" > object, where the finalizer could still be run? The ? statically decidable ? existence of the call to reachabilityFence with ?this? being passed into it (or call to anything else really? System.out.println would do too) is enough to ensure that the reference must live until at least the fence call was invoked (note: not necessarily until it returns, but that?s academical). There?s no race condition. FWIW, most of these optimizations only kick in after inlining - someMethod() would get inlined into the body of the method that created the new A() instance and then the resulting merged code with inlined body can be analyzed as a unit. I think you would most likely not even notice this in scenarios that don?t involve inlining, but honestly it?s possible to have a JIT that could decide to mark a local variable dead immediately after it got loaded on stack for a call if there are no further uses of it. Even so, from reachability point of view it?s an atomic operation to pop the arguments off the stack in the caller frame and assign them to parameter local variables in the callee frame. Having a gap in the reachability while transitioning from caller context to callee context would be a glaring bug indeed. Attila. From szegedia at gmail.com Wed Oct 10 14:01:40 2018 From: szegedia at gmail.com (Attila Szegedi) Date: Wed, 10 Oct 2018 16:01:40 +0200 Subject: Finalizer being run while class still in use (escape analysis bug) In-Reply-To: <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> References: <1a687eaf-0d7c-bbb7-faf3-3707b4389586@redhat.com> <19a5a48f-9c73-76be-813c-0bb093b8dc73@redhat.com> Message-ID: <719DC264-C858-43D2-9589-178564F67EC2@gmail.com> > On 2018. Oct 10., at 11:50, Andrew Haley wrote: > > On 10/10/2018 10:39 AM, Luke Hutchison wrote: >> [?] >> does not have a race condition between the "new A()" instance being >> invoked and reachability analysis realizing that there is a "this" >> reference in the finally block? Is it always true that there is an >> overlap between the reference to the invocation target being held >> and the reference to "this" in the method body being considered as >> reachable? > > You need two threads to have a race condition. I'm not sure what > you're trying to ask: the reachabilityFence must run strictly after > the body of the try clause, and it the object will be reachable until > then. I believe the finalizer thread is the second thread he had in mind. > [?] >>> >>> No. Believe it or not, a live method does not constitute an access >>> to an object. An object can die and be finalized even before one >>> if its methods is entered. >> >> That is highly surprising. > > Yes, it is . You are not the first to be surprised by this. > >> This makes finalizers a lot less reliable and more difficult to use >> than they could be. >> >> Wouldn't it make sense to consider the invocation targets in the stack >> during liveness analysis, so that out of the cases you gave, "finalizers >> may be run too soon (as here,) too late, or never", at least the "too soon" >> case is fixed? > > We considered doing that; the discussion is online somewhere. I was in > favour of doing what you suggest, but there is some performance impact > of preserving liveness for all methods. In the end it was decided that > we'd provide reachabilityFence. I think the general consensus is that finalization is a shunned feature[1]. Weakening optimizations, or really making any tradeoff no matter how small for the purpose of improving finalization ergonomics is not going to happen as there?s no perceived value in doing it. Attila. -- [1] unofficial term I just made up, but I like the ring of it. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From Michael.Rasmussen at roguewave.com Mon Oct 15 21:14:16 2018 From: Michael.Rasmussen at roguewave.com (Michael Rasmussen) Date: Mon, 15 Oct 2018 21:14:16 +0000 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com>, <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> Message-ID: > I logged issue in JDK-8212200, > https://bugs.openjdk.java.net/browse/JDK-8212200 > Thanks, > Jiangli Hi Jiangli, Thanks -- though regarding your comment there, I don't know if it only applies when changing Object's bytes? I did a small test (attached below) with adding a default method to Serializable, and it crashes as well, though differently -- don't know if what Volker mentioned fixes this though. Running the below with CDS off, I get: $ ../jdk/bin/java -Xshare:off -agentpath:./libagent.so Test.java public default java.lang.Object java.io.Serializable._$_$_blah() With it on, I get the following crash: $ ../jdk/bin/java -Xshare:on -agentpath:./libagent.so Test.java # Internal Error (.../src/hotspot/share/classfile/javaClasses.cpp:667), pid=277, tid=278 # assert(java_string->klass() == SystemDictionary::String_klass()) failed: must be java_string /Michael Stack trace: V [libjvm.so+0xe80ea5] java_lang_String::equals(oop, unsigned short*, int)+0x85 V [libjvm.so+0x9e6e44] CompactHashtable::decode_entry(CompactHashtable*, unsigned int, char const*, int)+0x54 V [libjvm.so+0x167fae6] StringTable::lookup_shared(unsigned short*, int, unsigned int)+0x166 V [libjvm.so+0x16801df] StringTable::intern(Handle, unsigned short*, int, Thread*)+0x6f V [libjvm.so+0x16802e3] StringTable::intern(Symbol*, Thread*)+0x53 V [libjvm.so+0xae9921] ConstantPool::uncached_string_at(int, Thread*)+0x31 V [libjvm.so+0xc4426c] fieldDescriptor::string_initial_value(Thread*) const+0x9c V [libjvm.so+0xe88056] initialize_static_field(fieldDescriptor*, Handle, Thread*)+0x1c6 V [libjvm.so+0xe23819] InstanceKlass::do_local_static_fields(void (*)(fieldDescriptor*, Handle, Thread*), Handle, Thread*)+0xd9 V [libjvm.so+0xe70157] java_lang_Class::initialize_mirror_fields(Klass*, Handle, Handle, Thread*)+0x117 V [libjvm.so+0xe723c3] java_lang_Class::create_mirror(Klass*, Handle, Handle, Handle, Thread*)+0x723 V [libjvm.so+0x95b8fd] ClassFileParser::fill_instance_klass(InstanceKlass*, bool, Thread*)+0x1c8d V [libjvm.so+0x95c12f] ClassFileParser::create_instance_klass(bool, Thread*)+0x5f V [libjvm.so+0x118e5f5] KlassFactory::create_from_stream(ClassFileStream*, Symbol*, ClassLoaderData*, Handle, InstanceKlass const*, GrowableArray*, Thread*)+0x615 V [libjvm.so+0x96b76c] ClassLoader::load_class(Symbol*, bool, Thread*)+0x1bc V [libjvm.so+0x171a3e6] SystemDictionary::load_instance_class(Symbol*, Handle, Thread*)+0x5f6 V [libjvm.so+0x1718139] SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, Handle, Thread*)+0xa69 V [libjvm.so+0x1718662] SystemDictionary::resolve_or_null(Symbol*, Handle, Handle, Thread*)+0x42 V [libjvm.so+0x17187de] SystemDictionary::resolve_or_fail(Symbol*, Handle, Handle, bool, Thread*)+0x1e V [libjvm.so+0x1718991] SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, Thread*)+0x151 V [libjvm.so+0x1718af1] SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, SystemDictionary::WKID&, Thread*)+0x61 V [libjvm.so+0x1718d33] SystemDictionary::initialize_preloaded_classes(Thread*)+0x153 V [libjvm.so+0x17190ea] SystemDictionary::initialize(Thread*)+0x21a V [libjvm.so+0x17a8140] Universe::genesis(Thread*)+0x5f0 V [libjvm.so+0x17a8b5c] universe2_init()+0x2c V [libjvm.so+0xe1e558] init_globals()+0xa8 V [libjvm.so+0x1766d17] Threads::create_vm(JavaVMInitArgs*, bool*)+0x2d7 Sample code: /* --- snip ---- */ #include #include #include static char *serializable_class_name = "java/io/Serializable"; /* public interface java.io.Serializable { default Object _$_$_blah() { return this; } } */ static jbyte serializable_class_bytes[] = { 202, 254, 186, 190, 0, 0, 0, 55, 0, 10, 1, 0, 20, 106, 97, 118, 97, 47, 105, 111, 47, 83, 101, 114, 105, 97, 108, 105, 122, 97, 98, 108, 101, 7, 0, 1, 1, 0, 16, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 79, 98, 106, 101, 99, 116, 7, 0, 3, 1, 0, 17, 83, 101, 114, 105, 97, 108, 105, 122, 97, 98, 108, 101, 46, 106, 97, 118, 97, 1, 0, 9, 95, 36, 95, 36, 95, 98, 108, 97, 104, 1, 0, 20, 40, 41, 76, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 79, 98, 106, 101, 99, 116, 59, 1, 0, 4, 67, 111, 100, 101, 1, 0, 10, 83, 111, 117, 114, 99, 101, 70, 105, 108, 101, 6, 1, 0, 2, 0, 4, 0, 0, 0, 0, 0, 1, 0, 1, 0, 6, 0, 7, 0, 1, 0, 8, 0, 0, 0, 14, 0, 1, 0, 1, 0, 0, 0, 2, 42, 176, 0, 0, 0, 0, 0, 1, 0, 9, 0, 0, 0, 2, 0, 5 }; static jsize serializable_class_bytes_len = 183; void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env, jclass class_being_redefined, jobject loader, const char* name, jobject protection_domain, jint class_data_len, const unsigned char* class_data, jint* new_class_data_len, unsigned char** new_class_data) { if (strcmp(serializable_class_name, name) == 0) { *new_class_data_len = serializable_class_bytes_len; (*jvmti)->Allocate(jvmti, *new_class_data_len, new_class_data); memcpy(*new_class_data, serializable_class_bytes, *new_class_data_len); } } JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void *reserved) { jvmtiEnv *jvmti; jint rc = (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9); jvmtiCapabilities caps; memset(&caps, 0, sizeof(caps)); caps.can_redefine_classes = 1; caps.can_generate_all_class_hook_events = 1; caps.can_generate_early_vmstart = 1; caps.can_generate_early_class_hook_events = 1; (*jvmti)->AddCapabilities(jvmti, &caps); jvmtiEventCallbacks callbacks; memset(&callbacks, 0, sizeof(callbacks)); callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook; (*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks)); (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL); return 0; } /* ---- snap --- */ import java.io.Serializable; import java.util.stream.Stream; public class Test { public static void main(String[] args) { Stream.of(Serializable.class.getDeclaredMethods()).forEach(System.out::println); } } /* ---- snude ---- */ From jiangli.zhou at oracle.com Mon Oct 15 21:59:20 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 15 Oct 2018 14:59:20 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> Message-ID: <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> Hi Michael, On 10/15/18 2:14 PM, Michael Rasmussen wrote: >> I logged issue in JDK-8212200, >> https://bugs.openjdk.java.net/browse/JDK-8212200 >> Thanks, >> Jiangli > Hi Jiangli, > > Thanks -- though regarding your comment there, I don't know if it only applies when changing Object's bytes? > I did a small test (attached below) with adding a default method to Serializable, and it crashes as well, though differently -- don't know if what Volker mentioned fixes this though. > > Running the below with CDS off, I get: > $ ../jdk/bin/java -Xshare:off -agentpath:./libagent.so Test.java > public default java.lang.Object java.io.Serializable._$_$_blah() > > With it on, I get the following crash: > $ ../jdk/bin/java -Xshare:on -agentpath:./libagent.so Test.java > # Internal Error (.../src/hotspot/share/classfile/javaClasses.cpp:667), pid=277, tid=278 > # assert(java_string->klass() == SystemDictionary::String_klass()) failed: must be java_string This appears to be a bug when archived java.lang.String class is not used (String class is dynamically loaded in this case) but the mapped compact string hash stable is still in use. We need to detect such case and disable the shared compact string table. Thanks for reporting this issue. I've added it to the JDK-8212200 report as well. One simple alternative fix is to disable sharing when some events (like CFLH) were enabled. This in fact was the behavior before JDK 9. In JDK 9, we enabled CDS for debugging. Looks like there are corner cases that are not uncovered by our testing. I'd like to seek opinions and feedback on the importance of supporting debugging with CDS enabled. Thanks! Jiangli > > /Michael > > Stack trace: > V [libjvm.so+0xe80ea5] java_lang_String::equals(oop, unsigned short*, int)+0x85 > V [libjvm.so+0x9e6e44] CompactHashtable::decode_entry(CompactHashtable*, unsigned int, char const*, int)+0x54 > V [libjvm.so+0x167fae6] StringTable::lookup_shared(unsigned short*, int, unsigned int)+0x166 > V [libjvm.so+0x16801df] StringTable::intern(Handle, unsigned short*, int, Thread*)+0x6f > V [libjvm.so+0x16802e3] StringTable::intern(Symbol*, Thread*)+0x53 > V [libjvm.so+0xae9921] ConstantPool::uncached_string_at(int, Thread*)+0x31 > V [libjvm.so+0xc4426c] fieldDescriptor::string_initial_value(Thread*) const+0x9c > V [libjvm.so+0xe88056] initialize_static_field(fieldDescriptor*, Handle, Thread*)+0x1c6 > V [libjvm.so+0xe23819] InstanceKlass::do_local_static_fields(void (*)(fieldDescriptor*, Handle, Thread*), Handle, Thread*)+0xd9 > V [libjvm.so+0xe70157] java_lang_Class::initialize_mirror_fields(Klass*, Handle, Handle, Thread*)+0x117 > V [libjvm.so+0xe723c3] java_lang_Class::create_mirror(Klass*, Handle, Handle, Handle, Thread*)+0x723 > V [libjvm.so+0x95b8fd] ClassFileParser::fill_instance_klass(InstanceKlass*, bool, Thread*)+0x1c8d > V [libjvm.so+0x95c12f] ClassFileParser::create_instance_klass(bool, Thread*)+0x5f > V [libjvm.so+0x118e5f5] KlassFactory::create_from_stream(ClassFileStream*, Symbol*, ClassLoaderData*, Handle, InstanceKlass const*, GrowableArray*, Thread*)+0x615 > V [libjvm.so+0x96b76c] ClassLoader::load_class(Symbol*, bool, Thread*)+0x1bc > V [libjvm.so+0x171a3e6] SystemDictionary::load_instance_class(Symbol*, Handle, Thread*)+0x5f6 > V [libjvm.so+0x1718139] SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, Handle, Thread*)+0xa69 > V [libjvm.so+0x1718662] SystemDictionary::resolve_or_null(Symbol*, Handle, Handle, Thread*)+0x42 > V [libjvm.so+0x17187de] SystemDictionary::resolve_or_fail(Symbol*, Handle, Handle, bool, Thread*)+0x1e > V [libjvm.so+0x1718991] SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, Thread*)+0x151 > V [libjvm.so+0x1718af1] SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, SystemDictionary::WKID&, Thread*)+0x61 > V [libjvm.so+0x1718d33] SystemDictionary::initialize_preloaded_classes(Thread*)+0x153 > V [libjvm.so+0x17190ea] SystemDictionary::initialize(Thread*)+0x21a > V [libjvm.so+0x17a8140] Universe::genesis(Thread*)+0x5f0 > V [libjvm.so+0x17a8b5c] universe2_init()+0x2c > V [libjvm.so+0xe1e558] init_globals()+0xa8 > V [libjvm.so+0x1766d17] Threads::create_vm(JavaVMInitArgs*, bool*)+0x2d7 > > > Sample code: > /* --- snip ---- */ > #include > #include > #include > > static char *serializable_class_name = "java/io/Serializable"; > /* > public interface java.io.Serializable { > default Object _$_$_blah() { return this; } > } > */ > static jbyte serializable_class_bytes[] = { > 202, 254, 186, 190, 0, 0, 0, 55, 0, 10, 1, 0, 20, 106, 97, 118, > 97, 47, 105, 111, 47, 83, 101, 114, 105, 97, 108, 105, 122, 97, 98, 108, > 101, 7, 0, 1, 1, 0, 16, 106, 97, 118, 97, 47, 108, 97, 110, 103, > 47, 79, 98, 106, 101, 99, 116, 7, 0, 3, 1, 0, 17, 83, 101, 114, > 105, 97, 108, 105, 122, 97, 98, 108, 101, 46, 106, 97, 118, 97, 1, 0, > 9, 95, 36, 95, 36, 95, 98, 108, 97, 104, 1, 0, 20, 40, 41, 76, > 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 79, 98, 106, 101, 99, 116, > 59, 1, 0, 4, 67, 111, 100, 101, 1, 0, 10, 83, 111, 117, 114, 99, > 101, 70, 105, 108, 101, 6, 1, 0, 2, 0, 4, 0, 0, 0, 0, 0, > 1, 0, 1, 0, 6, 0, 7, 0, 1, 0, 8, 0, 0, 0, 14, 0, > 1, 0, 1, 0, 0, 0, 2, 42, 176, 0, 0, 0, 0, 0, 1, 0, > 9, 0, 0, 0, 2, 0, 5 > }; > static jsize serializable_class_bytes_len = 183; > > void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env, > jclass class_being_redefined, jobject loader, const char* name, jobject protection_domain, > jint class_data_len, const unsigned char* class_data, > jint* new_class_data_len, unsigned char** new_class_data) { > > if (strcmp(serializable_class_name, name) == 0) { > *new_class_data_len = serializable_class_bytes_len; > (*jvmti)->Allocate(jvmti, *new_class_data_len, new_class_data); > memcpy(*new_class_data, serializable_class_bytes, *new_class_data_len); > } > } > > JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void *reserved) { > jvmtiEnv *jvmti; > jint rc = (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9); > > jvmtiCapabilities caps; > memset(&caps, 0, sizeof(caps)); > > caps.can_redefine_classes = 1; > caps.can_generate_all_class_hook_events = 1; > caps.can_generate_early_vmstart = 1; > caps.can_generate_early_class_hook_events = 1; > > (*jvmti)->AddCapabilities(jvmti, &caps); > > jvmtiEventCallbacks callbacks; > memset(&callbacks, 0, sizeof(callbacks)); > callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook; > (*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks)); > (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL); > > return 0; > } > /* ---- snap --- */ > import java.io.Serializable; > import java.util.stream.Stream; > > public class Test { > public static void main(String[] args) { > Stream.of(Serializable.class.getDeclaredMethods()).forEach(System.out::println); > } > } > /* ---- snude ---- */ From Michael.Rasmussen at roguewave.com Mon Oct 15 22:30:33 2018 From: Michael.Rasmussen at roguewave.com (Michael Rasmussen) Date: Mon, 15 Oct 2018 22:30:33 +0000 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> , <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> Message-ID: > One simple alternative fix is to disable sharing when some events (like > CFLH) were enabled. This in fact was the behavior before JDK 9. In JDK > 9, we enabled CDS for debugging. Looks like there are corner cases that > are not uncovered by our testing. I'd like to seek opinions and feedback > on the importance of supporting debugging with CDS enabled. Or at least CFLH together with can_generate_all_class_hook_events and can_generate_early_class_hook_events capabilities. CDS hasn't been part of our own testing scenarios so far, which is why we haven't notice these issues on our end before now. We only just discovered it because of user reports about crashes, which turned out to be because the java-11-openjdk package on Fedora comes with pre-generated CDS archive. But especially with JEP 341 just around the corner, it makes it very important to find and weed out these bugs now! /Michael From jiangli.zhou at oracle.com Mon Oct 15 22:35:54 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 15 Oct 2018 15:35:54 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> Message-ID: On 10/15/18 3:30 PM, Michael Rasmussen wrote: >> One simple alternative fix is to disable sharing when some events (like >> CFLH) were enabled. This in fact was the behavior before JDK 9. In JDK >> 9, we enabled CDS for debugging. Looks like there are corner cases that >> are not uncovered by our testing. I'd like to seek opinions and feedback >> on the importance of supporting debugging with CDS enabled. > Or at least CFLH together with can_generate_all_class_hook_events and can_generate_early_class_hook_events capabilities. > > CDS hasn't been part of our own testing scenarios so far, which is why we haven't notice these issues on our end before now. > We only just discovered it because of user reports about crashes, which turned out to be because the java-11-openjdk package on Fedora comes with pre-generated CDS archive. > > But especially with JEP 341 just around the corner, it makes it very important to find and weed out these bugs now! Totally agree. These issues are now more important with the default CDS archive. Thanks for reporting them! Best regards, Jiangli > > /Michael From kim.barrett at oracle.com Mon Oct 15 23:25:47 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Oct 2018 19:25:47 -0400 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> <10d8f2ea-883e-ec21-4fea-06a52a52fdc6@bell-sw.com> <407CE450-6EED-4955-9B1F-EA564E134D11@oracle.com> Message-ID: > On Oct 15, 2018, at 7:51 AM, Aleksei Voitylov wrote: > > Kim, > > If you were suggesting to just proceed with the change without giving a sufficient lead time for the ports that were willing to upgrade to do so, that sounds very alarming. What is "sufficient lead time"? I'm not proposing an answer, just suggesting that 3 months (approx time between JEP publication and JDK 12 fork) might be sufficient, and a worthy goal. That's a decision that ought to be made as part of the Targeting discussion for this JEP. Right now, it's not even a Candidate, since there's still work to be done. > Meanwhile, our testing has finished and I'm now confident we will be able to switch ARM port over to 7.x in 12 time frame. There are several issues that we still need to diagnose, but this does not look like a blocker. That's good news, though pretty much what I expected. I'm more worried about Solaris. Having a goal of JDK 12 may help move things along. From vladimir.kozlov at oracle.com Tue Oct 16 00:05:11 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 15 Oct 2018 17:05:11 -0700 Subject: RFC: How to deal with/abstract some offset-related Shenandoah changes? In-Reply-To: <27467623-88df-c77d-20a3-2e2e26a3ecee@redhat.com> References: <27467623-88df-c77d-20a3-2e2e26a3ecee@redhat.com> Message-ID: Hi Roman, I and others looked on this and agree with you that BrooksPointer::byte_offset() checks and other code you showed should be cleaned up. It may take time to come up with good suggestion. Please, wait. Regards, Vladimir On 10/12/18 1:57 AM, Roman Kennke wrote: > Hello, > > I'd like to get some opinions on how should we deal with some changes > that we need in Shenandoah, that really look a bit ugly, but that seem a > little out of place in a GC interface too. Let me show you the code: > > https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html > > There are 3 blocks there: > - The first deals with the aarch64 addressing bits. Only the lower > 48bits are used for addressing, and thus the upper 16 are masked away. > - the second block adjusts the offset by 8 bytes for Shenandoah when > computing the normalized heap-based-offset for narrow-oops. This is > needed to get combined decode+read-barrier instruction to work iirc. > - the last block allows implicit null-checks on Shenandoah fwd ptr loads > (now including the masking from block#1 above). > > I really am not sure how do deal with this. It does look like it would > require a two-way abstraction: > 1. CPU specific. i.e. move this method under src/cpu/$CPU and do the > right thing there (possibly duplicating for platforms that do common stuff) > 2. GC specific to deal with the -8 offset of Shenandoah > > However, I doubt that it would make the code any more easier to > understand/read/etc. And how would the GC abstraction look like? bool > BarrierSetAssembler::needs_explicit_null_check(..) ? And if GC says > 'false' then don't check any of the other stuff? > > I'm a bit at a loss here. Any suggestions are appreciated. > > > There is another place that deals with special offsets: > https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/ci/ciInstanceKlass.cpp.udiff.html > > (look at get_canonical_holder() there) > > Please let me know what you think. > > Roman > From david.holmes at oracle.com Tue Oct 16 06:55:40 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Oct 2018 16:55:40 +1000 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: Message-ID: Looks good - officially Reviewed! Thanks for fixing Serguei! David On 16/10/2018 3:03 PM, serguei.spitsyn at oracle.com wrote: > Please, review a fix for: > https://bugs.openjdk.java.net/browse/JDK-8024368 > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ > > > Summary: > > ? Private method should not allocate vtable indeces. > ? The fix tweaks several conditions in the functions: > klassVtable::update_inherited_vtable() and > klassVtable::needs_new_vtable_entry(), > ? adds a couple of asserts and makes several comments up-to-date. > > ? The fix was preliminary reviewed by Karen, David H. and Lois. > > Testing: > > ? The fix was verified with > runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} > ? and vmTestbase/vm/runtime/defmeth tests. > ? Also, I ran a mach5 job with the flags: > tier1,tier2,hs-tier3,hs-tier4,hs-tier5. > > ? By suggestion from David, it was also tested with the -Xlog output > for vtables before and after the fix. > ? I've done this comparison (awk was used to get rid of the first > column with the time stamps). > ? It shows that private methods were included into vtable before the > fix but not included after. > ? Please, see one fragment as an example: > > ?? 166c166 > ?? size 17 > ?? --- > ?? >? copy vtable from java.lang.Object to java.lang.StackFrameInfo > size 16 > ?? 180d179 > ?? index 16, flags: private > ?? 182c181 > ?? java.lang.LiveStackFrameInfo size 17 > ?? --- > ?? >? copy vtable from java.lang.StackFrameInfo to > java.lang.LiveStackFrameInfo size 16 > ?? 188,201c187,191 > ?? java.lang.StackStreamFactory$AbstractStackWalker size 18 > ?? at index 5, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I > at index 6, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; > at index 7, flags: protected abstract > ?? java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V at > index 8, flags: protected abstract > ?? java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at index > 9, flags: protected abstract > ?? java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I at > index 10, flags: protected > ?? java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z > at index 11, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; > at index 12, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at > index 13, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; > at index 14, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I at > index 15, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I > at index 16, flags: private native > ?? java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; > at index 17, flags: private native > ?? --- > ?? >? copy vtable from java.lang.Object to > java.lang.StackStreamFactory$AbstractStackWalker size 9 > ?? >? adding > java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; > at index 5, flags: protected abstract > ?? >? adding > java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V at > index 6, flags: protected abstract > ?? >? adding > java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at index > 7, flags: protected abstract > ?? >? adding > java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I at > index 8, flags: protected > > > Thanks, > Serguei > From serguei.spitsyn at oracle.com Tue Oct 16 07:17:19 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 16 Oct 2018 00:17:19 -0700 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: Message-ID: Thank you a lot, David! Serguei On 10/15/18 23:55, David Holmes wrote: > Looks good - officially Reviewed! > > Thanks for fixing Serguei! > > David > > On 16/10/2018 3:03 PM, serguei.spitsyn at oracle.com wrote: >> Please, review a fix for: >> https://bugs.openjdk.java.net/browse/JDK-8024368 >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ >> >> >> Summary: >> >> ?? Private method should not allocate vtable indeces. >> ?? The fix tweaks several conditions in the functions: >> klassVtable::update_inherited_vtable() and >> klassVtable::needs_new_vtable_entry(), >> ?? adds a couple of asserts and makes several comments up-to-date. >> >> ?? The fix was preliminary reviewed by Karen, David H. and Lois. >> >> Testing: >> >> ?? The fix was verified with >> runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} >> ?? and vmTestbase/vm/runtime/defmeth tests. >> ?? Also, I ran a mach5 job with the flags: >> tier1,tier2,hs-tier3,hs-tier4,hs-tier5. >> >> ?? By suggestion from David, it was also tested with the -Xlog output >> for vtables before and after the fix. >> ?? I've done this comparison (awk was used to get rid of the first >> column with the time stamps). >> ?? It shows that private methods were included into vtable before the >> fix but not included after. >> ?? Please, see one fragment as an example: >> >> ??? 166c166 >> ??? > size 17 >> ??? --- >> ??? >? copy vtable from java.lang.Object to java.lang.StackFrameInfo >> size 16 >> ??? 180d179 >> ??? > at index 16, flags: private >> ??? 182c181 >> ??? > java.lang.LiveStackFrameInfo size 17 >> ??? --- >> ??? >? copy vtable from java.lang.StackFrameInfo to >> java.lang.LiveStackFrameInfo size 16 >> ??? 188,201c187,191 >> ??? > java.lang.StackStreamFactory$AbstractStackWalker size 18 >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.close()V at index 5, >> flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I >> at index 6, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >> at index 7, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >> at index 8, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >> index 9, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >> at index 10, flags: protected >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z >> at index 11, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; >> at index 12, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at >> index 13, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; >> at index 14, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I >> at index 15, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I >> at index 16, flags: private native >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; >> at index 17, flags: private native >> ??? --- >> ??? >? copy vtable from java.lang.Object to >> java.lang.StackStreamFactory$AbstractStackWalker size 9 >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >> at index 5, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >> at index 6, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >> index 7, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >> at index 8, flags: protected >> >> >> Thanks, >> Serguei >> From rkennke at redhat.com Tue Oct 16 07:20:23 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 Oct 2018 09:20:23 +0200 Subject: RFC: How to deal with/abstract some offset-related Shenandoah changes? In-Reply-To: References: <27467623-88df-c77d-20a3-2e2e26a3ecee@redhat.com> Message-ID: <5413DFC4-170B-4C2F-93F4-EC963D8F2100@redhat.com> Hi Vladimir, thanks for the heads-up. Sure, I'll wait and try to come up with something reasonable myself in the meantime. Thanks! /Roman Am 16. Oktober 2018 02:05:11 MESZ schrieb Vladimir Kozlov : >Hi Roman, > >I and others looked on this and agree with you that >BrooksPointer::byte_offset() checks and other >code you showed should be cleaned up. It may take time to come up with >good suggestion. Please, wait. > >Regards, >Vladimir > >On 10/12/18 1:57 AM, Roman Kennke wrote: >> Hello, >> >> I'd like to get some opinions on how should we deal with some changes >> that we need in Shenandoah, that really look a bit ugly, but that >seem a >> little out of place in a GC interface too. Let me show you the code: >> >> >https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html >> >> There are 3 blocks there: >> - The first deals with the aarch64 addressing bits. Only the lower >> 48bits are used for addressing, and thus the upper 16 are masked >away. >> - the second block adjusts the offset by 8 bytes for Shenandoah when >> computing the normalized heap-based-offset for narrow-oops. This is >> needed to get combined decode+read-barrier instruction to work iirc. >> - the last block allows implicit null-checks on Shenandoah fwd ptr >loads >> (now including the masking from block#1 above). >> >> I really am not sure how do deal with this. It does look like it >would >> require a two-way abstraction: >> 1. CPU specific. i.e. move this method under src/cpu/$CPU and do the >> right thing there (possibly duplicating for platforms that do common >stuff) >> 2. GC specific to deal with the -8 offset of Shenandoah >> >> However, I doubt that it would make the code any more easier to >> understand/read/etc. And how would the GC abstraction look like? bool >> BarrierSetAssembler::needs_explicit_null_check(..) ? And if GC says >> 'false' then don't check any of the other stuff? >> >> I'm a bit at a loss here. Any suggestions are appreciated. >> >> >> There is another place that deals with special offsets: >> >https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/ci/ciInstanceKlass.cpp.udiff.html >> >> (look at get_canonical_holder() there) >> >> Please let me know what you think. >> >> Roman >> From coleen.phillimore at oracle.com Tue Oct 16 13:43:04 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 16 Oct 2018 09:43:04 -0400 Subject: Result: New hotspot group member: Mikael Vidstedt Message-ID: <2a8ca1eb-18ea-5075-ed28-d4f470c836dc@oracle.com> The vote for Mikael Vidstedt [1] is now closed. Yes: 15 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Coleen Phillimore [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-October/034450.html From shade at redhat.com Tue Oct 16 15:24:36 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 16 Oct 2018 17:24:36 +0200 Subject: RFC: "in kb" typo in UL JEP? Message-ID: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> Hi, Somebody asked on #openjdk @ OFTC this: (05:08:54 PM) tpunder: Anybody know anything about the JEP 158 (Unified JVM Logging) "filesize" argument? JEP 158 says that value is in KB and uses "filesize=1024" as an example for 1MB gc log file sizes. However on both Linux and OS X "filesize=1024" gives me gc log files that are 1024 bytes in length. (05:11:46 PM) shade: I think JEP grammar is incorrect, and UL accepts the usual suffixes (K, M, G), and no-suffix is bytes (05:13:41 PM) Maldivia_: tpunder: java -Xlog:help gives this as an example: -Xlog:gc=trace:file=gctrace.txt:uptimemillis,pids:filecount=5,filesize=1m (05:14:30 PM) tpunder: Yes, filesize=1M does work. Just wondering why the JEP says it should be KB. (05:15:42 PM) shade: it think it was drafted before implementation and tests (05:15:56 PM) shade: so, mistake (or rather, optimistic typo) in grammar comment (05:16:01 PM) tpunder: I specifically ran into a problem upgrading Apache Solr to Java 11 because the launch scripts were setup to use filesize=20000 (they meant wanted 20M) based on the JEP documentation. Manually patching the launch script to filesize=20M works. Indeed, JEP page (http://openjdk.java.net/jeps/158) says: output-option := filecount= filesize= parameter=value ...while the parsing code does it via the usual Arguments::atojulong, which would treat non-suffix space as bytes: logFileOutput.cpp: } else if (strcmp(FileSizeOptionKey, key) == 0) { julong value; success = Arguments::atojulong(value_str, &value); May I drop "in kb" from the JEP text? Asking here, because all people responsible for that JEP apparently left. I think I have enough permissions to edit the underlying bug (https://bugs.openjdk.java.net/browse/JDK-8046148), so that changes propagate automatically, but I would like someone to confirm this change is the right thing to do. -Aleksey From igor.ignatyev at oracle.com Tue Oct 16 16:06:06 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 16 Oct 2018 09:06:06 -0700 Subject: RFR(XS) : 8171097 : Convert TestReservedSpace_test to Gtest Message-ID: http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > 188 lines changed: 187 ins; 1 del; 0 mod; Hi all, could you please review this small (and hopefully trivial) patch which converts internal TestReservedSpace_test to gtest? since the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. webrev: http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8171097 testing: - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants - build w/ precompiled-headers enabled and disabled PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review Thanks, -- Igor From mandy.chung at oracle.com Tue Oct 16 16:08:27 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 Oct 2018 09:08:27 -0700 Subject: Review Request: JDK-8207146: Rename jdk.internal.misc.Unsafe::xxxObject to xxxReference Message-ID: <3bed4fd6-1f6f-b95d-1eb9-62396a8e9815@oracle.com> Webrev: http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8207146/webrev.00/ Unsafe::getObject returns a reference to an object. Similarly Unsafe::putObject sets a reference in the given base+offset. This patch proposes to rename Unsafe xxxObject to xxxReference that will make the xxxReference API very clear that these are getters/setters for a reference (instance of object class) and ready for adding xxxValue in the futurefor values. Note that this renaming only applies to jdk.internal.misc.Unsafe but not sun.misc.Unsafe.? So no impact to existing libraries that depends on sun.misc.Unsafe in jdk.unsupported module. I ran jdk_core, core_tools, langtools and nashorn tier1-3, hotspot_runtime, hotspot_compiler test groups. thanks Mandy From jcbeyler at google.com Tue Oct 16 16:15:41 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 16 Oct 2018 09:15:41 -0700 Subject: RFC: "in kb" typo in UL JEP? In-Reply-To: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> References: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> Message-ID: Hi Aleksey, If the consensus is to change the text, I would change the example as well: -Xlog:gc=trace:file=gctrace.txt:uptimemillis,pids:filecount=5,filesize=1024 - log messages tagged with 'gc' tag using 'trace' level to a rotating fileset with 5 files with size 1MB with base name Thanks, Jc On Tue, Oct 16, 2018 at 8:25 AM Aleksey Shipilev wrote: > Hi, > > Somebody asked on #openjdk @ OFTC this: > > (05:08:54 PM) tpunder: Anybody know anything about the JEP 158 (Unified > JVM Logging) "filesize" > argument? JEP 158 says that value is in KB and uses "filesize=1024" as an > example for 1MB gc log > file sizes. However on both Linux and OS X "filesize=1024" gives me gc log > files that are 1024 bytes > in length. > (05:11:46 PM) shade: I think JEP grammar is incorrect, and UL accepts the > usual suffixes (K, M, G), > and no-suffix is bytes > (05:13:41 PM) Maldivia_: tpunder: java -Xlog:help gives this as an example: > -Xlog:gc=trace:file=gctrace.txt:uptimemillis,pids:filecount=5,filesize=1m > (05:14:30 PM) tpunder: Yes, filesize=1M does work. Just wondering why the > JEP says it should be KB. > (05:15:42 PM) shade: it think it was drafted before implementation and > tests > (05:15:56 PM) shade: so, mistake (or rather, optimistic typo) in grammar > comment > (05:16:01 PM) tpunder: I specifically ran into a problem upgrading Apache > Solr to Java 11 because > the launch scripts were setup to use filesize=20000 (they meant wanted > 20M) based on the JEP > documentation. Manually patching the launch script to filesize=20M works. > > Indeed, JEP page (http://openjdk.java.net/jeps/158) says: > > output-option := filecount= > filesize= > parameter=value > > ...while the parsing code does it via the usual Arguments::atojulong, > which would treat non-suffix > space as bytes: > > logFileOutput.cpp: > > } else if (strcmp(FileSizeOptionKey, key) == 0) { > julong value; > success = Arguments::atojulong(value_str, &value); > > > May I drop "in kb" from the JEP text? Asking here, because all people > responsible for that JEP > apparently left. I think I have enough permissions to edit the underlying > bug > (https://bugs.openjdk.java.net/browse/JDK-8046148), so that changes > propagate automatically, but I > would like someone to confirm this change is the right thing to do. > > -Aleksey > > -- Thanks, Jc From dean.long at oracle.com Tue Oct 16 16:51:43 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 16 Oct 2018 09:51:43 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count Message-ID: https://bugs.openjdk.java.net/browse/JDK-8021335 http://cr.openjdk.java.net/~dlong/8021335/webrev.3/ This change attempts to fix an old and intermittent problem with ThreadMXBean thread counts. Changes have 3 main parts: 1. Make sure hidden threads don't affect counts (even when exiting) 2. Fix race reading counts using atomic counters 3. Remove some workarounds in test (because they hide bugs) dl From jcbeyler at google.com Tue Oct 16 17:28:40 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 16 Oct 2018 10:28:40 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: Message-ID: Hi Dean, I noticed a typo here : "are atomically" is in the comment but should no longer be there I believe; the sentence probably should be: // These 2 counters are like the above thread counts, but are Any way we could move this test into a helper method and both add_thread/current_thread_exiting could use the same "ignore" thread code so that we ensure the two never get out of sync? + if (jt->is_hidden_from_external_view() || + jt->is_jvmti_agent_thread()) { + return; + } (Also by curiosity, add_thread has an assert on the Threads_lock but not thread_exiting?) Thanks, Jc On Tue, Oct 16, 2018 at 9:52 AM wrote: > https://bugs.openjdk.java.net/browse/JDK-8021335 > http://cr.openjdk.java.net/~dlong/8021335/webrev.3/ > > This change attempts to fix an old and intermittent problem with > ThreadMXBean thread counts. > Changes have 3 main parts: > 1. Make sure hidden threads don't affect counts (even when exiting) > 2. Fix race reading counts using atomic counters > 3. Remove some workarounds in test (because they hide bugs) > > dl > -- Thanks, Jc From serguei.spitsyn at oracle.com Tue Oct 16 18:34:39 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 16 Oct 2018 11:34:39 -0700 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: Message-ID: Re-sending my RFR. Behive managed not to deliver it to the target mailing lists. David got it because he was on the cc-list. Please, review a fix for: https://bugs.openjdk.java.net/browse/JDK-8024368 Webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ Summary: ? Private method should not allocate vtable indeces. ? The fix tweaks several conditions in the functions: klassVtable::update_inherited_vtable() and klassVtable::needs_new_vtable_entry(), ? adds a couple of asserts and makes several comments up-to-date. ? The fix was preliminary reviewed by Karen, David H. and Lois. Testing: ? The fix was verified with runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} ? and vmTestbase/vm/runtime/defmeth tests. ? Also, I ran a mach5 job with the flags: tier1,tier2,hs-tier3,hs-tier4,hs-tier5. ? By suggestion from David, it was also tested with the -Xlog output for vtables before and after the fix. ? I've done this comparison (awk was used to get rid of the first column with the time stamps). ? It shows that private methods were included into vtable before the fix but not included after. ? Please, see one fragment as an example: ?? 166c166 ?? ? copy vtable from java.lang.Object to java.lang.StackFrameInfo size 16 ?? 180d179 ?? ? copy vtable from java.lang.StackFrameInfo to java.lang.LiveStackFrameInfo size 16 ?? 188,201c187,191 ?? ? copy vtable from java.lang.Object to java.lang.StackStreamFactory$AbstractStackWalker size 9 ?? >? adding java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; at index 5, flags: protected abstract ?? >? adding java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V at index 6, flags: protected abstract ?? >? adding java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at index 7, flags: protected abstract ?? >? adding java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I at index 8, flags: protected Thanks, Serguei From dean.long at oracle.com Tue Oct 16 18:59:38 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 16 Oct 2018 11:59:38 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: Message-ID: On 10/16/18 10:28 AM, JC Beyler wrote: > Hi Dean, > > I noticed a typo here : > "are atomically" is in the comment but should no longer be there I > believe; the sentence probably should be: > // These 2 counters are like the above thread counts, but are > Fixed! > Any way we could move this test into a helper method and both > add_thread/current_thread_exiting could use the same "ignore" thread > code so that we ensure the two never get out of sync? > > +? if (jt->is_hidden_from_external_view() || > +? ? ? jt->is_jvmti_agent_thread()) { > +? ? return; > +? } > Good idea.? Fixed. > (Also by curiosity, add_thread has an assert on the Threads_lock but > not thread_exiting?) > Right, I followed the existing pattern where current_thread_exiting() only uses the atomic counters, so it doesn't need Threads_lock. dl > Thanks, > Jc > > > On Tue, Oct 16, 2018 at 9:52 AM > wrote: > > https://bugs.openjdk.java.net/browse/JDK-8021335 > http://cr.openjdk.java.net/~dlong/8021335/webrev.3/ > > > This change attempts to fix an old and intermittent problem with > ThreadMXBean thread counts. > Changes have 3 main parts: > 1. Make sure hidden threads don't affect counts (even when exiting) > 2. Fix race reading counts using atomic counters > 3. Remove some workarounds in test (because they hide bugs) > > dl > > > > -- > > Thanks, > Jc From jcbeyler at google.com Tue Oct 16 19:51:36 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 16 Oct 2018 12:51:36 -0700 Subject: RFR(XS) : 8171097 : Convert TestReservedSpace_test to Gtest In-Reply-To: References: Message-ID: Hi Igor, The change looks good to me, I compared old/new, the port looks good. For the reviewers, here is the original test code: http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1071 Thanks, Jc On Tue, Oct 16, 2018 at 9:06 AM Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > > 188 lines changed: 187 ins; 1 del; 0 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which > converts internal TestReservedSpace_test to gtest? > since the old test is still used by WhiteBox::runMemoryUnitTests, the old > code hasn't been removed. it will be removed later when all 4 tests used by > runMemoryUnitTests are converted. > > webrev: > http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8171097 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 > in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been > sent out for official review > > Thanks, > -- Igor -- Thanks, Jc From gromero at linux.vnet.ibm.com Tue Oct 16 21:09:45 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 16 Oct 2018 18:09:45 -0300 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection Message-ID: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> Hi, Could the following small change be reviewed please? Bug : https://bugs.openjdk.java.net/browse/JDK-8212481 Webrev: http://cr.openjdk.java.net/~gromero/8212481/v1/ It adds 'darn' instruction introduced with POWER9 CPUs to the Macroassembler and uses it to set PowerArchitecturePPC64 to POWER9 when the instruction is available on the CPU so it can be used in the future by any POWER9-specific JVM code or by any JVM code specifically dependent on the 'darn' instruction, using VM_Version::has_darn(). It was checked on POWER9 as the following: gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 176 On-line CPU(s) list: 0-175 Thread(s) per core: 4 Core(s) per socket: 22 Socket(s): 2 NUMA node(s): 8 Model: 2.2 (pvr 004e 1202) Model name: POWER9 (raw), altivec supported CPU max MHz: 2800.0000 CPU min MHz: 2300.0000 L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 10240K NUMA node0 CPU(s): 0-87 NUMA node8 CPU(s): 88-175 NUMA node250 CPU(s): NUMA node251 CPU(s): NUMA node252 CPU(s): NUMA node253 CPU(s): NUMA node254 CPU(s): NUMA node255 CPU(s): gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ LD_SHOW_AUXV=1 /bin/true | egrep "HWCAP|PLATFORM" AT_HWCAP: true_le archpmu vsx arch_2_06 dfp ic_snoop smt mmu fpu altivec ppc64 ppc32 AT_HWCAP2: darn ieee128 arch_3_00 vcrypto tar isel ebb dscr arch_2_07 AT_PLATFORM: power9 AT_BASE_PLATFORM:power9 gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ ./java -XX:+Verbose -version dscr value was 0x10 Version: ppc64 fsqrt isel lxarxeh cmpb popcntb popcntw fcfids vand lqarx aes vpmsumb mfdscr vsx ldbrx stdbrx sha darn L1_data_cache_line_size=128 openjdk version "12-internal" 2019-03-19 OpenJDK Runtime Environment (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9) OpenJDK 64-Bit Server VM (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9, mixed mode, sharing) It was also checked on POWER8 and no regression was observed: gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 80 On-line CPU(s) list: 0-79 Thread(s) per core: 8 Core(s) per socket: 10 Socket(s): 1 NUMA node(s): 1 Model: 2.1 (pvr 004b 0201) Model name: POWER8 (architected), altivec supported Hypervisor vendor: horizontal Virtualization type: full L1d cache: 64K L1i cache: 32K NUMA node0 CPU(s): 0-79 gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ LD_SHOW_AUXV=1 /bin/true | egrep "HWCAP|PLATFORM" AT_HWCAP: archpmu vsx arch_2_06 dfp ic_snoop smt mmu fpu altivec ppc64 ppc32 AT_HWCAP2: htm-nosc vcrypto tar isel ebb dscr htm arch_2_07 AT_PLATFORM: power8 AT_BASE_PLATFORM:power8 gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ ./java -XX:+Verbose -version dscr value was 0x0 Version: ppc64 fsqrt isel lxarxeh cmpb popcntb popcntw fcfids vand lqarx aes vpmsumb mfdscr vsx ldbrx stdbrx sha rtm L1_data_cache_line_size=128 openjdk version "12-internal" 2019-03-19 OpenJDK Runtime Environment (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9) OpenJDK 64-Bit Server VM (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9, mixed mode, sharing) Thank you. Best regards, Gustavo From dean.long at oracle.com Tue Oct 16 23:43:16 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 16 Oct 2018 16:43:16 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: Message-ID: New webrev: http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ dl On 10/16/18 11:59 AM, dean.long at oracle.com wrote: > On 10/16/18 10:28 AM, JC Beyler wrote: >> Hi Dean, >> >> I noticed a typo here : >> "are atomically" is in the comment but should no longer be there I >> believe; the sentence probably should be: >> // These 2 counters are like the above thread counts, but are >> > > Fixed! > >> Any way we could move this test into a helper method and both >> add_thread/current_thread_exiting could use the same "ignore" thread >> code so that we ensure the two never get out of sync? >> >> +? if (jt->is_hidden_from_external_view() || >> +? ? ? jt->is_jvmti_agent_thread()) { >> +? ? return; >> +? } >> > > Good idea.? Fixed. > >> (Also by curiosity, add_thread has an assert on the Threads_lock but >> not thread_exiting?) >> > > Right, I followed the existing pattern where current_thread_exiting() > only uses the atomic counters, so it doesn't need Threads_lock. > > dl > >> Thanks, >> Jc >> >> >> On Tue, Oct 16, 2018 at 9:52 AM > > wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8021335 >> http://cr.openjdk.java.net/~dlong/8021335/webrev.3/ >> >> >> This change attempts to fix an old and intermittent problem with >> ThreadMXBean thread counts. >> Changes have 3 main parts: >> 1. Make sure hidden threads don't affect counts (even when exiting) >> 2. Fix race reading counts using atomic counters >> 3. Remove some workarounds in test (because they hide bugs) >> >> dl >> >> >> >> -- >> >> Thanks, >> Jc > From david.holmes at oracle.com Wed Oct 17 01:29:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 Oct 2018 11:29:54 +1000 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: Message-ID: <8af982c0-8e2f-a0fc-cd2c-2f35ffd47480@oracle.com> Bizarre! Looks good - officially Reviewed again! Thanks for fixing Serguei! David On 17/10/2018 4:34 AM, serguei.spitsyn at oracle.com wrote: > Re-sending my RFR. > Behive managed not to deliver it to the target mailing lists. > David got it because he was on the cc-list. > > > Please, review a fix for: > https://bugs.openjdk.java.net/browse/JDK-8024368 > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ > > > Summary: > > ? Private method should not allocate vtable indeces. > ? The fix tweaks several conditions in the functions: > klassVtable::update_inherited_vtable() and > klassVtable::needs_new_vtable_entry(), > ? adds a couple of asserts and makes several comments up-to-date. > > ? The fix was preliminary reviewed by Karen, David H. and Lois. > > Testing: > > ? The fix was verified with > runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} > ? and vmTestbase/vm/runtime/defmeth tests. > ? Also, I ran a mach5 job with the flags: > tier1,tier2,hs-tier3,hs-tier4,hs-tier5. > > ? By suggestion from David, it was also tested with the -Xlog output > for vtables before and after the fix. > ? I've done this comparison (awk was used to get rid of the first > column with the time stamps). > ? It shows that private methods were included into vtable before the > fix but not included after. > ? Please, see one fragment as an example: > > ?? 166c166 > ?? size 17 > ?? --- > ?? >? copy vtable from java.lang.Object to java.lang.StackFrameInfo > size 16 > ?? 180d179 > ?? index 16, flags: private > ?? 182c181 > ?? java.lang.LiveStackFrameInfo size 17 > ?? --- > ?? >? copy vtable from java.lang.StackFrameInfo to > java.lang.LiveStackFrameInfo size 16 > ?? 188,201c187,191 > ?? java.lang.StackStreamFactory$AbstractStackWalker size 18 > ?? at index 5, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I > at index 6, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; > at index 7, flags: protected abstract > ?? java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V at > index 8, flags: protected abstract > ?? java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at index > 9, flags: protected abstract > ?? java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I at > index 10, flags: protected > ?? java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z > at index 11, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; > at index 12, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at > index 13, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; > at index 14, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I at > index 15, flags: private > ?? java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I > at index 16, flags: private native > ?? java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; > at index 17, flags: private native > ?? --- > ?? >? copy vtable from java.lang.Object to > java.lang.StackStreamFactory$AbstractStackWalker size 9 > ?? >? adding > java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; > at index 5, flags: protected abstract > ?? >? adding > java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V at > index 6, flags: protected abstract > ?? >? adding > java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at index > 7, flags: protected abstract > ?? >? adding > java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I at > index 8, flags: protected > > > Thanks, > Serguei > From david.holmes at oracle.com Wed Oct 17 02:33:55 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 Oct 2018 12:33:55 +1000 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: Message-ID: Hi Dean, Thanks for tackling this. I'm still struggling to fully grasp why we need both the PerfCounters and the regular counters. I get that we have to decrement the live counts before ensure_join() has allowed Thread.join() to return, to ensure that if we then check the number of threads it has dropped by one. But I don't understand why that means we need to manage the thread count in two parts. Particularly as now you don't use the PerfCounter to return the live count, so it makes me wonder what role the PerfCounter is playing as it is temporarily inconsistent with the reported live count? Is it simply that we can't atomically decrement the PerfCounters, and we can't require the Threads_lock when decrementing? On the code itself one thing I find irksome is that we already have a daemon flag in remove_thread but decrement_thread_counts doesn't get passed that and so always re-examines the thread to see if it is a daemon. I know only one of the code paths has the daemon flag, so perhaps we should hoist the daemon check up into JavaThread::exit and then pass a daemon parameter to decrement_thread_counts. There's also some ambiguity (existing) in relation to the existence of the jt->threadObj() and its affect on whether the thread is considered a daemon or not. Your check has to mirror the existing checks - the threadObj may be NULL at removal if it was a failed attach, but the addition considers a JNI-attached thread to be non-daemon. Overall this seems buggy to me but as long as your check follows the same rules that is okay. In that regard JavaThread::exit should never encounter a NULL threadObj(). Minor nit: I suggest moving the two occurrences of the comment // Do not count VM internal or JVMTI agent threads inside is_hidden_thread so that we don't have to remember to update the comments if the definition changes. Thanks, David ----- On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: > New webrev: > > http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ > > dl > > On 10/16/18 11:59 AM, dean.long at oracle.com wrote: >> On 10/16/18 10:28 AM, JC Beyler wrote: >>> Hi Dean, >>> >>> I noticed a typo here : >>> "are atomically" is in the comment but should no longer be there I >>> believe; the sentence probably should be: >>> // These 2 counters are like the above thread counts, but are >>> >> >> Fixed! >> >>> Any way we could move this test into a helper method and both >>> add_thread/current_thread_exiting could use the same "ignore" thread >>> code so that we ensure the two never get out of sync? >>> >>> +? if (jt->is_hidden_from_external_view() || >>> +? ? ? jt->is_jvmti_agent_thread()) { >>> +? ? return; >>> +? } >>> >> >> Good idea.? Fixed. >> >>> (Also by curiosity, add_thread has an assert on the Threads_lock but >>> not thread_exiting?) >>> >> >> Right, I followed the existing pattern where current_thread_exiting() >> only uses the atomic counters, so it doesn't need Threads_lock. >> >> dl >> >>> Thanks, >>> Jc >>> >>> >>> On Tue, Oct 16, 2018 at 9:52 AM >> > wrote: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8021335 >>> http://cr.openjdk.java.net/~dlong/8021335/webrev.3/ >>> >>> >>> This change attempts to fix an old and intermittent problem with >>> ThreadMXBean thread counts. >>> Changes have 3 main parts: >>> 1. Make sure hidden threads don't affect counts (even when exiting) >>> 2. Fix race reading counts using atomic counters >>> 3. Remove some workarounds in test (because they hide bugs) >>> >>> dl >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> > From david.holmes at oracle.com Wed Oct 17 03:33:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 Oct 2018 13:33:19 +1000 Subject: Review Request: JDK-8207146: Rename jdk.internal.misc.Unsafe::xxxObject to xxxReference In-Reply-To: <3bed4fd6-1f6f-b95d-1eb9-62396a8e9815@oracle.com> References: <3bed4fd6-1f6f-b95d-1eb9-62396a8e9815@oracle.com> Message-ID: <237430e1-5aea-6c0b-3317-14f12aec72c8@oracle.com> Hi Mandy, I took a look through all of this and it seems okay - though it was a little surprising how far the name change needed to spread. Thanks, David On 17/10/2018 2:08 AM, Mandy Chung wrote: > Webrev: > http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8207146/webrev.00/ > > Unsafe::getObject returns a reference to an object. Similarly > Unsafe::putObject sets a reference in the given base+offset. > This patch proposes to rename Unsafe xxxObject to xxxReference > that will make the xxxReference API very clear that these > are getters/setters for a reference (instance of object class) > and ready for adding xxxValue in the futurefor values. > Note that this renaming only applies to jdk.internal.misc.Unsafe > but not sun.misc.Unsafe.? So no impact to existing libraries > that depends on sun.misc.Unsafe in jdk.unsupported module. > > I ran jdk_core, core_tools, langtools and nashorn tier1-3, > hotspot_runtime, hotspot_compiler test groups. > > thanks > Mandy From serguei.spitsyn at oracle.com Wed Oct 17 05:38:38 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 16 Oct 2018 22:38:38 -0700 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: Message-ID: Thank you again, David! Serguei On 10/15/18 23:55, David Holmes wrote: > Looks good - officially Reviewed! > > Thanks for fixing Serguei! > > David > > On 16/10/2018 3:03 PM, serguei.spitsyn at oracle.com wrote: >> Please, review a fix for: >> https://bugs.openjdk.java.net/browse/JDK-8024368 >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ >> >> >> Summary: >> >> ?? Private method should not allocate vtable indeces. >> ?? The fix tweaks several conditions in the functions: >> klassVtable::update_inherited_vtable() and >> klassVtable::needs_new_vtable_entry(), >> ?? adds a couple of asserts and makes several comments up-to-date. >> >> ?? The fix was preliminary reviewed by Karen, David H. and Lois. >> >> Testing: >> >> ?? The fix was verified with >> runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} >> ?? and vmTestbase/vm/runtime/defmeth tests. >> ?? Also, I ran a mach5 job with the flags: >> tier1,tier2,hs-tier3,hs-tier4,hs-tier5. >> >> ?? By suggestion from David, it was also tested with the -Xlog output >> for vtables before and after the fix. >> ?? I've done this comparison (awk was used to get rid of the first >> column with the time stamps). >> ?? It shows that private methods were included into vtable before the >> fix but not included after. >> ?? Please, see one fragment as an example: >> >> ??? 166c166 >> ??? > size 17 >> ??? --- >> ??? >? copy vtable from java.lang.Object to java.lang.StackFrameInfo >> size 16 >> ??? 180d179 >> ??? > at index 16, flags: private >> ??? 182c181 >> ??? > java.lang.LiveStackFrameInfo size 17 >> ??? --- >> ??? >? copy vtable from java.lang.StackFrameInfo to >> java.lang.LiveStackFrameInfo size 16 >> ??? 188,201c187,191 >> ??? > java.lang.StackStreamFactory$AbstractStackWalker size 18 >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.close()V at index 5, >> flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I >> at index 6, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >> at index 7, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >> at index 8, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >> index 9, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >> at index 10, flags: protected >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z >> at index 11, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; >> at index 12, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at >> index 13, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; >> at index 14, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I >> at index 15, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I >> at index 16, flags: private native >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; >> at index 17, flags: private native >> ??? --- >> ??? >? copy vtable from java.lang.Object to >> java.lang.StackStreamFactory$AbstractStackWalker size 9 >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >> at index 5, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >> at index 6, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >> index 7, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >> at index 8, flags: protected >> >> >> Thanks, >> Serguei >> From thomas.stuefe at gmail.com Wed Oct 17 07:32:35 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 17 Oct 2018 09:32:35 +0200 Subject: RFC: "in kb" typo in UL JEP? In-Reply-To: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> References: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> Message-ID: This seems very reasonable. I noticed this myself in the past. ..Thomas On Tue, Oct 16, 2018 at 5:25 PM Aleksey Shipilev wrote: > > Hi, > > Somebody asked on #openjdk @ OFTC this: > > (05:08:54 PM) tpunder: Anybody know anything about the JEP 158 (Unified JVM Logging) "filesize" > argument? JEP 158 says that value is in KB and uses "filesize=1024" as an example for 1MB gc log > file sizes. However on both Linux and OS X "filesize=1024" gives me gc log files that are 1024 bytes > in length. > (05:11:46 PM) shade: I think JEP grammar is incorrect, and UL accepts the usual suffixes (K, M, G), > and no-suffix is bytes > (05:13:41 PM) Maldivia_: tpunder: java -Xlog:help gives this as an example: > -Xlog:gc=trace:file=gctrace.txt:uptimemillis,pids:filecount=5,filesize=1m > (05:14:30 PM) tpunder: Yes, filesize=1M does work. Just wondering why the JEP says it should be KB. > (05:15:42 PM) shade: it think it was drafted before implementation and tests > (05:15:56 PM) shade: so, mistake (or rather, optimistic typo) in grammar comment > (05:16:01 PM) tpunder: I specifically ran into a problem upgrading Apache Solr to Java 11 because > the launch scripts were setup to use filesize=20000 (they meant wanted 20M) based on the JEP > documentation. Manually patching the launch script to filesize=20M works. > > Indeed, JEP page (http://openjdk.java.net/jeps/158) says: > > output-option := filecount= > filesize= > parameter=value > > ...while the parsing code does it via the usual Arguments::atojulong, which would treat non-suffix > space as bytes: > > logFileOutput.cpp: > > } else if (strcmp(FileSizeOptionKey, key) == 0) { > julong value; > success = Arguments::atojulong(value_str, &value); > > > May I drop "in kb" from the JEP text? Asking here, because all people responsible for that JEP > apparently left. I think I have enough permissions to edit the underlying bug > (https://bugs.openjdk.java.net/browse/JDK-8046148), so that changes propagate automatically, but I > would like someone to confirm this change is the right thing to do. > > -Aleksey > From erik.osterlund at oracle.com Wed Oct 17 15:36:14 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 17 Oct 2018 17:36:14 +0200 Subject: Clean up CompiledMethod::oops_reloc_begin() Message-ID: <5BC756EE.6040900@oracle.com> Hi, In CompiledMethod::oops_reloc_begin() there is code to prevent looking for oops at the verified entry of an nmethod that is not entrant, as a native jump instruction has been overwritten there. Except there would *never* be any immediate oops there for any compiler, even before becoming not entrant, with or without OSR compilation, so this special handling of not entrant vs entrant nmethods is seemingly completely unnecessary. This gets increasingly awkward when oops_do is called concurrently with concurrent class unloading, where an nmethod is racing to become not entrant. To clean this up, I propose to change the boundary for immediate oop scanning and start looking for oops only after the frame building is completed, as there is absolutely no point in looking for oops before that in the first place. This removes unnecessary workarounds that exist today, and removes unnecessary races going forward. It seems like Graal as JIT also inlines oops into the code stream, but never sets the CodeOffsets::Frame_Complete in the nmethod. So I recognize such nmethods and unconditionally start scanning at the verified entry + native jump instruction size. I spoke to the Graal folks, and they said they do not put oops in there either. So I think everyone involved should be happy with this solution. Bug: https://bugs.openjdk.java.net/browse/JDK-8212585 Webrev: http://cr.openjdk.java.net/~eosterlund/8212585/webrev.00/ Thanks, /Erik From shade at redhat.com Wed Oct 17 15:43:40 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 17:43:40 +0200 Subject: RFC: "in kb" typo in UL JEP? In-Reply-To: References: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> Message-ID: <4733772b-1880-dba7-8135-c2d236168f35@redhat.com> Okay, anyone from Oracle wants to confirm? Changing the text in 5... 4... -Aleksey On 10/17/2018 09:32 AM, Thomas St?fe wrote: > This seems very reasonable. I noticed this myself in the past. > > ..Thomas > On Tue, Oct 16, 2018 at 5:25 PM Aleksey Shipilev wrote: >> >> Hi, >> >> Somebody asked on #openjdk @ OFTC this: >> >> (05:08:54 PM) tpunder: Anybody know anything about the JEP 158 (Unified JVM Logging) "filesize" >> argument? JEP 158 says that value is in KB and uses "filesize=1024" as an example for 1MB gc log >> file sizes. However on both Linux and OS X "filesize=1024" gives me gc log files that are 1024 bytes >> in length. >> (05:11:46 PM) shade: I think JEP grammar is incorrect, and UL accepts the usual suffixes (K, M, G), >> and no-suffix is bytes >> (05:13:41 PM) Maldivia_: tpunder: java -Xlog:help gives this as an example: >> -Xlog:gc=trace:file=gctrace.txt:uptimemillis,pids:filecount=5,filesize=1m >> (05:14:30 PM) tpunder: Yes, filesize=1M does work. Just wondering why the JEP says it should be KB. >> (05:15:42 PM) shade: it think it was drafted before implementation and tests >> (05:15:56 PM) shade: so, mistake (or rather, optimistic typo) in grammar comment >> (05:16:01 PM) tpunder: I specifically ran into a problem upgrading Apache Solr to Java 11 because >> the launch scripts were setup to use filesize=20000 (they meant wanted 20M) based on the JEP >> documentation. Manually patching the launch script to filesize=20M works. >> >> Indeed, JEP page (http://openjdk.java.net/jeps/158) says: >> >> output-option := filecount= >> filesize= >> parameter=value >> >> ...while the parsing code does it via the usual Arguments::atojulong, which would treat non-suffix >> space as bytes: >> >> logFileOutput.cpp: >> >> } else if (strcmp(FileSizeOptionKey, key) == 0) { >> julong value; >> success = Arguments::atojulong(value_str, &value); >> >> >> May I drop "in kb" from the JEP text? Asking here, because all people responsible for that JEP >> apparently left. I think I have enough permissions to edit the underlying bug >> (https://bugs.openjdk.java.net/browse/JDK-8046148), so that changes propagate automatically, but I >> would like someone to confirm this change is the right thing to do. >> >> -Aleksey >> From shade at redhat.com Wed Oct 17 15:47:39 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 17:47:39 +0200 Subject: RFR (XS) 8212609: Minimal VM build failure after JDK-8210498 (nmethod entry barriers) Message-ID: <4ccd5e16-b92e-ab32-e42f-3e12a36ea578@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8212609 Fix: diff -r cba34f27d9ce src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp --- a/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Wed Oct 17 22:06:55 2018 +0800 +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Wed Oct 17 17:42:14 2018 +0200 @@ -34,6 +34,8 @@ #include "code/vtableStubs.hpp" #include "gc/shared/collectedHeap.hpp" #include "gc/shared/gcLocker.hpp" +#include "gc/shared/barrierSet.hpp" +#include "gc/shared/barrierSetAssembler.hpp" #include "interpreter/interpreter.hpp" #include "logging/log.hpp" #include "memory/resourceArea.hpp" I contemplated ifdef-ing the actual call to BSAsm here: BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); bs->nmethod_entry_barrier(masm); ...but thought it is cleaner to let barrier BSAsm figure out whatever compiler-specific thing needs to be done. Testing: x86_64-minimal builds Thanks, -Aleksey From erik.osterlund at oracle.com Wed Oct 17 16:08:58 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 17 Oct 2018 18:08:58 +0200 Subject: RFR (XS) 8212609: Minimal VM build failure after JDK-8210498 (nmethod entry barriers) In-Reply-To: <4ccd5e16-b92e-ab32-e42f-3e12a36ea578@redhat.com> References: <4ccd5e16-b92e-ab32-e42f-3e12a36ea578@redhat.com> Message-ID: <6B52D0D5-6544-4B24-81EC-550FE89D580A@oracle.com> Hi Aleksey, Looks good. Thanks for fixing. /Erik > On 17 Oct 2018, at 17:47, Aleksey Shipilev wrote: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212609 > > Fix: > > diff -r cba34f27d9ce src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp > --- a/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Wed Oct 17 22:06:55 2018 +0800 > +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Wed Oct 17 17:42:14 2018 +0200 > @@ -34,6 +34,8 @@ > #include "code/vtableStubs.hpp" > #include "gc/shared/collectedHeap.hpp" > #include "gc/shared/gcLocker.hpp" > +#include "gc/shared/barrierSet.hpp" > +#include "gc/shared/barrierSetAssembler.hpp" > #include "interpreter/interpreter.hpp" > #include "logging/log.hpp" > #include "memory/resourceArea.hpp" > > > I contemplated ifdef-ing the actual call to BSAsm here: > > BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); > bs->nmethod_entry_barrier(masm); > > ...but thought it is cleaner to let barrier BSAsm figure out whatever compiler-specific thing needs > to be done. > > Testing: x86_64-minimal builds > > Thanks, > -Aleksey > From shade at redhat.com Wed Oct 17 16:26:36 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 18:26:36 +0200 Subject: RFR (XS) 8212609: Minimal VM build failure after JDK-8210498 (nmethod entry barriers) In-Reply-To: <6B52D0D5-6544-4B24-81EC-550FE89D580A@oracle.com> References: <4ccd5e16-b92e-ab32-e42f-3e12a36ea578@redhat.com> <6B52D0D5-6544-4B24-81EC-550FE89D580A@oracle.com> Message-ID: <13fb1755-f2a0-1ab8-31e3-11a427fa7948@redhat.com> I'd like to treat this fix as trivial, and push it without further testing. Okay with everyone? -Aleksey On 10/17/2018 06:08 PM, Erik Osterlund wrote: > Hi Aleksey, > > Looks good. Thanks for fixing. > > /Erik > >> On 17 Oct 2018, at 17:47, Aleksey Shipilev wrote: >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212609 >> >> Fix: >> >> diff -r cba34f27d9ce src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp >> --- a/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Wed Oct 17 22:06:55 2018 +0800 >> +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Wed Oct 17 17:42:14 2018 +0200 >> @@ -34,6 +34,8 @@ >> #include "code/vtableStubs.hpp" >> #include "gc/shared/collectedHeap.hpp" >> #include "gc/shared/gcLocker.hpp" >> +#include "gc/shared/barrierSet.hpp" >> +#include "gc/shared/barrierSetAssembler.hpp" >> #include "interpreter/interpreter.hpp" >> #include "logging/log.hpp" >> #include "memory/resourceArea.hpp" >> >> >> I contemplated ifdef-ing the actual call to BSAsm here: >> >> BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); >> bs->nmethod_entry_barrier(masm); >> >> ...but thought it is cleaner to let barrier BSAsm figure out whatever compiler-specific thing needs >> to be done. >> >> Testing: x86_64-minimal builds >> >> Thanks, >> -Aleksey >> > From erik.osterlund at oracle.com Wed Oct 17 16:27:41 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 17 Oct 2018 18:27:41 +0200 Subject: RFR (XS) 8212609: Minimal VM build failure after JDK-8210498 (nmethod entry barriers) In-Reply-To: <13fb1755-f2a0-1ab8-31e3-11a427fa7948@redhat.com> References: <4ccd5e16-b92e-ab32-e42f-3e12a36ea578@redhat.com> <6B52D0D5-6544-4B24-81EC-550FE89D580A@oracle.com> <13fb1755-f2a0-1ab8-31e3-11a427fa7948@redhat.com> Message-ID: Hi Alexey, Ship it. Thanks, /Erik > On 17 Oct 2018, at 18:26, Aleksey Shipilev wrote: > > I'd like to treat this fix as trivial, and push it without further testing. Okay with everyone? > > -Aleksey > >> On 10/17/2018 06:08 PM, Erik Osterlund wrote: >> Hi Aleksey, >> >> Looks good. Thanks for fixing. >> >> /Erik >> >>> On 17 Oct 2018, at 17:47, Aleksey Shipilev wrote: >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212609 >>> >>> Fix: >>> >>> diff -r cba34f27d9ce src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp >>> --- a/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Wed Oct 17 22:06:55 2018 +0800 >>> +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Wed Oct 17 17:42:14 2018 +0200 >>> @@ -34,6 +34,8 @@ >>> #include "code/vtableStubs.hpp" >>> #include "gc/shared/collectedHeap.hpp" >>> #include "gc/shared/gcLocker.hpp" >>> +#include "gc/shared/barrierSet.hpp" >>> +#include "gc/shared/barrierSetAssembler.hpp" >>> #include "interpreter/interpreter.hpp" >>> #include "logging/log.hpp" >>> #include "memory/resourceArea.hpp" >>> >>> >>> I contemplated ifdef-ing the actual call to BSAsm here: >>> >>> BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); >>> bs->nmethod_entry_barrier(masm); >>> >>> ...but thought it is cleaner to let barrier BSAsm figure out whatever compiler-specific thing needs >>> to be done. >>> >>> Testing: x86_64-minimal builds >>> >>> Thanks, >>> -Aleksey >>> >> > > From ioi.lam at oracle.com Wed Oct 17 16:33:12 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 17 Oct 2018 09:33:12 -0700 Subject: RFR(XS) 8212612 - Add documentation about Arguments::_exit_hook Message-ID: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> http://cr.openjdk.java.net/~iklam/jdk12/8212612-exit-hook-docs.v01/ https://bugs.openjdk.java.net/browse/JDK-8212612 The following fields are assigned to according to JavaVMOption.optionString. ? abort_hook_t Arguments::_abort_hook = NULL; ? exit_hook_t Arguments::_exit_hook = NULL; ? vfprintf_hook_t Arguments::_vfprintf_hook = NULL; However, there's no code in the JDK repo that uses these options. Instead, they are intended for programs that embed the JVM using JNI_CreateJavaVM. We should add comments in arguments.cpp for clarification. Otherwise this would look like dead code and someone might be tempted to remove it. Thanks - Ioi From harold.seigel at oracle.com Wed Oct 17 17:10:54 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Wed, 17 Oct 2018 13:10:54 -0400 Subject: RFR(XS) 8212612 - Add documentation about Arguments::_exit_hook In-Reply-To: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> References: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> Message-ID: <98ff588a-34cf-a628-4717-98d658890250@oracle.com> Hi Ioi, Looks good!? Thanks for doing this. Harold On 10/17/2018 12:33 PM, Ioi Lam wrote: > http://cr.openjdk.java.net/~iklam/jdk12/8212612-exit-hook-docs.v01/ > https://bugs.openjdk.java.net/browse/JDK-8212612 > > The following fields are assigned to according to > JavaVMOption.optionString. > > ? abort_hook_t Arguments::_abort_hook = NULL; > ? exit_hook_t Arguments::_exit_hook = NULL; > ? vfprintf_hook_t Arguments::_vfprintf_hook = NULL; > > However, there's no code in the JDK repo that uses these options. > Instead, they are intended for programs that embed the JVM using > JNI_CreateJavaVM. > > We should add comments in arguments.cpp for clarification. Otherwise > this would look like dead code and someone might be tempted to > remove it. > > Thanks > - Ioi From shade at redhat.com Wed Oct 17 17:14:22 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 19:14:22 +0200 Subject: RFR (XS) 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8212616 Fix is to do what nmethod entry barriers patch did for x86_64.ad: # HG changeset patch # User shade # Date 1539796271 -7200 # Wed Oct 17 19:11:11 2018 +0200 # Node ID 92a18a1292533c334021aef21a66c5c9bc1b663c # Parent 199658d1ef860cdc17055b4fd3e94b057f292fe9 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) Reviewed-by: XXX diff -r 199658d1ef86 -r 92a18a129253 src/hotspot/cpu/x86/x86_32.ad --- a/src/hotspot/cpu/x86/x86_32.ad Wed Oct 17 18:31:48 2018 +0200 +++ b/src/hotspot/cpu/x86/x86_32.ad Wed Oct 17 19:11:11 2018 +0200 @@ -619,7 +619,7 @@ int framesize = C->frame_size_in_bytes(); int bangsize = C->bang_size_in_bytes(); - __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode()); + __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode(), C->stub_function() != NULL); C->set_frame_complete(cbuf.insts_size()); Testing: {x86_32, x86_64} build Thanks, -Aleksey From vladimir.kozlov at oracle.com Wed Oct 17 17:26:51 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 17 Oct 2018 10:26:51 -0700 Subject: RFR (XS) 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) In-Reply-To: References: Message-ID: <44f36dea-51ff-955c-60d0-7aafc8b38797@oracle.com> Looks good. Thanks, Vladimir On 10/17/18 10:14 AM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212616 > > Fix is to do what nmethod entry barriers patch did for x86_64.ad: > > # HG changeset patch > # User shade > # Date 1539796271 -7200 > # Wed Oct 17 19:11:11 2018 +0200 > # Node ID 92a18a1292533c334021aef21a66c5c9bc1b663c > # Parent 199658d1ef860cdc17055b4fd3e94b057f292fe9 > 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) > Reviewed-by: XXX > > diff -r 199658d1ef86 -r 92a18a129253 src/hotspot/cpu/x86/x86_32.ad > --- a/src/hotspot/cpu/x86/x86_32.ad Wed Oct 17 18:31:48 2018 +0200 > +++ b/src/hotspot/cpu/x86/x86_32.ad Wed Oct 17 19:11:11 2018 +0200 > @@ -619,7 +619,7 @@ > int framesize = C->frame_size_in_bytes(); > int bangsize = C->bang_size_in_bytes(); > > - __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode()); > + __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode(), > C->stub_function() != NULL); > > C->set_frame_complete(cbuf.insts_size()); > > Testing: {x86_32, x86_64} build > > Thanks, > -Aleksey > From igor.ignatyev at oracle.com Wed Oct 17 17:54:51 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 10:54:51 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest Message-ID: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > 141 lines changed: 140 ins; 1 del; 0 mod; Hi all, could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 testing: - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants - build w/ precompiled-headers enabled and disabled Thanks, -- Igor From igor.ignatyev at oracle.com Wed Oct 17 17:55:05 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 10:55:05 -0700 Subject: RFR(XS) : 8171097 : Convert TestReservedSpace_test to Gtest In-Reply-To: References: Message-ID: Hi JC, thanks for your review! Cheers, -- Igor > On Oct 16, 2018, at 12:51 PM, JC Beyler wrote: > > Hi Igor, > > The change looks good to me, I compared old/new, the port looks good. > > For the reviewers, here is the original test code: > http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1071 > > Thanks, > Jc > > On Tue, Oct 16, 2018 at 9:06 AM Igor Ignatyev > wrote: > http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > > 188 lines changed: 187 ins; 1 del; 0 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which converts internal TestReservedSpace_test to gtest? > since the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. > > webrev: http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8171097 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review > > Thanks, > -- Igor > > > -- > > Thanks, > Jc From thomas.stuefe at gmail.com Wed Oct 17 18:16:55 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 17 Oct 2018 20:16:55 +0200 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> Message-ID: Hi Igor, test_virtual_space_actual_committed_space() and actual_committed_space_one_large_page(): Won't we leak the reserved space now when we run into an ASSERT and then continue with the next gtests? Cheers, Thomas On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > > 141 lines changed: 140 ins; 1 del; 0 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? > again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. > > for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. > > webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > Thanks, > -- Igor From john.r.rose at oracle.com Wed Oct 17 18:23:54 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 17 Oct 2018 11:23:54 -0700 Subject: RFC: "in kb" typo in UL JEP? In-Reply-To: <4733772b-1880-dba7-8135-c2d236168f35@redhat.com> References: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> <4733772b-1880-dba7-8135-c2d236168f35@redhat.com> Message-ID: <4320F772-93AE-40CF-A96B-FC75797CC8CD@oracle.com> On Oct 17, 2018, at 8:43 AM, Aleksey Shipilev wrote: > > Okay, anyone from Oracle wants to confirm? Changing the text in 5... 4... I support this change, because I agree that "in kb" is nothing more than a shallow editing error. As JC points out, so is "filesize=1024", which is missing a "k". From igor.ignatyev at oracle.com Wed Oct 17 18:29:11 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 11:29:11 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> Message-ID: <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> Hi Thomas, 1st of all, thanks for looking at it, appreciate that. it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? -- Igor > On Oct 17, 2018, at 11:16 AM, Thomas St?fe wrote: > > Hi Igor, > > test_virtual_space_actual_committed_space() and > actual_committed_space_one_large_page(): > > Won't we leak the reserved space now when we run into an ASSERT and > then continue with the next gtests? > > Cheers, Thomas > > On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >>> 141 lines changed: 140 ins; 1 del; 0 mod; >> >> Hi all, >> >> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? >> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. >> >> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 >> testing: >> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >> - build w/ precompiled-headers enabled and disabled >> >> Thanks, >> -- Igor From thomas.stuefe at gmail.com Wed Oct 17 18:53:37 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 17 Oct 2018 20:53:37 +0200 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> Message-ID: On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev wrote: > > Hi Thomas, > > 1st of all, thanks for looking at it, appreciate that. > > it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? > > -- Igor Slight preference for SetUp/TearDown (EXCEPT would quit the testing altogether, wont it?). Lazy me would have just used an RAII object like this: struct Cleaner { ReservedSpace* rs; ~Cleaner() { rs.release(); } }; But I leave it up to you. ... Thomas > > > On Oct 17, 2018, at 11:16 AM, Thomas St?fe wrote: > > > > Hi Igor, > > > > test_virtual_space_actual_committed_space() and > > actual_committed_space_one_large_page(): > > > > Won't we leak the reserved space now when we run into an ASSERT and > > then continue with the next gtests? > > > > Cheers, Thomas > > > > On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: > >> > >> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>> 141 lines changed: 140 ins; 1 del; 0 mod; > >> > >> Hi all, > >> > >> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? > >> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. > >> > >> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. > >> > >> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > >> testing: > >> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > >> - build w/ precompiled-headers enabled and disabled > >> > >> Thanks, > >> -- Igor > From jcbeyler at google.com Wed Oct 17 18:58:54 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 17 Oct 2018 11:58:54 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> Message-ID: Hi Igor, Looks good to me as well, I agree with Thomas' comments and also prefer RAII, especially since SetUp/TearDown here is going to be weird since you don't always create the object the same way. If you make a good RAII object, then both tests could use the same kind of code instead of one using a helper method and another creating the ReservedSpace directly too. Thanks! Jc On Wed, Oct 17, 2018 at 11:54 AM Thomas St?fe wrote: > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev > wrote: > > > > Hi Thomas, > > > > 1st of all, thanks for looking at it, appreciate that. > > > > it seems we can. there are two ways I can fix that either replace ASSERT > by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you > prefer? > > > > -- Igor > > Slight preference for SetUp/TearDown (EXCEPT would quit the testing > altogether, wont it?). > > Lazy me would have just used an RAII object like this: > > struct Cleaner { > ReservedSpace* rs; > ~Cleaner() { rs.release(); } > }; > > But I leave it up to you. > > ... Thomas > > > > > > On Oct 17, 2018, at 11:16 AM, Thomas St?fe > wrote: > > > > > > Hi Igor, > > > > > > test_virtual_space_actual_committed_space() and > > > actual_committed_space_one_large_page(): > > > > > > Won't we leak the reserved space now when we run into an ASSERT and > > > then continue with the next gtests? > > > > > > Cheers, Thomas > > > > > > On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev < > igor.ignatyev at oracle.com> wrote: > > >> > > >> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > > >>> 141 lines changed: 140 ins; 1 del; 0 mod; > > >> > > >> Hi all, > > >> > > >> could you please review this small (and hopefully trivial) patch > which converts internal TestVirtualSpace_test to gtest? > > >> again, the old test is still used by WhiteBox::runMemoryUnitTests, > the old code hasn't been removed. it will be removed later when all 4 tests > used by runMemoryUnitTests are converted. > > >> > > >> for the sake of reviewer, > http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 > is the old version. > > >> > > >> webrev: > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > > >> testing: > > >> - converted tests on linux-x64, windows-x64, macosx-x64, > solaris-sparcv9 in product and fastdebug variants > > >> - build w/ precompiled-headers enabled and disabled > > >> > > >> Thanks, > > >> -- Igor > > > -- Thanks, Jc From igor.ignatyev at oracle.com Wed Oct 17 19:00:43 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 12:00:43 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> Message-ID: <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> EXPECT would just mark a test as failed and allow the method to continue, so rs.release() will be called. test fixture will basically do the same as an RAII object, but it will be a bit more obvious as test preparation and cleaning steps will be separated from test actions, so I'll go w/ setUp/tearDown. -- Igor > On Oct 17, 2018, at 11:53 AM, Thomas St?fe wrote: > > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev wrote: >> >> Hi Thomas, >> >> 1st of all, thanks for looking at it, appreciate that. >> >> it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? >> >> -- Igor > > Slight preference for SetUp/TearDown (EXCEPT would quit the testing > altogether, wont it?). > > Lazy me would have just used an RAII object like this: > > struct Cleaner { > ReservedSpace* rs; > ~Cleaner() { rs.release(); } > }; > > But I leave it up to you. > > ... Thomas > >> >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe wrote: >>> >>> Hi Igor, >>> >>> test_virtual_space_actual_committed_space() and >>> actual_committed_space_one_large_page(): >>> >>> Won't we leak the reserved space now when we run into an ASSERT and >>> then continue with the next gtests? >>> >>> Cheers, Thomas >>> >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: >>>> >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; >>>> >>>> Hi all, >>>> >>>> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. >>>> >>>> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. >>>> >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 >>>> testing: >>>> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >>>> - build w/ precompiled-headers enabled and disabled >>>> >>>> Thanks, >>>> -- Igor >> From thomas.stuefe at gmail.com Wed Oct 17 19:04:07 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 17 Oct 2018 21:04:07 +0200 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> Message-ID: On Wed 17. Oct 2018 at 21:00, Igor Ignatyev wrote: > EXPECT would just mark a test as failed and allow the method to continue, > so rs.release() will be called. Oh, okay. Thanks for clarifying. > > test fixture will basically do the same as an RAII object, but it will be > a bit more obvious as test preparation and cleaning steps will be separated > from test actions, so I'll go w/ setUp/tearDown. Okay fine by me. ..thomas > > -- Igor > > > On Oct 17, 2018, at 11:53 AM, Thomas St?fe > wrote: > > > > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev > wrote: > >> > >> Hi Thomas, > >> > >> 1st of all, thanks for looking at it, appreciate that. > >> > >> it seems we can. there are two ways I can fix that either replace > ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which > do you prefer? > >> > >> -- Igor > > > > Slight preference for SetUp/TearDown (EXCEPT would quit the testing > > altogether, wont it?). > > > > Lazy me would have just used an RAII object like this: > > > > struct Cleaner { > > ReservedSpace* rs; > > ~Cleaner() { rs.release(); } > > }; > > > > But I leave it up to you. > > > > ... Thomas > > > >> > >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe > wrote: > >>> > >>> Hi Igor, > >>> > >>> test_virtual_space_actual_committed_space() and > >>> actual_committed_space_one_large_page(): > >>> > >>> Won't we leak the reserved space now when we run into an ASSERT and > >>> then continue with the next gtests? > >>> > >>> Cheers, Thomas > >>> > >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev < > igor.ignatyev at oracle.com> wrote: > >>>> > >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; > >>>> > >>>> Hi all, > >>>> > >>>> could you please review this small (and hopefully trivial) patch > which converts internal TestVirtualSpace_test to gtest? > >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, > the old code hasn't been removed. it will be removed later when all 4 tests > used by runMemoryUnitTests are converted. > >>>> > >>>> for the sake of reviewer, > http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 > is the old version. > >>>> > >>>> webrev: > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > >>>> testing: > >>>> - converted tests on linux-x64, windows-x64, macosx-x64, > solaris-sparcv9 in product and fastdebug variants > >>>> - build w/ precompiled-headers enabled and disabled > >>>> > >>>> Thanks, > >>>> -- Igor > >> > > From shade at redhat.com Wed Oct 17 19:49:22 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 21:49:22 +0200 Subject: RFC: "in kb" typo in UL JEP? In-Reply-To: <4320F772-93AE-40CF-A96B-FC75797CC8CD@oracle.com> References: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> <4733772b-1880-dba7-8135-c2d236168f35@redhat.com> <4320F772-93AE-40CF-A96B-FC75797CC8CD@oracle.com> Message-ID: On 10/17/2018 08:23 PM, John Rose wrote: > On Oct 17, 2018, at 8:43 AM, Aleksey Shipilev > wrote: >> >> Okay, anyone from Oracle wants to confirm? Changing the text in 5... 4... > > I support this change, because I agree that "in kb" is nothing more > than a shallow editing error. ?As JC points out, so is "filesize=1024", > which is missing a "k". Better yet, I changed it to "filesize=1M". And dropped "(in kb)" from grammar. -Aleksey From shade at redhat.com Wed Oct 17 19:50:47 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 21:50:47 +0200 Subject: RFR (XS) 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) In-Reply-To: <44f36dea-51ff-955c-60d0-7aafc8b38797@oracle.com> References: <44f36dea-51ff-955c-60d0-7aafc8b38797@oracle.com> Message-ID: Thanks Vladimir. Is this trivial? Or, other Reviewers, please? -Aleksey On 10/17/2018 07:26 PM, Vladimir Kozlov wrote: > Looks good. > > Thanks, > Vladimir > > On 10/17/18 10:14 AM, Aleksey Shipilev wrote: >> Bug: >> ?? https://bugs.openjdk.java.net/browse/JDK-8212616 >> >> Fix is to do what nmethod entry barriers patch did for x86_64.ad: >> >> # HG changeset patch >> # User shade >> # Date 1539796271 -7200 >> #????? Wed Oct 17 19:11:11 2018 +0200 >> # Node ID 92a18a1292533c334021aef21a66c5c9bc1b663c >> # Parent? 199658d1ef860cdc17055b4fd3e94b057f292fe9 >> 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) >> Reviewed-by: XXX >> >> diff -r 199658d1ef86 -r 92a18a129253 src/hotspot/cpu/x86/x86_32.ad >> --- a/src/hotspot/cpu/x86/x86_32.ad???? Wed Oct 17 18:31:48 2018 +0200 >> +++ b/src/hotspot/cpu/x86/x86_32.ad???? Wed Oct 17 19:11:11 2018 +0200 >> @@ -619,7 +619,7 @@ >> ??? int framesize = C->frame_size_in_bytes(); >> ??? int bangsize = C->bang_size_in_bytes(); >> >> -? __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode()); >> +? __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode(), >> C->stub_function() != NULL); >> >> ??? C->set_frame_complete(cbuf.insts_size()); >> >> Testing: {x86_32, x86_64} build >> >> Thanks, >> -Aleksey >> From john.r.rose at oracle.com Wed Oct 17 20:19:48 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 17 Oct 2018 13:19:48 -0700 Subject: RFC: "in kb" typo in UL JEP? In-Reply-To: References: <429ccc3e-9363-f369-a946-65aeb1ff2082@redhat.com> <4733772b-1880-dba7-8135-c2d236168f35@redhat.com> <4320F772-93AE-40CF-A96B-FC75797CC8CD@oracle.com> Message-ID: <148C0EFE-D434-4160-A625-23EA5B1EB6C8@oracle.com> On Oct 17, 2018, at 12:49 PM, Aleksey Shipilev wrote: > >> I support this change, because I agree that "in kb" is nothing more >> than a shallow editing error. As JC points out, so is "filesize=1024", >> which is missing a "k". > > Better yet, I changed it to "filesize=1M". And dropped "(in kb)" from grammar. editorial discretion FTW From erik.osterlund at oracle.com Wed Oct 17 20:20:02 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 17 Oct 2018 22:20:02 +0200 Subject: RFR (XS) 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) In-Reply-To: References: <44f36dea-51ff-955c-60d0-7aafc8b38797@oracle.com> Message-ID: Hi Aleksey, Looks great. Thanks, /Erik On 2018-10-17 21:50, Aleksey Shipilev wrote: > Thanks Vladimir. Is this trivial? Or, other Reviewers, please? > > -Aleksey > > On 10/17/2018 07:26 PM, Vladimir Kozlov wrote: >> Looks good. >> >> Thanks, >> Vladimir >> >> On 10/17/18 10:14 AM, Aleksey Shipilev wrote: >>> Bug: >>> ?? https://bugs.openjdk.java.net/browse/JDK-8212616 >>> >>> Fix is to do what nmethod entry barriers patch did for x86_64.ad: >>> >>> # HG changeset patch >>> # User shade >>> # Date 1539796271 -7200 >>> #????? Wed Oct 17 19:11:11 2018 +0200 >>> # Node ID 92a18a1292533c334021aef21a66c5c9bc1b663c >>> # Parent? 199658d1ef860cdc17055b4fd3e94b057f292fe9 >>> 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) >>> Reviewed-by: XXX >>> >>> diff -r 199658d1ef86 -r 92a18a129253 src/hotspot/cpu/x86/x86_32.ad >>> --- a/src/hotspot/cpu/x86/x86_32.ad???? Wed Oct 17 18:31:48 2018 +0200 >>> +++ b/src/hotspot/cpu/x86/x86_32.ad???? Wed Oct 17 19:11:11 2018 +0200 >>> @@ -619,7 +619,7 @@ >>> ??? int framesize = C->frame_size_in_bytes(); >>> ??? int bangsize = C->bang_size_in_bytes(); >>> >>> -? __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode()); >>> +? __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode(), >>> C->stub_function() != NULL); >>> >>> ??? C->set_frame_complete(cbuf.insts_size()); >>> >>> Testing: {x86_32, x86_64} build >>> >>> Thanks, >>> -Aleksey >>> > From mandy.chung at oracle.com Wed Oct 17 20:41:55 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Oct 2018 13:41:55 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: Message-ID: On 10/16/18 7:33 PM, David Holmes wrote: > Hi Dean, > > Thanks for tackling this. > > I'm still struggling to fully grasp why we need both the PerfCounters > and the regular counters. I get that we have to decrement the live > counts before ensure_join() has allowed Thread.join() to return, to > ensure that if we then check the number of threads it has dropped by > one. But I don't understand why that means we need to manage the > thread count in two parts. Particularly as now you don't use the > PerfCounter to return the live count, so it makes me wonder what role > the PerfCounter is playing as it is temporarily inconsistent with the > reported live count? Perf counters were added long time back in JDK 1.4.2 for performance measurement before java.lang.management API.? One can use jstat tool to monitor VM perf counters of a running VM.?? One could look into the possibility of deprecating these counters and remove them over time. > On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: > New webrev: > > http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ When the perf counters are updated when a thread is added/removed, it's holding Threads_lock.? Are the asserts in ThreadService::remove_thread necessary? For clarify, I think we could simply set _live_threads_count to the value of _atomic_threads_count and set _daemon_threads_count to the value of _atomic_daemon_threads_count. Mandy From aleksei.voitylov at bell-sw.com Wed Oct 17 20:50:30 2018 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Wed, 17 Oct 2018 23:50:30 +0300 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <3690d846-5f4d-cf7d-5e43-51a1915fd36e@bell-sw.com> <99659828-1665-4B62-9E24-D6BD4EE96C98@oracle.com> <10d8f2ea-883e-ec21-4fea-06a52a52fdc6@bell-sw.com> <407CE450-6EED-4955-9B1F-EA564E134D11@oracle.com> Message-ID: <32419fac-84fe-c247-b91d-e8a5ead67cf2@bell-sw.com> On 16/10/2018 02:25, Kim Barrett wrote: >> On Oct 15, 2018, at 7:51 AM, Aleksei Voitylov wrote: >> >> Kim, >> >> If you were suggesting to just proceed with the change without giving a sufficient lead time for the ports that were willing to upgrade to do so, that sounds very alarming. > What is "sufficient lead time"? I'm not proposing an answer, just > suggesting that 3 months (approx time between JEP publication and JDK > 12 fork) might be sufficient, and a worthy goal. That's a decision > that ought to be made as part of the Targeting discussion for this > JEP. Right now, it's not even a Candidate, since there's still work > to be done. I'm fine with that, and you probably can assume to document this goal in the JEP if noone speaks up. > >> Meanwhile, our testing has finished and I'm now confident we will be able to switch ARM port over to 7.x in 12 time frame. There are several issues that we still need to diagnose, but this does not look like a blocker. > That's good news, though pretty much what I expected. > > I'm more worried about Solaris. Having a goal of JDK 12 may help move > things along. > From shade at redhat.com Wed Oct 17 20:54:46 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 22:54:46 +0200 Subject: RFR (XS) 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) In-Reply-To: References: <44f36dea-51ff-955c-60d0-7aafc8b38797@oracle.com> Message-ID: <0dcec0f3-0919-fa0e-43bc-ae7224e6c221@redhat.com> Thanks Erik, pushed. -Aleksey On 10/17/2018 10:20 PM, Erik ?sterlund wrote: > Looks great. > > Thanks, > /Erik > > On 2018-10-17 21:50, Aleksey Shipilev wrote: >> Thanks Vladimir. Is this trivial? Or, other Reviewers, please? >> >> -Aleksey >> >> On 10/17/2018 07:26 PM, Vladimir Kozlov wrote: >>> Looks good. >>> >>> Thanks, >>> Vladimir >>> >>> On 10/17/18 10:14 AM, Aleksey Shipilev wrote: >>>> Bug: >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8212616 >>>> >>>> Fix is to do what nmethod entry barriers patch did for x86_64.ad: >>>> >>>> # HG changeset patch >>>> # User shade >>>> # Date 1539796271 -7200 >>>> #????? Wed Oct 17 19:11:11 2018 +0200 >>>> # Node ID 92a18a1292533c334021aef21a66c5c9bc1b663c >>>> # Parent? 199658d1ef860cdc17055b4fd3e94b057f292fe9 >>>> 8212616: x86_32 build failures after JDK-8210498 (nmethod entry barriers) >>>> Reviewed-by: XXX >>>> >>>> diff -r 199658d1ef86 -r 92a18a129253 src/hotspot/cpu/x86/x86_32.ad >>>> --- a/src/hotspot/cpu/x86/x86_32.ad???? Wed Oct 17 18:31:48 2018 +0200 >>>> +++ b/src/hotspot/cpu/x86/x86_32.ad???? Wed Oct 17 19:11:11 2018 +0200 >>>> @@ -619,7 +619,7 @@ >>>> ???? int framesize = C->frame_size_in_bytes(); >>>> ???? int bangsize = C->bang_size_in_bytes(); >>>> >>>> -? __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode()); >>>> +? __ verified_entry(framesize, C->need_stack_bang(bangsize)?bangsize:0, C->in_24_bit_fp_mode(), >>>> C->stub_function() != NULL); >>>> >>>> ???? C->set_frame_complete(cbuf.insts_size()); >>>> >>>> Testing: {x86_32, x86_64} build >>>> >>>> Thanks, >>>> -Aleksey >>>> >> > From dean.long at oracle.com Wed Oct 17 21:10:48 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 14:10:48 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: Message-ID: <81de4bbe-2461-e969-7b82-99a7854bb990@oracle.com> On 10/16/18 7:33 PM, David Holmes wrote: > Hi Dean, > > Thanks for tackling this. > > I'm still struggling to fully grasp why we need both the PerfCounters > and the regular counters. I get that we have to decrement the live > counts before ensure_join() has allowed Thread.join() to return, to > ensure that if we then check the number of threads it has dropped by > one. But I don't understand why that means we need to manage the > thread count in two parts. Particularly as now you don't use the > PerfCounter to return the live count, so it makes me wonder what role > the PerfCounter is playing as it is temporarily inconsistent with the > reported live count? Is it simply that we can't atomically decrement > the PerfCounters, and we can't require the Threads_lock when > decrementing? > Good questions.? I didn't know the history, so I tried not to change too much.? The PerfCounters appear to be there to support StatSampler.? I think it's OK for them to be a little out of sync. If we wanted to make them more in sync with the atomic counters, I think we could do any of the following: - Grab Threads_lock before SR_lock() where we call current_thread_exiting, and update perf counts at that time - instead of decrementing in remove_thread, do this in decrement_thread_counts ? _live_threads_count->set_value(_atomic_threads_count); ? _daemon_threads_count->set_value(_atomic_daemon_threads_count); ( I see that Mandy has suggested the same thing.) - DO update the perf counts atomically However, I consider the PerfCounters inconsistency a separate issue from this bug. > On the code itself one thing I find irksome is that we already have a > daemon flag in remove_thread but decrement_thread_counts doesn't get > passed that and so always re-examines the thread to see if it is a > daemon. I know only one of the code paths has the daemon flag, so > perhaps we should hoist the daemon check up into JavaThread::exit and > then pass a daemon parameter to decrement_thread_counts. > OK, I've fixed this. > There's also some ambiguity (existing) in relation to the existence of > the jt->threadObj() and its affect on whether the thread is considered > a daemon or not. Your check has to mirror the existing checks - the > threadObj may be NULL at removal if it was a failed attach, but the > addition considers a JNI-attached thread to be non-daemon. Overall > this seems buggy to me but as long as your check follows the same > rules that is okay. In that regard JavaThread::exit should never > encounter a NULL threadObj(). > I refactored is_daemon checks into a single helper function. > Minor nit: I suggest moving the two occurrences of the comment > > // Do not count VM internal or JVMTI agent threads > > inside is_hidden_thread so that we don't have to remember to update > the comments if the definition changes. > OK.? New webrev: http://cr.openjdk.java.net/~dlong/8021335/webrev.5/ dl > Thanks, > David > ----- > > On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: >> New webrev: >> >> http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ >> >> dl >> >> On 10/16/18 11:59 AM, dean.long at oracle.com wrote: >>> On 10/16/18 10:28 AM, JC Beyler wrote: >>>> Hi Dean, >>>> >>>> I noticed a typo here : >>>> "are atomically" is in the comment but should no longer be there I >>>> believe; the sentence probably should be: >>>> // These 2 counters are like the above thread counts, but are >>>> >>> >>> Fixed! >>> >>>> Any way we could move this test into a helper method and both >>>> add_thread/current_thread_exiting could use the same "ignore" >>>> thread code so that we ensure the two never get out of sync? >>>> >>>> +? if (jt->is_hidden_from_external_view() || >>>> +? ? ? jt->is_jvmti_agent_thread()) { >>>> +? ? return; >>>> +? } >>>> >>> >>> Good idea.? Fixed. >>> >>>> (Also by curiosity, add_thread has an assert on the Threads_lock >>>> but not thread_exiting?) >>>> >>> >>> Right, I followed the existing pattern where >>> current_thread_exiting() only uses the atomic counters, so it >>> doesn't need Threads_lock. >>> >>> dl >>> >>>> Thanks, >>>> Jc >>>> >>>> >>>> On Tue, Oct 16, 2018 at 9:52 AM >>> > wrote: >>>> >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8021335 >>>> ??? http://cr.openjdk.java.net/~dlong/8021335/webrev.3/ >>>> >>>> >>>> ??? This change attempts to fix an old and intermittent problem with >>>> ??? ThreadMXBean thread counts. >>>> ??? Changes have 3 main parts: >>>> ??? 1. Make sure hidden threads don't affect counts (even when >>>> exiting) >>>> ??? 2. Fix race reading counts using atomic counters >>>> ??? 3. Remove some workarounds in test (because they hide bugs) >>>> >>>> ??? dl >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>> >> From dean.long at oracle.com Wed Oct 17 21:13:57 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 14:13:57 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: Message-ID: On 10/17/18 1:41 PM, Mandy Chung wrote: > > > On 10/16/18 7:33 PM, David Holmes wrote: >> Hi Dean, >> >> Thanks for tackling this. >> >> I'm still struggling to fully grasp why we need both the PerfCounters >> and the regular counters. I get that we have to decrement the live >> counts before ensure_join() has allowed Thread.join() to return, to >> ensure that if we then check the number of threads it has dropped by >> one. But I don't understand why that means we need to manage the >> thread count in two parts. Particularly as now you don't use the >> PerfCounter to return the live count, so it makes me wonder what role >> the PerfCounter is playing as it is temporarily inconsistent with the >> reported live count? > > Perf counters were added long time back in JDK 1.4.2 for performance > measurement before java.lang.management API.? One can use jstat tool > to monitor VM perf counters of a running VM.?? One could look into the > possibility of deprecating these counters and remove them over time. > >> On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: >> New webrev: >> >> http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ > > When the perf counters are updated when a thread is added/removed, > it's holding Threads_lock.? Are the asserts in > ThreadService::remove_thread necessary? > Not really.? They were intended to catch the case where the atomic counters weren't decremented for some reason, not for the perf counters. Should I remove them? > For clarify, I think we could simply set _live_threads_count to the > value of _atomic_threads_count and set _daemon_threads_count to the > value of _atomic_daemon_threads_count. > I think that works, even inside decrement_thread_counts() without holding the Threads_lock.? If you agree, I'll make that change. dl > Mandy From dean.long at oracle.com Wed Oct 17 21:27:28 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 14:27:28 -0700 Subject: RFR(XS) 8212612 - Add documentation about Arguments::_exit_hook In-Reply-To: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> References: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> Message-ID: <89b3afbf-94ed-4485-1381-724e5cb1d622@oracle.com> On 10/17/18 9:33 AM, Ioi Lam wrote: > http://cr.openjdk.java.net/~iklam/jdk12/8212612-exit-hook-docs.v01/ > https://bugs.openjdk.java.net/browse/JDK-8212612 > > The following fields are assigned to according to > JavaVMOption.optionString. > > ? abort_hook_t Arguments::_abort_hook = NULL; > ? exit_hook_t Arguments::_exit_hook = NULL; > ? vfprintf_hook_t Arguments::_vfprintf_hook = NULL; > > However, there's no code in the JDK repo that uses these options. > Instead, they are intended for programs that embed the JVM using > JNI_CreateJavaVM. > > We should add comments in arguments.cpp for clarification. Otherwise > this would look like dead code and someone might be tempted to > remove it. > > Thanks > - Ioi But it's not dead code.? All three are used.? For example, see jio_vfprintf(). dl From ioi.lam at oracle.com Wed Oct 17 21:35:58 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 17 Oct 2018 14:35:58 -0700 Subject: RFR(XS) 8212612 - Add documentation about Arguments::_exit_hook In-Reply-To: <89b3afbf-94ed-4485-1381-724e5cb1d622@oracle.com> References: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> <89b3afbf-94ed-4485-1381-724e5cb1d622@oracle.com> Message-ID: On 10/17/2018 02:27 PM, dean.long at oracle.com wrote: > On 10/17/18 9:33 AM, Ioi Lam wrote: >> http://cr.openjdk.java.net/~iklam/jdk12/8212612-exit-hook-docs.v01/ >> https://bugs.openjdk.java.net/browse/JDK-8212612 >> >> The following fields are assigned to according to >> JavaVMOption.optionString. >> >> ? abort_hook_t Arguments::_abort_hook = NULL; >> ? exit_hook_t Arguments::_exit_hook = NULL; >> ? vfprintf_hook_t Arguments::_vfprintf_hook = NULL; >> >> However, there's no code in the JDK repo that uses these options. >> Instead, they are intended for programs that embed the JVM using >> JNI_CreateJavaVM. >> >> We should add comments in arguments.cpp for clarification. Otherwise >> this would look like dead code and someone might be tempted to >> remove it. >> >> Thanks >> - Ioi > > But it's not dead code.? All three are used.? For example, see > jio_vfprintf(). > > dl Hi Dean, thanks for looking into this. I was afraid someone would infer that since nowhere in the JDK code would do anything to assign _vfprintf_hook to a non-NULL value, the block ? if (Arguments::vfprintf_hook() != NULL) { ??? jio_fprintf(defaultStream::output_stream(), "%.*s", (int)len, s); ? } would be effectively dead code. Thanks - Ioi From mandy.chung at oracle.com Wed Oct 17 21:38:41 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Oct 2018 14:38:41 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: Message-ID: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> On 10/17/18 2:13 PM, dean.long at oracle.com wrote: > On 10/17/18 1:41 PM, Mandy Chung wrote: >> >> >> On 10/16/18 7:33 PM, David Holmes wrote: >>> Hi Dean, >>> >>> Thanks for tackling this. >>> >>> I'm still struggling to fully grasp why we need both the >>> PerfCounters and the regular counters. I get that we have to >>> decrement the live counts before ensure_join() has allowed >>> Thread.join() to return, to ensure that if we then check the number >>> of threads it has dropped by one. But I don't understand why that >>> means we need to manage the thread count in two parts. Particularly >>> as now you don't use the PerfCounter to return the live count, so it >>> makes me wonder what role the PerfCounter is playing as it is >>> temporarily inconsistent with the reported live count? >> >> Perf counters were added long time back in JDK 1.4.2 for performance >> measurement before java.lang.management API.? One can use jstat tool >> to monitor VM perf counters of a running VM.?? One could look into >> the possibility of deprecating these counters and remove them over time. >> >>> On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: >>> New webrev: >>> >>> http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ >> >> When the perf counters are updated when a thread is added/removed, >> it's holding Threads_lock.? Are the asserts in >> ThreadService::remove_thread necessary? >> > > Not really.? They were intended to catch the case where the atomic > counters weren't decremented for some reason, not for the perf counters. > Should I remove them? > Hmm... when remove_thread is called but decrement_thread_counts has not been called. ? It's a bug in thread accounting.? It happens to have the perf counters that can be compared to assert.? It seems not obvious.? Setting the perf counters same values as _atomic_threads_count and _atomic_daemon_threads_count makes sense to me. I would opt for removing the asserts but I can't think of an alternative how to catch the issue you concern about. >> For clarify, I think we could simply set _live_threads_count to the >> value of _atomic_threads_count and set _daemon_threads_count to the >> value of _atomic_daemon_threads_count. >> > > I think that works, even inside decrement_thread_counts() without > holding the Threads_lock.? If you agree, I'll make that change. > +1 Mandy From mandy.chung at oracle.com Wed Oct 17 21:41:56 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Oct 2018 14:41:56 -0700 Subject: Review Request: JDK-8207146: Rename jdk.internal.misc.Unsafe::xxxObject to xxxReference In-Reply-To: <237430e1-5aea-6c0b-3317-14f12aec72c8@oracle.com> References: <3bed4fd6-1f6f-b95d-1eb9-62396a8e9815@oracle.com> <237430e1-5aea-6c0b-3317-14f12aec72c8@oracle.com> Message-ID: David, thanks for the review. Mandy On 10/16/18 8:33 PM, David Holmes wrote: > Hi Mandy, > > I took a look through all of this and it seems okay - though it was a > little surprising how far the name change needed to spread. > > Thanks, > David > > On 17/10/2018 2:08 AM, Mandy Chung wrote: >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8207146/webrev.00/ >> >> Unsafe::getObject returns a reference to an object. Similarly >> Unsafe::putObject sets a reference in the given base+offset. >> This patch proposes to rename Unsafe xxxObject to xxxReference >> that will make the xxxReference API very clear that these >> are getters/setters for a reference (instance of object class) >> and ready for adding xxxValue in the futurefor values. >> Note that this renaming only applies to jdk.internal.misc.Unsafe >> but not sun.misc.Unsafe.? So no impact to existing libraries >> that depends on sun.misc.Unsafe in jdk.unsupported module. >> >> I ran jdk_core, core_tools, langtools and nashorn tier1-3, >> hotspot_runtime, hotspot_compiler test groups. >> >> thanks >> Mandy From igor.ignatyev at oracle.com Wed Oct 17 21:45:38 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 14:45:38 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> Message-ID: Hi Thomas, JC I tried to use a test fixture and the code became very clumsy, so I went w/ a simple RAII. http://cr.openjdk.java.net/~iignatyev//8177709/webrev.0-1/index.html I've also noticed that we have same problem in TestReservedSpace_test tests -- http://cr.openjdk.java.net/~iignatyev//8171097/webrev.0-1/index.html Thanks, -- Igor > On Oct 17, 2018, at 12:04 PM, Thomas St?fe wrote: > > > > On Wed 17. Oct 2018 at 21:00, Igor Ignatyev > wrote: > EXPECT would just mark a test as failed and allow the method to continue, so rs.release() will be called. > > Oh, okay. Thanks for clarifying. > > > > test fixture will basically do the same as an RAII object, but it will be a bit more obvious as test preparation and cleaning steps will be separated from test actions, so I'll go w/ setUp/tearDown. > > Okay fine by me. > > ..thomas > > > > -- Igor > > > On Oct 17, 2018, at 11:53 AM, Thomas St?fe > wrote: > > > > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev > wrote: > >> > >> Hi Thomas, > >> > >> 1st of all, thanks for looking at it, appreciate that. > >> > >> it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? > >> > >> -- Igor > > > > Slight preference for SetUp/TearDown (EXCEPT would quit the testing > > altogether, wont it?). > > > > Lazy me would have just used an RAII object like this: > > > > struct Cleaner { > > ReservedSpace* rs; > > ~Cleaner() { rs.release(); } > > }; > > > > But I leave it up to you. > > > > ... Thomas > > > >> > >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe > wrote: > >>> > >>> Hi Igor, > >>> > >>> test_virtual_space_actual_committed_space() and > >>> actual_committed_space_one_large_page(): > >>> > >>> Won't we leak the reserved space now when we run into an ASSERT and > >>> then continue with the next gtests? > >>> > >>> Cheers, Thomas > >>> > >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev > wrote: > >>>> > >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; > >>>> > >>>> Hi all, > >>>> > >>>> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? > >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. > >>>> > >>>> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. > >>>> > >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > >>>> testing: > >>>> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > >>>> - build w/ precompiled-headers enabled and disabled > >>>> > >>>> Thanks, > >>>> -- Igor > >> > From dean.long at oracle.com Wed Oct 17 21:59:46 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 14:59:46 -0700 Subject: RFR(XS) 8212612 - Add documentation about Arguments::_exit_hook In-Reply-To: References: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> <89b3afbf-94ed-4485-1381-724e5cb1d622@oracle.com> Message-ID: On 10/17/18 2:35 PM, Ioi Lam wrote: > > > On 10/17/2018 02:27 PM, dean.long at oracle.com wrote: >> On 10/17/18 9:33 AM, Ioi Lam wrote: >>> http://cr.openjdk.java.net/~iklam/jdk12/8212612-exit-hook-docs.v01/ >>> https://bugs.openjdk.java.net/browse/JDK-8212612 >>> >>> The following fields are assigned to according to >>> JavaVMOption.optionString. >>> >>> ? abort_hook_t Arguments::_abort_hook = NULL; >>> ? exit_hook_t Arguments::_exit_hook = NULL; >>> ? vfprintf_hook_t Arguments::_vfprintf_hook = NULL; >>> >>> However, there's no code in the JDK repo that uses these options. >>> Instead, they are intended for programs that embed the JVM using >>> JNI_CreateJavaVM. >>> >>> We should add comments in arguments.cpp for clarification. Otherwise >>> this would look like dead code and someone might be tempted to >>> remove it. >>> >>> Thanks >>> - Ioi >> >> But it's not dead code.? All three are used.? For example, see >> jio_vfprintf(). >> >> dl > > Hi Dean, thanks for looking into this. > > I was afraid someone would infer that since nowhere in the JDK code > would do anything to assign _vfprintf_hook to a non-NULL value, the block > > ? if (Arguments::vfprintf_hook() != NULL) { > ??? jio_fprintf(defaultStream::output_stream(), "%.*s", (int)len, s); > ? } > > would be effectively dead code. > OK, then this comment was confusing: These are not used by the JDK itself I think it's more accurate to say that JDK launchers don't set these, but the JVM does use and honor them if set. dl > Thanks > - Ioi From jcbeyler at google.com Wed Oct 17 22:15:20 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 17 Oct 2018 15:15:20 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> Message-ID: Hi Igor, Both look good to me now :) Thanks! Jc On Wed, Oct 17, 2018 at 2:45 PM Igor Ignatyev wrote: > Hi Thomas, JC > > I tried to use a test fixture and the code became very clumsy, so I went > w/ a simple RAII. > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.0-1/index.html > > I've also noticed that we have same problem in TestReservedSpace_test > tests -- > http://cr.openjdk.java.net/~iignatyev//8171097/webrev.0-1/index.html > > Thanks, > -- Igor > > On Oct 17, 2018, at 12:04 PM, Thomas St?fe > wrote: > > > > On Wed 17. Oct 2018 at 21:00, Igor Ignatyev > wrote: > >> EXPECT would just mark a test as failed and allow the method to continue, >> so rs.release() will be called. > > > Oh, okay. Thanks for clarifying. > > >> >> test fixture will basically do the same as an RAII object, but it will be >> a bit more obvious as test preparation and cleaning steps will be separated >> from test actions, so I'll go w/ setUp/tearDown. > > > Okay fine by me. > > ..thomas > > >> >> -- Igor >> >> > On Oct 17, 2018, at 11:53 AM, Thomas St?fe >> wrote: >> > >> > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev >> wrote: >> >> >> >> Hi Thomas, >> >> >> >> 1st of all, thanks for looking at it, appreciate that. >> >> >> >> it seems we can. there are two ways I can fix that either replace >> ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which >> do you prefer? >> >> >> >> -- Igor >> > >> > Slight preference for SetUp/TearDown (EXCEPT would quit the testing >> > altogether, wont it?). >> > >> > Lazy me would have just used an RAII object like this: >> > >> > struct Cleaner { >> > ReservedSpace* rs; >> > ~Cleaner() { rs.release(); } >> > }; >> > >> > But I leave it up to you. >> > >> > ... Thomas >> > >> >> >> >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe >> wrote: >> >>> >> >>> Hi Igor, >> >>> >> >>> test_virtual_space_actual_committed_space() and >> >>> actual_committed_space_one_large_page(): >> >>> >> >>> Won't we leak the reserved space now when we run into an ASSERT and >> >>> then continue with the next gtests? >> >>> >> >>> Cheers, Thomas >> >>> >> >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev < >> igor.ignatyev at oracle.com> wrote: >> >>>> >> >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; >> >>>> >> >>>> Hi all, >> >>>> >> >>>> could you please review this small (and hopefully trivial) patch >> which converts internal TestVirtualSpace_test to gtest? >> >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, >> the old code hasn't been removed. it will be removed later when all 4 tests >> used by runMemoryUnitTests are converted. >> >>>> >> >>>> for the sake of reviewer, >> http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 >> is the old version. >> >>>> >> >>>> webrev: >> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 >> >>>> testing: >> >>>> - converted tests on linux-x64, windows-x64, macosx-x64, >> solaris-sparcv9 in product and fastdebug variants >> >>>> - build w/ precompiled-headers enabled and disabled >> >>>> >> >>>> Thanks, >> >>>> -- Igor >> >> >> >> > -- Thanks, Jc From dean.long at oracle.com Wed Oct 17 22:18:32 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 15:18:32 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> Message-ID: <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> On 10/17/18 2:38 PM, Mandy Chung wrote: > > > On 10/17/18 2:13 PM, dean.long at oracle.com wrote: >> On 10/17/18 1:41 PM, Mandy Chung wrote: >>> >>> >>> On 10/16/18 7:33 PM, David Holmes wrote: >>>> Hi Dean, >>>> >>>> Thanks for tackling this. >>>> >>>> I'm still struggling to fully grasp why we need both the >>>> PerfCounters and the regular counters. I get that we have to >>>> decrement the live counts before ensure_join() has allowed >>>> Thread.join() to return, to ensure that if we then check the number >>>> of threads it has dropped by one. But I don't understand why that >>>> means we need to manage the thread count in two parts. Particularly >>>> as now you don't use the PerfCounter to return the live count, so >>>> it makes me wonder what role the PerfCounter is playing as it is >>>> temporarily inconsistent with the reported live count? >>> >>> Perf counters were added long time back in JDK 1.4.2 for performance >>> measurement before java.lang.management API.? One can use jstat tool >>> to monitor VM perf counters of a running VM.?? One could look into >>> the possibility of deprecating these counters and remove them over time. >>> >>>> On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: >>>> New webrev: >>>> >>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ >>> >>> When the perf counters are updated when a thread is added/removed, >>> it's holding Threads_lock.? Are the asserts in >>> ThreadService::remove_thread necessary? >>> >> >> Not really.? They were intended to catch the case where the atomic >> counters weren't decremented for some reason, not for the perf counters. >> Should I remove them? >> > > Hmm... when remove_thread is called but decrement_thread_counts has > not been called. ? It's a bug in thread accounting.? It happens to > have the perf counters that can be compared to assert. It seems not > obvious.? Setting the perf counters same values as > _atomic_threads_count and _atomic_daemon_threads_count makes sense to me. > > I would opt for removing the asserts but I can't think of an > alternative how to catch the issue you concern about. > >>> For clarify, I think we could simply set _live_threads_count to the >>> value of _atomic_threads_count and set _daemon_threads_count to the >>> value of _atomic_daemon_threads_count. >>> >> >> I think that works, even inside decrement_thread_counts() without >> holding the Threads_lock.? If you agree, I'll make that change. >> > +1 > New webrevs, full and incremental: http://cr.openjdk.java.net/~dlong/8021335/webrev.6/ http://cr.openjdk.java.net/~dlong/8021335/webrev.6.diff/ I like it better without all the asserts too. dl > Mandy From ioi.lam at oracle.com Wed Oct 17 22:44:37 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 17 Oct 2018 15:44:37 -0700 Subject: RFR(XS) 8212612 - Add documentation about Arguments::_exit_hook In-Reply-To: References: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> <89b3afbf-94ed-4485-1381-724e5cb1d622@oracle.com> Message-ID: On 10/17/2018 02:59 PM, dean.long at oracle.com wrote: > On 10/17/18 2:35 PM, Ioi Lam wrote: >> >> >> On 10/17/2018 02:27 PM, dean.long at oracle.com wrote: >>> On 10/17/18 9:33 AM, Ioi Lam wrote: >>>> http://cr.openjdk.java.net/~iklam/jdk12/8212612-exit-hook-docs.v01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8212612 >>>> >>>> The following fields are assigned to according to >>>> JavaVMOption.optionString. >>>> >>>> ? abort_hook_t Arguments::_abort_hook = NULL; >>>> ? exit_hook_t Arguments::_exit_hook = NULL; >>>> ? vfprintf_hook_t Arguments::_vfprintf_hook = NULL; >>>> >>>> However, there's no code in the JDK repo that uses these options. >>>> Instead, they are intended for programs that embed the JVM using >>>> JNI_CreateJavaVM. >>>> >>>> We should add comments in arguments.cpp for clarification. Otherwise >>>> this would look like dead code and someone might be tempted to >>>> remove it. >>>> >>>> Thanks >>>> - Ioi >>> >>> But it's not dead code.? All three are used.? For example, see >>> jio_vfprintf(). >>> >>> dl >> >> Hi Dean, thanks for looking into this. >> >> I was afraid someone would infer that since nowhere in the JDK code >> would do anything to assign _vfprintf_hook to a non-NULL value, the >> block >> >> ? if (Arguments::vfprintf_hook() != NULL) { >> ??? jio_fprintf(defaultStream::output_stream(), "%.*s", (int)len, s); >> ? } >> >> would be effectively dead code. >> > > OK, then this comment was confusing: > > These are not used by the JDK itself > > I think it's more accurate to say that JDK launchers don't set these, > but the JVM does use and honor them if set. > How about this: // These are not set by the JDK's built-in launchers, but they can be set by programs that // embed the JVM using JNI_CreateJavaVM. See comments around JavaVMOption in jni.h. abort_hook_t???? Arguments::_abort_hook???????? = NULL; exit_hook_t????? Arguments::_exit_hook????????? = NULL; vfprintf_hook_t? Arguments::_vfprintf_hook????? = NULL; Thanks - Ioi From ioi.lam at oracle.com Wed Oct 17 23:54:06 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 17 Oct 2018 16:54:06 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> Message-ID: Hi Michael & Volker, Thanks for the bug report and the initial investigation! Following Volker's investigation, I tried to make a less drastic fix (than the "nuclear option" of disable CDS entirely), but run into a lot of issues when class-file-load-hook is specified: 1. The shared heap objects are mapped at JVM start-up. At this point, we don't know if any of the archived object's class may be replaced by the CFLH. 2. If java.lang.{Object, Cloneable, Serializable} are replaced by CFLH, we cannot use the archived primitive arrays. 3. There's a lot of code that assumes if UseSharedSpaces==true, certain InstanceKlasses must be loaded from CDS. E.g., ?? JavaClasses::compute_offsets() does nothing if UseSharedSpaces==true, ?? because it assumes that offsets calculated during dump time will match ?? the class loaded at runtime. I have developed a jtreg test case for this, and attempted to do a less drastic fix, but I am nowhere near done and it's already messy: http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v00/ Instead, I think we should do this: http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v01/ I will test the patch (probably some old tests may fail because they assumed that CDS can be enabled when CFLH is in use). I'll post a formal RFR afterwards. I'll also see if it's possible to allow CDS if can_generate_early_class_hook_events==false. I am not entirely sure if all classes affected by UseSharedSpaces are all loaded in the "early" stage. We can try the "messy" fix later, if we have more time to get it right. Thanks - Ioi On 10/15/2018 03:35 PM, Jiangli Zhou wrote: > On 10/15/18 3:30 PM, Michael Rasmussen wrote: > >>> One simple alternative fix is to disable sharing when some events (like >>> CFLH) were enabled. This in fact was the behavior before JDK 9. In JDK >>> 9, we enabled CDS for debugging. Looks like there are corner cases that >>> are not uncovered by our testing. I'd like to seek opinions and >>> feedback >>> on the importance of supporting debugging with CDS enabled. >> Or at least CFLH together with can_generate_all_class_hook_events and >> can_generate_early_class_hook_events capabilities. >> >> CDS hasn't been part of our own testing scenarios so far, which is >> why we haven't notice these issues on our end before now. >> We only just discovered it because of user reports about crashes, >> which turned out to be because the java-11-openjdk package on Fedora >> comes with pre-generated CDS archive. >> >> But especially with JEP 341 just around the corner, it makes it very >> important to find and weed out these bugs now! > > Totally agree. These issues are now more important with the default > CDS archive. Thanks for reporting them! > > Best regards, > Jiangli >> >> /Michael > From dean.long at oracle.com Wed Oct 17 23:59:11 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 16:59:11 -0700 Subject: RFR(XS) 8212612 - Add documentation about Arguments::_exit_hook In-Reply-To: References: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> <89b3afbf-94ed-4485-1381-724e5cb1d622@oracle.com> Message-ID: <69fd7fbe-1099-c452-dcc3-0b23fc59b6e9@oracle.com> On 10/17/18 3:44 PM, Ioi Lam wrote: > > > > On 10/17/2018 02:59 PM, dean.long at oracle.com wrote: >> On 10/17/18 2:35 PM, Ioi Lam wrote: >>> >>> >>> On 10/17/2018 02:27 PM, dean.long at oracle.com wrote: >>>> On 10/17/18 9:33 AM, Ioi Lam wrote: >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8212612-exit-hook-docs.v01/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8212612 >>>>> >>>>> The following fields are assigned to according to >>>>> JavaVMOption.optionString. >>>>> >>>>> ? abort_hook_t Arguments::_abort_hook = NULL; >>>>> ? exit_hook_t Arguments::_exit_hook = NULL; >>>>> ? vfprintf_hook_t Arguments::_vfprintf_hook = NULL; >>>>> >>>>> However, there's no code in the JDK repo that uses these options. >>>>> Instead, they are intended for programs that embed the JVM using >>>>> JNI_CreateJavaVM. >>>>> >>>>> We should add comments in arguments.cpp for clarification. Otherwise >>>>> this would look like dead code and someone might be tempted to >>>>> remove it. >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>>> But it's not dead code.? All three are used.? For example, see >>>> jio_vfprintf(). >>>> >>>> dl >>> >>> Hi Dean, thanks for looking into this. >>> >>> I was afraid someone would infer that since nowhere in the JDK code >>> would do anything to assign _vfprintf_hook to a non-NULL value, the >>> block >>> >>> ? if (Arguments::vfprintf_hook() != NULL) { >>> ??? jio_fprintf(defaultStream::output_stream(), "%.*s", (int)len, s); >>> ? } >>> >>> would be effectively dead code. >>> >> >> OK, then this comment was confusing: >> >> These are not used by the JDK itself >> >> I think it's more accurate to say that JDK launchers don't set these, >> but the JVM does use and honor them if set. >> > > How about this: > > // These are not set by the JDK's built-in launchers, but they can be > set by programs that > // embed the JVM using JNI_CreateJavaVM. See comments around > JavaVMOption in jni.h. > abort_hook_t???? Arguments::_abort_hook???????? = NULL; > exit_hook_t????? Arguments::_exit_hook????????? = NULL; > vfprintf_hook_t? Arguments::_vfprintf_hook????? = NULL; > > That's good! dl > Thanks > - Ioi > From jcbeyler at google.com Thu Oct 18 00:06:40 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 17 Oct 2018 17:06:40 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> Message-ID: Hi Dean, The new webrev looks much better :) LGTM (not a reviewer though :-)). Thanks, Jc On Wed, Oct 17, 2018 at 3:19 PM wrote: > On 10/17/18 2:38 PM, Mandy Chung wrote: > > > > On 10/17/18 2:13 PM, dean.long at oracle.com wrote: > > On 10/17/18 1:41 PM, Mandy Chung wrote: > > > > On 10/16/18 7:33 PM, David Holmes wrote: > > Hi Dean, > > Thanks for tackling this. > > I'm still struggling to fully grasp why we need both the PerfCounters and > the regular counters. I get that we have to decrement the live counts > before ensure_join() has allowed Thread.join() to return, to ensure that if > we then check the number of threads it has dropped by one. But I don't > understand why that means we need to manage the thread count in two parts. > Particularly as now you don't use the PerfCounter to return the live count, > so it makes me wonder what role the PerfCounter is playing as it is > temporarily inconsistent with the reported live count? > > > Perf counters were added long time back in JDK 1.4.2 for performance > measurement before java.lang.management API. One can use jstat tool to > monitor VM perf counters of a running VM. One could look into the > possibility of deprecating these counters and remove them over time. > > On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: > New webrev: > > http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ > > > When the perf counters are updated when a thread is added/removed, it's > holding Threads_lock. Are the asserts in ThreadService::remove_thread > necessary? > > > Not really. They were intended to catch the case where the atomic > counters weren't decremented for some reason, not for the perf counters. > Should I remove them? > > > Hmm... when remove_thread is called but decrement_thread_counts has not > been called. It's a bug in thread accounting. It happens to have the > perf counters that can be compared to assert. It seems not obvious. > Setting the perf counters same values as _atomic_threads_count and > _atomic_daemon_threads_count makes sense to me. > > I would opt for removing the asserts but I can't think of an alternative > how to catch the issue you concern about. > > For clarify, I think we could simply set _live_threads_count to the value > of _atomic_threads_count and set _daemon_threads_count to the value of > _atomic_daemon_threads_count. > > > I think that works, even inside decrement_thread_counts() without holding > the Threads_lock. If you agree, I'll make that change. > > +1 > > > New webrevs, full and incremental: > > http://cr.openjdk.java.net/~dlong/8021335/webrev.6/ > http://cr.openjdk.java.net/~dlong/8021335/webrev.6.diff/ > > I like it better without all the asserts too. > > dl > > Mandy > > > -- Thanks, Jc From dean.long at oracle.com Thu Oct 18 00:38:14 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 17:38:14 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> Message-ID: Thanks JC! dl On 10/17/18 5:06 PM, JC Beyler wrote: > Hi Dean, > > The new webrev looks much better :) LGTM (not a reviewer though :-)). > > Thanks, > Jc > On Wed, Oct 17, 2018 at 3:19 PM > wrote: > > On 10/17/18 2:38 PM, Mandy Chung wrote: >> >> >> On 10/17/18 2:13 PM, dean.long at oracle.com >> wrote: >>> On 10/17/18 1:41 PM, Mandy Chung wrote: >>>> >>>> >>>> On 10/16/18 7:33 PM, David Holmes wrote: >>>>> Hi Dean, >>>>> >>>>> Thanks for tackling this. >>>>> >>>>> I'm still struggling to fully grasp why we need both the >>>>> PerfCounters and the regular counters. I get that we have to >>>>> decrement the live counts before ensure_join() has allowed >>>>> Thread.join() to return, to ensure that if we then check the >>>>> number of threads it has dropped by one. But I don't >>>>> understand why that means we need to manage the thread count >>>>> in two parts. Particularly as now you don't use the >>>>> PerfCounter to return the live count, so it makes me wonder >>>>> what role the PerfCounter is playing as it is temporarily >>>>> inconsistent with the reported live count? >>>> >>>> Perf counters were added long time back in JDK 1.4.2 for >>>> performance measurement before java.lang.management API.? One >>>> can use jstat tool to monitor VM perf counters of a running >>>> VM.?? One could look into the possibility of deprecating these >>>> counters and remove them over time. >>>> >>>>> On 17/10/2018 9:43 AM, dean.long at oracle.com >>>>> wrote: >>>>> New webrev: >>>>> >>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ >>>>> >>>> >>>> When the perf counters are updated when a thread is >>>> added/removed, it's holding Threads_lock.? Are the asserts in >>>> ThreadService::remove_thread necessary? >>>> >>> >>> Not really.? They were intended to catch the case where the >>> atomic counters weren't decremented for some reason, not for the >>> perf counters. >>> Should I remove them? >>> >> >> Hmm... when remove_thread is called but decrement_thread_counts >> has not been called. ? It's a bug in thread accounting.? It >> happens to have the perf counters that can be compared to >> assert.? It seems not obvious.? Setting the perf counters same >> values as _atomic_threads_count and _atomic_daemon_threads_count >> makes sense to me. >> >> I would opt for removing the asserts but I can't think of an >> alternative how to catch the issue you concern about. >> >>>> For clarify, I think we could simply set _live_threads_count to >>>> the value of _atomic_threads_count and set >>>> _daemon_threads_count to the value of _atomic_daemon_threads_count. >>>> >>> >>> I think that works, even inside decrement_thread_counts() >>> without holding the Threads_lock.? If you agree, I'll make that >>> change. >>> >> +1 >> > > New webrevs, full and incremental: > > http://cr.openjdk.java.net/~dlong/8021335/webrev.6/ > > http://cr.openjdk.java.net/~dlong/8021335/webrev.6.diff/ > > > I like it better without all the asserts too. > > dl > >> Mandy > > > > -- > > Thanks, > Jc From mandy.chung at oracle.com Thu Oct 18 01:35:21 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Oct 2018 18:35:21 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> Message-ID: <08740d81-34dd-a356-1f2e-9bd1994b16f1@oracle.com> On 10/17/18 3:18 PM, dean.long at oracle.com wrote: > > New webrevs, full and incremental: > > http://cr.openjdk.java.net/~dlong/8021335/webrev.6/ > http://cr.openjdk.java.net/~dlong/8021335/webrev.6.diff/ > > I like it better without all the asserts too. Looks good to me. Mandy From david.holmes at oracle.com Thu Oct 18 02:07:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Oct 2018 12:07:27 +1000 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> Message-ID: <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> Hi Dean, This still seems racy to me. We increment all counts under the Threads_lock but we still decrement without the Threads_lock. So we can lose updates to the perfCounters. 117 _total_threads_count->inc(); 118 Atomic::inc(&_atomic_threads_count); 119 int count = _atomic_threads_count; <= context switch here 120 _live_threads_count->set_value(count); If a second thread now exits while the above thread is descheduled, it will decrement _atomic_threads_count and _live_threads_count, but when the first thread resumes at line 120 above we will set _live_threads_count to the wrong value! You can't maintain two counters in sync without all changes using locking across both. David On 18/10/2018 8:18 AM, dean.long at oracle.com wrote: > On 10/17/18 2:38 PM, Mandy Chung wrote: >> >> >> On 10/17/18 2:13 PM, dean.long at oracle.com wrote: >>> On 10/17/18 1:41 PM, Mandy Chung wrote: >>>> >>>> >>>> On 10/16/18 7:33 PM, David Holmes wrote: >>>>> Hi Dean, >>>>> >>>>> Thanks for tackling this. >>>>> >>>>> I'm still struggling to fully grasp why we need both the >>>>> PerfCounters and the regular counters. I get that we have to >>>>> decrement the live counts before ensure_join() has allowed >>>>> Thread.join() to return, to ensure that if we then check the number >>>>> of threads it has dropped by one. But I don't understand why that >>>>> means we need to manage the thread count in two parts. Particularly >>>>> as now you don't use the PerfCounter to return the live count, so >>>>> it makes me wonder what role the PerfCounter is playing as it is >>>>> temporarily inconsistent with the reported live count? >>>> >>>> Perf counters were added long time back in JDK 1.4.2 for performance >>>> measurement before java.lang.management API.? One can use jstat tool >>>> to monitor VM perf counters of a running VM.?? One could look into >>>> the possibility of deprecating these counters and remove them over time. >>>> >>>>> On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: >>>>> New webrev: >>>>> >>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ >>>> >>>> When the perf counters are updated when a thread is added/removed, >>>> it's holding Threads_lock.? Are the asserts in >>>> ThreadService::remove_thread necessary? >>>> >>> >>> Not really.? They were intended to catch the case where the atomic >>> counters weren't decremented for some reason, not for the perf counters. >>> Should I remove them? >>> >> >> Hmm... when remove_thread is called but decrement_thread_counts has >> not been called. ? It's a bug in thread accounting.? It happens to >> have the perf counters that can be compared to assert. It seems not >> obvious.? Setting the perf counters same values as >> _atomic_threads_count and _atomic_daemon_threads_count makes sense to me. >> >> I would opt for removing the asserts but I can't think of an >> alternative how to catch the issue you concern about. >> >>>> For clarify, I think we could simply set _live_threads_count to the >>>> value of _atomic_threads_count and set _daemon_threads_count to the >>>> value of _atomic_daemon_threads_count. >>>> >>> >>> I think that works, even inside decrement_thread_counts() without >>> holding the Threads_lock.? If you agree, I'll make that change. >>> >> +1 >> > > New webrevs, full and incremental: > > http://cr.openjdk.java.net/~dlong/8021335/webrev.6/ > http://cr.openjdk.java.net/~dlong/8021335/webrev.6.diff/ > > I like it better without all the asserts too. > > dl > >> Mandy > From david.holmes at oracle.com Thu Oct 18 02:13:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Oct 2018 12:13:01 +1000 Subject: RFR(XS) 8212612 - Add documentation about Arguments::_exit_hook In-Reply-To: <69fd7fbe-1099-c452-dcc3-0b23fc59b6e9@oracle.com> References: <2ac6a1db-ea86-1525-09dd-ff79b443665a@oracle.com> <89b3afbf-94ed-4485-1381-724e5cb1d622@oracle.com> <69fd7fbe-1099-c452-dcc3-0b23fc59b6e9@oracle.com> Message-ID: <39c09e71-89ab-0b02-cdc1-249265ae4cb3@oracle.com> +1 on the updated comment. Thanks, David On 18/10/2018 9:59 AM, dean.long at oracle.com wrote: > On 10/17/18 3:44 PM, Ioi Lam wrote: >> >> >> >> On 10/17/2018 02:59 PM, dean.long at oracle.com wrote: >>> On 10/17/18 2:35 PM, Ioi Lam wrote: >>>> >>>> >>>> On 10/17/2018 02:27 PM, dean.long at oracle.com wrote: >>>>> On 10/17/18 9:33 AM, Ioi Lam wrote: >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8212612-exit-hook-docs.v01/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8212612 >>>>>> >>>>>> The following fields are assigned to according to >>>>>> JavaVMOption.optionString. >>>>>> >>>>>> ? abort_hook_t Arguments::_abort_hook = NULL; >>>>>> ? exit_hook_t Arguments::_exit_hook = NULL; >>>>>> ? vfprintf_hook_t Arguments::_vfprintf_hook = NULL; >>>>>> >>>>>> However, there's no code in the JDK repo that uses these options. >>>>>> Instead, they are intended for programs that embed the JVM using >>>>>> JNI_CreateJavaVM. >>>>>> >>>>>> We should add comments in arguments.cpp for clarification. Otherwise >>>>>> this would look like dead code and someone might be tempted to >>>>>> remove it. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>> >>>>> But it's not dead code.? All three are used.? For example, see >>>>> jio_vfprintf(). >>>>> >>>>> dl >>>> >>>> Hi Dean, thanks for looking into this. >>>> >>>> I was afraid someone would infer that since nowhere in the JDK code >>>> would do anything to assign _vfprintf_hook to a non-NULL value, the >>>> block >>>> >>>> ? if (Arguments::vfprintf_hook() != NULL) { >>>> ??? jio_fprintf(defaultStream::output_stream(), "%.*s", (int)len, s); >>>> ? } >>>> >>>> would be effectively dead code. >>>> >>> >>> OK, then this comment was confusing: >>> >>> These are not used by the JDK itself >>> >>> I think it's more accurate to say that JDK launchers don't set these, >>> but the JVM does use and honor them if set. >>> >> >> How about this: >> >> // These are not set by the JDK's built-in launchers, but they can be >> set by programs that >> // embed the JVM using JNI_CreateJavaVM. See comments around >> JavaVMOption in jni.h. >> abort_hook_t???? Arguments::_abort_hook???????? = NULL; >> exit_hook_t????? Arguments::_exit_hook????????? = NULL; >> vfprintf_hook_t? Arguments::_vfprintf_hook????? = NULL; >> >> > > That's good! > > dl > >> Thanks >> - Ioi >> > From jiangli.zhou at oracle.com Thu Oct 18 02:39:17 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 17 Oct 2018 19:39:17 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> Message-ID: On 10/17/18 4:54 PM, Ioi Lam wrote: > Hi Michael & Volker, > > > Thanks for the bug report and the initial investigation! > > Following Volker's investigation, I tried to make a less drastic fix > (than the "nuclear option" of disable CDS entirely), but run into a > lot of > issues when class-file-load-hook is specified: > > > 1. The shared heap objects are mapped at JVM start-up. At this > point, we don't know if any of the archived object's class may > be replaced by the CFLH. I've given more thoughts to this. Besides Strings, other archived objects are already disabled automatically. Archived mirrors and cp reference arrays are not installed if their archived class are not used. The archived subg-raphs are not used if any of the classes (for objects in the subgraphs) are not the same as the archived ones. > > 2. If java.lang.{Object, Cloneable, Serializable} are replaced > by CFLH, we cannot use the archived primitive arrays. > > 3. There's a lot of code that assumes if UseSharedSpaces==true, > certain InstanceKlasses must be loaded from CDS. E.g., > ?? JavaClasses::compute_offsets() does nothing if UseSharedSpaces==true, > ?? because it assumes that offsets calculated during dump time will match > ?? the class loaded at runtime. Yes, the assumption is baked in our code. We'd have to exam each UseSharedSpaces check if we want to support the behavior. > > > I have developed a jtreg test case for this, and attempted to do a > less drastic fix, but I am nowhere near done and it's already messy: > > http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v00/ > > > > Instead, I think we should do this: > > http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v01/ > This would brings back the behavior of JDK-8141341. To be more cautious this time, is there any case that's covered by the check for can_generate_all_class_hook_events/can_generate_early_class_hook_events capabilities but not for JvmtiExport::should_post_class_file_load_hook()? As part of the JDK-8141341, we also stop loading any additional archived classes if JvmtiExport::should_post_class_file_load_hook() is set after mapping the shared archive. Should we do that also if we are going with this approach? Thanks, Jiangli > > > I will test the patch (probably some old tests may fail because they > assumed > that CDS can be enabled when CFLH is in use). I'll post a formal RFR > afterwards. > > I'll also see if it's possible to allow CDS if > can_generate_early_class_hook_events==false. > I am not entirely sure if all classes affected by UseSharedSpaces are all > loaded in the "early" stage. > > We can try the "messy" fix later, if we have more time to get it right. > > > Thanks > - Ioi > > > On 10/15/2018 03:35 PM, Jiangli Zhou wrote: >> On 10/15/18 3:30 PM, Michael Rasmussen wrote: >> >>>> One simple alternative fix is to disable sharing when some events >>>> (like >>>> CFLH) were enabled. This in fact was the behavior before JDK 9. In JDK >>>> 9, we enabled CDS for debugging. Looks like there are corner cases >>>> that >>>> are not uncovered by our testing. I'd like to seek opinions and >>>> feedback >>>> on the importance of supporting debugging with CDS enabled. >>> Or at least CFLH together with can_generate_all_class_hook_events >>> and can_generate_early_class_hook_events capabilities. >>> >>> CDS hasn't been part of our own testing scenarios so far, which is >>> why we haven't notice these issues on our end before now. >>> We only just discovered it because of user reports about crashes, >>> which turned out to be because the java-11-openjdk package on Fedora >>> comes with pre-generated CDS archive. >>> >>> But especially with JEP 341 just around the corner, it makes it very >>> important to find and weed out these bugs now! >> >> Totally agree. These issues are now more important with the default >> CDS archive. Thanks for reporting them! >> >> Best regards, >> Jiangli >>> >>> /Michael >> > From dean.long at oracle.com Thu Oct 18 03:09:18 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 20:09:18 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <08740d81-34dd-a356-1f2e-9bd1994b16f1@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <08740d81-34dd-a356-1f2e-9bd1994b16f1@oracle.com> Message-ID: Thanks, Mandy! dl On 10/17/18 6:35 PM, Mandy Chung wrote: > > > On 10/17/18 3:18 PM, dean.long at oracle.com wrote: >> >> New webrevs, full and incremental: >> >> http://cr.openjdk.java.net/~dlong/8021335/webrev.6/ >> http://cr.openjdk.java.net/~dlong/8021335/webrev.6.diff/ >> >> I like it better without all the asserts too. > > Looks good to me. > > Mandy From dean.long at oracle.com Thu Oct 18 04:06:41 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 17 Oct 2018 21:06:41 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> Message-ID: <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> On 10/17/18 7:07 PM, David Holmes wrote: > Hi Dean, > > This still seems racy to me. We increment all counts under the > Threads_lock but we still decrement without the Threads_lock. So we > can lose updates to the perfCounters. > > ?117?? _total_threads_count->inc(); > ?118?? Atomic::inc(&_atomic_threads_count); > ?119?? int count = _atomic_threads_count; > <= context switch here > ?120?? _live_threads_count->set_value(count); > > If a second thread now exits while the above thread is descheduled, it > will decrement _atomic_threads_count and _live_threads_count, but when > the first thread resumes at line 120 above we will set > _live_threads_count to the wrong value! > > You can't maintain two counters in sync without all changes using > locking across both. > You're right, I missed that.? I think the right thing to do is call current_thread_exiting while holding the Threads_lock. Then we can get rid of the parallel atomic counters.? So, here's one more try: ??? http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ dl > David > > > > On 18/10/2018 8:18 AM, dean.long at oracle.com wrote: >> On 10/17/18 2:38 PM, Mandy Chung wrote: >>> >>> >>> On 10/17/18 2:13 PM, dean.long at oracle.com wrote: >>>> On 10/17/18 1:41 PM, Mandy Chung wrote: >>>>> >>>>> >>>>> On 10/16/18 7:33 PM, David Holmes wrote: >>>>>> Hi Dean, >>>>>> >>>>>> Thanks for tackling this. >>>>>> >>>>>> I'm still struggling to fully grasp why we need both the >>>>>> PerfCounters and the regular counters. I get that we have to >>>>>> decrement the live counts before ensure_join() has allowed >>>>>> Thread.join() to return, to ensure that if we then check the >>>>>> number of threads it has dropped by one. But I don't understand >>>>>> why that means we need to manage the thread count in two parts. >>>>>> Particularly as now you don't use the PerfCounter to return the >>>>>> live count, so it makes me wonder what role the PerfCounter is >>>>>> playing as it is temporarily inconsistent with the reported live >>>>>> count? >>>>> >>>>> Perf counters were added long time back in JDK 1.4.2 for >>>>> performance measurement before java.lang.management API. One can >>>>> use jstat tool to monitor VM perf counters of a running VM.?? One >>>>> could look into the possibility of deprecating these counters and >>>>> remove them over time. >>>>> >>>>>> On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: >>>>>> New webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ >>>>> >>>>> When the perf counters are updated when a thread is added/removed, >>>>> it's holding Threads_lock.? Are the asserts in >>>>> ThreadService::remove_thread necessary? >>>>> >>>> >>>> Not really.? They were intended to catch the case where the atomic >>>> counters weren't decremented for some reason, not for the perf >>>> counters. >>>> Should I remove them? >>>> >>> >>> Hmm... when remove_thread is called but decrement_thread_counts has >>> not been called. ? It's a bug in thread accounting.? It happens to >>> have the perf counters that can be compared to assert. It seems not >>> obvious.? Setting the perf counters same values as >>> _atomic_threads_count and _atomic_daemon_threads_count makes sense >>> to me. >>> >>> I would opt for removing the asserts but I can't think of an >>> alternative how to catch the issue you concern about. >>> >>>>> For clarify, I think we could simply set _live_threads_count to >>>>> the value of _atomic_threads_count and set _daemon_threads_count >>>>> to the value of _atomic_daemon_threads_count. >>>>> >>>> >>>> I think that works, even inside decrement_thread_counts() without >>>> holding the Threads_lock.? If you agree, I'll make that change. >>>> >>> +1 >>> >> >> New webrevs, full and incremental: >> >> http://cr.openjdk.java.net/~dlong/8021335/webrev.6/ >> http://cr.openjdk.java.net/~dlong/8021335/webrev.6.diff/ >> >> I like it better without all the asserts too. >> >> dl >> >>> Mandy >> From ioi.lam at oracle.com Thu Oct 18 05:31:11 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 17 Oct 2018 22:31:11 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> Message-ID: <5493aca2-f67b-4e9f-1f10-02511d701eca@oracle.com> On 10/17/18 7:39 PM, Jiangli Zhou wrote: > On 10/17/18 4:54 PM, Ioi Lam wrote: > >> Hi Michael & Volker, >> >> >> Thanks for the bug report and the initial investigation! >> >> Following Volker's investigation, I tried to make a less drastic fix >> (than the "nuclear option" of disable CDS entirely), but run into a >> lot of >> issues when class-file-load-hook is specified: >> >> >> 1. The shared heap objects are mapped at JVM start-up. At this >> point, we don't know if any of the archived object's class may >> be replaced by the CFLH. > I've given more thoughts to this. Besides Strings, other archived > objects are already disabled automatically. Archived mirrors and cp > reference arrays are not installed if their archived class are not > used. The archived subg-raphs are not used if any of the classes (for > objects in the subgraphs) are not the same as the archived ones. Thanks for the clarification. This makes the patch simpler. >> >> 2. If java.lang.{Object, Cloneable, Serializable} are replaced >> by CFLH, we cannot use the archived primitive arrays. >> >> 3. There's a lot of code that assumes if UseSharedSpaces==true, >> certain InstanceKlasses must be loaded from CDS. E.g., >> ?? JavaClasses::compute_offsets() does nothing if UseSharedSpaces==true, >> ?? because it assumes that offsets calculated during dump time will >> match >> ?? the class loaded at runtime. > Yes, the assumption is baked in our code. We'd have to exam each > UseSharedSpaces check if we want to support the behavior. I think all of the UseSharedSpaces assumptions affect only classes loaded inside SystemDictionary::resolve_preloaded_classes(), which is called in the JVMTI "early" stage. So I've changed the code to disable CDS only like this (as Michael suggested): ? if (JvmtiExport::should_post_class_file_load_hook() && JvmtiExport::early_class_hook_env()) { ??? // CDS assumes that no classes resolved in SystemDictionary::resolve_preloaded_classes ??? // are replaced at runtime by JVMTI ClassFileLoadHook. All of those classes are resolved ??? // during the JVMTI "early" stage, so we're OK if JvmtiExport::early_class_hook_env() ??? // is not specified by the agent(s). ??? FileMapInfo::fail_continue("CDS is disabled because early JVMTI ClassFileLoadHook is in use."); ??? return false; ? } Updated webrev: http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v02/ I'll add more asserts and test cases to validate the patch. >> >> >> I have developed a jtreg test case for this, and attempted to do a >> less drastic fix, but I am nowhere near done and it's already messy: >> >> http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v00/ >> >> >> >> Instead, I think we should do this: >> >> http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v01/ >> > This would brings back the behavior of JDK-8141341. To be more > cautious this time, is there any case that's covered by the check for > can_generate_all_class_hook_events/can_generate_early_class_hook_events > capabilities but not for JvmtiExport::should_post_class_file_load_hook()? > I looked at JvmtiClassFileLoadHookPoster::post_to_env(). The "preloaded" classes can be replaced only if at least one agent has (1) set the following 2 capabilities: ??? caps.can_generate_all_class_hook_events = 1; ??? caps.can_generate_early_class_hook_events = 1; AND (2) Specified JVMTI_EVENT_CLASS_FILE_LOAD_HOOK See http://hg.openjdk.java.net/jdk/jdk/file/672bc2213cef/src/hotspot/share/prims/jvmtiExport.cpp#l932 > As part of the JDK-8141341, we also stop loading any additional > archived classes if JvmtiExport::should_post_class_file_load_hook() is > set after mapping the shared archive. Should we do that also if we are > going with this approach? > I don't think that's necessary. It should be OK to replace any classes that are not in the "preloaded" list. Originally (before JDK 9) we would disable CDS so that classes are loaded normally, in order for the class file data to be posted to the CFLH. However, since JDK 9, we have added the ability for posting the class file data of shared classes to the CFLH, so there's no need to disable CDS. Thanks - Ioi > Thanks, > Jiangli >> >> >> I will test the patch (probably some old tests may fail because they >> assumed >> that CDS can be enabled when CFLH is in use). I'll post a formal RFR >> afterwards. >> >> I'll also see if it's possible to allow CDS if >> can_generate_early_class_hook_events==false. >> I am not entirely sure if all classes affected by UseSharedSpaces are >> all >> loaded in the "early" stage. >> >> We can try the "messy" fix later, if we have more time to get it right. >> >> >> Thanks >> - Ioi >> >> >> On 10/15/2018 03:35 PM, Jiangli Zhou wrote: >>> On 10/15/18 3:30 PM, Michael Rasmussen wrote: >>> >>>>> One simple alternative fix is to disable sharing when some events >>>>> (like >>>>> CFLH) were enabled. This in fact was the behavior before JDK 9. In >>>>> JDK >>>>> 9, we enabled CDS for debugging. Looks like there are corner cases >>>>> that >>>>> are not uncovered by our testing. I'd like to seek opinions and >>>>> feedback >>>>> on the importance of supporting debugging with CDS enabled. >>>> Or at least CFLH together with can_generate_all_class_hook_events >>>> and can_generate_early_class_hook_events capabilities. >>>> >>>> CDS hasn't been part of our own testing scenarios so far, which is >>>> why we haven't notice these issues on our end before now. >>>> We only just discovered it because of user reports about crashes, >>>> which turned out to be because the java-11-openjdk package on >>>> Fedora comes with pre-generated CDS archive. >>>> >>>> But especially with JEP 341 just around the corner, it makes it >>>> very important to find and weed out these bugs now! >>> >>> Totally agree. These issues are now more important with the default >>> CDS archive. Thanks for reporting them! >>> >>> Best regards, >>> Jiangli >>>> >>>> /Michael >>> >> > From thomas.stuefe at gmail.com Thu Oct 18 05:39:53 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 18 Oct 2018 07:39:53 +0200 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> Message-ID: Looks both good Igor. +1 for the const pointers :) ..Thomas On Wed, Oct 17, 2018 at 11:45 PM Igor Ignatyev wrote: > > Hi Thomas, JC > > I tried to use a test fixture and the code became very clumsy, so I went w/ a simple RAII. > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.0-1/index.html > > I've also noticed that we have same problem in TestReservedSpace_test tests -- http://cr.openjdk.java.net/~iignatyev//8171097/webrev.0-1/index.html > > Thanks, > -- Igor > > On Oct 17, 2018, at 12:04 PM, Thomas St?fe wrote: > > > > On Wed 17. Oct 2018 at 21:00, Igor Ignatyev wrote: >> >> EXPECT would just mark a test as failed and allow the method to continue, so rs.release() will be called. > > > Oh, okay. Thanks for clarifying. > >> >> >> test fixture will basically do the same as an RAII object, but it will be a bit more obvious as test preparation and cleaning steps will be separated from test actions, so I'll go w/ setUp/tearDown. > > > Okay fine by me. > > ..thomas > >> >> >> -- Igor >> >> > On Oct 17, 2018, at 11:53 AM, Thomas St?fe wrote: >> > >> > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev wrote: >> >> >> >> Hi Thomas, >> >> >> >> 1st of all, thanks for looking at it, appreciate that. >> >> >> >> it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? >> >> >> >> -- Igor >> > >> > Slight preference for SetUp/TearDown (EXCEPT would quit the testing >> > altogether, wont it?). >> > >> > Lazy me would have just used an RAII object like this: >> > >> > struct Cleaner { >> > ReservedSpace* rs; >> > ~Cleaner() { rs.release(); } >> > }; >> > >> > But I leave it up to you. >> > >> > ... Thomas >> > >> >> >> >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe wrote: >> >>> >> >>> Hi Igor, >> >>> >> >>> test_virtual_space_actual_committed_space() and >> >>> actual_committed_space_one_large_page(): >> >>> >> >>> Won't we leak the reserved space now when we run into an ASSERT and >> >>> then continue with the next gtests? >> >>> >> >>> Cheers, Thomas >> >>> >> >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: >> >>>> >> >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; >> >>>> >> >>>> Hi all, >> >>>> >> >>>> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? >> >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. >> >>>> >> >>>> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. >> >>>> >> >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 >> >>>> testing: >> >>>> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >> >>>> - build w/ precompiled-headers enabled and disabled >> >>>> >> >>>> Thanks, >> >>>> -- Igor >> >> >> > From david.holmes at oracle.com Thu Oct 18 07:27:29 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Oct 2018 17:27:29 +1000 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> Message-ID: Hi Dean, On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: > On 10/17/18 7:07 PM, David Holmes wrote: >> Hi Dean, >> >> This still seems racy to me. We increment all counts under the >> Threads_lock but we still decrement without the Threads_lock. So we >> can lose updates to the perfCounters. >> >> ?117?? _total_threads_count->inc(); >> ?118?? Atomic::inc(&_atomic_threads_count); >> ?119?? int count = _atomic_threads_count; >> <= context switch here >> ?120?? _live_threads_count->set_value(count); >> >> If a second thread now exits while the above thread is descheduled, it >> will decrement _atomic_threads_count and _live_threads_count, but when >> the first thread resumes at line 120 above we will set >> _live_threads_count to the wrong value! >> >> You can't maintain two counters in sync without all changes using >> locking across both. >> > > You're right, I missed that.? I think the right thing to do is call > current_thread_exiting while holding the Threads_lock. > Then we can get rid of the parallel atomic counters.? So, here's one > more try: > > ??? http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ Okay that is the simple and obvious solution that doesn't require split counts. So I have to ask Mandy if she recalls why this approach wasn't taken 15 years ago when the exit counts were added as part of: https://bugs.openjdk.java.net/browse/JDK-4530538 ? Does taking the Threads_lock here cost too much and cause a thread termination bottleneck? Thanks, David ----- > dl > >> David >> >> >> >> On 18/10/2018 8:18 AM, dean.long at oracle.com wrote: >>> On 10/17/18 2:38 PM, Mandy Chung wrote: >>>> >>>> >>>> On 10/17/18 2:13 PM, dean.long at oracle.com wrote: >>>>> On 10/17/18 1:41 PM, Mandy Chung wrote: >>>>>> >>>>>> >>>>>> On 10/16/18 7:33 PM, David Holmes wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> Thanks for tackling this. >>>>>>> >>>>>>> I'm still struggling to fully grasp why we need both the >>>>>>> PerfCounters and the regular counters. I get that we have to >>>>>>> decrement the live counts before ensure_join() has allowed >>>>>>> Thread.join() to return, to ensure that if we then check the >>>>>>> number of threads it has dropped by one. But I don't understand >>>>>>> why that means we need to manage the thread count in two parts. >>>>>>> Particularly as now you don't use the PerfCounter to return the >>>>>>> live count, so it makes me wonder what role the PerfCounter is >>>>>>> playing as it is temporarily inconsistent with the reported live >>>>>>> count? >>>>>> >>>>>> Perf counters were added long time back in JDK 1.4.2 for >>>>>> performance measurement before java.lang.management API. One can >>>>>> use jstat tool to monitor VM perf counters of a running VM.?? One >>>>>> could look into the possibility of deprecating these counters and >>>>>> remove them over time. >>>>>> >>>>>>> On 17/10/2018 9:43 AM, dean.long at oracle.com wrote: >>>>>>> New webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.4/ >>>>>> >>>>>> When the perf counters are updated when a thread is added/removed, >>>>>> it's holding Threads_lock.? Are the asserts in >>>>>> ThreadService::remove_thread necessary? >>>>>> >>>>> >>>>> Not really.? They were intended to catch the case where the atomic >>>>> counters weren't decremented for some reason, not for the perf >>>>> counters. >>>>> Should I remove them? >>>>> >>>> >>>> Hmm... when remove_thread is called but decrement_thread_counts has >>>> not been called. ? It's a bug in thread accounting.? It happens to >>>> have the perf counters that can be compared to assert. It seems not >>>> obvious.? Setting the perf counters same values as >>>> _atomic_threads_count and _atomic_daemon_threads_count makes sense >>>> to me. >>>> >>>> I would opt for removing the asserts but I can't think of an >>>> alternative how to catch the issue you concern about. >>>> >>>>>> For clarify, I think we could simply set _live_threads_count to >>>>>> the value of _atomic_threads_count and set _daemon_threads_count >>>>>> to the value of _atomic_daemon_threads_count. >>>>>> >>>>> >>>>> I think that works, even inside decrement_thread_counts() without >>>>> holding the Threads_lock.? If you agree, I'll make that change. >>>>> >>>> +1 >>>> >>> >>> New webrevs, full and incremental: >>> >>> http://cr.openjdk.java.net/~dlong/8021335/webrev.6/ >>> http://cr.openjdk.java.net/~dlong/8021335/webrev.6.diff/ >>> >>> I like it better without all the asserts too. >>> >>> dl >>> >>>> Mandy >>> > From matthias.baesken at sap.com Thu Oct 18 14:30:20 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 18 Oct 2018 14:30:20 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: Hello, please review my updated webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.1/ - check changed to build number of recently released Windows Server 2019 GA version (17763) - adjusted comments a bit ( see also https://en.wikipedia.org/wiki/Windows_Server_2019 for the list of build numbers ) Thanks, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 12. Oktober 2018 08:57 > To: 'Bob Vandette' > Cc: 'hotspot-dev at openjdk.java.net' ; > 'core-libs-dev at openjdk.java.net' ; > Langer, Christoph > Subject: RE: RFR : 8211106: [windows] Update OS detection code to > recognize Windows Server 2019 > > I got the info that the GA of Windows Server 2019 has build number 17763 > . > Should I update the webrev to check this build number (would mean we do > not detect the preview as Windows Server 2019) , or keep it the way it is ? > > Current webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ > > > Best regards, Matthias > > > > > > > Hello Bob, > > > > > Hi Bob, > > > no problem , we can wait a few days until the final version of Win > > server > > > 2019 is out, and check the buildNumber again : > > > > > > > seems Windows server 2019 is GA now : > > > > > > https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows- > > server-2019-now-generally-available/ > > > > > > We don?t usually add logic to the JDK related to unreleased operating > > system > > > > previews. > > > > > > so I think we can process with the review . > > Still have to find out the buildNumber of the GA version, probably it has > been > > incremented again . > > > > > > > > > > > > Bug/webrev : > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211106 > > > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ > > > > > > > > > > > > > > > Thanks, Matthias From bob.vandette at oracle.com Thu Oct 18 14:35:49 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 18 Oct 2018 10:35:49 -0400 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: It sounds like this is still not the last one but I?m ok with your check since the real GA will be higher and we?re real close. "On October 6, 2018, distribution of Windows version 1809 (build 17763[18] ) was paused while Microsoft is investigating an issue with user data being deleted during an in-place upgrade.[19] It affected systems where a user profile folder (e.g. Documents, Music or Pictures) had been moved to another location, but data was left in the original location.[20] As Windows Server 2019 is based on the Windows version 1809 codebase, it too was removed from distribution.[21] ?? Bob. > On Oct 18, 2018, at 10:30 AM, Baesken, Matthias wrote: > > Hello, please review my updated webrev : > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.1/ > > - check changed to build number of recently released Windows Server 2019 GA version (17763) > - adjusted comments a bit > > > ( see also > > https://en.wikipedia.org/wiki/Windows_Server_2019 > > for the list of build numbers ) > > Thanks, Matthias > > >> -----Original Message----- >> From: Baesken, Matthias >> Sent: Freitag, 12. Oktober 2018 08:57 >> To: 'Bob Vandette' >> Cc: 'hotspot-dev at openjdk.java.net' ; >> 'core-libs-dev at openjdk.java.net' ; >> Langer, Christoph >> Subject: RE: RFR : 8211106: [windows] Update OS detection code to >> recognize Windows Server 2019 >> >> I got the info that the GA of Windows Server 2019 has build number 17763 >> . >> Should I update the webrev to check this build number (would mean we do >> not detect the preview as Windows Server 2019) , or keep it the way it is ? >> >> Current webrev : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ >> >> >> Best regards, Matthias >> >> >> >>> >>> Hello Bob, >>> >>>> Hi Bob, >>>> no problem , we can wait a few days until the final version of Win >>> server >>>> 2019 is out, and check the buildNumber again : >>>> >>> >>> seems Windows server 2019 is GA now : >>> >>> >>> https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows- >>> server-2019-now-generally-available/ >>> >>>>> We don?t usually add logic to the JDK related to unreleased operating >>> system >>>>> previews. >>> >>> >>> so I think we can process with the review . >>> Still have to find out the buildNumber of the GA version, probably it has >> been >>> incremented again . >>> >>>>>> >>>>>> Bug/webrev : >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8211106 >>>>>> >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ >>>>>> >>>>>> >>>>>> Thanks, Matthias > From ioi.lam at oracle.com Thu Oct 18 16:25:50 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 18 Oct 2018 09:25:50 -0700 Subject: RFR (S) Remove SystemDictionary::InitOption enum Message-ID: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8212642 http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/ SystemDictionary::InitOption is no longer needed because we don't support multiple version of the JDK class libraries anymore. All classes preloaded by SystemDictionary must exist. The JVMCI classes were declared using InitOption::Jvmci, so that they are loaded only when EnableJVMCI==true. However, this is rather convoluted and I just added a few null checks instead. Tested using hs tiers 1-5 Thanks - Ioi From volker.simonis at gmail.com Thu Oct 18 16:33:25 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 18 Oct 2018 18:33:25 +0200 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> Message-ID: On Tue, Oct 16, 2018 at 12:30 AM Michael Rasmussen wrote: > > > One simple alternative fix is to disable sharing when some events (like > > CFLH) were enabled. This in fact was the behavior before JDK 9. In JDK > > 9, we enabled CDS for debugging. Looks like there are corner cases that > > are not uncovered by our testing. I'd like to seek opinions and feedback > > on the importance of supporting debugging with CDS enabled. > > Or at least CFLH together with can_generate_all_class_hook_events and can_generate_early_class_hook_events capabilities. > > CDS hasn't been part of our own testing scenarios so far, which is why we haven't notice these issues on our end before now. > We only just discovered it because of user reports about crashes, which turned out to be because the java-11-openjdk package on Fedora comes with pre-generated CDS archive. > > But especially with JEP 341 just around the corner, it makes it very important to find and weed out these bugs now! > @Michael: just wondering if it is possible to redefine j.l.Object/j.l.String/etc. later on after startup has been completed with the help of RedefineClass. Have you tried or woudln't this be useful for you anyway? Thanks, Volker > /Michael From vladimir.kozlov at oracle.com Thu Oct 18 16:50:06 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 18 Oct 2018 09:50:06 -0700 Subject: RFR (S) Remove SystemDictionary::InitOption enum In-Reply-To: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> References: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> Message-ID: <9cc79d53-764b-3da5-c644-e90d2208e896@oracle.com> Good. Thanks, Vladimir On 10/18/18 9:25 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8212642 > http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/ > > SystemDictionary::InitOption is no longer needed because we don't support > multiple version of the JDK class libraries anymore. All classes preloaded > by SystemDictionary must exist. > > The JVMCI classes were declared using InitOption::Jvmci, so that they are loaded > only when EnableJVMCI==true. However, this is rather convoluted and > I just added a few null checks instead. > > Tested using hs tiers 1-5 > > Thanks > - Ioi From claes.redestad at oracle.com Thu Oct 18 17:18:32 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 18 Oct 2018 19:18:32 +0200 Subject: RFR (S) Remove SystemDictionary::InitOption enum In-Reply-To: <9cc79d53-764b-3da5-c644-e90d2208e896@oracle.com> References: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> <9cc79d53-764b-3da5-c644-e90d2208e896@oracle.com> Message-ID: <625fbabf-9089-97ae-bb1b-784ac750c834@oracle.com> +1 Thanks /Claes On 2018-10-18 18:50, Vladimir Kozlov wrote: > Good. > > Thanks, > Vladimir > > On 10/18/18 9:25 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8212642 >> http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/ >> >> >> SystemDictionary::InitOption is no longer needed because we don't >> support >> multiple version of the JDK class libraries anymore. All classes >> preloaded >> by SystemDictionary must exist. >> >> The JVMCI classes were declared using InitOption::Jvmci, so that they >> are loaded >> only when EnableJVMCI==true. However, this is rather convoluted and >> I just added a few null checks instead. >> >> Tested using hs tiers 1-5 >> >> Thanks >> - Ioi From Michael.Rasmussen at roguewave.com Thu Oct 18 17:40:45 2018 From: Michael.Rasmussen at roguewave.com (Michael Rasmussen) Date: Thu, 18 Oct 2018 17:40:45 +0000 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> , Message-ID: From: Volker Simonis > @Michael: just wondering if it is possible to redefine > j.l.Object/j.l.String/etc. later on after startup has been completed > with the help of RedefineClass. Have you tried or woudln't this be > useful for you anyway? For Object we could (we only modify toString and equals there), but otherwise we do hierarchical or structural changes to basically all other classes, which is not possible with a redefineClass. String we could probably do some very big workarounds for to support only redefine for, other primordial classes like Class and Thread we do need, and it simplifies our code a great deal to not have a list of excluded classes to handle differently from everything else. We have yet to come up with a way where we can still support reloading of hierarchy without instrumenting these classes. /Michael From erik.osterlund at oracle.com Thu Oct 18 21:07:55 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 18 Oct 2018 23:07:55 +0200 Subject: RFR: 8212585: Clean up CompiledMethod::oops_reloc_begin() In-Reply-To: <5BC756EE.6040900@oracle.com> References: <5BC756EE.6040900@oracle.com> Message-ID: Hi, I decided to make this less risky for platforms that do not yet need to be scanned concurrently, and hence don't really need to be "fixed". In this new webrev, I decided to check for the frame completed offset if available and further away than verified entry + native jump size. Then it is safe for me to go ahead and scan this stuff concurrently using nmethod entry barriers. Otherwise, the same good old behaviour we used to rely on applies so that we won't get any surprises where we do not want them. Webrev: http://cr.openjdk.java.net/~eosterlund/8212585/webrev.01/ Thanks, /Erik On 2018-10-17 17:36, Erik ?sterlund wrote: > Hi, > > In CompiledMethod::oops_reloc_begin() there is code to prevent looking > for oops at the verified entry of an nmethod that is not entrant, as a > native jump instruction has been overwritten there. Except there would > *never* be any immediate oops there for any compiler, even before > becoming not entrant, with or without OSR compilation, so this special > handling of not entrant vs entrant nmethods is seemingly completely > unnecessary. > > This gets increasingly awkward when oops_do is called concurrently > with concurrent class unloading, where an nmethod is racing to become > not entrant. > > To clean this up, I propose to change the boundary for immediate oop > scanning and start looking for oops only after the frame building is > completed, as there is absolutely no point in looking for oops before > that in the first place. This removes unnecessary workarounds that > exist today, and removes unnecessary races going forward. > > It seems like Graal as JIT also inlines oops into the code stream, but > never sets the CodeOffsets::Frame_Complete in the nmethod. So I > recognize such nmethods and unconditionally start scanning at the > verified entry + native jump instruction size. I spoke to the Graal > folks, and they said they do not put oops in there either. So I think > everyone involved should be happy with this solution. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212585 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212585/webrev.00/ > > Thanks, > /Erik From mandy.chung at oracle.com Fri Oct 19 01:12:31 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Oct 2018 18:12:31 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> Message-ID: On 10/18/18 12:27 AM, David Holmes wrote: > Hi Dean, > > On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: >> >> You're right, I missed that.? I think the right thing to do is call >> current_thread_exiting while holding the Threads_lock. >> Then we can get rid of the parallel atomic counters.? So, here's one >> more try: >> >> ???? http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ > > Okay that is the simple and obvious solution that doesn't require > split counts. So I have to ask Mandy if she recalls why this approach > wasn't taken 15 years ago when the exit counts were added as part of: > It has been so long.? I think it's likely an oversight. > https://bugs.openjdk.java.net/browse/JDK-4530538 ? > > Does taking the Threads_lock here cost too much and cause a thread > termination bottleneck? If the contention on Threads_lock is not high (that seems to me), it should be okay. ? I'm not close to the VM implementation (lot of changes since then) and I don't have a definitive answer unless I study the code closely.?? You and others have a better judgement on this. AFAICT the change is okay. Mandy From david.holmes at oracle.com Fri Oct 19 01:14:57 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Oct 2018 11:14:57 +1000 Subject: RFR (S) Remove SystemDictionary::InitOption enum In-Reply-To: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> References: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> Message-ID: Hi Ioi, Please include the bug number in RFR subject - thanks. On 19/10/2018 2:25 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8212642 > http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/ > > > SystemDictionary::InitOption is no longer needed because we don't support > multiple version of the JDK class libraries anymore. All classes preloaded > by SystemDictionary must exist. In systemDictionary.cpp/hpp: 1974 bool SystemDictionary::resolve_wk_klass(WKID id, int init_opt, TRAPS) { The init_opt parameter is now unused. --- src/hotspot/share/classfile/systemDictionary.hpp 96 // Each well-known class has a short klass name (like object_klass), 97 // a vmSymbol name (like java_lang_Object), and a flag word 98 // that makes some minor distinctions, like whether the klass 99 // is preloaded, optional, release-specific, etc. The comment is no longer accurate/correct. --- src/hotspot/share/runtime/reflection.cpp Why is MagicAccessorImpl still treated as optional? > The JVMCI classes were declared using InitOption::Jvmci, so that they > are loaded > only when EnableJVMCI==true. However, this is rather convoluted and > I just added a few null checks instead. Can't help but think check_klass is redundant now. It would simplify things if it were removed, then you wouldn't need to check is_loaded. The only reason for the check is to avoid the NULL-check in the assertion in check_klass when called from name(). The JVMCI code either has to handle the NULL anyway, or else is never called unless it can't be NULL. Thanks, David > Tested using hs tiers 1-5 > > Thanks > - Ioi From ioi.lam at oracle.com Fri Oct 19 03:43:29 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 18 Oct 2018 20:43:29 -0700 Subject: RFR (S) Remove SystemDictionary::InitOption enum In-Reply-To: References: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> Message-ID: <3cf7882b-9b46-9e23-dd72-eb50c5348e34@oracle.com> Hi David, Updated diff here: http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/delta_1.diff On 10/18/18 6:14 PM, David Holmes wrote: > Hi Ioi, > > Please include the bug number in RFR subject - thanks. > > On 19/10/2018 2:25 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8212642 >> http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/ >> >> >> SystemDictionary::InitOption is no longer needed because we don't >> support >> multiple version of the JDK class libraries anymore. All classes >> preloaded >> by SystemDictionary must exist. > > In systemDictionary.cpp/hpp: > > 1974 bool SystemDictionary::resolve_wk_klass(WKID id, int init_opt, > TRAPS) { > > The init_opt parameter is now unused. > Removed. > --- > > src/hotspot/share/classfile/systemDictionary.hpp > > ?96 // Each well-known class has a short klass name (like object_klass), > ?97 // a vmSymbol name (like java_lang_Object), and a flag word > ?98 // that makes some minor distinctions, like whether the klass > ?99 // is preloaded, optional, release-specific, etc. > > The comment is no longer accurate/correct. > Fixed. > --- > > src/hotspot/share/runtime/reflection.cpp > > Why is MagicAccessorImpl still treated as optional? > > Reflection::verify_class_access() is called for every class that has a super class. So it gets called for String, which is loaded immediately after Object. At this point, MagicAccessorImpl is not loaded yet, so the old code marks it as "optional" to get around this. The old code is further convoluted by the fact that current_class->is_subclass_of(NULL) happily returns false :-( I think the new code expresses what we want -- if MagicAccessorImpl is not loaded yet, obviously none of its subclasses are loaded, so there's nothing to check. >> The JVMCI classes were declared using InitOption::Jvmci, so that they >> are loaded >> only when EnableJVMCI==true. However, this is rather convoluted and >> I just added a few null checks instead. > > Can't help but think check_klass is redundant now. It would simplify > things if it were removed, then you wouldn't need to check is_loaded. > The only reason for the check is to avoid the NULL-check in the > assertion in check_klass when called from name(). The JVMCI code > either has to handle the NULL anyway, or else is never called unless > it can't be NULL. > check_klass() probably has a purpose -- before all the preloaded classes are resolved, someone may call SystemDictionary::xxx_klass() and store the result somewhere, and will get a crash only when the stored result is used later. A lot of code is executed inside SystemDictionary::resolve_preloaded_classes -- class file parsing, JVMTI, verification if requested by the user, etc. So I think it's better to preserve this check. Thanks - Ioi > Thanks, > David > >> Tested using hs tiers 1-5 >> >> Thanks >> - Ioi From david.holmes at oracle.com Fri Oct 19 03:55:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Oct 2018 13:55:12 +1000 Subject: RFR (S) Remove SystemDictionary::InitOption enum In-Reply-To: <3cf7882b-9b46-9e23-dd72-eb50c5348e34@oracle.com> References: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> <3cf7882b-9b46-9e23-dd72-eb50c5348e34@oracle.com> Message-ID: <922f2ec1-b298-308a-3397-e49039666eda@oracle.com> Hi Ioi, On 19/10/2018 1:43 PM, Ioi Lam wrote: > Hi David, > > Updated diff here: > > http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/delta_1.diff Updates look good - thanks. More below ... > > > On 10/18/18 6:14 PM, David Holmes wrote: >> Hi Ioi, >> >> Please include the bug number in RFR subject - thanks. >> >> On 19/10/2018 2:25 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8212642 >>> http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/ >>> >>> >>> SystemDictionary::InitOption is no longer needed because we don't >>> support >>> multiple version of the JDK class libraries anymore. All classes >>> preloaded >>> by SystemDictionary must exist. >> >> In systemDictionary.cpp/hpp: >> >> 1974 bool SystemDictionary::resolve_wk_klass(WKID id, int init_opt, >> TRAPS) { >> >> The init_opt parameter is now unused. >> > > Removed. > >> --- >> >> src/hotspot/share/classfile/systemDictionary.hpp >> >> ?96 // Each well-known class has a short klass name (like object_klass), >> ?97 // a vmSymbol name (like java_lang_Object), and a flag word >> ?98 // that makes some minor distinctions, like whether the klass >> ?99 // is preloaded, optional, release-specific, etc. >> >> The comment is no longer accurate/correct. >> > > Fixed. > >> --- >> >> src/hotspot/share/runtime/reflection.cpp >> >> Why is MagicAccessorImpl still treated as optional? >> >> > Reflection::verify_class_access() is called for every class that has a > super class. So it gets called for String, which is loaded immediately > after Object. At this point, MagicAccessorImpl is not loaded yet, so the > old code marks it as "optional" to get around this. But we should never get that far for core classes: 499 if ((current_class == NULL) || 500 (current_class == new_class) || 501 is_same_class_package(current_class, new_class)) { 502 return ACCESS_OK; 503 } as they are in the same package. ?? > The old code is > further convoluted by the fact that current_class->is_subclass_of(NULL) > happily returns false :-( That seems perfectly reasonable to me. What am I missing? :) > I think the new code expresses what we want -- if MagicAccessorImpl is > not loaded yet, obviously none of its subclasses are loaded, so there's > nothing to check. > >>> The JVMCI classes were declared using InitOption::Jvmci, so that they >>> are loaded >>> only when EnableJVMCI==true. However, this is rather convoluted and >>> I just added a few null checks instead. >> >> Can't help but think check_klass is redundant now. It would simplify >> things if it were removed, then you wouldn't need to check is_loaded. >> The only reason for the check is to avoid the NULL-check in the >> assertion in check_klass when called from name(). The JVMCI code >> either has to handle the NULL anyway, or else is never called unless >> it can't be NULL. >> > check_klass() probably has a purpose -- before all the preloaded classes > are resolved, someone may call SystemDictionary::xxx_klass() and store > the result somewhere, and will get a crash only when the stored result > is used later. > > A lot of code is executed inside > SystemDictionary::resolve_preloaded_classes -- class file parsing, > JVMTI, verification if requested by the user, etc. So I think it's > better to preserve this check. Ok. I suspect there will be other NULL checks in various places anyway, but this is not a big deal. Thanks, David > Thanks > - Ioi >> Thanks, >> David >> >>> Tested using hs tiers 1-5 >>> >>> Thanks >>> - Ioi > From ioi.lam at oracle.com Fri Oct 19 04:26:07 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 18 Oct 2018 21:26:07 -0700 Subject: RFR (S) Remove SystemDictionary::InitOption enum In-Reply-To: <922f2ec1-b298-308a-3397-e49039666eda@oracle.com> References: <0e3f8e0b-6ef6-317c-c869-404b56a1b839@oracle.com> <3cf7882b-9b46-9e23-dd72-eb50c5348e34@oracle.com> <922f2ec1-b298-308a-3397-e49039666eda@oracle.com> Message-ID: <696b5cfc-62be-33f8-8457-6c7a83ad22c2@oracle.com> Hi David, Thanks for the review. Comments below: On 10/18/18 8:55 PM, David Holmes wrote: > Hi Ioi, > > On 19/10/2018 1:43 PM, Ioi Lam wrote: >> Hi David, >> >> Updated diff here: >> >> http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/delta_1.diff > > > Updates look good - thanks. > > More below ... > >> >> >> On 10/18/18 6:14 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> Please include the bug number in RFR subject - thanks. >>> >>> On 19/10/2018 2:25 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8212642 >>>> http://cr.openjdk.java.net/~iklam/jdk12/8212642-remove-sysdict-initoption.v01/ >>>> >>>> >>>> SystemDictionary::InitOption is no longer needed because we don't >>>> support >>>> multiple version of the JDK class libraries anymore. All classes >>>> preloaded >>>> by SystemDictionary must exist. >>> >>> In systemDictionary.cpp/hpp: >>> >>> 1974 bool SystemDictionary::resolve_wk_klass(WKID id, int init_opt, >>> TRAPS) { >>> >>> The init_opt parameter is now unused. >>> >> >> Removed. >> >>> --- >>> >>> src/hotspot/share/classfile/systemDictionary.hpp >>> >>> ?96 // Each well-known class has a short klass name (like >>> object_klass), >>> ?97 // a vmSymbol name (like java_lang_Object), and a flag word >>> ?98 // that makes some minor distinctions, like whether the klass >>> ?99 // is preloaded, optional, release-specific, etc. >>> >>> The comment is no longer accurate/correct. >>> >> >> Fixed. >> >>> --- >>> >>> src/hotspot/share/runtime/reflection.cpp >>> >>> Why is MagicAccessorImpl still treated as optional? >>> >>> >> Reflection::verify_class_access() is called for every class that has >> a super class. So it gets called for String, which is loaded >> immediately after Object. At this point, MagicAccessorImpl is not >> loaded yet, so the old code marks it as "optional" to get around this. > > But we should never get that far for core classes: > > ?499?? if ((current_class == NULL) || > ?500?????? (current_class == new_class) || > ?501?????? is_same_class_package(current_class, new_class)) { > ?502???? return ACCESS_OK; > ?503?? } > > as they are in the same package. ?? > Oh, I missed a few steps: String implements java.io.Serializable. Which is a subclass of j.l.Object but a different package. >> The old code is further convoluted by the fact that >> current_class->is_subclass_of(NULL) happily returns false :-( > > That seems perfectly reasonable to me. What am I missing? :) > > >> I think the new code expresses what we want -- if MagicAccessorImpl >> is not loaded yet, obviously none of its subclasses are loaded, so >> there's nothing to check. >> >>>> The JVMCI classes were declared using InitOption::Jvmci, so that >>>> they are loaded >>>> only when EnableJVMCI==true. However, this is rather convoluted and >>>> I just added a few null checks instead. >>> >>> Can't help but think check_klass is redundant now. It would simplify >>> things if it were removed, then you wouldn't need to check >>> is_loaded. The only reason for the check is to avoid the NULL-check >>> in the assertion in check_klass when called from name(). The JVMCI >>> code either has to handle the NULL anyway, or else is never called >>> unless it can't be NULL. >>> >> check_klass() probably has a purpose -- before all the preloaded >> classes are resolved, someone may call SystemDictionary::xxx_klass() >> and store the result somewhere, and will get a crash only when the >> stored result is used later. >> >> A lot of code is executed inside >> SystemDictionary::resolve_preloaded_classes -- class file parsing, >> JVMTI, verification if requested by the user, etc. So I think it's >> better to preserve this check. > > Ok. I suspect there will be other NULL checks in various places > anyway, but this is not a big deal. > Or, perhaps the current code is so disciplined that it never touches a class that's not yet preloaded (with MagicAccessorImpl being the only exception :-) I am sure if you reorder the WK_KLASSES list all hell will break. Thanks - Ioi > Thanks, > David > >> Thanks >> - Ioi >>> Thanks, >>> David >>> >>>> Tested using hs tiers 1-5 >>>> >>>> Thanks >>>> - Ioi >> From matthias.baesken at sap.com Fri Oct 19 09:22:20 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 19 Oct 2018 09:22:20 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: > >java_props_md.c: 564. > >Please move the buildNumber check inside the "case 0:" above. >Then it won't need the extra check for minorVersion == 0. > Thanks for the remarks , I updated my changes , new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.2/ Best regards, Matthias From: Bob Vandette Sent: Donnerstag, 18. Oktober 2018 16:36 To: Baesken, Matthias Cc: hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Langer, Christoph Subject: Re: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 It sounds like this is still not the last one but I?m ok with your check since the real GA will be higher and we?re real close. "On October 6, 2018, distribution of Windows version 1809 (build 17763[18]) was paused while Microsoft is investigating an issue with user data being deleted during an in-place upgrade.[19] It affected systems where a user profile folder (e.g. Documents, Music or Pictures) had been moved to another location, but data was left in the original location.[20]As Windows Server 2019 is based on the Windows version 1809 codebase, it too was removed from distribution.[21]?? Bob. On Oct 18, 2018, at 10:30 AM, Baesken, Matthias > wrote: Hello, please review my updated webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.1/ - check changed to build number of recently released Windows Server 2019 GA version (17763) - adjusted comments a bit ( see also https://en.wikipedia.org/wiki/Windows_Server_2019 for the list of build numbers ) Thanks, Matthias -----Original Message----- From: Baesken, Matthias Sent: Freitag, 12. Oktober 2018 08:57 To: 'Bob Vandette' > Cc: 'hotspot-dev at openjdk.java.net' >; 'core-libs-dev at openjdk.java.net' >; Langer, Christoph > Subject: RE: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 I got the info that the GA of Windows Server 2019 has build number 17763 . Should I update the webrev to check this build number (would mean we do not detect the preview as Windows Server 2019) , or keep it the way it is ? Current webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ Best regards, Matthias Hello Bob, Hi Bob, no problem , we can wait a few days until the final version of Win server 2019 is out, and check the buildNumber again : seems Windows server 2019 is GA now : https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows- server-2019-now-generally-available/ We don?t usually add logic to the JDK related to unreleased operating system previews. so I think we can process with the review . Still have to find out the buildNumber of the GA version, probably it has been incremented again . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8211106 http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.0/webrev/ Thanks, Matthias From Alan.Bateman at oracle.com Fri Oct 19 10:05:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Oct 2018 11:05:29 +0100 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: On 19/10/2018 10:22, Baesken, Matthias wrote: > Thanks for the remarks , I updated my changes , new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.2/ > Do you know if there is anything on the Microsoft site to explain why the minor version wasn't bumped? It looks unusual to have both Windows Server 2016 and 2019 report the same major and minor versions. -Alan From weijun.wang at oracle.com Fri Oct 19 10:18:32 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 19 Oct 2018 18:18:32 +0800 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: According to https://en.wikipedia.org/wiki/Windows_Server: Microsoft releases Windows Server in a LTS channel every few years named Windows Server 2016, Windows Server 2019, ... It also releases in a semi-annual channel named Windows Server, version 1803, 1809, ... I'm not sure if we can find out from which channel a system is from simply by looking at the version number. --Max > On Oct 19, 2018, at 6:05 PM, Alan Bateman wrote: > > On 19/10/2018 10:22, Baesken, Matthias wrote: >> Thanks for the remarks , I updated my changes , new webrev : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.2/ >> > Do you know if there is anything on the Microsoft site to explain why the minor version wasn't bumped? It looks unusual to have both Windows Server 2016 and 2019 report the same major and minor versions. > > -Alan From matthias.baesken at sap.com Fri Oct 19 12:11:14 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 19 Oct 2018 12:11:14 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: Hi Alan, I agree it is a pity that the minor version (dwMinorVersion) was not increased . Looking at the build number was not my preferred way to handle this . My understanding (checked with our internal SAP-MS porting guys) is that the buildNumber of Windows server 2016 will not get larger than the minimum introduced buildNumber of Windows server 2019 . Best regards, Matthias > -----Original Message----- > From: Weijun Wang > Sent: Freitag, 19. Oktober 2018 12:19 > To: Alan Bateman > Cc: Baesken, Matthias ; Bob Vandette > ; roger.riggs at oracle.com; hotspot-dev Source > Developers ; core-libs- > dev at openjdk.java.net > Subject: Re: RFR : 8211106: [windows] Update OS detection code to > recognize Windows Server 2019 > > According to https://en.wikipedia.org/wiki/Windows_Server: > > Microsoft releases Windows Server in a LTS channel every few years named > Windows Server 2016, Windows Server 2019, ... > > It also releases in a semi-annual channel named Windows Server, version > 1803, 1809, ... > > I'm not sure if we can find out from which channel a system is from simply by > looking at the version number. > > --Max > > > On Oct 19, 2018, at 6:05 PM, Alan Bateman > wrote: > > > > On 19/10/2018 10:22, Baesken, Matthias wrote: > >> Thanks for the remarks , I updated my changes , new webrev : > >> > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211106.2/ > >> > > Do you know if there is anything on the Microsoft site to explain why the > minor version wasn't bumped? It looks unusual to have both Windows > Server 2016 and 2019 report the same major and minor versions. > > > > -Alan From erik.osterlund at oracle.com Fri Oct 19 14:22:50 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 19 Oct 2018 16:22:50 +0200 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent Message-ID: <5BC9E8BA.50407@oracle.com> Hi, Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. Bug ID: https://bugs.openjdk.java.net/browse/JDK-8209189 Webrev: cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ Thanks, /Erik From bsrbnd at gmail.com Fri Oct 19 15:42:59 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Fri, 19 Oct 2018 17:42:59 +0200 Subject: Bit set intrinsic (was: Primitive boolean array packing) In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Mon, 8 Oct 2018 at 10:53, Andrew Haley wrote: > > On 10/07/2018 08:54 PM, Aleksey Shipilev wrote: > > > Of course, if you do packed boolean[] with locked/CAS-ed writes or > > otherwise guarantee the absence of word tearing, it does not break > > the spec. > > Exactly. it's an interesting idea for large bit vectors, and it could > well generate good code. However, it probably makes more sense to > implement it as a bit vector class with suitable helper intrinsics. So, I turned the existing prototype into a bit set intrinsic (System.setBit()) as I agree with Andrew that it would make more sense than modifying baload/bastore: http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.01/ The current x86_64 implementation uses BTS/BTR (bit set and reset) instructions as Roman suggested with the LOCK prefix to assure atomicity and thus avoiding any word-tearing problem that Aleksey mentioned. Note that BTS/BTR are locking 32-bit words but AND/OR (8-bit) could be used instead, also in conjunction with LOCK. I tried example [1] with both synchronized BitSet and dedicated intrinsic on x86_64 (8 cores) which revealed that the intrinsic would be at least 2-3x faster than the synchronized set. Then, to the question of how much do we need this, BitSet usages in the JDK is a beginning of answer: $ find ./src/ -name '*.java' -type f -print | xargs grep -e "BitSet" -e "BitArray" There are also other places like javac flags [2] or EnumSet [3] where the same thing is remade by hand. Finally, note that using the intrinsic inside EnumSet would probably make it 2-3x faster than using a SynchronizedCollection [4] in multi-threaded environments. While I agree with Maurizio that Panama structs might be well suited in some situations requiring a rigid layout like javac flags, I think they are only complementary to bit arrays but not a replacement (see EnumSet, etc...). I've seen that Panama is also about exploring hardware optimizations, so maybe the latest intrinsic prototype could be a good starting point for this (hotspot:tier1 being OK)? In any case, feedbacks about this intrinsic would be welcome. Thanks, Bernard [1] https://docs.oracle.com/javase/specs/jls/se11/html/jls-17.html#jls-17.6 [2] http://mail.openjdk.java.net/pipermail/compiler-dev/2018-September/012504.html [3] http://hg.openjdk.java.net/jdk/jdk/file/cb94f3a51aed/src/java.base/share/classes/java/util/JumboEnumSet.java#l44 [4] http://hg.openjdk.java.net/jdk/jdk/file/cb94f3a51aed/src/java.base/share/classes/java/util/Collections.java#l1998 From serguei.spitsyn at oracle.com Fri Oct 19 16:43:35 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 19 Oct 2018 09:43:35 -0700 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: <8af982c0-8e2f-a0fc-cd2c-2f35ffd47480@oracle.com> References: <8af982c0-8e2f-a0fc-cd2c-2f35ffd47480@oracle.com> Message-ID: Hi Karen and Lois, I have just one official review from David. May I count your internal reviews as official? Do we need the Compiler team to review this as well? Thanks, Serguei On 10/16/18 18:29, David Holmes wrote: > Bizarre! > > Looks good - officially Reviewed again! > > Thanks for fixing Serguei! > > David > > On 17/10/2018 4:34 AM, serguei.spitsyn at oracle.com wrote: >> Re-sending my RFR. >> Behive managed not to deliver it to the target mailing lists. >> David got it because he was on the cc-list. >> >> >> Please, review a fix for: >> https://bugs.openjdk.java.net/browse/JDK-8024368 >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ >> >> >> Summary: >> >> ?? Private method should not allocate vtable indeces. >> ?? The fix tweaks several conditions in the functions: >> klassVtable::update_inherited_vtable() and >> klassVtable::needs_new_vtable_entry(), >> ?? adds a couple of asserts and makes several comments up-to-date. >> >> ?? The fix was preliminary reviewed by Karen, David H. and Lois. >> >> Testing: >> >> ?? The fix was verified with >> runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} >> ?? and vmTestbase/vm/runtime/defmeth tests. >> ?? Also, I ran a mach5 job with the flags: >> tier1,tier2,hs-tier3,hs-tier4,hs-tier5. >> >> ?? By suggestion from David, it was also tested with the -Xlog output >> for vtables before and after the fix. >> ?? I've done this comparison (awk was used to get rid of the first >> column with the time stamps). >> ?? It shows that private methods were included into vtable before the >> fix but not included after. >> ?? Please, see one fragment as an example: >> >> ??? 166c166 >> ??? > size 17 >> ??? --- >> ??? >? copy vtable from java.lang.Object to java.lang.StackFrameInfo >> size 16 >> ??? 180d179 >> ??? > at index 16, flags: private >> ??? 182c181 >> ??? > java.lang.LiveStackFrameInfo size 17 >> ??? --- >> ??? >? copy vtable from java.lang.StackFrameInfo to >> java.lang.LiveStackFrameInfo size 16 >> ??? 188,201c187,191 >> ??? > java.lang.StackStreamFactory$AbstractStackWalker size 18 >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.close()V at index 5, >> flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I >> at index 6, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >> at index 7, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >> at index 8, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >> index 9, flags: protected abstract >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >> at index 10, flags: protected >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z >> at index 11, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; >> at index 12, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at >> index 13, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; >> at index 14, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I >> at index 15, flags: private >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I >> at index 16, flags: private native >> ??? > java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; >> at index 17, flags: private native >> ??? --- >> ??? >? copy vtable from java.lang.Object to >> java.lang.StackStreamFactory$AbstractStackWalker size 9 >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >> at index 5, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >> at index 6, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >> index 7, flags: protected abstract >> ??? >? adding >> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >> at index 8, flags: protected >> >> >> Thanks, >> Serguei >> From hohensee at amazon.com Fri Oct 19 18:40:12 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 19 Oct 2018 18:40:12 +0000 Subject: RFR (XXS): 8212698: Minor g1 #include changes and memoryService.hpp copyright date update Message-ID: <7693E8BA-7A72-4D9E-B8B0-3E7566D00050@amazon.com> A trivial patch to fix 3 misc. issues. Bug: https://bugs.openjdk.java.net/browse/JDK-8212698 Webrev: http://cr.openjdk.java.net/~phh/8212698/webrev.00/ Thanks, Paul From dean.long at oracle.com Fri Oct 19 19:07:00 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 19 Oct 2018 12:07:00 -0700 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: <8af982c0-8e2f-a0fc-cd2c-2f35ffd47480@oracle.com> Message-ID: It sounds like the vtable slot was there for backwards compatibility, but I don't understand how a vtable slot for a private method ensured compatibility (how does it affect behavior?), so I don't understand how we continue to ensure backwards compatibility without it.? Also, which tests check for the desired backwards compatibility? dl On 10/19/18 9:43 AM, serguei.spitsyn at oracle.com wrote: > Hi Karen and Lois, > > I have just one official review from David. > May I count your internal reviews as official? > Do we need the Compiler team to review this as well? > > Thanks, > Serguei > > On 10/16/18 18:29, David Holmes wrote: >> Bizarre! >> >> Looks good - officially Reviewed again! >> >> Thanks for fixing Serguei! >> >> David >> >> On 17/10/2018 4:34 AM, serguei.spitsyn at oracle.com wrote: >>> Re-sending my RFR. >>> Behive managed not to deliver it to the target mailing lists. >>> David got it because he was on the cc-list. >>> >>> >>> Please, review a fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8024368 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ >>> >>> >>> >>> Summary: >>> >>> ?? Private method should not allocate vtable indeces. >>> ?? The fix tweaks several conditions in the functions: >>> klassVtable::update_inherited_vtable() and >>> klassVtable::needs_new_vtable_entry(), >>> ?? adds a couple of asserts and makes several comments up-to-date. >>> >>> ?? The fix was preliminary reviewed by Karen, David H. and Lois. >>> >>> Testing: >>> >>> ?? The fix was verified with >>> runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} >>> ?? and vmTestbase/vm/runtime/defmeth tests. >>> ?? Also, I ran a mach5 job with the flags: >>> tier1,tier2,hs-tier3,hs-tier4,hs-tier5. >>> >>> ?? By suggestion from David, it was also tested with the -Xlog >>> output for vtables before and after the fix. >>> ?? I've done this comparison (awk was used to get rid of the first >>> column with the time stamps). >>> ?? It shows that private methods were included into vtable before >>> the fix but not included after. >>> ?? Please, see one fragment as an example: >>> >>> ??? 166c166 >>> ??? >> size 17 >>> ??? --- >>> ??? >? copy vtable from java.lang.Object to java.lang.StackFrameInfo >>> size 16 >>> ??? 180d179 >>> ??? >> java.lang.StackFrameInfo.ensureRetainClassRefEnabled()V at index 16, >>> flags: private >>> ??? 182c181 >>> ??? >> java.lang.LiveStackFrameInfo size 17 >>> ??? --- >>> ??? >? copy vtable from java.lang.StackFrameInfo to >>> java.lang.LiveStackFrameInfo size 16 >>> ??? 188,201c187,191 >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker size 18 >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.close()V at index >>> 5, flags: private >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I >>> at index 6, flags: private >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >>> at index 7, flags: protected abstract >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >>> at index 8, flags: protected abstract >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >>> index 9, flags: protected abstract >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >>> at index 10, flags: protected >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z >>> at index 11, flags: private >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; >>> at index 12, flags: private >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at >>> index 13, flags: private >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; >>> at index 14, flags: private >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I >>> at index 15, flags: private >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I >>> at index 16, flags: private native >>> ??? >> java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; >>> at index 17, flags: private native >>> ??? --- >>> ??? >? copy vtable from java.lang.Object to >>> java.lang.StackStreamFactory$AbstractStackWalker size 9 >>> ??? >? adding >>> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >>> at index 5, flags: protected abstract >>> ??? >? adding >>> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >>> at index 6, flags: protected abstract >>> ??? >? adding >>> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >>> index 7, flags: protected abstract >>> ??? >? adding >>> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >>> at index 8, flags: protected >>> >>> >>> Thanks, >>> Serguei >>> > From serguei.spitsyn at oracle.com Fri Oct 19 19:13:49 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 19 Oct 2018 12:13:49 -0700 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: <8af982c0-8e2f-a0fc-cd2c-2f35ffd47480@oracle.com> Message-ID: Hi Dean, Thank you for looking at it and the question! Karen told that it is was a mistake back in 2005 to add non-final private methods into vtable. I don't know anything about compatibility here and related tests. Keren is traveling now and will be able to clarify it next week only. Thanks, Serguei On 10/19/18 12:07 PM, dean.long at oracle.com wrote: > It sounds like the vtable slot was there for backwards compatibility, > but I don't understand how a vtable slot for a private method ensured > compatibility (how does it affect behavior?), so I don't understand > how we continue to ensure backwards compatibility without it.? Also, > which tests check for the desired backwards compatibility? > > dl > > On 10/19/18 9:43 AM, serguei.spitsyn at oracle.com wrote: >> Hi Karen and Lois, >> >> I have just one official review from David. >> May I count your internal reviews as official? >> Do we need the Compiler team to review this as well? >> >> Thanks, >> Serguei >> >> On 10/16/18 18:29, David Holmes wrote: >>> Bizarre! >>> >>> Looks good - officially Reviewed again! >>> >>> Thanks for fixing Serguei! >>> >>> David >>> >>> On 17/10/2018 4:34 AM, serguei.spitsyn at oracle.com wrote: >>>> Re-sending my RFR. >>>> Behive managed not to deliver it to the target mailing lists. >>>> David got it because he was on the cc-list. >>>> >>>> >>>> Please, review a fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8024368 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ >>>> >>>> >>>> >>>> Summary: >>>> >>>> ?? Private method should not allocate vtable indeces. >>>> ?? The fix tweaks several conditions in the functions: >>>> klassVtable::update_inherited_vtable() and >>>> klassVtable::needs_new_vtable_entry(), >>>> ?? adds a couple of asserts and makes several comments up-to-date. >>>> >>>> ?? The fix was preliminary reviewed by Karen, David H. and Lois. >>>> >>>> Testing: >>>> >>>> ?? The fix was verified with >>>> runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} >>>> ?? and vmTestbase/vm/runtime/defmeth tests. >>>> ?? Also, I ran a mach5 job with the flags: >>>> tier1,tier2,hs-tier3,hs-tier4,hs-tier5. >>>> >>>> ?? By suggestion from David, it was also tested with the -Xlog >>>> output for vtables before and after the fix. >>>> ?? I've done this comparison (awk was used to get rid of the first >>>> column with the time stamps). >>>> ?? It shows that private methods were included into vtable before >>>> the fix but not included after. >>>> ?? Please, see one fragment as an example: >>>> >>>> ??? 166c166 >>>> ??? >>> java.lang.StackFrameInfo size 17 >>>> ??? --- >>>> ??? >? copy vtable from java.lang.Object to >>>> java.lang.StackFrameInfo size 16 >>>> ??? 180d179 >>>> ??? >>> java.lang.StackFrameInfo.ensureRetainClassRefEnabled()V at index >>>> 16, flags: private >>>> ??? 182c181 >>>> ??? >>> java.lang.LiveStackFrameInfo size 17 >>>> ??? --- >>>> ??? >? copy vtable from java.lang.StackFrameInfo to >>>> java.lang.LiveStackFrameInfo size 16 >>>> ??? 188,201c187,191 >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker size 18 >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.close()V at index >>>> 5, flags: private >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I >>>> at index 6, flags: private >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >>>> at index 7, flags: protected abstract >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >>>> at index 8, flags: protected abstract >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >>>> index 9, flags: protected abstract >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >>>> at index 10, flags: protected >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z >>>> at index 11, flags: private >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; >>>> at index 12, flags: private >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at >>>> index 13, flags: private >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; >>>> at index 14, flags: private >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I >>>> at index 15, flags: private >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I >>>> at index 16, flags: private native >>>> ??? >>> java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; >>>> at index 17, flags: private native >>>> ??? --- >>>> ??? >? copy vtable from java.lang.Object to >>>> java.lang.StackStreamFactory$AbstractStackWalker size 9 >>>> ??? >? adding >>>> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >>>> at index 5, flags: protected abstract >>>> ??? >? adding >>>> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >>>> at index 6, flags: protected abstract >>>> ??? >? adding >>>> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >>>> index 7, flags: protected abstract >>>> ??? >? adding >>>> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >>>> at index 8, flags: protected >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >> > From kim.barrett at oracle.com Fri Oct 19 20:47:24 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 19 Oct 2018 16:47:24 -0400 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread Message-ID: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> Please review this change to the Service thread to delete empty OopStorage blocks when needed. As part of this, eliminated delete_empty_blocks_safepoint, and renamed delete_empty_blocks_concurrent to remove the now redundant _concurrent suffix. Also added a predicate for the Service thread's use, to test whether there is cleanup work to be done. The previously unused empty block deletion has been revised for this new usage. This includes making loops safepoint polite. Various parts of OopStorage are now aware of the Service thread cleanup and notify that thread when appropriate. (allocate's obtaining the block to allocate from was refactored to make it easier to insert that notification.) As part of this, OopStorage::Block::release_entries now takes the owning storage object as an argument (rather than a pointer to its _deferred_updates list). This allows release_entries to perform additional operations on the owner (so long as C++ DR45 has been fixed, and we've added more AIX-specific workarounds for that). While touching the Service thread, fixed Service_lock locking to use use MonitorLockerEx rather than MutexLockerEx. CR: https://bugs.openjdk.java.net/browse/JDK-8210986 Webrev: http://cr.openjdk.java.net/~kbarrett/8210986/open.00/ Testing: Mach5 tier1-5. Ran some performance tests to verify no regressions due to additional load on the Service thread. From jiangli.zhou at oracle.com Fri Oct 19 20:48:30 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 19 Oct 2018 13:48:30 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: <5493aca2-f67b-4e9f-1f10-02511d701eca@oracle.com> References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> <5493aca2-f67b-4e9f-1f10-02511d701eca@oracle.com> Message-ID: <4e474a5e-9d65-ca27-94ac-e04a5aafbb0f@oracle.com> Hi Ioi, On 10/17/18 10:31 PM, Ioi Lam wrote: > On 10/17/18 7:39 PM, Jiangli Zhou wrote: > >> On 10/17/18 4:54 PM, Ioi Lam wrote: >> >>> Hi Michael & Volker, >>> >>> >>> Thanks for the bug report and the initial investigation! >>> >>> Following Volker's investigation, I tried to make a less drastic fix >>> (than the "nuclear option" of disable CDS entirely), but run into a >>> lot of >>> issues when class-file-load-hook is specified: >>> >>> >>> 1. The shared heap objects are mapped at JVM start-up. At this >>> point, we don't know if any of the archived object's class may >>> be replaced by the CFLH. >> I've given more thoughts to this. Besides Strings, other archived >> objects are already disabled automatically. Archived mirrors and cp >> reference arrays are not installed if their archived class are not >> used. The archived subg-raphs are not used if any of the classes (for >> objects in the subgraphs) are not the same as the archived ones. > > Thanks for the clarification. This makes the patch simpler. > >>> >>> 2. If java.lang.{Object, Cloneable, Serializable} are replaced >>> by CFLH, we cannot use the archived primitive arrays. >>> >>> 3. There's a lot of code that assumes if UseSharedSpaces==true, >>> certain InstanceKlasses must be loaded from CDS. E.g., >>> ?? JavaClasses::compute_offsets() does nothing if >>> UseSharedSpaces==true, >>> ?? because it assumes that offsets calculated during dump time will >>> match >>> ?? the class loaded at runtime. >> Yes, the assumption is baked in our code. We'd have to exam each >> UseSharedSpaces check if we want to support the behavior. > > I think all of the UseSharedSpaces assumptions affect only classes > loaded inside SystemDictionary::resolve_preloaded_classes(), which is > called in the JVMTI "early" stage. So I've changed the code to disable > CDS only like this (as Michael suggested): > > ? if (JvmtiExport::should_post_class_file_load_hook() && > JvmtiExport::early_class_hook_env()) { > ??? // CDS assumes that no classes resolved in > SystemDictionary::resolve_preloaded_classes > ??? // are replaced at runtime by JVMTI ClassFileLoadHook. All of > those classes are resolved > ??? // during the JVMTI "early" stage, so we're OK if > JvmtiExport::early_class_hook_env() > ??? // is not specified by the agent(s). > ??? FileMapInfo::fail_continue("CDS is disabled because early JVMTI > ClassFileLoadHook is in use."); > ??? return false; > ? } > > Updated webrev: > > http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v02/ > > > I'll add more asserts and test cases to validate the patch. Do you have an updated webev with more test cases ready, or you are still working on it? We should consider including more classes from the default classlist in the test. Archived classes loaded during both 'early' phase and after should be tested. We want to make sure with shared archive being used, the visible behavior should be the same as with -Xshare:off (no crash, no assert, no different exception) when working with a JVMTI agent. > >>> >>> >>> I have developed a jtreg test case for this, and attempted to do a >>> less drastic fix, but I am nowhere near done and it's already messy: >>> >>> http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v00/ >>> >>> >>> >>> Instead, I think we should do this: >>> >>> http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v01/ >>> >> This would brings back the behavior of JDK-8141341. To be more >> cautious this time, is there any case that's covered by the check for >> can_generate_all_class_hook_events/can_generate_early_class_hook_events >> capabilities but not for >> JvmtiExport::should_post_class_file_load_hook()? >> > > I looked at JvmtiClassFileLoadHookPoster::post_to_env(). The > "preloaded" classes can be replaced only if at least one agent has > > (1) set the following 2 capabilities: > > ??? caps.can_generate_all_class_hook_events = 1; > ??? caps.can_generate_early_class_hook_events = 1; > > AND > > (2) Specified JVMTI_EVENT_CLASS_FILE_LOAD_HOOK > > See > > http://hg.openjdk.java.net/jdk/jdk/file/672bc2213cef/src/hotspot/share/prims/jvmtiExport.cpp#l932 > Ok, thanks. To make sure we don't miss any corner cases, we should include serviceability-dev mailing list for the code review also. > > >> As part of the JDK-8141341, we also stop loading any additional >> archived classes if JvmtiExport::should_post_class_file_load_hook() >> is set after mapping the shared archive. Should we do that also if we >> are going with this approach? >> > > I don't think that's necessary. It should be OK to replace any classes > that are not in the "preloaded" list. > > Originally (before JDK 9) we would disable CDS so that classes are > loaded normally, in order for the class file data to be posted to the > CFLH. > > However, since JDK 9, we have added the ability for posting the class > file data of shared classes to the CFLH, so there's no need to disable > CDS. For future optimizations, we might want to prevent loading additional shared classes if any of the archived system classes is changed. However, that's out of the scope of the current issue. Please let me know when you have a new webrev ready. Thanks, Jiangli > > Thanks > - Ioi > > >> Thanks, >> Jiangli >>> >>> >>> I will test the patch (probably some old tests may fail because they >>> assumed >>> that CDS can be enabled when CFLH is in use). I'll post a formal RFR >>> afterwards. >>> >>> I'll also see if it's possible to allow CDS if >>> can_generate_early_class_hook_events==false. >>> I am not entirely sure if all classes affected by UseSharedSpaces >>> are all >>> loaded in the "early" stage. >>> >>> We can try the "messy" fix later, if we have more time to get it right. >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 10/15/2018 03:35 PM, Jiangli Zhou wrote: >>>> On 10/15/18 3:30 PM, Michael Rasmussen wrote: >>>> >>>>>> One simple alternative fix is to disable sharing when some events >>>>>> (like >>>>>> CFLH) were enabled. This in fact was the behavior before JDK 9. >>>>>> In JDK >>>>>> 9, we enabled CDS for debugging. Looks like there are corner >>>>>> cases that >>>>>> are not uncovered by our testing. I'd like to seek opinions and >>>>>> feedback >>>>>> on the importance of supporting debugging with CDS enabled. >>>>> Or at least CFLH together with can_generate_all_class_hook_events >>>>> and can_generate_early_class_hook_events capabilities. >>>>> >>>>> CDS hasn't been part of our own testing scenarios so far, which is >>>>> why we haven't notice these issues on our end before now. >>>>> We only just discovered it because of user reports about crashes, >>>>> which turned out to be because the java-11-openjdk package on >>>>> Fedora comes with pre-generated CDS archive. >>>>> >>>>> But especially with JEP 341 just around the corner, it makes it >>>>> very important to find and weed out these bugs now! >>>> >>>> Totally agree. These issues are now more important with the default >>>> CDS archive. Thanks for reporting them! >>>> >>>> Best regards, >>>> Jiangli >>>>> >>>>> /Michael >>>> >>> >> > From jcbeyler at google.com Fri Oct 19 21:31:06 2018 From: jcbeyler at google.com (JC Beyler) Date: Fri, 19 Oct 2018 14:31:06 -0700 Subject: RFR (XXS): 8212698: Minor g1 #include changes and memoryService.hpp copyright date update In-Reply-To: <7693E8BA-7A72-4D9E-B8B0-3E7566D00050@amazon.com> References: <7693E8BA-7A72-4D9E-B8B0-3E7566D00050@amazon.com> Message-ID: Hi Paul, Looks good to me! Jc On Fri, Oct 19, 2018 at 11:40 AM Hohensee, Paul wrote: > A trivial patch to fix 3 misc. issues. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212698 > Webrev: http://cr.openjdk.java.net/~phh/8212698/webrev.00/ > > Thanks, > > Paul > > -- Thanks, Jc From vladimir.kozlov at oracle.com Fri Oct 19 21:38:25 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 19 Oct 2018 14:38:25 -0700 Subject: Clean up CompiledMethod::oops_reloc_begin() In-Reply-To: <5BC756EE.6040900@oracle.com> References: <5BC756EE.6040900@oracle.com> Message-ID: Hi Erik, I think it was simple optimization to skip jump instruction - nothing more. Yes, there is special handling of Verified entry code generation to make sure there is space for jump instructions there - there is no way oop will be there: http://hg.openjdk.java.net/jdk/jdk/file/4d1e5697b32b/src/hotspot/cpu/x86/nativeInst_x86.cpp#l540 http://hg.openjdk.java.net/jdk/jdk/file/4d1e5697b32b/src/hotspot/cpu/x86/macroAssembler_x86.cpp#l5455 Yes, code in zNMethodTable.cpp is not needed. And change in compiledMethod.cpp is good. Reviewed. Thanks, Vladimir On 10/17/18 8:36 AM, Erik ?sterlund wrote: > Hi, > > In CompiledMethod::oops_reloc_begin() there is code to prevent looking for oops at the verified entry of an nmethod that > is not entrant, as a native jump instruction has been overwritten there. Except there would *never* be any immediate > oops there for any compiler, even before becoming not entrant, with or without OSR compilation, so this special handling > of not entrant vs entrant nmethods is seemingly completely unnecessary. > > This gets increasingly awkward when oops_do is called concurrently with concurrent class unloading, where an nmethod is > racing to become not entrant. > > To clean this up, I propose to change the boundary for immediate oop scanning and start looking for oops only after the > frame building is completed, as there is absolutely no point in looking for oops before that in the first place. This > removes unnecessary workarounds that exist today, and removes unnecessary races going forward. > > It seems like Graal as JIT also inlines oops into the code stream, but never sets the CodeOffsets::Frame_Complete in the > nmethod. So I recognize such nmethods and unconditionally start scanning at the verified entry + native jump instruction > size. I spoke to the Graal folks, and they said they do not put oops in there either. So I think everyone involved > should be happy with this solution. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212585 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212585/webrev.00/ > > Thanks, > /Erik From thomas.schatzl at oracle.com Fri Oct 19 21:39:42 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 19 Oct 2018 23:39:42 +0200 Subject: RFR (XXS): 8212698: Minor g1 #include changes and memoryService.hpp copyright date update In-Reply-To: References: <7693E8BA-7A72-4D9E-B8B0-3E7566D00050@amazon.com> Message-ID: <492842a28787f03d0e4a7402319be8128144dd68.camel@oracle.com> Hi Paul, On Fri, 2018-10-19 at 14:31 -0700, JC Beyler wrote: > Hi Paul, > > Looks good to me! > Jc > > On Fri, Oct 19, 2018 at 11:40 AM Hohensee, Paul > wrote: > > > A trivial patch to fix 3 misc. issues. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212698 > > Webrev: http://cr.openjdk.java.net/~phh/8212698/webrev.00/ > > ship it. Thomas From kim.barrett at oracle.com Fri Oct 19 21:50:01 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 19 Oct 2018 17:50:01 -0400 Subject: RFR (XXS): 8212698: Minor g1 #include changes and memoryService.hpp copyright date update In-Reply-To: <7693E8BA-7A72-4D9E-B8B0-3E7566D00050@amazon.com> References: <7693E8BA-7A72-4D9E-B8B0-3E7566D00050@amazon.com> Message-ID: <0AAF7EAC-CE1B-48E0-B05F-FF709D015DF3@oracle.com> > On Oct 19, 2018, at 2:40 PM, Hohensee, Paul wrote: > > A trivial patch to fix 3 misc. issues. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212698 > Webrev: http://cr.openjdk.java.net/~phh/8212698/webrev.00/ > > Thanks, > > Paul Looks good. From hohensee at amazon.com Fri Oct 19 21:55:41 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 19 Oct 2018 21:55:41 +0000 Subject: RFR (XXS): 8212698: Minor g1 #include changes and memoryService.hpp copyright date update In-Reply-To: <492842a28787f03d0e4a7402319be8128144dd68.camel@oracle.com> References: <7693E8BA-7A72-4D9E-B8B0-3E7566D00050@amazon.com> <492842a28787f03d0e4a7402319be8128144dd68.camel@oracle.com> Message-ID: Pushed. Thanks! ?On 10/19/18, 5:40 PM, "Thomas Schatzl" wrote: Hi Paul, On Fri, 2018-10-19 at 14:31 -0700, JC Beyler wrote: > Hi Paul, > > Looks good to me! > Jc > > On Fri, Oct 19, 2018 at 11:40 AM Hohensee, Paul > wrote: > > > A trivial patch to fix 3 misc. issues. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212698 > > Webrev: http://cr.openjdk.java.net/~phh/8212698/webrev.00/ > > ship it. Thomas From dean.long at oracle.com Sat Oct 20 03:28:48 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 19 Oct 2018 20:28:48 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> Message-ID: On 10/18/18 6:12 PM, Mandy Chung wrote: > > > On 10/18/18 12:27 AM, David Holmes wrote: >> Hi Dean, >> >> On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: >>> >>> You're right, I missed that.? I think the right thing to do is call >>> current_thread_exiting while holding the Threads_lock. >>> Then we can get rid of the parallel atomic counters.? So, here's one >>> more try: >>> >>> http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ >> >> Okay that is the simple and obvious solution that doesn't require >> split counts. So I have to ask Mandy if she recalls why this approach >> wasn't taken 15 years ago when the exit counts were added as part of: >> > > It has been so long.? I think it's likely an oversight. >> https://bugs.openjdk.java.net/browse/JDK-4530538 ? >> >> Does taking the Threads_lock here cost too much and cause a thread >> termination bottleneck? > > If the contention on Threads_lock is not high (that seems to me), it > should be okay. ? I'm not close to the VM implementation (lot of > changes since then) and I don't have a definitive answer unless I > study the code closely.?? You and others have a better judgement on this. > > AFAICT the change is okay. > Thanks Mandy.? David, OK to push? dl > Mandy > > From Alan.Bateman at oracle.com Sat Oct 20 08:06:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 20 Oct 2018 09:06:14 +0100 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> Message-ID: <02bc7237-d7c7-dd8f-2d3d-eebe64a63eee@oracle.com> On 19/10/2018 13:11, Baesken, Matthias wrote: > Hi Alan, I agree it is a pity that the minor version (dwMinorVersion) was not increased . > Looking at the build number was not my preferred way to handle this . > > My understanding (checked with our internal SAP-MS porting guys) is that the buildNumber of Windows server 2016 will not get larger than > the minimum introduced buildNumber of Windows server 2019 . > The changes in your webrev look okay to me although I assume we need to be cautious about back porting until it becomes clear if Windows Server 2019 will be re-released or not.? The Windows Server blog [1] suggests that the release is paused while an issue is investigated and maybe a re-release will mean a new build number at least. -Alan [1] https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows-server-2019-now-generally-available From erik.osterlund at oracle.com Sat Oct 20 11:59:44 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Sat, 20 Oct 2018 13:59:44 +0200 Subject: Clean up CompiledMethod::oops_reloc_begin() In-Reply-To: References: <5BC756EE.6040900@oracle.com> Message-ID: <295B8007-3800-41EF-BAB4-1CDA4464B260@oracle.com> Hi Vladimir, Thanks for the review. /Erik > On 19 Oct 2018, at 23:38, Vladimir Kozlov wrote: > > Hi Erik, > > I think it was simple optimization to skip jump instruction - nothing more. > > Yes, there is special handling of Verified entry code generation to make sure there is space for jump instructions there - there is no way oop will be there: > > http://hg.openjdk.java.net/jdk/jdk/file/4d1e5697b32b/src/hotspot/cpu/x86/nativeInst_x86.cpp#l540 > http://hg.openjdk.java.net/jdk/jdk/file/4d1e5697b32b/src/hotspot/cpu/x86/macroAssembler_x86.cpp#l5455 > > Yes, code in zNMethodTable.cpp is not needed. And change in compiledMethod.cpp is good. Reviewed. > > Thanks, > Vladimir > >> On 10/17/18 8:36 AM, Erik ?sterlund wrote: >> Hi, >> In CompiledMethod::oops_reloc_begin() there is code to prevent looking for oops at the verified entry of an nmethod that is not entrant, as a native jump instruction has been overwritten there. Except there would *never* be any immediate oops there for any compiler, even before becoming not entrant, with or without OSR compilation, so this special handling of not entrant vs entrant nmethods is seemingly completely unnecessary. >> This gets increasingly awkward when oops_do is called concurrently with concurrent class unloading, where an nmethod is racing to become not entrant. >> To clean this up, I propose to change the boundary for immediate oop scanning and start looking for oops only after the frame building is completed, as there is absolutely no point in looking for oops before that in the first place. This removes unnecessary workarounds that exist today, and removes unnecessary races going forward. >> It seems like Graal as JIT also inlines oops into the code stream, but never sets the CodeOffsets::Frame_Complete in the nmethod. So I recognize such nmethods and unconditionally start scanning at the verified entry + native jump instruction size. I spoke to the Graal folks, and they said they do not put oops in there either. So I think everyone involved should be happy with this solution. >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212585 >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212585/webrev.00/ >> Thanks, >> /Erik From aph at redhat.com Sun Oct 21 09:38:09 2018 From: aph at redhat.com (Andrew Haley) Date: Sun, 21 Oct 2018 10:38:09 +0100 Subject: Bit set intrinsic In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On 10/19/2018 04:42 PM, B. Blaser wrote: > I tried example [1] with both synchronized BitSet and dedicated > intrinsic on x86_64 (8 cores) which revealed that the intrinsic would > be at least 2-3x faster than the synchronized set. Which makes it very useful for things like highly-concurrent Bloom Filters. Here's an example of current usage: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/BloomFilter.java https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/util/Memory.java If we can provide a really fast on/off-heap bitset in standard Java the need for this Unsafe hackery will go away. It's interesting that the Cassandra Bloom Filter is not thread safe; I don't know why this is. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From ioi.lam at oracle.com Mon Oct 22 01:16:02 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 21 Oct 2018 18:16:02 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: <4e474a5e-9d65-ca27-94ac-e04a5aafbb0f@oracle.com> References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> <5493aca2-f67b-4e9f-1f10-02511d701eca@oracle.com> <4e474a5e-9d65-ca27-94ac-e04a5aafbb0f@oracle.com> Message-ID: I sent RFR to http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030612.html http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-October/025640.html Let's move the discussion to those threads. Thanks - Ioi On 10/19/18 1:48 PM, Jiangli Zhou wrote: > Hi Ioi, > > > On 10/17/18 10:31 PM, Ioi Lam wrote: >> On 10/17/18 7:39 PM, Jiangli Zhou wrote: >> >>> On 10/17/18 4:54 PM, Ioi Lam wrote: >>> >>>> Hi Michael & Volker, >>>> >>>> >>>> Thanks for the bug report and the initial investigation! >>>> >>>> Following Volker's investigation, I tried to make a less drastic fix >>>> (than the "nuclear option" of disable CDS entirely), but run into a >>>> lot of >>>> issues when class-file-load-hook is specified: >>>> >>>> >>>> 1. The shared heap objects are mapped at JVM start-up. At this >>>> point, we don't know if any of the archived object's class may >>>> be replaced by the CFLH. >>> I've given more thoughts to this. Besides Strings, other archived >>> objects are already disabled automatically. Archived mirrors and cp >>> reference arrays are not installed if their archived class are not >>> used. The archived subg-raphs are not used if any of the classes >>> (for objects in the subgraphs) are not the same as the archived ones. >> >> Thanks for the clarification. This makes the patch simpler. >> >>>> >>>> 2. If java.lang.{Object, Cloneable, Serializable} are replaced >>>> by CFLH, we cannot use the archived primitive arrays. >>>> >>>> 3. There's a lot of code that assumes if UseSharedSpaces==true, >>>> certain InstanceKlasses must be loaded from CDS. E.g., >>>> ?? JavaClasses::compute_offsets() does nothing if >>>> UseSharedSpaces==true, >>>> ?? because it assumes that offsets calculated during dump time will >>>> match >>>> ?? the class loaded at runtime. >>> Yes, the assumption is baked in our code. We'd have to exam each >>> UseSharedSpaces check if we want to support the behavior. >> >> I think all of the UseSharedSpaces assumptions affect only classes >> loaded inside SystemDictionary::resolve_preloaded_classes(), which is >> called in the JVMTI "early" stage. So I've changed the code to >> disable CDS only like this (as Michael suggested): >> >> ? if (JvmtiExport::should_post_class_file_load_hook() && >> JvmtiExport::early_class_hook_env()) { >> ??? // CDS assumes that no classes resolved in >> SystemDictionary::resolve_preloaded_classes >> ??? // are replaced at runtime by JVMTI ClassFileLoadHook. All of >> those classes are resolved >> ??? // during the JVMTI "early" stage, so we're OK if >> JvmtiExport::early_class_hook_env() >> ??? // is not specified by the agent(s). >> ??? FileMapInfo::fail_continue("CDS is disabled because early JVMTI >> ClassFileLoadHook is in use."); >> ??? return false; >> ? } >> >> Updated webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v02/ >> >> >> I'll add more asserts and test cases to validate the patch. > > Do you have an updated webev with more test cases ready, or you are > still working on it? We should consider including more classes from > the default classlist in the test. Archived classes loaded during both > 'early' phase and after should be tested. We want to make sure with > shared archive being used, the visible behavior should be the same as > with -Xshare:off (no crash, no assert, no different exception) when > working with a JVMTI agent. > >> >>>> >>>> >>>> I have developed a jtreg test case for this, and attempted to do a >>>> less drastic fix, but I am nowhere near done and it's already messy: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v00/ >>>> >>>> >>>> >>>> Instead, I think we should do this: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk12/8212200-cds-jvmti-clfh-critical-classes.v01/ >>>> >>> This would brings back the behavior of JDK-8141341. To be more >>> cautious this time, is there any case that's covered by the check >>> for >>> can_generate_all_class_hook_events/can_generate_early_class_hook_events >>> capabilities but not for >>> JvmtiExport::should_post_class_file_load_hook()? >>> >> >> I looked at JvmtiClassFileLoadHookPoster::post_to_env(). The >> "preloaded" classes can be replaced only if at least one agent has >> >> (1) set the following 2 capabilities: >> >> ??? caps.can_generate_all_class_hook_events = 1; >> ??? caps.can_generate_early_class_hook_events = 1; >> >> AND >> >> (2) Specified JVMTI_EVENT_CLASS_FILE_LOAD_HOOK >> >> See >> >> http://hg.openjdk.java.net/jdk/jdk/file/672bc2213cef/src/hotspot/share/prims/jvmtiExport.cpp#l932 >> > > Ok, thanks. To make sure we don't miss any corner cases, we should > include serviceability-dev mailing list for the code review also. > >> >> >>> As part of the JDK-8141341, we also stop loading any additional >>> archived classes if JvmtiExport::should_post_class_file_load_hook() >>> is set after mapping the shared archive. Should we do that also if >>> we are going with this approach? >>> >> >> I don't think that's necessary. It should be OK to replace any >> classes that are not in the "preloaded" list. >> >> Originally (before JDK 9) we would disable CDS so that classes are >> loaded normally, in order for the class file data to be posted to the >> CFLH. >> >> However, since JDK 9, we have added the ability for posting the class >> file data of shared classes to the CFLH, so there's no need to >> disable CDS. > > For future optimizations, we might want to prevent loading additional > shared classes if any of the archived system classes is changed. > However, that's out of the scope of the current issue. > > Please let me know when you have a new webrev ready. > > Thanks, > Jiangli > >> >> Thanks >> - Ioi >> >> >>> Thanks, >>> Jiangli >>>> >>>> >>>> I will test the patch (probably some old tests may fail because >>>> they assumed >>>> that CDS can be enabled when CFLH is in use). I'll post a formal >>>> RFR afterwards. >>>> >>>> I'll also see if it's possible to allow CDS if >>>> can_generate_early_class_hook_events==false. >>>> I am not entirely sure if all classes affected by UseSharedSpaces >>>> are all >>>> loaded in the "early" stage. >>>> >>>> We can try the "messy" fix later, if we have more time to get it >>>> right. >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 10/15/2018 03:35 PM, Jiangli Zhou wrote: >>>>> On 10/15/18 3:30 PM, Michael Rasmussen wrote: >>>>> >>>>>>> One simple alternative fix is to disable sharing when some >>>>>>> events (like >>>>>>> CFLH) were enabled. This in fact was the behavior before JDK 9. >>>>>>> In JDK >>>>>>> 9, we enabled CDS for debugging. Looks like there are corner >>>>>>> cases that >>>>>>> are not uncovered by our testing. I'd like to seek opinions and >>>>>>> feedback >>>>>>> on the importance of supporting debugging with CDS enabled. >>>>>> Or at least CFLH together with can_generate_all_class_hook_events >>>>>> and can_generate_early_class_hook_events capabilities. >>>>>> >>>>>> CDS hasn't been part of our own testing scenarios so far, which >>>>>> is why we haven't notice these issues on our end before now. >>>>>> We only just discovered it because of user reports about crashes, >>>>>> which turned out to be because the java-11-openjdk package on >>>>>> Fedora comes with pre-generated CDS archive. >>>>>> >>>>>> But especially with JEP 341 just around the corner, it makes it >>>>>> very important to find and weed out these bugs now! >>>>> >>>>> Totally agree. These issues are now more important with the >>>>> default CDS archive. Thanks for reporting them! >>>>> >>>>> Best regards, >>>>> Jiangli >>>>>> >>>>>> /Michael >>>>> >>>> >>> >> > From tobias.hartmann at oracle.com Mon Oct 22 06:26:56 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 22 Oct 2018 08:26:56 +0200 Subject: Review Request: JDK-8207146: Rename jdk.internal.misc.Unsafe::xxxObject to xxxReference In-Reply-To: <3bed4fd6-1f6f-b95d-1eb9-62396a8e9815@oracle.com> References: <3bed4fd6-1f6f-b95d-1eb9-62396a8e9815@oracle.com> Message-ID: <7adbb064-d0da-e1a2-c78e-ebc389b11f0f@oracle.com> Hi Mandy, the compiler related changes look good to me. Please run hs-tier1-3 if you haven't done so yet. Best regards, Tobias On 16.10.18 18:08, Mandy Chung wrote: > Webrev: > http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8207146/webrev.00/ > > Unsafe::getObject returns a reference to an object. Similarly > Unsafe::putObject sets a reference in the given base+offset. > This patch proposes to rename Unsafe xxxObject to xxxReference > that will make the xxxReference API very clear that these > are getters/setters for a reference (instance of object class) > and ready for adding xxxValue in the futurefor values. > Note that this renaming only applies to jdk.internal.misc.Unsafe > but not sun.misc.Unsafe.? So no impact to existing libraries > that depends on sun.misc.Unsafe in jdk.unsupported module. > > I ran jdk_core, core_tools, langtools and nashorn tier1-3, > hotspot_runtime, hotspot_compiler test groups. > > thanks > Mandy From martin.doerr at sap.com Mon Oct 22 10:07:00 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 22 Oct 2018 10:07:00 +0000 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> Message-ID: Hi Gustavo, thanks for contributing. I have some minor remarks. I'm not sure if it's good to have L=1 hard coded in the assembler instruction, but I think it's ok for now. (One could also pass L as parameter with default value 1.) I think it would be nice to have the capability bit usages in the same order at all places in the source code. We could add a comment line like "rtm is determined by OS" where it was omitted. An empty line before has_mtfprd() would be good because it's just an alias. Having them everywhere as one block in the same order helps avoiding confusion. Besides this, the change looks good. Best regards, Martin -----Original Message----- From: ppc-aix-port-dev On Behalf Of Gustavo Romero Sent: Dienstag, 16. Oktober 2018 23:10 To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection Hi, Could the following small change be reviewed please? Bug : https://bugs.openjdk.java.net/browse/JDK-8212481 Webrev: http://cr.openjdk.java.net/~gromero/8212481/v1/ It adds 'darn' instruction introduced with POWER9 CPUs to the Macroassembler and uses it to set PowerArchitecturePPC64 to POWER9 when the instruction is available on the CPU so it can be used in the future by any POWER9-specific JVM code or by any JVM code specifically dependent on the 'darn' instruction, using VM_Version::has_darn(). It was checked on POWER9 as the following: gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 176 On-line CPU(s) list: 0-175 Thread(s) per core: 4 Core(s) per socket: 22 Socket(s): 2 NUMA node(s): 8 Model: 2.2 (pvr 004e 1202) Model name: POWER9 (raw), altivec supported CPU max MHz: 2800.0000 CPU min MHz: 2300.0000 L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 10240K NUMA node0 CPU(s): 0-87 NUMA node8 CPU(s): 88-175 NUMA node250 CPU(s): NUMA node251 CPU(s): NUMA node252 CPU(s): NUMA node253 CPU(s): NUMA node254 CPU(s): NUMA node255 CPU(s): gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ LD_SHOW_AUXV=1 /bin/true | egrep "HWCAP|PLATFORM" AT_HWCAP: true_le archpmu vsx arch_2_06 dfp ic_snoop smt mmu fpu altivec ppc64 ppc32 AT_HWCAP2: darn ieee128 arch_3_00 vcrypto tar isel ebb dscr arch_2_07 AT_PLATFORM: power9 AT_BASE_PLATFORM:power9 gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ ./java -XX:+Verbose -version dscr value was 0x10 Version: ppc64 fsqrt isel lxarxeh cmpb popcntb popcntw fcfids vand lqarx aes vpmsumb mfdscr vsx ldbrx stdbrx sha darn L1_data_cache_line_size=128 openjdk version "12-internal" 2019-03-19 OpenJDK Runtime Environment (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9) OpenJDK 64-Bit Server VM (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9, mixed mode, sharing) It was also checked on POWER8 and no regression was observed: gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 80 On-line CPU(s) list: 0-79 Thread(s) per core: 8 Core(s) per socket: 10 Socket(s): 1 NUMA node(s): 1 Model: 2.1 (pvr 004b 0201) Model name: POWER8 (architected), altivec supported Hypervisor vendor: horizontal Virtualization type: full L1d cache: 64K L1i cache: 32K NUMA node0 CPU(s): 0-79 gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ LD_SHOW_AUXV=1 /bin/true | egrep "HWCAP|PLATFORM" AT_HWCAP: archpmu vsx arch_2_06 dfp ic_snoop smt mmu fpu altivec ppc64 ppc32 AT_HWCAP2: htm-nosc vcrypto tar isel ebb dscr htm arch_2_07 AT_PLATFORM: power8 AT_BASE_PLATFORM:power8 gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ ./java -XX:+Verbose -version dscr value was 0x0 Version: ppc64 fsqrt isel lxarxeh cmpb popcntb popcntw fcfids vand lqarx aes vpmsumb mfdscr vsx ldbrx stdbrx sha rtm L1_data_cache_line_size=128 openjdk version "12-internal" 2019-03-19 OpenJDK Runtime Environment (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9) OpenJDK 64-Bit Server VM (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9, mixed mode, sharing) Thank you. Best regards, Gustavo From erik.osterlund at oracle.com Mon Oct 22 10:59:16 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 22 Oct 2018 12:59:16 +0200 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker Message-ID: <5BCDAD84.4000808@oracle.com> Hi, Today, one must acquire the global CompiledIC_lock to have permission to perform certain IC updating activities. The lock is used outside of safepoints, but for example during nmethod unloading due to GC, lock free IC cache cleaning is performed, which is safe due to being in a safepoint. With concurrent class unloading introduced, the global lock is too coarse grained, and a per-nmethod mechanism is needed instead. In order to allow this, an abstract CompiledICLocker class could help ensuring the safety of IC caches, and allow both the fine grained (but density costly) approach for users of concurrent class unloading, and resort to the default global locking approach otherwise. The idea is to retain the coarse grained implementation of the synchronization when concurrent class unloading is not being used, as supporting fine grained locking could be more costly in memory, and there does not seem to be any use in doing this unless concurrent class unloading is being used. Webrev: http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212681 Thanks, /Erik From matthias.baesken at sap.com Mon Oct 22 11:18:47 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 22 Oct 2018 11:18:47 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: <02bc7237-d7c7-dd8f-2d3d-eebe64a63eee@oracle.com> References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> <02bc7237-d7c7-dd8f-2d3d-eebe64a63eee@oracle.com> Message-ID: Hi Alan , in case there is a re-release of Windows server 2019 the buildNumber might (or might not) increase. But it will not decrease. So I think we are fine with the current check . > although I assume we need to be cautious about back porting Sure we can wait a couple of weeks before backporting it to jdk11u . Best regards, Matthias > -----Original Message----- > From: Alan Bateman > Sent: Samstag, 20. Oktober 2018 10:06 > To: Baesken, Matthias > Cc: hotspot-dev Source Developers ; core- > libs-dev at openjdk.java.net; Moldenhauer, Niclas > > Subject: Re: RFR : 8211106: [windows] Update OS detection code to > recognize Windows Server 2019 > > On 19/10/2018 13:11, Baesken, Matthias wrote: > > Hi Alan, I agree it is a pity that the minor version (dwMinorVersion) was > not increased . > > Looking at the build number was not my preferred way to handle this . > > > > My understanding (checked with our internal SAP-MS porting guys) is > that the buildNumber of Windows server 2016 will not get larger than > > the minimum introduced buildNumber of Windows server 2019 . > > > The changes in your webrev look okay to me although I assume we need to > be cautious about back porting until it becomes clear if Windows Server > 2019 will be re-released or not.? The Windows Server blog [1] suggests > that the release is paused while an issue is investigated and maybe a > re-release will mean a new build number at least. > > -Alan > > [1] > https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows- > server-2019-now-generally-available From david.holmes at oracle.com Mon Oct 22 11:22:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 22 Oct 2018 21:22:56 +1000 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <5BCDAD84.4000808@oracle.com> References: <5BCDAD84.4000808@oracle.com> Message-ID: <80f77438-9061-4afc-f09e-59c420a54d19@oracle.com> Hi Erik, The CompiledIC_lock also "guards" use of other nested locks and thereby avoids the possibility of lock sneaking on those nested locks [1]. Will your concurrent unloading changes change this? Thanks, David [1] https://bugs.openjdk.java.net/browse/JDK-8210832 On 22/10/2018 8:59 PM, Erik ?sterlund wrote: > Hi, > > Today, one must acquire the global CompiledIC_lock to have permission to > perform certain IC updating activities. The lock is used outside of > safepoints, but for example during nmethod unloading due to GC, lock > free IC cache cleaning is performed, which is safe due to being in a > safepoint. > > With concurrent class unloading introduced, the global lock is too > coarse grained, and a per-nmethod mechanism is needed instead. In order > to allow this, an abstract CompiledICLocker class could help ensuring > the safety of IC caches, and allow both the fine grained (but density > costly) approach for users of concurrent class unloading, and resort to > the default global locking approach otherwise. > > The idea is to retain the coarse grained implementation of the > synchronization when concurrent class unloading is not being used, as > supporting fine grained locking could be more costly in memory, and > there does not seem to be any use in doing this unless concurrent class > unloading is being used. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212681 > > Thanks, > /Erik From shade at redhat.com Mon Oct 22 11:35:19 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 22 Oct 2018 13:35:19 +0200 Subject: RFR 8212754 (XS): Build failure: undefined JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8212754 When JVMTI is disabled (e.g. with Minimal VM), jvmtiExport.cpp is not compiled, and so definition is not present during linkage. This actually manifests in jdk-updates/jdk11u, because memAllocator.cpp calls this method only there. jdk/jdk has no problematic call from memAllocator.cpp. Still, I believe the code should not deviate between jdk and jdk11u, so let's push to to development head, and then backport. Fix: # User shade 8212754: Build failure: undefined JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample Reviewed-by: XXX diff -r d5a96cafdd4a -r f9c30b92d0dc src/hotspot/share/prims/jvmtiExport.hpp --- a/src/hotspot/share/prims/jvmtiExport.hpp Wed Oct 17 22:47:59 2018 +0200 +++ b/src/hotspot/share/prims/jvmtiExport.hpp Mon Oct 22 13:20:21 2018 +0200 @@ -538,7 +538,7 @@ JvmtiSampledObjectAllocEventCollector() NOT_JVMTI_RETURN; ~JvmtiSampledObjectAllocEventCollector() NOT_JVMTI_RETURN; bool is_sampled_object_alloc_event() { return true; } - static bool object_alloc_is_safe_to_sample(); + static bool object_alloc_is_safe_to_sample() NOT_JVMTI_RETURN_(false); }; Testing: jdk-updates/jdk11u minimal+slowdebug; jdk/jdk {server,minimal}+slowdebug Thanks, -Aleksey From erik.osterlund at oracle.com Mon Oct 22 11:48:35 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 22 Oct 2018 13:48:35 +0200 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <80f77438-9061-4afc-f09e-59c420a54d19@oracle.com> References: <5BCDAD84.4000808@oracle.com> <80f77438-9061-4afc-f09e-59c420a54d19@oracle.com> Message-ID: <5BCDB913.4050803@oracle.com> Hi David, With concurrent class unloading, there is a possibility that two JavaThreads both miss an IC, run the SharedRuntime::handle_ic_miss_helper, lock disjoint fine grained IC locks, both ask if a CompiledIC::is_megamorphic(), causing the VtableStubs_lock in VtableStubs::entry_point to become contended, and hence possibly rely on the VM thread using lock sneaking to not cause a deadlock. If we want to get rid of sneaky locking, it seems like we should use MutexLockerEx and conditionally lock VtableStubs_lock only when we are not the VM thread in a safepoint. It seems kind of evil to build the correctness of a solution based on redundant locking on a higher level that removes the contention on another lower level lock that would if contended cause a deadlock. I think the InlineCacheBuffer_lock should be okay, because in the paths that grab that lock, I make sure to take the CompiledIC_lock first, with or without fine grained IC locking. Thanks, /Erik On 2018-10-22 13:22, David Holmes wrote: > Hi Erik, > > The CompiledIC_lock also "guards" use of other nested locks and > thereby avoids the possibility of lock sneaking on those nested locks > [1]. Will your concurrent unloading changes change this? > > Thanks, > David > > [1] https://bugs.openjdk.java.net/browse/JDK-8210832 > > On 22/10/2018 8:59 PM, Erik ?sterlund wrote: >> Hi, >> >> Today, one must acquire the global CompiledIC_lock to have permission >> to perform certain IC updating activities. The lock is used outside >> of safepoints, but for example during nmethod unloading due to GC, >> lock free IC cache cleaning is performed, which is safe due to being >> in a safepoint. >> >> With concurrent class unloading introduced, the global lock is too >> coarse grained, and a per-nmethod mechanism is needed instead. In >> order to allow this, an abstract CompiledICLocker class could help >> ensuring the safety of IC caches, and allow both the fine grained >> (but density costly) approach for users of concurrent class >> unloading, and resort to the default global locking approach otherwise. >> >> The idea is to retain the coarse grained implementation of the >> synchronization when concurrent class unloading is not being used, as >> supporting fine grained locking could be more costly in memory, and >> there does not seem to be any use in doing this unless concurrent >> class unloading is being used. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212681 >> >> Thanks, >> /Erik From jcbeyler at google.com Mon Oct 22 16:26:34 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 22 Oct 2018 09:26:34 -0700 Subject: RFR 8212754 (XS): Build failure: undefined JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample In-Reply-To: References: Message-ID: Hi Aleksey, Looks good to me; let me know if I can do anything to help, though it seems like an easy push once reviews are in, Thanks for the fix, Jc On Mon, Oct 22, 2018 at 4:36 AM Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212754 > > When JVMTI is disabled (e.g. with Minimal VM), jvmtiExport.cpp is not > compiled, and so definition is > not present during linkage. This actually manifests in jdk-updates/jdk11u, > because memAllocator.cpp > calls this method only there. jdk/jdk has no problematic call from > memAllocator.cpp. Still, I > believe the code should not deviate between jdk and jdk11u, so let's push > to to development head, > and then backport. > > Fix: > > # User shade > 8212754: Build failure: undefined > JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample > Reviewed-by: XXX > > diff -r d5a96cafdd4a -r f9c30b92d0dc > src/hotspot/share/prims/jvmtiExport.hpp > --- a/src/hotspot/share/prims/jvmtiExport.hpp Wed Oct 17 22:47:59 2018 > +0200 > +++ b/src/hotspot/share/prims/jvmtiExport.hpp Mon Oct 22 13:20:21 2018 > +0200 > @@ -538,7 +538,7 @@ > JvmtiSampledObjectAllocEventCollector() NOT_JVMTI_RETURN; > ~JvmtiSampledObjectAllocEventCollector() NOT_JVMTI_RETURN; > bool is_sampled_object_alloc_event() { return true; } > - static bool object_alloc_is_safe_to_sample(); > + static bool object_alloc_is_safe_to_sample() NOT_JVMTI_RETURN_(false); > }; > > > Testing: jdk-updates/jdk11u minimal+slowdebug; jdk/jdk > {server,minimal}+slowdebug > > Thanks, > -Aleksey > > -- Thanks, Jc From mandy.chung at oracle.com Mon Oct 22 17:08:40 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 22 Oct 2018 10:08:40 -0700 Subject: Review Request: JDK-8207146: Rename jdk.internal.misc.Unsafe::xxxObject to xxxReference In-Reply-To: <7adbb064-d0da-e1a2-c78e-ebc389b11f0f@oracle.com> References: <3bed4fd6-1f6f-b95d-1eb9-62396a8e9815@oracle.com> <7adbb064-d0da-e1a2-c78e-ebc389b11f0f@oracle.com> Message-ID: Thanks for the review, Tobias. I will rerun the tests and add hs-tier1-3 jobs before the push. Mandy On 10/21/18 11:26 PM, Tobias Hartmann wrote: > Hi Mandy, > > the compiler related changes look good to me. > > Please run hs-tier1-3 if you haven't done so yet. > > Best regards, > Tobias > > > On 16.10.18 18:08, Mandy Chung wrote: >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8207146/webrev.00/ >> >> Unsafe::getObject returns a reference to an object. Similarly >> Unsafe::putObject sets a reference in the given base+offset. >> This patch proposes to rename Unsafe xxxObject to xxxReference >> that will make the xxxReference API very clear that these >> are getters/setters for a reference (instance of object class) >> and ready for adding xxxValue in the futurefor values. >> Note that this renaming only applies to jdk.internal.misc.Unsafe >> but not sun.misc.Unsafe.? So no impact to existing libraries >> that depends on sun.misc.Unsafe in jdk.unsupported module. >> >> I ran jdk_core, core_tools, langtools and nashorn tier1-3, >> hotspot_runtime, hotspot_compiler test groups. >> >> thanks >> Mandy From shade at redhat.com Mon Oct 22 18:43:05 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 22 Oct 2018 20:43:05 +0200 Subject: RFR 8212754 (XS): Build failure: undefined JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample In-Reply-To: References: Message-ID: Thanks! No need for additional help, but I would appreciate another reviewer (and maybe accepting this is trivial). -Aleksey On 10/22/2018 06:26 PM, JC Beyler wrote: > Hi Aleksey, > > Looks good to me; let me know if I can do anything to help, though it seems like an easy push once > reviews are in, > > Thanks for the fix, > Jc > > On Mon, Oct 22, 2018 at 4:36 AM Aleksey Shipilev > wrote: > > Bug: > ?https://bugs.openjdk.java.net/browse/JDK-8212754 > > When JVMTI is disabled (e.g. with Minimal VM), jvmtiExport.cpp is not compiled, and so definition is > not present during linkage. This actually manifests in jdk-updates/jdk11u, because memAllocator.cpp > calls this method only there. jdk/jdk has no problematic call from memAllocator.cpp. Still, I > believe the code should not deviate between jdk and jdk11u, so let's push to to development head, > and then backport. > > Fix: > > # User shade > 8212754: Build failure: undefined > JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample > Reviewed-by: XXX > > diff -r d5a96cafdd4a -r f9c30b92d0dc src/hotspot/share/prims/jvmtiExport.hpp > --- a/src/hotspot/share/prims/jvmtiExport.hpp? ?Wed Oct 17 22:47:59 2018 +0200 > +++ b/src/hotspot/share/prims/jvmtiExport.hpp? ?Mon Oct 22 13:20:21 2018 +0200 > @@ -538,7 +538,7 @@ > ? ?JvmtiSampledObjectAllocEventCollector()? NOT_JVMTI_RETURN; > ? ?~JvmtiSampledObjectAllocEventCollector()? NOT_JVMTI_RETURN; > ? ?bool is_sampled_object_alloc_event()? ? { return true; } > -? static bool object_alloc_is_safe_to_sample(); > +? static bool object_alloc_is_safe_to_sample() NOT_JVMTI_RETURN_(false); > ?}; > > > Testing: jdk-updates/jdk11u minimal+slowdebug; jdk/jdk {server,minimal}+slowdebug > > Thanks, > -Aleksey > > > > -- > > Thanks, > Jc From zgu at redhat.com Mon Oct 22 18:46:13 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 22 Oct 2018 14:46:13 -0400 Subject: RFR 8212754 (XS): Build failure: undefined JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample In-Reply-To: References: Message-ID: <58d91392-8a3b-1aa4-b727-79425daf43fb@redhat.com> Looks good and trivial. Thanks, -Zhengyu On 10/22/2018 02:43 PM, Aleksey Shipilev wrote: > Thanks! No need for additional help, but I would appreciate another reviewer (and maybe accepting > this is trivial). > > -Aleksey > > On 10/22/2018 06:26 PM, JC Beyler wrote: >> Hi Aleksey, >> >> Looks good to me; let me know if I can do anything to help, though it seems like an easy push once >> reviews are in, >> >> Thanks for the fix, >> Jc >> >> On Mon, Oct 22, 2018 at 4:36 AM Aleksey Shipilev > wrote: >> >> Bug: >> ?https://bugs.openjdk.java.net/browse/JDK-8212754 >> >> When JVMTI is disabled (e.g. with Minimal VM), jvmtiExport.cpp is not compiled, and so definition is >> not present during linkage. This actually manifests in jdk-updates/jdk11u, because memAllocator.cpp >> calls this method only there. jdk/jdk has no problematic call from memAllocator.cpp. Still, I >> believe the code should not deviate between jdk and jdk11u, so let's push to to development head, >> and then backport. >> >> Fix: >> >> # User shade >> 8212754: Build failure: undefined >> JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample >> Reviewed-by: XXX >> >> diff -r d5a96cafdd4a -r f9c30b92d0dc src/hotspot/share/prims/jvmtiExport.hpp >> --- a/src/hotspot/share/prims/jvmtiExport.hpp? ?Wed Oct 17 22:47:59 2018 +0200 >> +++ b/src/hotspot/share/prims/jvmtiExport.hpp? ?Mon Oct 22 13:20:21 2018 +0200 >> @@ -538,7 +538,7 @@ >> ? ?JvmtiSampledObjectAllocEventCollector()? NOT_JVMTI_RETURN; >> ? ?~JvmtiSampledObjectAllocEventCollector()? NOT_JVMTI_RETURN; >> ? ?bool is_sampled_object_alloc_event()? ? { return true; } >> -? static bool object_alloc_is_safe_to_sample(); >> +? static bool object_alloc_is_safe_to_sample() NOT_JVMTI_RETURN_(false); >> ?}; >> >> >> Testing: jdk-updates/jdk11u minimal+slowdebug; jdk/jdk {server,minimal}+slowdebug >> >> Thanks, >> -Aleksey >> >> >> >> -- >> >> Thanks, >> Jc > > From coleen.phillimore at oracle.com Mon Oct 22 20:43:34 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 22 Oct 2018 16:43:34 -0400 Subject: RFR 8212754 (XS): Build failure: undefined JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample In-Reply-To: References: Message-ID: <18f3c224-9f72-b3e3-1ca9-a7021ec4bb19@oracle.com> Yes, this is trivial and looks good. Coleen On 10/22/18 2:43 PM, Aleksey Shipilev wrote: > Thanks! No need for additional help, but I would appreciate another reviewer (and maybe accepting > this is trivial). > > -Aleksey > > On 10/22/2018 06:26 PM, JC Beyler wrote: >> Hi Aleksey, >> >> Looks good to me; let me know if I can do anything to help, though it seems like an easy push once >> reviews are in, >> >> Thanks for the fix, >> Jc >> >> On Mon, Oct 22, 2018 at 4:36 AM Aleksey Shipilev > wrote: >> >> Bug: >> ?https://bugs.openjdk.java.net/browse/JDK-8212754 >> >> When JVMTI is disabled (e.g. with Minimal VM), jvmtiExport.cpp is not compiled, and so definition is >> not present during linkage. This actually manifests in jdk-updates/jdk11u, because memAllocator.cpp >> calls this method only there. jdk/jdk has no problematic call from memAllocator.cpp. Still, I >> believe the code should not deviate between jdk and jdk11u, so let's push to to development head, >> and then backport. >> >> Fix: >> >> # User shade >> 8212754: Build failure: undefined >> JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample >> Reviewed-by: XXX >> >> diff -r d5a96cafdd4a -r f9c30b92d0dc src/hotspot/share/prims/jvmtiExport.hpp >> --- a/src/hotspot/share/prims/jvmtiExport.hpp? ?Wed Oct 17 22:47:59 2018 +0200 >> +++ b/src/hotspot/share/prims/jvmtiExport.hpp? ?Mon Oct 22 13:20:21 2018 +0200 >> @@ -538,7 +538,7 @@ >> ? ?JvmtiSampledObjectAllocEventCollector()? NOT_JVMTI_RETURN; >> ? ?~JvmtiSampledObjectAllocEventCollector()? NOT_JVMTI_RETURN; >> ? ?bool is_sampled_object_alloc_event()? ? { return true; } >> -? static bool object_alloc_is_safe_to_sample(); >> +? static bool object_alloc_is_safe_to_sample() NOT_JVMTI_RETURN_(false); >> ?}; >> >> >> Testing: jdk-updates/jdk11u minimal+slowdebug; jdk/jdk {server,minimal}+slowdebug >> >> Thanks, >> -Aleksey >> >> >> >> -- >> >> Thanks, >> Jc > From david.holmes at oracle.com Mon Oct 22 22:31:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 Oct 2018 08:31:10 +1000 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> Message-ID: Sorry Dean I'm concerned about a thread termination bottleneck with this. A simple microbenchmark that creates 500,000 threads that have to run and terminate, shows a 15+% slowdown on my Linux box. I tried to find some kind of real benchmarks that covers thread termination but couldn't see anything specific. Can you at least run this through our performance system to see if any of the regular benchmarks are affected. Thanks, David On 20/10/2018 1:28 PM, dean.long at oracle.com wrote: > On 10/18/18 6:12 PM, Mandy Chung wrote: >> >> >> On 10/18/18 12:27 AM, David Holmes wrote: >>> Hi Dean, >>> >>> On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: >>>> >>>> You're right, I missed that.? I think the right thing to do is call >>>> current_thread_exiting while holding the Threads_lock. >>>> Then we can get rid of the parallel atomic counters.? So, here's one >>>> more try: >>>> >>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ >>> >>> Okay that is the simple and obvious solution that doesn't require >>> split counts. So I have to ask Mandy if she recalls why this approach >>> wasn't taken 15 years ago when the exit counts were added as part of: >>> >> >> It has been so long.? I think it's likely an oversight. >>> https://bugs.openjdk.java.net/browse/JDK-4530538 ? >>> >>> Does taking the Threads_lock here cost too much and cause a thread >>> termination bottleneck? >> >> If the contention on Threads_lock is not high (that seems to me), it >> should be okay. ? I'm not close to the VM implementation (lot of >> changes since then) and I don't have a definitive answer unless I >> study the code closely.?? You and others have a better judgement on this. >> >> AFAICT the change is okay. >> > > Thanks Mandy.? David, OK to push? > > dl > >> Mandy >> >> > From david.holmes at oracle.com Tue Oct 23 01:46:21 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 Oct 2018 11:46:21 +1000 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <5BCDB913.4050803@oracle.com> References: <5BCDAD84.4000808@oracle.com> <80f77438-9061-4afc-f09e-59c420a54d19@oracle.com> <5BCDB913.4050803@oracle.com> Message-ID: Hi Erik, On 22/10/2018 9:48 PM, Erik ?sterlund wrote: > Hi David, > > With concurrent class unloading, there is a possibility that two > JavaThreads both miss an IC, run the > SharedRuntime::handle_ic_miss_helper, lock disjoint fine grained IC > locks, both ask if a CompiledIC::is_megamorphic(), causing the > VtableStubs_lock in VtableStubs::entry_point to become contended, and > hence possibly rely on the VM thread using lock sneaking to not cause a > deadlock. That was my concern. We might go from a situation where sneaking is so rare that we can't observe it, to one where it is more common. > If we want to get rid of sneaky locking, it seems like we should use > MutexLockerEx and conditionally lock VtableStubs_lock only when we are > not the VM thread in a safepoint. It seems kind of evil to build the > correctness of a solution based on redundant locking on a higher level > that removes the contention on another lower level lock that would if > contended cause a deadlock. It's a usage protocol that allows us to show certain locks are immune from sneaking - which can be seen as a good thing to know at least until sneaking is removed, or which might allow sneaking to be turned off. But arguably it also shows that the lower-level locks are redundant and we should be able to rely on the higher-level lock only (at least prior to your change.) > I think the InlineCacheBuffer_lock should be okay, because in the paths > that grab that lock, I make sure to take the CompiledIC_lock first, with > or without fine grained IC locking. So you hold the CompiledIC_lock, a fine-grained IC lock and the InlineCacheBuffer_lock all at the same time? That seems somewhat excessive. :) Thanks, David > Thanks, > /Erik > > On 2018-10-22 13:22, David Holmes wrote: >> Hi Erik, >> >> The CompiledIC_lock also "guards" use of other nested locks and >> thereby avoids the possibility of lock sneaking on those nested locks >> [1]. Will your concurrent unloading changes change this? >> >> Thanks, >> David >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8210832 >> >> On 22/10/2018 8:59 PM, Erik ?sterlund wrote: >>> Hi, >>> >>> Today, one must acquire the global CompiledIC_lock to have permission >>> to perform certain IC updating activities. The lock is used outside >>> of safepoints, but for example during nmethod unloading due to GC, >>> lock free IC cache cleaning is performed, which is safe due to being >>> in a safepoint. >>> >>> With concurrent class unloading introduced, the global lock is too >>> coarse grained, and a per-nmethod mechanism is needed instead. In >>> order to allow this, an abstract CompiledICLocker class could help >>> ensuring the safety of IC caches, and allow both the fine grained >>> (but density costly) approach for users of concurrent class >>> unloading, and resort to the default global locking approach otherwise. >>> >>> The idea is to retain the coarse grained implementation of the >>> synchronization when concurrent class unloading is not being used, as >>> supporting fine grained locking could be more costly in memory, and >>> there does not seem to be any use in doing this unless concurrent >>> class unloading is being used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212681 >>> >>> Thanks, >>> /Erik > From robbin.ehn at oracle.com Tue Oct 23 12:50:52 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 23 Oct 2018 14:50:52 +0200 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <5BCDAD84.4000808@oracle.com> References: <5BCDAD84.4000808@oracle.com> Message-ID: Hi Erik, Looks better than before :), don't forget copyright years before push. /Robbin On 10/22/18 12:59 PM, Erik ?sterlund wrote: > Hi, > > Today, one must acquire the global CompiledIC_lock to have permission to perform > certain IC updating activities. The lock is used outside of safepoints, but for > example during nmethod unloading due to GC, lock free IC cache cleaning is > performed, which is safe due to being in a safepoint. > > With concurrent class unloading introduced, the global lock is too coarse > grained, and a per-nmethod mechanism is needed instead. In order to allow this, > an abstract CompiledICLocker class could help ensuring the safety of IC caches, > and allow both the fine grained (but density costly) approach for users of > concurrent class unloading, and resort to the default global locking approach > otherwise. > > The idea is to retain the coarse grained implementation of the synchronization > when concurrent class unloading is not being used, as supporting fine grained > locking could be more costly in memory, and there does not seem to be any use in > doing this unless concurrent class unloading is being used. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212681 > > Thanks, > /Erik From erik.osterlund at oracle.com Tue Oct 23 13:03:15 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 23 Oct 2018 15:03:15 +0200 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: References: <5BCDAD84.4000808@oracle.com> Message-ID: <5BCF1C13.5020103@oracle.com> Hi Robbin, Thanks for the review. /Erik On 2018-10-23 14:50, Robbin Ehn wrote: > Hi Erik, > > Looks better than before :), don't forget copyright years before push. > > /Robbin > > On 10/22/18 12:59 PM, Erik ?sterlund wrote: >> Hi, >> >> Today, one must acquire the global CompiledIC_lock to have permission >> to perform certain IC updating activities. The lock is used outside >> of safepoints, but for example during nmethod unloading due to GC, >> lock free IC cache cleaning is performed, which is safe due to being >> in a safepoint. >> >> With concurrent class unloading introduced, the global lock is too >> coarse grained, and a per-nmethod mechanism is needed instead. In >> order to allow this, an abstract CompiledICLocker class could help >> ensuring the safety of IC caches, and allow both the fine grained >> (but density costly) approach for users of concurrent class >> unloading, and resort to the default global locking approach otherwise. >> >> The idea is to retain the coarse grained implementation of the >> synchronization when concurrent class unloading is not being used, as >> supporting fine grained locking could be more costly in memory, and >> there does not seem to be any use in doing this unless concurrent >> class unloading is being used. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212681 >> >> Thanks, >> /Erik From aph at redhat.com Tue Oct 23 13:11:09 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 23 Oct 2018 14:11:09 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BC9E8BA.50407@oracle.com> References: <5BC9E8BA.50407@oracle.com> Message-ID: <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> On 10/19/2018 03:22 PM, Erik ?sterlund wrote: > Hi, > > Today the nmethods are unloaded either serially or in parallel due to GC > (and e.g. class unloading). With ZGC concurrent class unloading, a > concurrent fashion is required. Rather than inventing yet a third way of > unloading nmethods due to class unloading, it would be ideal if there > could be one unified way of doing it that makes everyone happy. > > To solve this problem in a more general way, a new member function on > CompiledMethod is added: is_unloading(). It returns whether a > CompiledMethod has broken oops. In order to not have to iterate over all > the oops every time this question is asked, it caches the result, so it > needs to be computed only once per "epoch". Every time a CodeCache > unloading is triggered by the GC, a new epoch is started, which forces > re-computation of whether the CompiledMethod is_unloading() the first > time it is called. > > So by having is_unloading() be lazily evaluated, it is now possible to > build a do_unloading() method on nmethod that simply makes the nmethod > unloaded if it is_unloading(), and otherwise cleans the various caches. > Cleaning e.g. the inline caches of said nmethod, uses the same > is_unloading() method on its targets to figure out if the IC cache > should be cleaned or not. Again, the epoched caching mechanism makes > sure we don't recompute this value. > > The new do_unloading() method may be called in both serial, parallel and > concurrent contexts. So I removed the parallel variation of this that we > had before, that unnecessarily postponed the unloading due to not having > computed whether the nmethod should be unloaded or not. Since that is > now computed on-demand lazily, there is no longer a need for postponing > this, nor to have two phases for parallel nmethod unloading. > > While there is more work involved in making concurrent nmethod unloading > work, this is a good cleanup towards that goal. > > Bug ID: > https://bugs.openjdk.java.net/browse/JDK-8209189 > > Webrev: > cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ http://cr.openjdk.java.net/home/eosterlu/oracle/jdk/old_repos/jdk-hs/open 404 - Not Found > > Thanks, > /Erik > -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Tue Oct 23 13:21:19 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 23 Oct 2018 15:21:19 +0200 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> Message-ID: <5BCF204F.6030705@oracle.com> Hi Andrew, You may prepend "http://" to the URL I posted in the email: http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ Thanks, /Erik On 2018-10-23 15:11, Andrew Haley wrote: > On 10/19/2018 03:22 PM, Erik ?sterlund wrote: >> Hi, >> >> Today the nmethods are unloaded either serially or in parallel due to GC >> (and e.g. class unloading). With ZGC concurrent class unloading, a >> concurrent fashion is required. Rather than inventing yet a third way of >> unloading nmethods due to class unloading, it would be ideal if there >> could be one unified way of doing it that makes everyone happy. >> >> To solve this problem in a more general way, a new member function on >> CompiledMethod is added: is_unloading(). It returns whether a >> CompiledMethod has broken oops. In order to not have to iterate over all >> the oops every time this question is asked, it caches the result, so it >> needs to be computed only once per "epoch". Every time a CodeCache >> unloading is triggered by the GC, a new epoch is started, which forces >> re-computation of whether the CompiledMethod is_unloading() the first >> time it is called. >> >> So by having is_unloading() be lazily evaluated, it is now possible to >> build a do_unloading() method on nmethod that simply makes the nmethod >> unloaded if it is_unloading(), and otherwise cleans the various caches. >> Cleaning e.g. the inline caches of said nmethod, uses the same >> is_unloading() method on its targets to figure out if the IC cache >> should be cleaned or not. Again, the epoched caching mechanism makes >> sure we don't recompute this value. >> >> The new do_unloading() method may be called in both serial, parallel and >> concurrent contexts. So I removed the parallel variation of this that we >> had before, that unnecessarily postponed the unloading due to not having >> computed whether the nmethod should be unloaded or not. Since that is >> now computed on-demand lazily, there is no longer a need for postponing >> this, nor to have two phases for parallel nmethod unloading. >> >> While there is more work involved in making concurrent nmethod unloading >> work, this is a good cleanup towards that goal. >> >> Bug ID: >> https://bugs.openjdk.java.net/browse/JDK-8209189 >> >> Webrev: >> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ > http://cr.openjdk.java.net/home/eosterlu/oracle/jdk/old_repos/jdk-hs/open > > 404 - Not Found > >> Thanks, >> /Erik >> > From aph at redhat.com Tue Oct 23 13:35:59 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 23 Oct 2018 14:35:59 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BCF204F.6030705@oracle.com> References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> <5BCF204F.6030705@oracle.com> Message-ID: On 10/23/2018 02:21 PM, Erik ?sterlund wrote: > Hi Andrew, > > You may prepend "http://" to the URL I posted in the email: > > http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ It doesn't work. Please go to http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ and click on the top link on the page. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Tue Oct 23 14:32:49 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 23 Oct 2018 16:32:49 +0200 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> <5BCF204F.6030705@oracle.com> Message-ID: <5BCF3111.6020105@oracle.com> Hi Andrew, That link is to the repository. Yes it is wrong because it's a local URL (because I have a local remote that I push to, to prevent accidental pushing). Please ignore that. It's based on jdk-jdk. The links to the files should work though. Thanks, /Erik On 2018-10-23 15:35, Andrew Haley wrote: > On 10/23/2018 02:21 PM, Erik ?sterlund wrote: >> Hi Andrew, >> >> You may prepend "http://" to the URL I posted in the email: >> >> http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ > It doesn't work. Please go to > > http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ > > and click on the top link on the page. > From gromero at linux.vnet.ibm.com Tue Oct 23 14:57:13 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 23 Oct 2018 11:57:13 -0300 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> Message-ID: Hi Martin, On 10/22/2018 07:07 AM, Doerr, Martin wrote: > Hi Gustavo, > > thanks for contributing. I have some minor remarks. Thanks a lot for reviewing it. > I'm not sure if it's good to have L=1 hard coded in the assembler instruction, but I think it's ok for now. > (One could also pass L as parameter with default value 1.) I agree. So, if you don't mind, to reduce any future rework on that I removed the hardcoded L=1 and now it's just a default. I introduced l14() function that matches two bits in the proper field as I don't see any collision with any other one bit field at the same position (bit 14). > I think it would be nice to have the capability bit usages in the same order at all places in the source code. We could add a comment line like "rtm is determined by OS" where it was omitted. An empty line before has_mtfprd() would be good because it's just an alias. Having them everywhere as one block in the same order helps avoiding confusion. Sure. I think that the confusion came from the fact that I split the "CPU instruction support" section in two by adding a new one called "OS instruction support" when I refactored the TM detection. Given a second thought tho I think it's not necessary, so I've put has_tm() back to the block as you instructed, adding an empty line before has_mtfprd() as it's an alias, and added the "rtm is deter- mined by OS" comment were it applies. v2 webrev: http://cr.openjdk.java.net/~gromero/8212481/v2/ I just have one question regarding the feature-string. I see instrumentation for "fsqrts" feature but it's not currently advertised by the feature-string. On the other hand, ISA info looks to indicate that fsqrt and fsqrts are not aliases, because: fsqrt P2 -> Instruction introduced in the POWER2 Architecture. fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 so I'm wondering if just for completeness the "fsqrts" feature should be included in the feature-string. Besides that I don't see instrumentation to support has_mftgpr(), which is commented out, thus should we remove it? Like the following: diff -r c860b03741ab src/hotspot/cpu/ppc/vm_version_ppc.cpp --- a/src/hotspot/cpu/ppc/vm_version_ppc.cpp Tue Oct 16 16:26:28 2018 -0400 +++ b/src/hotspot/cpu/ppc/vm_version_ppc.cpp Tue Oct 23 10:51:22 2018 -0400 @@ -134,12 +134,12 @@ // Create and print feature-string. char buf[(num_features+1) * 16]; // Max 16 chars per feature. jio_snprintf(buf, sizeof(buf), - "ppc64%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s", + "ppc64%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s", (has_fsqrt() ? " fsqrt" : ""), + (has_fsqrts() ? " fsqrts" : ""), (has_isel() ? " isel" : ""), (has_lxarxeh() ? " lxarxeh" : ""), (has_cmpb() ? " cmpb" : ""), - //(has_mftgpr()? " mftgpr" : ""), (has_popcntb() ? " popcntb" : ""), (has_popcntw() ? " popcntw" : ""), (has_fcfids() ? " fcfids" : ""), Thank you. Best regards, Gustavo > Besides this, the change looks good. > > Best regards, > Martin > > > -----Original Message----- > From: ppc-aix-port-dev On Behalf Of Gustavo Romero > Sent: Dienstag, 16. Oktober 2018 23:10 > To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection > > Hi, > > Could the following small change be reviewed please? > > Bug : https://bugs.openjdk.java.net/browse/JDK-8212481 > Webrev: http://cr.openjdk.java.net/~gromero/8212481/v1/ > > It adds 'darn' instruction introduced with POWER9 CPUs to the Macroassembler and > uses it to set PowerArchitecturePPC64 to POWER9 when the instruction is > available on the CPU so it can be used in the future by any POWER9-specific > JVM code or by any JVM code specifically dependent on the 'darn' instruction, > using VM_Version::has_darn(). > > It was checked on POWER9 as the following: > > gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ lscpu > Architecture: ppc64le > Byte Order: Little Endian > CPU(s): 176 > On-line CPU(s) list: 0-175 > Thread(s) per core: 4 > Core(s) per socket: 22 > Socket(s): 2 > NUMA node(s): 8 > Model: 2.2 (pvr 004e 1202) > Model name: POWER9 (raw), altivec supported > CPU max MHz: 2800.0000 > CPU min MHz: 2300.0000 > L1d cache: 32K > L1i cache: 32K > L2 cache: 512K > L3 cache: 10240K > NUMA node0 CPU(s): 0-87 > NUMA node8 CPU(s): 88-175 > NUMA node250 CPU(s): > NUMA node251 CPU(s): > NUMA node252 CPU(s): > NUMA node253 CPU(s): > NUMA node254 CPU(s): > NUMA node255 CPU(s): > gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ LD_SHOW_AUXV=1 /bin/true | egrep "HWCAP|PLATFORM" > AT_HWCAP: true_le archpmu vsx arch_2_06 dfp ic_snoop smt mmu fpu altivec ppc64 ppc32 > AT_HWCAP2: darn ieee128 arch_3_00 vcrypto tar isel ebb dscr arch_2_07 > AT_PLATFORM: power9 > AT_BASE_PLATFORM:power9 > gromero at ltc-wspoon3:~/POWER9/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ ./java -XX:+Verbose -version > dscr value was 0x10 > Version: ppc64 fsqrt isel lxarxeh cmpb popcntb popcntw fcfids vand lqarx aes vpmsumb mfdscr vsx ldbrx stdbrx sha darn L1_data_cache_line_size=128 > > openjdk version "12-internal" 2019-03-19 > OpenJDK Runtime Environment (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9) > OpenJDK 64-Bit Server VM (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9, mixed mode, sharing) > > > It was also checked on POWER8 and no regression was observed: > > gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ lscpu > Architecture: ppc64le > Byte Order: Little Endian > CPU(s): 80 > On-line CPU(s) list: 0-79 > Thread(s) per core: 8 > Core(s) per socket: 10 > Socket(s): 1 > NUMA node(s): 1 > Model: 2.1 (pvr 004b 0201) > Model name: POWER8 (architected), altivec supported > Hypervisor vendor: horizontal > Virtualization type: full > L1d cache: 64K > L1i cache: 32K > NUMA node0 CPU(s): 0-79 > gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ LD_SHOW_AUXV=1 /bin/true | egrep "HWCAP|PLATFORM" > AT_HWCAP: archpmu vsx arch_2_06 dfp ic_snoop smt mmu fpu altivec ppc64 ppc32 > AT_HWCAP2: htm-nosc vcrypto tar isel ebb dscr htm arch_2_07 > AT_PLATFORM: power8 > AT_BASE_PLATFORM:power8 > gromero at gromero16:~/openjdks/jdk12_tip_f53671e05660+/jvm/openjdk-12-internal/bin$ ./java -XX:+Verbose -version > dscr value was 0x0 > Version: ppc64 fsqrt isel lxarxeh cmpb popcntb popcntw fcfids vand lqarx aes vpmsumb mfdscr vsx ldbrx stdbrx sha rtm L1_data_cache_line_size=128 > > openjdk version "12-internal" 2019-03-19 > OpenJDK Runtime Environment (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9) > OpenJDK 64-Bit Server VM (fastdebug build 12-internal+0-f53671e05660.dirty.debug.POWER9, mixed mode, sharing) > > > Thank you. > > Best regards, > Gustavo > From erik.osterlund at oracle.com Tue Oct 23 14:59:50 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 23 Oct 2018 16:59:50 +0200 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() Message-ID: <5BCF3766.3050504@oracle.com> Hi, We occasionally blockingly wait for GC in allocations in ObjArrayKlass::allocate_objArray_klass(), while holding the Compile_lock. This is problematic for concurrent class unloading that needs to hold that lock in the unloading path. This introduces a potential deadlock situation. After staring enough at this code together with help from Coleen, I have convinced myself that the Compile_lock is in fact not needed in this path, and there is nothing to my knowledge it actually protects here. The vague comment next to the lock suggests its purpose is to protect vtable stubs. But it doesn't. There is a VtableStubs_lock for that, and I find no traces if the Compile_lock having anything to do with that. Coleen helped me trying to trace down where this locking code came from. It takes us back to before anyone was around, and does not seem to have changed in the past 2 decades, and its origins are a bit unclear. The theory is that perhaps vtable stubs used to be protected by the Compile_lock in ancient times, and the locking code is still around because nobody dared to touch it. Since both code analysis and mach5 suggest there is no reason for this lock to be held here, I propose to remove it with this patch. If anyone that has been around for more than 2 decades in HotSpot happens to know the reason why this locking was introduced, any commentary around that would be very appreciated. Webrev: http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212682 Thanks, /Erik From aph at redhat.com Tue Oct 23 15:48:40 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 23 Oct 2018 16:48:40 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BCF3111.6020105@oracle.com> References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> <5BCF204F.6030705@oracle.com> <5BCF3111.6020105@oracle.com> Message-ID: <2e538f61-0c4d-3e29-2829-357bc3b60baf@redhat.com> On 10/23/2018 03:32 PM, Erik ?sterlund wrote: > That link is to the repository. Yes it is wrong because it's a local URL > (because I have a local remote that I push to, to prevent accidental > pushing). Please ignore that. It's based on jdk-jdk. > The links to the files should work though. Please put the patch on cr.openjdk. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Tue Oct 23 16:25:04 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Tue, 23 Oct 2018 18:25:04 +0200 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <2e538f61-0c4d-3e29-2829-357bc3b60baf@redhat.com> References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> <5BCF204F.6030705@oracle.com> <5BCF3111.6020105@oracle.com> <2e538f61-0c4d-3e29-2829-357bc3b60baf@redhat.com> Message-ID: <2678A53A-31C0-4F00-9CE7-8371C5CF0C08@oracle.com> Hi Andrew, It is already there though: http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/open.patch Does this link not work for you? Thanks, /Erik > On 23 Oct 2018, at 17:48, Andrew Haley wrote: > >> On 10/23/2018 03:32 PM, Erik ?sterlund wrote: >> That link is to the repository. Yes it is wrong because it's a local URL >> (because I have a local remote that I push to, to prevent accidental >> pushing). Please ignore that. It's based on jdk-jdk. >> The links to the files should work though. > > Please put the patch on cr.openjdk. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From dean.long at oracle.com Tue Oct 23 16:46:48 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 23 Oct 2018 09:46:48 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> Message-ID: On 10/22/18 3:31 PM, David Holmes wrote: > Sorry Dean I'm concerned about a thread termination bottleneck with > this. A simple microbenchmark that creates 500,000 threads that have > to run and terminate, shows a 15+% slowdown on my Linux box. I tried > to find some kind of real benchmarks that covers thread termination > but couldn't see anything specific. > > Can you at least run this through our performance system to see if any > of the regular benchmarks are affected. > OK, but even if the regular benchmarks don't show a difference, I'd feel better if microbenchmarks were not affected.? What if I go back to the original approach and add locking: ?? static jlong get_live_thread_count()??????? { MutexLocker mu(Threads_lock); return _live_threads_count->get_value() - _exiting_threads_count; } ?? static jlong get_daemon_thread_count()????? { MutexLocker mu(Threads_lock); return _daemon_threads_count->get_value() - _exiting_daemon_threads_count; } along with the other cleanups around is_daemon and is_hidden_thread? dl > Thanks, > David > > On 20/10/2018 1:28 PM, dean.long at oracle.com wrote: >> On 10/18/18 6:12 PM, Mandy Chung wrote: >>> >>> >>> On 10/18/18 12:27 AM, David Holmes wrote: >>>> Hi Dean, >>>> >>>> On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: >>>>> >>>>> You're right, I missed that.? I think the right thing to do is >>>>> call current_thread_exiting while holding the Threads_lock. >>>>> Then we can get rid of the parallel atomic counters.? So, here's >>>>> one more try: >>>>> >>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ >>>> >>>> Okay that is the simple and obvious solution that doesn't require >>>> split counts. So I have to ask Mandy if she recalls why this >>>> approach wasn't taken 15 years ago when the exit counts were added >>>> as part of: >>>> >>> >>> It has been so long.? I think it's likely an oversight. >>>> https://bugs.openjdk.java.net/browse/JDK-4530538 ? >>>> >>>> Does taking the Threads_lock here cost too much and cause a thread >>>> termination bottleneck? >>> >>> If the contention on Threads_lock is not high (that seems to me), it >>> should be okay. ? I'm not close to the VM implementation (lot of >>> changes since then) and I don't have a definitive answer unless I >>> study the code closely.?? You and others have a better judgement on >>> this. >>> >>> AFAICT the change is okay. >>> >> >> Thanks Mandy.? David, OK to push? >> >> dl >> >>> Mandy >>> >>> >> From martin.doerr at sap.com Tue Oct 23 16:51:50 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 23 Oct 2018 16:51:50 +0000 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> Message-ID: Hi Gustavo, > I agree. So, if you don't mind, to reduce any future rework on that > I removed the hardcoded L=1 and now it's just a default. I introduced > l14() function that matches two bits in the proper field as I don't > see any collision with any other one bit field at the same position > (bit 14). I like it, but the default value needs to get specified in the .hpp file. > Sure. I think that the confusion came from the fact that I split > the "CPU instruction support" section in two by adding a new one > called "OS instruction support" when I refactored the TM detection. > Given a second thought tho I think it's not necessary, so I've put > has_tm() back to the block as you instructed, adding an empty line > before has_mtfprd() as it's an alias, and added the "rtm is deter- > mined by OS" comment were it applies. Thanks for cleaning it up. > I just have one question regarding the feature-string. I see > instrumentation for "fsqrts" feature but it's not currently > advertised by the feature-string. On the other hand, ISA info > looks to indicate that fsqrt and fsqrts are not aliases, because: > > fsqrt P2 -> Instruction introduced in the POWER2 Architecture. > fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 > > so I'm wondering if just for completeness the "fsqrts" feature > should be included in the feature-string. Besides that I don't > see instrumentation to support has_mftgpr(), which is commented > out, thus should we remove it? Like the following: This code is very old. If we are sure nobody can run the VM on a machine without these instructions they can get removed from the feature checking. The mftgpr was only designed for an extension of Power6 and the opcode was reused for something else later on if I remember correctly. I think you can remove the remaining comment. Best regards, Martin From kim.barrett at oracle.com Tue Oct 23 17:07:30 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Oct 2018 13:07:30 -0400 Subject: RFR: 8212827: GlobalCounter should support nested critical sections Message-ID: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> Please review this change to GlobalCounter to support nested critical sections. While in the area, cleaned up a few things: - Improved comments for GlobalCounter API. - Cleaned up write_synchronize global counter increment. - Removed a useless acquire barrier in critical_section_begin. CR: https://bugs.openjdk.java.net/browse/JDK-8212827 Webrev: http://cr.openjdk.java.net/~kbarrett/8212827/open.00/ Testing: mach5 tier1-5 New gtest directly testing nested critical sections. From dean.long at oracle.com Tue Oct 23 18:05:00 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 23 Oct 2018 11:05:00 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> Message-ID: <5b6265b6-36b3-4317-1e6a-8ba51c8a48b3@oracle.com> On 10/23/18 9:46 AM, dean.long at oracle.com wrote: > On 10/22/18 3:31 PM, David Holmes wrote: >> Sorry Dean I'm concerned about a thread termination bottleneck with >> this. A simple microbenchmark that creates 500,000 threads that have >> to run and terminate, shows a 15+% slowdown on my Linux box. I tried >> to find some kind of real benchmarks that covers thread termination >> but couldn't see anything specific. >> >> Can you at least run this through our performance system to see if >> any of the regular benchmarks are affected. >> > > OK, but even if the regular benchmarks don't show a difference, I'd > feel better if microbenchmarks were not affected.? What if I go back > to the original approach and add locking: > > ?? static jlong get_live_thread_count()??????? { MutexLocker > mu(Threads_lock); return _live_threads_count->get_value() - > _exiting_threads_count; } > ?? static jlong get_daemon_thread_count()????? { MutexLocker > mu(Threads_lock); return _daemon_threads_count->get_value() - > _exiting_daemon_threads_count; } > > along with the other cleanups around is_daemon and is_hidden_thread? > Some micro-benchmarks like SecureRandomBench show a regression with webrev.7.? I could go back to webrev.5 and then we shouldn't need any locking in the get_*() functions. dl > dl > >> Thanks, >> David >> >> On 20/10/2018 1:28 PM, dean.long at oracle.com wrote: >>> On 10/18/18 6:12 PM, Mandy Chung wrote: >>>> >>>> >>>> On 10/18/18 12:27 AM, David Holmes wrote: >>>>> Hi Dean, >>>>> >>>>> On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: >>>>>> >>>>>> You're right, I missed that.? I think the right thing to do is >>>>>> call current_thread_exiting while holding the Threads_lock. >>>>>> Then we can get rid of the parallel atomic counters. So, here's >>>>>> one more try: >>>>>> >>>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ >>>>> >>>>> Okay that is the simple and obvious solution that doesn't require >>>>> split counts. So I have to ask Mandy if she recalls why this >>>>> approach wasn't taken 15 years ago when the exit counts were added >>>>> as part of: >>>>> >>>> >>>> It has been so long.? I think it's likely an oversight. >>>>> https://bugs.openjdk.java.net/browse/JDK-4530538 ? >>>>> >>>>> Does taking the Threads_lock here cost too much and cause a thread >>>>> termination bottleneck? >>>> >>>> If the contention on Threads_lock is not high (that seems to me), >>>> it should be okay. ? I'm not close to the VM implementation (lot of >>>> changes since then) and I don't have a definitive answer unless I >>>> study the code closely.?? You and others have a better judgement on >>>> this. >>>> >>>> AFAICT the change is okay. >>>> >>> >>> Thanks Mandy.? David, OK to push? >>> >>> dl >>> >>>> Mandy >>>> >>>> >>> > From coleen.phillimore at oracle.com Tue Oct 23 18:51:22 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 23 Oct 2018 14:51:22 -0400 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <5BCF3766.3050504@oracle.com> References: <5BCF3766.3050504@oracle.com> Message-ID: <05dd085d-08a0-ca41-202c-12a7452985b7@oracle.com> I wonder if the Compile_lock is added for this in ci/ciEnv.cpp (and it's copy in jvmtiEnv.cpp): ciKlass* ciEnv::get_klass_by_name_impl(ciKlass* accessing_klass, ... ??? MutexLocker ml(Compile_lock); ??? Klass* kls; ??? if (!require_local) { ????? kls = SystemDictionary::find_constrained_instance_or_array_klass(sym, loader, KILL_COMPILE_ON_FATAL_(fail_type)); ??? } else { ????? kls = SystemDictionary::find_instance_or_array_klass(sym, loader, domain, KILL_COMPILE_ON_FATAL_(fail_type)); ??? } ??? found_klass = kls; ? } which eventually calls in SystemDictionary:: ??? if (k != NULL) { ????? k = k->array_klass_or_null(fd.dimension()); ??? } And maybe the Compile_lock keeps the Klass::_higher_dimension have a consistent value.? Maybe the code in ciEnv.cpp and jvmciEnv.cpp should take MultiArray_lock instead.? The problem is that Compile_lock protects too many things. I don't know if this is safe to remove unfortunately. Coleen On 10/23/18 10:59 AM, Erik ?sterlund wrote: > Hi, > > We occasionally blockingly wait for GC in allocations in > ObjArrayKlass::allocate_objArray_klass(), while holding the Compile_lock. > This is problematic for concurrent class unloading that needs to hold > that lock in the unloading path. This introduces a potential deadlock > situation. > > After staring enough at this code together with help from Coleen, I > have convinced myself that the Compile_lock is in fact not needed in > this path, and there is nothing to my knowledge it actually protects > here. The vague comment next to the lock suggests its purpose is to > protect vtable stubs. But it doesn't. There is a VtableStubs_lock for > that, and I find no traces if the Compile_lock having anything to do > with that. > > Coleen helped me trying to trace down where this locking code came > from. It takes us back to before anyone was around, and does not seem > to have changed in the past 2 decades, and its origins are a bit > unclear. The theory is that perhaps vtable stubs used to be protected > by the Compile_lock in ancient times, and the locking code is still > around because nobody dared to touch it. > > Since both code analysis and mach5 suggest there is no reason for this > lock to be held here, I propose to remove it with this patch. > > If anyone that has been around for more than 2 decades in HotSpot > happens to know the reason why this locking was introduced, any > commentary around that would be very appreciated. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212682 > > Thanks, > /Erik From dean.long at oracle.com Tue Oct 23 20:27:12 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 23 Oct 2018 13:27:12 -0700 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <5BCF3766.3050504@oracle.com> References: <5BCF3766.3050504@oracle.com> Message-ID: Can allocate_objArray_klass() end up calling SystemDictionary::add_to_hierarchy()?? If so, then you'll end up locking Compile_lock anway, but in the wrong order with respect to MultiArray_lock. dl On 10/23/18 7:59 AM, Erik ?sterlund wrote: > Hi, > > We occasionally blockingly wait for GC in allocations in > ObjArrayKlass::allocate_objArray_klass(), while holding the Compile_lock. > This is problematic for concurrent class unloading that needs to hold > that lock in the unloading path. This introduces a potential deadlock > situation. > > After staring enough at this code together with help from Coleen, I > have convinced myself that the Compile_lock is in fact not needed in > this path, and there is nothing to my knowledge it actually protects > here. The vague comment next to the lock suggests its purpose is to > protect vtable stubs. But it doesn't. There is a VtableStubs_lock for > that, and I find no traces if the Compile_lock having anything to do > with that. > > Coleen helped me trying to trace down where this locking code came > from. It takes us back to before anyone was around, and does not seem > to have changed in the past 2 decades, and its origins are a bit > unclear. The theory is that perhaps vtable stubs used to be protected > by the Compile_lock in ancient times, and the locking code is still > around because nobody dared to touch it. > > Since both code analysis and mach5 suggest there is no reason for this > lock to be held here, I propose to remove it with this patch. > > If anyone that has been around for more than 2 decades in HotSpot > happens to know the reason why this locking was introduced, any > commentary around that would be very appreciated. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212682 > > Thanks, > /Erik From gromero at linux.vnet.ibm.com Tue Oct 23 21:46:14 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 23 Oct 2018 18:46:14 -0300 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> Message-ID: <271b6743-1611-4201-cdf6-9fdc5b87086e@linux.vnet.ibm.com> Hi Martin, Thanks for your comments on fsqrt{,s} and mftgpr opcodes. On 10/23/2018 01:51 PM, Doerr, Martin wrote: > Hi Gustavo, > >> I agree. So, if you don't mind, to reduce any future rework on that >> I removed the hardcoded L=1 and now it's just a default. I introduced >> l14() function that matches two bits in the proper field as I don't >> see any collision with any other one bit field at the same position >> (bit 14). > I like it, but the default value needs to get specified in the .hpp file. Fixed. >> I just have one question regarding the feature-string. I see >> instrumentation for "fsqrts" feature but it's not currently >> advertised by the feature-string. On the other hand, ISA info >> looks to indicate that fsqrt and fsqrts are not aliases, because: >> >> fsqrt P2 -> Instruction introduced in the POWER2 Architecture. >> fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 >> >> so I'm wondering if just for completeness the "fsqrts" feature >> should be included in the feature-string. Besides that I don't >> see instrumentation to support has_mftgpr(), which is commented >> out, thus should we remove it? Like the following: > > This code is very old. If we are sure nobody can run the VM on a machine without these instructions they can get removed from the feature checking. I'm not sure about it. Maybe somebody is using out there, in the community for example, so kept it. > The mftgpr was only designed for an extension of Power6 and the opcode was reused for something else later on if I remember correctly. I think you can remove the remaining comment. I was not aware of that, thanks for the info. Just out of curiosity I asked the toolchain folks and they said that mftgpr is present in some POWER6 hardware but on the so-called raw mode (ie power6x, which I think it's the extension you mentioned), but not in the architected / normal mode (power6). So even if it's present in the hardware it's disabled unless you boot the system up in raw mode. On that raw mode (power6x) one would see the following in AUXV, for instance: $ LD_SHOW_AUXV=1 /bin/true | grep PLATFORM AT_PLATFORM: power6x AT_BASE_PLATFORM:power6x But probably theses machines w/ power6x never got out of IBM indeed. Anyway, they said that power6x is unsupported. So, as you said, I removed the comment about mftgpr. v3 webrev: http://cr.openjdk.java.net/~gromero/8212481/v3/ Thanks! Best regards, Gustavo From david.holmes at oracle.com Tue Oct 23 21:51:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Oct 2018 07:51:34 +1000 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <5b6265b6-36b3-4317-1e6a-8ba51c8a48b3@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> <5b6265b6-36b3-4317-1e6a-8ba51c8a48b3@oracle.com> Message-ID: <0560343b-94b4-2261-0f1d-a08d7067e97b@oracle.com> Hi Dean, On 24/10/2018 4:05 AM, dean.long at oracle.com wrote: > On 10/23/18 9:46 AM, dean.long at oracle.com wrote: >> On 10/22/18 3:31 PM, David Holmes wrote: >>> Sorry Dean I'm concerned about a thread termination bottleneck with >>> this. A simple microbenchmark that creates 500,000 threads that have >>> to run and terminate, shows a 15+% slowdown on my Linux box. I tried >>> to find some kind of real benchmarks that covers thread termination >>> but couldn't see anything specific. >>> >>> Can you at least run this through our performance system to see if >>> any of the regular benchmarks are affected. >>> >> >> OK, but even if the regular benchmarks don't show a difference, I'd >> feel better if microbenchmarks were not affected.? What if I go back >> to the original approach and add locking: >> >> ?? static jlong get_live_thread_count()??????? { MutexLocker >> mu(Threads_lock); return _live_threads_count->get_value() - >> _exiting_threads_count; } >> ?? static jlong get_daemon_thread_count()????? { MutexLocker >> mu(Threads_lock); return _daemon_threads_count->get_value() - >> _exiting_daemon_threads_count; } >> >> along with the other cleanups around is_daemon and is_hidden_thread? >> > > Some micro-benchmarks like SecureRandomBench show a regression with > webrev.7.? I could go back to webrev.5 and then we shouldn't need any > locking in the get_*() functions. I don't see version 5 discussed but I took a look and it seems okay. My only query with that version is that it appears the actual perfCounters no longer get read by anything - is that the case? Thanks, David > dl > >> dl >> >>> Thanks, >>> David >>> >>> On 20/10/2018 1:28 PM, dean.long at oracle.com wrote: >>>> On 10/18/18 6:12 PM, Mandy Chung wrote: >>>>> >>>>> >>>>> On 10/18/18 12:27 AM, David Holmes wrote: >>>>>> Hi Dean, >>>>>> >>>>>> On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: >>>>>>> >>>>>>> You're right, I missed that.? I think the right thing to do is >>>>>>> call current_thread_exiting while holding the Threads_lock. >>>>>>> Then we can get rid of the parallel atomic counters. So, here's >>>>>>> one more try: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ >>>>>> >>>>>> Okay that is the simple and obvious solution that doesn't require >>>>>> split counts. So I have to ask Mandy if she recalls why this >>>>>> approach wasn't taken 15 years ago when the exit counts were added >>>>>> as part of: >>>>>> >>>>> >>>>> It has been so long.? I think it's likely an oversight. >>>>>> https://bugs.openjdk.java.net/browse/JDK-4530538 ? >>>>>> >>>>>> Does taking the Threads_lock here cost too much and cause a thread >>>>>> termination bottleneck? >>>>> >>>>> If the contention on Threads_lock is not high (that seems to me), >>>>> it should be okay. ? I'm not close to the VM implementation (lot of >>>>> changes since then) and I don't have a definitive answer unless I >>>>> study the code closely.?? You and others have a better judgement on >>>>> this. >>>>> >>>>> AFAICT the change is okay. >>>>> >>>> >>>> Thanks Mandy.? David, OK to push? >>>> >>>> dl >>>> >>>>> Mandy >>>>> >>>>> >>>> >> > From david.holmes at oracle.com Tue Oct 23 22:11:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Oct 2018 08:11:35 +1000 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <2e538f61-0c4d-3e29-2829-357bc3b60baf@redhat.com> References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> <5BCF204F.6030705@oracle.com> <5BCF3111.6020105@oracle.com> <2e538f61-0c4d-3e29-2829-357bc3b60baf@redhat.com> Message-ID: <53f86f99-21d1-16c0-cd9f-181ac69a7faf@oracle.com> On 24/10/2018 1:48 AM, Andrew Haley wrote: > On 10/23/2018 03:32 PM, Erik ?sterlund wrote: >> That link is to the repository. Yes it is wrong because it's a local URL >> (because I have a local remote that I push to, to prevent accidental >> pushing). Please ignore that. It's based on jdk-jdk. >> The links to the files should work though. > > Please put the patch on cr.openjdk. The patch is on cr.openjdk. The only "issue" is that the webrev contains a "link" to the original repository that is completely bogus because the repository (naturally) is not on cr.openjdk. The version of webrev I use doesn't do that it just has this non-link entry: Workspace: /export/users/dh198349/jdk-dev/open Cheers, David From dean.long at oracle.com Tue Oct 23 22:30:35 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 23 Oct 2018 15:30:35 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <0560343b-94b4-2261-0f1d-a08d7067e97b@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> <5b6265b6-36b3-4317-1e6a-8ba51c8a48b3@oracle.com> <0560343b-94b4-2261-0f1d-a08d7067e97b@oracle.com> Message-ID: <43be9a19-d0c5-298a-9bcb-e98223f4566a@oracle.com> On 10/23/18 2:51 PM, David Holmes wrote: > Hi Dean, > > On 24/10/2018 4:05 AM, dean.long at oracle.com wrote: >> On 10/23/18 9:46 AM, dean.long at oracle.com wrote: >>> On 10/22/18 3:31 PM, David Holmes wrote: >>>> Sorry Dean I'm concerned about a thread termination bottleneck with >>>> this. A simple microbenchmark that creates 500,000 threads that >>>> have to run and terminate, shows a 15+% slowdown on my Linux box. I >>>> tried to find some kind of real benchmarks that covers thread >>>> termination but couldn't see anything specific. >>>> >>>> Can you at least run this through our performance system to see if >>>> any of the regular benchmarks are affected. >>>> >>> >>> OK, but even if the regular benchmarks don't show a difference, I'd >>> feel better if microbenchmarks were not affected.? What if I go back >>> to the original approach and add locking: >>> >>> ?? static jlong get_live_thread_count()??????? { MutexLocker >>> mu(Threads_lock); return _live_threads_count->get_value() - >>> _exiting_threads_count; } >>> ?? static jlong get_daemon_thread_count()????? { MutexLocker >>> mu(Threads_lock); return _daemon_threads_count->get_value() - >>> _exiting_daemon_threads_count; } >>> >>> along with the other cleanups around is_daemon and is_hidden_thread? >>> >> >> Some micro-benchmarks like SecureRandomBench show a regression with >> webrev.7.? I could go back to webrev.5 and then we shouldn't need any >> locking in the get_*() functions. > > I don't see version 5 discussed but I took a look and it seems okay. Mandy had questions about the asserts in .5, and it seemed like we could just set the perf counters to the same value as the atomic counters, which resulted in .6.? I think the only problem with .6 is that I set the perf counters in decrement_thread_counts without the lock.? If I "sync" the perf counters to the atomic counters only in add_thread and remove_thread, with the lock, then it's about the same as .5, but without the asserts and parallel inc/dec.? If anyone likes the sound of that, I can send out a new webrev.? Or we can go with webrev.5. > My only query with that version is that it appears the actual > perfCounters no longer get read by anything - is that the case? > There does seem to be code that references them, through their name string.? That's a difference interface that I'm not familiar with, so I didn't want to break it. dl > Thanks, > David > >> dl >> >>> dl >>> >>>> Thanks, >>>> David >>>> >>>> On 20/10/2018 1:28 PM, dean.long at oracle.com wrote: >>>>> On 10/18/18 6:12 PM, Mandy Chung wrote: >>>>>> >>>>>> >>>>>> On 10/18/18 12:27 AM, David Holmes wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: >>>>>>>> >>>>>>>> You're right, I missed that.? I think the right thing to do is >>>>>>>> call current_thread_exiting while holding the Threads_lock. >>>>>>>> Then we can get rid of the parallel atomic counters. So, here's >>>>>>>> one more try: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ >>>>>>> >>>>>>> Okay that is the simple and obvious solution that doesn't >>>>>>> require split counts. So I have to ask Mandy if she recalls why >>>>>>> this approach wasn't taken 15 years ago when the exit counts >>>>>>> were added as part of: >>>>>>> >>>>>> >>>>>> It has been so long.? I think it's likely an oversight. >>>>>>> https://bugs.openjdk.java.net/browse/JDK-4530538 ? >>>>>>> >>>>>>> Does taking the Threads_lock here cost too much and cause a >>>>>>> thread termination bottleneck? >>>>>> >>>>>> If the contention on Threads_lock is not high (that seems to me), >>>>>> it should be okay. ? I'm not close to the VM implementation (lot >>>>>> of changes since then) and I don't have a definitive answer >>>>>> unless I study the code closely.?? You and others have a better >>>>>> judgement on this. >>>>>> >>>>>> AFAICT the change is okay. >>>>>> >>>>> >>>>> Thanks Mandy.? David, OK to push? >>>>> >>>>> dl >>>>> >>>>>> Mandy >>>>>> >>>>>> >>>>> >>> >> From mandy.chung at oracle.com Wed Oct 24 03:22:15 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Oct 2018 20:22:15 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <43be9a19-d0c5-298a-9bcb-e98223f4566a@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> <5b6265b6-36b3-4317-1e6a-8ba51c8a48b3@oracle.com> <0560343b-94b4-2261-0f1d-a08d7067e97b@oracle.com> <43be9a19-d0c5-298a-9bcb-e98223f4566a@oracle.com> Message-ID: <385be5ad-c124-c9b2-9339-006683abd953@oracle.com> On 10/23/18 3:30 PM, dean.long at oracle.com wrote: > On 10/23/18 2:51 PM, David Holmes wrote: >> >>>> OK, but even if the regular benchmarks don't show a difference, I'd >>>> feel better if microbenchmarks were not affected.? What if I go >>>> back to the original approach and add locking: >>>> >>>> ?? static jlong get_live_thread_count()??????? { MutexLocker >>>> mu(Threads_lock); return _live_threads_count->get_value() - >>>> _exiting_threads_count; } >>>> ?? static jlong get_daemon_thread_count()????? { MutexLocker >>>> mu(Threads_lock); return _daemon_threads_count->get_value() - >>>> _exiting_daemon_threads_count; } >>>> >>>> along with the other cleanups around is_daemon and is_hidden_thread? >>>> >>> >>> Some micro-benchmarks like SecureRandomBench show a regression with >>> webrev.7.? I could go back to webrev.5 and then we shouldn't need >>> any locking in the get_*() functions. >> >> I don't see version 5 discussed but I took a look and it seems okay. > > Mandy had questions about the asserts in .5, and it seemed like we > could just set the perf counters to the same value as the atomic > counters, which resulted in .6.? I think the only problem with .6 is > that I set the perf counters in decrement_thread_counts without the > lock.? If I "sync" the perf counters to the atomic counters only in > add_thread and remove_thread, with the lock, then it's about the same > as .5, but without the asserts and parallel inc/dec.? If anyone likes > the sound of that, I can send out a new webrev. ThreadService::current_thread_exiting will decrement the atomic counters. add_thread will increase atomic counters and sync the perf counters with the atomic counters. remove_thread will decrement the atomic counters if thread is not exiting and update the perf counters to the atomic counters. I'm good with that. > Or we can go with webrev.5. > >> My only query with that version is that it appears the actual >> perfCounters no longer get read by anything - is that the case? >> > > There does seem to be code that references them, through their name > string.? That's a difference interface that I'm not familiar with, so > I didn't want to break it. The perf counters are exposed via jstat tool.? I don't know who relies on jstat to monitor thread counts.?? I suggest to check with the performance team if they depend on these counters.?? With JFR, I think we should re-examine whether these perf counters should stay. Mandy From david.holmes at oracle.com Wed Oct 24 05:34:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Oct 2018 15:34:03 +1000 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <43be9a19-d0c5-298a-9bcb-e98223f4566a@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> <5b6265b6-36b3-4317-1e6a-8ba51c8a48b3@oracle.com> <0560343b-94b4-2261-0f1d-a08d7067e97b@oracle.com> <43be9a19-d0c5-298a-9bcb-e98223f4566a@oracle.com> Message-ID: <98af2e32-b9e7-e06a-91fc-447baa445bed@oracle.com> On 24/10/2018 8:30 AM, dean.long at oracle.com wrote: > On 10/23/18 2:51 PM, David Holmes wrote: >> Hi Dean, >> >> On 24/10/2018 4:05 AM, dean.long at oracle.com wrote: >>> On 10/23/18 9:46 AM, dean.long at oracle.com wrote: >>>> On 10/22/18 3:31 PM, David Holmes wrote: >>>>> Sorry Dean I'm concerned about a thread termination bottleneck with >>>>> this. A simple microbenchmark that creates 500,000 threads that >>>>> have to run and terminate, shows a 15+% slowdown on my Linux box. I >>>>> tried to find some kind of real benchmarks that covers thread >>>>> termination but couldn't see anything specific. >>>>> >>>>> Can you at least run this through our performance system to see if >>>>> any of the regular benchmarks are affected. >>>>> >>>> >>>> OK, but even if the regular benchmarks don't show a difference, I'd >>>> feel better if microbenchmarks were not affected.? What if I go back >>>> to the original approach and add locking: >>>> >>>> ?? static jlong get_live_thread_count()??????? { MutexLocker >>>> mu(Threads_lock); return _live_threads_count->get_value() - >>>> _exiting_threads_count; } >>>> ?? static jlong get_daemon_thread_count()????? { MutexLocker >>>> mu(Threads_lock); return _daemon_threads_count->get_value() - >>>> _exiting_daemon_threads_count; } >>>> >>>> along with the other cleanups around is_daemon and is_hidden_thread? >>>> >>> >>> Some micro-benchmarks like SecureRandomBench show a regression with >>> webrev.7.? I could go back to webrev.5 and then we shouldn't need any >>> locking in the get_*() functions. >> >> I don't see version 5 discussed but I took a look and it seems okay. > > Mandy had questions about the asserts in .5, and it seemed like we could > just set the perf counters to the same value as the atomic counters, > which resulted in .6.? I think the only problem with .6 is that I set > the perf counters in decrement_thread_counts without the lock.? If I > "sync" the perf counters to the atomic counters only in add_thread and > remove_thread, with the lock, then it's about the same as .5, but > without the asserts and parallel inc/dec.? If anyone likes the sound of > that, I can send out a new webrev.? Or we can go with webrev.5. I'm not sure what the concern was with the asserts - if they mis-fire we'll know soon enough. So I'm okay with .5 >> My only query with that version is that it appears the actual >> perfCounters no longer get read by anything - is that the case? >> > > There does seem to be code that references them, through their name > string.? That's a difference interface that I'm not familiar with, so I > didn't want to break it. Right - they can be accessed directly through other means. I was concerned that the perfCounter was out of sync with get_live_thread-count() but that's already the case so not an issue. If all tests and benchmarks are happy I say go with version .5 Thanks, David > dl > >> Thanks, >> David >> >>> dl >>> >>>> dl >>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 20/10/2018 1:28 PM, dean.long at oracle.com wrote: >>>>>> On 10/18/18 6:12 PM, Mandy Chung wrote: >>>>>>> >>>>>>> >>>>>>> On 10/18/18 12:27 AM, David Holmes wrote: >>>>>>>> Hi Dean, >>>>>>>> >>>>>>>> On 18/10/2018 2:06 PM, dean.long at oracle.com wrote: >>>>>>>>> >>>>>>>>> You're right, I missed that.? I think the right thing to do is >>>>>>>>> call current_thread_exiting while holding the Threads_lock. >>>>>>>>> Then we can get rid of the parallel atomic counters. So, here's >>>>>>>>> one more try: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dlong/8021335/webrev.7/ >>>>>>>> >>>>>>>> Okay that is the simple and obvious solution that doesn't >>>>>>>> require split counts. So I have to ask Mandy if she recalls why >>>>>>>> this approach wasn't taken 15 years ago when the exit counts >>>>>>>> were added as part of: >>>>>>>> >>>>>>> >>>>>>> It has been so long.? I think it's likely an oversight. >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-4530538 ? >>>>>>>> >>>>>>>> Does taking the Threads_lock here cost too much and cause a >>>>>>>> thread termination bottleneck? >>>>>>> >>>>>>> If the contention on Threads_lock is not high (that seems to me), >>>>>>> it should be okay. ? I'm not close to the VM implementation (lot >>>>>>> of changes since then) and I don't have a definitive answer >>>>>>> unless I study the code closely.?? You and others have a better >>>>>>> judgement on this. >>>>>>> >>>>>>> AFAICT the change is okay. >>>>>>> >>>>>> >>>>>> Thanks Mandy.? David, OK to push? >>>>>> >>>>>> dl >>>>>> >>>>>>> Mandy >>>>>>> >>>>>>> >>>>>> >>>> >>> > From mandy.chung at oracle.com Wed Oct 24 06:04:30 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Oct 2018 23:04:30 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <98af2e32-b9e7-e06a-91fc-447baa445bed@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> <5b6265b6-36b3-4317-1e6a-8ba51c8a48b3@oracle.com> <0560343b-94b4-2261-0f1d-a08d7067e97b@oracle.com> <43be9a19-d0c5-298a-9bcb-e98223f4566a@oracle.com> <98af2e32-b9e7-e06a-91fc-447baa445bed@oracle.com> Message-ID: <4febfc04-21c3-fd91-db83-54777ebd7ca2@oracle.com> On 10/23/18 10:34 PM, David Holmes wrote: > On 24/10/2018 8:30 AM, dean.long at oracle.com wrote: >> On 10/23/18 2:51 PM, David Holmes wrote: >>> Hi Dean, >>> >>> On 24/10/2018 4:05 AM, dean.long at oracle.com wrote: >>>> On 10/23/18 9:46 AM, dean.long at oracle.com wrote: >>>>> On 10/22/18 3:31 PM, David Holmes wrote: >>>>>> Sorry Dean I'm concerned about a thread termination bottleneck >>>>>> with this. A simple microbenchmark that creates 500,000 threads >>>>>> that have to run and terminate, shows a 15+% slowdown on my Linux >>>>>> box. I tried to find some kind of real benchmarks that covers >>>>>> thread termination but couldn't see anything specific. >>>>>> >>>>>> Can you at least run this through our performance system to see >>>>>> if any of the regular benchmarks are affected. >>>>>> >>>>> >>>>> OK, but even if the regular benchmarks don't show a difference, >>>>> I'd feel better if microbenchmarks were not affected.? What if I >>>>> go back to the original approach and add locking: >>>>> >>>>> ?? static jlong get_live_thread_count()??????? { MutexLocker >>>>> mu(Threads_lock); return _live_threads_count->get_value() - >>>>> _exiting_threads_count; } >>>>> ?? static jlong get_daemon_thread_count()????? { MutexLocker >>>>> mu(Threads_lock); return _daemon_threads_count->get_value() - >>>>> _exiting_daemon_threads_count; } >>>>> >>>>> along with the other cleanups around is_daemon and is_hidden_thread? >>>>> >>>> >>>> Some micro-benchmarks like SecureRandomBench show a regression with >>>> webrev.7.? I could go back to webrev.5 and then we shouldn't need >>>> any locking in the get_*() functions. >>> >>> I don't see version 5 discussed but I took a look and it seems okay. >> >> Mandy had questions about the asserts in .5, and it seemed like we >> could just set the perf counters to the same value as the atomic >> counters, which resulted in .6.? I think the only problem with .6 is >> that I set the perf counters in decrement_thread_counts without the >> lock.? If I "sync" the perf counters to the atomic counters only in >> add_thread and remove_thread, with the lock, then it's about the same >> as .5, but without the asserts and parallel inc/dec.? If anyone likes >> the sound of that, I can send out a new webrev.? Or we can go with >> webrev.5. > > I'm not sure what the concern was with the asserts - if they mis-fire > we'll know soon enough. So I'm okay with .5 > >>> My only query with that version is that it appears the actual >>> perfCounters no longer get read by anything - is that the case? >>> >> >> There does seem to be code that references them, through their name >> string.? That's a difference interface that I'm not familiar with, so >> I didn't want to break it. > > Right - they can be accessed directly through other means. I was > concerned that the perfCounter was out of sync with > get_live_thread-count() but that's already the case so not an issue. > > If all tests and benchmarks are happy I say go with version .5 > I have no objection to version .5 if most people prefer that.? My comment was that I don't think the asserts are necessary. Mandy From thomas.stuefe at gmail.com Wed Oct 24 06:19:22 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 24 Oct 2018 08:19:22 +0200 Subject: RFR(xxs): 8212896: AIX build breaks after 8212611 Message-ID: Hi all, may I please have a review (trivial rule) for this tiny build fix: https://bugs.openjdk.java.net/browse/JDK-8212896 http://cr.openjdk.java.net/~stuefe/webrevs/8212896-aix-build-broken-after-8212611/webrev.00/webrev/ Thank you, Thomas From david.holmes at oracle.com Wed Oct 24 06:23:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Oct 2018 16:23:37 +1000 Subject: RFR(xxs): 8212896: AIX build breaks after 8212611 In-Reply-To: References: Message-ID: <560b9877-e962-a472-31db-f23f4a463fbf@oracle.com> Looks fine - and consistent with other files affected by 8212611. Thanks, David On 24/10/2018 4:19 PM, Thomas St?fe wrote: > Hi all, > > may I please have a review (trivial rule) for this tiny build fix: > > https://bugs.openjdk.java.net/browse/JDK-8212896 > http://cr.openjdk.java.net/~stuefe/webrevs/8212896-aix-build-broken-after-8212611/webrev.00/webrev/ > > Thank you, > > Thomas > From volker.simonis at gmail.com Wed Oct 24 06:51:23 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 24 Oct 2018 08:51:23 +0200 Subject: RFR(xxs): 8212896: AIX build breaks after 8212611 In-Reply-To: References: Message-ID: Looks good! However I don't understand why this bug does not also break the non-PCH build on Linux for example (I've just tried and it still works)? Do you understand? Regards, Volker On Wed, Oct 24, 2018 at 8:19 AM Thomas St?fe wrote: > > Hi all, > > may I please have a review (trivial rule) for this tiny build fix: > > https://bugs.openjdk.java.net/browse/JDK-8212896 > http://cr.openjdk.java.net/~stuefe/webrevs/8212896-aix-build-broken-after-8212611/webrev.00/webrev/ > > Thank you, > > Thomas From thomas.stuefe at gmail.com Wed Oct 24 07:49:00 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 24 Oct 2018 09:49:00 +0200 Subject: RFR(xxs): 8212896: AIX build breaks after 8212611 In-Reply-To: References: Message-ID: Yes I wondered about that too. Turns out this header sneaks in on other platforms via JFR: 471 ........ /shared/projects/openjdk/jdk-jdk/output-fastdebug-nopch/hotspot/variant-server/gensrc/jfrfiles/jfrEventClasses.hpp 472 ......... /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/jfr/recorder/service/jfrEvent.hpp 477 .......... /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/jfr/writers/jfrNativeEventWriter.hpp 482 ........... /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/jfr/writers/jfrEventWriterHost.inline.hpp 483 ............ /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/jfr/writers/jfrEventWriterHost.hpp 484 ............. /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/jfr/writers/jfrWriterHost.inline.hpp 505 .............. /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/oops/typeArrayOop.inline.hpp 506 ............... /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/oops/access.inline.hpp 507 ................ /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/shared/barrierSetConfig.inline.hpp 508 ................. /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/shared/modRefBarrierSet.inline.hpp 509 .................. /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/shared/barrierSet.hpp Since on AIX INCLUDE_JFR is not set, we miss out on those includes. ...Thomas On Wed, Oct 24, 2018 at 8:51 AM Volker Simonis wrote: > > Looks good! > > However I don't understand why this bug does not also break the > non-PCH build on Linux for example (I've just tried and it still > works)? Do you understand? > > Regards, > Volker > On Wed, Oct 24, 2018 at 8:19 AM Thomas St?fe wrote: > > > > Hi all, > > > > may I please have a review (trivial rule) for this tiny build fix: > > > > https://bugs.openjdk.java.net/browse/JDK-8212896 > > http://cr.openjdk.java.net/~stuefe/webrevs/8212896-aix-build-broken-after-8212611/webrev.00/webrev/ > > > > Thank you, > > > > Thomas From erik.osterlund at oracle.com Wed Oct 24 08:18:00 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 24 Oct 2018 10:18:00 +0200 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <05dd085d-08a0-ca41-202c-12a7452985b7@oracle.com> References: <5BCF3766.3050504@oracle.com> <05dd085d-08a0-ca41-202c-12a7452985b7@oracle.com> Message-ID: <5BD02AB8.5000305@oracle.com> Hi Coleen, On 2018-10-23 20:51, coleen.phillimore at oracle.com wrote: > > I wonder if the Compile_lock is added for this in ci/ciEnv.cpp (and > it's copy in jvmtiEnv.cpp): > > ciKlass* ciEnv::get_klass_by_name_impl(ciKlass* accessing_klass, > ... > MutexLocker ml(Compile_lock); > Klass* kls; > if (!require_local) { > kls = > SystemDictionary::find_constrained_instance_or_array_klass(sym, loader, > KILL_COMPILE_ON_FATAL_(fail_type)); > } else { > kls = SystemDictionary::find_instance_or_array_klass(sym, > loader, domain, > KILL_COMPILE_ON_FATAL_(fail_type)); > } > found_klass = kls; > } > > which eventually calls in SystemDictionary:: > > if (k != NULL) { > k = k->array_klass_or_null(fd.dimension()); > } > If that path was ever taken before, it seems to me that would result in a deadlock with the existing code, if allocating the array klass is necessary, as the Compile_lock is not conditionally taken in array_klass_impl. So if the callers you point out already have the Compile_lock, it would just hang, and in debug mode, it would assert that the lock is already held. So it seems like already today, we rely on that path never being taken for correctness. > > And maybe the Compile_lock keeps the Klass::_higher_dimension have a > consistent value. Maybe the code in ciEnv.cpp and jvmciEnv.cpp should > take MultiArray_lock instead. The problem is that Compile_lock > protects too many things. I have gone through all callers of higher_dimension() using rtags, and did not find anything suggesting the Compile_lock keeps it consistent. The callers are either safepoint operations, *::array_klass_impl itself, or "protected" by another lock (VtableStats::compute is protected by the CLDG_lock only, which is not being used to protect the higher or lower dimensions, so this code is possibly wrong, but only called when using -XX:+PrintVtableStats). > I don't know if this is safe to remove unfortunately. Hope it feels safer now. Thanks, /Erik > Coleen > > > On 10/23/18 10:59 AM, Erik ?sterlund wrote: >> Hi, >> >> We occasionally blockingly wait for GC in allocations in >> ObjArrayKlass::allocate_objArray_klass(), while holding the >> Compile_lock. >> This is problematic for concurrent class unloading that needs to hold >> that lock in the unloading path. This introduces a potential deadlock >> situation. >> >> After staring enough at this code together with help from Coleen, I >> have convinced myself that the Compile_lock is in fact not needed in >> this path, and there is nothing to my knowledge it actually protects >> here. The vague comment next to the lock suggests its purpose is to >> protect vtable stubs. But it doesn't. There is a VtableStubs_lock for >> that, and I find no traces if the Compile_lock having anything to do >> with that. >> >> Coleen helped me trying to trace down where this locking code came >> from. It takes us back to before anyone was around, and does not seem >> to have changed in the past 2 decades, and its origins are a bit >> unclear. The theory is that perhaps vtable stubs used to be protected >> by the Compile_lock in ancient times, and the locking code is still >> around because nobody dared to touch it. >> >> Since both code analysis and mach5 suggest there is no reason for >> this lock to be held here, I propose to remove it with this patch. >> >> If anyone that has been around for more than 2 decades in HotSpot >> happens to know the reason why this locking was introduced, any >> commentary around that would be very appreciated. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212682 >> >> Thanks, >> /Erik > From martin.doerr at sap.com Wed Oct 24 08:20:34 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 24 Oct 2018 08:20:34 +0000 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: <271b6743-1611-4201-cdf6-9fdc5b87086e@linux.vnet.ibm.com> References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> <271b6743-1611-4201-cdf6-9fdc5b87086e@linux.vnet.ibm.com> Message-ID: <0c176a41bd274bf795382f26e476b59a@sap.com> Hi Gustavo, looks good. Thank you for cleaning things up and also for sharing the story about mftgpr. Interesting. Reviewed. Best regards, Martin -----Original Message----- From: Gustavo Romero Sent: Dienstag, 23. Oktober 2018 23:46 To: Doerr, Martin ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection Hi Martin, Thanks for your comments on fsqrt{,s} and mftgpr opcodes. On 10/23/2018 01:51 PM, Doerr, Martin wrote: > Hi Gustavo, > >> I agree. So, if you don't mind, to reduce any future rework on that >> I removed the hardcoded L=1 and now it's just a default. I introduced >> l14() function that matches two bits in the proper field as I don't >> see any collision with any other one bit field at the same position >> (bit 14). > I like it, but the default value needs to get specified in the .hpp file. Fixed. >> I just have one question regarding the feature-string. I see >> instrumentation for "fsqrts" feature but it's not currently >> advertised by the feature-string. On the other hand, ISA info >> looks to indicate that fsqrt and fsqrts are not aliases, because: >> >> fsqrt P2 -> Instruction introduced in the POWER2 Architecture. >> fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 >> >> so I'm wondering if just for completeness the "fsqrts" feature >> should be included in the feature-string. Besides that I don't >> see instrumentation to support has_mftgpr(), which is commented >> out, thus should we remove it? Like the following: > > This code is very old. If we are sure nobody can run the VM on a machine without these instructions they can get removed from the feature checking. I'm not sure about it. Maybe somebody is using out there, in the community for example, so kept it. > The mftgpr was only designed for an extension of Power6 and the opcode was reused for something else later on if I remember correctly. I think you can remove the remaining comment. I was not aware of that, thanks for the info. Just out of curiosity I asked the toolchain folks and they said that mftgpr is present in some POWER6 hardware but on the so-called raw mode (ie power6x, which I think it's the extension you mentioned), but not in the architected / normal mode (power6). So even if it's present in the hardware it's disabled unless you boot the system up in raw mode. On that raw mode (power6x) one would see the following in AUXV, for instance: $ LD_SHOW_AUXV=1 /bin/true | grep PLATFORM AT_PLATFORM: power6x AT_BASE_PLATFORM:power6x But probably theses machines w/ power6x never got out of IBM indeed. Anyway, they said that power6x is unsupported. So, as you said, I removed the comment about mftgpr. v3 webrev: http://cr.openjdk.java.net/~gromero/8212481/v3/ Thanks! Best regards, Gustavo From thomas.stuefe at gmail.com Wed Oct 24 08:36:35 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 24 Oct 2018 10:36:35 +0200 Subject: RFR(xxs): 8212896: AIX build breaks after 8212611 In-Reply-To: <560b9877-e962-a472-31db-f23f4a463fbf@oracle.com> References: <560b9877-e962-a472-31db-f23f4a463fbf@oracle.com> Message-ID: Thanks David! Submit ran thru, will push now. ..Thomas On Wed, Oct 24, 2018 at 10:35 AM David Holmes wrote: > > Looks fine - and consistent with other files affected by 8212611. > > Thanks, > David > > On 24/10/2018 4:19 PM, Thomas St?fe wrote: > > Hi all, > > > > may I please have a review (trivial rule) for this tiny build fix: > > > > https://bugs.openjdk.java.net/browse/JDK-8212896 > > http://cr.openjdk.java.net/~stuefe/webrevs/8212896-aix-build-broken-after-8212611/webrev.00/webrev/ > > > > Thank you, > > > > Thomas > > From aph at redhat.com Wed Oct 24 08:49:33 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 24 Oct 2018 09:49:33 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <53f86f99-21d1-16c0-cd9f-181ac69a7faf@oracle.com> References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> <5BCF204F.6030705@oracle.com> <5BCF3111.6020105@oracle.com> <2e538f61-0c4d-3e29-2829-357bc3b60baf@redhat.com> <53f86f99-21d1-16c0-cd9f-181ac69a7faf@oracle.com> Message-ID: On 10/23/2018 11:11 PM, David Holmes wrote: > The patch is on cr.openjdk. > > The only "issue" is that the webrev contains a "link" to the original > repository that is completely bogus because the repository (naturally) > is not on cr.openjdk. The version of webrev I use doesn't do that it > just has this non-link entry: > > Workspace: /export/users/dh198349/jdk-dev/open OK, so it's a matter of everyone updating to the latest webrev. Thanks. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Oct 24 08:50:10 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 24 Oct 2018 09:50:10 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <2678A53A-31C0-4F00-9CE7-8371C5CF0C08@oracle.com> References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> <5BCF204F.6030705@oracle.com> <5BCF3111.6020105@oracle.com> <2e538f61-0c4d-3e29-2829-357bc3b60baf@redhat.com> <2678A53A-31C0-4F00-9CE7-8371C5CF0C08@oracle.com> Message-ID: Hi, On 10/23/2018 05:25 PM, Erik Osterlund wrote: > It is already there though: > http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/open.patch > > Does this link not work for you? Got it now, thanks. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Wed Oct 24 08:52:23 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 24 Oct 2018 10:52:23 +0200 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: References: <5BCF3766.3050504@oracle.com> Message-ID: <5BD032C7.1000807@oracle.com> Hi Dean, On 2018-10-23 22:27, dean.long at oracle.com wrote: > Can allocate_objArray_klass() end up calling > SystemDictionary::add_to_hierarchy()? If so, then you'll end up > locking Compile_lock anway, but in the wrong order with respect to > MultiArray_lock. No, I can't see that allocate_objArray_klass() ever calls SystemDictionary::add_to_hierarchy(). If it did, we would assert that the Compile_lock is held; it never re-acquires the lock. Thanks, /Erik > dl > > On 10/23/18 7:59 AM, Erik ?sterlund wrote: >> Hi, >> >> We occasionally blockingly wait for GC in allocations in >> ObjArrayKlass::allocate_objArray_klass(), while holding the >> Compile_lock. >> This is problematic for concurrent class unloading that needs to hold >> that lock in the unloading path. This introduces a potential deadlock >> situation. >> >> After staring enough at this code together with help from Coleen, I >> have convinced myself that the Compile_lock is in fact not needed in >> this path, and there is nothing to my knowledge it actually protects >> here. The vague comment next to the lock suggests its purpose is to >> protect vtable stubs. But it doesn't. There is a VtableStubs_lock for >> that, and I find no traces if the Compile_lock having anything to do >> with that. >> >> Coleen helped me trying to trace down where this locking code came >> from. It takes us back to before anyone was around, and does not seem >> to have changed in the past 2 decades, and its origins are a bit >> unclear. The theory is that perhaps vtable stubs used to be protected >> by the Compile_lock in ancient times, and the locking code is still >> around because nobody dared to touch it. >> >> Since both code analysis and mach5 suggest there is no reason for >> this lock to be held here, I propose to remove it with this patch. >> >> If anyone that has been around for more than 2 decades in HotSpot >> happens to know the reason why this locking was introduced, any >> commentary around that would be very appreciated. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212682 >> >> Thanks, >> /Erik > From david.holmes at oracle.com Wed Oct 24 09:15:44 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Oct 2018 19:15:44 +1000 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: References: <5BC9E8BA.50407@oracle.com> <57b663f3-44e4-b0bb-86bf-3cf02d90de84@redhat.com> <5BCF204F.6030705@oracle.com> <5BCF3111.6020105@oracle.com> <2e538f61-0c4d-3e29-2829-357bc3b60baf@redhat.com> <53f86f99-21d1-16c0-cd9f-181ac69a7faf@oracle.com> Message-ID: On 24/10/2018 6:49 PM, Andrew Haley wrote: > On 10/23/2018 11:11 PM, David Holmes wrote: >> The patch is on cr.openjdk. >> >> The only "issue" is that the webrev contains a "link" to the original >> repository that is completely bogus because the repository (naturally) >> is not on cr.openjdk. The version of webrev I use doesn't do that it >> just has this non-link entry: >> >> Workspace: /export/users/dh198349/jdk-dev/open > > OK, so it's a matter of everyone updating to the latest webrev. Thanks. Not really. My point was that entry (whether a link or not) is not really relevant to the patch/webrev. I'm probably using an older version of webrev. David From gromero at linux.vnet.ibm.com Wed Oct 24 13:35:03 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Wed, 24 Oct 2018 10:35:03 -0300 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: <0c176a41bd274bf795382f26e476b59a@sap.com> References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> <271b6743-1611-4201-cdf6-9fdc5b87086e@linux.vnet.ibm.com> <0c176a41bd274bf795382f26e476b59a@sap.com> Message-ID: Hi Martin, On 10/24/2018 05:20 AM, Doerr, Martin wrote: > looks good. Thank you for cleaning things up and also for sharing the story about mftgpr. Interesting. > Reviewed. Thanks for reviewing! Best regards, Gustavo > Best regards, > Martin > > > -----Original Message----- > From: Gustavo Romero > Sent: Dienstag, 23. Oktober 2018 23:46 > To: Doerr, Martin ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: Re: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection > > Hi Martin, > > Thanks for your comments on fsqrt{,s} and mftgpr opcodes. > > On 10/23/2018 01:51 PM, Doerr, Martin wrote: >> Hi Gustavo, >> >>> I agree. So, if you don't mind, to reduce any future rework on that >>> I removed the hardcoded L=1 and now it's just a default. I introduced >>> l14() function that matches two bits in the proper field as I don't >>> see any collision with any other one bit field at the same position >>> (bit 14). >> I like it, but the default value needs to get specified in the .hpp file. > > Fixed. > > >>> I just have one question regarding the feature-string. I see >>> instrumentation for "fsqrts" feature but it's not currently >>> advertised by the feature-string. On the other hand, ISA info >>> looks to indicate that fsqrt and fsqrts are not aliases, because: >>> >>> fsqrt P2 -> Instruction introduced in the POWER2 Architecture. >>> fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 >>> >>> so I'm wondering if just for completeness the "fsqrts" feature >>> should be included in the feature-string. Besides that I don't >>> see instrumentation to support has_mftgpr(), which is commented >>> out, thus should we remove it? Like the following: >> >> This code is very old. If we are sure nobody can run the VM on a machine without these instructions they can get removed from the feature checking. > > I'm not sure about it. Maybe somebody is using out there, in the community > for example, so kept it. > > >> The mftgpr was only designed for an extension of Power6 and the opcode was reused for something else later on if I remember correctly. I think you can remove the remaining comment. > > I was not aware of that, thanks for the info. > Just out of curiosity I asked the toolchain folks and they said that mftgpr > is present in some POWER6 hardware but on the so-called raw mode (ie power6x, > which I think it's the extension you mentioned), but not in the architected / > normal mode (power6). So even if it's present in the hardware it's disabled > unless you boot the system up in raw mode. > > On that raw mode (power6x) one would see the following in AUXV, for instance: > > $ LD_SHOW_AUXV=1 /bin/true | grep PLATFORM > AT_PLATFORM: power6x > AT_BASE_PLATFORM:power6x > > But probably theses machines w/ power6x never got out of IBM indeed. > > Anyway, they said that power6x is unsupported. So, as you said, I removed the > comment about mftgpr. > > v3 webrev: > > http://cr.openjdk.java.net/~gromero/8212481/v3/ > > > Thanks! > > Best regards, > Gustavo > From karen.kinnear at oracle.com Wed Oct 24 16:36:32 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 24 Oct 2018 12:36:32 -0400 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: Message-ID: <26730C6C-1528-4520-91D1-4AF7FD040D81@oracle.com> Serguei, Fix looks good. Officially reviewed. Many thanks for the tests and the -Xlog output. Many thanks for fixing this. thanks, Karen > On Oct 16, 2018, at 1:03 AM, serguei.spitsyn at oracle.com wrote: > > Please, review a fix for: > https://bugs.openjdk.java.net/browse/JDK-8024368 > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ > > > Summary: > > Private method should not allocate vtable indeces. > The fix tweaks several conditions in the functions: > klassVtable::update_inherited_vtable() and klassVtable::needs_new_vtable_entry(), > adds a couple of asserts and makes several comments up-to-date. > > The fix was preliminary reviewed by Karen, David H. and Lois. > > Testing: > > The fix was verified with runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} > and vmTestbase/vm/runtime/defmeth tests. > Also, I ran a mach5 job with the flags: tier1,tier2,hs-tier3,hs-tier4,hs-tier5. > > By suggestion from David, it was also tested with the -Xlog output for vtables before and after the fix. > I've done this comparison (awk was used to get rid of the first column with the time stamps). > It shows that private methods were included into vtable before the fix but not included after. > Please, see one fragment as an example: > > 166c166 > < copy vtable from java.lang.Object to java.lang.StackFrameInfo size 17 > --- > > copy vtable from java.lang.Object to java.lang.StackFrameInfo size 16 > 180d179 > < adding java.lang.StackFrameInfo.ensureRetainClassRefEnabled()V at index 16, flags: private > 182c181 > < copy vtable from java.lang.StackFrameInfo to java.lang.LiveStackFrameInfo size 17 > --- > > copy vtable from java.lang.StackFrameInfo to java.lang.LiveStackFrameInfo size 16 > 188,201c187,191 > < copy vtable from java.lang.Object to java.lang.StackStreamFactory$AbstractStackWalker size 18 > < adding java.lang.StackStreamFactory$AbstractStackWalker.close()V at index 5, flags: private > < adding java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I at index 6, flags: private > < adding java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; at index 7, flags: protected abstract > < adding java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V at index 8, flags: protected abstract > < adding java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at index 9, flags: protected abstract > < adding java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I at index 10, flags: protected > < adding java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z at index 11, flags: private > < adding java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; at index 12, flags: private > < adding java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at index 13, flags: private > < adding java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; at index 14, flags: private > < adding java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I at index 15, flags: private > < adding java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I at index 16, flags: private native > < adding java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; at index 17, flags: private native > --- > > copy vtable from java.lang.Object to java.lang.StackStreamFactory$AbstractStackWalker size 9 > > adding java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; at index 5, flags: protected abstract > > adding java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V at index 6, flags: protected abstract > > adding java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at index 7, flags: protected abstract > > adding java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I at index 8, flags: protected > > > Thanks, > Serguei > From lois.foltan at oracle.com Wed Oct 24 17:44:47 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 24 Oct 2018 13:44:47 -0400 Subject: RFR(S): 8024368: private methods are allocated vtable slots In-Reply-To: References: <26730C6C-1528-4520-91D1-4AF7FD040D81@oracle.com> Message-ID: On 10/24/2018 1:33 PM, serguei.spitsyn at oracle.com wrote: > Hi Karen, > > It is great you had a chance to review it officially - thanks a lot! > Probably, I have to add Lois to the list of reviewers before pushing it. Please do!? I did review and looks good. Lois > > Thanks, > Serguei > > > On 10/24/18 09:36, Karen Kinnear wrote: >> Serguei, >> >> Fix looks good. Officially reviewed. >> >> Many thanks for the tests and the -Xlog output. Many thanks for >> fixing this. >> >> >> thanks, >> Karen >> >>> On Oct 16, 2018, at 1:03 AM, serguei.spitsyn at oracle.com >>> wrote: >>> >>> Please, review a fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8024368 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8024368-priv-vtable.2/ >>> >>> >>> Summary: >>> >>> ? Private method should not allocate vtable indeces. >>> ? The fix tweaks several conditions in the functions: >>> ? klassVtable::update_inherited_vtable() and >>> klassVtable::needs_new_vtable_entry(), >>> ? adds a couple of asserts and makes several comments up-to-date. >>> >>> ? The fix was preliminary reviewed by Karen, David H. and Lois. >>> >>> Testing: >>> >>> ? The fix was verified with >>> runtime/{appcds+Nestmates+RedefineTests+SelectionResolution} >>> ? and vmTestbase/vm/runtime/defmeth tests. >>> ? Also, I ran a mach5 job with the flags: >>> tier1,tier2,hs-tier3,hs-tier4,hs-tier5. >>> >>> ? By suggestion from David, it was also tested with the -Xlog output >>> for vtables before and after the fix. >>> ? I've done this comparison (awk was used to get rid of the first >>> column with the time stamps). >>> ? It shows that private methods were included into vtable before the >>> fix but not included after. >>> ? Please, see one fragment as an example: >>> >>> ?? 166c166 >>> ?? >> size 17 >>> ?? --- >>> ?? >? copy vtable from java.lang.Object to java.lang.StackFrameInfo >>> size 16 >>> ?? 180d179 >>> ?? >> at index 16, flags: private >>> ?? 182c181 >>> ?? >> java.lang.LiveStackFrameInfo size 17 >>> ?? --- >>> ?? >? copy vtable from java.lang.StackFrameInfo to >>> java.lang.LiveStackFrameInfo size 16 >>> ?? 188,201c187,191 >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker size 18 >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.close()V at index >>> 5, flags: private >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.toStackWalkMode(Ljava/lang/StackWalker;I)I >>> at index 6, flags: private >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >>> at index 7, flags: protected abstract >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >>> at index 8, flags: protected abstract >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >>> index 9, flags: protected abstract >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >>> at index 10, flags: protected >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.skipReflectionFrames()Z >>> at index 11, flags: private >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object; >>> at index 12, flags: private >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I at >>> index 13, flags: private >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object; >>> at index 14, flags: private >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I >>> at index 15, flags: private >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I >>> at index 16, flags: private native >>> ?? >> java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object; >>> at index 17, flags: private native >>> ?? --- >>> ?? >? copy vtable from java.lang.Object to >>> java.lang.StackStreamFactory$AbstractStackWalker size 9 >>> ?? >? adding >>> java.lang.StackStreamFactory$AbstractStackWalker.consumeFrames()Ljava/lang/Object; >>> at index 5, flags: protected abstract >>> ?? >? adding >>> java.lang.StackStreamFactory$AbstractStackWalker.initFrameBuffer()V >>> at index 6, flags: protected abstract >>> ?? >? adding >>> java.lang.StackStreamFactory$AbstractStackWalker.batchSize(I)I at >>> index 7, flags: protected abstract >>> ?? >? adding >>> java.lang.StackStreamFactory$AbstractStackWalker.getNextBatchSize()I >>> at index 8, flags: protected >>> >>> >>> Thanks, >>> Serguei >>> >> > From mbalao at redhat.com Wed Oct 24 19:58:44 2018 From: mbalao at redhat.com (Martin Balao) Date: Wed, 24 Oct 2018 16:58:44 -0300 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader Message-ID: Hi, Can I have a review for JDK-8212937 [1]? Webrev.00: * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00.zip By obtaining the ClassLoaderData instance through a call to SystemDictionary::register_loader, we make sure that either an existing instance is obtained or a new one created. This same idiom has already been used in other places where there are no guarantees that a previous ClassLoaderData instance exists for a given class loader. See futher information in JDK-8212937 ticket [1]. Thanks, Martin.- -- [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 From dean.long at oracle.com Wed Oct 24 20:20:10 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 24 Oct 2018 13:20:10 -0700 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <5BD032C7.1000807@oracle.com> References: <5BCF3766.3050504@oracle.com> <5BD032C7.1000807@oracle.com> Message-ID: <55e11251-f8da-aeed-8372-6a26efbb7822@oracle.com> On 10/24/18 1:52 AM, Erik ?sterlund wrote: > Hi Dean, > > On 2018-10-23 22:27, dean.long at oracle.com wrote: >> Can allocate_objArray_klass() end up calling >> SystemDictionary::add_to_hierarchy()?? If so, then you'll end up >> locking Compile_lock anway, but in the wrong order with respect to >> MultiArray_lock. > > No, I can't see that allocate_objArray_klass() ever calls > SystemDictionary::add_to_hierarchy(). If it did, we would assert that > the Compile_lock is held; it never re-acquires the lock. > OK, I forgot that mutex's can't be locked recursively. I found a clue to the reason for original 1998 change.? This was also added at the same time, to klassVtable::initialize_vtable(): ? // We need the compile lock in order to force the super vtables and ? // methodOop code pointers to stay constant. ? // (Race scenario: compiler thread updates a vtable entry while we're copying ? // down entries from the superclass vtable.) ? assert_lock(Compile_lock); By 1999, the vtable code changed a lot and the assert above went away, so it was probably just an oversight that the extra uses of Compile_lock weren't removed at the same time. dl > Thanks, > /Erik > >> dl >> >> On 10/23/18 7:59 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> We occasionally blockingly wait for GC in allocations in >>> ObjArrayKlass::allocate_objArray_klass(), while holding the >>> Compile_lock. >>> This is problematic for concurrent class unloading that needs to >>> hold that lock in the unloading path. This introduces a potential >>> deadlock situation. >>> >>> After staring enough at this code together with help from Coleen, I >>> have convinced myself that the Compile_lock is in fact not needed in >>> this path, and there is nothing to my knowledge it actually protects >>> here. The vague comment next to the lock suggests its purpose is to >>> protect vtable stubs. But it doesn't. There is a VtableStubs_lock >>> for that, and I find no traces if the Compile_lock having anything >>> to do with that. >>> >>> Coleen helped me trying to trace down where this locking code came >>> from. It takes us back to before anyone was around, and does not >>> seem to have changed in the past 2 decades, and its origins are a >>> bit unclear. The theory is that perhaps vtable stubs used to be >>> protected by the Compile_lock in ancient times, and the locking code >>> is still around because nobody dared to touch it. >>> >>> Since both code analysis and mach5 suggest there is no reason for >>> this lock to be held here, I propose to remove it with this patch. >>> >>> If anyone that has been around for more than 2 decades in HotSpot >>> happens to know the reason why this locking was introduced, any >>> commentary around that would be very appreciated. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212682 >>> >>> Thanks, >>> /Erik >> > From lois.foltan at oracle.com Wed Oct 24 20:52:45 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 24 Oct 2018 16:52:45 -0400 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: On 10/24/2018 3:58 PM, Martin Balao wrote: > Hi, > > Can I have a review for JDK-8212937 [1]? > > Webrev.00: > > * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ > * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00.zip > > By obtaining the ClassLoaderData instance through a call to > SystemDictionary::register_loader, we make sure that either an existing > instance is obtained or a new one created. This same idiom has already been > used in other places where there are no guarantees that a previous > ClassLoaderData instance exists for a given class loader. See futher > information in JDK-8212937 ticket [1]. > > Thanks, > Martin.- > > -- > [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 Hi Martin, I agree that the same idiom is used in other places, but there are also a couple of occurrences of java_lang_ClassLoader::parent in classfile/classLoaderStats.cpp and classfile/classLoaderHierarchyDCmd.cpp that do not.? In order to better understand, do you have a test case that demonstrates this issue? Thanks, Lois From erik.osterlund at oracle.com Wed Oct 24 20:59:30 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 24 Oct 2018 22:59:30 +0200 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <55e11251-f8da-aeed-8372-6a26efbb7822@oracle.com> References: <5BCF3766.3050504@oracle.com> <5BD032C7.1000807@oracle.com> <55e11251-f8da-aeed-8372-6a26efbb7822@oracle.com> Message-ID: Hi Dean, On 2018-10-24 22:20, dean.long at oracle.com wrote: > On 10/24/18 1:52 AM, Erik ?sterlund wrote: >> Hi Dean, >> >> On 2018-10-23 22:27, dean.long at oracle.com wrote: >>> Can allocate_objArray_klass() end up calling >>> SystemDictionary::add_to_hierarchy()?? If so, then you'll end up >>> locking Compile_lock anway, but in the wrong order with respect to >>> MultiArray_lock. >> >> No, I can't see that allocate_objArray_klass() ever calls >> SystemDictionary::add_to_hierarchy(). If it did, we would assert that >> the Compile_lock is held; it never re-acquires the lock. >> > > OK, I forgot that mutex's can't be locked recursively. > > I found a clue to the reason for original 1998 change.? This was also > added at the same time, to klassVtable::initialize_vtable(): > > ? // We need the compile lock in order to force the super vtables and > ? // methodOop code pointers to stay constant. > ? // (Race scenario: compiler thread updates a vtable entry while we're > copying > ? // down entries from the superclass vtable.) > ? assert_lock(Compile_lock); > > By 1999, the vtable code changed a lot and the assert above went away, > so it was probably just an oversight that the extra uses of Compile_lock > weren't removed at the same time. Thank you for the code archaeology. That really helps a lot. So it still looks like this lock doesn't really seem to do anything for us today. And perhaps that has been the case for nearly two decades. Thanks, /Erik > dl > >> Thanks, >> /Erik >> >>> dl >>> >>> On 10/23/18 7:59 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> We occasionally blockingly wait for GC in allocations in >>>> ObjArrayKlass::allocate_objArray_klass(), while holding the >>>> Compile_lock. >>>> This is problematic for concurrent class unloading that needs to >>>> hold that lock in the unloading path. This introduces a potential >>>> deadlock situation. >>>> >>>> After staring enough at this code together with help from Coleen, I >>>> have convinced myself that the Compile_lock is in fact not needed in >>>> this path, and there is nothing to my knowledge it actually protects >>>> here. The vague comment next to the lock suggests its purpose is to >>>> protect vtable stubs. But it doesn't. There is a VtableStubs_lock >>>> for that, and I find no traces if the Compile_lock having anything >>>> to do with that. >>>> >>>> Coleen helped me trying to trace down where this locking code came >>>> from. It takes us back to before anyone was around, and does not >>>> seem to have changed in the past 2 decades, and its origins are a >>>> bit unclear. The theory is that perhaps vtable stubs used to be >>>> protected by the Compile_lock in ancient times, and the locking code >>>> is still around because nobody dared to touch it. >>>> >>>> Since both code analysis and mach5 suggest there is no reason for >>>> this lock to be held here, I propose to remove it with this patch. >>>> >>>> If anyone that has been around for more than 2 decades in HotSpot >>>> happens to know the reason why this locking was introduced, any >>>> commentary around that would be very appreciated. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8212682 >>>> >>>> Thanks, >>>> /Erik >>> >> > From dean.long at oracle.com Wed Oct 24 21:18:00 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 24 Oct 2018 14:18:00 -0700 Subject: RFR(S) 8021335: Missing synchronization when reading counters for live threads and peak thread count In-Reply-To: <4febfc04-21c3-fd91-db83-54777ebd7ca2@oracle.com> References: <3345d539-54e0-c048-ddf7-d2919edc585c@oracle.com> <90728cf7-3711-e91c-cf9a-534c8d5fca15@oracle.com> <78dedbbf-cfc0-6910-dc28-ee7eec0f5373@oracle.com> <88d9ae30-d08c-8bab-35a5-b4601f452883@oracle.com> <5b6265b6-36b3-4317-1e6a-8ba51c8a48b3@oracle.com> <0560343b-94b4-2261-0f1d-a08d7067e97b@oracle.com> <43be9a19-d0c5-298a-9bcb-e98223f4566a@oracle.com> <98af2e32-b9e7-e06a-91fc-447baa445bed@oracle.com> <4febfc04-21c3-fd91-db83-54777ebd7ca2@oracle.com> Message-ID: <57b7cbe0-78ed-4a27-b1b0-3f733321ecee@oracle.com> On 10/23/18 11:04 PM, Mandy Chung wrote: > > > On 10/23/18 10:34 PM, David Holmes wrote: >> On 24/10/2018 8:30 AM, dean.long at oracle.com wrote: >>> On 10/23/18 2:51 PM, David Holmes wrote: >>>> Hi Dean, >>>> >>>> On 24/10/2018 4:05 AM, dean.long at oracle.com wrote: >>>>> On 10/23/18 9:46 AM, dean.long at oracle.com wrote: >>>>>> On 10/22/18 3:31 PM, David Holmes wrote: >>>>>>> Sorry Dean I'm concerned about a thread termination bottleneck >>>>>>> with this. A simple microbenchmark that creates 500,000 threads >>>>>>> that have to run and terminate, shows a 15+% slowdown on my >>>>>>> Linux box. I tried to find some kind of real benchmarks that >>>>>>> covers thread termination but couldn't see anything specific. >>>>>>> >>>>>>> Can you at least run this through our performance system to see >>>>>>> if any of the regular benchmarks are affected. >>>>>>> >>>>>> >>>>>> OK, but even if the regular benchmarks don't show a difference, >>>>>> I'd feel better if microbenchmarks were not affected.? What if I >>>>>> go back to the original approach and add locking: >>>>>> >>>>>> ?? static jlong get_live_thread_count()??????? { MutexLocker >>>>>> mu(Threads_lock); return _live_threads_count->get_value() - >>>>>> _exiting_threads_count; } >>>>>> ?? static jlong get_daemon_thread_count()????? { MutexLocker >>>>>> mu(Threads_lock); return _daemon_threads_count->get_value() - >>>>>> _exiting_daemon_threads_count; } >>>>>> >>>>>> along with the other cleanups around is_daemon and is_hidden_thread? >>>>>> >>>>> >>>>> Some micro-benchmarks like SecureRandomBench show a regression >>>>> with webrev.7.? I could go back to webrev.5 and then we shouldn't >>>>> need any locking in the get_*() functions. >>>> >>>> I don't see version 5 discussed but I took a look and it seems okay. >>> >>> Mandy had questions about the asserts in .5, and it seemed like we >>> could just set the perf counters to the same value as the atomic >>> counters, which resulted in .6.? I think the only problem with .6 is >>> that I set the perf counters in decrement_thread_counts without the >>> lock.? If I "sync" the perf counters to the atomic counters only in >>> add_thread and remove_thread, with the lock, then it's about the >>> same as .5, but without the asserts and parallel inc/dec.? If anyone >>> likes the sound of that, I can send out a new webrev.? Or we can go >>> with webrev.5. >> >> I'm not sure what the concern was with the asserts - if they mis-fire >> we'll know soon enough. So I'm okay with .5 >> >>>> My only query with that version is that it appears the actual >>>> perfCounters no longer get read by anything - is that the case? >>>> >>> >>> There does seem to be code that references them, through their name >>> string.? That's a difference interface that I'm not familiar with, >>> so I didn't want to break it. >> >> Right - they can be accessed directly through other means. I was >> concerned that the perfCounter was out of sync with >> get_live_thread-count() but that's already the case so not an issue. >> >> If all tests and benchmarks are happy I say go with version .5 >> > > I have no objection to version .5 if most people prefer that.? My > comment was that I don't think the asserts are necessary. > OK, I'll rerun some performance benchmarks and push .5 if the results look OK.? David, can you send me your micro-benchmark? Thanks for the reviews! dl > Mandy From coleen.phillimore at oracle.com Thu Oct 25 12:30:21 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 25 Oct 2018 08:30:21 -0400 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <5BD02AB8.5000305@oracle.com> References: <5BCF3766.3050504@oracle.com> <05dd085d-08a0-ca41-202c-12a7452985b7@oracle.com> <5BD02AB8.5000305@oracle.com> Message-ID: On 10/24/18 4:18 AM, Erik ?sterlund wrote: > Hi Coleen, > > On 2018-10-23 20:51, coleen.phillimore at oracle.com wrote: >> >> I wonder if the Compile_lock is added for this in ci/ciEnv.cpp (and >> it's copy in jvmtiEnv.cpp): >> >> ciKlass* ciEnv::get_klass_by_name_impl(ciKlass* accessing_klass, >> ... >> ??? MutexLocker ml(Compile_lock); >> ??? Klass* kls; >> ??? if (!require_local) { >> ????? kls = >> SystemDictionary::find_constrained_instance_or_array_klass(sym, loader, >> KILL_COMPILE_ON_FATAL_(fail_type)); >> ??? } else { >> ????? kls = SystemDictionary::find_instance_or_array_klass(sym, >> loader, domain, >> KILL_COMPILE_ON_FATAL_(fail_type)); >> ??? } >> ??? found_klass = kls; >> ? } >> >> which eventually calls in SystemDictionary:: >> >> ??? if (k != NULL) { >> ????? k = k->array_klass_or_null(fd.dimension()); >> ??? } >> > > If that path was ever taken before, it seems to me that would result > in a deadlock with the existing code, if allocating the array klass is > necessary, as the Compile_lock is not conditionally taken in > array_klass_impl. So if the callers you point out already have the > Compile_lock, it would just hang, and in debug mode, it would assert > that the lock is already held. So it seems like already today, we rely > on that path never being taken for correctness. > >> >> And maybe the Compile_lock keeps the Klass::_higher_dimension have a >> consistent value.? Maybe the code in ciEnv.cpp and jvmciEnv.cpp >> should take MultiArray_lock instead.? The problem is that >> Compile_lock protects too many things. > > I have gone through all callers of higher_dimension() using rtags, and > did not find anything suggesting the Compile_lock keeps it consistent. > The callers are either safepoint operations, *::array_klass_impl > itself, or "protected" by another lock (VtableStats::compute is > protected by the CLDG_lock only, which is not being used to protect > the higher or lower dimensions, so this code is possibly wrong, but > only called when using -XX:+PrintVtableStats). > >> I don't know if this is safe to remove unfortunately. > > Hope it feels safer now. Yes, thank you for the explanation.? I trust rtags better than my eyeballs.? Thanks to Dean for the archeology! Looks good! Coleen > > Thanks, > /Erik > >> Coleen >> >> >> On 10/23/18 10:59 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> We occasionally blockingly wait for GC in allocations in >>> ObjArrayKlass::allocate_objArray_klass(), while holding the >>> Compile_lock. >>> This is problematic for concurrent class unloading that needs to >>> hold that lock in the unloading path. This introduces a potential >>> deadlock situation. >>> >>> After staring enough at this code together with help from Coleen, I >>> have convinced myself that the Compile_lock is in fact not needed in >>> this path, and there is nothing to my knowledge it actually protects >>> here. The vague comment next to the lock suggests its purpose is to >>> protect vtable stubs. But it doesn't. There is a VtableStubs_lock >>> for that, and I find no traces if the Compile_lock having anything >>> to do with that. >>> >>> Coleen helped me trying to trace down where this locking code came >>> from. It takes us back to before anyone was around, and does not >>> seem to have changed in the past 2 decades, and its origins are a >>> bit unclear. The theory is that perhaps vtable stubs used to be >>> protected by the Compile_lock in ancient times, and the locking code >>> is still around because nobody dared to touch it. >>> >>> Since both code analysis and mach5 suggest there is no reason for >>> this lock to be held here, I propose to remove it with this patch. >>> >>> If anyone that has been around for more than 2 decades in HotSpot >>> happens to know the reason why this locking was introduced, any >>> commentary around that would be very appreciated. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212682 >>> >>> Thanks, >>> /Erik >> > From mbalao at redhat.com Thu Oct 25 15:15:22 2018 From: mbalao at redhat.com (Martin Balao) Date: Thu, 25 Oct 2018 12:15:22 -0300 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: Hi Lois, Thanks for having a look at this. Yes, I have a test that triggers this execution path. Unfortunately, this is a security test that I cannot publicly share. What I can tell you, though, is that the parent class loader is a "delegating class loader" which never defined a class and that's the reason why it doesn't have a ClassLoaderData instance created. I'm not sure if this would be needed in other places where the idiom is not used. The reason is that there might be any other implicit assumptions that makes it impossible. That's why I limited the scope of this changeset to the only case that I'm sure it's possible to trigger. Kind regards, Martin.- On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan wrote: > On 10/24/2018 3:58 PM, Martin Balao wrote: > > Hi, >> >> Can I have a review for JDK-8212937 [1]? >> >> Webrev.00: >> >> * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ >> * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937. >> webrev.00.zip >> >> By obtaining the ClassLoaderData instance through a call to >> SystemDictionary::register_loader, we make sure that either an existing >> instance is obtained or a new one created. This same idiom has already >> been >> used in other places where there are no guarantees that a previous >> ClassLoaderData instance exists for a given class loader. See futher >> information in JDK-8212937 ticket [1]. >> >> Thanks, >> Martin.- >> >> -- >> [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 >> > Hi Martin, > > I agree that the same idiom is used in other places, but there are also a > couple of occurrences of java_lang_ClassLoader::parent in > classfile/classLoaderStats.cpp and classfile/classLoaderHierarchyDCmd.cpp > that do not. In order to better understand, do you have a test case that > demonstrates this issue? > > Thanks, > Lois > From vladimir.kozlov at oracle.com Thu Oct 25 17:34:23 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 25 Oct 2018 10:34:23 -0700 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BC9E8BA.50407@oracle.com> References: <5BC9E8BA.50407@oracle.com> Message-ID: Hi Erik, First, what testing you did? This is significant rewrite and requires a lot of testing. I don't see a link to test results in RFE. Any performance testing? It seems you removed the only case (in do_unloading()) where CodeCache::do_unloading_nmethod_caches() is called. I can't find other place where it is called after your change. Why keep it then? It seems CompiledMethod::is_unloading() is performance critical. Can we make IsUnloadingBehaviour::current()->is_unloading(this) as non virtual call? I see only ClosureIsUnloadingBehaviour subclass of IsUnloadingBehaviour. Thanks, Vladimir On 10/19/18 7:22 AM, Erik ?sterlund wrote: > Hi, > > Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent > class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to > class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. > > To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It > returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this > question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache > unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod > is_unloading() the first time it is called. > > So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that > simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the > inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should > be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. > > The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel > variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the > nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing > this, nor to have two phases for parallel nmethod unloading. > > While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. > > Bug ID: > https://bugs.openjdk.java.net/browse/JDK-8209189 > > Webrev: > cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ > > Thanks, > /Erik From erik.osterlund at oracle.com Thu Oct 25 19:29:36 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 25 Oct 2018 21:29:36 +0200 Subject: RFR: 8212996: Use AS_NO_KEEPALIVE when accessing dead java.lang.invoke.CallSites during nmethod unloading Message-ID: Hi, When nmethods are unloaded, they may have dead java.lang.invoke.CallSites loaded, and then we follow their dead java.lang.invoke.MethodHandleNatives.CallSiteContext, so that its dependency context can be cleaned. These dead oops should be loaded with AS_NO_KEEPALIVE accessors, otherwise GC barriers can become very unhappy. Webrev: http://cr.openjdk.java.net/~eosterlund/8212996/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212996 Thanks, /Erik From coleen.phillimore at oracle.com Thu Oct 25 20:42:00 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 25 Oct 2018 16:42:00 -0400 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes Message-ID: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> Summary: Don't return unloaded klasses. Make sure access is protected by Compile_lock. See bug for more description.? This test has compiler changes since the weak klass links in Klass (and implementor in InstanceKlass are used by the compiler).? All the compiler jtreg tests passed. Tested with mach5 tier1-7 and test added. open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8212958 Thanks, Coleen From coleen.phillimore at oracle.com Thu Oct 25 20:53:19 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 25 Oct 2018 16:53:19 -0400 Subject: RFR: 8212996: Use AS_NO_KEEPALIVE when accessing dead java.lang.invoke.CallSites during nmethod unloading In-Reply-To: References: Message-ID: <0daef4c4-797b-cdde-7e0a-10380a68e514@oracle.com> This doesn't look bad to me.? It's pretty specific for unloading so the NO_KEEP_ALIVE makes sense here even though we're loading dead oops through dead oops. Thanks, Coleen On 10/25/18 3:29 PM, Erik ?sterlund wrote: > Hi, > > When nmethods are unloaded, they may have dead > java.lang.invoke.CallSites loaded, and then we follow their dead > java.lang.invoke.MethodHandleNatives.CallSiteContext, so that its > dependency context can be cleaned. These dead oops should be loaded > with AS_NO_KEEPALIVE accessors, otherwise GC barriers can become very > unhappy. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212996/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212996 > > Thanks, > /Erik From coleen.phillimore at oracle.com Thu Oct 25 22:01:36 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 25 Oct 2018 18:01:36 -0400 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> Message-ID: <27ebdbbd-833e-5b78-93b0-83af04834835@oracle.com> http://cr.openjdk.java.net/~kbarrett/8210986/open.00/src/hotspot/share/gc/shared/oopStorage.cpp.frames.html 392 // it is called in all kinds of contexts where even quote low ranked locks typo: s/quote/quite/ 425 Block* block = block_for_allocation(); 426 if (block == NULL) return NULL; // Block allocation failed. This could be "get_block_for_allocation" because it's not blocking (afaict).? Name is somewhat ambiguous. http://cr.openjdk.java.net/~kbarrett/8210986/open.00/src/hotspot/share/runtime/serviceThread.cpp.frames.html ick.? Since this is a static list, can it be declared outside the scope of ServiceThread::entry(), like before the functions that loop through the oopstorages? 109 OopStorage* const oopstorages[] = { 110 JNIHandles::global_handles(), 111 JNIHandles::weak_global_handles(), 112 StringTable::weak_storage(), 113 SystemDictionary::vm_weak_oop_storage() 114 }; 115 const size_t oopstorage_count = ARRAY_SIZE(oopstorages); 116 I think it would bother me less that we have to have all the OopStorages listed somewhere, if it were static scope and not in the service thread function.? Are these all the ones we have? Otherwise I think this is fine. Thanks, Coleen On 10/19/18 4:47 PM, Kim Barrett wrote: > Please review this change to the Service thread to delete empty > OopStorage blocks when needed. > > As part of this, eliminated delete_empty_blocks_safepoint, and renamed > delete_empty_blocks_concurrent to remove the now redundant _concurrent > suffix. Also added a predicate for the Service thread's use, to test > whether there is cleanup work to be done. > > The previously unused empty block deletion has been revised for this > new usage. This includes making loops safepoint polite. > > Various parts of OopStorage are now aware of the Service thread > cleanup and notify that thread when appropriate. (allocate's > obtaining the block to allocate from was refactored to make it easier > to insert that notification.) > > As part of this, OopStorage::Block::release_entries now takes the > owning storage object as an argument (rather than a pointer to its > _deferred_updates list). This allows release_entries to perform > additional operations on the owner (so long as C++ DR45 has been > fixed, and we've added more AIX-specific workarounds for that). > > While touching the Service thread, fixed Service_lock locking to use > use MonitorLockerEx rather than MutexLockerEx. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8210986 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8210986/open.00/ > > Testing: > Mach5 tier1-5. > Ran some performance tests to verify no regressions due to additional > load on the Service thread. > From kim.barrett at oracle.com Thu Oct 25 22:26:30 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Oct 2018 18:26:30 -0400 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: <27ebdbbd-833e-5b78-93b0-83af04834835@oracle.com> References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> <27ebdbbd-833e-5b78-93b0-83af04834835@oracle.com> Message-ID: <08A02CFE-EB69-409F-9DA5-BA2B82F2B249@oracle.com> > On Oct 25, 2018, at 6:01 PM, coleen.phillimore at oracle.com wrote: Thanks for looking at this. > http://cr.openjdk.java.net/~kbarrett/8210986/open.00/src/hotspot/share/gc/shared/oopStorage.cpp.frames.html > > 392 // it is called in all kinds of contexts where even quote low ranked locks > > typo: s/quote/quite/ Well spotted. Will fix before pushing. > 425 Block* block = block_for_allocation(); > 426 if (block == NULL) return NULL; // Block allocation failed. > > This could be "get_block_for_allocation" because it's not blocking (afaict). Name is somewhat ambiguous. Hotspot style guide says getters are noun phrases, with no ?get_" noise word. > http://cr.openjdk.java.net/~kbarrett/8210986/open.00/src/hotspot/share/runtime/serviceThread.cpp.frames.html > > ick. Since this is a static list, can it be declared outside the scope of ServiceThread::entry(), like before the functions that loop through the oopstorages? > > 109 OopStorage* const oopstorages[] = { > 110 JNIHandles::global_handles(), > 111 JNIHandles::weak_global_handles(), > 112 StringTable::weak_storage(), > 113 SystemDictionary::vm_weak_oop_storage() > 114 }; > 115 const size_t oopstorage_count = ARRAY_SIZE(oopstorages); > 116 > > I think it would bother me less that we have to have all the OopStorages listed somewhere, if it were static scope and not in the service thread function. Are these all the ones we have? It has to be at function scope. If it were at file scope, it might (and likely would) be initialized before the various OopStorage objects have been initialized, and we'd have an array of NULLs. I didn't declare it (function scope) static since it doesn't matter. It's only constructed once on the single entry to the function. And yes, this array is supposed to be complete. If there were any other places that needed a similarly complete set, I'd make a helper function. But so far, there isn't such a place, so I'm not sure where I'd put such a function. But I admit I'm not crazy about this array being here. It's not an obvious place to look if one were adding another OopStorage. > Otherwise I think this is fine. Thanks. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8210986 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8210986/open.00/ >> >> Testing: >> Mach5 tier1-5. >> Ran some performance tests to verify no regressions due to additional >> load on the Service thread. From coleen.phillimore at oracle.com Thu Oct 25 23:00:04 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 25 Oct 2018 19:00:04 -0400 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: <08A02CFE-EB69-409F-9DA5-BA2B82F2B249@oracle.com> References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> <27ebdbbd-833e-5b78-93b0-83af04834835@oracle.com> <08A02CFE-EB69-409F-9DA5-BA2B82F2B249@oracle.com> Message-ID: <3945eb38-b053-a802-666b-7d520822360a@oracle.com> On 10/25/18 6:26 PM, Kim Barrett wrote: >> On Oct 25, 2018, at 6:01 PM, coleen.phillimore at oracle.com wrote: > Thanks for looking at this. > >> http://cr.openjdk.java.net/~kbarrett/8210986/open.00/src/hotspot/share/gc/shared/oopStorage.cpp.frames.html >> >> 392 // it is called in all kinds of contexts where even quote low ranked locks >> >> typo: s/quote/quite/ > Well spotted. Will fix before pushing. > >> 425 Block* block = block_for_allocation(); >> 426 if (block == NULL) return NULL; // Block allocation failed. >> >> This could be "get_block_for_allocation" because it's not blocking (afaict). Name is somewhat ambiguous. > Hotspot style guide says getters are noun phrases, with no ?get_" noise word. The style guide says that we avoid of noise word get for cases like this: Block* block() const { return _block; } void set_block(Block* b) { _block = b; } If the function is doing anything else, you can say get.? Also because 'block' is a verb, it makes it confusing. > >> http://cr.openjdk.java.net/~kbarrett/8210986/open.00/src/hotspot/share/runtime/serviceThread.cpp.frames.html >> >> ick. Since this is a static list, can it be declared outside the scope of ServiceThread::entry(), like before the functions that loop through the oopstorages? >> >> 109 OopStorage* const oopstorages[] = { >> 110 JNIHandles::global_handles(), >> 111 JNIHandles::weak_global_handles(), >> 112 StringTable::weak_storage(), >> 113 SystemDictionary::vm_weak_oop_storage() >> 114 }; >> 115 const size_t oopstorage_count = ARRAY_SIZE(oopstorages); >> 116 >> >> I think it would bother me less that we have to have all the OopStorages listed somewhere, if it were static scope and not in the service thread function. Are these all the ones we have? > It has to be at function scope. If it were at file scope, it might > (and likely would) be initialized before the various OopStorage > objects have been initialized, and we'd have an array of NULLs. > > I didn't declare it (function scope) static since it doesn't matter. > It's only constructed once on the single entry to the function. > > And yes, this array is supposed to be complete. If there were any > other places that needed a similarly complete set, I'd make a helper > function. But so far, there isn't such a place, so I'm not sure where > I'd put such a function. But I admit I'm not crazy about this array > being here. It's not an obvious place to look if one were adding > another OopStorage. Oh well, I see why it can't be static.? Maybe somewhere better will occur to us when we add more.? This looks okay to me. Coleen > >> Otherwise I think this is fine. > Thanks. > >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8210986 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8210986/open.00/ >>> >>> Testing: >>> Mach5 tier1-5. >>> Ran some performance tests to verify no regressions due to additional >>> load on the Service thread. > From dean.long at oracle.com Fri Oct 26 06:17:49 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 25 Oct 2018 23:17:49 -0700 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> Message-ID: Hi Coleen.? Most of this looks good to me, but I have a few questions: Is the need for Compile_lock around accesses new in 12, because the implementor and subclass links can now be cleaned without being at a safepoint? Style typo: In ciInstanceKlass::compute_has_subklass(), I think it should be return "ik->subklass() != NULL". I see that you removed the memoizing of _has_subklass but left _implementor.? I'm concerned that this might have a performance impact and change behavior.? It seems to allow has_subklass() to change from true to false (and is_leaf_type() to change from false to true) where previously that should have been impossible.? With some fiddling with the locking, I was able to add back _has_subklass.? Do you remember why you removed it? dl On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: > Summary: Don't return unloaded klasses. Make sure access is protected > by Compile_lock. > > See bug for more description.? This test has compiler changes since > the weak klass links in Klass (and implementor in InstanceKlass are > used by the compiler).? All the compiler jtreg tests passed. > > Tested with mach5 tier1-7 and test added. > > open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8212958 > > Thanks, > Coleen From erik.osterlund at oracle.com Fri Oct 26 06:18:23 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 26 Oct 2018 08:18:23 +0200 Subject: RFR: 8212996: Use AS_NO_KEEPALIVE when accessing dead java.lang.invoke.CallSites during nmethod unloading In-Reply-To: <0daef4c4-797b-cdde-7e0a-10380a68e514@oracle.com> References: <0daef4c4-797b-cdde-7e0a-10380a68e514@oracle.com> Message-ID: Hi Coleen, Thanks for the review. And yeah, this could have turned out a lot worse. /Erik On 2018-10-25 22:53, coleen.phillimore at oracle.com wrote: > > This doesn't look bad to me.? It's pretty specific for unloading so the > NO_KEEP_ALIVE makes sense here even though we're loading dead oops > through dead oops. > > Thanks, > Coleen > > On 10/25/18 3:29 PM, Erik ?sterlund wrote: >> Hi, >> >> When nmethods are unloaded, they may have dead >> java.lang.invoke.CallSites loaded, and then we follow their dead >> java.lang.invoke.MethodHandleNatives.CallSiteContext, so that its >> dependency context can be cleaned. These dead oops should be loaded >> with AS_NO_KEEPALIVE accessors, otherwise GC barriers can become very >> unhappy. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212996/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212996 >> >> Thanks, >> /Erik > From robbin.ehn at oracle.com Fri Oct 26 08:06:18 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 26 Oct 2018 10:06:18 +0200 Subject: RFR: 8212827: GlobalCounter should support nested critical sections In-Reply-To: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> References: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> Message-ID: <943c0c03-5936-bbd4-afcc-fdeb21000994@oracle.com> Hi Kim, On 23/10/2018 19:07, Kim Barrett wrote: > Please review this change to GlobalCounter to support nested critical > sections. Great! > > While in the area, cleaned up a few things: > - Improved comments for GlobalCounter API. > - Cleaned up write_synchronize global counter increment. > - Removed a useless acquire barrier in critical_section_begin. Thanks for doing that! > > CR: > https://bugs.openjdk.java.net/browse/JDK-8212827 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8212827/open.00/ Looks good, ship it, below is just some notes. I would like the return value here: inline uintx GlobalCounter::critical_section_begin(Thread *thread) { And thus the parameter here: 46 inline void GlobalCounter::critical_section_end(Thread *thread, uintx begin_value) { be an opaque type so we don't leak out the implementation, but as long as the GlobalCounter have the implementation as name, that doesn't help much. I'm thinking of something like: inline RCUContext RCU::critical_section_begin(Thread *thread) { inline void RCU::critical_section_end(Thread *thread, RCUContext context) { Or inline CSContext CriticalSection::begin(Thread *thread) { inline void CriticalSection::end(Thread *thread, CSContext context) { Thanks, Robbin > > Testing: > mach5 tier1-5 > New gtest directly testing nested critical sections. > > From coleen.phillimore at oracle.com Fri Oct 26 12:09:56 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 26 Oct 2018 08:09:56 -0400 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> Message-ID: <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> Hi Dean,? Thank you for reviewing this. On 10/26/18 2:17 AM, dean.long at oracle.com wrote: > Hi Coleen.? Most of this looks good to me, but I have a few questions: > > Is the need for Compile_lock around accesses new in 12, because the > implementor and subclass links can now be cleaned without being at a > safepoint? Yes.? Erik's going to take Compile_lock to clean these during concurrent class unloading.? There were some unlocked accesses though that seemed suspicious before though. > > Style typo: In ciInstanceKlass::compute_has_subklass(), I think it > should be return "ik->subklass() != NULL". Fixed. > > I see that you removed the memoizing of _has_subklass but left > _implementor.? I'm concerned that this might have a performance impact > and change behavior.? It seems to allow has_subklass() to change from > true to false (and is_leaf_type() to change from false to true) where > previously that should have been impossible.? With some fiddling with > the locking, I was able to add back _has_subklass.? Do you remember > why you removed it? Yes, I removed the memoizing for _has_subklass because I couldn't take the Compile_lock in the ciInstanceKlass constructor, because some callers had it.?? If you have a change that will make that work, I'd be happy to use yours.? I was trying to not change anything semantically, just grab fresh results from InstanceKlass which I didn't think would affect performance (although it grabs Compile_lock so maybe it could).?? We can revert this part and add locking to ciInstanceKlass constructor.? I've been trying to avoid conditional locking. Thanks, Coleen > > dl > > On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >> Summary: Don't return unloaded klasses. Make sure access is protected >> by Compile_lock. >> >> See bug for more description.? This test has compiler changes since >> the weak klass links in Klass (and implementor in InstanceKlass are >> used by the compiler).? All the compiler jtreg tests passed. >> >> Tested with mach5 tier1-7 and test added. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >> >> Thanks, >> Coleen > From sgehwolf at redhat.com Fri Oct 26 12:18:23 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 26 Oct 2018 14:18:23 +0200 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: <82f2ada623c83dfb76805dd72d5522d2d97fa801.camel@redhat.com> On Thu, 2018-10-25 at 12:15 -0300, Martin Balao wrote: > Hi Lois, > > Thanks for having a look at this. > > Yes, I have a test that triggers this execution path. Unfortunately, this > is a security test that I cannot publicly share. What I can tell you, > though, is that the parent class loader is a "delegating class loader" > which never defined a class and that's the reason why it doesn't have a > ClassLoaderData instance created. > > I'm not sure if this would be needed in other places where the idiom is not > used. The reason is that there might be any other implicit assumptions that > makes it impossible. That's why I limited the scope of this changeset to > the only case that I'm sure it's possible to trigger. FWIW, I can confirm that the proposed patch fixes the issue we were seeing. Thanks, Severin On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan wrote: > > > On 10/24/2018 3:58 PM, Martin Balao wrote: > > > > Hi, > > > > > > Can I have a review for JDK-8212937 [1]? > > > > > > Webrev.00: > > > > > > * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ > > > * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937. > > > webrev.00.zip > > > > > > By obtaining the ClassLoaderData instance through a call to > > > SystemDictionary::register_loader, we make sure that either an existing > > > instance is obtained or a new one created. This same idiom has already > > > been > > > used in other places where there are no guarantees that a previous > > > ClassLoaderData instance exists for a given class loader. See futher > > > information in JDK-8212937 ticket [1]. > > > > > > Thanks, > > > Martin.- > > > > > > -- > > > [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 > > > > > > > Hi Martin, > > > > I agree that the same idiom is used in other places, but there are also a > > couple of occurrences of java_lang_ClassLoader::parent in > > classfile/classLoaderStats.cpp and classfile/classLoaderHierarchyDCmd.cpp > > that do not. In order to better understand, do you have a test case that > > demonstrates this issue? > > > > Thanks, > > Lois > > From erik.osterlund at oracle.com Fri Oct 26 12:20:33 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 26 Oct 2018 14:20:33 +0200 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: References: <5BC9E8BA.50407@oracle.com> Message-ID: <5BD30691.8020508@oracle.com> Hi Vladimir, Thank you for having a look at these changes. On 2018-10-25 19:34, Vladimir Kozlov wrote: > Hi Erik, > > First, what testing you did? This is significant rewrite and requires > a lot of testing. I don't see a link to test results in RFE. Any > performance testing? In terms of testing, I have run this through tier 1-6 in mach 5 with no failures related to my changes. As far as performance testing goes, I did spend some extra attention to the G1 parallel case, as it is most sensitive to performance. I ran through DaCapo2009, SPECjvm98, SPECjvm2008, SPECjbb2000, SPECjbb2005 and SPECjbb2015 with fairly constrained heap size to cause a lot of old GCs. I recorded the time taken for the parallel class unloading and cleanup task which is where my changes would affect things. Here is a summary of the results: new DaCapo2009: mean 19.46 min 4.97 max 41.94 std 10.74 old DaCapo2009: mean 20.22 min 6.54 max 42.55 std 10.41 new SPECjvm98: mean 2.4 min 0.34 max 6.43 std 1.66 old SPECjvm98: mean 3.18 min 0.88 max 8.15 std 2.01 new SPECjvm2008: mean 374.75 min 69.72 max 477.76 std 73.42 old SPECjvm2008: mean 345.14 min 84.85 max 496.55 std 101.24 new SPECjbb2000: mean 354.69 min 0.61 max 482.97 std 87.87 old SPECjbb2000: mean 360.56 min 0.56 max 482.86 std 85.83 new SPECjbb2005: mean 174.69 min 0.8 max 242.71 std 36.37 old SPECjbb2005: mean 174.66 min 0.74 max 242.76 std 36.82 new SPECjbb2015: mean 2.71 min 2.38 max 3.05 std 0.47 old SPECjbb2015: mean 4.05 min 3.52 max 4.58 std 0.75 As we can see, my approach seems to perform mostly "a bit better". Looks like the odd one I don't perform better is SPECjbb2008, but that is also the benchmark where I see highest standard deviations, so that might just be noise. The reason I think why I get overall better results, is because I always do everything in one pass over the code heap, instead of two with an intrusive wait barrier separating the two parallel phases. And, as a bonus when you do it this way, for each nmethod, you first walk the oop reloc entries to figure out if the nmethod is alive, and then immediately clean the same nmethod, walking the same reloc entries again, to find IC caches, while the caches are warm. That improved data locality gives this approach an edge. And the bigger the code cache, the bigger that advantage becomes. I have not looked with the same level of detail on SerialGC and ParallelGC, because it seems strictly less interesting (when you do a full STW GC, the unloading times is by far not the biggest problem, which is why it's done with 1 thread). However, FWIW, the same improved data locality should be beneficial there too by again doing everything in one single pass over the code cache instead of two. > It seems you removed the only case (in do_unloading()) where > CodeCache::do_unloading_nmethod_caches() is called. I can't find other > place where it is called after your change. Why keep it then? Good point. Removed. > It seems CompiledMethod::is_unloading() is performance critical. Can > we make IsUnloadingBehaviour::current()->is_unloading(this) as non > virtual call? I see only ClosureIsUnloadingBehaviour subclass of > IsUnloadingBehaviour. Yes, is_unloading does get pretty hot indeed. However, after it has been called once on an nmethod, the result of the computation is cached. So the virtual call you point out will only be called once per nmethod, and then the cached value will be used instead. And I need to override that virtual call with ZGC, as this will be called outside of safepoints and we have to use locking to guarantee safety of computing the value, especially since the immediate oops in the code stream are misaligned, and will get patched concurrently by nmethod entry barriers. Thanks, /Erik > Thanks, > Vladimir > > On 10/19/18 7:22 AM, Erik ?sterlund wrote: >> Hi, >> >> Today the nmethods are unloaded either serially or in parallel due to >> GC (and e.g. class unloading). With ZGC concurrent class unloading, a >> concurrent fashion is required. Rather than inventing yet a third way >> of unloading nmethods due to class unloading, it would be ideal if >> there could be one unified way of doing it that makes everyone happy. >> >> To solve this problem in a more general way, a new member function on >> CompiledMethod is added: is_unloading(). It returns whether a >> CompiledMethod has broken oops. In order to not have to iterate over >> all the oops every time this question is asked, it caches the result, >> so it needs to be computed only once per "epoch". Every time a >> CodeCache unloading is triggered by the GC, a new epoch is started, >> which forces re-computation of whether the CompiledMethod >> is_unloading() the first time it is called. >> >> So by having is_unloading() be lazily evaluated, it is now possible >> to build a do_unloading() method on nmethod that simply makes the >> nmethod unloaded if it is_unloading(), and otherwise cleans the >> various caches. Cleaning e.g. the inline caches of said nmethod, uses >> the same is_unloading() method on its targets to figure out if the IC >> cache should be cleaned or not. Again, the epoched caching mechanism >> makes sure we don't recompute this value. >> >> The new do_unloading() method may be called in both serial, parallel >> and concurrent contexts. So I removed the parallel variation of this >> that we had before, that unnecessarily postponed the unloading due to >> not having computed whether the nmethod should be unloaded or not. >> Since that is now computed on-demand lazily, there is no longer a >> need for postponing this, nor to have two phases for parallel nmethod >> unloading. >> >> While there is more work involved in making concurrent nmethod >> unloading work, this is a good cleanup towards that goal. >> >> Bug ID: >> https://bugs.openjdk.java.net/browse/JDK-8209189 >> >> Webrev: >> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >> >> Thanks, >> /Erik From erik.osterlund at oracle.com Fri Oct 26 14:56:48 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 26 Oct 2018 16:56:48 +0200 Subject: RFR: 8212827: GlobalCounter should support nested critical sections In-Reply-To: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> References: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> Message-ID: <5BD32B30.9040707@oracle.com> Hi Kim, Looks good. Thanks, /Erik On 2018-10-23 19:07, Kim Barrett wrote: > Please review this change to GlobalCounter to support nested critical > sections. > > While in the area, cleaned up a few things: > - Improved comments for GlobalCounter API. > - Cleaned up write_synchronize global counter increment. > - Removed a useless acquire barrier in critical_section_begin. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8212827 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8212827/open.00/ > > Testing: > mach5 tier1-5 > New gtest directly testing nested critical sections. > > From erik.osterlund at oracle.com Fri Oct 26 15:04:03 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 26 Oct 2018 17:04:03 +0200 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> Message-ID: <5BD32CE3.5070704@oracle.com> Hi Coleen, On 2018-10-26 14:09, coleen.phillimore at oracle.com wrote: > > Hi Dean, Thank you for reviewing this. > > On 10/26/18 2:17 AM, dean.long at oracle.com wrote: >> Hi Coleen. Most of this looks good to me, but I have a few questions: >> >> Is the need for Compile_lock around accesses new in 12, because the >> implementor and subclass links can now be cleaned without being at a >> safepoint? > > Yes. Erik's going to take Compile_lock to clean these during > concurrent class unloading. There were some unlocked accesses though > that seemed suspicious before though. >> >> Style typo: In ciInstanceKlass::compute_has_subklass(), I think it >> should be return "ik->subklass() != NULL". > > Fixed. >> >> I see that you removed the memoizing of _has_subklass but left >> _implementor. I'm concerned that this might have a performance >> impact and change behavior. It seems to allow has_subklass() to >> change from true to false (and is_leaf_type() to change from false to >> true) where previously that should have been impossible. With some >> fiddling with the locking, I was able to add back _has_subklass. Do >> you remember why you removed it? > > Yes, I removed the memoizing for _has_subklass because I couldn't take > the Compile_lock in the ciInstanceKlass constructor, because some > callers had it. If you have a change that will make that work, I'd > be happy to use yours. I was trying to not change anything > semantically, just grab fresh results from InstanceKlass which I > didn't think would affect performance (although it grabs Compile_lock > so maybe it could). We can revert this part and add locking to > ciInstanceKlass constructor. I've been trying to avoid conditional > locking. Can you really safely rely on a cached boolean w.r.t. unloading? The instant we release the mark end safepoint, compiler threads will come asking if there are subklasses, before we have started doing unloading, and they have to get consistent answers that there are no subklasses, if the one subklass had died. So it seems to me that this value must be computed and not cached. You can however use the same trick I used for CompiledMethod::is_unloading in 8212958, and make the answer belong to an epoch. That way, we force computation for the first time only that the question is asked, and use the cached result that we know to be consistent w.r.t. unloading in subsequent calls. But I wonder if we are micro optimizing by doing that here. Thanks, /Erik > Thanks, > Coleen >> >> dl >> >> On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >>> Summary: Don't return unloaded klasses. Make sure access is >>> protected by Compile_lock. >>> >>> See bug for more description. This test has compiler changes since >>> the weak klass links in Klass (and implementor in InstanceKlass are >>> used by the compiler). All the compiler jtreg tests passed. >>> >>> Tested with mach5 tier1-7 and test added. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >>> >>> Thanks, >>> Coleen >> > From vladimir.kozlov at oracle.com Fri Oct 26 15:15:31 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 26 Oct 2018 08:15:31 -0700 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BD30691.8020508@oracle.com> References: <5BC9E8BA.50407@oracle.com> <5BD30691.8020508@oracle.com> Message-ID: Thank you, Erik, for data. Looks good. And I am fine with virtual call in is_unloading() since you will have other variant for ZGC - I thought it is only one. Thanks, Vladimir On 10/26/18 5:20 AM, Erik ?sterlund wrote: > Hi Vladimir, > > Thank you for having a look at these changes. > > On 2018-10-25 19:34, Vladimir Kozlov wrote: >> Hi Erik, >> >> First, what testing you did? This is significant rewrite and requires a lot of testing. I don't see a link to test >> results in RFE. Any performance testing? > > In terms of testing, I have run this through tier 1-6 in mach 5 with no failures related to my changes. > > As far as performance testing goes, I did spend some extra attention to the G1 parallel case, as it is most sensitive to > performance. > I ran through DaCapo2009, SPECjvm98, SPECjvm2008, SPECjbb2000, SPECjbb2005 and SPECjbb2015 with fairly constrained heap > size to cause a lot of old GCs. I recorded the time taken for the parallel class unloading and cleanup task which is > where my changes would affect things. > > Here is a summary of the results: > new DaCapo2009: > mean??? 19.46?? min???? 4.97??? max???? 41.94?? std???? 10.74 > old DaCapo2009: > mean??? 20.22?? min???? 6.54??? max???? 42.55?? std???? 10.41 > > new SPECjvm98: > mean??? 2.4???? min???? 0.34??? max???? 6.43??? std???? 1.66 > old SPECjvm98: > mean??? 3.18??? min???? 0.88??? max???? 8.15??? std???? 2.01 > > new SPECjvm2008: > mean??? 374.75? min???? 69.72?? max???? 477.76? std???? 73.42 > old SPECjvm2008: > mean??? 345.14? min???? 84.85?? max???? 496.55? std???? 101.24 > > new SPECjbb2000: > mean??? 354.69? min???? 0.61??? max???? 482.97? std???? 87.87 > old SPECjbb2000: > mean??? 360.56? min???? 0.56??? max???? 482.86? std???? 85.83 > > new SPECjbb2005: > mean??? 174.69? min???? 0.8???? max???? 242.71? std???? 36.37 > old SPECjbb2005: > mean??? 174.66? min???? 0.74??? max???? 242.76? std???? 36.82 > > new SPECjbb2015: > mean??? 2.71??? min???? 2.38??? max???? 3.05??? std???? 0.47 > old SPECjbb2015: > mean??? 4.05??? min???? 3.52??? max???? 4.58??? std???? 0.75 > > As we can see, my approach seems to perform mostly "a bit better". Looks like the odd one I don't perform better is > SPECjbb2008, but that is also the benchmark where I see highest standard deviations, so that might just be noise. The > reason I think why I get overall better results, is because I always do everything in one pass over the code heap, > instead of two with an intrusive wait barrier separating the two parallel phases. And, as a bonus when you do it this > way, for each nmethod, you first walk the oop reloc entries to figure out if the nmethod is alive, and then immediately > clean the same nmethod, walking the same reloc entries again, to find IC caches, while the caches are warm. That > improved data locality gives this approach an edge. And the bigger the code cache, the bigger that advantage becomes. > > I have not looked with the same level of detail on SerialGC and ParallelGC, because it seems strictly less interesting > (when you do a full STW GC, the unloading times is by far not the biggest problem, which is why it's done with 1 > thread). However, FWIW, the same improved data locality should be beneficial there too by again doing everything in one > single pass over the code cache instead of two. > >> It seems you removed the only case (in do_unloading()) where CodeCache::do_unloading_nmethod_caches() is called. I >> can't find other place where it is called after your change. Why keep it then? > > Good point. Removed. > >> It seems CompiledMethod::is_unloading() is performance critical. Can we make >> IsUnloadingBehaviour::current()->is_unloading(this) as non virtual call? I see only ClosureIsUnloadingBehaviour >> subclass of IsUnloadingBehaviour. > > Yes, is_unloading does get pretty hot indeed. However, after it has been called once on an nmethod, the result of the > computation is cached. So the virtual call you point out will only be called once per nmethod, and then the cached value > will be used instead. And I need to override that virtual call with ZGC, as this will be called outside of safepoints > and we have to use locking to guarantee safety of computing the value, especially since the immediate oops in the code > stream are misaligned, and will get patched concurrently by nmethod entry barriers. > > Thanks, > /Erik > >> Thanks, >> Vladimir >> >> On 10/19/18 7:22 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC >>> concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading >>> nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone >>> happy. >>> >>> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It >>> returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this >>> question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache >>> unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod >>> is_unloading() the first time it is called. >>> >>> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that >>> simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the >>> inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache >>> should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >>> >>> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the >>> parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed >>> whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need >>> for postponing this, nor to have two phases for parallel nmethod unloading. >>> >>> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >>> >>> Bug ID: >>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>> >>> Webrev: >>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>> >>> Thanks, >>> /Erik > From erik.osterlund at oracle.com Fri Oct 26 15:18:07 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 26 Oct 2018 17:18:07 +0200 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: References: <5BC9E8BA.50407@oracle.com> <5BD30691.8020508@oracle.com> Message-ID: <5BD3302F.9040203@oracle.com> Hi Vladimir, Thank you for the review! /Erik On 2018-10-26 17:15, Vladimir Kozlov wrote: > Thank you, Erik, for data. > > Looks good. And I am fine with virtual call in is_unloading() since > you will have other variant for ZGC - I thought it is only one. > > Thanks, > Vladimir > > On 10/26/18 5:20 AM, Erik ?sterlund wrote: >> Hi Vladimir, >> >> Thank you for having a look at these changes. >> >> On 2018-10-25 19:34, Vladimir Kozlov wrote: >>> Hi Erik, >>> >>> First, what testing you did? This is significant rewrite and >>> requires a lot of testing. I don't see a link to test results in >>> RFE. Any performance testing? >> >> In terms of testing, I have run this through tier 1-6 in mach 5 with >> no failures related to my changes. >> >> As far as performance testing goes, I did spend some extra attention >> to the G1 parallel case, as it is most sensitive to performance. >> I ran through DaCapo2009, SPECjvm98, SPECjvm2008, SPECjbb2000, >> SPECjbb2005 and SPECjbb2015 with fairly constrained heap size to >> cause a lot of old GCs. I recorded the time taken for the parallel >> class unloading and cleanup task which is where my changes would >> affect things. >> >> Here is a summary of the results: >> new DaCapo2009: >> mean 19.46 min 4.97 max 41.94 std 10.74 >> old DaCapo2009: >> mean 20.22 min 6.54 max 42.55 std 10.41 >> >> new SPECjvm98: >> mean 2.4 min 0.34 max 6.43 std 1.66 >> old SPECjvm98: >> mean 3.18 min 0.88 max 8.15 std 2.01 >> >> new SPECjvm2008: >> mean 374.75 min 69.72 max 477.76 std 73.42 >> old SPECjvm2008: >> mean 345.14 min 84.85 max 496.55 std 101.24 >> >> new SPECjbb2000: >> mean 354.69 min 0.61 max 482.97 std 87.87 >> old SPECjbb2000: >> mean 360.56 min 0.56 max 482.86 std 85.83 >> >> new SPECjbb2005: >> mean 174.69 min 0.8 max 242.71 std 36.37 >> old SPECjbb2005: >> mean 174.66 min 0.74 max 242.76 std 36.82 >> >> new SPECjbb2015: >> mean 2.71 min 2.38 max 3.05 std 0.47 >> old SPECjbb2015: >> mean 4.05 min 3.52 max 4.58 std 0.75 >> >> As we can see, my approach seems to perform mostly "a bit better". >> Looks like the odd one I don't perform better is SPECjbb2008, but >> that is also the benchmark where I see highest standard deviations, >> so that might just be noise. The reason I think why I get overall >> better results, is because I always do everything in one pass over >> the code heap, instead of two with an intrusive wait barrier >> separating the two parallel phases. And, as a bonus when you do it >> this way, for each nmethod, you first walk the oop reloc entries to >> figure out if the nmethod is alive, and then immediately clean the >> same nmethod, walking the same reloc entries again, to find IC >> caches, while the caches are warm. That improved data locality gives >> this approach an edge. And the bigger the code cache, the bigger that >> advantage becomes. >> >> I have not looked with the same level of detail on SerialGC and >> ParallelGC, because it seems strictly less interesting (when you do a >> full STW GC, the unloading times is by far not the biggest problem, >> which is why it's done with 1 thread). However, FWIW, the same >> improved data locality should be beneficial there too by again doing >> everything in one single pass over the code cache instead of two. >> >>> It seems you removed the only case (in do_unloading()) where >>> CodeCache::do_unloading_nmethod_caches() is called. I can't find >>> other place where it is called after your change. Why keep it then? >> >> Good point. Removed. >> >>> It seems CompiledMethod::is_unloading() is performance critical. Can >>> we make IsUnloadingBehaviour::current()->is_unloading(this) as non >>> virtual call? I see only ClosureIsUnloadingBehaviour subclass of >>> IsUnloadingBehaviour. >> >> Yes, is_unloading does get pretty hot indeed. However, after it has >> been called once on an nmethod, the result of the computation is >> cached. So the virtual call you point out will only be called once >> per nmethod, and then the cached value will be used instead. And I >> need to override that virtual call with ZGC, as this will be called >> outside of safepoints and we have to use locking to guarantee safety >> of computing the value, especially since the immediate oops in the >> code stream are misaligned, and will get patched concurrently by >> nmethod entry barriers. >> >> Thanks, >> /Erik >> >>> Thanks, >>> Vladimir >>> >>> On 10/19/18 7:22 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> Today the nmethods are unloaded either serially or in parallel due >>>> to GC (and e.g. class unloading). With ZGC concurrent class >>>> unloading, a concurrent fashion is required. Rather than inventing >>>> yet a third way of unloading nmethods due to class unloading, it >>>> would be ideal if there could be one unified way of doing it that >>>> makes everyone happy. >>>> >>>> To solve this problem in a more general way, a new member function >>>> on CompiledMethod is added: is_unloading(). It returns whether a >>>> CompiledMethod has broken oops. In order to not have to iterate >>>> over all the oops every time this question is asked, it caches the >>>> result, so it needs to be computed only once per "epoch". Every >>>> time a CodeCache unloading is triggered by the GC, a new epoch is >>>> started, which forces re-computation of whether the CompiledMethod >>>> is_unloading() the first time it is called. >>>> >>>> So by having is_unloading() be lazily evaluated, it is now possible >>>> to build a do_unloading() method on nmethod that simply makes the >>>> nmethod unloaded if it is_unloading(), and otherwise cleans the >>>> various caches. Cleaning e.g. the inline caches of said nmethod, >>>> uses the same is_unloading() method on its targets to figure out if >>>> the IC cache should be cleaned or not. Again, the epoched caching >>>> mechanism makes sure we don't recompute this value. >>>> >>>> The new do_unloading() method may be called in both serial, >>>> parallel and concurrent contexts. So I removed the parallel >>>> variation of this that we had before, that unnecessarily postponed >>>> the unloading due to not having computed whether the nmethod should >>>> be unloaded or not. Since that is now computed on-demand lazily, >>>> there is no longer a need for postponing this, nor to have two >>>> phases for parallel nmethod unloading. >>>> >>>> While there is more work involved in making concurrent nmethod >>>> unloading work, this is a good cleanup towards that goal. >>>> >>>> Bug ID: >>>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>>> >>>> Webrev: >>>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>>> >>>> Thanks, >>>> /Erik >> From coleen.phillimore at oracle.com Fri Oct 26 15:58:12 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 26 Oct 2018 11:58:12 -0400 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BC9E8BA.50407@oracle.com> References: <5BC9E8BA.50407@oracle.com> Message-ID: <46f30a22-c2b3-bc18-2334-e84476b0ba64@oracle.com> I think this looks really good. I appreciate the cleanup in this complexity.? It's nice that nmethod/CompiledMethod as non-gc structure with oops, there's simply an oops_do() vs. all these do_unloading_scopes() functions.?? It reduces the special handling. I spent a lot of time trying to figure out how this "postponed" cleaning works.? This make a lot of sense. http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/src/hotspot/share/code/compiledMethod.cpp.udiff.html 508 bool result = IsUnloadingBehaviour::current()->is_unloading(this); This line is essentially the special sauce to this change.? What is hard to see immediately is that this doesn't just test the flag but will do an oops_do in order to set the flag.? Can you add a comment to this effect here? Also, two GC threads could be visiting the same nmethods at the same time with this cleaning.? This is ok though as long as they set the is_unloading state correctly. 500 state._value = RawAccess::load(&_is_unloading_state); Are these required or are you being pedantic?? Shouldn't it be ordered for multiple GC threads? http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/src/hotspot/share/gc/shared/gcBehaviours.new.hpp.html Despite the fact that you spelled "behavoirs" wrong, I think I'm warming up to this use of the concept here.? I'm having trouble thinking of a more specific word for it. I have to confess to not reviewing the epoch handling very well. Since there's a big alignment gap in CompiledMethod, why not just have two fields and skip this odd union?? One bool for is_unloading and one short for unloading_cycle?? Reading the word _inflated is strange here. Thanks, Coleen On 10/19/18 10:22 AM, Erik ?sterlund wrote: > Hi, > > Today the nmethods are unloaded either serially or in parallel due to > GC (and e.g. class unloading). With ZGC concurrent class unloading, a > concurrent fashion is required. Rather than inventing yet a third way > of unloading nmethods due to class unloading, it would be ideal if > there could be one unified way of doing it that makes everyone happy. > > To solve this problem in a more general way, a new member function on > CompiledMethod is added: is_unloading(). It returns whether a > CompiledMethod has broken oops. In order to not have to iterate over > all the oops every time this question is asked, it caches the result, > so it needs to be computed only once per "epoch". Every time a > CodeCache unloading is triggered by the GC, a new epoch is started, > which forces re-computation of whether the CompiledMethod > is_unloading() the first time it is called. > > So by having is_unloading() be lazily evaluated, it is now possible to > build a do_unloading() method on nmethod that simply makes the nmethod > unloaded if it is_unloading(), and otherwise cleans the various > caches. Cleaning e.g. the inline caches of said nmethod, uses the same > is_unloading() method on its targets to figure out if the IC cache > should be cleaned or not. Again, the epoched caching mechanism makes > sure we don't recompute this value. > > The new do_unloading() method may be called in both serial, parallel > and concurrent contexts. So I removed the parallel variation of this > that we had before, that unnecessarily postponed the unloading due to > not having computed whether the nmethod should be unloaded or not. > Since that is now computed on-demand lazily, there is no longer a need > for postponing this, nor to have two phases for parallel nmethod > unloading. > > While there is more work involved in making concurrent nmethod > unloading work, this is a good cleanup towards that goal. > > Bug ID: > https://bugs.openjdk.java.net/browse/JDK-8209189 > > Webrev: > cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ > > Thanks, > /Erik From erik.osterlund at oracle.com Fri Oct 26 16:25:33 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 26 Oct 2018 18:25:33 +0200 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <46f30a22-c2b3-bc18-2334-e84476b0ba64@oracle.com> References: <5BC9E8BA.50407@oracle.com> <46f30a22-c2b3-bc18-2334-e84476b0ba64@oracle.com> Message-ID: <97336d1e-0a9d-0e35-3dc7-d1c41b35d321@oracle.com> Hi Coleen, Thank you for reviewing this. On 2018-10-26 17:58, coleen.phillimore at oracle.com wrote: > > I think this looks really good. I appreciate the cleanup in this > complexity.? It's nice that nmethod/CompiledMethod as non-gc structure > with oops, there's simply an oops_do() vs. all these > do_unloading_scopes() functions.?? It reduces the special handling. I > spent a lot of time trying to figure out how this "postponed" cleaning > works.? This make a lot of sense. I'm glad you like it! I also think the new model is more intuitive. > http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/src/hotspot/share/code/compiledMethod.cpp.udiff.html > > > 508 bool result = IsUnloadingBehaviour::current()->is_unloading(this); > > > This line is essentially the special sauce to this change.? What is hard > to see immediately is that this doesn't just test the flag but will do > an oops_do in order to set the flag.? Can you add a comment to this > effect here? Done. New incremental (including Vladimir's suggestion to remove now dead code): http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00_01/ Full: http://cr.openjdk.java.net/~eosterlund/8209189/webrev.01/ > Also, two GC threads could be visiting the same nmethods at the same > time with this cleaning.? This is ok though as long as they set the > is_unloading state correctly. Exactly. The value will monotonically change, possibly by multiple threads (but in practice mostly by one thread) to the same value. If two threads race, they will race for writing the same value, which is harmless. > 500 state._value = RawAccess::load(&_is_unloading_state); > > > Are these required or are you being pedantic?? Shouldn't it be ordered > for multiple GC threads? No, ordering is irrelevant here. What is relevant is that the access is atomic; that's why I use MO_RELAXED, which gives me those guarantees. As mentioned previously, two threads may race, and that is harmless; they will race for writing the same cached value. There are no ordering constraints. > http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/src/hotspot/share/gc/shared/gcBehaviours.new.hpp.html > > > Despite the fact that you spelled "behavoirs" wrong, I think I'm warming > up to this use of the concept here.? I'm having trouble thinking of a > more specific word for it. Sorry Coleen, but "behaviours" is the correct English spelling. American English is not English. Shots fired... > I have to confess to not reviewing the epoch handling very well. > > Since there's a big alignment gap in CompiledMethod, why not just have > two fields and skip this odd union?? One bool for is_unloading and one > short for unloading_cycle?? Reading the word _inflated is strange here. I use the union to make sure that both values are accessed as a pair atomically. The union allows me to represent the same data either as a single value that can be accessed atomically, and then interpret it as an "inflated" view where it comprises a tuple representing both an epoch and result value. The single value is used to ensure atomicity, and the inflated view is used for interpreting what the state represents. Thanks, /Erik > Thanks, > Coleen > > On 10/19/18 10:22 AM, Erik ?sterlund wrote: >> Hi, >> >> Today the nmethods are unloaded either serially or in parallel due to >> GC (and e.g. class unloading). With ZGC concurrent class unloading, a >> concurrent fashion is required. Rather than inventing yet a third way >> of unloading nmethods due to class unloading, it would be ideal if >> there could be one unified way of doing it that makes everyone happy. >> >> To solve this problem in a more general way, a new member function on >> CompiledMethod is added: is_unloading(). It returns whether a >> CompiledMethod has broken oops. In order to not have to iterate over >> all the oops every time this question is asked, it caches the result, >> so it needs to be computed only once per "epoch". Every time a >> CodeCache unloading is triggered by the GC, a new epoch is started, >> which forces re-computation of whether the CompiledMethod >> is_unloading() the first time it is called. >> >> So by having is_unloading() be lazily evaluated, it is now possible to >> build a do_unloading() method on nmethod that simply makes the nmethod >> unloaded if it is_unloading(), and otherwise cleans the various >> caches. Cleaning e.g. the inline caches of said nmethod, uses the same >> is_unloading() method on its targets to figure out if the IC cache >> should be cleaned or not. Again, the epoched caching mechanism makes >> sure we don't recompute this value. >> >> The new do_unloading() method may be called in both serial, parallel >> and concurrent contexts. So I removed the parallel variation of this >> that we had before, that unnecessarily postponed the unloading due to >> not having computed whether the nmethod should be unloaded or not. >> Since that is now computed on-demand lazily, there is no longer a need >> for postponing this, nor to have two phases for parallel nmethod >> unloading. >> >> While there is more work involved in making concurrent nmethod >> unloading work, this is a good cleanup towards that goal. >> >> Bug ID: >> https://bugs.openjdk.java.net/browse/JDK-8209189 >> >> Webrev: >> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >> >> Thanks, >> /Erik > From dean.long at oracle.com Fri Oct 26 17:24:51 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 26 Oct 2018 10:24:51 -0700 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <5BD32CE3.5070704@oracle.com> References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> <5BD32CE3.5070704@oracle.com> Message-ID: <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> On 10/26/18 8:04 AM, Erik ?sterlund wrote: > Hi Coleen, > > On 2018-10-26 14:09, coleen.phillimore at oracle.com wrote: >> >> Hi Dean,? Thank you for reviewing this. >> >> On 10/26/18 2:17 AM, dean.long at oracle.com wrote: >>> Hi Coleen.? Most of this looks good to me, but I have a few questions: >>> >>> Is the need for Compile_lock around accesses new in 12, because the >>> implementor and subclass links can now be cleaned without being at a >>> safepoint? >> >> Yes.? Erik's going to take Compile_lock to clean these during >> concurrent class unloading.? There were some unlocked accesses though >> that seemed suspicious before though. >>> >>> Style typo: In ciInstanceKlass::compute_has_subklass(), I think it >>> should be return "ik->subklass() != NULL". >> >> Fixed. >>> >>> I see that you removed the memoizing of _has_subklass but left >>> _implementor.? I'm concerned that this might have a performance >>> impact and change behavior.? It seems to allow has_subklass() to >>> change from true to false (and is_leaf_type() to change from false >>> to true) where previously that should have been impossible.? With >>> some fiddling with the locking, I was able to add back >>> _has_subklass.? Do you remember why you removed it? >> >> Yes, I removed the memoizing for _has_subklass because I couldn't >> take the Compile_lock in the ciInstanceKlass constructor, because >> some callers had it.?? If you have a change that will make that work, >> I'd be happy to use yours.? I was trying to not change anything >> semantically, just grab fresh results from InstanceKlass which I >> didn't think would affect performance (although it grabs Compile_lock >> so maybe it could).?? We can revert this part and add locking to >> ciInstanceKlass constructor.? I've been trying to avoid conditional >> locking. > > Can you really safely rely on a cached boolean w.r.t. unloading? The > instant we release the mark end safepoint, compiler threads will come > asking if there are subklasses, before we have started doing > unloading, and they have to get consistent answers that there are no > subklasses, if the one subklass had died. So it seems to me that this > value must be computed and not cached. You can however use the same > trick I used for CompiledMethod::is_unloading in 8212958, and make the > answer belong to an epoch. That way, we force computation for the > first time only that the question is asked, and use the cached result > that we know to be consistent w.r.t. unloading in subsequent calls. > But I wonder if we are micro optimizing by doing that here. > The compiler tasks protect against has_subklass changing from false to true by checking SystemDictionary::_number_of_modifications. Optimizations are done when has_subklass is false, so if it changes from true to false, then we may have missed some optimizations before the change, but we don't have to throw out the generated code.? Do we need to make changes here for concurrent unloading? dl > Thanks, > /Erik > >> Thanks, >> Coleen >>> >>> dl >>> >>> On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >>>> Summary: Don't return unloaded klasses. Make sure access is >>>> protected by Compile_lock. >>>> >>>> See bug for more description.? This test has compiler changes since >>>> the weak klass links in Klass (and implementor in InstanceKlass are >>>> used by the compiler).? All the compiler jtreg tests passed. >>>> >>>> Tested with mach5 tier1-7 and test added. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >>>> >>>> Thanks, >>>> Coleen >>> >> > From coleen.phillimore at oracle.com Fri Oct 26 17:31:17 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 26 Oct 2018 13:31:17 -0400 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <97336d1e-0a9d-0e35-3dc7-d1c41b35d321@oracle.com> References: <5BC9E8BA.50407@oracle.com> <46f30a22-c2b3-bc18-2334-e84476b0ba64@oracle.com> <97336d1e-0a9d-0e35-3dc7-d1c41b35d321@oracle.com> Message-ID: <99ffe9e5-9678-f7ca-882b-1e7a684e0df2@oracle.com> On 10/26/18 12:25 PM, Erik ?sterlund wrote: > Hi Coleen, > > Thank you for reviewing this. > > On 2018-10-26 17:58, coleen.phillimore at oracle.com wrote: >> >> I think this looks really good. I appreciate the cleanup in this >> complexity.? It's nice that nmethod/CompiledMethod as non-gc >> structure with oops, there's simply an oops_do() vs. all these >> do_unloading_scopes() functions.?? It reduces the special handling. I >> spent a lot of time trying to figure out how this "postponed" >> cleaning works.? This make a lot of sense. > > I'm glad you like it! I also think the new model is more intuitive. > >> http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/src/hotspot/share/code/compiledMethod.cpp.udiff.html >> >> >> 508 bool result = IsUnloadingBehaviour::current()->is_unloading(this); >> >> >> This line is essentially the special sauce to this change.? What is >> hard to see immediately is that this doesn't just test the flag but >> will do an oops_do in order to set the flag.? Can you add a comment >> to this effect here? > > Done. > > New incremental (including Vladimir's suggestion to remove now dead > code): > http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00_01/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8209189/webrev.01/ > > >> Also, two GC threads could be visiting the same nmethods at the same >> time with this cleaning.? This is ok though as long as they set the >> is_unloading state correctly. > > Exactly. The value will monotonically change, possibly by multiple > threads (but in practice mostly by one thread) to the same value. If > two threads race, they will race for writing the same value, which is > harmless. > >> 500 state._value = RawAccess::load(&_is_unloading_state); >> >> >> Are these required or are you being pedantic?? Shouldn't it be >> ordered for multiple GC threads? > > No, ordering is irrelevant here. What is relevant is that the access > is atomic; that's why I use MO_RELAXED, which gives me those guarantees. > > As mentioned previously, two threads may race, and that is harmless; > they will race for writing the same cached value. There are no > ordering constraints. So is RawAccess the same as: state._value = _is_unloading_state; ? > >> http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/src/hotspot/share/gc/shared/gcBehaviours.new.hpp.html >> >> >> Despite the fact that you spelled "behavoirs" wrong, I think I'm >> warming up to this use of the concept here.? I'm having trouble >> thinking of a more specific word for it. > > Sorry Coleen, but "behaviours" is the correct English spelling. > American English is not English. Shots fired... :) > >> I have to confess to not reviewing the epoch handling very well. >> >> Since there's a big alignment gap in CompiledMethod, why not just >> have two fields and skip this odd union?? One bool for is_unloading >> and one short for unloading_cycle?? Reading the word _inflated is >> strange here. > > I use the union to make sure that both values are accessed as a pair > atomically. The union allows me to represent the same data either as a > single value that can be accessed atomically, and then interpret it as > an "inflated" view where it comprises a tuple representing both an > epoch and result value. The single value is used to ensure atomicity, > and the inflated view is used for interpreting what the state represents. Can you put some sort of comment above the IsUnloading{Union,Struct} to that effect?? I don't need to see it.? I'm sure your English will be good :) Coleen > > Thanks, > /Erik > >> Thanks, >> Coleen >> >> On 10/19/18 10:22 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> Today the nmethods are unloaded either serially or in parallel due >>> to GC (and e.g. class unloading). With ZGC concurrent class >>> unloading, a concurrent fashion is required. Rather than inventing >>> yet a third way of unloading nmethods due to class unloading, it >>> would be ideal if there could be one unified way of doing it that >>> makes everyone happy. >>> >>> To solve this problem in a more general way, a new member function >>> on CompiledMethod is added: is_unloading(). It returns whether a >>> CompiledMethod has broken oops. In order to not have to iterate over >>> all the oops every time this question is asked, it caches the >>> result, so it needs to be computed only once per "epoch". Every time >>> a CodeCache unloading is triggered by the GC, a new epoch is >>> started, which forces re-computation of whether the CompiledMethod >>> is_unloading() the first time it is called. >>> >>> So by having is_unloading() be lazily evaluated, it is now possible >>> to build a do_unloading() method on nmethod that simply makes the >>> nmethod unloaded if it is_unloading(), and otherwise cleans the >>> various caches. Cleaning e.g. the inline caches of said nmethod, >>> uses the same is_unloading() method on its targets to figure out if >>> the IC cache should be cleaned or not. Again, the epoched caching >>> mechanism makes sure we don't recompute this value. >>> >>> The new do_unloading() method may be called in both serial, parallel >>> and concurrent contexts. So I removed the parallel variation of this >>> that we had before, that unnecessarily postponed the unloading due >>> to not having computed whether the nmethod should be unloaded or >>> not. Since that is now computed on-demand lazily, there is no longer >>> a need for postponing this, nor to have two phases for parallel >>> nmethod unloading. >>> >>> While there is more work involved in making concurrent nmethod >>> unloading work, this is a good cleanup towards that goal. >>> >>> Bug ID: >>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>> >>> Webrev: >>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>> >>> Thanks, >>> /Erik >> From hohensee at amazon.com Fri Oct 26 17:49:31 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 26 Oct 2018 17:49:31 +0000 Subject: JMathConstant submission for upcoming JDK In-Reply-To: References: Message-ID: <45477CFC-5C52-4EE1-9807-CD203A410D25@amazon.com> This would go to core-libs-dev, and would be a spec change since it adds a class. The concept could be expanded to BigInteger, Long, Integer, etc. Instead of adding a class, one could add these constants directly to BigDecimal (and the others), but that would also be a spec change since it adds public members. You should file a JBS bug. If you don't have access to JBS, I'd be happy to file it for you. Thanks, Paul ?On 10/26/18, 9:41 AM, "jdk-dev on behalf of robert at liguorisoftware.com" wrote: Hi Chris, I wrote this simple utility class you may want to use in an upcoming version of the JDK. Btw, I'm the author of the Java Pocket Guide. :) Or is there a better place I can submit simple code like this for the JDK? Thanks, Robert import java.math.BigDecimal; public class JMathConstant { private JMathConstant() { } // ZERO, 1/2 AND ONE public static final BigDecimal ZERO = new BigDecimal("0.0"); public static final BigDecimal LANDAUS = new BigDecimal("0.5"); public static final BigDecimal ONE = new BigDecimal("1.0"); public static final BigDecimal LEGENDRE = new BigDecimal("1.0"); // SILVER AND GOLD public static final BigDecimal ARCHIMEDES_RATIO = new BigDecimal("3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019 27876611 1959092164201989"); public static final BigDecimal UNKNOWN_NAME = new BigDecimal("2.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019 27876611 1959092164201989"); public static final BigDecimal SILVER_RATIO = new BigDecimal("2.4142135623730950488"); public static final BigDecimal PYTHAGORAS = new BigDecimal("1.4142135623730950488"); public static final BigDecimal GOLDEN_RATIO = new BigDecimal("1.61803398874989484820458683436563811772030917980576286213544862270526046281890"); public static final BigDecimal INVERSE_GOLDEN_RATIO = new BigDecimal("0.61803398874989484820458683436563811772030917980576286213544862270526046281890"); // ADDITIONAL RATIO CONSTANTS public static final BigDecimal FEIGENBAUMS_RATIO_1 = new BigDecimal("4.66920160910299067185320382046620161"); public static final BigDecimal FEIGENBAUMS_RATIO_2 = new BigDecimal("2.50290787509589282228390287321821578"); // OTHERS public static final BigDecimal MEISSEL_MERTENS = new BigDecimal("0.26149721284764278375542683860869585"); public static final BigDecimal BERNSTEINS = new BigDecimal("0.28016949902386913303"); public static final BigDecimal GAUSS_KKUZMIN_WIRSING = new BigDecimal("0.30366300289873265859744812190155623"); public static final BigDecimal HAFNER_SARNAK_MCCURLEY = new BigDecimal("0.35323637185499598454351655043268201"); public static final BigDecimal OMEGA = new BigDecimal("0.56714329040978387299996866221035555"); public static final BigDecimal EULER_MASCHERONI = new BigDecimal("0.57721"); public static final BigDecimal GOLOMB_DICKMAN = new BigDecimal("0.6243299885435508709929363831083724"); public static final BigDecimal CAHEN = new BigDecimal("0.6434105463"); public static final BigDecimal TWO_PRIME = new BigDecimal("0.6601618158468695739281211001455577"); public static final BigDecimal LAPLANE_LIMIT = new BigDecimal("0.66274341934918158097474209710925290"); public static final BigDecimal EMBREE_TREFETHEN = new BigDecimal("0.70258"); public static final BigDecimal LANDAU_RAMANUJAN = new BigDecimal("0.76422365358922066299069873125009232"); public static final BigDecimal ALLADI_GRINSTEAD = new BigDecimal("0.8093940205"); public static final BigDecimal BRUN = new BigDecimal("0.8705883800"); public static final BigDecimal CATALANS = new BigDecimal("0.91596559417721901505460351493238411"); public static final BigDecimal LENGYELS = new BigDecimal("1.0986858055"); public static final BigDecimal VISWANATHS = new BigDecimal("1.13198824"); public static final BigDecimal APERYS = new BigDecimal("1.20205690315959428539973816151144999"); public static final BigDecimal GLAISHER_KINKELIN = new BigDecimal("1.2824271291"); public static final BigDecimal CONWAYS = new BigDecimal("1.30357726903429639125709911215255189"); public static final BigDecimal MILLS = new BigDecimal("1.30637788386308069046861449260260571"); public static final BigDecimal PLASTIC = new BigDecimal("1.32471795724474602596090885447809734"); public static final BigDecimal RAMANUJAN_SOLDNER = new BigDecimal("1.45136923488338105028396848589202744"); public static final BigDecimal PORTERS = new BigDecimal("1.4670780794"); public static final BigDecimal BACKHOUSE = new BigDecimal("1.45607494858268967139959535111654356"); public static final BigDecimal LIEBS_SQUARE_ICE = new BigDecimal("1.5396007178"); public static final BigDecimal ERDOS_BORWEIN = new BigDecimal("1.60669515241529176378330152319092458"); public static final BigDecimal NIVENS = new BigDecimal("1.70521114010536776428855145343450816"); public static final BigDecimal THEODORUS = new BigDecimal("1.73205080756887729352744634150587236"); public static final BigDecimal BRUNS = new BigDecimal("1.9021605823"); public static final BigDecimal UNIVERSAL_PARABOLIC = new BigDecimal("2.29558714939263807403429804918949039"); public static final BigDecimal SIERPINSKI = new BigDecimal("2.58498175957925321706589358738317116"); public static final BigDecimal KHINCHIN = new BigDecimal("2.68545200106530644530971483548179569"); public static final BigDecimal NAPIERS = new BigDecimal("2.7182818284590452353602874713527"); public static final BigDecimal FRANSEN_ROBINSON = new BigDecimal("2.80777024202851936522150118655777293"); public static final BigDecimal LEVYS = new BigDecimal("3.27582291872181115978768188245384386"); public static final BigDecimal RECIPROCAL_FIBONACCI = new BigDecimal("3.35988566624317755317201130291892717"); public static final BigDecimal GRAVITY = new BigDecimal("32.17404856"); // ft/sec^2 public static final BigDecimal NEAR_ONE = new BigDecimal("0.9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 99999999 9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999"); public static final BigDecimal DIVIDE_BY_998001 = new BigDecimal("0.0000010020030040050060070080090100110120130140150160170180190200210220230240250260270280290300310320330340350360370380390400410420430440450460470480490500510520530540550560570580590600610620630640650660670680690700710720730740750760770780790800810820830840850860870880890900910920930940950960970980991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243 25326327 328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657 65865966 066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999 09919929 93994995996997999"); // ALTERNATE NAMES public static final BigDecimal PI = ARCHIMEDES_RATIO; public static void main(String[] args) { System.out.println("MEISSEL_MERTENS CONSTANT: " + MEISSEL_MERTENS); } } Robert From coleen.phillimore at oracle.com Fri Oct 26 18:09:18 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 26 Oct 2018 14:09:18 -0400 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> <5BD32CE3.5070704@oracle.com> <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> Message-ID: <1cb5c68e-ec10-f619-adf8-55338871c7eb@oracle.com> On 10/26/18 1:24 PM, dean.long at oracle.com wrote: > On 10/26/18 8:04 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> On 2018-10-26 14:09, coleen.phillimore at oracle.com wrote: >>> >>> Hi Dean,? Thank you for reviewing this. >>> >>> On 10/26/18 2:17 AM, dean.long at oracle.com wrote: >>>> Hi Coleen.? Most of this looks good to me, but I have a few questions: >>>> >>>> Is the need for Compile_lock around accesses new in 12, because the >>>> implementor and subclass links can now be cleaned without being at >>>> a safepoint? >>> >>> Yes.? Erik's going to take Compile_lock to clean these during >>> concurrent class unloading.? There were some unlocked accesses >>> though that seemed suspicious before though. >>>> >>>> Style typo: In ciInstanceKlass::compute_has_subklass(), I think it >>>> should be return "ik->subklass() != NULL". >>> >>> Fixed. >>>> >>>> I see that you removed the memoizing of _has_subklass but left >>>> _implementor.? I'm concerned that this might have a performance >>>> impact and change behavior.? It seems to allow has_subklass() to >>>> change from true to false (and is_leaf_type() to change from false >>>> to true) where previously that should have been impossible.? With >>>> some fiddling with the locking, I was able to add back >>>> _has_subklass.? Do you remember why you removed it? >>> >>> Yes, I removed the memoizing for _has_subklass because I couldn't >>> take the Compile_lock in the ciInstanceKlass constructor, because >>> some callers had it.?? If you have a change that will make that >>> work, I'd be happy to use yours.? I was trying to not change >>> anything semantically, just grab fresh results from InstanceKlass >>> which I didn't think would affect performance (although it grabs >>> Compile_lock so maybe it could).?? We can revert this part and add >>> locking to ciInstanceKlass constructor.? I've been trying to avoid >>> conditional locking. >> >> Can you really safely rely on a cached boolean w.r.t. unloading? The >> instant we release the mark end safepoint, compiler threads will come >> asking if there are subklasses, before we have started doing >> unloading, and they have to get consistent answers that there are no >> subklasses, if the one subklass had died. So it seems to me that this >> value must be computed and not cached. You can however use the same >> trick I used for CompiledMethod::is_unloading in 8212958, and make >> the answer belong to an epoch. That way, we force computation for the >> first time only that the question is asked, and use the cached result >> that we know to be consistent w.r.t. unloading in subsequent calls. >> But I wonder if we are micro optimizing by doing that here. >> > > The compiler tasks protect against has_subklass changing from false to > true by checking SystemDictionary::_number_of_modifications. > Optimizations are done when has_subklass is false, so if it changes > from true to false, then we may have missed some optimizations before > the change, but we don't have to throw out the generated code.? Do we > need to make changes here for concurrent unloading? I don't know if we do.? That sounds like the right sort of behavior to me. Coleen > > dl > >> Thanks, >> /Erik >> >>> Thanks, >>> Coleen >>>> >>>> dl >>>> >>>> On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >>>>> Summary: Don't return unloaded klasses. Make sure access is >>>>> protected by Compile_lock. >>>>> >>>>> See bug for more description.? This test has compiler changes >>>>> since the weak klass links in Klass (and implementor in >>>>> InstanceKlass are used by the compiler). All the compiler jtreg >>>>> tests passed. >>>>> >>>>> Tested with mach5 tier1-7 and test added. >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >>>>> >>>>> Thanks, >>>>> Coleen >>>> >>> >> > From coleen.phillimore at oracle.com Fri Oct 26 18:20:24 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 26 Oct 2018 14:20:24 -0400 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> <5BD32CE3.5070704@oracle.com> <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> Message-ID: Hi, I have tested a version that memoizes _has_subklass in ciInstanceKlass on first use, which doesn't require locking in the ciInstanceKlass constructor. http://cr.openjdk.java.net/~coleenp/8212958.02/webrev/index.html Thanks, Coleen On 10/26/18 1:24 PM, dean.long at oracle.com wrote: > On 10/26/18 8:04 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> On 2018-10-26 14:09, coleen.phillimore at oracle.com wrote: >>> >>> Hi Dean,? Thank you for reviewing this. >>> >>> On 10/26/18 2:17 AM, dean.long at oracle.com wrote: >>>> Hi Coleen.? Most of this looks good to me, but I have a few questions: >>>> >>>> Is the need for Compile_lock around accesses new in 12, because the >>>> implementor and subclass links can now be cleaned without being at >>>> a safepoint? >>> >>> Yes.? Erik's going to take Compile_lock to clean these during >>> concurrent class unloading.? There were some unlocked accesses >>> though that seemed suspicious before though. >>>> >>>> Style typo: In ciInstanceKlass::compute_has_subklass(), I think it >>>> should be return "ik->subklass() != NULL". >>> >>> Fixed. >>>> >>>> I see that you removed the memoizing of _has_subklass but left >>>> _implementor.? I'm concerned that this might have a performance >>>> impact and change behavior.? It seems to allow has_subklass() to >>>> change from true to false (and is_leaf_type() to change from false >>>> to true) where previously that should have been impossible.? With >>>> some fiddling with the locking, I was able to add back >>>> _has_subklass.? Do you remember why you removed it? >>> >>> Yes, I removed the memoizing for _has_subklass because I couldn't >>> take the Compile_lock in the ciInstanceKlass constructor, because >>> some callers had it.?? If you have a change that will make that >>> work, I'd be happy to use yours.? I was trying to not change >>> anything semantically, just grab fresh results from InstanceKlass >>> which I didn't think would affect performance (although it grabs >>> Compile_lock so maybe it could).?? We can revert this part and add >>> locking to ciInstanceKlass constructor.? I've been trying to avoid >>> conditional locking. >> >> Can you really safely rely on a cached boolean w.r.t. unloading? The >> instant we release the mark end safepoint, compiler threads will come >> asking if there are subklasses, before we have started doing >> unloading, and they have to get consistent answers that there are no >> subklasses, if the one subklass had died. So it seems to me that this >> value must be computed and not cached. You can however use the same >> trick I used for CompiledMethod::is_unloading in 8212958, and make >> the answer belong to an epoch. That way, we force computation for the >> first time only that the question is asked, and use the cached result >> that we know to be consistent w.r.t. unloading in subsequent calls. >> But I wonder if we are micro optimizing by doing that here. >> > > The compiler tasks protect against has_subklass changing from false to > true by checking SystemDictionary::_number_of_modifications. > Optimizations are done when has_subklass is false, so if it changes > from true to false, then we may have missed some optimizations before > the change, but we don't have to throw out the generated code.? Do we > need to make changes here for concurrent unloading? > > dl > >> Thanks, >> /Erik >> >>> Thanks, >>> Coleen >>>> >>>> dl >>>> >>>> On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >>>>> Summary: Don't return unloaded klasses. Make sure access is >>>>> protected by Compile_lock. >>>>> >>>>> See bug for more description.? This test has compiler changes >>>>> since the weak klass links in Klass (and implementor in >>>>> InstanceKlass are used by the compiler). All the compiler jtreg >>>>> tests passed. >>>>> >>>>> Tested with mach5 tier1-7 and test added. >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >>>>> >>>>> Thanks, >>>>> Coleen >>>> >>> >> > From erik.osterlund at oracle.com Fri Oct 26 18:51:20 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 26 Oct 2018 20:51:20 +0200 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <99ffe9e5-9678-f7ca-882b-1e7a684e0df2@oracle.com> References: <5BC9E8BA.50407@oracle.com> <46f30a22-c2b3-bc18-2334-e84476b0ba64@oracle.com> <97336d1e-0a9d-0e35-3dc7-d1c41b35d321@oracle.com> <99ffe9e5-9678-f7ca-882b-1e7a684e0df2@oracle.com> Message-ID: <6f30dcc5-6980-12f2-b8e0-97c1b89cc4f6@oracle.com> Hi Coleen, On 2018-10-26 19:31, coleen.phillimore at oracle.com wrote: > > > On 10/26/18 12:25 PM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Thank you for reviewing this. >> >> On 2018-10-26 17:58, coleen.phillimore at oracle.com wrote: >>> >>> I think this looks really good. I appreciate the cleanup in this >>> complexity.? It's nice that nmethod/CompiledMethod as non-gc >>> structure with oops, there's simply an oops_do() vs. all these >>> do_unloading_scopes() functions.?? It reduces the special handling. I >>> spent a lot of time trying to figure out how this "postponed" >>> cleaning works.? This make a lot of sense. >> >> I'm glad you like it! I also think the new model is more intuitive. >> >>> http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/src/hotspot/share/code/compiledMethod.cpp.udiff.html >>> >>> >>> 508 bool result = IsUnloadingBehaviour::current()->is_unloading(this); >>> >>> >>> This line is essentially the special sauce to this change.? What is >>> hard to see immediately is that this doesn't just test the flag but >>> will do an oops_do in order to set the flag.? Can you add a comment >>> to this effect here? >> >> Done. >> >> New incremental (including Vladimir's suggestion to remove now dead >> code): >> http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00_01/ >> >> Full: >> http://cr.openjdk.java.net/~eosterlund/8209189/webrev.01/ >> >> >>> Also, two GC threads could be visiting the same nmethods at the same >>> time with this cleaning.? This is ok though as long as they set the >>> is_unloading state correctly. >> >> Exactly. The value will monotonically change, possibly by multiple >> threads (but in practice mostly by one thread) to the same value. If >> two threads race, they will race for writing the same value, which is >> harmless. >> >>> 500 state._value = RawAccess::load(&_is_unloading_state); >>> >>> >>> Are these required or are you being pedantic?? Shouldn't it be >>> ordered for multiple GC threads? >> >> No, ordering is irrelevant here. What is relevant is that the access >> is atomic; that's why I use MO_RELAXED, which gives me those guarantees. >> >> As mentioned previously, two threads may race, and that is harmless; >> they will race for writing the same cached value. There are no >> ordering constraints. > > So is RawAccess the same as: > > state._value = _is_unloading_state; ? Yes basically. It's equivalent to Atomic::store, which is (currently) equivalent to a normal volatile store as you say. Except I managed to use way more templates to express the same thing in a semantically more satisfying way. And that makes me feel a lot better. I'm not biased... After Kim's C++14 upgrades, it will map to memory_order_relaxed accesses (hence the name MO_RELAXED), as volatile accesses don't really guarantee atomicity according to the C++14 standard. In fact, it will be explicitly undefined behaviour then. >> >>> http://cr.openjdk.java.net/~eosterlund/8209189/webrev.00/src/hotspot/share/gc/shared/gcBehaviours.new.hpp.html >>> >>> >>> Despite the fact that you spelled "behavoirs" wrong, I think I'm >>> warming up to this use of the concept here.? I'm having trouble >>> thinking of a more specific word for it. >> >> Sorry Coleen, but "behaviours" is the correct English spelling. >> American English is not English. Shots fired... > > :) >> >>> I have to confess to not reviewing the epoch handling very well. >>> >>> Since there's a big alignment gap in CompiledMethod, why not just >>> have two fields and skip this odd union?? One bool for is_unloading >>> and one short for unloading_cycle?? Reading the word _inflated is >>> strange here. >> >> I use the union to make sure that both values are accessed as a pair >> atomically. The union allows me to represent the same data either as a >> single value that can be accessed atomically, and then interpret it as >> an "inflated" view where it comprises a tuple representing both an >> epoch and result value. The single value is used to ensure atomicity, >> and the inflated view is used for interpreting what the state represents. > > Can you put some sort of comment above the IsUnloading{Union,Struct} to > that effect?? I don't need to see it.? I'm sure your English will be > good :) Will do! Thanks for the review. /Erik > Coleen > >> >> Thanks, >> /Erik >> >>> Thanks, >>> Coleen >>> >>> On 10/19/18 10:22 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> Today the nmethods are unloaded either serially or in parallel due >>>> to GC (and e.g. class unloading). With ZGC concurrent class >>>> unloading, a concurrent fashion is required. Rather than inventing >>>> yet a third way of unloading nmethods due to class unloading, it >>>> would be ideal if there could be one unified way of doing it that >>>> makes everyone happy. >>>> >>>> To solve this problem in a more general way, a new member function >>>> on CompiledMethod is added: is_unloading(). It returns whether a >>>> CompiledMethod has broken oops. In order to not have to iterate over >>>> all the oops every time this question is asked, it caches the >>>> result, so it needs to be computed only once per "epoch". Every time >>>> a CodeCache unloading is triggered by the GC, a new epoch is >>>> started, which forces re-computation of whether the CompiledMethod >>>> is_unloading() the first time it is called. >>>> >>>> So by having is_unloading() be lazily evaluated, it is now possible >>>> to build a do_unloading() method on nmethod that simply makes the >>>> nmethod unloaded if it is_unloading(), and otherwise cleans the >>>> various caches. Cleaning e.g. the inline caches of said nmethod, >>>> uses the same is_unloading() method on its targets to figure out if >>>> the IC cache should be cleaned or not. Again, the epoched caching >>>> mechanism makes sure we don't recompute this value. >>>> >>>> The new do_unloading() method may be called in both serial, parallel >>>> and concurrent contexts. So I removed the parallel variation of this >>>> that we had before, that unnecessarily postponed the unloading due >>>> to not having computed whether the nmethod should be unloaded or >>>> not. Since that is now computed on-demand lazily, there is no longer >>>> a need for postponing this, nor to have two phases for parallel >>>> nmethod unloading. >>>> >>>> While there is more work involved in making concurrent nmethod >>>> unloading work, this is a good cleanup towards that goal. >>>> >>>> Bug ID: >>>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>>> >>>> Webrev: >>>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>>> >>>> Thanks, >>>> /Erik >>> > From dean.long at oracle.com Fri Oct 26 19:06:50 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 26 Oct 2018 12:06:50 -0700 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> <5BD32CE3.5070704@oracle.com> <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> Message-ID: <2cf8ca3d-75ed-c1c4-4052-35c97982fe0c@oracle.com> Looks good to me. dl On 10/26/18 11:20 AM, coleen.phillimore at oracle.com wrote: > > Hi, I have tested a version that memoizes _has_subklass in > ciInstanceKlass on first use, which doesn't require locking in the > ciInstanceKlass constructor. > > http://cr.openjdk.java.net/~coleenp/8212958.02/webrev/index.html > > Thanks, > Coleen > > > On 10/26/18 1:24 PM, dean.long at oracle.com wrote: >> On 10/26/18 8:04 AM, Erik ?sterlund wrote: >>> Hi Coleen, >>> >>> On 2018-10-26 14:09, coleen.phillimore at oracle.com wrote: >>>> >>>> Hi Dean,? Thank you for reviewing this. >>>> >>>> On 10/26/18 2:17 AM, dean.long at oracle.com wrote: >>>>> Hi Coleen.? Most of this looks good to me, but I have a few >>>>> questions: >>>>> >>>>> Is the need for Compile_lock around accesses new in 12, because >>>>> the implementor and subclass links can now be cleaned without >>>>> being at a safepoint? >>>> >>>> Yes.? Erik's going to take Compile_lock to clean these during >>>> concurrent class unloading.? There were some unlocked accesses >>>> though that seemed suspicious before though. >>>>> >>>>> Style typo: In ciInstanceKlass::compute_has_subklass(), I think it >>>>> should be return "ik->subklass() != NULL". >>>> >>>> Fixed. >>>>> >>>>> I see that you removed the memoizing of _has_subklass but left >>>>> _implementor.? I'm concerned that this might have a performance >>>>> impact and change behavior.? It seems to allow has_subklass() to >>>>> change from true to false (and is_leaf_type() to change from false >>>>> to true) where previously that should have been impossible.? With >>>>> some fiddling with the locking, I was able to add back >>>>> _has_subklass.? Do you remember why you removed it? >>>> >>>> Yes, I removed the memoizing for _has_subklass because I couldn't >>>> take the Compile_lock in the ciInstanceKlass constructor, because >>>> some callers had it.?? If you have a change that will make that >>>> work, I'd be happy to use yours. I was trying to not change >>>> anything semantically, just grab fresh results from InstanceKlass >>>> which I didn't think would affect performance (although it grabs >>>> Compile_lock so maybe it could).?? We can revert this part and add >>>> locking to ciInstanceKlass constructor.? I've been trying to avoid >>>> conditional locking. >>> >>> Can you really safely rely on a cached boolean w.r.t. unloading? The >>> instant we release the mark end safepoint, compiler threads will >>> come asking if there are subklasses, before we have started doing >>> unloading, and they have to get consistent answers that there are no >>> subklasses, if the one subklass had died. So it seems to me that >>> this value must be computed and not cached. You can however use the >>> same trick I used for CompiledMethod::is_unloading in 8212958, and >>> make the answer belong to an epoch. That way, we force computation >>> for the first time only that the question is asked, and use the >>> cached result that we know to be consistent w.r.t. unloading in >>> subsequent calls. But I wonder if we are micro optimizing by doing >>> that here. >>> >> >> The compiler tasks protect against has_subklass changing from false >> to true by checking SystemDictionary::_number_of_modifications. >> Optimizations are done when has_subklass is false, so if it changes >> from true to false, then we may have missed some optimizations before >> the change, but we don't have to throw out the generated code.? Do we >> need to make changes here for concurrent unloading? >> >> dl >> >>> Thanks, >>> /Erik >>> >>>> Thanks, >>>> Coleen >>>>> >>>>> dl >>>>> >>>>> On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Don't return unloaded klasses. Make sure access is >>>>>> protected by Compile_lock. >>>>>> >>>>>> See bug for more description.? This test has compiler changes >>>>>> since the weak klass links in Klass (and implementor in >>>>>> InstanceKlass are used by the compiler). All the compiler jtreg >>>>>> tests passed. >>>>>> >>>>>> Tested with mach5 tier1-7 and test added. >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Fri Oct 26 19:19:34 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 26 Oct 2018 15:19:34 -0400 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <2cf8ca3d-75ed-c1c4-4052-35c97982fe0c@oracle.com> References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> <5BD32CE3.5070704@oracle.com> <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> <2cf8ca3d-75ed-c1c4-4052-35c97982fe0c@oracle.com> Message-ID: Dean, Thank you for the review and comments. Coleen On 10/26/18 3:06 PM, dean.long at oracle.com wrote: > Looks good to me. > > dl > > On 10/26/18 11:20 AM, coleen.phillimore at oracle.com wrote: >> >> Hi, I have tested a version that memoizes _has_subklass in >> ciInstanceKlass on first use, which doesn't require locking in the >> ciInstanceKlass constructor. >> >> http://cr.openjdk.java.net/~coleenp/8212958.02/webrev/index.html >> >> Thanks, >> Coleen >> >> >> On 10/26/18 1:24 PM, dean.long at oracle.com wrote: >>> On 10/26/18 8:04 AM, Erik ?sterlund wrote: >>>> Hi Coleen, >>>> >>>> On 2018-10-26 14:09, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Hi Dean,? Thank you for reviewing this. >>>>> >>>>> On 10/26/18 2:17 AM, dean.long at oracle.com wrote: >>>>>> Hi Coleen.? Most of this looks good to me, but I have a few >>>>>> questions: >>>>>> >>>>>> Is the need for Compile_lock around accesses new in 12, because >>>>>> the implementor and subclass links can now be cleaned without >>>>>> being at a safepoint? >>>>> >>>>> Yes.? Erik's going to take Compile_lock to clean these during >>>>> concurrent class unloading.? There were some unlocked accesses >>>>> though that seemed suspicious before though. >>>>>> >>>>>> Style typo: In ciInstanceKlass::compute_has_subklass(), I think >>>>>> it should be return "ik->subklass() != NULL". >>>>> >>>>> Fixed. >>>>>> >>>>>> I see that you removed the memoizing of _has_subklass but left >>>>>> _implementor.? I'm concerned that this might have a performance >>>>>> impact and change behavior.? It seems to allow has_subklass() to >>>>>> change from true to false (and is_leaf_type() to change from >>>>>> false to true) where previously that should have been >>>>>> impossible.? With some fiddling with the locking, I was able to >>>>>> add back _has_subklass.? Do you remember why you removed it? >>>>> >>>>> Yes, I removed the memoizing for _has_subklass because I couldn't >>>>> take the Compile_lock in the ciInstanceKlass constructor, because >>>>> some callers had it.?? If you have a change that will make that >>>>> work, I'd be happy to use yours. I was trying to not change >>>>> anything semantically, just grab fresh results from InstanceKlass >>>>> which I didn't think would affect performance (although it grabs >>>>> Compile_lock so maybe it could).?? We can revert this part and add >>>>> locking to ciInstanceKlass constructor.? I've been trying to avoid >>>>> conditional locking. >>>> >>>> Can you really safely rely on a cached boolean w.r.t. unloading? >>>> The instant we release the mark end safepoint, compiler threads >>>> will come asking if there are subklasses, before we have started >>>> doing unloading, and they have to get consistent answers that there >>>> are no subklasses, if the one subklass had died. So it seems to me >>>> that this value must be computed and not cached. You can however >>>> use the same trick I used for CompiledMethod::is_unloading in >>>> 8212958, and make the answer belong to an epoch. That way, we force >>>> computation for the first time only that the question is asked, and >>>> use the cached result that we know to be consistent w.r.t. >>>> unloading in subsequent calls. But I wonder if we are micro >>>> optimizing by doing that here. >>>> >>> >>> The compiler tasks protect against has_subklass changing from false >>> to true by checking SystemDictionary::_number_of_modifications. >>> Optimizations are done when has_subklass is false, so if it changes >>> from true to false, then we may have missed some optimizations >>> before the change, but we don't have to throw out the generated >>> code.? Do we need to make changes here for concurrent unloading? >>> >>> dl >>> >>>> Thanks, >>>> /Erik >>>> >>>>> Thanks, >>>>> Coleen >>>>>> >>>>>> dl >>>>>> >>>>>> On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >>>>>>> Summary: Don't return unloaded klasses. Make sure access is >>>>>>> protected by Compile_lock. >>>>>>> >>>>>>> See bug for more description.? This test has compiler changes >>>>>>> since the weak klass links in Klass (and implementor in >>>>>>> InstanceKlass are used by the compiler). All the compiler jtreg >>>>>>> tests passed. >>>>>>> >>>>>>> Tested with mach5 tier1-7 and test added. >>>>>>> >>>>>>> open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>> >>>>> >>>> >>> >> > From kim.barrett at oracle.com Fri Oct 26 20:16:12 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Oct 2018 16:16:12 -0400 Subject: RFR: 8212827: GlobalCounter should support nested critical sections In-Reply-To: <943c0c03-5936-bbd4-afcc-fdeb21000994@oracle.com> References: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> <943c0c03-5936-bbd4-afcc-fdeb21000994@oracle.com> Message-ID: > On Oct 26, 2018, at 4:06 AM, Robbin Ehn wrote: > I would like the return value here: > inline uintx GlobalCounter::critical_section_begin(Thread *thread) { > And thus the parameter here: > 46 inline void GlobalCounter::critical_section_end(Thread *thread, uintx begin_value) { > be an opaque type so we don't leak out the implementation, but as long as the > GlobalCounter have the implementation as name, that doesn't help much. > > I'm thinking of something like: > inline RCUContext RCU::critical_section_begin(Thread *thread) { > inline void RCU::critical_section_end(Thread *thread, RCUContext context) { > Or > inline CSContext CriticalSection::begin(Thread *thread) { > inline void CriticalSection::end(Thread *thread, CSContext context) { Giving this value it's own type name seems like a good idea. I didn't want to go so far as an actual class wrapper though, so just added a typedef for CSContext, even though that's weak. Better would be a C++11 enum class, e.g. enum class CSContext : uintx {}; I've written the code as if it was that (e.g. using static_casts for conversions where needed). I'll file an RFE to change the type, blocked by the C++11/14 JEP. Making that change required adding an #include of globalCounter.hpp to concurrentHashTable.hpp. While there, I added a couple more obviously missing #includes to that file. I didn't verify that what I added was sufficient to #include this file without other preceeding #includes. There were enough changes involved in this that I'm providing new webrevs for your consideration: full: http://cr.openjdk.java.net/~kbarrett/8212827/open.01/ incr: http://cr.openjdk.java.net/~kbarrett/8212827/open.01.inc/ Testing: local (linux-x64) tier1. From kim.barrett at oracle.com Fri Oct 26 20:16:44 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Oct 2018 16:16:44 -0400 Subject: RFR: 8212827: GlobalCounter should support nested critical sections In-Reply-To: <5BD32B30.9040707@oracle.com> References: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> <5BD32B30.9040707@oracle.com> Message-ID: <22F8C483-61FE-41B5-9757-F558D6EC5C21@oracle.com> > On Oct 26, 2018, at 10:56 AM, Erik ?sterlund wrote: > > Hi Kim, > > Looks good. Thanks. Note there are new webrevs in response to Robbin?s suggestion. From kim.barrett at oracle.com Fri Oct 26 20:29:12 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Oct 2018 16:29:12 -0400 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: <3945eb38-b053-a802-666b-7d520822360a@oracle.com> References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> <27ebdbbd-833e-5b78-93b0-83af04834835@oracle.com> <08A02CFE-EB69-409F-9DA5-BA2B82F2B249@oracle.com> <3945eb38-b053-a802-666b-7d520822360a@oracle.com> Message-ID: <5417DFAA-9829-4A21-B4FA-8E313DEFD51B@oracle.com> > On Oct 25, 2018, at 7:00 PM, coleen.phillimore at oracle.com wrote: > On 10/25/18 6:26 PM, Kim Barrett wrote: >>> On Oct 25, 2018, at 6:01 PM, coleen.phillimore at oracle.com wrote: >>> 425 Block* block = block_for_allocation(); >>> 426 if (block == NULL) return NULL; // Block allocation failed. >>> >>> This could be "get_block_for_allocation" because it's not blocking (afaict). Name is somewhat ambiguous. >> Hotspot style guide says getters are noun phrases, with no ?get_" noise word. > > The style guide says that we avoid of noise word get for cases like this: > > Block* block() const { return _block; } > void set_block(Block* b) { _block = b; } > > If the function is doing anything else, you can say get. Also because 'block' is a verb, it makes it confusing. Sorry, but I don't agree. That's a very narrow interpretation of "getter", and suggests the naming convention depends on the underlying implementation. But a primary purpose of providing a function-based API is information hiding; the implementation can be changed without affecting clients. Tying the name to the implementation as suggested is contrary to that purpose. So I think that interpretation is incorrect. As to the potential noun/verb confusion for "block", this function appears in a context where objects of type "Block" are being used, and it returns such an object. It seems to me that ought to be sufficient disambiguation. >>> http://cr.openjdk.java.net/~kbarrett/8210986/open.00/src/hotspot/share/runtime/serviceThread.cpp.frames.html >>> >>> ick. Since this is a static list, can it be declared outside the scope of ServiceThread::entry(), like before the functions that loop through the oopstorages? >>> >>> 109 OopStorage* const oopstorages[] = { >>> 110 JNIHandles::global_handles(), >>> 111 JNIHandles::weak_global_handles(), >>> 112 StringTable::weak_storage(), >>> 113 SystemDictionary::vm_weak_oop_storage() >>> 114 }; >>> 115 const size_t oopstorage_count = ARRAY_SIZE(oopstorages); >>> 116 >>> >>> I think it would bother me less that we have to have all the OopStorages listed somewhere, if it were static scope and not in the service thread function. Are these all the ones we have? >> [?] But I admit I'm not crazy about this array >> being here. It's not an obvious place to look if one were adding >> another OopStorage. > > Oh well, I see why it can't be static. Maybe somewhere better will occur to us when we add more. This looks okay to me. Darn. I was really hoping a better location would come up. From kim.barrett at oracle.com Fri Oct 26 20:32:31 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Oct 2018 16:32:31 -0400 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: <5417DFAA-9829-4A21-B4FA-8E313DEFD51B@oracle.com> References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> <27ebdbbd-833e-5b78-93b0-83af04834835@oracle.com> <08A02CFE-EB69-409F-9DA5-BA2B82F2B249@oracle.com> <3945eb38-b053-a802-666b-7d520822360a@oracle.com> <5417DFAA-9829-4A21-B4FA-8E313DEFD51B@oracle.com> Message-ID: > On Oct 26, 2018, at 4:29 PM, Kim Barrett wrote: > >> On Oct 25, 2018, at 7:00 PM, coleen.phillimore at oracle.com wrote: >> On 10/25/18 6:26 PM, Kim Barrett wrote: >>>> On Oct 25, 2018, at 6:01 PM, coleen.phillimore at oracle.com wrote: >>>> 425 Block* block = block_for_allocation(); >>>> 426 if (block == NULL) return NULL; // Block allocation failed. >>>> >>>> This could be "get_block_for_allocation" because it's not blocking (afaict). Name is somewhat ambiguous. >>> Hotspot style guide says getters are noun phrases, with no ?get_" noise word. >> >> The style guide says that we avoid of noise word get for cases like this: >> >> Block* block() const { return _block; } >> void set_block(Block* b) { _block = b; } >> >> If the function is doing anything else, you can say get. Also because 'block' is a verb, it makes it confusing. > > Sorry, but I don't agree. > > That's a very narrow interpretation of "getter", and suggests the > naming convention depends on the underlying implementation. But a > primary purpose of providing a function-based API is information > hiding; the implementation can be changed without affecting clients. > Tying the name to the implementation as suggested is contrary to that > purpose. So I think that interpretation is incorrect. Or if that *is* the ?original intent? of tat coding guideline, then I think the guideline is wrong. From igor.ignatyev at oracle.com Fri Oct 26 23:23:32 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 26 Oct 2018 16:23:32 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> Message-ID: <9805F73F-2631-455A-8C44-3A2A2E4A7C51@oracle.com> ping... still looking for a Reviewer -- Igor > On Oct 10, 2018, at 11:33 AM, Igor Ignatyev wrote: > > Hi David, > > thanks for spotting the typo, I've fixed the commit message. > still looking for a Reviewer though. > > Cheers, > -- Igor > >> On Oct 9, 2018, at 6:49 PM, David Holmes wrote: >> >> Hi Igor, >> >> Not a review - I fixed the typo in the bug synopsis: Covert -> Convert :) >> >> Please ensure you commit with corrected synopsis. >> >> Thanks, >> David >> >> On 10/10/2018 3:23 AM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >>>> 450 lines changed: 238 ins; 211 del; 1 mod; >>> Hi all, >>> could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? >>> webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 >>> testing: >>> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >>> - build w/ precompiled-headers enabled and disabled >>> PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review >>> Thanks, >>> -- Igor >>> > From kim.barrett at oracle.com Sat Oct 27 00:18:23 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Oct 2018 20:18:23 -0400 Subject: RFR: 6735527: Bitmap - speed up searches Message-ID: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> Please review this improvement to the performance of BitMap searches. The functions BitMap::get_next_one_offset and friends now use count_trailing_zeros to find the position of a 1 bit in a word, rather than using a loop. That set of functions is also now implemented using a parameterized helper, eliminating a bunch of code (near) duplication. There is a new gtest-based "microbenchmark" included in the change: test_bitMap_search_perf.cpp. It fills a 64KB bitmap to some specified "density", and then counts the bits in the bitmap by iteration over get_next_one_offset. Repeat for various densities. With the webrev is an OpenDocument spreadsheet comparing running the microbenchmark before and after the change. Comparisons were made on a Xeon E5-2630 @ 2.6GHz running Ubuntu 16.04, code compiled with gcc7.3.0. It shows that across the entire range of densities the new code is no worse than the old, and for some ranges is significantly better. It's nice that the ranges with improvements are likely relevant to GC. Question for reviewers: Should the microbenchmark be checked in? It doesn't really test anything, just executes some code and prints some timing information. CR: https://bugs.openjdk.java.net/browse/JDK-6735527 Webrev: http://cr.openjdk.java.net/~kbarrett/6735527/open.00/ http://cr.openjdk.java.net/~kbarrett/6735527/comparison.ods Testing: mach5 tier1-5 New gtest-based "microbenchmark". From kim.barrett at oracle.com Sat Oct 27 02:52:21 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Oct 2018 22:52:21 -0400 Subject: RFR: 6735527: Bitmap - speed up searches In-Reply-To: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> References: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> Message-ID: <4C6F8E9D-6BA7-4DBF-BF1B-87FEB390BEBC@oracle.com> > On Oct 26, 2018, at 8:18 PM, Kim Barrett wrote: > > Please review this improvement to the performance of BitMap searches. > > [?] > CR: > https://bugs.openjdk.java.net/browse/JDK-6735527 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/6735527/open.00/ > http://cr.openjdk.java.net/~kbarrett/6735527/comparison.ods It seems that .ods files can't be downloaded from cr.openjdk.java.net. I've added a zip of the file, which does seem to be accessible. http://cr.openjdk.java.net/~kbarrett/6735527/comparison.ods.zip > Testing: > mach5 tier1-5 > New gtest-based "microbenchmark". From robbin.ehn at oracle.com Sun Oct 28 19:41:38 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Sun, 28 Oct 2018 20:41:38 +0100 Subject: RFR: 8212827: GlobalCounter should support nested critical sections In-Reply-To: References: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> <943c0c03-5936-bbd4-afcc-fdeb21000994@oracle.com> Message-ID: <501a9d5e-2635-3873-fb64-92234db9878e@oracle.com> Hi Kim, Looks good, thanks! /Robbin On 26/10/2018 22:16, Kim Barrett wrote: >> On Oct 26, 2018, at 4:06 AM, Robbin Ehn wrote: >> I would like the return value here: >> inline uintx GlobalCounter::critical_section_begin(Thread *thread) { >> And thus the parameter here: >> 46 inline void GlobalCounter::critical_section_end(Thread *thread, uintx begin_value) { >> be an opaque type so we don't leak out the implementation, but as long as the >> GlobalCounter have the implementation as name, that doesn't help much. >> >> I'm thinking of something like: >> inline RCUContext RCU::critical_section_begin(Thread *thread) { >> inline void RCU::critical_section_end(Thread *thread, RCUContext context) { >> Or >> inline CSContext CriticalSection::begin(Thread *thread) { >> inline void CriticalSection::end(Thread *thread, CSContext context) { > > Giving this value it's own type name seems like a good idea. I didn't > want to go so far as an actual class wrapper though, so just added a > typedef for CSContext, even though that's weak. Better would be a > C++11 enum class, e.g. > enum class CSContext : uintx {}; > I've written the code as if it was that (e.g. using static_casts for > conversions where needed). I'll file an RFE to change the type, > blocked by the C++11/14 JEP. > > Making that change required adding an #include of globalCounter.hpp to > concurrentHashTable.hpp. While there, I added a couple more obviously > missing #includes to that file. I didn't verify that what I added was > sufficient to #include this file without other preceeding #includes. > > There were enough changes involved in this that I'm providing new > webrevs for your consideration: > full: http://cr.openjdk.java.net/~kbarrett/8212827/open.01/ > incr: http://cr.openjdk.java.net/~kbarrett/8212827/open.01.inc/ > > Testing: local (linux-x64) tier1. > From thomas.schatzl at oracle.com Mon Oct 29 08:05:59 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 29 Oct 2018 09:05:59 +0100 Subject: RFR: 8212827: GlobalCounter should support nested critical sections In-Reply-To: References: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> <943c0c03-5936-bbd4-afcc-fdeb21000994@oracle.com> Message-ID: <862aa8574e353bf79ec58efa061738df1c214e12.camel@oracle.com> Hi, On Fri, 2018-10-26 at 16:16 -0400, Kim Barrett wrote: > > On Oct 26, 2018, at 4:06 AM, Robbin Ehn > > wrote: > > I would like the return value here: > > [...] > There were enough changes involved in this that I'm providing new > webrevs for your consideration: > full: http://cr.openjdk.java.net/~kbarrett/8212827/open.01/ > incr: http://cr.openjdk.java.net/~kbarrett/8212827/open.01.inc/ > > Testing: local (linux-x64) tier1. > looks good to me. Thomas From thomas.schatzl at oracle.com Mon Oct 29 08:58:49 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 29 Oct 2018 09:58:49 +0100 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> Message-ID: Hi Igor, On Wed, 2018-10-10 at 11:33 -0700, Igor Ignatyev wrote: > Hi David, > > thanks for spotting the typo, I've fixed the commit message. > still looking for a Reviewer though. looks good. Thomas From erik.osterlund at oracle.com Mon Oct 29 14:46:39 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 29 Oct 2018 15:46:39 +0100 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> <5BD32CE3.5070704@oracle.com> <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> Message-ID: <5BD71D4F.8060900@oracle.com> Hi Coleen, Looks good. Thanks, /Erik On 2018-10-26 20:20, coleen.phillimore at oracle.com wrote: > > Hi, I have tested a version that memoizes _has_subklass in > ciInstanceKlass on first use, which doesn't require locking in the > ciInstanceKlass constructor. > > http://cr.openjdk.java.net/~coleenp/8212958.02/webrev/index.html > > Thanks, > Coleen > > > On 10/26/18 1:24 PM, dean.long at oracle.com wrote: >> On 10/26/18 8:04 AM, Erik ?sterlund wrote: >>> Hi Coleen, >>> >>> On 2018-10-26 14:09, coleen.phillimore at oracle.com wrote: >>>> >>>> Hi Dean, Thank you for reviewing this. >>>> >>>> On 10/26/18 2:17 AM, dean.long at oracle.com wrote: >>>>> Hi Coleen. Most of this looks good to me, but I have a few >>>>> questions: >>>>> >>>>> Is the need for Compile_lock around accesses new in 12, because >>>>> the implementor and subclass links can now be cleaned without >>>>> being at a safepoint? >>>> >>>> Yes. Erik's going to take Compile_lock to clean these during >>>> concurrent class unloading. There were some unlocked accesses >>>> though that seemed suspicious before though. >>>>> >>>>> Style typo: In ciInstanceKlass::compute_has_subklass(), I think it >>>>> should be return "ik->subklass() != NULL". >>>> >>>> Fixed. >>>>> >>>>> I see that you removed the memoizing of _has_subklass but left >>>>> _implementor. I'm concerned that this might have a performance >>>>> impact and change behavior. It seems to allow has_subklass() to >>>>> change from true to false (and is_leaf_type() to change from false >>>>> to true) where previously that should have been impossible. With >>>>> some fiddling with the locking, I was able to add back >>>>> _has_subklass. Do you remember why you removed it? >>>> >>>> Yes, I removed the memoizing for _has_subklass because I couldn't >>>> take the Compile_lock in the ciInstanceKlass constructor, because >>>> some callers had it. If you have a change that will make that >>>> work, I'd be happy to use yours. I was trying to not change >>>> anything semantically, just grab fresh results from InstanceKlass >>>> which I didn't think would affect performance (although it grabs >>>> Compile_lock so maybe it could). We can revert this part and add >>>> locking to ciInstanceKlass constructor. I've been trying to avoid >>>> conditional locking. >>> >>> Can you really safely rely on a cached boolean w.r.t. unloading? The >>> instant we release the mark end safepoint, compiler threads will >>> come asking if there are subklasses, before we have started doing >>> unloading, and they have to get consistent answers that there are no >>> subklasses, if the one subklass had died. So it seems to me that >>> this value must be computed and not cached. You can however use the >>> same trick I used for CompiledMethod::is_unloading in 8212958, and >>> make the answer belong to an epoch. That way, we force computation >>> for the first time only that the question is asked, and use the >>> cached result that we know to be consistent w.r.t. unloading in >>> subsequent calls. But I wonder if we are micro optimizing by doing >>> that here. >>> >> >> The compiler tasks protect against has_subklass changing from false >> to true by checking SystemDictionary::_number_of_modifications. >> Optimizations are done when has_subklass is false, so if it changes >> from true to false, then we may have missed some optimizations before >> the change, but we don't have to throw out the generated code. Do we >> need to make changes here for concurrent unloading? >> >> dl >> >>> Thanks, >>> /Erik >>> >>>> Thanks, >>>> Coleen >>>>> >>>>> dl >>>>> >>>>> On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Don't return unloaded klasses. Make sure access is >>>>>> protected by Compile_lock. >>>>>> >>>>>> See bug for more description. This test has compiler changes >>>>>> since the weak klass links in Klass (and implementor in >>>>>> InstanceKlass are used by the compiler). All the compiler jtreg >>>>>> tests passed. >>>>>> >>>>>> Tested with mach5 tier1-7 and test added. >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Mon Oct 29 14:49:20 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 29 Oct 2018 10:49:20 -0400 Subject: RFR 8212958: Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <5BD71D4F.8060900@oracle.com> References: <907c9862-eb74-dbbb-5ab8-7561e47a3bd3@oracle.com> <04c6aae2-3c08-0b0d-f4ab-9410b47b4a17@oracle.com> <5BD32CE3.5070704@oracle.com> <220ffc53-a66d-cd90-bc6f-beceb7b1b480@oracle.com> <5BD71D4F.8060900@oracle.com> Message-ID: <3267aefe-c3f8-7627-49e5-d58f5ce184fc@oracle.com> Thanks Erik. Coleen On 10/29/18 10:46 AM, Erik ?sterlund wrote: > Hi Coleen, > > Looks good. > > Thanks, > /Erik > > On 2018-10-26 20:20, coleen.phillimore at oracle.com wrote: >> >> Hi, I have tested a version that memoizes _has_subklass in >> ciInstanceKlass on first use, which doesn't require locking in the >> ciInstanceKlass constructor. >> >> http://cr.openjdk.java.net/~coleenp/8212958.02/webrev/index.html >> >> Thanks, >> Coleen >> >> >> On 10/26/18 1:24 PM, dean.long at oracle.com wrote: >>> On 10/26/18 8:04 AM, Erik ?sterlund wrote: >>>> Hi Coleen, >>>> >>>> On 2018-10-26 14:09, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Hi Dean,? Thank you for reviewing this. >>>>> >>>>> On 10/26/18 2:17 AM, dean.long at oracle.com wrote: >>>>>> Hi Coleen.? Most of this looks good to me, but I have a few >>>>>> questions: >>>>>> >>>>>> Is the need for Compile_lock around accesses new in 12, because >>>>>> the implementor and subclass links can now be cleaned without >>>>>> being at a safepoint? >>>>> >>>>> Yes.? Erik's going to take Compile_lock to clean these during >>>>> concurrent class unloading.? There were some unlocked accesses >>>>> though that seemed suspicious before though. >>>>>> >>>>>> Style typo: In ciInstanceKlass::compute_has_subklass(), I think >>>>>> it should be return "ik->subklass() != NULL". >>>>> >>>>> Fixed. >>>>>> >>>>>> I see that you removed the memoizing of _has_subklass but left >>>>>> _implementor.? I'm concerned that this might have a performance >>>>>> impact and change behavior.? It seems to allow has_subklass() to >>>>>> change from true to false (and is_leaf_type() to change from >>>>>> false to true) where previously that should have been >>>>>> impossible.? With some fiddling with the locking, I was able to >>>>>> add back _has_subklass.? Do you remember why you removed it? >>>>> >>>>> Yes, I removed the memoizing for _has_subklass because I couldn't >>>>> take the Compile_lock in the ciInstanceKlass constructor, because >>>>> some callers had it.?? If you have a change that will make that >>>>> work, I'd be happy to use yours. I was trying to not change >>>>> anything semantically, just grab fresh results from InstanceKlass >>>>> which I didn't think would affect performance (although it grabs >>>>> Compile_lock so maybe it could).?? We can revert this part and add >>>>> locking to ciInstanceKlass constructor.? I've been trying to avoid >>>>> conditional locking. >>>> >>>> Can you really safely rely on a cached boolean w.r.t. unloading? >>>> The instant we release the mark end safepoint, compiler threads >>>> will come asking if there are subklasses, before we have started >>>> doing unloading, and they have to get consistent answers that there >>>> are no subklasses, if the one subklass had died. So it seems to me >>>> that this value must be computed and not cached. You can however >>>> use the same trick I used for CompiledMethod::is_unloading in >>>> 8212958, and make the answer belong to an epoch. That way, we force >>>> computation for the first time only that the question is asked, and >>>> use the cached result that we know to be consistent w.r.t. >>>> unloading in subsequent calls. But I wonder if we are micro >>>> optimizing by doing that here. >>>> >>> >>> The compiler tasks protect against has_subklass changing from false >>> to true by checking SystemDictionary::_number_of_modifications. >>> Optimizations are done when has_subklass is false, so if it changes >>> from true to false, then we may have missed some optimizations >>> before the change, but we don't have to throw out the generated >>> code.? Do we need to make changes here for concurrent unloading? >>> >>> dl >>> >>>> Thanks, >>>> /Erik >>>> >>>>> Thanks, >>>>> Coleen >>>>>> >>>>>> dl >>>>>> >>>>>> On 10/25/18 1:42 PM, coleen.phillimore at oracle.com wrote: >>>>>>> Summary: Don't return unloaded klasses. Make sure access is >>>>>>> protected by Compile_lock. >>>>>>> >>>>>>> See bug for more description.? This test has compiler changes >>>>>>> since the weak klass links in Klass (and implementor in >>>>>>> InstanceKlass are used by the compiler). All the compiler jtreg >>>>>>> tests passed. >>>>>>> >>>>>>> Tested with mach5 tier1-7 and test added. >>>>>>> >>>>>>> open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/8212958.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8212958 >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>> >>>>> >>>> >>> >> > From kim.barrett at oracle.com Mon Oct 29 17:54:13 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 29 Oct 2018 13:54:13 -0400 Subject: RFR: 8212827: GlobalCounter should support nested critical sections In-Reply-To: <501a9d5e-2635-3873-fb64-92234db9878e@oracle.com> References: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> <943c0c03-5936-bbd4-afcc-fdeb21000994@oracle.com> <501a9d5e-2635-3873-fb64-92234db9878e@oracle.com> Message-ID: <0A89D526-3C18-463D-AF7C-2F1FA175D695@oracle.com> > On Oct 28, 2018, at 3:41 PM, Robbin Ehn wrote: > > Hi Kim, > > Looks good, thanks! Thanks. > > /Robbin > > On 26/10/2018 22:16, Kim Barrett wrote: >>> On Oct 26, 2018, at 4:06 AM, Robbin Ehn wrote: >>> I would like the return value here: >>> inline uintx GlobalCounter::critical_section_begin(Thread *thread) { >>> And thus the parameter here: >>> 46 inline void GlobalCounter::critical_section_end(Thread *thread, uintx begin_value) { >>> be an opaque type so we don't leak out the implementation, but as long as the >>> GlobalCounter have the implementation as name, that doesn't help much. >>> >>> I'm thinking of something like: >>> inline RCUContext RCU::critical_section_begin(Thread *thread) { >>> inline void RCU::critical_section_end(Thread *thread, RCUContext context) { >>> Or >>> inline CSContext CriticalSection::begin(Thread *thread) { >>> inline void CriticalSection::end(Thread *thread, CSContext context) { >> Giving this value it's own type name seems like a good idea. I didn't >> want to go so far as an actual class wrapper though, so just added a >> typedef for CSContext, even though that's weak. Better would be a >> C++11 enum class, e.g. >> enum class CSContext : uintx {}; >> I've written the code as if it was that (e.g. using static_casts for >> conversions where needed). I'll file an RFE to change the type, >> blocked by the C++11/14 JEP. >> Making that change required adding an #include of globalCounter.hpp to >> concurrentHashTable.hpp. While there, I added a couple more obviously >> missing #includes to that file. I didn't verify that what I added was >> sufficient to #include this file without other preceeding #includes. >> There were enough changes involved in this that I'm providing new >> webrevs for your consideration: >> full: http://cr.openjdk.java.net/~kbarrett/8212827/open.01/ >> incr: http://cr.openjdk.java.net/~kbarrett/8212827/open.01.inc/ >> Testing: local (linux-x64) tier1. From kim.barrett at oracle.com Mon Oct 29 17:54:22 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 29 Oct 2018 13:54:22 -0400 Subject: RFR: 8212827: GlobalCounter should support nested critical sections In-Reply-To: <862aa8574e353bf79ec58efa061738df1c214e12.camel@oracle.com> References: <5E186910-B7B5-4C3F-BE88-54A519A9912B@oracle.com> <943c0c03-5936-bbd4-afcc-fdeb21000994@oracle.com> <862aa8574e353bf79ec58efa061738df1c214e12.camel@oracle.com> Message-ID: > On Oct 29, 2018, at 4:05 AM, Thomas Schatzl wrote: > > Hi, > > On Fri, 2018-10-26 at 16:16 -0400, Kim Barrett wrote: >>> On Oct 26, 2018, at 4:06 AM, Robbin Ehn >>> wrote: >>> I would like the return value here: >>> > [...] >> There were enough changes involved in this that I'm providing new >> webrevs for your consideration: >> full: http://cr.openjdk.java.net/~kbarrett/8212827/open.01/ >> incr: http://cr.openjdk.java.net/~kbarrett/8212827/open.01.inc/ >> >> Testing: local (linux-x64) tier1. >> > > looks good to me. > > Thomas Thanks. From lois.foltan at oracle.com Mon Oct 29 19:47:42 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 29 Oct 2018 15:47:42 -0400 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: On 10/25/2018 11:15 AM, Martin Balao wrote: > Hi Lois, > > Thanks for having a look at this. > > Yes, I have a test that triggers this execution path. Unfortunately, > this is a security test that I cannot publicly share. What I can tell > you, though, is that the parent class loader is a "delegating class > loader" which never defined a class and that's the reason why it > doesn't have a ClassLoaderData instance created. > > I'm not sure if this would be needed in other places where the idiom > is not used. The reason is that there might be any other implicit > assumptions that makes it impossible. That's why I limited the scope > of this changeset to the only case that I'm sure it's possible to trigger. Hi Martin, I wrote a test case that reproduces this failure and demonstrates that invoking SystemDictionary::register_loader() will not work. With a non-product (fastdebug) build, the test case shows that calling register_loader is problematic, a SystemDictionary lock is currently held and acquiring a ClassLoaderDataGraph lock to create the new ClassLoaderData causes an out of order locking fatal error. Please consider this test case and proposed change: ??? http://cr.openjdk.java.net/~lfoltan/bug_jdk8212937/webrev/ Thanks, Lois > > Kind regards, > Martin.- > > On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan > wrote: > > On 10/24/2018 3:58 PM, Martin Balao wrote: > > Hi, > > Can I have a review for JDK-8212937 [1]? > > Webrev.00: > > ? * > http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ > > ? * > http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00.zip > > > By obtaining the ClassLoaderData instance through a call to > SystemDictionary::register_loader, we make sure that either an > existing > instance is obtained or a new one created. This same idiom has > already been > used in other places where there are no guarantees that a previous > ClassLoaderData instance exists for a given class loader. See > futher > information in JDK-8212937 ticket [1]. > > Thanks, > Martin.- > > -- > [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 > > > Hi Martin, > > I agree that the same idiom is used in other places, but there are > also a couple of occurrences of java_lang_ClassLoader::parent in > classfile/classLoaderStats.cpp and > classfile/classLoaderHierarchyDCmd.cpp that do not.? In order to > better understand, do you have a test case that demonstrates this > issue? > > Thanks, > Lois > > From Michael.Rasmussen at roguewave.com Tue Oct 30 08:45:30 2018 From: Michael.Rasmussen at roguewave.com (Michael Rasmussen) Date: Tue, 30 Oct 2018 08:45:30 +0000 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> <5493aca2-f67b-4e9f-1f10-02511d701eca@oracle.com> <4e474a5e-9d65-ca27-94ac-e04a5aafbb0f@oracle.com>, Message-ID: From: Ioi Lam > I sent RFR to > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030612.html > http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-October/025640.html Hi I saw that a fix was pushed to jdk/jdk now. Any plans of back-porting it to jdk11u? /Michael From thomas.schatzl at oracle.com Tue Oct 30 14:29:59 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 30 Oct 2018 15:29:59 +0100 Subject: RFR: 6735527: Bitmap - speed up searches In-Reply-To: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> References: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> Message-ID: Hi, On Fri, 2018-10-26 at 20:18 -0400, Kim Barrett wrote: > Please review this improvement to the performance of BitMap searches. > > The functions BitMap::get_next_one_offset and friends now use > count_trailing_zeros to find the position of a 1 bit in a word, > rather > than using a loop. > > That set of functions is also now implemented using a parameterized > helper, eliminating a bunch of code (near) duplication. > > There is a new gtest-based "microbenchmark" included in the change: > test_bitMap_search_perf.cpp. It fills a 64KB bitmap to some specified > "density", and then counts the bits in the bitmap by iteration over > get_next_one_offset. Repeat for various densities. > > With the webrev is an OpenDocument spreadsheet comparing running the > microbenchmark before and after the change. Comparisons were made on > a Xeon E5-2630 @ 2.6GHz running Ubuntu 16.04, code compiled with > gcc7.3.0. It shows that across the entire range of densities the new > code is no worse than the old, and for some ranges is significantly > better. It's nice that the ranges with improvements are likely > relevant to GC. > > Question for reviewers: Should the microbenchmark be checked in? It > doesn't really test anything, just executes some code and prints some > timing information. > > CR: > https://bugs.openjdk.java.net/browse/JDK-6735527 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/6735527/open.00/ > http://cr.openjdk.java.net/~kbarrett/6735527/comparison.ods the change looks good to me. The microbenchmark unfortunately does not seem to not fit anywhere in the rest of the repo at this time. Maybe somebody else has an idea where to put it. Thanks, Thomas From bob.vandette at oracle.com Tue Oct 30 14:56:36 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 30 Oct 2018 10:56:36 -0400 Subject: RFR: 8209093 - JEP 340: One AArch64 Port, Not Two In-Reply-To: References: <285AC12C-3A77-41FB-A756-ED15354B61FE@oracle.com> <6C784061-6256-4D40-BFB3-CA1ABB7F29EB@oracle.com> <5BA38920.8020203@cjnash.com> Message-ID: <43E51870-3583-42A1-B1A3-CFD86CF975A0@oracle.com> This JEP is now targeted to JDK 12. I?ve merged up to the latest sources and will be integrating these changes in the next day or two. Here?s the latest webrev post merge in the event anyone spots any last minute issues. http://cr.openjdk.java.net/~bobv/8209093/webrev.02 Bob,. From sgehwolf at redhat.com Tue Oct 30 15:01:47 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 30 Oct 2018 16:01:47 +0100 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: Hi Lois, On Mon, 2018-10-29 at 15:47 -0400, Lois Foltan wrote: > On 10/25/2018 11:15 AM, Martin Balao wrote: > > > Hi Lois, > > > > Thanks for having a look at this. > > > > Yes, I have a test that triggers this execution path. Unfortunately, > > this is a security test that I cannot publicly share. What I can tell > > you, though, is that the parent class loader is a "delegating class > > loader" which never defined a class and that's the reason why it > > doesn't have a ClassLoaderData instance created. > > > > I'm not sure if this would be needed in other places where the idiom > > is not used. The reason is that there might be any other implicit > > assumptions that makes it impossible. That's why I limited the scope > > of this changeset to the only case that I'm sure it's possible to trigger. > > Hi Martin, > > I wrote a test case that reproduces this failure and demonstrates that > invoking SystemDictionary::register_loader() will not work. With a > non-product (fastdebug) build, the test case shows that calling > register_loader is problematic, a SystemDictionary lock is currently > held and acquiring a ClassLoaderDataGraph lock to create the new > ClassLoaderData causes an out of order locking fatal error. > > Please consider this test case and proposed change: > http://cr.openjdk.java.net/~lfoltan/bug_jdk8212937/webrev/ FWIW, I've tested this webrev with our reproducer and this fix works for us. Thanks, Severin > > Kind regards, > > Martin.- > > > > On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan > > wrote: > > > > On 10/24/2018 3:58 PM, Martin Balao wrote: > > > > Hi, > > > > Can I have a review for JDK-8212937 [1]? > > > > Webrev.00: > > > > * > > http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ > > > > * > > http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00.zip > > > > > > By obtaining the ClassLoaderData instance through a call to > > SystemDictionary::register_loader, we make sure that either an > > existing > > instance is obtained or a new one created. This same idiom has > > already been > > used in other places where there are no guarantees that a previous > > ClassLoaderData instance exists for a given class loader. See > > futher > > information in JDK-8212937 ticket [1]. > > > > Thanks, > > Martin.- > > > > -- > > [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 > > > > > > Hi Martin, > > > > I agree that the same idiom is used in other places, but there are > > also a couple of occurrences of java_lang_ClassLoader::parent in > > classfile/classLoaderStats.cpp and > > classfile/classLoaderHierarchyDCmd.cpp that do not. In order to > > better understand, do you have a test case that demonstrates this > > issue? > > > > Thanks, > > Lois > > > > > > From volker.simonis at gmail.com Tue Oct 30 17:55:08 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 30 Oct 2018 18:55:08 +0100 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> <271b6743-1611-4201-cdf6-9fdc5b87086e@linux.vnet.ibm.com> <0c176a41bd274bf795382f26e476b59a@sap.com> Message-ID: Hi Gustavo, thanks for adding Power9 support. Your change looks good! Just two minor comments (no need to publish a new webrev): - can you please update the copyright year to 2018 in assembler_ppc.inline.hpp and vm_version_ppc.hpp - in assembler_ppc.inline.hpp can you pleas add a comment with the default value as shown below? I often find that useful when browsing the code. +// Deliver A Random Number (introduced with POWER9) +inline void Assembler::darn(Register d, int l /* = 1 */) { emit_int32( DARN_OPCODE | rt(d) | l14(l)); } + Thank you and best regards, Volker On Wed, Oct 24, 2018 at 3:35 PM Gustavo Romero wrote: > > Hi Martin, > > On 10/24/2018 05:20 AM, Doerr, Martin wrote: > > looks good. Thank you for cleaning things up and also for sharing the story about mftgpr. Interesting. > > Reviewed. > > Thanks for reviewing! > > > Best regards, > Gustavo > > > Best regards, > > Martin > > > > > > -----Original Message----- > > From: Gustavo Romero > > Sent: Dienstag, 23. Oktober 2018 23:46 > > To: Doerr, Martin ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > > Subject: Re: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection > > > > Hi Martin, > > > > Thanks for your comments on fsqrt{,s} and mftgpr opcodes. > > > > On 10/23/2018 01:51 PM, Doerr, Martin wrote: > >> Hi Gustavo, > >> > >>> I agree. So, if you don't mind, to reduce any future rework on that > >>> I removed the hardcoded L=1 and now it's just a default. I introduced > >>> l14() function that matches two bits in the proper field as I don't > >>> see any collision with any other one bit field at the same position > >>> (bit 14). > >> I like it, but the default value needs to get specified in the .hpp file. > > > > Fixed. > > > > > >>> I just have one question regarding the feature-string. I see > >>> instrumentation for "fsqrts" feature but it's not currently > >>> advertised by the feature-string. On the other hand, ISA info > >>> looks to indicate that fsqrt and fsqrts are not aliases, because: > >>> > >>> fsqrt P2 -> Instruction introduced in the POWER2 Architecture. > >>> fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 > >>> > >>> so I'm wondering if just for completeness the "fsqrts" feature > >>> should be included in the feature-string. Besides that I don't > >>> see instrumentation to support has_mftgpr(), which is commented > >>> out, thus should we remove it? Like the following: > >> > >> This code is very old. If we are sure nobody can run the VM on a machine without these instructions they can get removed from the feature checking. > > > > I'm not sure about it. Maybe somebody is using out there, in the community > > for example, so kept it. > > > > > >> The mftgpr was only designed for an extension of Power6 and the opcode was reused for something else later on if I remember correctly. I think you can remove the remaining comment. > > > > I was not aware of that, thanks for the info. > > Just out of curiosity I asked the toolchain folks and they said that mftgpr > > is present in some POWER6 hardware but on the so-called raw mode (ie power6x, > > which I think it's the extension you mentioned), but not in the architected / > > normal mode (power6). So even if it's present in the hardware it's disabled > > unless you boot the system up in raw mode. > > > > On that raw mode (power6x) one would see the following in AUXV, for instance: > > > > $ LD_SHOW_AUXV=1 /bin/true | grep PLATFORM > > AT_PLATFORM: power6x > > AT_BASE_PLATFORM:power6x > > > > But probably theses machines w/ power6x never got out of IBM indeed. > > > > Anyway, they said that power6x is unsupported. So, as you said, I removed the > > comment about mftgpr. > > > > v3 webrev: > > > > http://cr.openjdk.java.net/~gromero/8212481/v3/ > > > > > > Thanks! > > > > Best regards, > > Gustavo > > > From lois.foltan at oracle.com Tue Oct 30 18:13:53 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 30 Oct 2018 14:13:53 -0400 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: <5976703e-b4b0-d865-1e2f-b793f8cfb03b@oracle.com> On 10/30/2018 11:01 AM, Severin Gehwolf wrote: > Hi Lois, > > On Mon, 2018-10-29 at 15:47 -0400, Lois Foltan wrote: >> On 10/25/2018 11:15 AM, Martin Balao wrote: >> >>> Hi Lois, >>> >>> Thanks for having a look at this. >>> >>> Yes, I have a test that triggers this execution path. Unfortunately, >>> this is a security test that I cannot publicly share. What I can tell >>> you, though, is that the parent class loader is a "delegating class >>> loader" which never defined a class and that's the reason why it >>> doesn't have a ClassLoaderData instance created. >>> >>> I'm not sure if this would be needed in other places where the idiom >>> is not used. The reason is that there might be any other implicit >>> assumptions that makes it impossible. That's why I limited the scope >>> of this changeset to the only case that I'm sure it's possible to trigger. >> Hi Martin, >> >> I wrote a test case that reproduces this failure and demonstrates that >> invoking SystemDictionary::register_loader() will not work. With a >> non-product (fastdebug) build, the test case shows that calling >> register_loader is problematic, a SystemDictionary lock is currently >> held and acquiring a ClassLoaderDataGraph lock to create the new >> ClassLoaderData causes an out of order locking fatal error. >> >> Please consider this test case and proposed change: >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8212937/webrev/ > FWIW, I've tested this webrev with our reproducer and this fix works > for us. Thank you Severin for verifying this! Lois > > Thanks, > Severin > >>> Kind regards, >>> Martin.- >>> >>> On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan >> > wrote: >>> >>> On 10/24/2018 3:58 PM, Martin Balao wrote: >>> >>> Hi, >>> >>> Can I have a review for JDK-8212937 [1]? >>> >>> Webrev.00: >>> >>> * >>> http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ >>> >>> * >>> http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00.zip >>> >>> >>> By obtaining the ClassLoaderData instance through a call to >>> SystemDictionary::register_loader, we make sure that either an >>> existing >>> instance is obtained or a new one created. This same idiom has >>> already been >>> used in other places where there are no guarantees that a previous >>> ClassLoaderData instance exists for a given class loader. See >>> futher >>> information in JDK-8212937 ticket [1]. >>> >>> Thanks, >>> Martin.- >>> >>> -- >>> [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 >>> >>> >>> Hi Martin, >>> >>> I agree that the same idiom is used in other places, but there are >>> also a couple of occurrences of java_lang_ClassLoader::parent in >>> classfile/classLoaderStats.cpp and >>> classfile/classLoaderHierarchyDCmd.cpp that do not. In order to >>> better understand, do you have a test case that demonstrates this >>> issue? >>> >>> Thanks, >>> Lois >>> >>> >> From ioi.lam at oracle.com Tue Oct 30 20:41:09 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 30 Oct 2018 13:41:09 -0700 Subject: CDS and JVM-TI agent questions and crash In-Reply-To: References: <2ba503c7-c6f4-4870-06eb-8347648a9bca@oracle.com> <3ff00d0e-8a1f-43c7-7bf5-6595be832b0a@oracle.com> <25e48d53-90ab-01c2-2b1c-dab55a25360c@oracle.com> <5493aca2-f67b-4e9f-1f10-02511d701eca@oracle.com> <4e474a5e-9d65-ca27-94ac-e04a5aafbb0f@oracle.com> Message-ID: <5578a3ef-fbbf-2a60-75d1-04a129dec957@oracle.com> On 10/30/2018 01:45 AM, Michael Rasmussen wrote: > From: Ioi Lam >> I sent RFR to >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030612.html >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-October/025640.html > Hi > > I saw that a fix was pushed to jdk/jdk now. Any plans of back-porting it to jdk11u? > > /Michael Yes, I will backport it to JDK 11u. The issue is https://bugs.openjdk.java.net/browse/JDK-8213164 I'll start the backport once the jdk/jdk testing shows no regression. Thanks - Ioi From coleen.phillimore at oracle.com Tue Oct 30 22:32:04 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 30 Oct 2018 18:32:04 -0400 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <5BCF1C13.5020103@oracle.com> References: <5BCDAD84.4000808@oracle.com> <5BCF1C13.5020103@oracle.com> Message-ID: Erik, It looks like you've removed the only use of VerifyMutexLocker.? I was going to ask if you ran with logging that called nmethod::print_calls() but I don't see any callers. Maybe nmethod::print_nmethod() should call it.? Can you experiment with calling this print_calls to see if you can remove VerifyMutexLocker? http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/sweeper.cpp.frames.html 676 MutexLockerEx mex(CompiledIC_lock); Why isn't this CompiledICLocker (nm)?? Can youy add a comment why it's different? http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/vmBehaviours.new.hpp.html I see you've added more (funny spelling) behaviours.? I think this one should go in the code directory with the user of it and be called compiledICBehaviours.hpp or even codeBehaviours.hpp. thanks, Coleen On 10/23/18 9:03 AM, Erik ?sterlund wrote: > Hi Robbin, > > Thanks for the review. > > /Erik > > On 2018-10-23 14:50, Robbin Ehn wrote: >> Hi Erik, >> >> Looks better than before :), don't forget copyright years before push. >> >> /Robbin >> >> On 10/22/18 12:59 PM, Erik ?sterlund wrote: >>> Hi, >>> >>> Today, one must acquire the global CompiledIC_lock to have >>> permission to perform certain IC updating activities. The lock is >>> used outside of safepoints, but for example during nmethod unloading >>> due to GC, lock free IC cache cleaning is performed, which is safe >>> due to being in a safepoint. >>> >>> With concurrent class unloading introduced, the global lock is too >>> coarse grained, and a per-nmethod mechanism is needed instead. In >>> order to allow this, an abstract CompiledICLocker class could help >>> ensuring the safety of IC caches, and allow both the fine grained >>> (but density costly) approach for users of concurrent class >>> unloading, and resort to the default global locking approach otherwise. >>> >>> The idea is to retain the coarse grained implementation of the >>> synchronization when concurrent class unloading is not being used, >>> as supporting fine grained locking could be more costly in memory, >>> and there does not seem to be any use in doing this unless >>> concurrent class unloading is being used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212681 >>> >>> Thanks, >>> /Erik > From coleen.phillimore at oracle.com Tue Oct 30 22:40:18 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 30 Oct 2018 18:40:18 -0400 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: <5976703e-b4b0-d865-1e2f-b793f8cfb03b@oracle.com> References: <5976703e-b4b0-d865-1e2f-b793f8cfb03b@oracle.com> Message-ID: On 10/30/18 2:13 PM, Lois Foltan wrote: > On 10/30/2018 11:01 AM, Severin Gehwolf wrote: > >> Hi Lois, >> >> On Mon, 2018-10-29 at 15:47 -0400, Lois Foltan wrote: >>> On 10/25/2018 11:15 AM, Martin Balao wrote: >>> >>>> Hi Lois, >>>> >>>> Thanks for having a look at this. >>>> >>>> Yes, I have a test that triggers this execution path. Unfortunately, >>>> this is a security test that I cannot publicly share. What I can tell >>>> you, though, is that the parent class loader is a "delegating class >>>> loader" which never defined a class and that's the reason why it >>>> doesn't have a ClassLoaderData instance created. >>>> >>>> I'm not sure if this would be needed in other places where the idiom >>>> is not used. The reason is that there might be any other implicit >>>> assumptions that makes it impossible. That's why I limited the scope >>>> of this changeset to the only case that I'm sure it's possible to >>>> trigger. >>> Hi Martin, >>> >>> I wrote a test case that reproduces this failure and demonstrates that >>> invoking SystemDictionary::register_loader() will not work. With a >>> non-product (fastdebug) build, the test case shows that calling >>> register_loader is problematic, a SystemDictionary lock is currently >>> held and acquiring a ClassLoaderDataGraph lock to create the new >>> ClassLoaderData causes an out of order locking fatal error. >>> >>> Please consider this test case and proposed change: >>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8212937/webrev/ This change looks good to me.? I'd prefer not calling register_loader() when one is only needed to produce an error message.? This also avoids the lock inversion problem. Thanks, Coleen >> FWIW, I've tested this webrev with our reproducer and this fix works >> for us. > > Thank you Severin for verifying this! > Lois > >> >> Thanks, >> Severin >> >>>> Kind regards, >>>> Martin.- >>>> >>>> On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan >>> > wrote: >>>> >>>> ???? On 10/24/2018 3:58 PM, Martin Balao wrote: >>>> >>>> ???????? Hi, >>>> >>>> ???????? Can I have a review for JDK-8212937 [1]? >>>> >>>> ???????? Webrev.00: >>>> >>>> ?????????? * >>>> http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ >>>> >>>> ?????????? * >>>> http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00.zip >>>> >>>> >>>> >>>> ???????? By obtaining the ClassLoaderData instance through a call to >>>> ???????? SystemDictionary::register_loader, we make sure that >>>> either an >>>> ???????? existing >>>> ???????? instance is obtained or a new one created. This same idiom >>>> has >>>> ???????? already been >>>> ???????? used in other places where there are no guarantees that a >>>> previous >>>> ???????? ClassLoaderData instance exists for a given class loader. See >>>> ???????? futher >>>> ???????? information in JDK-8212937 ticket [1]. >>>> >>>> ???????? Thanks, >>>> ???????? Martin.- >>>> >>>> ???????? -- >>>> ???????? [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 >>>> >>>> >>>> ???? Hi Martin, >>>> >>>> ???? I agree that the same idiom is used in other places, but there >>>> are >>>> ???? also a couple of occurrences of java_lang_ClassLoader::parent in >>>> ???? classfile/classLoaderStats.cpp and >>>> ???? classfile/classLoaderHierarchyDCmd.cpp that do not.? In order to >>>> ???? better understand, do you have a test case that demonstrates this >>>> ???? issue? >>>> >>>> ???? Thanks, >>>> ???? Lois >>>> >>>> >>> > From vicente.romero at oracle.com Wed Oct 31 00:59:31 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 30 Oct 2018 20:59:31 -0400 Subject: [amber-record] feedback on a Record class attribute implementation In-Reply-To: References: Message-ID: <730f6520-1a52-5bb0-cfc4-1827c5dd5564@oracle.com> Hi all, I have sent the patch to the hotspot list as this patch is touching some native code and I would like to have some feedback from the runtime wizards, please let me know if there is a better list for that. This patch has been pushed the the record branch in amber project [1]. The records project is about, well adding data classes to the language so that this declaration: record Record(int i); gets lowered to: class Record { ??? final int i; ?? // automatically generated equals, getters, hashCode, toString and more } apart from the usual information generated for the lowered version, the javac compiler is generating this new attribute in the class file: ??????? Record { ??????????? u2 name_index; ??????????? u4 length; ??????????? u2 num_record_params; ??????????? { ??????????????? u2 param_name_idx;? // [1] ??????????????? u2 param_flag; ??????????????? u2 param_desc; ??????????????? u2 param_signature; ??????????? } record_params[num_record_params]; ??????? } which have a lot in common with the fields information but we don't want to depend on the order of the fields etc. The attached patch provides for parsing this attribute, plus additional helper classes, plus all the pipes needed. As a way to provide a way for users to peek the information contained in the Record attribute, I have added a method to java.lang.Class, Class::getRecordParameters. In the background I'm using JNI to extract the information from the related InstanceKlass in the native world. Method java.lang.Class::getRecordParameters just returns an array of fields but only those that have being defined in the header of the record. For example if the record would have been defined as: record Record(int i) { ??? static final int j = 0;? //no instance fields can be defined in records } then an invocation of java.lang.Class::getRecordParameters will return only field `i` ignoring the static field `j` TIA, Vicente [1] http://hg.openjdk.java.net/amber/amber -------------- next part -------------- A non-text attachment was scrubbed... Name: record.hotspot.patch Type: text/x-patch Size: 34444 bytes Desc: not available URL: From mbalao at redhat.com Wed Oct 31 01:33:47 2018 From: mbalao at redhat.com (Martin Balao) Date: Tue, 30 Oct 2018 22:33:47 -0300 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: Hi Lois, Thanks for your proposal. I'm not a reviewer but it looks good to me too. Kind regards, Martin.- On Mon, Oct 29, 2018 at 4:47 PM, Lois Foltan wrote: > On 10/25/2018 11:15 AM, Martin Balao wrote: > > Hi Lois, > > Thanks for having a look at this. > > Yes, I have a test that triggers this execution path. Unfortunately, this > is a security test that I cannot publicly share. What I can tell you, > though, is that the parent class loader is a "delegating class loader" > which never defined a class and that's the reason why it doesn't have a > ClassLoaderData instance created. > > I'm not sure if this would be needed in other places where the idiom is > not used. The reason is that there might be any other implicit assumptions > that makes it impossible. That's why I limited the scope of this changeset > to the only case that I'm sure it's possible to trigger. > > > Hi Martin, > > I wrote a test case that reproduces this failure and demonstrates that > invoking SystemDictionary::register_loader() will not work. With a > non-product (fastdebug) build, the test case shows that calling > register_loader is problematic, a SystemDictionary lock is currently held > and acquiring a ClassLoaderDataGraph lock to create the new ClassLoaderData > causes an out of order locking fatal error. > > Please consider this test case and proposed change: > http://cr.openjdk.java.net/~lfoltan/bug_jdk8212937/webrev/ > > Thanks, > Lois > > > Kind regards, > Martin.- > > On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan > wrote: > >> On 10/24/2018 3:58 PM, Martin Balao wrote: >> >> Hi, >>> >>> Can I have a review for JDK-8212937 [1]? >>> >>> Webrev.00: >>> >>> * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.w >>> ebrev.00/ >>> * http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.w >>> ebrev.00.zip >>> >>> By obtaining the ClassLoaderData instance through a call to >>> SystemDictionary::register_loader, we make sure that either an existing >>> instance is obtained or a new one created. This same idiom has already >>> been >>> used in other places where there are no guarantees that a previous >>> ClassLoaderData instance exists for a given class loader. See futher >>> information in JDK-8212937 ticket [1]. >>> >>> Thanks, >>> Martin.- >>> >>> -- >>> [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 >>> >> Hi Martin, >> >> I agree that the same idiom is used in other places, but there are also a >> couple of occurrences of java_lang_ClassLoader::parent in >> classfile/classLoaderStats.cpp and classfile/classLoaderHierarchyDCmd.cpp >> that do not. In order to better understand, do you have a test case that >> demonstrates this issue? >> >> Thanks, >> Lois >> > > > From david.holmes at oracle.com Wed Oct 31 01:57:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 31 Oct 2018 11:57:30 +1000 Subject: [amber-record] feedback on a Record class attribute implementation In-Reply-To: <730f6520-1a52-5bb0-cfc4-1827c5dd5564@oracle.com> References: <730f6520-1a52-5bb0-cfc4-1827c5dd5564@oracle.com> Message-ID: Hi Vicente, On 31/10/2018 10:59 AM, Vicente Romero wrote: > Hi all, > > I have sent the patch to the hotspot list as this patch is touching some > native code and I would like to have some feedback from the runtime > wizards, please let me know if there is a better list for that. This hotspot-runtime-dev :) A couple of minor comments: src/hotspot/share/classfile/classFileParser.cpp + // Set nest members attribute to default sentinel + _record_params = Universe::the_empty_short_array(); Comment needs updating. - } else { - // Unknown attribute - cfs->skip_u1(attribute_length, CHECK); + } else if (tag == vmSymbols::tag_record()) { ... You need to insert the new check before the existing else clause but need to keep the check for an unknown attribute in the given classfile version. Presumably this will eventually be processed only for some future classfile version. --- src/hotspot/share/oops/instanceKlass.hpp + u2 _record_params_count; Do you need a separate count when you can query the array length? Cheers, David > patch has been pushed the the record branch in amber project [1]. The > records project is about, well adding data classes to the language so > that this declaration: > > record Record(int i); > > gets lowered to: > > class Record { > ??? final int i; > > ?? // automatically generated equals, getters, hashCode, toString and more > } > > apart from the usual information generated for the lowered version, the > javac compiler is generating this new attribute in the class file: > > ??????? Record { > ??????????? u2 name_index; > ??????????? u4 length; > ??????????? u2 num_record_params; > ??????????? { > ??????????????? u2 param_name_idx;? // [1] > ??????????????? u2 param_flag; > ??????????????? u2 param_desc; > ??????????????? u2 param_signature; > ??????????? } record_params[num_record_params]; > ??????? } > > which have a lot in common with the fields information but we don't want > to depend on the order of the fields etc. The attached patch provides > for parsing this attribute, plus additional helper classes, plus all the > pipes needed. As a way to provide a way for users to peek the > information contained in the Record attribute, I have added a method to > java.lang.Class, Class::getRecordParameters. In the background I'm using > JNI to extract the information from the related InstanceKlass in the > native world. Method java.lang.Class::getRecordParameters just returns > an array of fields but only those that have being defined in the header > of the record. For example if the record would have been defined as: > > record Record(int i) { > ??? static final int j = 0;? //no instance fields can be defined in > records > } > > then an invocation of java.lang.Class::getRecordParameters will return > only field `i` ignoring the static field `j` > > TIA, > Vicente > > [1] http://hg.openjdk.java.net/amber/amber From shade at redhat.com Wed Oct 31 11:59:32 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 31 Oct 2018 12:59:32 +0100 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <5BCDAD84.4000808@oracle.com> References: <5BCDAD84.4000808@oracle.com> Message-ID: On 10/22/2018 12:59 PM, Erik ?sterlund wrote: > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ Ah, I missed this is already in review here. Erik, you need this addition to build on AArch64: http://mail.openjdk.java.net/pipermail/zgc-dev/2018-October/000497.html diff -r 3913e1dc09f5 src/hotspot/cpu/aarch64/nativeInst_aarch64.cpp --- a/src/hotspot/cpu/aarch64/nativeInst_aarch64.cpp Mon Oct 22 12:19:23 2018 +0200 +++ b/src/hotspot/cpu/aarch64/nativeInst_aarch64.cpp Wed Oct 31 12:54:39 2018 +0100 @@ -25,6 +25,7 @@ #include "precompiled.hpp" #include "asm/macroAssembler.hpp" +#include "code/compiledIC.hpp" #include "memory/resourceArea.hpp" #include "nativeInst_aarch64.hpp" #include "oops/oop.inline.hpp" Thanks, -Aleksey From gromero at linux.vnet.ibm.com Wed Oct 31 12:26:59 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Wed, 31 Oct 2018 09:26:59 -0300 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> <271b6743-1611-4201-cdf6-9fdc5b87086e@linux.vnet.ibm.com> <0c176a41bd274bf795382f26e476b59a@sap.com> Message-ID: <71628cc4-fbdb-922d-2993-a72093941d5e@linux.vnet.ibm.com> Hi Volker! On 10/30/2018 02:55 PM, Volker Simonis wrote: > Hi Gustavo, > > thanks for adding Power9 support. Your change looks good! Thanks a lot for reviewing. > Just two minor comments (no need to publish a new webrev): > > - can you please update the copyright year to 2018 in > assembler_ppc.inline.hpp and vm_version_ppc.hpp I think that the copyright update applies to Oracle and SAP, like the following, but I would like to confirm if that is indeed correct as I see some examples "out of sync" in other files: diff -r 57a87060bd09 src/hotspot/cpu/ppc/assembler_ppc.inline.hpp --- a/src/hotspot/cpu/ppc/assembler_ppc.inline.hpp Tue Oct 16 16:26:28 2018 -0400 +++ b/src/hotspot/cpu/ppc/assembler_ppc.inline.hpp Wed Oct 31 08:03:21 2018 -0400 @@ -1,6 +1,6 @@ /* - * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2017 SAP SE. All rights reserved. + * Copyright (c) 2002, 2018, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018 SAP SE. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it diff -r 57a87060bd09 src/hotspot/cpu/ppc/vm_version_ppc.hpp --- a/src/hotspot/cpu/ppc/vm_version_ppc.hpp Tue Oct 16 16:26:28 2018 -0400 +++ b/src/hotspot/cpu/ppc/vm_version_ppc.hpp Wed Oct 31 08:03:21 2018 -0400 @@ -1,6 +1,6 @@ /* - * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2017 SAP SE. All rights reserved. + * Copyright (c) 1997, 2018, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018 SAP SE. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it > - in assembler_ppc.inline.hpp can you pleas add a comment with the > default value as shown below? I often find that useful when browsing > the code. > > +// Deliver A Random Number (introduced with POWER9) > +inline void Assembler::darn(Register d, int l /* = 1 */) { > emit_int32( DARN_OPCODE | rt(d) | l14(l)); } > + Sure, I'll update that before pushing the change. Thank you. Best regards, Gustavo > Thank you and best regards, > Volker > > On Wed, Oct 24, 2018 at 3:35 PM Gustavo Romero > wrote: >> >> Hi Martin, >> >> On 10/24/2018 05:20 AM, Doerr, Martin wrote: >>> looks good. Thank you for cleaning things up and also for sharing the story about mftgpr. Interesting. >>> Reviewed. >> >> Thanks for reviewing! >> >> >> Best regards, >> Gustavo >> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: Gustavo Romero >>> Sent: Dienstag, 23. Oktober 2018 23:46 >>> To: Doerr, Martin ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >>> Subject: Re: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection >>> >>> Hi Martin, >>> >>> Thanks for your comments on fsqrt{,s} and mftgpr opcodes. >>> >>> On 10/23/2018 01:51 PM, Doerr, Martin wrote: >>>> Hi Gustavo, >>>> >>>>> I agree. So, if you don't mind, to reduce any future rework on that >>>>> I removed the hardcoded L=1 and now it's just a default. I introduced >>>>> l14() function that matches two bits in the proper field as I don't >>>>> see any collision with any other one bit field at the same position >>>>> (bit 14). >>>> I like it, but the default value needs to get specified in the .hpp file. >>> >>> Fixed. >>> >>> >>>>> I just have one question regarding the feature-string. I see >>>>> instrumentation for "fsqrts" feature but it's not currently >>>>> advertised by the feature-string. On the other hand, ISA info >>>>> looks to indicate that fsqrt and fsqrts are not aliases, because: >>>>> >>>>> fsqrt P2 -> Instruction introduced in the POWER2 Architecture. >>>>> fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 >>>>> >>>>> so I'm wondering if just for completeness the "fsqrts" feature >>>>> should be included in the feature-string. Besides that I don't >>>>> see instrumentation to support has_mftgpr(), which is commented >>>>> out, thus should we remove it? Like the following: >>>> >>>> This code is very old. If we are sure nobody can run the VM on a machine without these instructions they can get removed from the feature checking. >>> >>> I'm not sure about it. Maybe somebody is using out there, in the community >>> for example, so kept it. >>> >>> >>>> The mftgpr was only designed for an extension of Power6 and the opcode was reused for something else later on if I remember correctly. I think you can remove the remaining comment. >>> >>> I was not aware of that, thanks for the info. >>> Just out of curiosity I asked the toolchain folks and they said that mftgpr >>> is present in some POWER6 hardware but on the so-called raw mode (ie power6x, >>> which I think it's the extension you mentioned), but not in the architected / >>> normal mode (power6). So even if it's present in the hardware it's disabled >>> unless you boot the system up in raw mode. >>> >>> On that raw mode (power6x) one would see the following in AUXV, for instance: >>> >>> $ LD_SHOW_AUXV=1 /bin/true | grep PLATFORM >>> AT_PLATFORM: power6x >>> AT_BASE_PLATFORM:power6x >>> >>> But probably theses machines w/ power6x never got out of IBM indeed. >>> >>> Anyway, they said that power6x is unsupported. So, as you said, I removed the >>> comment about mftgpr. >>> >>> v3 webrev: >>> >>> http://cr.openjdk.java.net/~gromero/8212481/v3/ >>> >>> >>> Thanks! >>> >>> Best regards, >>> Gustavo >>> >> > From erik.osterlund at oracle.com Wed Oct 31 13:16:51 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 31 Oct 2018 14:16:51 +0100 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: References: <5BCDAD84.4000808@oracle.com> <5BCF1C13.5020103@oracle.com> Message-ID: <5BD9AB43.4020609@oracle.com> Hi Coleen, Thanks for the review. New full webrev: http://cr.openjdk.java.net/~eosterlund/8212681/webrev.01/ Incremental: http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00_01/ Fixed Robbin's copyright headers, Aleksey's AArch64 build aaaaand... On 2018-10-30 23:32, coleen.phillimore at oracle.com wrote: > > Erik, > > It looks like you've removed the only use of VerifyMutexLocker. I was > going to ask if you ran with logging that called > nmethod::print_calls() but I don't see any callers. Maybe > nmethod::print_nmethod() should call it. Can you experiment with > calling this print_calls to see if you can remove VerifyMutexLocker? Removed. > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/sweeper.cpp.frames.html > > > 676 MutexLockerEx mex(CompiledIC_lock); > > > Why isn't this CompiledICLocker (nm)? Can youy add a comment why it's > different? Good catch. Should really use the CompiledICLocker, so I changed it so it does that. > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/vmBehaviours.new.hpp.html > > > I see you've added more (funny spelling) behaviours. I think this one > should go in the code directory with the user of it and be called > compiledICBehaviours.hpp or even codeBehaviours.hpp. Changed to code/codeBehaviours.hpp as requested. Thanks, /Erik > thanks, > Coleen > > On 10/23/18 9:03 AM, Erik ?sterlund wrote: >> Hi Robbin, >> >> Thanks for the review. >> >> /Erik >> >> On 2018-10-23 14:50, Robbin Ehn wrote: >>> Hi Erik, >>> >>> Looks better than before :), don't forget copyright years before push. >>> >>> /Robbin >>> >>> On 10/22/18 12:59 PM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> Today, one must acquire the global CompiledIC_lock to have >>>> permission to perform certain IC updating activities. The lock is >>>> used outside of safepoints, but for example during nmethod >>>> unloading due to GC, lock free IC cache cleaning is performed, >>>> which is safe due to being in a safepoint. >>>> >>>> With concurrent class unloading introduced, the global lock is too >>>> coarse grained, and a per-nmethod mechanism is needed instead. In >>>> order to allow this, an abstract CompiledICLocker class could help >>>> ensuring the safety of IC caches, and allow both the fine grained >>>> (but density costly) approach for users of concurrent class >>>> unloading, and resort to the default global locking approach >>>> otherwise. >>>> >>>> The idea is to retain the coarse grained implementation of the >>>> synchronization when concurrent class unloading is not being used, >>>> as supporting fine grained locking could be more costly in memory, >>>> and there does not seem to be any use in doing this unless >>>> concurrent class unloading is being used. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8212681 >>>> >>>> Thanks, >>>> /Erik >> > From erik.osterlund at oracle.com Wed Oct 31 13:17:16 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 31 Oct 2018 14:17:16 +0100 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: References: <5BCDAD84.4000808@oracle.com> Message-ID: <5BD9AB5C.8040900@oracle.com> Hi Aleksey, Thanks for pointing this out. Fixed in the latest increment. /Erik On 2018-10-31 12:59, Aleksey Shipilev wrote: > On 10/22/2018 12:59 PM, Erik ?sterlund wrote: >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ > Ah, I missed this is already in review here. > > Erik, you need this addition to build on AArch64: > http://mail.openjdk.java.net/pipermail/zgc-dev/2018-October/000497.html > > diff -r 3913e1dc09f5 src/hotspot/cpu/aarch64/nativeInst_aarch64.cpp > --- a/src/hotspot/cpu/aarch64/nativeInst_aarch64.cpp Mon Oct 22 12:19:23 2018 +0200 > +++ b/src/hotspot/cpu/aarch64/nativeInst_aarch64.cpp Wed Oct 31 12:54:39 2018 +0100 > @@ -25,6 +25,7 @@ > > #include "precompiled.hpp" > #include "asm/macroAssembler.hpp" > +#include "code/compiledIC.hpp" > #include "memory/resourceArea.hpp" > #include "nativeInst_aarch64.hpp" > #include "oops/oop.inline.hpp" > > Thanks, > -Aleksey > From robbin.ehn at oracle.com Wed Oct 31 13:23:48 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 31 Oct 2018 14:23:48 +0100 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <5BD9AB43.4020609@oracle.com> References: <5BCDAD84.4000808@oracle.com> <5BCF1C13.5020103@oracle.com> <5BD9AB43.4020609@oracle.com> Message-ID: > Incremental: > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00_01/ Another 33 lines bites the dust, great! /Robbin > > Fixed Robbin's copyright headers, Aleksey's AArch64 build aaaaand... > > On 2018-10-30 23:32, coleen.phillimore at oracle.com wrote: >> >> Erik, >> >> It looks like you've removed the only use of VerifyMutexLocker.? I was going >> to ask if you ran with logging that called nmethod::print_calls() but I don't >> see any callers. Maybe nmethod::print_nmethod() should call it.? Can you >> experiment with calling this print_calls to see if you can remove >> VerifyMutexLocker? > > Removed. > >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/sweeper.cpp.frames.html >> >> >> 676 MutexLockerEx mex(CompiledIC_lock); >> >> >> Why isn't this CompiledICLocker (nm)?? Can youy add a comment why it's different? > > Good catch. Should really use the CompiledICLocker, so I changed it so it does > that. > >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/vmBehaviours.new.hpp.html >> >> >> I see you've added more (funny spelling) behaviours.? I think this one should >> go in the code directory with the user of it and be called >> compiledICBehaviours.hpp or even codeBehaviours.hpp. > > Changed to code/codeBehaviours.hpp as requested. > > Thanks, > /Erik > >> thanks, >> Coleen >> >> On 10/23/18 9:03 AM, Erik ?sterlund wrote: >>> Hi Robbin, >>> >>> Thanks for the review. >>> >>> /Erik >>> >>> On 2018-10-23 14:50, Robbin Ehn wrote: >>>> Hi Erik, >>>> >>>> Looks better than before :), don't forget copyright years before push. >>>> >>>> /Robbin >>>> >>>> On 10/22/18 12:59 PM, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> Today, one must acquire the global CompiledIC_lock to have permission to >>>>> perform certain IC updating activities. The lock is used outside of >>>>> safepoints, but for example during nmethod unloading due to GC, lock free >>>>> IC cache cleaning is performed, which is safe due to being in a safepoint. >>>>> >>>>> With concurrent class unloading introduced, the global lock is too coarse >>>>> grained, and a per-nmethod mechanism is needed instead. In order to allow >>>>> this, an abstract CompiledICLocker class could help ensuring the safety of >>>>> IC caches, and allow both the fine grained (but density costly) approach >>>>> for users of concurrent class unloading, and resort to the default global >>>>> locking approach otherwise. >>>>> >>>>> The idea is to retain the coarse grained implementation of the >>>>> synchronization when concurrent class unloading is not being used, as >>>>> supporting fine grained locking could be more costly in memory, and there >>>>> does not seem to be any use in doing this unless concurrent class unloading >>>>> is being used. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8212681 >>>>> >>>>> Thanks, >>>>> /Erik >>> >> > From kim.barrett at oracle.com Wed Oct 31 13:52:36 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Oct 2018 09:52:36 -0400 Subject: RFR: 6735527: Bitmap - speed up searches In-Reply-To: References: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> Message-ID: > On Oct 30, 2018, at 10:29 AM, Thomas Schatzl wrote: > On Fri, 2018-10-26 at 20:18 -0400, Kim Barrett wrote: >> Please review this improvement to the performance of BitMap searches. >> >> [?] >> Question for reviewers: Should the microbenchmark be checked in? It >> doesn't really test anything, just executes some code and prints some >> timing information. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-6735527 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/6735527/open.00/ >> http://cr.openjdk.java.net/~kbarrett/6735527/comparison.ods > > the change looks good to me. Thanks. > The microbenchmark unfortunately does not seem to not fit anywhere in > the rest of the repo at this time. Maybe somebody else has an idea > where to put it. Maybe the new test/micro area that Claes is creating for JEP 230? But that all seems to be Java-based. Since the code isn?t all that interesting (and can be recovered from the review package if anyone cares), I think I?m going to exclude it from the push, unless someone objects. From martin.doerr at sap.com Wed Oct 31 14:09:07 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 31 Oct 2018 14:09:07 +0000 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: <71628cc4-fbdb-922d-2993-a72093941d5e@linux.vnet.ibm.com> References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> <271b6743-1611-4201-cdf6-9fdc5b87086e@linux.vnet.ibm.com> <0c176a41bd274bf795382f26e476b59a@sap.com> <71628cc4-fbdb-922d-2993-a72093941d5e@linux.vnet.ibm.com> Message-ID: <5cc8fc9a0659404993c94c46ed050356@sap.com> Hi Gustavo, your copyright update looks correct. Ideally, both copyright lines should be in sync. But some people use scripts which only update Oracle's copyright. I believe this is why they sometimes get out of sync. Best regards, Martin -----Original Message----- From: Gustavo Romero Sent: Mittwoch, 31. Oktober 2018 13:27 To: Volker Simonis Cc: Doerr, Martin ; HotSpot Open Source Developers ; ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection Hi Volker! On 10/30/2018 02:55 PM, Volker Simonis wrote: > Hi Gustavo, > > thanks for adding Power9 support. Your change looks good! Thanks a lot for reviewing. > Just two minor comments (no need to publish a new webrev): > > - can you please update the copyright year to 2018 in > assembler_ppc.inline.hpp and vm_version_ppc.hpp I think that the copyright update applies to Oracle and SAP, like the following, but I would like to confirm if that is indeed correct as I see some examples "out of sync" in other files: diff -r 57a87060bd09 src/hotspot/cpu/ppc/assembler_ppc.inline.hpp --- a/src/hotspot/cpu/ppc/assembler_ppc.inline.hpp Tue Oct 16 16:26:28 2018 -0400 +++ b/src/hotspot/cpu/ppc/assembler_ppc.inline.hpp Wed Oct 31 08:03:21 2018 -0400 @@ -1,6 +1,6 @@ /* - * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2017 SAP SE. All rights reserved. + * Copyright (c) 2002, 2018, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018 SAP SE. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it diff -r 57a87060bd09 src/hotspot/cpu/ppc/vm_version_ppc.hpp --- a/src/hotspot/cpu/ppc/vm_version_ppc.hpp Tue Oct 16 16:26:28 2018 -0400 +++ b/src/hotspot/cpu/ppc/vm_version_ppc.hpp Wed Oct 31 08:03:21 2018 -0400 @@ -1,6 +1,6 @@ /* - * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2017 SAP SE. All rights reserved. + * Copyright (c) 1997, 2018, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018 SAP SE. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it > - in assembler_ppc.inline.hpp can you pleas add a comment with the > default value as shown below? I often find that useful when browsing > the code. > > +// Deliver A Random Number (introduced with POWER9) > +inline void Assembler::darn(Register d, int l /* = 1 */) { > emit_int32( DARN_OPCODE | rt(d) | l14(l)); } > + Sure, I'll update that before pushing the change. Thank you. Best regards, Gustavo > Thank you and best regards, > Volker > > On Wed, Oct 24, 2018 at 3:35 PM Gustavo Romero > wrote: >> >> Hi Martin, >> >> On 10/24/2018 05:20 AM, Doerr, Martin wrote: >>> looks good. Thank you for cleaning things up and also for sharing the story about mftgpr. Interesting. >>> Reviewed. >> >> Thanks for reviewing! >> >> >> Best regards, >> Gustavo >> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: Gustavo Romero >>> Sent: Dienstag, 23. Oktober 2018 23:46 >>> To: Doerr, Martin ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >>> Subject: Re: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection >>> >>> Hi Martin, >>> >>> Thanks for your comments on fsqrt{,s} and mftgpr opcodes. >>> >>> On 10/23/2018 01:51 PM, Doerr, Martin wrote: >>>> Hi Gustavo, >>>> >>>>> I agree. So, if you don't mind, to reduce any future rework on that >>>>> I removed the hardcoded L=1 and now it's just a default. I introduced >>>>> l14() function that matches two bits in the proper field as I don't >>>>> see any collision with any other one bit field at the same position >>>>> (bit 14). >>>> I like it, but the default value needs to get specified in the .hpp file. >>> >>> Fixed. >>> >>> >>>>> I just have one question regarding the feature-string. I see >>>>> instrumentation for "fsqrts" feature but it's not currently >>>>> advertised by the feature-string. On the other hand, ISA info >>>>> looks to indicate that fsqrt and fsqrts are not aliases, because: >>>>> >>>>> fsqrt P2 -> Instruction introduced in the POWER2 Architecture. >>>>> fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 >>>>> >>>>> so I'm wondering if just for completeness the "fsqrts" feature >>>>> should be included in the feature-string. Besides that I don't >>>>> see instrumentation to support has_mftgpr(), which is commented >>>>> out, thus should we remove it? Like the following: >>>> >>>> This code is very old. If we are sure nobody can run the VM on a machine without these instructions they can get removed from the feature checking. >>> >>> I'm not sure about it. Maybe somebody is using out there, in the community >>> for example, so kept it. >>> >>> >>>> The mftgpr was only designed for an extension of Power6 and the opcode was reused for something else later on if I remember correctly. I think you can remove the remaining comment. >>> >>> I was not aware of that, thanks for the info. >>> Just out of curiosity I asked the toolchain folks and they said that mftgpr >>> is present in some POWER6 hardware but on the so-called raw mode (ie power6x, >>> which I think it's the extension you mentioned), but not in the architected / >>> normal mode (power6). So even if it's present in the hardware it's disabled >>> unless you boot the system up in raw mode. >>> >>> On that raw mode (power6x) one would see the following in AUXV, for instance: >>> >>> $ LD_SHOW_AUXV=1 /bin/true | grep PLATFORM >>> AT_PLATFORM: power6x >>> AT_BASE_PLATFORM:power6x >>> >>> But probably theses machines w/ power6x never got out of IBM indeed. >>> >>> Anyway, they said that power6x is unsupported. So, as you said, I removed the >>> comment about mftgpr. >>> >>> v3 webrev: >>> >>> http://cr.openjdk.java.net/~gromero/8212481/v3/ >>> >>> >>> Thanks! >>> >>> Best regards, >>> Gustavo >>> >> > From gromero at linux.vnet.ibm.com Wed Oct 31 14:34:48 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Wed, 31 Oct 2018 11:34:48 -0300 Subject: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection In-Reply-To: <5cc8fc9a0659404993c94c46ed050356@sap.com> References: <7af8967f-cb19-0293-b688-474d82a03895@linux.vnet.ibm.com> <271b6743-1611-4201-cdf6-9fdc5b87086e@linux.vnet.ibm.com> <0c176a41bd274bf795382f26e476b59a@sap.com> <71628cc4-fbdb-922d-2993-a72093941d5e@linux.vnet.ibm.com> <5cc8fc9a0659404993c94c46ed050356@sap.com> Message-ID: <48556ba8-cf41-04e5-95f7-64b06aa76b07@linux.vnet.ibm.com> Hi Martin, On 10/31/2018 11:09 AM, Doerr, Martin wrote: > Hi Gustavo, > > your copyright update looks correct. Ideally, both copyright lines should be in sync. > But some people use scripts which only update Oracle's copyright. I believe this is why they sometimes get out of sync. oh I see. Thanks for clarifying and confirming the copyright update is correct. I'll push the change with Volker's comments today. Best regards, Gustavo > Best regards, > Martin > > > -----Original Message----- > From: Gustavo Romero > Sent: Mittwoch, 31. Oktober 2018 13:27 > To: Volker Simonis > Cc: Doerr, Martin ; HotSpot Open Source Developers ; ppc-aix-port-dev at openjdk.java.net > Subject: Re: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection > > Hi Volker! > > On 10/30/2018 02:55 PM, Volker Simonis wrote: >> Hi Gustavo, >> >> thanks for adding Power9 support. Your change looks good! > > Thanks a lot for reviewing. > > >> Just two minor comments (no need to publish a new webrev): >> >> - can you please update the copyright year to 2018 in >> assembler_ppc.inline.hpp and vm_version_ppc.hpp > > I think that the copyright update applies to Oracle and SAP, > like the following, but I would like to confirm if that is > indeed correct as I see some examples "out of sync" in other > files: > > diff -r 57a87060bd09 src/hotspot/cpu/ppc/assembler_ppc.inline.hpp > --- a/src/hotspot/cpu/ppc/assembler_ppc.inline.hpp Tue Oct 16 16:26:28 2018 -0400 > +++ b/src/hotspot/cpu/ppc/assembler_ppc.inline.hpp Wed Oct 31 08:03:21 2018 -0400 > @@ -1,6 +1,6 @@ > /* > - * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights reserved. > - * Copyright (c) 2012, 2017 SAP SE. All rights reserved. > + * Copyright (c) 2002, 2018, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2012, 2018 SAP SE. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > diff -r 57a87060bd09 src/hotspot/cpu/ppc/vm_version_ppc.hpp > --- a/src/hotspot/cpu/ppc/vm_version_ppc.hpp Tue Oct 16 16:26:28 2018 -0400 > +++ b/src/hotspot/cpu/ppc/vm_version_ppc.hpp Wed Oct 31 08:03:21 2018 -0400 > @@ -1,6 +1,6 @@ > /* > - * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights reserved. > - * Copyright (c) 2012, 2017 SAP SE. All rights reserved. > + * Copyright (c) 1997, 2018, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2012, 2018 SAP SE. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > > >> - in assembler_ppc.inline.hpp can you pleas add a comment with the >> default value as shown below? I often find that useful when browsing >> the code. >> >> +// Deliver A Random Number (introduced with POWER9) >> +inline void Assembler::darn(Register d, int l /* = 1 */) { >> emit_int32( DARN_OPCODE | rt(d) | l14(l)); } >> + > > Sure, I'll update that before pushing the change. > > Thank you. > > > Best regards, > Gustavo > >> Thank you and best regards, >> Volker >> >> On Wed, Oct 24, 2018 at 3:35 PM Gustavo Romero >> wrote: >>> >>> Hi Martin, >>> >>> On 10/24/2018 05:20 AM, Doerr, Martin wrote: >>>> looks good. Thank you for cleaning things up and also for sharing the story about mftgpr. Interesting. >>>> Reviewed. >>> >>> Thanks for reviewing! >>> >>> >>> Best regards, >>> Gustavo >>> >>>> Best regards, >>>> Martin >>>> >>>> >>>> -----Original Message----- >>>> From: Gustavo Romero >>>> Sent: Dienstag, 23. Oktober 2018 23:46 >>>> To: Doerr, Martin ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net >>>> Subject: Re: RFR(S): 8212481: PPC64: Enable POWER9 CPU detection >>>> >>>> Hi Martin, >>>> >>>> Thanks for your comments on fsqrt{,s} and mftgpr opcodes. >>>> >>>> On 10/23/2018 01:51 PM, Doerr, Martin wrote: >>>>> Hi Gustavo, >>>>> >>>>>> I agree. So, if you don't mind, to reduce any future rework on that >>>>>> I removed the hardcoded L=1 and now it's just a default. I introduced >>>>>> l14() function that matches two bits in the proper field as I don't >>>>>> see any collision with any other one bit field at the same position >>>>>> (bit 14). >>>>> I like it, but the default value needs to get specified in the .hpp file. >>>> >>>> Fixed. >>>> >>>> >>>>>> I just have one question regarding the feature-string. I see >>>>>> instrumentation for "fsqrts" feature but it's not currently >>>>>> advertised by the feature-string. On the other hand, ISA info >>>>>> looks to indicate that fsqrt and fsqrts are not aliases, because: >>>>>> >>>>>> fsqrt P2 -> Instruction introduced in the POWER2 Architecture. >>>>>> fsqrts PPC -> Instruction introduced in the PowerPC Architecture prior to ISA v2.00 >>>>>> >>>>>> so I'm wondering if just for completeness the "fsqrts" feature >>>>>> should be included in the feature-string. Besides that I don't >>>>>> see instrumentation to support has_mftgpr(), which is commented >>>>>> out, thus should we remove it? Like the following: >>>>> >>>>> This code is very old. If we are sure nobody can run the VM on a machine without these instructions they can get removed from the feature checking. >>>> >>>> I'm not sure about it. Maybe somebody is using out there, in the community >>>> for example, so kept it. >>>> >>>> >>>>> The mftgpr was only designed for an extension of Power6 and the opcode was reused for something else later on if I remember correctly. I think you can remove the remaining comment. >>>> >>>> I was not aware of that, thanks for the info. >>>> Just out of curiosity I asked the toolchain folks and they said that mftgpr >>>> is present in some POWER6 hardware but on the so-called raw mode (ie power6x, >>>> which I think it's the extension you mentioned), but not in the architected / >>>> normal mode (power6). So even if it's present in the hardware it's disabled >>>> unless you boot the system up in raw mode. >>>> >>>> On that raw mode (power6x) one would see the following in AUXV, for instance: >>>> >>>> $ LD_SHOW_AUXV=1 /bin/true | grep PLATFORM >>>> AT_PLATFORM: power6x >>>> AT_BASE_PLATFORM:power6x >>>> >>>> But probably theses machines w/ power6x never got out of IBM indeed. >>>> >>>> Anyway, they said that power6x is unsupported. So, as you said, I removed the >>>> comment about mftgpr. >>>> >>>> v3 webrev: >>>> >>>> http://cr.openjdk.java.net/~gromero/8212481/v3/ >>>> >>>> >>>> Thanks! >>>> >>>> Best regards, >>>> Gustavo >>>> >>> >> > From bsrbnd at gmail.com Wed Oct 31 14:51:34 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 31 Oct 2018 15:51:34 +0100 Subject: Bit set intrinsic In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Sun, 21 Oct 2018 at 11:38, Andrew Haley wrote: > > On 10/19/2018 04:42 PM, B. Blaser wrote: > > I tried example [1] with both synchronized BitSet and dedicated > > intrinsic on x86_64 (8 cores) which revealed that the intrinsic would > > be at least 2-3x faster than the synchronized set. > > Which makes it very useful for things like highly-concurrent Bloom > Filters. Here's an example of current usage: > > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/BloomFilter.java > > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/util/Memory.java > > If we can provide a really fast on/off-heap bitset in standard Java > the need for this Unsafe hackery will go away. > > It's interesting that the Cassandra Bloom Filter is not thread safe; I > don't know why this is. The last but not least, I implemented the c2 part (using the 8-bit AND/OR variant) to do sharper comparisons also on non-concurrent execution: http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.02/ With 10e6 iterations the lock latency seems to be more or less negligible and removing it would make the intrinsic about 10% faster than BitSet without synchronization. I used it naively inside RegularEnumSet without optimization nor atomicity guarantee only to evaluate its robustness and tier1 was successful. I'm not sure what would be the process for adding something like this to the JDK (maybe a JEP would be necessary?) but I cannot implement the intrinsic on other architectures myself. Also, I guess a more precise API specification along with advanced profiling would be necessary but I'm probably not an expert in these areas... In any case, I hope these experiments will be useful for something. Please let me know of any feedback, Bernard From coleen.phillimore at oracle.com Wed Oct 31 15:44:45 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 31 Oct 2018 11:44:45 -0400 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <5BD9AB43.4020609@oracle.com> References: <5BCDAD84.4000808@oracle.com> <5BCF1C13.5020103@oracle.com> <5BD9AB43.4020609@oracle.com> Message-ID: Looks good! Coleen On 10/31/18 9:16 AM, Erik ?sterlund wrote: > Hi Coleen, > > Thanks for the review. > > New full webrev: > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00_01/ > > Fixed Robbin's copyright headers, Aleksey's AArch64 build aaaaand... > > On 2018-10-30 23:32, coleen.phillimore at oracle.com wrote: >> >> Erik, >> >> It looks like you've removed the only use of VerifyMutexLocker. I was >> going to ask if you ran with logging that called >> nmethod::print_calls() but I don't see any callers. Maybe >> nmethod::print_nmethod() should call it.? Can you experiment with >> calling this print_calls to see if you can remove VerifyMutexLocker? > > Removed. > >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/sweeper.cpp.frames.html >> >> >> 676 MutexLockerEx mex(CompiledIC_lock); >> >> >> Why isn't this CompiledICLocker (nm)?? Can youy add a comment why >> it's different? > > Good catch. Should really use the CompiledICLocker, so I changed it so > it does that. > >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/vmBehaviours.new.hpp.html >> >> >> I see you've added more (funny spelling) behaviours.? I think this >> one should go in the code directory with the user of it and be called >> compiledICBehaviours.hpp or even codeBehaviours.hpp. > > Changed to code/codeBehaviours.hpp as requested. > > Thanks, > /Erik > >> thanks, >> Coleen >> >> On 10/23/18 9:03 AM, Erik ?sterlund wrote: >>> Hi Robbin, >>> >>> Thanks for the review. >>> >>> /Erik >>> >>> On 2018-10-23 14:50, Robbin Ehn wrote: >>>> Hi Erik, >>>> >>>> Looks better than before :), don't forget copyright years before push. >>>> >>>> /Robbin >>>> >>>> On 10/22/18 12:59 PM, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> Today, one must acquire the global CompiledIC_lock to have >>>>> permission to perform certain IC updating activities. The lock is >>>>> used outside of safepoints, but for example during nmethod >>>>> unloading due to GC, lock free IC cache cleaning is performed, >>>>> which is safe due to being in a safepoint. >>>>> >>>>> With concurrent class unloading introduced, the global lock is too >>>>> coarse grained, and a per-nmethod mechanism is needed instead. In >>>>> order to allow this, an abstract CompiledICLocker class could help >>>>> ensuring the safety of IC caches, and allow both the fine grained >>>>> (but density costly) approach for users of concurrent class >>>>> unloading, and resort to the default global locking approach >>>>> otherwise. >>>>> >>>>> The idea is to retain the coarse grained implementation of the >>>>> synchronization when concurrent class unloading is not being used, >>>>> as supporting fine grained locking could be more costly in memory, >>>>> and there does not seem to be any use in doing this unless >>>>> concurrent class unloading is being used. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8212681 >>>>> >>>>> Thanks, >>>>> /Erik >>> >> > From erik.osterlund at oracle.com Wed Oct 31 15:45:31 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 31 Oct 2018 16:45:31 +0100 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: References: <5BCDAD84.4000808@oracle.com> <5BCF1C13.5020103@oracle.com> <5BD9AB43.4020609@oracle.com> Message-ID: <5BD9CE1B.5050904@oracle.com> Hi Coleen, Thanks for the review! /Erik On 2018-10-31 16:44, coleen.phillimore at oracle.com wrote: > Looks good! > Coleen > > On 10/31/18 9:16 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Thanks for the review. >> >> New full webrev: >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.01/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00_01/ >> >> Fixed Robbin's copyright headers, Aleksey's AArch64 build aaaaand... >> >> On 2018-10-30 23:32, coleen.phillimore at oracle.com wrote: >>> >>> Erik, >>> >>> It looks like you've removed the only use of VerifyMutexLocker. I >>> was going to ask if you ran with logging that called >>> nmethod::print_calls() but I don't see any callers. Maybe >>> nmethod::print_nmethod() should call it. Can you experiment with >>> calling this print_calls to see if you can remove VerifyMutexLocker? >> >> Removed. >> >>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/sweeper.cpp.frames.html >>> >>> >>> 676 MutexLockerEx mex(CompiledIC_lock); >>> >>> >>> Why isn't this CompiledICLocker (nm)? Can youy add a comment why >>> it's different? >> >> Good catch. Should really use the CompiledICLocker, so I changed it >> so it does that. >> >>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/vmBehaviours.new.hpp.html >>> >>> >>> I see you've added more (funny spelling) behaviours. I think this >>> one should go in the code directory with the user of it and be >>> called compiledICBehaviours.hpp or even codeBehaviours.hpp. >> >> Changed to code/codeBehaviours.hpp as requested. >> >> Thanks, >> /Erik >> >>> thanks, >>> Coleen >>> >>> On 10/23/18 9:03 AM, Erik ?sterlund wrote: >>>> Hi Robbin, >>>> >>>> Thanks for the review. >>>> >>>> /Erik >>>> >>>> On 2018-10-23 14:50, Robbin Ehn wrote: >>>>> Hi Erik, >>>>> >>>>> Looks better than before :), don't forget copyright years before >>>>> push. >>>>> >>>>> /Robbin >>>>> >>>>> On 10/22/18 12:59 PM, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> Today, one must acquire the global CompiledIC_lock to have >>>>>> permission to perform certain IC updating activities. The lock is >>>>>> used outside of safepoints, but for example during nmethod >>>>>> unloading due to GC, lock free IC cache cleaning is performed, >>>>>> which is safe due to being in a safepoint. >>>>>> >>>>>> With concurrent class unloading introduced, the global lock is >>>>>> too coarse grained, and a per-nmethod mechanism is needed >>>>>> instead. In order to allow this, an abstract CompiledICLocker >>>>>> class could help ensuring the safety of IC caches, and allow both >>>>>> the fine grained (but density costly) approach for users of >>>>>> concurrent class unloading, and resort to the default global >>>>>> locking approach otherwise. >>>>>> >>>>>> The idea is to retain the coarse grained implementation of the >>>>>> synchronization when concurrent class unloading is not being >>>>>> used, as supporting fine grained locking could be more costly in >>>>>> memory, and there does not seem to be any use in doing this >>>>>> unless concurrent class unloading is being used. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8212681 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>> >>> >> > From vladimir.kozlov at oracle.com Wed Oct 31 16:46:04 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 31 Oct 2018 09:46:04 -0700 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <5BD9AB43.4020609@oracle.com> References: <5BCDAD84.4000808@oracle.com> <5BCF1C13.5020103@oracle.com> <5BD9AB43.4020609@oracle.com> Message-ID: <14de6e12-04e8-2a21-4075-ab8811e71553@oracle.com> Looks good to me too. Thanks, Vladimir On 10/31/18 6:16 AM, Erik ?sterlund wrote: > Hi Coleen, > > Thanks for the review. > > New full webrev: > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00_01/ > > Fixed Robbin's copyright headers, Aleksey's AArch64 build aaaaand... > > On 2018-10-30 23:32, coleen.phillimore at oracle.com wrote: >> >> Erik, >> >> It looks like you've removed the only use of VerifyMutexLocker.? I was going to ask if you ran with logging that >> called nmethod::print_calls() but I don't see any callers. Maybe nmethod::print_nmethod() should call it.? Can you >> experiment with calling this print_calls to see if you can remove VerifyMutexLocker? > > Removed. > >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/sweeper.cpp.frames.html >> >> 676 MutexLockerEx mex(CompiledIC_lock); >> >> >> Why isn't this CompiledICLocker (nm)?? Can youy add a comment why it's different? > > Good catch. Should really use the CompiledICLocker, so I changed it so it does that. > >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/vmBehaviours.new.hpp.html >> >> I see you've added more (funny spelling) behaviours.? I think this one should go in the code directory with the user >> of it and be called compiledICBehaviours.hpp or even codeBehaviours.hpp. > > Changed to code/codeBehaviours.hpp as requested. > > Thanks, > /Erik > >> thanks, >> Coleen >> >> On 10/23/18 9:03 AM, Erik ?sterlund wrote: >>> Hi Robbin, >>> >>> Thanks for the review. >>> >>> /Erik >>> >>> On 2018-10-23 14:50, Robbin Ehn wrote: >>>> Hi Erik, >>>> >>>> Looks better than before :), don't forget copyright years before push. >>>> >>>> /Robbin >>>> >>>> On 10/22/18 12:59 PM, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> Today, one must acquire the global CompiledIC_lock to have permission to perform certain IC updating activities. >>>>> The lock is used outside of safepoints, but for example during nmethod unloading due to GC, lock free IC cache >>>>> cleaning is performed, which is safe due to being in a safepoint. >>>>> >>>>> With concurrent class unloading introduced, the global lock is too coarse grained, and a per-nmethod mechanism is >>>>> needed instead. In order to allow this, an abstract CompiledICLocker class could help ensuring the safety of IC >>>>> caches, and allow both the fine grained (but density costly) approach for users of concurrent class unloading, and >>>>> resort to the default global locking approach otherwise. >>>>> >>>>> The idea is to retain the coarse grained implementation of the synchronization when concurrent class unloading is >>>>> not being used, as supporting fine grained locking could be more costly in memory, and there does not seem to be >>>>> any use in doing this unless concurrent class unloading is being used. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8212681 >>>>> >>>>> Thanks, >>>>> /Erik >>> >> > From rkennke at redhat.com Wed Oct 31 17:27:15 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 31 Oct 2018 18:27:15 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() Message-ID: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> GCs (like Shenandoah) may emit additional loads or stores. In the case of Shenandoah, we emit forward pointer loads from offset -8. Those accesses don't match the pattern currently checked in Assembler::needs_explicit_null_check() which means that explicit null checks would be emitted for forwarding pointer loads. But we don't want that because we know that the base object cannot be NULL (everything else would be an error). I'm proposing to put an abstraction in needs_explicit_null_check(). I am basically trying to solve the mess that we have in Shenandoah in this place: https://builds.shipilev.net/patch-openjdk-shenandoah-jdk-only-shared/latest/src/hotspot/share/asm/assembler.cpp.sdiff.html After experimenting back-and-forth with a bunch of approaches, it seems to be most useful and least intrusive to simply put the whole method under GC's control by moving its body into BarrierSetAssembler: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/src/hotspot/share/gc/shared/barrierSetAssembler.cpp.html The corresponding Shenandoah implementation for x86 would then look like this: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.shenandoah/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp.sdiff.html As an additional bonus, this approach provides not only a GC-abstraction but also implicitely a cpu-abstraction, which helps to avoid the #ifdef mess that we made in that method. Bug: https://bugs.openjdk.java.net/browse/JDK-8213199 Full webrev: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/ Testing: passes hotspot/jtreg:tier3 on x86/fastdebug Can I please get reviews on this? Also, any suggestions for improvement are very welcome! Thanks, Roman From erik.osterlund at oracle.com Wed Oct 31 18:44:03 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 31 Oct 2018 19:44:03 +0100 Subject: RFR: 8212681: Refactor IC locking to use a fine grained CompiledICLocker In-Reply-To: <14de6e12-04e8-2a21-4075-ab8811e71553@oracle.com> References: <5BCDAD84.4000808@oracle.com> <5BCF1C13.5020103@oracle.com> <5BD9AB43.4020609@oracle.com> <14de6e12-04e8-2a21-4075-ab8811e71553@oracle.com> Message-ID: <798470c1-28e4-1704-c90b-3a6a1eb9ed07@oracle.com> Hi Vladimir, Thanks for the review. /Erik On 2018-10-31 17:46, Vladimir Kozlov wrote: > Looks good to me too. > > Thanks, > Vladimir > > On 10/31/18 6:16 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Thanks for the review. >> >> New full webrev: >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.01/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00_01/ >> >> Fixed Robbin's copyright headers, Aleksey's AArch64 build aaaaand... >> >> On 2018-10-30 23:32, coleen.phillimore at oracle.com wrote: >>> >>> Erik, >>> >>> It looks like you've removed the only use of VerifyMutexLocker.? I >>> was going to ask if you ran with logging that called >>> nmethod::print_calls() but I don't see any callers. Maybe >>> nmethod::print_nmethod() should call it.? Can you experiment with >>> calling this print_calls to see if you can remove VerifyMutexLocker? >> >> Removed. >> >>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/sweeper.cpp.frames.html >>> >>> >>> 676 MutexLockerEx mex(CompiledIC_lock); >>> >>> >>> Why isn't this CompiledICLocker (nm)?? Can youy add a comment why >>> it's different? >> >> Good catch. Should really use the CompiledICLocker, so I changed it so >> it does that. >> >>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/src/hotspot/share/runtime/vmBehaviours.new.hpp.html >>> >>> >>> I see you've added more (funny spelling) behaviours.? I think this >>> one should go in the code directory with the user of it and be called >>> compiledICBehaviours.hpp or even codeBehaviours.hpp. >> >> Changed to code/codeBehaviours.hpp as requested. >> >> Thanks, >> /Erik >> >>> thanks, >>> Coleen >>> >>> On 10/23/18 9:03 AM, Erik ?sterlund wrote: >>>> Hi Robbin, >>>> >>>> Thanks for the review. >>>> >>>> /Erik >>>> >>>> On 2018-10-23 14:50, Robbin Ehn wrote: >>>>> Hi Erik, >>>>> >>>>> Looks better than before :), don't forget copyright years before push. >>>>> >>>>> /Robbin >>>>> >>>>> On 10/22/18 12:59 PM, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> Today, one must acquire the global CompiledIC_lock to have >>>>>> permission to perform certain IC updating activities. The lock is >>>>>> used outside of safepoints, but for example during nmethod >>>>>> unloading due to GC, lock free IC cache cleaning is performed, >>>>>> which is safe due to being in a safepoint. >>>>>> >>>>>> With concurrent class unloading introduced, the global lock is too >>>>>> coarse grained, and a per-nmethod mechanism is needed instead. In >>>>>> order to allow this, an abstract CompiledICLocker class could help >>>>>> ensuring the safety of IC caches, and allow both the fine grained >>>>>> (but density costly) approach for users of concurrent class >>>>>> unloading, and resort to the default global locking approach >>>>>> otherwise. >>>>>> >>>>>> The idea is to retain the coarse grained implementation of the >>>>>> synchronization when concurrent class unloading is not being used, >>>>>> as supporting fine grained locking could be more costly in memory, >>>>>> and there does not seem to be any use in doing this unless >>>>>> concurrent class unloading is being used. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8212681/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8212681 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>> >>> >> From coleen.phillimore at oracle.com Wed Oct 31 19:16:26 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 31 Oct 2018 15:16:26 -0400 Subject: RFR (trivial) 8213211: [BACKOUT] Allow Klass::_subklass and _next_sibling to have unloaded classes Message-ID: <547d8b95-6565-4569-fd65-e50c8110ef61@oracle.com> hg backout succeeded without any problems.?? I made ErikO the reviewer in the hg commit. open webrev at http://cr.openjdk.java.net/~coleenp/8213211.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8213211 There are several hangs.? One identified cause but others are iffy. Would rather backout and go on vacation and fix it when I'm back and mach5 is stable. Thanks, Coleen From jesper.wilhelmsson at oracle.com Wed Oct 31 19:18:31 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Wed, 31 Oct 2018 20:18:31 +0100 Subject: RFR (trivial) 8213211: [BACKOUT] Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <547d8b95-6565-4569-fd65-e50c8110ef61@oracle.com> References: <547d8b95-6565-4569-fd65-e50c8110ef61@oracle.com> Message-ID: <9E5A91C8-38DB-49A6-B56B-DEA227E7A100@oracle.com> Looks good! /Jesper > On 31 Oct 2018, at 20:16, coleen.phillimore at oracle.com wrote: > > hg backout succeeded without any problems. I made ErikO the reviewer in the hg commit. > > open webrev at http://cr.openjdk.java.net/~coleenp/8213211.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8213211 > > There are several hangs. One identified cause but others are iffy. Would rather backout and go on vacation and fix it when I'm back and mach5 is stable. > > Thanks, > Coleen From jiangli.zhou at oracle.com Wed Oct 31 19:19:51 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 31 Oct 2018 12:19:51 -0700 Subject: RFR (trivial) 8213211: [BACKOUT] Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <9E5A91C8-38DB-49A6-B56B-DEA227E7A100@oracle.com> References: <547d8b95-6565-4569-fd65-e50c8110ef61@oracle.com> <9E5A91C8-38DB-49A6-B56B-DEA227E7A100@oracle.com> Message-ID: +1 Thanks, Jiangli On 10/31/18 12:18 PM, jesper.wilhelmsson at oracle.com wrote: > Looks good! > /Jesper > >> On 31 Oct 2018, at 20:16, coleen.phillimore at oracle.com wrote: >> >> hg backout succeeded without any problems. I made ErikO the reviewer in the hg commit. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8213211.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8213211 >> >> There are several hangs. One identified cause but others are iffy. Would rather backout and go on vacation and fix it when I'm back and mach5 is stable. >> >> Thanks, >> Coleen From coleen.phillimore at oracle.com Wed Oct 31 19:24:29 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 31 Oct 2018 15:24:29 -0400 Subject: RFR (trivial) 8213211: [BACKOUT] Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: References: <547d8b95-6565-4569-fd65-e50c8110ef61@oracle.com> <9E5A91C8-38DB-49A6-B56B-DEA227E7A100@oracle.com> Message-ID: <8efcf93f-9ebe-bd87-8eda-a07ec72dd40c@oracle.com> Thanks Jesper and Jiangli. Coleen On 10/31/18 3:19 PM, Jiangli Zhou wrote: > +1 > > Thanks, > > Jiangli > > > On 10/31/18 12:18 PM, jesper.wilhelmsson at oracle.com wrote: >> Looks good! >> /Jesper >> >>> On 31 Oct 2018, at 20:16, coleen.phillimore at oracle.com wrote: >>> >>> hg backout succeeded without any problems.?? I made ErikO the >>> reviewer in the hg commit. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8213211.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8213211 >>> >>> There are several hangs.? One identified cause but others are iffy. >>> Would rather backout and go on vacation and fix it when I'm back and >>> mach5 is stable. >>> >>> Thanks, >>> Coleen > From kim.barrett at oracle.com Wed Oct 31 19:27:22 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Oct 2018 15:27:22 -0400 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> Message-ID: <8496DA21-3DAD-4C20-BFCF-BB2EB1400AC9@oracle.com> > On Oct 31, 2018, at 1:27 PM, Roman Kennke wrote: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213199 > Full webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/ > > Testing: passes hotspot/jtreg:tier3 on x86/fastdebug > > Can I please get reviews on this? Also, any suggestions for improvement > are very welcome! > > Thanks, > Roman I think BarrierSetAssembler::needs_explicit_null_check() should be const. Other than that, looks good. From erik.osterlund at oracle.com Wed Oct 31 19:39:38 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 31 Oct 2018 20:39:38 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> Message-ID: <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> Hi Roman, Pretty sure this will break zero. Thanks, /Erik > On 31 Oct 2018, at 18:27, Roman Kennke wrote: > > GCs (like Shenandoah) may emit additional loads or stores. In the case > of Shenandoah, we emit forward pointer loads from offset -8. Those > accesses don't match the pattern currently checked in > Assembler::needs_explicit_null_check() which means that explicit null > checks would be emitted for forwarding pointer loads. But we don't want > that because we know that the base object cannot be NULL (everything > else would be an error). > > I'm proposing to put an abstraction in needs_explicit_null_check(). I am > basically trying to solve the mess that we have in Shenandoah in this place: > > https://builds.shipilev.net/patch-openjdk-shenandoah-jdk-only-shared/latest/src/hotspot/share/asm/assembler.cpp.sdiff.html > > After experimenting back-and-forth with a bunch of approaches, it seems > to be most useful and least intrusive to simply put the whole method > under GC's control by moving its body into BarrierSetAssembler: > > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/src/hotspot/share/gc/shared/barrierSetAssembler.cpp.html > > The corresponding Shenandoah implementation for x86 would then look like > this: > > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.shenandoah/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp.sdiff.html > > As an additional bonus, this approach provides not only a GC-abstraction > but also implicitely a cpu-abstraction, which helps to avoid the #ifdef > mess that we made in that method. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213199 > Full webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/ > > Testing: passes hotspot/jtreg:tier3 on x86/fastdebug > > Can I please get reviews on this? Also, any suggestions for improvement > are very welcome! > > Thanks, > Roman > From coleen.phillimore at oracle.com Wed Oct 31 19:42:05 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 31 Oct 2018 15:42:05 -0400 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> <27ebdbbd-833e-5b78-93b0-83af04834835@oracle.com> <08A02CFE-EB69-409F-9DA5-BA2B82F2B249@oracle.com> <3945eb38-b053-a802-666b-7d520822360a@oracle.com> <5417DFAA-9829-4A21-B4FA-8E313DEFD51B@oracle.com> Message-ID: <60f47d8d-e55d-0d3f-66bb-70ef689c3e65@oracle.com> On 10/26/18 4:32 PM, Kim Barrett wrote: >> On Oct 26, 2018, at 4:29 PM, Kim Barrett wrote: >> >>> On Oct 25, 2018, at 7:00 PM, coleen.phillimore at oracle.com wrote: >>> On 10/25/18 6:26 PM, Kim Barrett wrote: >>>>> On Oct 25, 2018, at 6:01 PM, coleen.phillimore at oracle.com wrote: >>>>> 425 Block* block = block_for_allocation(); >>>>> 426 if (block == NULL) return NULL; // Block allocation failed. >>>>> >>>>> This could be "get_block_for_allocation" because it's not blocking (afaict). Name is somewhat ambiguous. >>>> Hotspot style guide says getters are noun phrases, with no ?get_" noise word. >>> The style guide says that we avoid of noise word get for cases like this: >>> >>> Block* block() const { return _block; } >>> void set_block(Block* b) { _block = b; } >>> >>> If the function is doing anything else, you can say get. Also because 'block' is a verb, it makes it confusing. >> Sorry, but I don't agree. >> >> That's a very narrow interpretation of "getter", and suggests the >> naming convention depends on the underlying implementation. But a >> primary purpose of providing a function-based API is information >> hiding; the implementation can be changed without affecting clients. >> Tying the name to the implementation as suggested is contrary to that >> purpose. So I think that interpretation is incorrect. > Or if that *is* the ?original intent? of tat coding guideline, then I think the > guideline is wrong. > Ok, that's fine.? Leave out the "get_". Reviewed. Coleen From kim.barrett at oracle.com Wed Oct 31 21:23:42 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Oct 2018 17:23:42 -0400 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: <60f47d8d-e55d-0d3f-66bb-70ef689c3e65@oracle.com> References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> <27ebdbbd-833e-5b78-93b0-83af04834835@oracle.com> <08A02CFE-EB69-409F-9DA5-BA2B82F2B249@oracle.com> <3945eb38-b053-a802-666b-7d520822360a@oracle.com> <5417DFAA-9829-4A21-B4FA-8E313DEFD51B@oracle.com> <60f47d8d-e55d-0d3f-66bb-70ef689c3e65@oracle.com> Message-ID: > On Oct 31, 2018, at 3:42 PM, coleen.phillimore at oracle.com wrote: > > > > On 10/26/18 4:32 PM, Kim Barrett wrote: >>> On Oct 26, 2018, at 4:29 PM, Kim Barrett wrote: >>> >>>> On Oct 25, 2018, at 7:00 PM, coleen.phillimore at oracle.com wrote: >>>> On 10/25/18 6:26 PM, Kim Barrett wrote: >>>>>> On Oct 25, 2018, at 6:01 PM, coleen.phillimore at oracle.com wrote: >>>>>> 425 Block* block = block_for_allocation(); >>>>>> 426 if (block == NULL) return NULL; // Block allocation failed. >>>>>> >>>>>> This could be "get_block_for_allocation" because it's not blocking (afaict). Name is somewhat ambiguous. >>>>> Hotspot style guide says getters are noun phrases, with no ?get_" noise word. >>>> The style guide says that we avoid of noise word get for cases like this: >>>> >>>> Block* block() const { return _block; } >>>> void set_block(Block* b) { _block = b; } >>>> >>>> If the function is doing anything else, you can say get. Also because 'block' is a verb, it makes it confusing. >>> Sorry, but I don't agree. >>> >>> That's a very narrow interpretation of "getter", and suggests the >>> naming convention depends on the underlying implementation. But a >>> primary purpose of providing a function-based API is information >>> hiding; the implementation can be changed without affecting clients. >>> Tying the name to the implementation as suggested is contrary to that >>> purpose. So I think that interpretation is incorrect. >> Or if that *is* the ?original intent? of tat coding guideline, then I think the >> guideline is wrong. >> > Ok, that's fine. Leave out the "get_". > Reviewed. > Coleen Thanks. From rkennke at redhat.com Wed Oct 31 22:14:58 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 31 Oct 2018 23:14:58 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> Message-ID: <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> Hi Erik, right. Fixed this, and what what Kim mentioned plus a missing include: Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ Ok now? Roman > Hi Roman, > > Pretty sure this will break zero. > > Thanks, > /Erik > >> On 31 Oct 2018, at 18:27, Roman Kennke wrote: >> >> GCs (like Shenandoah) may emit additional loads or stores. In the case >> of Shenandoah, we emit forward pointer loads from offset -8. Those >> accesses don't match the pattern currently checked in >> Assembler::needs_explicit_null_check() which means that explicit null >> checks would be emitted for forwarding pointer loads. But we don't want >> that because we know that the base object cannot be NULL (everything >> else would be an error). >> >> I'm proposing to put an abstraction in needs_explicit_null_check(). I am >> basically trying to solve the mess that we have in Shenandoah in this place: >> >> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk-only-shared/latest/src/hotspot/share/asm/assembler.cpp.sdiff.html >> >> After experimenting back-and-forth with a bunch of approaches, it seems >> to be most useful and least intrusive to simply put the whole method >> under GC's control by moving its body into BarrierSetAssembler: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/src/hotspot/share/gc/shared/barrierSetAssembler.cpp.html >> >> The corresponding Shenandoah implementation for x86 would then look like >> this: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.shenandoah/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp.sdiff.html >> >> As an additional bonus, this approach provides not only a GC-abstraction >> but also implicitely a cpu-abstraction, which helps to avoid the #ifdef >> mess that we made in that method. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8213199 >> Full webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/ >> >> Testing: passes hotspot/jtreg:tier3 on x86/fastdebug >> >> Can I please get reviews on this? Also, any suggestions for improvement >> are very welcome! >> >> Thanks, >> Roman >> > From dean.long at oracle.com Wed Oct 31 22:23:12 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 31 Oct 2018 15:23:12 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged Message-ID: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8212605 http://cr.openjdk.java.net/~dlong/8212605/webrev.1 This change implements AccessController.doPrivileged in Java.? This gives a performance improvement while also being useful to Project Loom by removing the Java --> native --> Java transition.? One reason doPrivileged has historically been in native is because of the need to guarantee the cleanup of the privileged context when doPrivileged returns.? To do that in Java, the information that AccessController.getContext needs is pushed onto the Java stack as arguments to a method that getContext will recognize during its stack walk.? This allows us to remove the native privileged stack while guaranteeing that the privileged context goes away when the method returns. Tested with tier1-tier3 hotspot and jdk tests and JCK api/java_security tests.? For the first few rounds of testing, I kept the old native privileged stack and compared the results of the old and new implementations for each getContext call, which did catch some early bugs.? The changes were also examined by internal security experts and run through additional internal security tests. The improvement on this [1] doPrivileged microbenchmark is approximate 50x. There is no attempt to optimize getContext() or security permission checks in this change, however, this is intended to be a first step towards other possible improvements, for example those proposed here [2]. dl [1] http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java [2] http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html From kim.barrett at oracle.com Wed Oct 31 22:27:40 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Oct 2018 18:27:40 -0400 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> Message-ID: <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> > On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: > > Hi Erik, > > right. Fixed this, and what what Kim mentioned plus a missing include: > > Incremental: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ > > Ok now? Looks good. From david.holmes at oracle.com Wed Oct 31 23:06:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Nov 2018 09:06:02 +1000 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> Message-ID: <6523ec93-af32-da8f-704a-0322e56de00f@oracle.com> Hi Dean, Looking only at the hotspot changes. The removal of the DoPrivileged and related privileged_stack code seems okay. I have a few related comments: src/hotspot/share/classfile/systemDictionary.hpp You added the java_security_AccessController class after java_security_AccessControlContext. Did you actually verify where in the load/initialization order the AccessController class appears today, and where it appears after your change? (Note the comments at the start of WK_KLASSES_DO). Changes to the initialization order would be my biggest concern with this patch. --- I'm unclear about the change to the test: test/hotspot/jtreg/runtime/JVMDoPrivileged/DoPrivRunAbstract.jasm as it still refers to the now non-existent JVM_DoPrivileged in the summary. Does this test still make sense? Should it be moved to the Java side now it doesn't actually test anything in the VM? --- There may be further dead code to remove now: vmSymbols.hpp: codesource_permissioncollection_signature is now unreferenced and can be removed. javaClasses.*: - java_lang_System::has_security_manager - java_security_AccessControlContext::is_authorized ./share/memory/universe.hpp: static Method* protection_domain_implies_method() --- Thanks, David On 1/11/2018 8:23 AM, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8212605 > http://cr.openjdk.java.net/~dlong/8212605/webrev.1 > > This change implements AccessController.doPrivileged in Java.? This > gives a performance improvement while also being useful to Project Loom > by removing the Java --> native --> Java transition.? One reason > doPrivileged has historically been in native is because of the need to > guarantee the cleanup of the privileged context when doPrivileged > returns.? To do that in Java, the information that > AccessController.getContext needs is pushed onto the Java stack as > arguments to a method that getContext will recognize during its stack > walk.? This allows us to remove the native privileged stack while > guaranteeing that the privileged context goes away when the method returns. > > Tested with tier1-tier3 hotspot and jdk tests and JCK api/java_security > tests.? For the first few rounds of testing, I kept the old native > privileged stack and compared the results of the old and new > implementations for each getContext call, which did catch some early > bugs.? The changes were also examined by internal security experts and > run through additional internal security tests. > > The improvement on this [1] doPrivileged microbenchmark is approximate 50x. > > There is no attempt to optimize getContext() or security permission > checks in this change, however, this is intended to be a first step > towards other possible improvements, for example those proposed here [2]. > > dl > > [1] > http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java > > [2] > http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html > >