From jaroslav.bachorik at oracle.com Mon Aug 3 09:36:13 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 03 Aug 2015 11:36:13 +0200 Subject: RFR (XS) 8060245: update bsd version of jhelper.d to be in sync with the fix of 8009204 on solaris Message-ID: <55BF360D.7000201@oracle.com> Hi Serguey, reviving this old RFR. Looks good (just update the copyright year to 2015)! -JB- > Please, review the fix for: > https://bugs.openjdk.java.net/browse/JDK-8060245 > > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8060245-dtrace.1/ > > > Summary: > > The fix of 8009204 for jhelper.d was applied to the Solaris > version only but the bsd version must match it too. > > > Testing: > N/A: The jhelper.d is not used on bsd yet > > > Thanks, > Serguei From dmitry.samersoff at oracle.com Mon Aug 3 12:30:40 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Mon, 03 Aug 2015 15:30:40 +0300 Subject: RFR(S): JDK-8132648 sun/tools/jhsdb/BasicLauncherTest fails with java.lang.RuntimeException Message-ID: <55BF5EF0.1090005@oracle.com> Everybody, Please review the fix: http://cr.openjdk.java.net/~dsamersoff/JDK-8132648/webrev.01/ Changes: Check for standard conditions that cause attach to fail Remove word "Server" to match both Server and Client compiler configurations. -Dmitry -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From jaroslav.bachorik at oracle.com Mon Aug 3 15:11:46 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 03 Aug 2015 17:11:46 +0200 Subject: RFR(S): JDK-8132648 sun/tools/jhsdb/BasicLauncherTest fails with java.lang.RuntimeException In-Reply-To: <55BF5EF0.1090005@oracle.com> References: <55BF5EF0.1090005@oracle.com> Message-ID: <55BF84B2.5090501@oracle.com> Looks fine! -JB- On 3.8.2015 14:30, Dmitry Samersoff wrote: > Everybody, > > Please review the fix: > > http://cr.openjdk.java.net/~dsamersoff/JDK-8132648/webrev.01/ > > Changes: > > Check for standard conditions that cause attach to fail > > Remove word "Server" to match both Server and Client compiler > configurations. > > -Dmitry > From jaroslav.bachorik at oracle.com Mon Aug 3 15:31:19 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 03 Aug 2015 17:31:19 +0200 Subject: RFR 8085919: OperatingSystemMXBean/TestTotalSwap.java failure : Total Swap Space figures mismatch Message-ID: <55BF8947.5060009@oracle.com> Please, review the following test change Issue : https://bugs.openjdk.java.net/browse/JDK-8085919 Webrev: http://cr.openjdk.java.net/~jbachorik/8085919/webrev.00 The test fails on embedded machines with 'yocto' flavour of OS installed. For some reason the 'free -b' command ignores the request to report sizes in bytes and does report in kilobytes. I've changed the test to expect this behaviour (when the OS version string contains 'yocto') and in case the expected and reported total swap size are different to try conversion from kilobytes to bytes and check again. Thanks, -JB- From olivier.lagneau at oracle.com Mon Aug 3 16:00:58 2015 From: olivier.lagneau at oracle.com (olivier.lagneau at oracle.com) Date: Mon, 3 Aug 2015 18:00:58 +0200 Subject: RFR 8085919: OperatingSystemMXBean/TestTotalSwap.java failure : Total Swap Space figures mismatch In-Reply-To: <55BF8947.5060009@oracle.com> References: <55BF8947.5060009@oracle.com> Message-ID: <55BF903A.7030400@oracle.com> Looks good to me. Not a reviewer however. Olivier. On 03/08/2015 17:31, Jaroslav Bachorik wrote: > Please, review the following test change > > Issue : https://bugs.openjdk.java.net/browse/JDK-8085919 > Webrev: http://cr.openjdk.java.net/~jbachorik/8085919/webrev.00 > > The test fails on embedded machines with 'yocto' flavour of OS > installed. For some reason the 'free -b' command ignores the request > to report sizes in bytes and does report in kilobytes. > > I've changed the test to expect this behaviour (when the OS version > string contains 'yocto') and in case the expected and reported total > swap size are different to try conversion from kilobytes to bytes and > check again. > > Thanks, > > -JB- -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon Aug 3 23:22:28 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 03 Aug 2015 16:22:28 -0700 Subject: RFR(S): JDK-8132648 sun/tools/jhsdb/BasicLauncherTest fails with java.lang.RuntimeException In-Reply-To: <55BF5EF0.1090005@oracle.com> References: <55BF5EF0.1090005@oracle.com> Message-ID: <55BFF7B4.9020404@oracle.com> Hi Dmitry, Looks good. Thanks, Serguei On 8/3/15 5:30 AM, Dmitry Samersoff wrote: > Everybody, > > Please review the fix: > > http://cr.openjdk.java.net/~dsamersoff/JDK-8132648/webrev.01/ > > Changes: > > Check for standard conditions that cause attach to fail > > Remove word "Server" to match both Server and Client compiler > configurations. > > -Dmitry > From david.holmes at oracle.com Tue Aug 4 00:25:13 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2015 10:25:13 +1000 Subject: RFR 8085919: OperatingSystemMXBean/TestTotalSwap.java failure : Total Swap Space figures mismatch In-Reply-To: <55BF8947.5060009@oracle.com> References: <55BF8947.5060009@oracle.com> Message-ID: <55C00669.3090603@oracle.com> On 4/08/2015 1:31 AM, Jaroslav Bachorik wrote: > Please, review the following test change > > Issue : https://bugs.openjdk.java.net/browse/JDK-8085919 > Webrev: http://cr.openjdk.java.net/~jbachorik/8085919/webrev.00 > > The test fails on embedded machines with 'yocto' flavour of OS > installed. For some reason the 'free -b' command ignores the request to > report sizes in bytes and does report in kilobytes. > > I've changed the test to expect this behaviour (when the OS version > string contains 'yocto') and in case the expected and reported total > swap size are different to try conversion from kilobytes to bytes and > check again. Seems okay. I would have tweaked this: long expected_swap_size = getSwapSizeFromOs(); to long expected_swap_size = getSwapSizeFromOs() * (SwapInKB ? 1024 : 1); Cheers, David > Thanks, > > -JB- From jaroslav.bachorik at oracle.com Tue Aug 4 07:10:21 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 04 Aug 2015 09:10:21 +0200 Subject: RFR 8085919: OperatingSystemMXBean/TestTotalSwap.java failure : Total Swap Space figures mismatch In-Reply-To: <55C00669.3090603@oracle.com> References: <55BF8947.5060009@oracle.com> <55C00669.3090603@oracle.com> Message-ID: <55C0655D.4090807@oracle.com> Hi David, On 4.8.2015 02:25, David Holmes wrote: > On 4/08/2015 1:31 AM, Jaroslav Bachorik wrote: >> Please, review the following test change >> >> Issue : https://bugs.openjdk.java.net/browse/JDK-8085919 >> Webrev: http://cr.openjdk.java.net/~jbachorik/8085919/webrev.00 >> >> The test fails on embedded machines with 'yocto' flavour of OS >> installed. For some reason the 'free -b' command ignores the request to >> report sizes in bytes and does report in kilobytes. >> >> I've changed the test to expect this behaviour (when the OS version >> string contains 'yocto') and in case the expected and reported total >> swap size are different to try conversion from kilobytes to bytes and >> check again. > > Seems okay. I would have tweaked this: > > long expected_swap_size = getSwapSizeFromOs(); > > to > > long expected_swap_size = getSwapSizeFromOs() * (SwapInKB ? 1024 > : 1); I thought about this - but doing just this the test would start failing the minute they fix the 'free' tool in yocto linux distribution :/ -JB- > > Cheers, > David > >> Thanks, >> >> -JB- From david.holmes at oracle.com Tue Aug 4 07:13:39 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2015 17:13:39 +1000 Subject: RFR 8085919: OperatingSystemMXBean/TestTotalSwap.java failure : Total Swap Space figures mismatch In-Reply-To: <55C0655D.4090807@oracle.com> References: <55BF8947.5060009@oracle.com> <55C00669.3090603@oracle.com> <55C0655D.4090807@oracle.com> Message-ID: <55C06623.2040604@oracle.com> On 4/08/2015 5:10 PM, Jaroslav Bachorik wrote: > Hi David, > > On 4.8.2015 02:25, David Holmes wrote: >> On 4/08/2015 1:31 AM, Jaroslav Bachorik wrote: >>> Please, review the following test change >>> >>> Issue : https://bugs.openjdk.java.net/browse/JDK-8085919 >>> Webrev: http://cr.openjdk.java.net/~jbachorik/8085919/webrev.00 >>> >>> The test fails on embedded machines with 'yocto' flavour of OS >>> installed. For some reason the 'free -b' command ignores the request to >>> report sizes in bytes and does report in kilobytes. >>> >>> I've changed the test to expect this behaviour (when the OS version >>> string contains 'yocto') and in case the expected and reported total >>> swap size are different to try conversion from kilobytes to bytes and >>> check again. >> >> Seems okay. I would have tweaked this: >> >> long expected_swap_size = getSwapSizeFromOs(); >> >> to >> >> long expected_swap_size = getSwapSizeFromOs() * (SwapInKB ? 1024 >> : 1); > > I thought about this - but doing just this the test would start failing > the minute they fix the 'free' tool in yocto linux distribution :/ So instead we will carry a workaround for ever because we won't realize it isn't needed anymore :) Okay. Thanks, David > -JB- > >> >> Cheers, >> David >> >>> Thanks, >>> >>> -JB- > From jaroslav.bachorik at oracle.com Tue Aug 4 07:22:00 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 04 Aug 2015 09:22:00 +0200 Subject: RFR 8085919: OperatingSystemMXBean/TestTotalSwap.java failure : Total Swap Space figures mismatch In-Reply-To: <55C06623.2040604@oracle.com> References: <55BF8947.5060009@oracle.com> <55C00669.3090603@oracle.com> <55C0655D.4090807@oracle.com> <55C06623.2040604@oracle.com> Message-ID: <55C06818.4080706@oracle.com> On 4.8.2015 09:13, David Holmes wrote: > On 4/08/2015 5:10 PM, Jaroslav Bachorik wrote: >> Hi David, >> >> On 4.8.2015 02:25, David Holmes wrote: >>> On 4/08/2015 1:31 AM, Jaroslav Bachorik wrote: >>>> Please, review the following test change >>>> >>>> Issue : https://bugs.openjdk.java.net/browse/JDK-8085919 >>>> Webrev: http://cr.openjdk.java.net/~jbachorik/8085919/webrev.00 >>>> >>>> The test fails on embedded machines with 'yocto' flavour of OS >>>> installed. For some reason the 'free -b' command ignores the request to >>>> report sizes in bytes and does report in kilobytes. >>>> >>>> I've changed the test to expect this behaviour (when the OS version >>>> string contains 'yocto') and in case the expected and reported total >>>> swap size are different to try conversion from kilobytes to bytes and >>>> check again. >>> >>> Seems okay. I would have tweaked this: >>> >>> long expected_swap_size = getSwapSizeFromOs(); >>> >>> to >>> >>> long expected_swap_size = getSwapSizeFromOs() * (SwapInKB ? 1024 >>> : 1); >> >> I thought about this - but doing just this the test would start failing >> the minute they fix the 'free' tool in yocto linux distribution :/ > > So instead we will carry a workaround for ever because we won't realize > it isn't needed anymore :) Um, well. Something like that. Can't really think of a good solution to this. -JB- > > Okay. > > Thanks, > David > > >> -JB- >> >>> >>> Cheers, >>> David >>> >>>> Thanks, >>>> >>>> -JB- >> From staffan.larsen at oracle.com Tue Aug 4 11:04:00 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 4 Aug 2015 13:04:00 +0200 Subject: Low-Overhead Heap Profiling In-Reply-To: References: Message-ID: Hi, Sorry for being away for so long. If memory serves me right then the current AllocObjectInNewTLAB JFR event was written as a way to quickly get some allocation profiling information with a minimum of VM changes. It probably also carried over to Hotspot from JRockit. I agree that we can and should collect better information than what that event does. I like the approach of instrumenting the interpreter and compiler to do the sampling. I had thought it would be a lot of work to do it and was reluctant to go down that path. If the work is already done and Jeremy has maintained it for a few years without great problems, I think we should look at bringing it in. I am not worried about the overhead of a decrement and a compare in the allocation path, but of course we should benchmark that. It wasn?t clear to me from the discussion how (or if) this was being exposed to end users. Was the proposal to just have the native entry points in the VM and have these be used by various agents? Would this be part of JVMTI? I think a draft JEP would be the logical next step and make it easier for us all to discuss what exactly the proposal should contain. Also, let?s not forget the need for tests for the feature. Thanks, /Staffan From jaroslav.bachorik at oracle.com Tue Aug 4 14:20:52 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 04 Aug 2015 16:20:52 +0200 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <552D2B00.4090805@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> Message-ID: <55C0CA44.3030909@oracle.com> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: > On 14.4.2015 14:56, Daniel Fuchs wrote: >> Hi Jaroslav, >> >> I like this change, but it does introduce an incompatibility, >> so it probably needs a CCC and some release notes. >> >> For instance, this test passes with the current version of >> ObjectName: >> >> public class StringLengthTest { >> >> final static int smax = Short.MAX_VALUE; >> final static int smore = smax + 126; >> public static void main(String[] args) throws >> MalformedObjectNameException { >> char[] chars = new char[smore]; >> Arrays.fill(chars, 0, smax, 'a'); >> Arrays.fill(chars, smax, smore, 'b'); >> >> System.out.println(new ObjectName(new String(chars), "type", >> "Test")); >> } >> >> } >> >> I'm not sure what it will do with your changes :-) > > It will fail with assert (if enabled). Or truncate the domain name, I > suppose. > >> >> With that in mind I believe you should consider throwing >> InternalError - or IllegalArgumentException - instead of >> using 'assert' statements. > > This would probably be better. > >> >> BTW have you run the JCK? > > Yes. All passed. I don't think JCK is testing for unrealistic values :) > I don't see how one could come up with a domain name 32767 characters long. The proposed change has passed the CCC review. In case the domain name is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will be thrown. All the JMX tests and JCK are still passing. http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- > > -JB- > >> >> best regards, >> >> -- daniel >> >> On 13/04/15 17:07, Jaroslav Bachorik wrote: >>> Hi Roger, >>> >>> On 13.4.2015 16:07, Roger Riggs wrote: >>>> Hi Jaroslav, >>>> >>>> Minor comments: >>>> >>>> 1488+: In forms like: >>>> >>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" >>>> >>>> The &0xff seems unnecessary since the store is to a byte field. >>> >>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>> >>>> >>>> 1644: the ? and : operators should be surrounded by spaces. >>>> >>>> There are other style issues, such as then statements on the same line >>>> as the 'if' but that >>>> may be beyond the scope of this change. >>> >>> I know but these style issue have been around since the file was first >>> committed. I didn't address them because it didn't feel right to mix >>> style changes with the actual functionality. >>> >>> Cheers, >>> >>> -JB- >>> >>>> >>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>> serviceability reviewer). >>>> >>>> Thanks, Roger >>>> >>>> >>>> On 4/13/2015 5:43 AM, Jaroslav Bachorik wrote: >>>>> Please, review the following change >>>>> >>>>> Issue : https://bugs.openjdk.java.net/browse/JDK-8041565 >>>>> Webrev: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 >>>>> >>>>> In situations when there are 10s of thousands ObjectNname instances >>>>> around (enterprise setups etc.) the 3 separate internal boolean fields >>>>> can lead to a noticeable memory waste. Adding insult to the injury, >>>>> with the current field layout it is necessary to align the instances >>>>> by 4 bytes. >>>>> >>>>> When using JOL (http://openjdk.java.net/projects/code-tools/jol/) to >>>>> inspect the object layout we can see this: >>>>> >>>>> Before optimization (JDK8u40): >>>>> --- >>>>> javax.management.ObjectName object internals: >>>>> OFFSET SIZE TYPE DESCRIPTION VALUE >>>>> 0 12 (object header)| N/A >>>>> 12 4 int ObjectName._domain_length N/A >>>>> 16 1 boolean ObjectName._domain_pattern N/A >>>>> 17 1 boolean ObjectName._property_list_pattern N/A >>>>> 18 1 boolean ObjectName._property_value_pattern N/A >>>>> 19 1 (alignment/padding gap) N/A >>>>> 20 4 String ObjectName._canonicalName N/A >>>>> 24 4 Property[] ObjectName._kp_array N/A >>>>> 28 4 Property[] ObjectName._ca_array N/A >>>>> 32 4 Map ObjectName._propertyList N/A >>>>> 36 4 (loss due to the next object alignment) >>>>> Instance size: 40 bytes (estimated, the sample instance is not >>>>> available) >>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>> {noformat} >>>>> >>>>> After optimization (JDK9 internal build): >>>>> --- >>>>> >>>>> javax.management.ObjectName object internals: >>>>> OFFSET SIZE TYPE DESCRIPTION VALUE >>>>> 0 12 (object header) N/A >>>>> 12 2 short ObjectName._domain_length N/A >>>>> 14 1 byte ObjectName._pattern_flag N/A >>>>> 15 1 (alignment/padding gap) N/A >>>>> 16 4 String ObjectName._canonicalName N/A >>>>> 20 4 Property[] ObjectName._kp_array N/A >>>>> 24 4 Property[] ObjectName._ca_array N/A >>>>> 28 4 Map ObjectName._propertyList N/A >>>>> Instance size: 32 bytes (estimated, the sample instance is not >>>>> available) >>>>> Space losses: 1 bytes internal + 0 bytes external = 1 bytes total >>>>> >>>>> >>>>> After optimization we can save 8 bytes per instance which can >>>>> translate to very interesting numbers on large installations. >>>>> >>>>> >>>>> To achieve this the domain name length is set to be *short* instead of >>>>> *int* and the three booleans kept for the performance purposes are >>>>> encoded into one byte value (as proposed by the reporter, >>>>> Jean-Francois Denise). >>>>> >>>>> All the regression and JCK tests are passing after this change. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> -JB- >>>> >>> >> > From eamonn at mcmanus.net Tue Aug 4 15:24:09 2015 From: eamonn at mcmanus.net (Eamonn McManus) Date: Tue, 4 Aug 2015 08:24:09 -0700 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55C0CA44.3030909@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> Message-ID: I would like to suggest some cosmetic changes. The hex constants could have underscores, for example 0x8000_0000. The FLAG_MASK constant could be expressed as the OR of the three flags rather than as a literal, and the DOMAIN_LENGTH_MASK could be ~FLAG_MASK. In setDomainLength, the parameter name l isn't very good because it looks like 1. ?amonn 2015-08-04 7:20 GMT-07:00 Jaroslav Bachorik : > Hi, > > reviving this review. > > > On 14.4.2015 16:58, Jaroslav Bachorik wrote: >> >> On 14.4.2015 14:56, Daniel Fuchs wrote: >>> >>> Hi Jaroslav, >>> >>> I like this change, but it does introduce an incompatibility, >>> so it probably needs a CCC and some release notes. >>> >>> For instance, this test passes with the current version of >>> ObjectName: >>> >>> public class StringLengthTest { >>> >>> final static int smax = Short.MAX_VALUE; >>> final static int smore = smax + 126; >>> public static void main(String[] args) throws >>> MalformedObjectNameException { >>> char[] chars = new char[smore]; >>> Arrays.fill(chars, 0, smax, 'a'); >>> Arrays.fill(chars, smax, smore, 'b'); >>> >>> System.out.println(new ObjectName(new String(chars), "type", >>> "Test")); >>> } >>> >>> } >>> >>> I'm not sure what it will do with your changes :-) >> >> >> It will fail with assert (if enabled). Or truncate the domain name, I >> suppose. >> >>> >>> With that in mind I believe you should consider throwing >>> InternalError - or IllegalArgumentException - instead of >>> using 'assert' statements. >> >> >> This would probably be better. >> >>> >>> BTW have you run the JCK? >> >> >> Yes. All passed. I don't think JCK is testing for unrealistic values :) >> I don't see how one could come up with a domain name 32767 characters >> long. > > > The proposed change has passed the CCC review. > > In case the domain name is longer than Integer.MAX_VALUE/4 a > MalformedObjectNameException will be thrown. > > All the JMX tests and JCK are still passing. > > http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 > > -JB- > > >> >> -JB- >> >>> >>> best regards, >>> >>> -- daniel >>> >>> On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>> >>>> Hi Roger, >>>> >>>> On 13.4.2015 16:07, Roger Riggs wrote: >>>>> >>>>> Hi Jaroslav, >>>>> >>>>> Minor comments: >>>>> >>>>> 1488+: In forms like: >>>>> >>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" >>>>> >>>>> The &0xff seems unnecessary since the store is to a byte field. >>>> >>>> >>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>> >>>>> >>>>> 1644: the ? and : operators should be surrounded by spaces. >>>>> >>>>> There are other style issues, such as then statements on the same line >>>>> as the 'if' but that >>>>> may be beyond the scope of this change. >>>> >>>> >>>> I know but these style issue have been around since the file was first >>>> committed. I didn't address them because it didn't feel right to mix >>>> style changes with the actual functionality. >>>> >>>> Cheers, >>>> >>>> -JB- >>>> >>>>> >>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>> serviceability reviewer). >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>>> On 4/13/2015 5:43 AM, Jaroslav Bachorik wrote: >>>>>> >>>>>> Please, review the following change >>>>>> >>>>>> Issue : https://bugs.openjdk.java.net/browse/JDK-8041565 >>>>>> Webrev: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 >>>>>> >>>>>> In situations when there are 10s of thousands ObjectNname instances >>>>>> around (enterprise setups etc.) the 3 separate internal boolean fields >>>>>> can lead to a noticeable memory waste. Adding insult to the injury, >>>>>> with the current field layout it is necessary to align the instances >>>>>> by 4 bytes. >>>>>> >>>>>> When using JOL (http://openjdk.java.net/projects/code-tools/jol/) to >>>>>> inspect the object layout we can see this: >>>>>> >>>>>> Before optimization (JDK8u40): >>>>>> --- >>>>>> javax.management.ObjectName object internals: >>>>>> OFFSET SIZE TYPE DESCRIPTION VALUE >>>>>> 0 12 (object header)| N/A >>>>>> 12 4 int ObjectName._domain_length N/A >>>>>> 16 1 boolean ObjectName._domain_pattern N/A >>>>>> 17 1 boolean ObjectName._property_list_pattern N/A >>>>>> 18 1 boolean ObjectName._property_value_pattern N/A >>>>>> 19 1 (alignment/padding gap) N/A >>>>>> 20 4 String ObjectName._canonicalName N/A >>>>>> 24 4 Property[] ObjectName._kp_array N/A >>>>>> 28 4 Property[] ObjectName._ca_array N/A >>>>>> 32 4 Map ObjectName._propertyList N/A >>>>>> 36 4 (loss due to the next object alignment) >>>>>> Instance size: 40 bytes (estimated, the sample instance is not >>>>>> available) >>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>> {noformat} >>>>>> >>>>>> After optimization (JDK9 internal build): >>>>>> --- >>>>>> >>>>>> javax.management.ObjectName object internals: >>>>>> OFFSET SIZE TYPE DESCRIPTION VALUE >>>>>> 0 12 (object header) N/A >>>>>> 12 2 short ObjectName._domain_length N/A >>>>>> 14 1 byte ObjectName._pattern_flag N/A >>>>>> 15 1 (alignment/padding gap) N/A >>>>>> 16 4 String ObjectName._canonicalName N/A >>>>>> 20 4 Property[] ObjectName._kp_array N/A >>>>>> 24 4 Property[] ObjectName._ca_array N/A >>>>>> 28 4 Map ObjectName._propertyList N/A >>>>>> Instance size: 32 bytes (estimated, the sample instance is not >>>>>> available) >>>>>> Space losses: 1 bytes internal + 0 bytes external = 1 bytes total >>>>>> >>>>>> >>>>>> After optimization we can save 8 bytes per instance which can >>>>>> translate to very interesting numbers on large installations. >>>>>> >>>>>> >>>>>> To achieve this the domain name length is set to be *short* instead of >>>>>> *int* and the three booleans kept for the performance purposes are >>>>>> encoded into one byte value (as proposed by the reporter, >>>>>> Jean-Francois Denise). >>>>>> >>>>>> All the regression and JCK tests are passing after this change. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -JB- >>>>> >>>>> >>>> >>> >> > From daniel.fuchs at oracle.com Tue Aug 4 21:02:50 2015 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 4 Aug 2015 23:02:50 +0200 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55C0CA44.3030909@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> Message-ID: <55C1287A.6030004@oracle.com> Hi Jaroslav, 379 * This field encodes _domain_pattern, _property_list_pattern and 380 * _property_value_pattern booleans. If I'm not mistaken it also encodes the domain length. 1072 if ((l & FLAG_MASK) > 0 ) { Although it is not expected that 'l' could be negative, it might be better to write this test as: if ((l & FLAG_MASK) != 0 ) { (+ I agree with ?amonn that l is not a great parameter name - nice to see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, Jaroslav Bachorik wrote: > Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: >> On 14.4.2015 14:56, Daniel Fuchs wrote: >>> Hi Jaroslav, I like this change, but it does introduce an >>> incompatibility, so it probably needs a CCC and some release notes. >>> For instance, this test passes with the current version of >>> ObjectName: public class StringLengthTest { final static int >>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>> public static void main(String[] args) throws >>> MalformedObjectNameException { char[] chars = new >>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>> Arrays.fill(chars, smax, smore, 'b'); >>> System.out.println(new ObjectName(new String(chars), "type", >>> "Test")); } } I'm not sure what it will do with your changes :-) >> It will fail with assert (if enabled). Or truncate the domain name, I >> suppose. >>> With that in mind I believe you should consider throwing >>> InternalError - or IllegalArgumentException - instead of using >>> 'assert' statements. >> This would probably be better. >>> BTW have you run the JCK? >> Yes. All passed. I don't think JCK is testing for unrealistic values >> :) I don't see how one could come up with a domain name 32767 >> characters long. > The proposed change has passed the CCC review. In case the domain name > is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will > be thrown. All the JMX tests and JCK are still passing. > http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >> -JB- >>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>> unnecessary since the store is to a byte field. >>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>> 1644: the ? and : operators should be surrounded by spaces. There >>>>> are other style issues, such as then statements on the same line >>>>> as the 'if' but that may be beyond the scope of this change. >>>> I know but these style issue have been around since the file was >>>> first committed. I didn't address them because it didn't feel right >>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>> Jaroslav Bachorik wrote: >>>>>> Please, review the following change Issue : >>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>> situations when there are 10s of thousands ObjectNname instances >>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>> the injury, with the current field layout it is necessary to >>>>>> align the instances by 4 bytes. When using JOL >>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to inspect the >>>>>> object layout we can see this: Before optimization (JDK8u40): --- >>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>> (alignment/padding gap) N/A 20 4 String >>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>> size: 40 bytes (estimated, the sample instance is not available) >>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 short >>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>> not available) Space losses: 1 bytes internal + 0 bytes external >>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>> instance which can translate to very interesting numbers on large >>>>>> installations. To achieve this the domain name length is set to >>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>> performance purposes are encoded into one byte value (as proposed >>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>> JCK tests are passing after this change. Thanks, -JB- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Tue Aug 4 21:22:52 2015 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 4 Aug 2015 14:22:52 -0700 Subject: Low-Overhead Heap Profiling In-Reply-To: References: Message-ID: Thanks, Staffan. I've been tinkering with a draft JEP, but it has been going slowly because of other craziness in my life. Some responses: 1) It was a fair bit of work to do it, mostly because it was the first time I had tinkered with c2. This was 5 years ago. Most of the work since then has been dealing with merge conflicts. 2) I did envision a JVMTI interface. More in the JEP. 3) We have some internal tests, but obviously, we'd have to figure out how to externalize them. For native code, it's much easier only to have to worry about building and running on Linux. Presumably, there's already code in there for JVMTI. I'll try to get a JEP going for real, although it might not happen in the next week or so, because I'm moving next week (+the JVMLS). Jeremy On Tue, Aug 4, 2015 at 4:04 AM, Staffan Larsen wrote: > Hi, > > Sorry for being away for so long. > > If memory serves me right then the current AllocObjectInNewTLAB JFR event > was written as a way to quickly get some allocation profiling information > with a minimum of VM changes. It probably also carried over to Hotspot from > JRockit. I agree that we can and should collect better information than > what that event does. > > I like the approach of instrumenting the interpreter and compiler to do > the sampling. I had thought it would be a lot of work to do it and was > reluctant to go down that path. If the work is already done and Jeremy has > maintained it for a few years without great problems, I think we should > look at bringing it in. I am not worried about the overhead of a decrement > and a compare in the allocation path, but of course we should benchmark > that. > > It wasn?t clear to me from the discussion how (or if) this was being > exposed to end users. Was the proposal to just have the native entry points > in the VM and have these be used by various agents? Would this be part of > JVMTI? > > I think a draft JEP would be the logical next step and make it easier for > us all to discuss what exactly the proposal should contain. Also, let?s not > forget the need for tests for the feature. > > Thanks, > /Staffan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaroslav.bachorik at oracle.com Wed Aug 5 11:48:33 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 05 Aug 2015 13:48:33 +0200 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55C1287A.6030004@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> Message-ID: <55C1F811.2040804@oracle.com> Eamonn, Daniel, thanks for the comments. I've updated the webrev to address them. Also, I've added a test to exercise the boolean flag en-/decoding in ObjectName. http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 Cheers, -JB- On 4.8.2015 23:02, Daniel Fuchs wrote: > Hi Jaroslav, > > 379 * This field encodes _domain_pattern, _property_list_pattern and > 380 * _property_value_pattern booleans. > > If I'm not mistaken it also encodes the domain length. > > > 1072 if ((l & FLAG_MASK) > 0 ) { > > Although it is not expected that 'l' could be negative, it might be > better to write this test as: > > if ((l & FLAG_MASK) != 0 ) { > > (+ I agree with ?amonn that l is not a great parameter name - nice to > see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, > Jaroslav Bachorik wrote: >> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: >>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>> Hi Jaroslav, I like this change, but it does introduce an >>>> incompatibility, so it probably needs a CCC and some release notes. >>>> For instance, this test passes with the current version of >>>> ObjectName: public class StringLengthTest { final static int >>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>> public static void main(String[] args) throws >>>> MalformedObjectNameException { char[] chars = new >>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>> Arrays.fill(chars, smax, smore, 'b'); >>>> System.out.println(new ObjectName(new String(chars), "type", >>>> "Test")); } } I'm not sure what it will do with your changes :-) >>> It will fail with assert (if enabled). Or truncate the domain name, I >>> suppose. >>>> With that in mind I believe you should consider throwing >>>> InternalError - or IllegalArgumentException - instead of using >>>> 'assert' statements. >>> This would probably be better. >>>> BTW have you run the JCK? >>> Yes. All passed. I don't think JCK is testing for unrealistic values >>> :) I don't see how one could come up with a domain name 32767 >>> characters long. >> The proposed change has passed the CCC review. In case the domain name >> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will >> be thrown. All the JMX tests and JCK are still passing. >> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>> -JB- >>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>> unnecessary since the store is to a byte field. >>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>> 1644: the ? and : operators should be surrounded by spaces. There >>>>>> are other style issues, such as then statements on the same line >>>>>> as the 'if' but that may be beyond the scope of this change. >>>>> I know but these style issue have been around since the file was >>>>> first committed. I didn't address them because it didn't feel right >>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>> Jaroslav Bachorik wrote: >>>>>>> Please, review the following change Issue : >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>> situations when there are 10s of thousands ObjectNname instances >>>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>> the injury, with the current field layout it is necessary to >>>>>>> align the instances by 4 bytes. When using JOL >>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to inspect the >>>>>>> object layout we can see this: Before optimization (JDK8u40): --- >>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>> size: 40 bytes (estimated, the sample instance is not available) >>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 short >>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>> not available) Space losses: 1 bytes internal + 0 bytes external >>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>> instance which can translate to very interesting numbers on large >>>>>>> installations. To achieve this the domain name length is set to >>>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>>> performance purposes are encoded into one byte value (as proposed >>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>> JCK tests are passing after this change. Thanks, -JB- From daniel.daugherty at oracle.com Wed Aug 5 14:00:23 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 05 Aug 2015 08:00:23 -0600 Subject: RFR 8085919: OperatingSystemMXBean/TestTotalSwap.java failure : Total Swap Space figures mismatch In-Reply-To: <55C06818.4080706@oracle.com> References: <55BF8947.5060009@oracle.com> <55C00669.3090603@oracle.com> <55C0655D.4090807@oracle.com> <55C06623.2040604@oracle.com> <55C06818.4080706@oracle.com> Message-ID: <55C216F7.7050008@oracle.com> On 8/4/15 1:22 AM, Jaroslav Bachorik wrote: > On 4.8.2015 09:13, David Holmes wrote: >> On 4/08/2015 5:10 PM, Jaroslav Bachorik wrote: >>> Hi David, >>> >>> On 4.8.2015 02:25, David Holmes wrote: >>>> On 4/08/2015 1:31 AM, Jaroslav Bachorik wrote: >>>>> Please, review the following test change >>>>> >>>>> Issue : https://bugs.openjdk.java.net/browse/JDK-8085919 >>>>> Webrev: http://cr.openjdk.java.net/~jbachorik/8085919/webrev.00 >>>>> >>>>> The test fails on embedded machines with 'yocto' flavour of OS >>>>> installed. For some reason the 'free -b' command ignores the >>>>> request to >>>>> report sizes in bytes and does report in kilobytes. >>>>> >>>>> I've changed the test to expect this behaviour (when the OS version >>>>> string contains 'yocto') and in case the expected and reported total >>>>> swap size are different to try conversion from kilobytes to bytes and >>>>> check again. >>>> >>>> Seems okay. I would have tweaked this: >>>> >>>> long expected_swap_size = getSwapSizeFromOs(); >>>> >>>> to >>>> >>>> long expected_swap_size = getSwapSizeFromOs() * (SwapInKB ? >>>> 1024 >>>> : 1); >>> >>> I thought about this - but doing just this the test would start failing >>> the minute they fix the 'free' tool in yocto linux distribution :/ >> >> So instead we will carry a workaround for ever because we won't realize >> it isn't needed anymore :) > > Um, well. Something like that. Can't really think of a good solution > to this. So here is the theory: - We have a specific set of platforms for JDK9 and we make these types of accommodations for bugs in those platforms. - When we move on to JDK10, we will have a (hopefully) newer set of platforms and we'll make new accommodations for bugs in those platforms. We'll also remove accommodations for bugs that have been fixed, but this is more rare. If we don't know that a platform bug has been fixed because the test still passes, we won't know to remove the accommodation in JDK10. I like to see comments like this with such code: // Work around for XYZ bug in 'yocto' version of Linux. // When this platform bug is fixed, this test will fail // again and we'll know that this work around can be removed. So the result will be that JDK9 will have the work around in it and that's fine because the broken platform is supported in that release. JDK10 will not have the work around in it because the broken platform is no longer supported in that release. If someone tries to run the JDK10 version of the test on the 'yocto' platform it will fail and, when investigated, the triage engineer will hopefully realize that JDK10 should not be run on 'yocto'. Dan > > -JB- > >> >> Okay. >> >> Thanks, >> David >> >> >>> -JB- >>> >>>> >>>> Cheers, >>>> David >>>> >>>>> Thanks, >>>>> >>>>> -JB- >>> > > From eamonn at mcmanus.net Wed Aug 5 14:53:37 2015 From: eamonn at mcmanus.net (Eamonn McManus) Date: Wed, 5 Aug 2015 07:53:37 -0700 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55C1F811.2040804@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> Message-ID: I would remove the spec changes about the limit on the domain length, which are a property of this particular implementation. It's perfectly reasonable to blow up if the domain length is > 536,870,911, but there's no reason for it to be in the spec. ?amonn 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik : > Eamonn, Daniel, > > thanks for the comments. > > I've updated the webrev to address them. Also, I've added a test to exercise > the boolean flag en-/decoding in ObjectName. > > http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 > > > Cheers, > > -JB- > > > On 4.8.2015 23:02, Daniel Fuchs wrote: >> >> Hi Jaroslav, >> >> 379 * This field encodes _domain_pattern, _property_list_pattern >> and >> 380 * _property_value_pattern booleans. >> >> If I'm not mistaken it also encodes the domain length. >> >> >> 1072 if ((l & FLAG_MASK) > 0 ) { >> >> Although it is not expected that 'l' could be negative, it might be >> better to write this test as: >> >> if ((l & FLAG_MASK) != 0 ) { >> >> (+ I agree with ?amonn that l is not a great parameter name - nice to >> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >> Jaroslav Bachorik wrote: >>> >>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: >>>> >>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>> >>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>> incompatibility, so it probably needs a CCC and some release notes. >>>>> For instance, this test passes with the current version of >>>>> ObjectName: public class StringLengthTest { final static int >>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>> public static void main(String[] args) throws >>>>> MalformedObjectNameException { char[] chars = new >>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>> "Test")); } } I'm not sure what it will do with your changes :-) >>>> >>>> It will fail with assert (if enabled). Or truncate the domain name, I >>>> suppose. >>>>> >>>>> With that in mind I believe you should consider throwing >>>>> InternalError - or IllegalArgumentException - instead of using >>>>> 'assert' statements. >>>> >>>> This would probably be better. >>>>> >>>>> BTW have you run the JCK? >>>> >>>> Yes. All passed. I don't think JCK is testing for unrealistic values >>>> :) I don't see how one could come up with a domain name 32767 >>>> characters long. >>> >>> The proposed change has passed the CCC review. In case the domain name >>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will >>> be thrown. All the JMX tests and JCK are still passing. >>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>> >>>> -JB- >>>>> >>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>>>> >>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>> >>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>> unnecessary since the store is to a byte field. >>>>>> >>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>> >>>>>>> 1644: the ? and : operators should be surrounded by spaces. There >>>>>>> are other style issues, such as then statements on the same line >>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>> >>>>>> I know but these style issue have been around since the file was >>>>>> first committed. I didn't address them because it didn't feel right >>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>> >>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>> Jaroslav Bachorik wrote: >>>>>>>> >>>>>>>> Please, review the following change Issue : >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>> situations when there are 10s of thousands ObjectNname instances >>>>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to inspect the >>>>>>>> object layout we can see this: Before optimization (JDK8u40): --- >>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>> size: 40 bytes (estimated, the sample instance is not available) >>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 short >>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes external >>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>> instance which can translate to very interesting numbers on large >>>>>>>> installations. To achieve this the domain name length is set to >>>>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>>>> performance purposes are encoded into one byte value (as proposed >>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>> JCK tests are passing after this change. Thanks, -JB- > > From jaroslav.bachorik at oracle.com Wed Aug 5 15:02:19 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 05 Aug 2015 17:02:19 +0200 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> Message-ID: <55C2257B.1030500@oracle.com> On 5.8.2015 16:53, Eamonn McManus wrote: > I would remove the spec changes about the limit on the domain length, > which are a property of this particular implementation. It's perfectly > reasonable to blow up if the domain length is > 536,870,911, but > there's no reason for it to be in the spec. Well, CCC board had a different opinion. That's why this limit is documented in the spec. -JB- > ?amonn > > > 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik : >> Eamonn, Daniel, >> >> thanks for the comments. >> >> I've updated the webrev to address them. Also, I've added a test to exercise >> the boolean flag en-/decoding in ObjectName. >> >> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 >> >> >> Cheers, >> >> -JB- >> >> >> On 4.8.2015 23:02, Daniel Fuchs wrote: >>> >>> Hi Jaroslav, >>> >>> 379 * This field encodes _domain_pattern, _property_list_pattern >>> and >>> 380 * _property_value_pattern booleans. >>> >>> If I'm not mistaken it also encodes the domain length. >>> >>> >>> 1072 if ((l & FLAG_MASK) > 0 ) { >>> >>> Although it is not expected that 'l' could be negative, it might be >>> better to write this test as: >>> >>> if ((l & FLAG_MASK) != 0 ) { >>> >>> (+ I agree with ?amonn that l is not a great parameter name - nice to >>> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >>> Jaroslav Bachorik wrote: >>>> >>>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: >>>>> >>>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>>> >>>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>>> incompatibility, so it probably needs a CCC and some release notes. >>>>>> For instance, this test passes with the current version of >>>>>> ObjectName: public class StringLengthTest { final static int >>>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>>> public static void main(String[] args) throws >>>>>> MalformedObjectNameException { char[] chars = new >>>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>>> "Test")); } } I'm not sure what it will do with your changes :-) >>>>> >>>>> It will fail with assert (if enabled). Or truncate the domain name, I >>>>> suppose. >>>>>> >>>>>> With that in mind I believe you should consider throwing >>>>>> InternalError - or IllegalArgumentException - instead of using >>>>>> 'assert' statements. >>>>> >>>>> This would probably be better. >>>>>> >>>>>> BTW have you run the JCK? >>>>> >>>>> Yes. All passed. I don't think JCK is testing for unrealistic values >>>>> :) I don't see how one could come up with a domain name 32767 >>>>> characters long. >>>> >>>> The proposed change has passed the CCC review. In case the domain name >>>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will >>>> be thrown. All the JMX tests and JCK are still passing. >>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>>> >>>>> -JB- >>>>>> >>>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>>>>> >>>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>>> >>>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>>> unnecessary since the store is to a byte field. >>>>>>> >>>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>>> >>>>>>>> 1644: the ? and : operators should be surrounded by spaces. There >>>>>>>> are other style issues, such as then statements on the same line >>>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>>> >>>>>>> I know but these style issue have been around since the file was >>>>>>> first committed. I didn't address them because it didn't feel right >>>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>>> >>>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>>> Jaroslav Bachorik wrote: >>>>>>>>> >>>>>>>>> Please, review the following change Issue : >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>>> situations when there are 10s of thousands ObjectNname instances >>>>>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to inspect the >>>>>>>>> object layout we can see this: Before optimization (JDK8u40): --- >>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>>> size: 40 bytes (estimated, the sample instance is not available) >>>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 short >>>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes external >>>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>>> instance which can translate to very interesting numbers on large >>>>>>>>> installations. To achieve this the domain name length is set to >>>>>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>>>>> performance purposes are encoded into one byte value (as proposed >>>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>>> JCK tests are passing after this change. Thanks, -JB- >> >> From eamonn at mcmanus.net Wed Aug 5 15:04:06 2015 From: eamonn at mcmanus.net (Eamonn McManus) Date: Wed, 5 Aug 2015 08:04:06 -0700 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55C2257B.1030500@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> <55C2257B.1030500@oracle.com> Message-ID: That makes me sad. The limit is clearly a detail of this particular implementation and should not be enshrined in the spec. ?amonn 2015-08-05 8:02 GMT-07:00 Jaroslav Bachorik : > On 5.8.2015 16:53, Eamonn McManus wrote: >> >> I would remove the spec changes about the limit on the domain length, >> which are a property of this particular implementation. It's perfectly >> reasonable to blow up if the domain length is > 536,870,911, but >> there's no reason for it to be in the spec. > > > Well, CCC board had a different opinion. That's why this limit is documented > in the spec. > > -JB- > > >> ?amonn >> >> >> 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik >> : >>> >>> Eamonn, Daniel, >>> >>> thanks for the comments. >>> >>> I've updated the webrev to address them. Also, I've added a test to >>> exercise >>> the boolean flag en-/decoding in ObjectName. >>> >>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 >>> >>> >>> Cheers, >>> >>> -JB- >>> >>> >>> On 4.8.2015 23:02, Daniel Fuchs wrote: >>>> >>>> >>>> Hi Jaroslav, >>>> >>>> 379 * This field encodes _domain_pattern, _property_list_pattern >>>> and >>>> 380 * _property_value_pattern booleans. >>>> >>>> If I'm not mistaken it also encodes the domain length. >>>> >>>> >>>> 1072 if ((l & FLAG_MASK) > 0 ) { >>>> >>>> Although it is not expected that 'l' could be negative, it might be >>>> better to write this test as: >>>> >>>> if ((l & FLAG_MASK) != 0 ) { >>>> >>>> (+ I agree with ?amonn that l is not a great parameter name - nice to >>>> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >>>> Jaroslav Bachorik wrote: >>>>> >>>>> >>>>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: >>>>>> >>>>>> >>>>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>>>> >>>>>>> >>>>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>>>> incompatibility, so it probably needs a CCC and some release notes. >>>>>>> For instance, this test passes with the current version of >>>>>>> ObjectName: public class StringLengthTest { final static int >>>>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>>>> public static void main(String[] args) throws >>>>>>> MalformedObjectNameException { char[] chars = new >>>>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>>>> "Test")); } } I'm not sure what it will do with your changes :-) >>>>>> >>>>>> >>>>>> It will fail with assert (if enabled). Or truncate the domain name, I >>>>>> suppose. >>>>>>> >>>>>>> >>>>>>> With that in mind I believe you should consider throwing >>>>>>> InternalError - or IllegalArgumentException - instead of using >>>>>>> 'assert' statements. >>>>>> >>>>>> >>>>>> This would probably be better. >>>>>>> >>>>>>> >>>>>>> BTW have you run the JCK? >>>>>> >>>>>> >>>>>> Yes. All passed. I don't think JCK is testing for unrealistic values >>>>>> :) I don't see how one could come up with a domain name 32767 >>>>>> characters long. >>>>> >>>>> >>>>> The proposed change has passed the CCC review. In case the domain name >>>>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will >>>>> be thrown. All the JMX tests and JCK are still passing. >>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>>>> >>>>>> >>>>>> -JB- >>>>>>> >>>>>>> >>>>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>>>> unnecessary since the store is to a byte field. >>>>>>>> >>>>>>>> >>>>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>>>> >>>>>>>>> >>>>>>>>> 1644: the ? and : operators should be surrounded by spaces. There >>>>>>>>> are other style issues, such as then statements on the same line >>>>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>>>> >>>>>>>> >>>>>>>> I know but these style issue have been around since the file was >>>>>>>> first committed. I didn't address them because it didn't feel right >>>>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>>>> >>>>>>>>> >>>>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>>>> Jaroslav Bachorik wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Please, review the following change Issue : >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>>>> situations when there are 10s of thousands ObjectNname instances >>>>>>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to inspect the >>>>>>>>>> object layout we can see this: Before optimization (JDK8u40): --- >>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>>>> size: 40 bytes (estimated, the sample instance is not available) >>>>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 short >>>>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes external >>>>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>>>> instance which can translate to very interesting numbers on large >>>>>>>>>> installations. To achieve this the domain name length is set to >>>>>>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>>>>>> performance purposes are encoded into one byte value (as proposed >>>>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>>>> JCK tests are passing after this change. Thanks, -JB- >>> >>> >>> > From joe.darcy at oracle.com Wed Aug 5 16:56:06 2015 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 5 Aug 2015 09:56:06 -0700 Subject: JDK 9 RFR of JDK-8133060: Problem list BasicLauncherTest until fix for JDK-8132648 propagates Message-ID: <55C24026.7000106@oracle.com> Hello, With the most recent integration of HotSpot into dev, we started seeing test failures of sun/tools/jhsdb/BasicLauncherTest.java The underlying problem has been fixed by JDK-8132648 in hs-rt, but until that fix propagates, the test should be problem listed in dev: diff -r 9bce83952890 test/ProblemList.txt --- a/test/ProblemList.txt Tue Aug 04 11:26:51 2015 -0700 +++ b/test/ProblemList.txt Wed Aug 05 09:55:27 2015 -0700 @@ -382,4 +382,7 @@ # 8064572 8060736 8062938 sun/jvmstat/monitor/MonitoredVm/CR6672135.java generic-all +# 8132648 +sun/tools/jhsdb/BasicLauncherTest.java macosx-all + ############################################################################ Thanks, -Joe From dmitry.samersoff at oracle.com Wed Aug 5 17:36:38 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 05 Aug 2015 20:36:38 +0300 Subject: JDK 9 RFR of JDK-8133060: Problem list BasicLauncherTest until fix for JDK-8132648 propagates In-Reply-To: <55C24026.7000106@oracle.com> References: <55C24026.7000106@oracle.com> Message-ID: <55C249A6.7030304@oracle.com> Joe, > +sun/tools/jhsdb/BasicLauncherTest.java macosx-all It also failing on windows-i586 so please add generic-all (Fixed in hs-rt as well) -Dmitry On 2015-08-05 19:56, joe darcy wrote: > Hello, > > With the most recent integration of HotSpot into dev, we started seeing > test failures of > > sun/tools/jhsdb/BasicLauncherTest.java > > The underlying problem has been fixed by JDK-8132648 in hs-rt, but until > that fix propagates, the test should be problem listed in dev: > > diff -r 9bce83952890 test/ProblemList.txt > --- a/test/ProblemList.txt Tue Aug 04 11:26:51 2015 -0700 > +++ b/test/ProblemList.txt Wed Aug 05 09:55:27 2015 -0700 > @@ -382,4 +382,7 @@ > # 8064572 8060736 8062938 > sun/jvmstat/monitor/MonitoredVm/CR6672135.java generic-all > > +# 8132648 > +sun/tools/jhsdb/BasicLauncherTest.java macosx-all > + > ############################################################################ > > > Thanks, > > -Joe -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From stuart.marks at oracle.com Wed Aug 5 19:39:31 2015 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 5 Aug 2015 12:39:31 -0700 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55C2257B.1030500@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> <55C2257B.1030500@oracle.com> Message-ID: <55C26673.7050900@oracle.com> On 8/5/15 8:02 AM, Jaroslav Bachorik wrote: > On 5.8.2015 16:53, Eamonn McManus wrote: >> I would remove the spec changes about the limit on the domain length, >> which are a property of this particular implementation. It's perfectly >> reasonable to blow up if the domain length is > 536,870,911, but >> there's no reason for it to be in the spec. > > Well, CCC board had a different opinion. That's why this limit is documented in > the spec. (Coming in a bit from left field... sorry if this is misplaced. Also, I didn't participate in that [Oracle-internal] CCC discussion.) If the maximum value were considered to be implementation-specific, then one could separate the issues into two parts: 1) the existence of a maximum domain name length, which if exceeded will cause MalformedObjectNameException; and 2) the actual value of that maximum length. If this approach were taken, the spec would say that there is an implementation-specific maximum value, which if exceeded would result in MalformedObjectNameException. The actual value for "this implementation" could then be moved into an @implNote section. Would that make sense? Now, this would raise an interoperability risk, if there were multiple JMX implementations, and they chose different maximum lengths. But y'all probably know better than me how to assess risk. It might be preferable to baking some number into the formal part of the specification. s'marks > > -JB- > >> ?amonn >> >> >> 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik : >>> Eamonn, Daniel, >>> >>> thanks for the comments. >>> >>> I've updated the webrev to address them. Also, I've added a test to exercise >>> the boolean flag en-/decoding in ObjectName. >>> >>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 >>> >>> >>> Cheers, >>> >>> -JB- >>> >>> >>> On 4.8.2015 23:02, Daniel Fuchs wrote: >>>> >>>> Hi Jaroslav, >>>> >>>> 379 * This field encodes _domain_pattern, _property_list_pattern >>>> and >>>> 380 * _property_value_pattern booleans. >>>> >>>> If I'm not mistaken it also encodes the domain length. >>>> >>>> >>>> 1072 if ((l & FLAG_MASK) > 0 ) { >>>> >>>> Although it is not expected that 'l' could be negative, it might be >>>> better to write this test as: >>>> >>>> if ((l & FLAG_MASK) != 0 ) { >>>> >>>> (+ I agree with ?amonn that l is not a great parameter name - nice to >>>> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >>>> Jaroslav Bachorik wrote: >>>>> >>>>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: >>>>>> >>>>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>>>> >>>>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>>>> incompatibility, so it probably needs a CCC and some release notes. >>>>>>> For instance, this test passes with the current version of >>>>>>> ObjectName: public class StringLengthTest { final static int >>>>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>>>> public static void main(String[] args) throws >>>>>>> MalformedObjectNameException { char[] chars = new >>>>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>>>> "Test")); } } I'm not sure what it will do with your changes :-) >>>>>> >>>>>> It will fail with assert (if enabled). Or truncate the domain name, I >>>>>> suppose. >>>>>>> >>>>>>> With that in mind I believe you should consider throwing >>>>>>> InternalError - or IllegalArgumentException - instead of using >>>>>>> 'assert' statements. >>>>>> >>>>>> This would probably be better. >>>>>>> >>>>>>> BTW have you run the JCK? >>>>>> >>>>>> Yes. All passed. I don't think JCK is testing for unrealistic values >>>>>> :) I don't see how one could come up with a domain name 32767 >>>>>> characters long. >>>>> >>>>> The proposed change has passed the CCC review. In case the domain name >>>>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will >>>>> be thrown. All the JMX tests and JCK are still passing. >>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>>>> >>>>>> -JB- >>>>>>> >>>>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>>>>>> >>>>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>>>> >>>>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>>>> unnecessary since the store is to a byte field. >>>>>>>> >>>>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>>>> >>>>>>>>> 1644: the ? and : operators should be surrounded by spaces. There >>>>>>>>> are other style issues, such as then statements on the same line >>>>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>>>> >>>>>>>> I know but these style issue have been around since the file was >>>>>>>> first committed. I didn't address them because it didn't feel right >>>>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>>>> >>>>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>>>> Jaroslav Bachorik wrote: >>>>>>>>>> >>>>>>>>>> Please, review the following change Issue : >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>>>> situations when there are 10s of thousands ObjectNname instances >>>>>>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to inspect the >>>>>>>>>> object layout we can see this: Before optimization (JDK8u40): --- >>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>>>> size: 40 bytes (estimated, the sample instance is not available) >>>>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 short >>>>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes external >>>>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>>>> instance which can translate to very interesting numbers on large >>>>>>>>>> installations. To achieve this the domain name length is set to >>>>>>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>>>>>> performance purposes are encoded into one byte value (as proposed >>>>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>>>> JCK tests are passing after this change. Thanks, -JB- >>> >>> > From eamonn at mcmanus.net Wed Aug 5 19:50:27 2015 From: eamonn at mcmanus.net (Eamonn McManus) Date: Wed, 5 Aug 2015 12:50:27 -0700 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55C26673.7050900@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> <55C2257B.1030500@oracle.com> <55C26673.7050900@oracle.com> Message-ID: That makes sense to me. I think the phrase could even be as vague as "or the name exceeds one of the implementation's limits" without saying what those limits might be. That would leave having reasonable limits as a quality-of-implementation issue, as it should be. The acid test is whether we would want to add a check to the JCK. I would argue not, so a vague specification is appropriate. ?amonn 2015-08-05 12:39 GMT-07:00 Stuart Marks : > On 8/5/15 8:02 AM, Jaroslav Bachorik wrote: >> >> On 5.8.2015 16:53, Eamonn McManus wrote: >>> >>> I would remove the spec changes about the limit on the domain length, >>> which are a property of this particular implementation. It's perfectly >>> reasonable to blow up if the domain length is > 536,870,911, but >>> there's no reason for it to be in the spec. >> >> >> Well, CCC board had a different opinion. That's why this limit is >> documented in >> the spec. > > > (Coming in a bit from left field... sorry if this is misplaced. Also, I > didn't participate in that [Oracle-internal] CCC discussion.) > > If the maximum value were considered to be implementation-specific, then one > could separate the issues into two parts: > > 1) the existence of a maximum domain name length, which if exceeded will > cause MalformedObjectNameException; and > > 2) the actual value of that maximum length. > > If this approach were taken, the spec would say that there is an > implementation-specific maximum value, which if exceeded would result in > MalformedObjectNameException. > > The actual value for "this implementation" could then be moved into an > @implNote section. > > Would that make sense? > > Now, this would raise an interoperability risk, if there were multiple JMX > implementations, and they chose different maximum lengths. But y'all > probably know better than me how to assess risk. It might be preferable to > baking some number into the formal part of the specification. > > s'marks > > >> >> -JB- >> >>> ?amonn >>> >>> >>> 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik >>> : >>>> >>>> Eamonn, Daniel, >>>> >>>> thanks for the comments. >>>> >>>> I've updated the webrev to address them. Also, I've added a test to >>>> exercise >>>> the boolean flag en-/decoding in ObjectName. >>>> >>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 >>>> >>>> >>>> Cheers, >>>> >>>> -JB- >>>> >>>> >>>> On 4.8.2015 23:02, Daniel Fuchs wrote: >>>>> >>>>> >>>>> Hi Jaroslav, >>>>> >>>>> 379 * This field encodes _domain_pattern, >>>>> _property_list_pattern >>>>> and >>>>> 380 * _property_value_pattern booleans. >>>>> >>>>> If I'm not mistaken it also encodes the domain length. >>>>> >>>>> >>>>> 1072 if ((l & FLAG_MASK) > 0 ) { >>>>> >>>>> Although it is not expected that 'l' could be negative, it might be >>>>> better to write this test as: >>>>> >>>>> if ((l & FLAG_MASK) != 0 ) { >>>>> >>>>> (+ I agree with ?amonn that l is not a great parameter name - nice to >>>>> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >>>>> Jaroslav Bachorik wrote: >>>>>> >>>>>> >>>>>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: >>>>>>> >>>>>>> >>>>>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>>>>> incompatibility, so it probably needs a CCC and some release notes. >>>>>>>> For instance, this test passes with the current version of >>>>>>>> ObjectName: public class StringLengthTest { final static int >>>>>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>>>>> public static void main(String[] args) throws >>>>>>>> MalformedObjectNameException { char[] chars = new >>>>>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>>>>> "Test")); } } I'm not sure what it will do with your changes >>>>>>>> :-) >>>>>>> >>>>>>> >>>>>>> It will fail with assert (if enabled). Or truncate the domain name, I >>>>>>> suppose. >>>>>>>> >>>>>>>> >>>>>>>> With that in mind I believe you should consider throwing >>>>>>>> InternalError - or IllegalArgumentException - instead of using >>>>>>>> 'assert' statements. >>>>>>> >>>>>>> >>>>>>> This would probably be better. >>>>>>>> >>>>>>>> >>>>>>>> BTW have you run the JCK? >>>>>>> >>>>>>> >>>>>>> Yes. All passed. I don't think JCK is testing for unrealistic values >>>>>>> :) I don't see how one could come up with a domain name 32767 >>>>>>> characters long. >>>>>> >>>>>> >>>>>> The proposed change has passed the CCC review. In case the domain name >>>>>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will >>>>>> be thrown. All the JMX tests and JCK are still passing. >>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>>>>> >>>>>>> >>>>>>> -JB- >>>>>>>> >>>>>>>> >>>>>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>>>>> unnecessary since the store is to a byte field. >>>>>>>>> >>>>>>>>> >>>>>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 1644: the ? and : operators should be surrounded by spaces. There >>>>>>>>>> are other style issues, such as then statements on the same line >>>>>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>>>>> >>>>>>>>> >>>>>>>>> I know but these style issue have been around since the file was >>>>>>>>> first committed. I didn't address them because it didn't feel right >>>>>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>>>>> Jaroslav Bachorik wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Please, review the following change Issue : >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>>>>> situations when there are 10s of thousands ObjectNname instances >>>>>>>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to inspect the >>>>>>>>>>> object layout we can see this: Before optimization (JDK8u40): --- >>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>>>>> size: 40 bytes (estimated, the sample instance is not available) >>>>>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 short >>>>>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes external >>>>>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>>>>> instance which can translate to very interesting numbers on large >>>>>>>>>>> installations. To achieve this the domain name length is set to >>>>>>>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>>>>>>> performance purposes are encoded into one byte value (as proposed >>>>>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>>>>> JCK tests are passing after this change. Thanks, -JB- >>>> >>>> >>>> >> > From me at basilcrow.com Thu Aug 6 15:46:28 2015 From: me at basilcrow.com (Basil Crow) Date: Thu, 6 Aug 2015 08:46:28 -0700 Subject: Unable to create hprof heap dump from core file with jmap on Oracle Solaris In-Reply-To: <5520EBF7.6030808@oracle.com> References: <5520EBF7.6030808@oracle.com> Message-ID: On Sun, Apr 5, 2015 at 1:01 AM, Dmitry Samersoff wrote: > Basil, > > jmap uses SA backend to extract data from coredump and it doesn't > support lambda. > > See JDK-8073604 > > -Dmitry Hi Dmitry, While I'm pleased to see that JDK-8044416 (of which JDK-8073604 is a duplicate) was integrated and backported to 8u60, I'm sad to report that I am still unable to meaningfully work with hprof heap dumps generated from core files when lambdas are in use. Here is a minimal test case running on Oracle Solaris: $ uname -a SunOS solaris 5.11 11.2 i86pc i386 i86pc $ java -version java version "1.8.0_60-ea" Java(TM) SE Runtime Environment (build 1.8.0_60-ea-b25) Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode) $ cat LambdaSleep.java import java.util.Arrays; public class LambdaSleep { public static void main(String[] args) throws InterruptedException { String[] words = new String[] { "longer", "short" }; Arrays.sort(words, (first, second) -> Integer.compare(first.length(), second.length())); Thread.sleep(300000); } } $ javac LambdaSleep.java $ java LambdaSleep & [1] 4528 I took two hprof heap dumps of this program. The first, live4528.hprof, comes from the live process: $ jmap -dump:format=b,file=live4528.hprof 4528 Dumping heap to /export/home/basil/live4528.hprof ... Heap dump file created The second, core4528.hprof, comes from a core dump: $ gcore 4528 gcore: core.4528 dumped $ jmap -dump:format=b,file=core4528.hprof /opt/jdk/bin/java core.4528 Attaching to core core.4528 from executable /opt/jdk/bin/java, please wait... Debugger attached successfully. Server compiler detected. JVM version is 25.60-b23 Dumping heap to core4528.hprof ... Heap dump file created With the fix for JDK-8044416 in place, the heap dump is generated without any exceptions (which is an improvement), but it is still not usable in any meaningful way. While live4528.hprof loads in Eclipse MAT, core4528.hprof fails to load in Eclipse MAT with the following exception: !ENTRY org.eclipse.core.jobs 4 2 2015-08-06 08:32:45.883 !MESSAGE An internal error occurred during: "Parsing heap dump from 'core4528.hprof'". !STACK 0 java.lang.NullPointerException at org.eclipse.mat.hprof.HprofParserHandlerImpl.resolveClassHierarchy(HprofParserHandlerImpl.java:587) at org.eclipse.mat.hprof.Pass2Parser.readInstanceDump(Pass2Parser.java:205) at org.eclipse.mat.hprof.Pass2Parser.readDumpSegments(Pass2Parser.java:159) at org.eclipse.mat.hprof.Pass2Parser.read(Pass2Parser.java:89) at org.eclipse.mat.hprof.HprofIndexBuilder.fill(HprofIndexBuilder.java:94) at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.parse(SnapshotFactoryImpl.java:222) at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.openSnapshot(SnapshotFactoryImpl.java:126) at org.eclipse.mat.snapshot.SnapshotFactory.openSnapshot(SnapshotFactory.java:145) at org.eclipse.mat.ui.snapshot.ParseHeapDumpJob.run(ParseHeapDumpJob.java:83) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) While both live4528.hprof and core4528.hprof load in VisualVM, attempting to find objects by retained size in VisualVM hangs on "Computing retained sizes..." for core4528.hprof, while this operation succeeds for live4528.hprof. When lambdas are not in use, the above problem is not manifest. Clearly there is a difference in the hprof heap dump generated from the core file compared to the one generated from the live process. An update would be appreciated, as this bug greatly hampers our ability to debug memory leaks in production. Thanks, Basil From serguei.spitsyn at oracle.com Fri Aug 7 04:44:30 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 6 Aug 2015 21:44:30 -0700 Subject: RFR (XS) 8060245: update bsd version of jhelper.d to be in sync with the fix of 8009204 on solaris In-Reply-To: <55BF360D.7000201@oracle.com> References: <55BF360D.7000201@oracle.com> Message-ID: <55C437AE.90408@oracle.com> Hi Jaroslav, Thank you for reviewing it! This fix has been pushed now. Thanks, Serguei On 8/3/15 02:36, Jaroslav Bachorik wrote: > Hi Serguey, > > reviving this old RFR. > > Looks good (just update the copyright year to 2015)! > > -JB- > > > Please, review the fix for: > > https://bugs.openjdk.java.net/browse/JDK-8060245 > > > > > > Open webrev: > > > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8060245-dtrace.1/ > > > > > > Summary: > > > > The fix of 8009204 for jhelper.d was applied to the Solaris > > version only but the bsd version must match it too. > > > > > > Testing: > > N/A: The jhelper.d is not used on bsd yet > > > > > > Thanks, > > Serguei From staffan.larsen at oracle.com Fri Aug 7 06:57:11 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Aug 2015 08:57:11 +0200 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent Message-ID: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> Please review the following changes to remove the hprof JVMTI agent. There are changes in three different repositories. All tests that used the hprof agent has been removed in previous changesets. Note: This does not remove the ability of the Hotspot VM to output heap dumps in the hprof format. bug: https://bugs.openjdk.java.net/browse/JDK-8046661 top-level changes: http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ hotspot changes: http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ Thanks, /Staffan -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Fri Aug 7 09:24:14 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 07 Aug 2015 02:24:14 -0700 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> Message-ID: <55C4793E.2050608@oracle.com> Hi Staffan, Looks good. I'm re-posting the same question in this review: Q1: Should the folder jdk/src/demo/share/jvmti/java_crw_demo also be removed? Thanks, Serguei On 8/6/15 11:57 PM, Staffan Larsen wrote: > Please review the following changes to remove the hprof JVMTI agent. > There are changes in three different repositories. All tests that used > the hprof agent has been removed in previous changesets. > > Note: This does not remove the ability of the Hotspot VM to output > heap dumps in the hprof format. > > bug: https://bugs.openjdk.java.net/browse/JDK-8046661 > > top-level changes: > http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ > > jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ > > hotspot changes: > http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ > > > Thanks, > /Staffan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Fri Aug 7 09:27:24 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Aug 2015 11:27:24 +0200 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <55C4793E.2050608@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> <55C4793E.2050608@oracle.com> Message-ID: <85643090-BD65-48B5-B63C-F97D48304672@oracle.com> > On 7 aug 2015, at 11:24, serguei.spitsyn at oracle.com wrote: > > Hi Staffan, > > Looks good. Thanks! > > I'm re-posting the same question in this review: > Q1: Should the folder jdk/src/demo/share/jvmti/java_crw_demo also be removed? No, it is used by some of the demos (mtrace, minst, etc). But they statically link the code so the lib is no longer needed. /Staffan > > Thanks, > Serguei > > > On 8/6/15 11:57 PM, Staffan Larsen wrote: >> Please review the following changes to remove the hprof JVMTI agent. There are changes in three different repositories. All tests that used the hprof agent has been removed in previous changesets. >> >> Note: This does not remove the ability of the Hotspot VM to output heap dumps in the hprof format. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8046661 >> >> top-level changes: http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ >> jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ >> hotspot changes: http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ >> >> Thanks, >> /Staffan >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.samersoff at oracle.com Fri Aug 7 09:31:16 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 07 Aug 2015 12:31:16 +0300 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <55C4793E.2050608@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> <55C4793E.2050608@oracle.com> Message-ID: <55C47AE4.4040507@oracle.com> Serguei, > Q1: Should the folder jdk/src/demo/share/jvmti/java_crw_demo also be > removed? Yes. See https://bugs.openjdk.java.net/browse/JDK-8041639 -Dmitry On 2015-08-07 12:24, serguei.spitsyn at oracle.com wrote: > Hi Staffan, > > Looks good. > > I'm re-posting the same question in this review: > Q1: Should the folder jdk/src/demo/share/jvmti/java_crw_demo also be > removed? > > Thanks, > Serguei > > > On 8/6/15 11:57 PM, Staffan Larsen wrote: >> Please review the following changes to remove the hprof JVMTI agent. >> There are changes in three different repositories. All tests that >> used the hprof agent has been removed in previous changesets. >> >> Note: This does not remove the ability of the Hotspot VM to output >> heap dumps in the hprof format. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8046661 >> >> top-level changes: >> http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ >> >> jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ >> >> hotspot changes: >> http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ >> >> >> Thanks, >> /Staffan >> >> >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From serguei.spitsyn at oracle.com Fri Aug 7 09:33:46 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 07 Aug 2015 02:33:46 -0700 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <55C47AE4.4040507@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> <55C4793E.2050608@oracle.com> <55C47AE4.4040507@oracle.com> Message-ID: <55C47B7A.9030201@oracle.com> Dmitry, On 8/7/15 2:31 AM, Dmitry Samersoff wrote: > Serguei, >> Q1: Should the folder jdk/src/demo/share/jvmti/java_crw_demo also be >> removed? > Yes. See > > https://bugs.openjdk.java.net/browse/JDK-8041639 Thank you for the comment. But, please, see also the related comment from Staffan. Thanks, Serguei > > -Dmitry > > On 2015-08-07 12:24, serguei.spitsyn at oracle.com wrote: >> Hi Staffan, >> >> Looks good. >> >> I'm re-posting the same question in this review: >> Q1: Should the folder jdk/src/demo/share/jvmti/java_crw_demo also be >> removed? >> >> Thanks, >> Serguei >> >> >> On 8/6/15 11:57 PM, Staffan Larsen wrote: >>> Please review the following changes to remove the hprof JVMTI agent. >>> There are changes in three different repositories. All tests that >>> used the hprof agent has been removed in previous changesets. >>> >>> Note: This does not remove the ability of the Hotspot VM to output >>> heap dumps in the hprof format. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8046661 >>> >>> top-level changes: >>> http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ >>> >>> jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ >>> >>> hotspot changes: >>> http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ >>> >>> >>> Thanks, >>> /Staffan >>> >>> >>> > From serguei.spitsyn at oracle.com Fri Aug 7 09:34:16 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 07 Aug 2015 02:34:16 -0700 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <85643090-BD65-48B5-B63C-F97D48304672@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> <55C4793E.2050608@oracle.com> <85643090-BD65-48B5-B63C-F97D48304672@oracle.com> Message-ID: <55C47B98.5080506@oracle.com> Ok. Thanks, Staffan! Serguei On 8/7/15 2:27 AM, Staffan Larsen wrote: > >> On 7 aug 2015, at 11:24, serguei.spitsyn at oracle.com >> wrote: >> >> Hi Staffan, >> >> Looks good. > > Thanks! > >> >> I'm re-posting the same question in this review: >> Q1: Should the folder jdk/src/demo/share/jvmti/java_crw_demo also >> be removed? > > No, it is used by some of the demos (mtrace, minst, etc). But they > statically link the code so the lib is no longer needed. > > /Staffan > >> >> Thanks, >> Serguei >> >> >> On 8/6/15 11:57 PM, Staffan Larsen wrote: >>> Please review the following changes to remove the hprof JVMTI agent. >>> There are changes in three different repositories. All tests that >>> used the hprof agent has been removed in previous changesets. >>> >>> Note: This does not remove the ability of the Hotspot VM to output >>> heap dumps in the hprof format. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8046661 >>> >>> top-level changes: >>> http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ >>> >>> jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ >>> >>> hotspot changes: >>> http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ >>> >>> >>> Thanks, >>> /Staffan >>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Fri Aug 7 10:19:20 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 07 Aug 2015 03:19:20 -0700 Subject: RFR(S) 8033577: Message-ID: <55C48628.3000407@oracle.com> Please, review the jdk 9 fix for: https://bugs.openjdk.java.net/browse/JDK-8033577 Open webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8033577-dtrace-parfait1.1/ Summary: The fix includes: - the parfait fixes listed in the bug report and - a couple of adjustments for the bsd version to match the solaris version Testing: The JPRT build Thanks, Serguei From serguei.spitsyn at oracle.com Fri Aug 7 10:21:46 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 07 Aug 2015 03:21:46 -0700 Subject: RFR(S) 8033577: [parfait] warnings from b128 for hotspot/src/os/solaris/dtrace: Unportable format string argument mismatch In-Reply-To: <55C48628.3000407@oracle.com> References: <55C48628.3000407@oracle.com> Message-ID: <55C486BA.9030005@oracle.com> Re-posting this with the updated subject line. Please, review the jdk 9 fix for: https://bugs.openjdk.java.net/browse/JDK-8033577 Open webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8033577-dtrace-parfait1.1/ Summary: The fix includes: - the parfait fixes listed in the bug report and - a couple of adjustments for the bsd version to match the solaris version Testing: The JPRT build Thanks, Serguei From serguei.spitsyn at oracle.com Fri Aug 7 10:25:36 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 07 Aug 2015 03:25:36 -0700 Subject: RFR(S) 8080401: Uninitialised variable in hotspot/src/os/solaris/dtrace/ Message-ID: <55C487A0.9040904@oracle.com> Please, review the jdk 9 fix for: https://bugs.openjdk.java.net/browse/JDK-8080401 Open webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8080401-dtrace-parfait2.1/ Summary: The fix includes the parfait issues listed in the bug report. Testing: The JPRT build Thanks, Serguei From staffan.larsen at oracle.com Fri Aug 7 11:22:52 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Aug 2015 13:22:52 +0200 Subject: RFR(S) 8080401: Uninitialised variable in hotspot/src/os/solaris/dtrace/ In-Reply-To: <55C487A0.9040904@oracle.com> References: <55C487A0.9040904@oracle.com> Message-ID: <6A4AA882-4BAC-47D1-A0B5-A522B3096EDC@oracle.com> Looks good! Thanks, /Staffan > On 7 aug 2015, at 12:25, serguei.spitsyn at oracle.com wrote: > > > Please, review the jdk 9 fix for: > https://bugs.openjdk.java.net/browse/JDK-8080401 > > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8080401-dtrace-parfait2.1/ > > > Summary: > > The fix includes the parfait issues listed in the bug report. > > > Testing: > The JPRT build > > > Thanks, > Serguei From staffan.larsen at oracle.com Fri Aug 7 11:23:46 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Aug 2015 13:23:46 +0200 Subject: RFR(S) 8033577: [parfait] warnings from b128 for hotspot/src/os/solaris/dtrace: Unportable format string argument mismatch In-Reply-To: <55C486BA.9030005@oracle.com> References: <55C48628.3000407@oracle.com> <55C486BA.9030005@oracle.com> Message-ID: Looks good! Thanks, /Staffan > On 7 aug 2015, at 12:21, serguei.spitsyn at oracle.com wrote: > > Re-posting this with the updated subject line. > > Please, review the jdk 9 fix for: > https://bugs.openjdk.java.net/browse/JDK-8033577 > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8033577-dtrace-parfait1.1/ > > > Summary: > The fix includes: > - the parfait fixes listed in the bug report and > - a couple of adjustments for the bsd version to match the solaris > version > > > Testing: > The JPRT build > > > Thanks, > Serguei > > > From serguei.spitsyn at oracle.com Fri Aug 7 11:25:48 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Aug 2015 04:25:48 -0700 Subject: RFR(S) 8080401: Uninitialised variable in hotspot/src/os/solaris/dtrace/ In-Reply-To: <6A4AA882-4BAC-47D1-A0B5-A522B3096EDC@oracle.com> References: <55C487A0.9040904@oracle.com> <6A4AA882-4BAC-47D1-A0B5-A522B3096EDC@oracle.com> Message-ID: <55C495BC.6070503@oracle.com> Thanks, Staffan! Serguei On 8/7/15 04:22, Staffan Larsen wrote: > Looks good! > > Thanks, > /Staffan > >> On 7 aug 2015, at 12:25, serguei.spitsyn at oracle.com wrote: >> >> >> Please, review the jdk 9 fix for: >> https://bugs.openjdk.java.net/browse/JDK-8080401 >> >> >> Open webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8080401-dtrace-parfait2.1/ >> >> >> Summary: >> >> The fix includes the parfait issues listed in the bug report. >> >> >> Testing: >> The JPRT build >> >> >> Thanks, >> Serguei -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Fri Aug 7 11:26:03 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Aug 2015 04:26:03 -0700 Subject: RFR(S) 8033577: [parfait] warnings from b128 for hotspot/src/os/solaris/dtrace: Unportable format string argument mismatch In-Reply-To: References: <55C48628.3000407@oracle.com> <55C486BA.9030005@oracle.com> Message-ID: <55C495CB.4040203@oracle.com> Thanks, Staffan! Serguei On 8/7/15 04:23, Staffan Larsen wrote: > Looks good! > > Thanks, > /Staffan > >> On 7 aug 2015, at 12:21, serguei.spitsyn at oracle.com wrote: >> >> Re-posting this with the updated subject line. >> >> Please, review the jdk 9 fix for: >> https://bugs.openjdk.java.net/browse/JDK-8033577 >> >> Open webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8033577-dtrace-parfait1.1/ >> >> >> Summary: >> The fix includes: >> - the parfait fixes listed in the bug report and >> - a couple of adjustments for the bsd version to match the solaris >> version >> >> >> Testing: >> The JPRT build >> >> >> Thanks, >> Serguei >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.samersoff at oracle.com Fri Aug 7 12:11:22 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 07 Aug 2015 15:11:22 +0300 Subject: RFR(S) 8033577: [parfait] warnings from b128 for hotspot/src/os/solaris/dtrace: Unportable format string argument mismatch In-Reply-To: <55C486BA.9030005@oracle.com> References: <55C48628.3000407@oracle.com> <55C486BA.9030005@oracle.com> Message-ID: <55C4A06A.3030203@oracle.com> Looks good for me. On 2015-08-07 13:21, serguei.spitsyn at oracle.com wrote: > Re-posting this with the updated subject line. > > Please, review the jdk 9 fix for: > https://bugs.openjdk.java.net/browse/JDK-8033577 > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8033577-dtrace-parfait1.1/ > > > > Summary: > The fix includes: > - the parfait fixes listed in the bug report and > - a couple of adjustments for the bsd version to match the solaris > version > > > Testing: > The JPRT build > > > Thanks, > Serguei > > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From serguei.spitsyn at oracle.com Fri Aug 7 13:33:41 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Aug 2015 06:33:41 -0700 Subject: RFR(S) 8033577: [parfait] warnings from b128 for hotspot/src/os/solaris/dtrace: Unportable format string argument mismatch In-Reply-To: <55C4A06A.3030203@oracle.com> References: <55C48628.3000407@oracle.com> <55C486BA.9030005@oracle.com> <55C4A06A.3030203@oracle.com> Message-ID: <55C4B3B5.8060606@oracle.com> Thanks, Dmitry! May I ask you to look at another parfait fix (8080401)? I hope, it is simple enough. Thanks, Serguei On 8/7/15 05:11, Dmitry Samersoff wrote: > Looks good for me. > > On 2015-08-07 13:21, serguei.spitsyn at oracle.com wrote: >> Re-posting this with the updated subject line. >> >> Please, review the jdk 9 fix for: >> https://bugs.openjdk.java.net/browse/JDK-8033577 >> >> Open webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8033577-dtrace-parfait1.1/ >> >> >> >> Summary: >> The fix includes: >> - the parfait fixes listed in the bug report and >> - a couple of adjustments for the bsd version to match the solaris >> version >> >> >> Testing: >> The JPRT build >> >> >> Thanks, >> Serguei >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.samersoff at oracle.com Fri Aug 7 13:45:49 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 07 Aug 2015 16:45:49 +0300 Subject: RFR(S) 8080401: Uninitialised variable in hotspot/src/os/solaris/dtrace/ In-Reply-To: <55C487A0.9040904@oracle.com> References: <55C487A0.9040904@oracle.com> Message-ID: <55C4B68D.6040501@oracle.com> Serguei, Looks good for me. It might be better to use (uint64_t) -1 for initialization of pc_diff in libjvm_db.c -Dmitry On 2015-08-07 13:25, serguei.spitsyn at oracle.com wrote: > > Please, review the jdk 9 fix for: > https://bugs.openjdk.java.net/browse/JDK-8080401 > > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8080401-dtrace-parfait2.1/ > > > > Summary: > > The fix includes the parfait issues listed in the bug report. > > > Testing: > The JPRT build > > > Thanks, > Serguei -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From serguei.spitsyn at oracle.com Fri Aug 7 13:53:21 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Aug 2015 06:53:21 -0700 Subject: RFR(S) 8080401: Uninitialised variable in hotspot/src/os/solaris/dtrace/ In-Reply-To: <55C4B68D.6040501@oracle.com> References: <55C487A0.9040904@oracle.com> <55C4B68D.6040501@oracle.com> Message-ID: <55C4B851.1040908@oracle.com> On 8/7/15 06:45, Dmitry Samersoff wrote: > Serguei, > > Looks good for me. Thanks a lot! > > It might be better to use (uint64_t) -1 for initialization of pc_diff in > libjvm_db.c Thank you for the suggestion. It'd work but not that important. :) Thanks, Serguei > > -Dmitry > > On 2015-08-07 13:25, serguei.spitsyn at oracle.com wrote: >> Please, review the jdk 9 fix for: >> https://bugs.openjdk.java.net/browse/JDK-8080401 >> >> >> Open webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8080401-dtrace-parfait2.1/ >> >> >> >> Summary: >> >> The fix includes the parfait issues listed in the bug report. >> >> >> Testing: >> The JPRT build >> >> >> Thanks, >> Serguei > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yumin.qi at oracle.com Fri Aug 7 16:48:18 2015 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 07 Aug 2015 09:48:18 -0700 Subject: RFR: 8130115: REDO - Reduce Symbol::_identity_hash to 2 bytes In-Reply-To: <55B70538.1070109@oracle.com> References: <55A69A5F.3070504@oracle.com> <55B70538.1070109@oracle.com> Message-ID: <55C4E151.4020805@oracle.com> Ioi, I am trying to add a test case in SA for the testing as you mentioned. The easy part is adding a simple SA Tool (SymbolsInfo.java) to get the Symbol information but encountered a problem as: In the testing java process (1), create (spawn) another java process(2), which will run SA (SymbolsInfo) and attach back to the process (1). It failed due to time out waiting for response from target(1). I am investigating and trying to find a solution. It may have issue for such case. Webrev (Note: in the webrev, WhitBox.java, white_box.cpp, SymbolsInfo.java and IdentityHashForSymbols.java added to previous version webrev01) http://cr.openjdk.java.net/~minqi/8130115/webrev02/ Any one has comments how to solve the problem? Following are the two processes, you can see process 2) has parent as process 1): ($WS, $TEST are for real locations on my host machine) 1) 25939 25807 19 09:32 pts/1 00:00:00 $MYJDK/bin/java -Dtest.src=$WS/hotspot/test/serviceability/sa -Dtest.src.path=$WS/hotspot/test/serviceability/sa:$WS/hotspot/test/testlibrary:$WS/test/lib -Dtest.classes=$TEST/JTwork/classes/serviceability/sa -Dtest.class.path=$TEST/JTwork/classes/serviceability/sa:$TEST/JTwork/classes/testlibrary:$TEST/test/lib -Dtest.vm.opts= -Dtest.tool.vm.opts= -Dtest.compiler.opts= -Dtest.java.opts= -Dtest.jdk=$MYJDK -Dcompile.jdk=$MYJDK -Dtest.timeout.factor=1.0 -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI com.sun.javatest.regtest.agent.MainWrapper $TEST/JTwork/classes/serviceability/sa/IdentityHashForSymbols.jta 2) 25976 25939 63 09:32 pts/1 00:00:03 $MYJDK/bin/java -cp $TEST/jtreg/lib/javatest.jar:$TEST/jtreg/lib/jtreg.jar:$TEST/JTwork/classes/serviceability/sa:$WS/hotspot/test/serviceability/sa:$TEST/JTwork/classes/testlibrary:$TEST/test/lib sun.jvm.hotspot.tools.SymbolsInfo 25939 Thanks Yumin On 7/27/2015 9:29 PM, Ioi Lam wrote: > Hi Yumin, > > The C code changes look good to me. > > I am a little concerned about the Java code's calculation of > identityHash: > > Java version: > 86 public int identityHash() { > 87 long addr_value = getAddress().asLongValue(); > 88 int addr_bits = (int)(addr_value >> > (VM.getVM().getLogMinObjAlignmentInBytes() + 3)); > 89 int length = (int)getLength(); > 90 int byte0 = getByteAt(0); > 91 int byte1 = getByteAt(1); > 92 int id_hash = (int)(0xffff & idHash.getValue(this.addr)); > 93 return id_hash | > 94 ((addr_bits ^ (length << 8) ^ ((byte0 << 8) | byte1)) > << 16); > 95 } > > C version: > 148 unsigned identity_hash() { > 149 unsigned addr_bits = (unsigned)((uintptr_t)this >> > (LogMinObjAlignmentInBytes + 3)); > 150 return (unsigned)_identity_hash | > 151 ((addr_bits ^ (_length << 8) ^ (( _body[0] << 8) | > _body[1])) << 16); > 152 } > > The main problem is to correctly emulate the C unsigned operations in > the Java code. I've eyeballed the code and it seems correct, but I am > wondering if you have actually tested and verified that the Java > version indeed returns the same value as the C code? A unit test case > would be good: > > * Add a new test in hotspot/agent/test > * Get a few Symbols (e.g., call > sun.jvm.hotspot.runtime.VM.getSymbolTable and iterate over the first > 1000 Symbols) > * For each Symbol, call its Symbol.identityHash() method > * Add a new whitebox API to return the C version of the > identity_hash() value > * Check if the C value is the same as the Java value > > Please run the test on all platforms (both 32-bit and 64-bit, and all > OSes). > > Thanks > - Ioi > > > On 7/15/15 10:37 AM, Yumin Qi wrote: >> Hi, >> >> This is redo for bug 8087143, in that push, it caused failure on >> Serviceability Agent failed to get type for "_identity_hash": >> mistakenly used JShortField for it, but in fact it still is >> CIntegerField. In this change, besides of the original change in >> hotspot/src, I add code to calculate identity_hash in hotspot/agent >> based on the changed in hotspot. >> >> Old webrev for 8087143: >> bug: https://bugs.openjdk.java.net/browse/JDK-8087143 >> webrev: http://cr.openjdk.java.net/~minqi/8087143/webrev03/ >> >> Summary: _identity_hash is an integer in Symbol (SymbolBase), it is >> used to compute hash bucket index by modulus division of table size. >> Currently in hotspot, no table size is more than 65535 so we can use >> short instead. For case with table size over 65535 we can use the >> first two bytes of symbol data to be as the upper 16 bits for the >> calculation but rare cases. >> >> New webrev for 8130115: >> bug: https://bugs.openjdk.java.net/browse/JDK-8130115 >> webrev: http://cr.openjdk.java.net/~minqi/8130115/webrev01/ >> >> >> Tests: JPRT, SA manual tests, -atk quick, jtreg hotspot/runtime >> Also internal large application used for hashtable data analysis --- >> the No. of loaded classes is big(over 19K), and tested with different >> bucket sizes including over 65535 to see the new algorithm for >> identity_hash calculation, result shows the consistency before and >> after the fix. >> >> Thanks >> Yumin > From mandy.chung at oracle.com Sun Aug 9 03:13:10 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Sat, 8 Aug 2015 20:13:10 -0700 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> Message-ID: <93E8D7EA-FA2B-4987-A901-49C758A9764E@oracle.com> > On Aug 6, 2015, at 11:57 PM, Staffan Larsen wrote: > > Please review the following changes to remove the hprof JVMTI agent. There are changes in three different repositories. All tests that used the hprof agent has been removed in previous changesets. > > Note: This does not remove the ability of the Hotspot VM to output heap dumps in the hprof format. > > bug: https://bugs.openjdk.java.net/browse/JDK-8046661 > > top-level changes: http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ > jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ > hotspot changes: http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ > This patch looks fine to me. A minor comment: src/java.base/share/classes/sun/launcher/resources/launcher.properties - should this resource file be updated not to reference -agentlib:prof? I assume you have a JBS issue tracking the launcher man page and other documentation to be updated accordingly. Mandy From staffan.larsen at oracle.com Sun Aug 9 19:36:39 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Sun, 9 Aug 2015 21:36:39 +0200 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <93E8D7EA-FA2B-4987-A901-49C758A9764E@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> <93E8D7EA-FA2B-4987-A901-49C758A9764E@oracle.com> Message-ID: <462F8032-3D9B-499A-BC1C-1E7AA862E700@oracle.com> > On 9 aug 2015, at 05:13, Mandy Chung wrote: > > >> On Aug 6, 2015, at 11:57 PM, Staffan Larsen wrote: >> >> Please review the following changes to remove the hprof JVMTI agent. There are changes in three different repositories. All tests that used the hprof agent has been removed in previous changesets. >> >> Note: This does not remove the ability of the Hotspot VM to output heap dumps in the hprof format. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8046661 >> >> top-level changes: http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ >> jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ >> hotspot changes: http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ >> > > This patch looks fine to me. Thanks! > A minor comment: > > src/java.base/share/classes/sun/launcher/resources/launcher.properties - should this resource file be updated not to reference -agentlib:prof? > > I assume you have a JBS issue tracking the launcher man page and other documentation to be updated accordingly. I added this to the docs task for the JEP. Thanks, /Staffan > > Mandy > From staffan.larsen at oracle.com Mon Aug 10 08:19:14 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 10 Aug 2015 10:19:14 +0200 Subject: RFR: 8133245 Use external.lib.roots instead of relative paths for @library Message-ID: jtreg @library entries like these: @library /../../test/lib which refer to classes in the top-level repo, can be changed to * @library /test/lib if external.lib.roots=../../ is added to TEST.ROOT. This is a new feature in jtreg 4.1b12 (which we are currently using). Please review the changes to the serviceability tests to incorporate this: bug: https://bugs.openjdk.java.net/browse/JDK-8133245 webrev hotspot: http://cr.openjdk.java.net/~sla/8133245/hotspot/webrev.00/ webrev jdk: http://cr.openjdk.java.net/~sla/8133245/jdk/webrev.00/ Thanks, /Staffan From erik.joelsson at oracle.com Mon Aug 10 10:52:20 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 10 Aug 2015 12:52:20 +0200 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> Message-ID: <55C88264.9010006@oracle.com> Looks good to me. /Erik On 2015-08-07 08:57, Staffan Larsen wrote: > Please review the following changes to remove the hprof JVMTI agent. There are changes in three different repositories. All tests that used the hprof agent has been removed in previous changesets. > > Note: This does not remove the ability of the Hotspot VM to output heap dumps in the hprof format. > > bug: https://bugs.openjdk.java.net/browse/JDK-8046661 > > top-level changes: http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ > jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ > hotspot changes: http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ > > Thanks, > /Staffan > > > From staffan.larsen at oracle.com Mon Aug 10 11:10:20 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 10 Aug 2015 13:10:20 +0200 Subject: RFR: 8076470 - JEP 240: Remove the JVM TI hprof Agent In-Reply-To: <55C88264.9010006@oracle.com> References: <44EB3F1F-0796-4C33-A1AE-490BCE5AE3DA@oracle.com> <55C88264.9010006@oracle.com> Message-ID: <431AFB11-4806-4E76-BC72-A57FDECB8D73@oracle.com> Thanks Erik! > On 10 aug 2015, at 12:52, Erik Joelsson wrote: > > Looks good to me. > > /Erik > > On 2015-08-07 08:57, Staffan Larsen wrote: >> Please review the following changes to remove the hprof JVMTI agent. There are changes in three different repositories. All tests that used the hprof agent has been removed in previous changesets. >> >> Note: This does not remove the ability of the Hotspot VM to output heap dumps in the hprof format. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8046661 >> >> top-level changes: http://cr.openjdk.java.net/~sla/8076470/root/webrev.00/ >> jdk changes: http://cr.openjdk.java.net/~sla/8076470/jdk/webrev.00/ >> hotspot changes: http://cr.openjdk.java.net/~sla/8076470/hotspot/webrev.00/ >> >> Thanks, >> /Staffan >> >> >> > From jaroslav.bachorik at oracle.com Mon Aug 10 12:03:21 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 10 Aug 2015 05:03:21 -0700 Subject: RFR: 8133245 Use external.lib.roots instead of relative paths for @library In-Reply-To: References: Message-ID: <55C89309.9060703@oracle.com> Hi Staffan, On 10.8.2015 01:19, Staffan Larsen wrote: > jtreg @library entries like these: > > @library /../../test/lib > > which refer to classes in the top-level repo, can be changed to > > * @library /test/lib > > if external.lib.roots=../../ is added to TEST.ROOT. This is a new feature in jtreg 4.1b12 (which we are currently using). > > Please review the changes to the serviceability tests to incorporate this: > > bug: https://bugs.openjdk.java.net/browse/JDK-8133245 > webrev hotspot: http://cr.openjdk.java.net/~sla/8133245/hotspot/webrev.00/ Looks good. > webrev jdk: http://cr.openjdk.java.net/~sla/8133245/jdk/webrev.00/ test/TEST.ROOT - shouldn't there be '4.1 b12' instead of '4.1 b11'? Otherwise looks good too. -JB- > > Thanks, > /Staffan > From staffan.larsen at oracle.com Mon Aug 10 12:19:34 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 10 Aug 2015 14:19:34 +0200 Subject: RFR: 8133245 Use external.lib.roots instead of relative paths for @library In-Reply-To: <55C89309.9060703@oracle.com> References: <55C89309.9060703@oracle.com> Message-ID: <01989376-A36C-495B-AFB5-F4DC9D5D7AA3@oracle.com> > On 10 aug 2015, at 14:03, Jaroslav Bachorik wrote: > > Hi Staffan, > > On 10.8.2015 01:19, Staffan Larsen wrote: >> jtreg @library entries like these: >> >> @library /../../test/lib >> >> which refer to classes in the top-level repo, can be changed to >> >> * @library /test/lib >> >> if external.lib.roots=../../ is added to TEST.ROOT. This is a new feature in jtreg 4.1b12 (which we are currently using). >> >> Please review the changes to the serviceability tests to incorporate this: >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8133245 >> webrev hotspot: http://cr.openjdk.java.net/~sla/8133245/hotspot/webrev.00/ > > Looks good. > >> webrev jdk: http://cr.openjdk.java.net/~sla/8133245/jdk/webrev.00/ > > test/TEST.ROOT - shouldn't there be '4.1 b12' instead of '4.1 b11?? Yes, it does say ?4.1 b12?: http://cr.openjdk.java.net/~sla/8133245/jdk/webrev.00/test/TEST.ROOT.udiff.html /Staffan > > Otherwise looks good too. > > -JB- > >> >> Thanks, >> /Staffan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaroslav.bachorik at oracle.com Mon Aug 10 12:33:11 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 10 Aug 2015 05:33:11 -0700 Subject: RFR: 8133245 Use external.lib.roots instead of relative paths for @library In-Reply-To: <01989376-A36C-495B-AFB5-F4DC9D5D7AA3@oracle.com> References: <55C89309.9060703@oracle.com> <01989376-A36C-495B-AFB5-F4DC9D5D7AA3@oracle.com> Message-ID: <55C89A07.2050802@oracle.com> On 10.8.2015 05:19, Staffan Larsen wrote: > >> On 10 aug 2015, at 14:03, Jaroslav Bachorik >> > >> wrote: >> >> Hi Staffan, >> >> On 10.8.2015 01:19, Staffan Larsen wrote: >>> jtreg @library entries like these: >>> >>> @library /../../test/lib >>> >>> which refer to classes in the top-level repo, can be changed to >>> >>> * @library /test/lib >>> >>> if external.lib.roots=../../ is added to TEST.ROOT. This is a new >>> feature in jtreg 4.1b12 (which we are currently using). >>> >>> Please review the changes to the serviceability tests to incorporate >>> this: >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8133245 >>> webrev hotspot: >>> http://cr.openjdk.java.net/~sla/8133245/hotspot/webrev.00/ >> >> Looks good. >> >>> webrev jdk:http://cr.openjdk.java.net/~sla/8133245/jdk/webrev.00/ >> >> test/TEST.ROOT - shouldn't there be '4.1 b12' instead of '4.1 b11?? > > Yes, it does say ?4.1 b12?: > http://cr.openjdk.java.net/~sla/8133245/jdk/webrev.00/test/TEST.ROOT.udiff.html I completely missed it :( In that case - ship it! -JB- > > /Staffan > >> >> Otherwise looks good too. >> >> -JB- >> >>> >>> Thanks, >>> /Staffan > From staffan.larsen at oracle.com Mon Aug 10 12:44:11 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 10 Aug 2015 14:44:11 +0200 Subject: RFR: 8133245 Use external.lib.roots instead of relative paths for @library In-Reply-To: <55C89A07.2050802@oracle.com> References: <55C89309.9060703@oracle.com> <01989376-A36C-495B-AFB5-F4DC9D5D7AA3@oracle.com> <55C89A07.2050802@oracle.com> Message-ID: <2A936754-9616-4B18-BC9D-8C6D9AFC534B@oracle.com> > On 10 aug 2015, at 14:33, Jaroslav Bachorik wrote: > > On 10.8.2015 05:19, Staffan Larsen wrote: >> >>> On 10 aug 2015, at 14:03, Jaroslav Bachorik >>> >> >>> wrote: >>> >>> Hi Staffan, >>> >>> On 10.8.2015 01:19, Staffan Larsen wrote: >>>> jtreg @library entries like these: >>>> >>>> @library /../../test/lib >>>> >>>> which refer to classes in the top-level repo, can be changed to >>>> >>>> * @library /test/lib >>>> >>>> if external.lib.roots=../../ is added to TEST.ROOT. This is a new >>>> feature in jtreg 4.1b12 (which we are currently using). >>>> >>>> Please review the changes to the serviceability tests to incorporate >>>> this: >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8133245 >>>> webrev hotspot: >>>> http://cr.openjdk.java.net/~sla/8133245/hotspot/webrev.00/ >>> >>> Looks good. >>> >>>> webrev jdk:http://cr.openjdk.java.net/~sla/8133245/jdk/webrev.00/ >>> >>> test/TEST.ROOT - shouldn't there be '4.1 b12' instead of '4.1 b11?? >> >> Yes, it does say ?4.1 b12?: >> http://cr.openjdk.java.net/~sla/8133245/jdk/webrev.00/test/TEST.ROOT.udiff.html > > I completely missed it :( > > In that case - ship it! Thanks! > > -JB- > >> >> /Staffan >> >>> >>> Otherwise looks good too. >>> >>> -JB- >>> >>>> >>>> Thanks, >>>> /Staffan -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Tue Aug 11 08:26:32 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 11 Aug 2015 10:26:32 +0200 Subject: RFR: 8133314 Update launcher.properties to remove reference to hprof Message-ID: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> The usage text for the launcher has a reference to hprof which I didn?t remove when removing hprof. Please see that linked webrev for the changes to remove the mention. bug: https://bugs.openjdk.java.net/browse/JDK-8133314 webrev: http://cr.openjdk.java.net/~sla/8133314/webrev.00/ Thanks, /Staffan -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Aug 11 08:42:07 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2015 18:42:07 +1000 Subject: RFR: 8133314 Update launcher.properties to remove reference to hprof In-Reply-To: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> References: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> Message-ID: <55C9B55F.9090405@oracle.com> Looks okay to me. Thanks, David On 11/08/2015 6:26 PM, Staffan Larsen wrote: > The usage text for the launcher has a reference to hprof which I didn?t > remove when removing hprof. Please see that linked webrev for the > changes to remove the mention. > > bug: https://bugs.openjdk.java.net/browse/JDK-8133314 > webrev: http://cr.openjdk.java.net/~sla/8133314/webrev.00/ > > Thanks, > /Staffan > From staffan.larsen at oracle.com Tue Aug 11 10:34:51 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 11 Aug 2015 12:34:51 +0200 Subject: RFR: 8133314 Update launcher.properties to remove reference to hprof In-Reply-To: <55C9B55F.9090405@oracle.com> References: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> <55C9B55F.9090405@oracle.com> Message-ID: Thanks David! > On 11 aug 2015, at 10:42, David Holmes wrote: > > Looks okay to me. > > Thanks, > David > > On 11/08/2015 6:26 PM, Staffan Larsen wrote: >> The usage text for the launcher has a reference to hprof which I didn?t >> remove when removing hprof. Please see that linked webrev for the >> changes to remove the mention. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8133314 >> webrev: http://cr.openjdk.java.net/~sla/8133314/webrev.00/ >> >> Thanks, >> /Staffan >> From serguei.spitsyn at oracle.com Tue Aug 11 12:58:33 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 11 Aug 2015 05:58:33 -0700 Subject: RFR: 8133314 Update launcher.properties to remove reference to hprof In-Reply-To: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> References: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> Message-ID: <55C9F179.3040901@oracle.com> Looks good. Thanks, Serguei On 8/11/15 01:26, Staffan Larsen wrote: > The usage text for the launcher has a reference to hprof which I > didn?t remove when removing hprof. Please see that linked webrev for > the changes to remove the mention. > > bug: https://bugs.openjdk.java.net/browse/JDK-8133314 > webrev: http://cr.openjdk.java.net/~sla/8133314/webrev.00/ > > > Thanks, > /Staffan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Tue Aug 11 13:04:45 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 11 Aug 2015 15:04:45 +0200 Subject: RFR: 8133314 Update launcher.properties to remove reference to hprof In-Reply-To: <55C9F179.3040901@oracle.com> References: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> <55C9F179.3040901@oracle.com> Message-ID: Thanks Serguei! > On 11 aug 2015, at 14:58, serguei.spitsyn at oracle.com wrote: > > Looks good. > > Thanks, > Serguei > > On 8/11/15 01:26, Staffan Larsen wrote: >> The usage text for the launcher has a reference to hprof which I didn?t remove when removing hprof. Please see that linked webrev for the changes to remove the mention. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8133314 >> webrev: http://cr.openjdk.java.net/~sla/8133314/webrev.00/ >> >> Thanks, >> /Staffan >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Tue Aug 11 15:01:38 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 11 Aug 2015 08:01:38 -0700 Subject: RFR: 8133314 Update launcher.properties to remove reference to hprof In-Reply-To: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> References: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> Message-ID: <0950E44A-7D8A-48BC-BFEE-6BC65859DC3C@oracle.com> > On Aug 11, 2015, at 1:26 AM, Staffan Larsen wrote: > > The usage text for the launcher has a reference to hprof which I didn?t remove when removing hprof. Please see that linked webrev for the changes to remove the mention. > > bug: https://bugs.openjdk.java.net/browse/JDK-8133314 > webrev: http://cr.openjdk.java.net/~sla/8133314/webrev.00/ Thanks for fixing this up. Looks good. Mandy From daniel.daugherty at oracle.com Tue Aug 11 18:38:26 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 11 Aug 2015 12:38:26 -0600 Subject: assert(_exception_caught == false) failed: _exception_caught is out of phase In-Reply-To: <7F953F1E7B01F54E9F0B51450984507D3993D390@DEWDFEMB19B.global.corp.sap> References: <7F953F1E7B01F54E9F0B51450984507D3993D390@DEWDFEMB19B.global.corp.sap> Message-ID: <55CA4122.3080504@oracle.com> Adding serviceability-dev at ... since this JVM/TI... Dan On 8/11/15 9:06 AM, Reingruber, Richard wrote: > Hi, > > I would like to report that the assertion > > assert(_exception_caught == false) failed: _exception_caught is out of phase > > at jvmtiThreadState.hpp:170 fires when running the command > > ./images/jdk/bin/java -agentlib:jdwp=transport=dt_socket,address=9000,server=y,suspend=n -Xbatch ExceptionCaughtOutOfPhaseAssertion > > (when analyzing you might want to add -XX:-TieredCompilation -XX:-Inline '-XX:CompileCommand=compileonly *::run') > > Source Code: > > import java.security.AccessController; > import java.security.PrivilegedAction; > > public class ExceptionCaughtOutOfPhaseAssertion { > > public static void main(String[] args) { > PrivilegedAction action = new HotThrowingAction(); > System.out.println("### Warm-up"); > for(int i=0; i<11000; i++) { > try { > action.run(); // call run() to get it compiled > } catch(Throwable t) { /* ignored */ } > } > System.out.println("### Warm-up done"); > System.out.println("### Executing privileged action"); > AccessController.doPrivileged(action); > } > > public static class HotThrowingAction implements PrivilegedAction { > public Object run() { > throw new Error(); > } > } > } > > My Analysis: > > * Error is thrown in interpreted activation of run() > - JvmtiThreadState::_exception_detected is set to true > - JvmtiThreadState::_exception_caught is set to false > * Error is caught in main() method > - JvmtiThreadState::_exception_detected is set to false > - JvmtiThreadState::_exception_caught is set to true > * run() method is compiled > * PrivilegedAction is executed > * compiled activation of run() method > - Error object is allocated and initialized > - JavaThread::_should_post_on_exceptions_flag is checked and found to be false > -> *no* uncommon trap > - compiled frame is popped, rethrow stub calls OptoRuntime::rethrow_C() > - _exception_detected is *not* set, remains false > - _exception_caught is still true > * Java call in JVM_DoPrivileged() returns with pending exception > * CLEAR_PENDING_EXCEPTION triggers the assertion > > How to Fix? I'm not really an expert in this area, but here are my two cent: > > (a) Improve the assertion ...but how?! Toggling JVMTI exception notifications does not seem > to be synchronized with java execution. > > (b) Calling JvmtiExport::post_exception_throw() in OptoRuntime::rethrow_C() is probably not possible, > because of the JRT_LEAF comment for rethrow_C(), but _exception_detected = true could be set. > > (c) Remove the assertion. > > I guess (b) could be acceptable. > > What do you think? > > Best regards, > Richard. From staffan.larsen at oracle.com Tue Aug 11 18:38:28 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 11 Aug 2015 20:38:28 +0200 Subject: RFR: 8133314 Update launcher.properties to remove reference to hprof In-Reply-To: <0950E44A-7D8A-48BC-BFEE-6BC65859DC3C@oracle.com> References: <4A3EC1DB-DD0A-4858-B364-71429017C867@oracle.com> <0950E44A-7D8A-48BC-BFEE-6BC65859DC3C@oracle.com> Message-ID: <18866988-A94F-476D-95BC-6531F8780E50@oracle.com> Thanks Mandy! > On 11 aug 2015, at 17:01, Mandy Chung wrote: > > >> On Aug 11, 2015, at 1:26 AM, Staffan Larsen wrote: >> >> The usage text for the launcher has a reference to hprof which I didn?t remove when removing hprof. Please see that linked webrev for the changes to remove the mention. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8133314 >> webrev: http://cr.openjdk.java.net/~sla/8133314/webrev.00/ > > Thanks for fixing this up. Looks good. > > Mandy > From cheleswer.sahu at oracle.com Wed Aug 12 08:28:30 2015 From: cheleswer.sahu at oracle.com (cheleswer sahu) Date: Wed, 12 Aug 2015 13:58:30 +0530 Subject: [8u60] request for approval: 8075773 : jps running as root fails after the fix of JDK-8050807 In-Reply-To: <55CAFF6D.7060601@oracle.com> References: <55CAFF6D.7060601@oracle.com> Message-ID: <55CB03AE.5010009@oracle.com> Hi! May I please have approval to backport this fix from JDK9 to JDK8. I have built the JDK-8 hotspot and tested already. JDK9 fix applies cleanly to JDK8. As I do not have an account for OPENJDK, Kevin Walls will push the fix into jdk8u/hs-dev/hotspot. BUGURL: https://bugs.openjdk.java.net/browse/JDK-8075773 JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/0762dac98888 Review thread: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-August/015593.html Regards, Cheleswer From richard.reingruber at sap.com Wed Aug 12 08:36:54 2015 From: richard.reingruber at sap.com (Richard Reingruber) Date: Wed, 12 Aug 2015 10:36:54 +0200 Subject: assert(_exception_caught == false) failed: _exception_caught is out of phase In-Reply-To: <55CA4122.3080504@oracle.com> References: <7F953F1E7B01F54E9F0B51450984507D3993D390@DEWDFEMB19B.global.corp.sap> <55CA4122.3080504@oracle.com> Message-ID: <55CB05A6.1010503@sap.com> > Adding serviceability-dev at ... since this JVM/TI... Thanks Dan. Sorry, forgot to add a stack trace. This is from amd64: Stack: [0x00007fe6cdacc000,0x00007fe6cdbcd000], sp=0x00007fe6cdbc9d30, free space=1015k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0xfce6f1] VMError::report(outputStream*)+0x12fd V [libjvm.so+0xfcfbed] VMError::report_and_die()+0x3fd V [libjvm.so+0x86b82d] report_vm_error(char const*, int, char const*, char const*)+0xb4 V [libjvm.so+0xbe44ba] JvmtiThreadState::clear_exception_detected()+0x4e V [libjvm.so+0xbe3046] JvmtiExport::clear_detected_exception(JavaThread*)+0x72 V [libjvm.so+0xb50627] JVM_DoPrivileged+0x7e4 C [libjava.so+0xd04e] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedAction_2+0x42 j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object;+0 j ExceptionCaughtOutOfPhaseAssertion.main([Ljava/lang/String;)V+59 v ~StubRoutines::call_stub [...] I can reproduce this on amd64, sparcv9 and ppc64 with dbg builds of the current openjdk9. Richard. On 08/11/2015 08:38 PM, Daniel D. Daugherty wrote: > Adding serviceability-dev at ... since this JVM/TI... > > Dan > > > On 8/11/15 9:06 AM, Reingruber, Richard wrote: >> Hi, >> >> I would like to report that the assertion >> >> assert(_exception_caught == false) failed: _exception_caught is out of phase >> >> at jvmtiThreadState.hpp:170 fires when running the command >> >> ./images/jdk/bin/java -agentlib:jdwp=transport=dt_socket,address=9000,server=y,suspend=n -Xbatch ExceptionCaughtOutOfPhaseAssertion >> >> (when analyzing you might want to add -XX:-TieredCompilation -XX:-Inline '-XX:CompileCommand=compileonly *::run') >> >> Source Code: >> >> import java.security.AccessController; >> import java.security.PrivilegedAction; >> public class ExceptionCaughtOutOfPhaseAssertion { >> public static void main(String[] args) { >> PrivilegedAction action = new HotThrowingAction(); >> System.out.println("### Warm-up"); >> for(int i=0; i<11000; i++) { >> try { >> action.run(); // call run() to get it compiled >> } catch(Throwable t) { /* ignored */ } >> } >> System.out.println("### Warm-up done"); >> System.out.println("### Executing privileged action"); >> AccessController.doPrivileged(action); >> } >> public static class HotThrowingAction implements PrivilegedAction { >> public Object run() { >> throw new Error(); >> } >> } >> } >> My Analysis: >> >> * Error is thrown in interpreted activation of run() >> - JvmtiThreadState::_exception_detected is set to true >> - JvmtiThreadState::_exception_caught is set to false >> * Error is caught in main() method >> - JvmtiThreadState::_exception_detected is set to false >> - JvmtiThreadState::_exception_caught is set to true >> * run() method is compiled >> * PrivilegedAction is executed >> * compiled activation of run() method >> - Error object is allocated and initialized >> - JavaThread::_should_post_on_exceptions_flag is checked and found to be false >> -> *no* uncommon trap >> - compiled frame is popped, rethrow stub calls OptoRuntime::rethrow_C() >> - _exception_detected is *not* set, remains false >> - _exception_caught is still true >> * Java call in JVM_DoPrivileged() returns with pending exception >> * CLEAR_PENDING_EXCEPTION triggers the assertion >> >> How to Fix? I'm not really an expert in this area, but here are my two cent: >> >> (a) Improve the assertion ...but how?! Toggling JVMTI exception notifications does not seem >> to be synchronized with java execution. >> >> (b) Calling JvmtiExport::post_exception_throw() in OptoRuntime::rethrow_C() is probably not possible, >> because of the JRT_LEAF comment for rethrow_C(), but _exception_detected = true could be set. >> >> (c) Remove the assertion. >> >> I guess (b) could be acceptable. >> >> What do you think? >> >> Best regards, >> Richard. > From sean.coffey at oracle.com Wed Aug 12 09:51:28 2015 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Wed, 12 Aug 2015 10:51:28 +0100 Subject: [8u60] request for approval: 8075773 : jps running as root fails after the fix of JDK-8050807 In-Reply-To: <55CB03AE.5010009@oracle.com> References: <55CAFF6D.7060601@oracle.com> <55CB03AE.5010009@oracle.com> Message-ID: <55CB1720.8020409@oracle.com> Cheleswer, jdk8u approval requests need to be sent to the jdk8u-dev mailing list. Please send your request to there. Regards, Sean. On 12/08/15 09:28, cheleswer sahu wrote: > Hi! > > May I please have approval to backport this fix from JDK9 to JDK8. I > have built the JDK-8 hotspot and tested already. JDK9 fix applies > cleanly to JDK8. > As I do not have an account for OPENJDK, Kevin Walls will push the fix > into jdk8u/hs-dev/hotspot. > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8075773 > > JDK9 changeset: > http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/0762dac98888 > > Review thread: > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-August/015593.html > > Regards, > Cheleswer > > > > > From kjkoster at gmail.com Wed Aug 12 11:28:10 2015 From: kjkoster at gmail.com (Kees Jan Koster) Date: Wed, 12 Aug 2015 13:28:10 +0200 Subject: Low-Overhead Heap Profiling In-Reply-To: References: Message-ID: <035570A0-5A63-4A99-96EE-D8A067A33B14@gmail.com> Guys, This piqued my interest. Where can I find the draft JEP and the sources for this, please? Kees Jan > On 4 Aug 2015, at 23:22, Jeremy Manson wrote: > > Thanks, Staffan. I've been tinkering with a draft JEP, but it has been going slowly because of other craziness in my life. Some responses: > > 1) It was a fair bit of work to do it, mostly because it was the first time I had tinkered with c2. This was 5 years ago. Most of the work since then has been dealing with merge conflicts. > > 2) I did envision a JVMTI interface. More in the JEP. > > 3) We have some internal tests, but obviously, we'd have to figure out how to externalize them. For native code, it's much easier only to have to worry about building and running on Linux. Presumably, there's already code in there for JVMTI. > > I'll try to get a JEP going for real, although it might not happen in the next week or so, because I'm moving next week (+the JVMLS). > > Jeremy > > On Tue, Aug 4, 2015 at 4:04 AM, Staffan Larsen wrote: > Hi, > > Sorry for being away for so long. > > If memory serves me right then the current AllocObjectInNewTLAB JFR event was written as a way to quickly get some allocation profiling information with a minimum of VM changes. It probably also carried over to Hotspot from JRockit. I agree that we can and should collect better information than what that event does. > > I like the approach of instrumenting the interpreter and compiler to do the sampling. I had thought it would be a lot of work to do it and was reluctant to go down that path. If the work is already done and Jeremy has maintained it for a few years without great problems, I think we should look at bringing it in. I am not worried about the overhead of a decrement and a compare in the allocation path, but of course we should benchmark that. > > It wasn?t clear to me from the discussion how (or if) this was being exposed to end users. Was the proposal to just have the native entry points in the VM and have these be used by various agents? Would this be part of JVMTI? > > I think a draft JEP would be the logical next step and make it easier for us all to discuss what exactly the proposal should contain. Also, let?s not forget the need for tests for the feature. > > Thanks, > /Staffan > > > -- Kees Jan http://java-monitor.com/ kjkoster at kjkoster.org +31651838192 The secret of success lies in the stability of the goal. -- Benjamin Disraeli -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jaroslav.bachorik at oracle.com Wed Aug 12 16:05:07 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 12 Aug 2015 09:05:07 -0700 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> <55C2257B.1030500@oracle.com> Message-ID: <55CB6EB3.3010502@oracle.com> On 5.8.2015 08:04, Eamonn McManus wrote: > That makes me sad. The limit is clearly a detail of this particular > implementation and should not be enshrined in the spec. > ?amonn I re-requested CCC for not mentioning the exact limit in the docs and it has been approved (thanks to Joe for quick turnaround). The update webrev: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.04 -JB- > > > 2015-08-05 8:02 GMT-07:00 Jaroslav Bachorik : >> On 5.8.2015 16:53, Eamonn McManus wrote: >>> >>> I would remove the spec changes about the limit on the domain length, >>> which are a property of this particular implementation. It's perfectly >>> reasonable to blow up if the domain length is > 536,870,911, but >>> there's no reason for it to be in the spec. >> >> >> Well, CCC board had a different opinion. That's why this limit is documented >> in the spec. >> >> -JB- >> >> >>> ?amonn >>> >>> >>> 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik >>> : >>>> >>>> Eamonn, Daniel, >>>> >>>> thanks for the comments. >>>> >>>> I've updated the webrev to address them. Also, I've added a test to >>>> exercise >>>> the boolean flag en-/decoding in ObjectName. >>>> >>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 >>>> >>>> >>>> Cheers, >>>> >>>> -JB- >>>> >>>> >>>> On 4.8.2015 23:02, Daniel Fuchs wrote: >>>>> >>>>> >>>>> Hi Jaroslav, >>>>> >>>>> 379 * This field encodes _domain_pattern, _property_list_pattern >>>>> and >>>>> 380 * _property_value_pattern booleans. >>>>> >>>>> If I'm not mistaken it also encodes the domain length. >>>>> >>>>> >>>>> 1072 if ((l & FLAG_MASK) > 0 ) { >>>>> >>>>> Although it is not expected that 'l' could be negative, it might be >>>>> better to write this test as: >>>>> >>>>> if ((l & FLAG_MASK) != 0 ) { >>>>> >>>>> (+ I agree with ?amonn that l is not a great parameter name - nice to >>>>> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >>>>> Jaroslav Bachorik wrote: >>>>>> >>>>>> >>>>>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik wrote: >>>>>>> >>>>>>> >>>>>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>>>>> incompatibility, so it probably needs a CCC and some release notes. >>>>>>>> For instance, this test passes with the current version of >>>>>>>> ObjectName: public class StringLengthTest { final static int >>>>>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>>>>> public static void main(String[] args) throws >>>>>>>> MalformedObjectNameException { char[] chars = new >>>>>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>>>>> "Test")); } } I'm not sure what it will do with your changes :-) >>>>>>> >>>>>>> >>>>>>> It will fail with assert (if enabled). Or truncate the domain name, I >>>>>>> suppose. >>>>>>>> >>>>>>>> >>>>>>>> With that in mind I believe you should consider throwing >>>>>>>> InternalError - or IllegalArgumentException - instead of using >>>>>>>> 'assert' statements. >>>>>>> >>>>>>> >>>>>>> This would probably be better. >>>>>>>> >>>>>>>> >>>>>>>> BTW have you run the JCK? >>>>>>> >>>>>>> >>>>>>> Yes. All passed. I don't think JCK is testing for unrealistic values >>>>>>> :) I don't see how one could come up with a domain name 32767 >>>>>>> characters long. >>>>>> >>>>>> >>>>>> The proposed change has passed the CCC review. In case the domain name >>>>>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException will >>>>>> be thrown. All the JMX tests and JCK are still passing. >>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>>>>> >>>>>>> >>>>>>> -JB- >>>>>>>> >>>>>>>> >>>>>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>>>>> unnecessary since the store is to a byte field. >>>>>>>>> >>>>>>>>> >>>>>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 1644: the ? and : operators should be surrounded by spaces. There >>>>>>>>>> are other style issues, such as then statements on the same line >>>>>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>>>>> >>>>>>>>> >>>>>>>>> I know but these style issue have been around since the file was >>>>>>>>> first committed. I didn't address them because it didn't feel right >>>>>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>>>>> Jaroslav Bachorik wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Please, review the following change Issue : >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>>>>> situations when there are 10s of thousands ObjectNname instances >>>>>>>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to inspect the >>>>>>>>>>> object layout we can see this: Before optimization (JDK8u40): --- >>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>>>>> size: 40 bytes (estimated, the sample instance is not available) >>>>>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes total >>>>>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 short >>>>>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes external >>>>>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>>>>> instance which can translate to very interesting numbers on large >>>>>>>>>>> installations. To achieve this the domain name length is set to >>>>>>>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>>>>>>> performance purposes are encoded into one byte value (as proposed >>>>>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>>>>> JCK tests are passing after this change. Thanks, -JB- >>>> >>>> >>>> >> From daniel.fuchs at oracle.com Thu Aug 13 09:53:53 2015 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 13 Aug 2015 11:53:53 +0200 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55CB6EB3.3010502@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> <55C2257B.1030500@oracle.com> <55CB6EB3.3010502@oracle.com> Message-ID: <55CC6931.5020309@oracle.com> Hi Jaroslav, On 12/08/15 18:05, Jaroslav Bachorik wrote: > On 5.8.2015 08:04, Eamonn McManus wrote: >> That makes me sad. The limit is clearly a detail of this particular >> implementation and should not be enshrined in the spec. >> ?amonn > > I re-requested CCC for not mentioning the exact limit in the docs and it > has been approved (thanks to Joe for quick turnaround). > > The update webrev: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.04 Why are index and in_index now declared as short? Is that a left over from the original webrev? 445 short index = 0; 501 short in_index; should these be reverted to 'int' ? Otherwise - this looks good... -- daniel > > -JB- > >> >> >> 2015-08-05 8:02 GMT-07:00 Jaroslav Bachorik >> : >>> On 5.8.2015 16:53, Eamonn McManus wrote: >>>> >>>> I would remove the spec changes about the limit on the domain length, >>>> which are a property of this particular implementation. It's perfectly >>>> reasonable to blow up if the domain length is > 536,870,911, but >>>> there's no reason for it to be in the spec. >>> >>> >>> Well, CCC board had a different opinion. That's why this limit is >>> documented >>> in the spec. >>> >>> -JB- >>> >>> >>>> ?amonn >>>> >>>> >>>> 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik >>>> : >>>>> >>>>> Eamonn, Daniel, >>>>> >>>>> thanks for the comments. >>>>> >>>>> I've updated the webrev to address them. Also, I've added a test to >>>>> exercise >>>>> the boolean flag en-/decoding in ObjectName. >>>>> >>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 >>>>> >>>>> >>>>> Cheers, >>>>> >>>>> -JB- >>>>> >>>>> >>>>> On 4.8.2015 23:02, Daniel Fuchs wrote: >>>>>> >>>>>> >>>>>> Hi Jaroslav, >>>>>> >>>>>> 379 * This field encodes _domain_pattern, >>>>>> _property_list_pattern >>>>>> and >>>>>> 380 * _property_value_pattern booleans. >>>>>> >>>>>> If I'm not mistaken it also encodes the domain length. >>>>>> >>>>>> >>>>>> 1072 if ((l & FLAG_MASK) > 0 ) { >>>>>> >>>>>> Although it is not expected that 'l' could be negative, it might be >>>>>> better to write this test as: >>>>>> >>>>>> if ((l & FLAG_MASK) != 0 ) { >>>>>> >>>>>> (+ I agree with ?amonn that l is not a great parameter name - nice to >>>>>> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >>>>>> Jaroslav Bachorik wrote: >>>>>>> >>>>>>> >>>>>>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>>>>>> incompatibility, so it probably needs a CCC and some release >>>>>>>>> notes. >>>>>>>>> For instance, this test passes with the current version of >>>>>>>>> ObjectName: public class StringLengthTest { final static int >>>>>>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>>>>>> public static void main(String[] args) throws >>>>>>>>> MalformedObjectNameException { char[] chars = new >>>>>>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>>>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>>>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>>>>>> "Test")); } } I'm not sure what it will do with your >>>>>>>>> changes :-) >>>>>>>> >>>>>>>> >>>>>>>> It will fail with assert (if enabled). Or truncate the domain >>>>>>>> name, I >>>>>>>> suppose. >>>>>>>>> >>>>>>>>> >>>>>>>>> With that in mind I believe you should consider throwing >>>>>>>>> InternalError - or IllegalArgumentException - instead of using >>>>>>>>> 'assert' statements. >>>>>>>> >>>>>>>> >>>>>>>> This would probably be better. >>>>>>>>> >>>>>>>>> >>>>>>>>> BTW have you run the JCK? >>>>>>>> >>>>>>>> >>>>>>>> Yes. All passed. I don't think JCK is testing for unrealistic >>>>>>>> values >>>>>>>> :) I don't see how one could come up with a domain name 32767 >>>>>>>> characters long. >>>>>>> >>>>>>> >>>>>>> The proposed change has passed the CCC review. In case the domain >>>>>>> name >>>>>>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException >>>>>>> will >>>>>>> be thrown. All the JMX tests and JCK are still passing. >>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>>>>>> >>>>>>>> >>>>>>>> -JB- >>>>>>>>> >>>>>>>>> >>>>>>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>>>>>> unnecessary since the store is to a byte field. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 1644: the ? and : operators should be surrounded by spaces. >>>>>>>>>>> There >>>>>>>>>>> are other style issues, such as then statements on the same line >>>>>>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I know but these style issue have been around since the file was >>>>>>>>>> first committed. I didn't address them because it didn't feel >>>>>>>>>> right >>>>>>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>>>>>> Jaroslav Bachorik wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Please, review the following change Issue : >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>>>>>> situations when there are 10s of thousands ObjectNname >>>>>>>>>>>> instances >>>>>>>>>>>> around (enterprise setups etc.) the 3 separate internal boolean >>>>>>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to >>>>>>>>>>>> inspect the >>>>>>>>>>>> object layout we can see this: Before optimization >>>>>>>>>>>> (JDK8u40): --- >>>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 int >>>>>>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>>>>>> ObjectName._ca_array N/A 32 4 Map ObjectName._propertyList >>>>>>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>>>>>> size: 40 bytes (estimated, the sample instance is not >>>>>>>>>>>> available) >>>>>>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes >>>>>>>>>>>> total >>>>>>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 >>>>>>>>>>>> short >>>>>>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>>>>>> ObjectName._ca_array N/A 28 4 Map ObjectName._propertyList >>>>>>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes >>>>>>>>>>>> external >>>>>>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>>>>>> instance which can translate to very interesting numbers on >>>>>>>>>>>> large >>>>>>>>>>>> installations. To achieve this the domain name length is set to >>>>>>>>>>>> be *short* instead of *int* and the three booleans kept for the >>>>>>>>>>>> performance purposes are encoded into one byte value (as >>>>>>>>>>>> proposed >>>>>>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>>>>>> JCK tests are passing after this change. Thanks, -JB- >>>>> >>>>> >>>>> >>> > From jaroslav.bachorik at oracle.com Thu Aug 13 16:13:11 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Thu, 13 Aug 2015 09:13:11 -0700 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55CC6931.5020309@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> <55C2257B.1030500@oracle.com> <55CB6EB3.3010502@oracle.com> <55CC6931.5020309@oracle.com> Message-ID: <55CCC217.9050802@oracle.com> On 13.8.2015 02:53, Daniel Fuchs wrote: > Hi Jaroslav, > > On 12/08/15 18:05, Jaroslav Bachorik wrote: >> On 5.8.2015 08:04, Eamonn McManus wrote: >>> That makes me sad. The limit is clearly a detail of this particular >>> implementation and should not be enshrined in the spec. >>> ?amonn >> >> I re-requested CCC for not mentioning the exact limit in the docs and it >> has been approved (thanks to Joe for quick turnaround). >> >> The update webrev: >> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.04 > > Why are index and in_index now declared as short? Is that a left over > from the original webrev? > > 445 short index = 0; > 501 short in_index; Thanks for catching this! Updated: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.05 I'd better wait for Eamonn before proceeding with integration. -JB- > > should these be reverted to 'int' ? > > Otherwise - this looks good... > > -- daniel > > >> >> -JB- >> >>> >>> >>> 2015-08-05 8:02 GMT-07:00 Jaroslav Bachorik >>> : >>>> On 5.8.2015 16:53, Eamonn McManus wrote: >>>>> >>>>> I would remove the spec changes about the limit on the domain length, >>>>> which are a property of this particular implementation. It's perfectly >>>>> reasonable to blow up if the domain length is > 536,870,911, but >>>>> there's no reason for it to be in the spec. >>>> >>>> >>>> Well, CCC board had a different opinion. That's why this limit is >>>> documented >>>> in the spec. >>>> >>>> -JB- >>>> >>>> >>>>> ?amonn >>>>> >>>>> >>>>> 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik >>>>> : >>>>>> >>>>>> Eamonn, Daniel, >>>>>> >>>>>> thanks for the comments. >>>>>> >>>>>> I've updated the webrev to address them. Also, I've added a test to >>>>>> exercise >>>>>> the boolean flag en-/decoding in ObjectName. >>>>>> >>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 >>>>>> >>>>>> >>>>>> Cheers, >>>>>> >>>>>> -JB- >>>>>> >>>>>> >>>>>> On 4.8.2015 23:02, Daniel Fuchs wrote: >>>>>>> >>>>>>> >>>>>>> Hi Jaroslav, >>>>>>> >>>>>>> 379 * This field encodes _domain_pattern, >>>>>>> _property_list_pattern >>>>>>> and >>>>>>> 380 * _property_value_pattern booleans. >>>>>>> >>>>>>> If I'm not mistaken it also encodes the domain length. >>>>>>> >>>>>>> >>>>>>> 1072 if ((l & FLAG_MASK) > 0 ) { >>>>>>> >>>>>>> Although it is not expected that 'l' could be negative, it might be >>>>>>> better to write this test as: >>>>>>> >>>>>>> if ((l & FLAG_MASK) != 0 ) { >>>>>>> >>>>>>> (+ I agree with ?amonn that l is not a great parameter name - >>>>>>> nice to >>>>>>> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >>>>>>> Jaroslav Bachorik wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>>>>>>> incompatibility, so it probably needs a CCC and some release >>>>>>>>>> notes. >>>>>>>>>> For instance, this test passes with the current version of >>>>>>>>>> ObjectName: public class StringLengthTest { final static int >>>>>>>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>>>>>>> public static void main(String[] args) throws >>>>>>>>>> MalformedObjectNameException { char[] chars = new >>>>>>>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>>>>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>>>>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>>>>>>> "Test")); } } I'm not sure what it will do with your >>>>>>>>>> changes :-) >>>>>>>>> >>>>>>>>> >>>>>>>>> It will fail with assert (if enabled). Or truncate the domain >>>>>>>>> name, I >>>>>>>>> suppose. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> With that in mind I believe you should consider throwing >>>>>>>>>> InternalError - or IllegalArgumentException - instead of using >>>>>>>>>> 'assert' statements. >>>>>>>>> >>>>>>>>> >>>>>>>>> This would probably be better. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> BTW have you run the JCK? >>>>>>>>> >>>>>>>>> >>>>>>>>> Yes. All passed. I don't think JCK is testing for unrealistic >>>>>>>>> values >>>>>>>>> :) I don't see how one could come up with a domain name 32767 >>>>>>>>> characters long. >>>>>>>> >>>>>>>> >>>>>>>> The proposed change has passed the CCC review. In case the domain >>>>>>>> name >>>>>>>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException >>>>>>>> will >>>>>>>> be thrown. All the JMX tests and JCK are still passing. >>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>>>>>>> >>>>>>>>> >>>>>>>>> -JB- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>>>>>>> unnecessary since the store is to a byte field. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 1644: the ? and : operators should be surrounded by spaces. >>>>>>>>>>>> There >>>>>>>>>>>> are other style issues, such as then statements on the same >>>>>>>>>>>> line >>>>>>>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I know but these style issue have been around since the file was >>>>>>>>>>> first committed. I didn't address them because it didn't feel >>>>>>>>>>> right >>>>>>>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>>>>>>> Jaroslav Bachorik wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Please, review the following change Issue : >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>>>>>>> situations when there are 10s of thousands ObjectNname >>>>>>>>>>>>> instances >>>>>>>>>>>>> around (enterprise setups etc.) the 3 separate internal >>>>>>>>>>>>> boolean >>>>>>>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to >>>>>>>>>>>>> inspect the >>>>>>>>>>>>> object layout we can see this: Before optimization >>>>>>>>>>>>> (JDK8u40): --- >>>>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 >>>>>>>>>>>>> int >>>>>>>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>>>>>>> ObjectName._ca_array N/A 32 4 Map >>>>>>>>>>>>> ObjectName._propertyList >>>>>>>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>>>>>>> size: 40 bytes (estimated, the sample instance is not >>>>>>>>>>>>> available) >>>>>>>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes >>>>>>>>>>>>> total >>>>>>>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE >>>>>>>>>>>>> TYPE >>>>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 >>>>>>>>>>>>> short >>>>>>>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>>>>>>> ObjectName._ca_array N/A 28 4 Map >>>>>>>>>>>>> ObjectName._propertyList >>>>>>>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes >>>>>>>>>>>>> external >>>>>>>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>>>>>>> instance which can translate to very interesting numbers on >>>>>>>>>>>>> large >>>>>>>>>>>>> installations. To achieve this the domain name length is >>>>>>>>>>>>> set to >>>>>>>>>>>>> be *short* instead of *int* and the three booleans kept for >>>>>>>>>>>>> the >>>>>>>>>>>>> performance purposes are encoded into one byte value (as >>>>>>>>>>>>> proposed >>>>>>>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>>>>>>> JCK tests are passing after this change. Thanks, -JB- >>>>>> >>>>>> >>>>>> >>>> >> > From eamonn at mcmanus.net Thu Aug 13 19:31:15 2015 From: eamonn at mcmanus.net (Eamonn McManus) Date: Thu, 13 Aug 2015 12:31:15 -0700 Subject: RFR 8041565: JMX ObjectName could be refactored to save memory In-Reply-To: <55CCC217.9050802@oracle.com> References: <552B8FC6.6030409@oracle.com> <552BCDB1.704@Oracle.com> <552BDBB2.1050503@oracle.com> <552D0E65.1020805@oracle.com> <552D2B00.4090805@oracle.com> <55C0CA44.3030909@oracle.com> <55C1287A.6030004@oracle.com> <55C1F811.2040804@oracle.com> <55C2257B.1030500@oracle.com> <55CB6EB3.3010502@oracle.com> <55CC6931.5020309@oracle.com> <55CCC217.9050802@oracle.com> Message-ID: Looks fine to me (emcmanus). On Aug 13, 2015 11:13 AM, "Jaroslav Bachorik" wrote: > On 13.8.2015 02:53, Daniel Fuchs wrote: > >> Hi Jaroslav, >> >> On 12/08/15 18:05, Jaroslav Bachorik wrote: >> >>> On 5.8.2015 08:04, Eamonn McManus wrote: >>> >>>> That makes me sad. The limit is clearly a detail of this particular >>>> implementation and should not be enshrined in the spec. >>>> ?amonn >>>> >>> >>> I re-requested CCC for not mentioning the exact limit in the docs and it >>> has been approved (thanks to Joe for quick turnaround). >>> >>> The update webrev: >>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.04 >>> >> >> Why are index and in_index now declared as short? Is that a left over >> from the original webrev? >> >> 445 short index = 0; >> 501 short in_index; >> > > Thanks for catching this! > > Updated: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.05 > > I'd better wait for Eamonn before proceeding with integration. > > -JB- > > >> should these be reverted to 'int' ? >> >> Otherwise - this looks good... >> >> -- daniel >> >> >> >>> -JB- >>> >>> >>>> >>>> 2015-08-05 8:02 GMT-07:00 Jaroslav Bachorik >>>> : >>>> >>>>> On 5.8.2015 16:53, Eamonn McManus wrote: >>>>> >>>>>> >>>>>> I would remove the spec changes about the limit on the domain length, >>>>>> which are a property of this particular implementation. It's perfectly >>>>>> reasonable to blow up if the domain length is > 536,870,911, but >>>>>> there's no reason for it to be in the spec. >>>>>> >>>>> >>>>> >>>>> Well, CCC board had a different opinion. That's why this limit is >>>>> documented >>>>> in the spec. >>>>> >>>>> -JB- >>>>> >>>>> >>>>> ?amonn >>>>>> >>>>>> >>>>>> 2015-08-05 4:48 GMT-07:00 Jaroslav Bachorik >>>>>> : >>>>>> >>>>>>> >>>>>>> Eamonn, Daniel, >>>>>>> >>>>>>> thanks for the comments. >>>>>>> >>>>>>> I've updated the webrev to address them. Also, I've added a test to >>>>>>> exercise >>>>>>> the boolean flag en-/decoding in ObjectName. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.03 >>>>>>> >>>>>>> >>>>>>> Cheers, >>>>>>> >>>>>>> -JB- >>>>>>> >>>>>>> >>>>>>> On 4.8.2015 23:02, Daniel Fuchs wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi Jaroslav, >>>>>>>> >>>>>>>> 379 * This field encodes _domain_pattern, >>>>>>>> _property_list_pattern >>>>>>>> and >>>>>>>> 380 * _property_value_pattern booleans. >>>>>>>> >>>>>>>> If I'm not mistaken it also encodes the domain length. >>>>>>>> >>>>>>>> >>>>>>>> 1072 if ((l & FLAG_MASK) > 0 ) { >>>>>>>> >>>>>>>> Although it is not expected that 'l' could be negative, it might be >>>>>>>> better to write this test as: >>>>>>>> >>>>>>>> if ((l & FLAG_MASK) != 0 ) { >>>>>>>> >>>>>>>> (+ I agree with ?amonn that l is not a great parameter name - >>>>>>>> nice to >>>>>>>> see you back ?amonn :-)) best regards, -- daniel On 8/4/15 4:20 PM, >>>>>>>> Jaroslav Bachorik wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi, reviving this review. On 14.4.2015 16:58, Jaroslav Bachorik >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 14.4.2015 14:56, Daniel Fuchs wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Jaroslav, I like this change, but it does introduce an >>>>>>>>>>> incompatibility, so it probably needs a CCC and some release >>>>>>>>>>> notes. >>>>>>>>>>> For instance, this test passes with the current version of >>>>>>>>>>> ObjectName: public class StringLengthTest { final static int >>>>>>>>>>> smax = Short.MAX_VALUE; final static int smore = smax + 126; >>>>>>>>>>> public static void main(String[] args) throws >>>>>>>>>>> MalformedObjectNameException { char[] chars = new >>>>>>>>>>> char[smore]; Arrays.fill(chars, 0, smax, 'a'); >>>>>>>>>>> Arrays.fill(chars, smax, smore, 'b'); >>>>>>>>>>> System.out.println(new ObjectName(new String(chars), "type", >>>>>>>>>>> "Test")); } } I'm not sure what it will do with your >>>>>>>>>>> changes :-) >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> It will fail with assert (if enabled). Or truncate the domain >>>>>>>>>> name, I >>>>>>>>>> suppose. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> With that in mind I believe you should consider throwing >>>>>>>>>>> InternalError - or IllegalArgumentException - instead of using >>>>>>>>>>> 'assert' statements. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This would probably be better. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> BTW have you run the JCK? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes. All passed. I don't think JCK is testing for unrealistic >>>>>>>>>> values >>>>>>>>>> :) I don't see how one could come up with a domain name 32767 >>>>>>>>>> characters long. >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> The proposed change has passed the CCC review. In case the domain >>>>>>>>> name >>>>>>>>> is longer than Integer.MAX_VALUE/4 a MalformedObjectNameException >>>>>>>>> will >>>>>>>>> be thrown. All the JMX tests and JCK are still passing. >>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.02 -JB- >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -JB- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> best regards, -- daniel On 13/04/15 17:07, Jaroslav Bachorik >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Roger, On 13.4.2015 16:07, Roger Riggs wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Jaroslav, Minor comments: 1488+: In forms like: >>>>>>>>>>>>> _pattern_flag &= (~PROPLIST_PATTERN & 0xff);" The &0xff seems >>>>>>>>>>>>> unnecessary since the store is to a byte field. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Fixed: http://cr.openjdk.java.net/~jbachorik/8041565/webrev.01 >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 1644: the ? and : operators should be surrounded by spaces. >>>>>>>>>>>>> There >>>>>>>>>>>>> are other style issues, such as then statements on the same >>>>>>>>>>>>> line >>>>>>>>>>>>> as the 'if' but that may be beyond the scope of this change. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I know but these style issue have been around since the file was >>>>>>>>>>>> first committed. I didn't address them because it didn't feel >>>>>>>>>>>> right >>>>>>>>>>>> to mix style changes with the actual functionality. Cheers, -JB- >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Otherwise looks fine (as a 9 reviewer, but not specifically a >>>>>>>>>>>>> serviceability reviewer). Thanks, Roger On 4/13/2015 5:43 AM, >>>>>>>>>>>>> Jaroslav Bachorik wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please, review the following change Issue : >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8041565 Webrev: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~jbachorik/8041565/webrev.00 In >>>>>>>>>>>>>> situations when there are 10s of thousands ObjectNname >>>>>>>>>>>>>> instances >>>>>>>>>>>>>> around (enterprise setups etc.) the 3 separate internal >>>>>>>>>>>>>> boolean >>>>>>>>>>>>>> fields can lead to a noticeable memory waste. Adding insult to >>>>>>>>>>>>>> the injury, with the current field layout it is necessary to >>>>>>>>>>>>>> align the instances by 4 bytes. When using JOL >>>>>>>>>>>>>> (http://openjdk.java.net/projects/code-tools/jol/) to >>>>>>>>>>>>>> inspect the >>>>>>>>>>>>>> object layout we can see this: Before optimization >>>>>>>>>>>>>> (JDK8u40): --- >>>>>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE TYPE >>>>>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header)| N/A 12 4 >>>>>>>>>>>>>> int >>>>>>>>>>>>>> ObjectName._domain_length N/A 16 1 boolean >>>>>>>>>>>>>> ObjectName._domain_pattern N/A 17 1 boolean >>>>>>>>>>>>>> ObjectName._property_list_pattern N/A 18 1 boolean >>>>>>>>>>>>>> ObjectName._property_value_pattern N/A 19 1 >>>>>>>>>>>>>> (alignment/padding gap) N/A 20 4 String >>>>>>>>>>>>>> ObjectName._canonicalName N/A 24 4 Property[] >>>>>>>>>>>>>> ObjectName._kp_array N/A 28 4 Property[] >>>>>>>>>>>>>> ObjectName._ca_array N/A 32 4 Map >>>>>>>>>>>>>> ObjectName._propertyList >>>>>>>>>>>>>> N/A 36 4 (loss due to the next object alignment) Instance >>>>>>>>>>>>>> size: 40 bytes (estimated, the sample instance is not >>>>>>>>>>>>>> available) >>>>>>>>>>>>>> Space losses: 1 bytes internal + 4 bytes external = 5 bytes >>>>>>>>>>>>>> total >>>>>>>>>>>>>> {noformat} After optimization (JDK9 internal build): --- >>>>>>>>>>>>>> javax.management.ObjectName object internals: OFFSET SIZE >>>>>>>>>>>>>> TYPE >>>>>>>>>>>>>> DESCRIPTION VALUE 0 12 (object header) N/A 12 2 >>>>>>>>>>>>>> short >>>>>>>>>>>>>> ObjectName._domain_length N/A 14 1 byte >>>>>>>>>>>>>> ObjectName._pattern_flag N/A 15 1 (alignment/padding gap) >>>>>>>>>>>>>> N/A 16 4 String ObjectName._canonicalName N/A 20 4 >>>>>>>>>>>>>> Property[] ObjectName._kp_array N/A 24 4 Property[] >>>>>>>>>>>>>> ObjectName._ca_array N/A 28 4 Map >>>>>>>>>>>>>> ObjectName._propertyList >>>>>>>>>>>>>> N/A Instance size: 32 bytes (estimated, the sample instance is >>>>>>>>>>>>>> not available) Space losses: 1 bytes internal + 0 bytes >>>>>>>>>>>>>> external >>>>>>>>>>>>>> = 1 bytes total After optimization we can save 8 bytes per >>>>>>>>>>>>>> instance which can translate to very interesting numbers on >>>>>>>>>>>>>> large >>>>>>>>>>>>>> installations. To achieve this the domain name length is >>>>>>>>>>>>>> set to >>>>>>>>>>>>>> be *short* instead of *int* and the three booleans kept for >>>>>>>>>>>>>> the >>>>>>>>>>>>>> performance purposes are encoded into one byte value (as >>>>>>>>>>>>>> proposed >>>>>>>>>>>>>> by the reporter, Jean-Francois Denise). All the regression and >>>>>>>>>>>>>> JCK tests are passing after this change. Thanks, -JB- >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Mon Aug 17 05:03:39 2015 From: jeremymanson at google.com (Jeremy Manson) Date: Sun, 16 Aug 2015 22:03:39 -0700 Subject: Low-Overhead Heap Profiling In-Reply-To: <035570A0-5A63-4A99-96EE-D8A067A33B14@gmail.com> References: <035570A0-5A63-4A99-96EE-D8A067A33B14@gmail.com> Message-ID: Working on it. I hope to have something to share in the next couple of weeks. Jeremy On Wed, Aug 12, 2015 at 4:28 AM, Kees Jan Koster wrote: > Guys, > > This piqued my interest. Where can I find the draft JEP and the sources > for this, please? > > Kees Jan > > > > On 4 Aug 2015, at 23:22, Jeremy Manson wrote: > > > > Thanks, Staffan. I've been tinkering with a draft JEP, but it has been > going slowly because of other craziness in my life. Some responses: > > > > 1) It was a fair bit of work to do it, mostly because it was the first > time I had tinkered with c2. This was 5 years ago. Most of the work since > then has been dealing with merge conflicts. > > > > 2) I did envision a JVMTI interface. More in the JEP. > > > > 3) We have some internal tests, but obviously, we'd have to figure out > how to externalize them. For native code, it's much easier only to have to > worry about building and running on Linux. Presumably, there's already > code in there for JVMTI. > > > > I'll try to get a JEP going for real, although it might not happen in > the next week or so, because I'm moving next week (+the JVMLS). > > > > Jeremy > > > > On Tue, Aug 4, 2015 at 4:04 AM, Staffan Larsen < > staffan.larsen at oracle.com> wrote: > > Hi, > > > > Sorry for being away for so long. > > > > If memory serves me right then the current AllocObjectInNewTLAB JFR > event was written as a way to quickly get some allocation profiling > information with a minimum of VM changes. It probably also carried over to > Hotspot from JRockit. I agree that we can and should collect better > information than what that event does. > > > > I like the approach of instrumenting the interpreter and compiler to do > the sampling. I had thought it would be a lot of work to do it and was > reluctant to go down that path. If the work is already done and Jeremy has > maintained it for a few years without great problems, I think we should > look at bringing it in. I am not worried about the overhead of a decrement > and a compare in the allocation path, but of course we should benchmark > that. > > > > It wasn?t clear to me from the discussion how (or if) this was being > exposed to end users. Was the proposal to just have the native entry points > in the VM and have these be used by various agents? Would this be part of > JVMTI? > > > > I think a draft JEP would be the logical next step and make it easier > for us all to discuss what exactly the proposal should contain. Also, let?s > not forget the need for tests for the feature. > > > > Thanks, > > /Staffan > > > > > > > > > -- > Kees Jan > > http://java-monitor.com/ > kjkoster at kjkoster.org > +31651838192 > > The secret of success lies in the stability of the goal. -- Benjamin > Disraeli > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.buck at oracle.com Mon Aug 17 11:24:08 2015 From: david.buck at oracle.com (david buck) Date: Mon, 17 Aug 2015 20:24:08 +0900 Subject: RFR 8133666: OperatingSystemMXBean reports abnormally high machine CPU consumption on Linux Message-ID: <55D1C458.1070701@oracle.com> Hi! Please review my fix for this JMX issue. bug: https://bugs.openjdk.java.net/browse/JDK-8133666 webrev: http://cr.openjdk.java.net/~dbuck/8133666/ Once approved, I will push this fix into: http://hg.openjdk.java.net/jdk9/hs-rt Cheers, -Buck From markus.gronlund at oracle.com Mon Aug 17 11:23:44 2015 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Mon, 17 Aug 2015 04:23:44 -0700 (PDT) Subject: RFR 8133666: OperatingSystemMXBean reports abnormally high machine CPU consumption on Linux In-Reply-To: <55D1C458.1070701@oracle.com> References: <55D1C458.1070701@oracle.com> Message-ID: <79b4c65f-b889-4e1d-9603-4cbd53119ce0@default> Hi David, This looks good - thanks for fixing this issue! /Markus -----Original Message----- From: david buck Sent: den 17 augusti 2015 13:24 To: serviceability-dev at openjdk.java.net Subject: RFR 8133666: OperatingSystemMXBean reports abnormally high machine CPU consumption on Linux Hi! Please review my fix for this JMX issue. bug: https://bugs.openjdk.java.net/browse/JDK-8133666 webrev: http://cr.openjdk.java.net/~dbuck/8133666/ Once approved, I will push this fix into: http://hg.openjdk.java.net/jdk9/hs-rt Cheers, -Buck From staffan.larsen at oracle.com Mon Aug 17 11:30:42 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 17 Aug 2015 13:30:42 +0200 Subject: RFR 8133666: OperatingSystemMXBean reports abnormally high machine CPU consumption on Linux In-Reply-To: <55D1C458.1070701@oracle.com> References: <55D1C458.1070701@oracle.com> Message-ID: <39A04006-8240-4948-A259-61D8BAF5F071@oracle.com> Looks good! Thanks, /Staffan > On 17 aug 2015, at 13:24, david buck wrote: > > Hi! > > Please review my fix for this JMX issue. > > bug: https://bugs.openjdk.java.net/browse/JDK-8133666 > webrev: http://cr.openjdk.java.net/~dbuck/8133666/ > > Once approved, I will push this fix into: > > http://hg.openjdk.java.net/jdk9/hs-rt > > Cheers, > -Buck From jaroslav.bachorik at oracle.com Tue Aug 18 14:35:24 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 18 Aug 2015 16:35:24 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector Message-ID: <55D342AC.8000005@oracle.com> Please, review the following change Issue : https://bugs.openjdk.java.net/browse/JDK-8043937 Webrev: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.00 For Java SE 8, JSR 160 was updated to downgrade the support for the JXM RMI-IIOP transport from required to optional. This was the first step in a multi-release effort to remove support for the IIOP transport from the JMX Remote API. Early in JDK 9, the build was changed to not include the JMX RMI-IIOP connector by default. This is the last step in removing the JMX RMI-IIOP connector from the JMX specification. This changeset will remove the implementation classes as well as related tests. All jdk_jmx and jdk_management tests are passing after this change. JCK tests will need to be updated (tracked in a related issue). Thanks, -JB- From Alan.Bateman at oracle.com Wed Aug 19 06:19:20 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Aug 2015 07:19:20 +0100 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D342AC.8000005@oracle.com> References: <55D342AC.8000005@oracle.com> Message-ID: <55D41FE8.9000600@oracle.com> On 18/08/2015 15:35, Jaroslav Bachorik wrote: > Please, review the following change > > Issue : https://bugs.openjdk.java.net/browse/JDK-8043937 > Webrev: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.00 > > For Java SE 8, JSR 160 was updated to downgrade the support for the > JXM RMI-IIOP transport from required to optional. This was the first > step in a multi-release effort to remove support for the IIOP > transport from the JMX Remote API. > > Early in JDK 9, the build was changed to not include the JMX RMI-IIOP > connector by default. > > This is the last step in removing the JMX RMI-IIOP connector from the > JMX specification. This changeset will remove the implementation > classes as well as related tests. > > All jdk_jmx and jdk_management tests are passing after this change. > JCK tests will need to be updated (tracked in a related issue). Do you have a webrev for the changes to the top-level repo? I ask because part of the removal has to be the removal of the --enable-rmiconnector-iiop configure option, essentially removing the changes introduced by: http://hg.openjdk.java.net/jdk9/dev/rev/e9e5bd372a4e -Alan From jaroslav.bachorik at oracle.com Wed Aug 19 08:59:01 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 19 Aug 2015 10:59:01 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D41FE8.9000600@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> Message-ID: <55D44555.7090601@oracle.com> On 19.8.2015 08:19, Alan Bateman wrote: > On 18/08/2015 15:35, Jaroslav Bachorik wrote: >> Please, review the following change >> >> Issue : https://bugs.openjdk.java.net/browse/JDK-8043937 >> Webrev: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.00 >> >> For Java SE 8, JSR 160 was updated to downgrade the support for the >> JXM RMI-IIOP transport from required to optional. This was the first >> step in a multi-release effort to remove support for the IIOP >> transport from the JMX Remote API. >> >> Early in JDK 9, the build was changed to not include the JMX RMI-IIOP >> connector by default. >> >> This is the last step in removing the JMX RMI-IIOP connector from the >> JMX specification. This changeset will remove the implementation >> classes as well as related tests. >> >> All jdk_jmx and jdk_management tests are passing after this change. >> JCK tests will need to be updated (tracked in a related issue). > Do you have a webrev for the changes to the top-level repo? I ask > because part of the removal has to be the removal of the > --enable-rmiconnector-iiop configure option, essentially removing the > changes introduced by: > > http://hg.openjdk.java.net/jdk9/dev/rev/e9e5bd372a4e I missed this. However, I have my doubts about 'common/autoconf/generated-configure.sh' - the comment says this file should not be edited but it was changed in the aforementioned commit. Should I change it back? -JB- > > -Alan From jaroslav.bachorik at oracle.com Wed Aug 19 09:02:41 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 19 Aug 2015 11:02:41 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D41FE8.9000600@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> Message-ID: <55D44631.9080007@oracle.com> On 19.8.2015 08:19, Alan Bateman wrote: > On 18/08/2015 15:35, Jaroslav Bachorik wrote: >> Please, review the following change >> >> Issue : https://bugs.openjdk.java.net/browse/JDK-8043937 >> Webrev: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.00 >> >> For Java SE 8, JSR 160 was updated to downgrade the support for the >> JXM RMI-IIOP transport from required to optional. This was the first >> step in a multi-release effort to remove support for the IIOP >> transport from the JMX Remote API. >> >> Early in JDK 9, the build was changed to not include the JMX RMI-IIOP >> connector by default. >> >> This is the last step in removing the JMX RMI-IIOP connector from the >> JMX specification. This changeset will remove the implementation >> classes as well as related tests. >> >> All jdk_jmx and jdk_management tests are passing after this change. >> JCK tests will need to be updated (tracked in a related issue). > Do you have a webrev for the changes to the top-level repo? I ask > because part of the removal has to be the removal of the > --enable-rmiconnector-iiop configure option, essentially removing the > changes introduced by: > > http://hg.openjdk.java.net/jdk9/dev/rev/e9e5bd372a4e I missed this. However, I have my doubts about 'common/autoconf/generated-configure.sh' - the comment says this file should not be edited but it was changed in the aforementioned commit. Should I change it back? -JB- > > -Alan From Alan.Bateman at oracle.com Wed Aug 19 09:02:53 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Aug 2015 10:02:53 +0100 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D44555.7090601@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> Message-ID: <55D4463D.2010109@oracle.com> On 19/08/2015 09:59, Jaroslav Bachorik wrote: > > I missed this. However, I have my doubts about > 'common/autoconf/generated-configure.sh' - the comment says this file > should not be edited but it was changed in the aforementioned commit. > Should I change it back? You will need to edit spec.gmk.in and jdk-options.m4, then re-run autogen to regenerate generated-configure.sh. -Alan From jaroslav.bachorik at oracle.com Wed Aug 19 10:47:34 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 19 Aug 2015 12:47:34 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D4463D.2010109@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> <55D4463D.2010109@oracle.com> Message-ID: <55D45EC6.30606@oracle.com> On 19.8.2015 11:02, Alan Bateman wrote: > > > On 19/08/2015 09:59, Jaroslav Bachorik wrote: >> >> I missed this. However, I have my doubts about >> 'common/autoconf/generated-configure.sh' - the comment says this file >> should not be edited but it was changed in the aforementioned commit. >> Should I change it back? > You will need to edit spec.gmk.in and jdk-options.m4, then re-run > autogen to regenerate generated-configure.sh. Top level changes: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.01/ JDK changes: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.01/jdk -JB- > > -Alan From Alan.Bateman at oracle.com Wed Aug 19 11:20:58 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Aug 2015 12:20:58 +0100 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D45EC6.30606@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> <55D4463D.2010109@oracle.com> <55D45EC6.30606@oracle.com> Message-ID: <55D4669A.5080307@oracle.com> On 19/08/2015 11:47, Jaroslav Bachorik wrote: > Top level changes: > http://cr.openjdk.java.net/~jbachorik/8043937/webrev.01/ > JDK changes: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.01/jdk I think you also need to look at jake/make/rmic/Rmic-java.management.gmk which is where RMICONNECTOR_IIOP is used. This is the original change to the jdk repo but the make files have since been refactored: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/c952b81ad038 -Alan From jaroslav.bachorik at oracle.com Wed Aug 19 12:51:36 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 19 Aug 2015 14:51:36 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D4669A.5080307@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> <55D4463D.2010109@oracle.com> <55D45EC6.30606@oracle.com> <55D4669A.5080307@oracle.com> Message-ID: <55D47BD8.4000409@oracle.com> On 19.8.2015 13:20, Alan Bateman wrote: > On 19/08/2015 11:47, Jaroslav Bachorik wrote: >> Top level changes: >> http://cr.openjdk.java.net/~jbachorik/8043937/webrev.01/ >> JDK changes: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.01/jdk > I think you also need to look at jake/make/rmic/Rmic-java.management.gmk > which is where RMICONNECTOR_IIOP is used. This is the original change to > the jdk repo but the make files have since been refactored: > > http://hg.openjdk.java.net/jdk9/dev/jdk/rev/c952b81ad038 I hope I got it all this time. Used brute force search for any mention of 'iiop' and removed anything that had anything to do with JMX. Top level: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.02 JDK: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.02/jdk -JB- > > -Alan From Alan.Bateman at oracle.com Wed Aug 19 13:05:19 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Aug 2015 14:05:19 +0100 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D47BD8.4000409@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> <55D4463D.2010109@oracle.com> <55D45EC6.30606@oracle.com> <55D4669A.5080307@oracle.com> <55D47BD8.4000409@oracle.com> Message-ID: <55D47F0F.4050903@oracle.com> On 19/08/2015 13:51, Jaroslav Bachorik wrote: > > I hope I got it all this time. Used brute force search for any mention > of 'iiop' and removed anything that had anything to do with JMX. > > Top level: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.02 > JDK: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.02/jdk I think you are close :-) The @deprecated in class RMIIIOPServerImpl says "This method was ..." when it should be "This class". It might be simpler to just drop that sentence and say "The IIOP transport is no longer supported". I see the methods in this class throw UOE but I assume that can't happen because the constructor always throws an exception (except of course if someone does something sneaky, say with finalizers). So I think what you have is okay. There are several tests in jdk/test/javax/management that exercise IIOP if available, shouldn't they be updated too? -Alan. From jaroslav.bachorik at oracle.com Wed Aug 19 14:01:07 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 19 Aug 2015 16:01:07 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D47F0F.4050903@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> <55D4463D.2010109@oracle.com> <55D45EC6.30606@oracle.com> <55D4669A.5080307@oracle.com> <55D47BD8.4000409@oracle.com> <55D47F0F.4050903@oracle.com> Message-ID: <55D48C23.5040301@oracle.com> On 19.8.2015 15:05, Alan Bateman wrote: > On 19/08/2015 13:51, Jaroslav Bachorik wrote: >> >> I hope I got it all this time. Used brute force search for any mention >> of 'iiop' and removed anything that had anything to do with JMX. >> >> Top level: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.02 >> JDK: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.02/jdk > > I think you are close :-) > > The @deprecated in class RMIIIOPServerImpl says "This method was ..." > when it should be "This class". It might be simpler to just drop that > sentence and say "The IIOP transport is no longer supported". Ok. > > I see the methods in this class throw UOE but I assume that can't happen > because the constructor always throws an exception (except of course if > someone does something sneaky, say with finalizers). So I think what you > have is okay. Yes, under normal circumstances you should never be able to obtain an instance of this class. > > There are several tests in jdk/test/javax/management that exercise IIOP > if available, shouldn't they be updated too? Not sure about this. These tests are also exercising JMXMP if available - but that has never been a part of JDK. -JB- > > -Alan. > > From Alan.Bateman at oracle.com Wed Aug 19 14:32:29 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Aug 2015 15:32:29 +0100 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D48C23.5040301@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> <55D4463D.2010109@oracle.com> <55D45EC6.30606@oracle.com> <55D4669A.5080307@oracle.com> <55D47BD8.4000409@oracle.com> <55D47F0F.4050903@oracle.com> <55D48C23.5040301@oracle.com> Message-ID: <55D4937D.2050802@oracle.com> On 19/08/2015 15:01, Jaroslav Bachorik wrote: > : > >> >> There are several tests in jdk/test/javax/management that exercise IIOP >> if available, shouldn't they be updated too? > > Not sure about this. These tests are also exercising JMXMP if > available - but that has never been a part of JDK. I think the tests date back to when JMX was a standalone JSR and RI. In any case, with IIOP support removed then it seems like these tests should be updated but it's not critical to do it now. -Alan From jaroslav.bachorik at oracle.com Wed Aug 19 14:40:48 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 19 Aug 2015 16:40:48 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D4937D.2050802@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> <55D4463D.2010109@oracle.com> <55D45EC6.30606@oracle.com> <55D4669A.5080307@oracle.com> <55D47BD8.4000409@oracle.com> <55D47F0F.4050903@oracle.com> <55D48C23.5040301@oracle.com> <55D4937D.2050802@oracle.com> Message-ID: <55D49570.9020602@oracle.com> On 19.8.2015 16:32, Alan Bateman wrote: > > > On 19/08/2015 15:01, Jaroslav Bachorik wrote: >> : >> >>> >>> There are several tests in jdk/test/javax/management that exercise IIOP >>> if available, shouldn't they be updated too? >> >> Not sure about this. These tests are also exercising JMXMP if >> available - but that has never been a part of JDK. > I think the tests date back to when JMX was a standalone JSR and RI. In > any case, with IIOP support removed then it seems like these tests > should be updated but it's not critical to do it now. What we could do is to update the tests to accept the required protocol as a test parameter. That way they still can serve as a test harness for any potential new JMX connectors (eg. JMXMP etc.) -JB- > > -Alan From dmitry.samersoff at oracle.com Wed Aug 19 17:10:45 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 19 Aug 2015 20:10:45 +0300 Subject: RFR(S): 8086134 Deadlock detection fails to attach to core file Message-ID: <55D4B895.9000509@oracle.com> Everybody, Please, review test-only fix. http://cr.openjdk.java.net/~dsamersoff/JDK-8086134/webrev.01/ The test is rewritten to use testlibrary and to detect common bad environment conditions. Tests in Hotspot and JDK workspaces are identical, except imports and the fact that JDK version runs jstack but hotspot version runs jhsdb (sa launcher). -Dmitry -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From jaroslav.bachorik at oracle.com Fri Aug 21 09:35:22 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 21 Aug 2015 11:35:22 +0200 Subject: RFR(S): 8086134 Deadlock detection fails to attach to core file In-Reply-To: <55D4B895.9000509@oracle.com> References: <55D4B895.9000509@oracle.com> Message-ID: <55D6F0DA.7090902@oracle.com> Hi Dmitry, On 19.8.2015 19:10, Dmitry Samersoff wrote: > Everybody, > > Please, review test-only fix. > > http://cr.openjdk.java.net/~dsamersoff/JDK-8086134/webrev.01/ > > The test is rewritten to use testlibrary and to detect common bad environment conditions. > > Tests in Hotspot and JDK workspaces are identical, except imports and the fact that JDK version runs > jstack but hotspot version runs jhsdb (sa launcher). I will need more explanation here. * the issue is about the tmtools test failing; are you planning to remove the failing tests? * what is the reason for having the test in hotspot and in jdk? wouldn't it be possible to have it just in jdk and launch it once using jstack and once jhsdb? Thanks, -JB- > > -Dmitry > > From dmitry.samersoff at oracle.com Fri Aug 21 10:05:43 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 21 Aug 2015 13:05:43 +0300 Subject: RFR(S): 8086134 Deadlock detection fails to attach to core file In-Reply-To: <55D6F0DA.7090902@oracle.com> References: <55D4B895.9000509@oracle.com> <55D6F0DA.7090902@oracle.com> Message-ID: <55D6F7F7.4010503@oracle.com> On 2015-08-21 12:35, Jaroslav Bachorik wrote: > Hi Dmitry, > > On 19.8.2015 19:10, Dmitry Samersoff wrote: >> Everybody, >> >> Please, review test-only fix. >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8086134/webrev.01/ >> >> The test is rewritten to use testlibrary and to detect common bad >> environment conditions. >> >> Tests in Hotspot and JDK workspaces are identical, except imports and >> the fact that JDK version runs >> jstack but hotspot version runs jhsdb (sa launcher). > > I will need more explanation here. > > * the issue is about the tmtools test failing; are you planning to > remove the failing tests? Yes. Falling test have to be removed. > * what is the reason for having the test in hotspot and in jdk? > wouldn't it be possible to have it just in jdk and launch it once > using jstack and once jhsdb? Currently, tests for tmtools (jstack, jmap etc) resides in JDK, but tests for SA resides in Hotspot. I would love to place all tests to the single place but this work is out of scope of this fix. -Dmitry > > > Thanks, > > -JB- > >> >> -Dmitry >> >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From jaroslav.bachorik at oracle.com Fri Aug 21 11:44:28 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 21 Aug 2015 13:44:28 +0200 Subject: RFR(S): 8086134 Deadlock detection fails to attach to core file In-Reply-To: <55D6F7F7.4010503@oracle.com> References: <55D4B895.9000509@oracle.com> <55D6F0DA.7090902@oracle.com> <55D6F7F7.4010503@oracle.com> Message-ID: <55D70F1C.3030802@oracle.com> On 21.8.2015 12:05, Dmitry Samersoff wrote: > On 2015-08-21 12:35, Jaroslav Bachorik wrote: >> Hi Dmitry, >> >> On 19.8.2015 19:10, Dmitry Samersoff wrote: >>> Everybody, >>> >>> Please, review test-only fix. >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8086134/webrev.01/ >>> >>> The test is rewritten to use testlibrary and to detect common bad >>> environment conditions. >>> >>> Tests in Hotspot and JDK workspaces are identical, except imports and >>> the fact that JDK version runs >>> jstack but hotspot version runs jhsdb (sa launcher). >> >> I will need more explanation here. >> >> * the issue is about the tmtools test failing; are you planning to >> remove the failing tests? > > Yes. Falling test have to be removed. > >> * what is the reason for having the test in hotspot and in jdk? >> wouldn't it be possible to have it just in jdk and launch it once >> using jstack and once jhsdb? > > Currently, tests for tmtools (jstack, jmap etc) resides in JDK, but > tests for SA resides in Hotspot. I would love to place all tests to the > single place > but this work is out of scope of this fix. Ok. Thanks. Overall looks good. Just some minor nits. * instead of `Arrays.toString(processBuilder.command().toArray()).replace(",", "")` you can use `processBuilder.command().stream().collect(Collectors.joining(" "))` - which is slightly more concise * typo 'depends *to* operating system' -> 'depends *on* operating system' -JB- > > -Dmitry > >> >> >> Thanks, >> >> -JB- >> >>> >>> -Dmitry >>> >>> >> > > From jaroslav.bachorik at oracle.com Fri Aug 21 11:49:12 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 21 Aug 2015 13:49:12 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D41FE8.9000600@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> Message-ID: <55D71038.3010203@oracle.com> On 19.8.2015 08:19, Alan Bateman wrote: > On 18/08/2015 15:35, Jaroslav Bachorik wrote: >> Please, review the following change >> >> Issue : https://bugs.openjdk.java.net/browse/JDK-8043937 >> Webrev: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.00 >> >> For Java SE 8, JSR 160 was updated to downgrade the support for the >> JXM RMI-IIOP transport from required to optional. This was the first >> step in a multi-release effort to remove support for the IIOP >> transport from the JMX Remote API. >> >> Early in JDK 9, the build was changed to not include the JMX RMI-IIOP >> connector by default. >> >> This is the last step in removing the JMX RMI-IIOP connector from the >> JMX specification. This changeset will remove the implementation >> classes as well as related tests. >> >> All jdk_jmx and jdk_management tests are passing after this change. >> JCK tests will need to be updated (tracked in a related issue). > Do you have a webrev for the changes to the top-level repo? I ask > because part of the removal has to be the removal of the > --enable-rmiconnector-iiop configure option, essentially removing the > changes introduced by: > > http://hg.openjdk.java.net/jdk9/dev/rev/e9e5bd372a4e Updated webrev top-level: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.03 Updated webrev jdk: http://cr.openjdk.java.net/~jbachorik/8043937/webrev.03/jdk Contains changes to the top level repo as well as a correct version of jdk/make/rmic/Rmic-java.management.gmk (thanks to Erik Joelsson) -JB- > > -Alan From Alan.Bateman at oracle.com Mon Aug 24 06:52:58 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 24 Aug 2015 07:52:58 +0100 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D71038.3010203@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D71038.3010203@oracle.com> Message-ID: <55DABF4A.4090805@oracle.com> On 21/08/2015 12:49, Jaroslav Bachorik wrote: > > Updated webrev top-level: > http://cr.openjdk.java.net/~jbachorik/8043937/webrev.03 > Updated webrev jdk: > http://cr.openjdk.java.net/~jbachorik/8043937/webrev.03/jdk > > Contains changes to the top level repo as well as a correct version of > jdk/make/rmic/Rmic-java.management.gmk (thanks to Erik Joelsson) The updated webrev and the updated @deprecated in RMIIIOPServerImpl looks good. Can you make sure to test out boot cycle builds before pushing this? That would help with the confidence that the updates to the RMI stub generation are good. -Alan. From jaroslav.bachorik at oracle.com Mon Aug 24 10:18:08 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 24 Aug 2015 12:18:08 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55DABF4A.4090805@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D71038.3010203@oracle.com> <55DABF4A.4090805@oracle.com> Message-ID: <55DAEF60.3080401@oracle.com> On 24.8.2015 08:52, Alan Bateman wrote: > > On 21/08/2015 12:49, Jaroslav Bachorik wrote: >> >> Updated webrev top-level: >> http://cr.openjdk.java.net/~jbachorik/8043937/webrev.03 >> Updated webrev jdk: >> http://cr.openjdk.java.net/~jbachorik/8043937/webrev.03/jdk >> >> Contains changes to the top level repo as well as a correct version of >> jdk/make/rmic/Rmic-java.management.gmk (thanks to Erik Joelsson) > The updated webrev and the updated @deprecated in RMIIIOPServerImpl > looks good. Can you make sure to test out boot cycle builds before > pushing this? That would help with the confidence that the updates to > the RMI stub generation are good. Bootcycle build is ok. -JB- > > -Alan. From erik.joelsson at oracle.com Mon Aug 24 10:34:38 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 24 Aug 2015 12:34:38 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55DABF4A.4090805@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D71038.3010203@oracle.com> <55DABF4A.4090805@oracle.com> Message-ID: <55DAF33E.3050405@oracle.com> This version looks good to me. /Erik On 2015-08-24 08:52, Alan Bateman wrote: > > On 21/08/2015 12:49, Jaroslav Bachorik wrote: >> >> Updated webrev top-level: >> http://cr.openjdk.java.net/~jbachorik/8043937/webrev.03 >> Updated webrev jdk: >> http://cr.openjdk.java.net/~jbachorik/8043937/webrev.03/jdk >> >> Contains changes to the top level repo as well as a correct version >> of jdk/make/rmic/Rmic-java.management.gmk (thanks to Erik Joelsson) > The updated webrev and the updated @deprecated in RMIIIOPServerImpl > looks good. Can you make sure to test out boot cycle builds before > pushing this? That would help with the confidence that the updates to > the RMI stub generation are good. > > -Alan. From dmitry.samersoff at oracle.com Tue Aug 25 15:11:49 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 25 Aug 2015 18:11:49 +0300 Subject: RFR(S): 8086134 Deadlock detection fails to attach to core file In-Reply-To: <55D70F1C.3030802@oracle.com> References: <55D4B895.9000509@oracle.com> <55D6F0DA.7090902@oracle.com> <55D6F7F7.4010503@oracle.com> <55D70F1C.3030802@oracle.com> Message-ID: <55DC85B5.9000103@oracle.com> Jaroslav, > * instead of > `Arrays.toString(processBuilder.command().toArray()).replace(",", "")` > you can use > `processBuilder.command().stream().collect(Collectors.joining(" "))` - > which is slightly more concise > * typo 'depends *to* operating system' -> 'depends *on* operating system' fixed. (in-place, press shift-reload). -Dmitry On 2015-08-21 14:44, Jaroslav Bachorik wrote: > On 21.8.2015 12:05, Dmitry Samersoff wrote: >> On 2015-08-21 12:35, Jaroslav Bachorik wrote: >>> Hi Dmitry, >>> >>> On 19.8.2015 19:10, Dmitry Samersoff wrote: >>>> Everybody, >>>> >>>> Please, review test-only fix. >>>> >>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8086134/webrev.01/ >>>> >>>> The test is rewritten to use testlibrary and to detect common bad >>>> environment conditions. >>>> >>>> Tests in Hotspot and JDK workspaces are identical, except imports and >>>> the fact that JDK version runs >>>> jstack but hotspot version runs jhsdb (sa launcher). >>> >>> I will need more explanation here. >>> >>> * the issue is about the tmtools test failing; are you planning to >>> remove the failing tests? >> >> Yes. Falling test have to be removed. >> >>> * what is the reason for having the test in hotspot and in jdk? >>> wouldn't it be possible to have it just in jdk and launch it once >>> using jstack and once jhsdb? >> >> Currently, tests for tmtools (jstack, jmap etc) resides in JDK, but >> tests for SA resides in Hotspot. I would love to place all tests to the >> single place >> but this work is out of scope of this fix. > > Ok. Thanks. > > Overall looks good. Just some minor nits. > > * instead of > `Arrays.toString(processBuilder.command().toArray()).replace(",", "")` > you can use > `processBuilder.command().stream().collect(Collectors.joining(" "))` - > which is slightly more concise > * typo 'depends *to* operating system' -> 'depends *on* operating system' > > -JB- > > >> >> -Dmitry >> >>> >>> >>> Thanks, >>> >>> -JB- >>> >>>> >>>> -Dmitry >>>> >>>> >>> >> >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From jaroslav.bachorik at oracle.com Tue Aug 25 16:29:20 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 25 Aug 2015 18:29:20 +0200 Subject: RFR 8043937: Drop support for the IIOP transport from the JMX RMIConnector In-Reply-To: <55D4937D.2050802@oracle.com> References: <55D342AC.8000005@oracle.com> <55D41FE8.9000600@oracle.com> <55D44555.7090601@oracle.com> <55D4463D.2010109@oracle.com> <55D45EC6.30606@oracle.com> <55D4669A.5080307@oracle.com> <55D47BD8.4000409@oracle.com> <55D47F0F.4050903@oracle.com> <55D48C23.5040301@oracle.com> <55D4937D.2050802@oracle.com> Message-ID: <55DC97E0.2000604@oracle.com> On 19.8.2015 16:32, Alan Bateman wrote: > > > On 19/08/2015 15:01, Jaroslav Bachorik wrote: >> : >> >>> >>> There are several tests in jdk/test/javax/management that exercise IIOP >>> if available, shouldn't they be updated too? >> >> Not sure about this. These tests are also exercising JMXMP if >> available - but that has never been a part of JDK. > I think the tests date back to when JMX was a standalone JSR and RI. In > any case, with IIOP support removed then it seems like these tests > should be updated but it's not critical to do it now. The test clean up is tracked as a separate task now - JDK-8134421 -JB- > > -Alan From daniel.daugherty at oracle.com Tue Aug 25 20:11:39 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 25 Aug 2015 14:11:39 -0600 Subject: assert(_exception_caught == false) failed: _exception_caught is out of phase In-Reply-To: <55CB05A6.1010503@sap.com> References: <7F953F1E7B01F54E9F0B51450984507D3993D390@DEWDFEMB19B.global.corp.sap> <55CA4122.3080504@oracle.com> <55CB05A6.1010503@sap.com> Message-ID: <55DCCBFB.7050201@oracle.com> Richard, I didn't see any response from the Serviceability Team, but maybe I missed it. In any case, I filed: JDK-8134434 JVM_DoPrivileged() fires assert(_exception_caught == false) failed: _exception_caught is out of phase https://bugs.openjdk.java.net/browse/JDK-8134434 for your issue. Dan On 8/12/15 2:36 AM, Richard Reingruber wrote: > > Adding serviceability-dev at ... since this JVM/TI... > Thanks Dan. > > Sorry, forgot to add a stack trace. This is from amd64: > > Stack: [0x00007fe6cdacc000,0x00007fe6cdbcd000], > sp=0x00007fe6cdbc9d30, free space=1015k > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, > C=native code) > V [libjvm.so+0xfce6f1] VMError::report(outputStream*)+0x12fd > V [libjvm.so+0xfcfbed] VMError::report_and_die()+0x3fd > V [libjvm.so+0x86b82d] report_vm_error(char const*, int, char > const*, char const*)+0xb4 > V [libjvm.so+0xbe44ba] > JvmtiThreadState::clear_exception_detected()+0x4e > V [libjvm.so+0xbe3046] > JvmtiExport::clear_detected_exception(JavaThread*)+0x72 > V [libjvm.so+0xb50627] JVM_DoPrivileged+0x7e4 > C [libjava.so+0xd04e] > Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedAction_2+0x42 > j > java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object;+0 > j ExceptionCaughtOutOfPhaseAssertion.main([Ljava/lang/String;)V+59 > v ~StubRoutines::call_stub > [...] > > I can reproduce this on amd64, sparcv9 and ppc64 with dbg builds of > the current openjdk9. > > Richard. > > > On 08/11/2015 08:38 PM, Daniel D. Daugherty wrote: >> Adding serviceability-dev at ... since this JVM/TI... >> >> Dan >> >> >> On 8/11/15 9:06 AM, Reingruber, Richard wrote: >>> Hi, >>> >>> I would like to report that the assertion >>> >>> assert(_exception_caught == false) failed: _exception_caught is >>> out of phase >>> >>> at jvmtiThreadState.hpp:170 fires when running the command >>> >>> ./images/jdk/bin/java >>> -agentlib:jdwp=transport=dt_socket,address=9000,server=y,suspend=n >>> -Xbatch ExceptionCaughtOutOfPhaseAssertion >>> >>> (when analyzing you might want to add -XX:-TieredCompilation >>> -XX:-Inline '-XX:CompileCommand=compileonly *::run') >>> >>> Source Code: >>> >>> import java.security.AccessController; >>> import java.security.PrivilegedAction; >>> public class ExceptionCaughtOutOfPhaseAssertion { >>> public static void main(String[] args) { >>> PrivilegedAction action = new HotThrowingAction(); >>> System.out.println("### Warm-up"); >>> for(int i=0; i<11000; i++) { >>> try { >>> action.run(); // call run() to get it compiled >>> } catch(Throwable t) { /* ignored */ } >>> } >>> System.out.println("### Warm-up done"); >>> System.out.println("### Executing privileged action"); >>> AccessController.doPrivileged(action); >>> } >>> public static class HotThrowingAction implements >>> PrivilegedAction { >>> public Object run() { >>> throw new Error(); >>> } >>> } >>> } >>> My Analysis: >>> >>> * Error is thrown in interpreted activation of run() >>> - JvmtiThreadState::_exception_detected is set to true >>> - JvmtiThreadState::_exception_caught is set to false >>> * Error is caught in main() method >>> - JvmtiThreadState::_exception_detected is set to false >>> - JvmtiThreadState::_exception_caught is set to true >>> * run() method is compiled >>> * PrivilegedAction is executed >>> * compiled activation of run() method >>> - Error object is allocated and initialized >>> - JavaThread::_should_post_on_exceptions_flag is checked and >>> found to be false >>> -> *no* uncommon trap >>> - compiled frame is popped, rethrow stub calls >>> OptoRuntime::rethrow_C() >>> - _exception_detected is *not* set, remains false >>> - _exception_caught is still true >>> * Java call in JVM_DoPrivileged() returns with pending exception >>> * CLEAR_PENDING_EXCEPTION triggers the assertion >>> >>> How to Fix? I'm not really an expert in this area, but here are my >>> two cent: >>> >>> (a) Improve the assertion ...but how?! Toggling JVMTI exception >>> notifications does not seem >>> to be synchronized with java execution. >>> >>> (b) Calling JvmtiExport::post_exception_throw() in >>> OptoRuntime::rethrow_C() is probably not possible, >>> because of the JRT_LEAF comment for rethrow_C(), but >>> _exception_detected = true could be set. >>> >>> (c) Remove the assertion. >>> >>> I guess (b) could be acceptable. >>> >>> What do you think? >>> >>> Best regards, >>> Richard. >> From daniel.daugherty at oracle.com Tue Aug 25 21:08:26 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 25 Aug 2015 15:08:26 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() Message-ID: <55DCD94A.30705@oracle.com> Greetings, I have a "fix" for a long standing race between JVM shutdown and the JVM statistics subsystem: JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() https://bugs.openjdk.java.net/browse/JDK-8049304 Webrev URL: http://cr.openjdk.java.net/~dcubed/8049304-webrev/0-jdk9-hs-rt/ Testing: Aurora Adhoc RT-SVC nightly batch Aurora Adhoc vm.tmtools batch Kim's repro sequence for JDK-8049304 Kim's repro sequence for JDK-8129978 JPRT -testset hotspot This "fix": - adds a volatile flag to record whether PerfDataManager is holding data (PerfData objects) - adds PerfDataManager::has_PerfData() to return the flag - changes the Java monitor subsystem's use of PerfData to check both allocation of the monitor subsystem specific PerfData object and the new PerfDataManager::has_PerfData() return value If the global 'UsePerfData' option is false, the system works as it did before. If 'UsePerfData' is true (the default on non-embedded systems), the Java monitor subsystem will allocate a number of PerfData objects to record information. The objects will record information about Java monitor subsystem until the JVM shuts down. When the JVM starts to shutdown, the new PerfDataManager flag will change to false and the Java monitor subsystem will stop using the PerfData objects. This is the new behavior. As noted in the comments I added to the code, the race is still present; I'm just changing the order and the timing to reduce the likelihood of the crash. Thanks, in advance, for any comments, questions or suggestions. Dan From staffan.larsen at oracle.com Tue Aug 25 23:26:24 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 25 Aug 2015 16:26:24 -0700 Subject: RFR: JDK-8134458 Make sun/tools/jps tests non-concurrent with other tests Message-ID: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> A new test (tools/launcher/ArgFileSyntax.java) was recently added. One of the things that test does is create command lines with newlines in them. If this is run at the same time as the jps tests, the regexp in the jps tests fails on the newlines in the output. A simple workaround for this is to make sure the jps tests are not being run together with the other tests. We do this by adding the test path to exclusiveAccess.dirs in JTREG.ROOT. bug: https://bugs.openjdk.java.net/browse/JDK-8134458 Thanks, /Staffan diff --git a/test/TEST.ROOT b/test/TEST.ROOT --- a/test/TEST.ROOT +++ b/test/TEST.ROOT @@ -18,7 +18,7 @@ othervm.dirs=java/awt java/beans javax/accessibility javax/imageio javax/sound javax/print javax/management com/sun/awt sun/awt sun/java2d sun/pisces javax/xml/jaxp/testng/validation java/lang/ProcessHandle # Tests that cannot run concurrently -exclusiveAccess.dirs=java/rmi/Naming java/util/prefs sun/management/jmxremote sun/tools/jstatd sun/security/mscapi java/util/stream javax/rmi +exclusiveAccess.dirs=java/rmi/Naming java/util/prefs sun/management/jmxremote sun/tools/jstatd sun/security/mscapi java/util/stream javax/rmi sun/tools/jps # Group definitions groups=TEST.groups [closed/TEST.groups] -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Wed Aug 26 02:53:14 2015 From: martinrb at google.com (Martin Buchholz) Date: Tue, 25 Aug 2015 19:53:14 -0700 Subject: RFR: JDK-8134458 Make sun/tools/jps tests non-concurrent with other tests In-Reply-To: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> References: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> Message-ID: But it is a workaround that only reduces flakiness, right? If two jtreg invocations are on the same machine running concurrently, we still have a risk of failure? On Tue, Aug 25, 2015 at 4:26 PM, Staffan Larsen wrote: > A new test (tools/launcher/ArgFileSyntax.java) was recently added. One of > the things that test does is create command lines with newlines in them. If > this is run at the same time as the jps tests, the regexp in the jps tests > fails on the newlines in the output. A simple workaround for this is to > make sure the jps tests are not being run together with the other tests. We > do this by adding the test path to exclusiveAccess.dirs in JTREG.ROOT. > > bug: https://bugs.openjdk.java.net/browse/JDK-8134458 > > Thanks, > /Staffan > > > diff --git a/test/TEST.ROOT b/test/TEST.ROOT > --- a/test/TEST.ROOT > +++ b/test/TEST.ROOT > @@ -18,7 +18,7 @@ > othervm.dirs=java/awt java/beans javax/accessibility javax/imageio > javax/sound javax/print javax/management com/sun/awt sun/awt sun/java2d > sun/pisces javax/xml/jaxp/testng/validation java/lang/ProcessHandle > > # Tests that cannot run concurrently > -exclusiveAccess.dirs=java/rmi/Naming java/util/prefs > sun/management/jmxremote sun/tools/jstatd sun/security/mscapi > java/util/stream javax/rmi > +exclusiveAccess.dirs=java/rmi/Naming java/util/prefs > sun/management/jmxremote sun/tools/jstatd sun/security/mscapi > java/util/stream javax/rmi sun/tools/jps > > # Group definitions > groups=TEST.groups [closed/TEST.groups] > -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Wed Aug 26 04:32:36 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 25 Aug 2015 21:32:36 -0700 Subject: RFR: JDK-8134458 Make sun/tools/jps tests non-concurrent with other tests In-Reply-To: References: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> Message-ID: > On 25 aug 2015, at 19:53, Martin Buchholz wrote: > > But it is a workaround that only reduces flakiness, right? If two jtreg invocations are on the same machine running concurrently, we still have a risk of failure? Yes, but they have to be run by the same user and we ?never do that?. This isn?t the final fix for the tests, just a quick fix to get the tests to be mor stable in our CI server. /Staffan > > On Tue, Aug 25, 2015 at 4:26 PM, Staffan Larsen > wrote: > A new test (tools/launcher/ArgFileSyntax.java) was recently added. One of the things that test does is create command lines with newlines in them. If this is run at the same time as the jps tests, the regexp in the jps tests fails on the newlines in the output. A simple workaround for this is to make sure the jps tests are not being run together with the other tests. We do this by adding the test path to exclusiveAccess.dirs in JTREG.ROOT. > > bug: https://bugs.openjdk.java.net/browse/JDK-8134458 > > Thanks, > /Staffan > > > diff --git a/test/TEST.ROOT b/test/TEST.ROOT > --- a/test/TEST.ROOT > +++ b/test/TEST.ROOT > @@ -18,7 +18,7 @@ > othervm.dirs=java/awt java/beans javax/accessibility javax/imageio javax/sound javax/print javax/management com/sun/awt sun/awt sun/java2d sun/pisces javax/xml/jaxp/testng/validation java/lang/ProcessHandle > > # Tests that cannot run concurrently > -exclusiveAccess.dirs=java/rmi/Naming java/util/prefs sun/management/jmxremote sun/tools/jstatd sun/security/mscapi java/util/stream javax/rmi > +exclusiveAccess.dirs=java/rmi/Naming java/util/prefs sun/management/jmxremote sun/tools/jstatd sun/security/mscapi java/util/stream javax/rmi sun/tools/jps > > # Group definitions > groups=TEST.groups [closed/TEST.groups] > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Wed Aug 26 04:39:59 2015 From: martinrb at google.com (Martin Buchholz) Date: Tue, 25 Aug 2015 21:39:59 -0700 Subject: RFR: JDK-8134458 Make sun/tools/jps tests non-concurrent with other tests In-Reply-To: References: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> Message-ID: Looks ... OK! On Tue, Aug 25, 2015 at 9:32 PM, Staffan Larsen wrote: > > On 25 aug 2015, at 19:53, Martin Buchholz wrote: > > But it is a workaround that only reduces flakiness, right? If two jtreg > invocations are on the same machine running concurrently, we still have a > risk of failure? > > > Yes, but they have to be run by the same user and we ?never do that?. This > isn?t the final fix for the tests, just a quick fix to get the tests to be > mor stable in our CI server. > > /Staffan > > > On Tue, Aug 25, 2015 at 4:26 PM, Staffan Larsen > wrote: > >> A new test (tools/launcher/ArgFileSyntax.java) was recently added. One of >> the things that test does is create command lines with newlines in them. If >> this is run at the same time as the jps tests, the regexp in the jps tests >> fails on the newlines in the output. A simple workaround for this is to >> make sure the jps tests are not being run together with the other tests. We >> do this by adding the test path to exclusiveAccess.dirs in JTREG.ROOT. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8134458 >> >> Thanks, >> /Staffan >> >> >> diff --git a/test/TEST.ROOT b/test/TEST.ROOT >> --- a/test/TEST.ROOT >> +++ b/test/TEST.ROOT >> @@ -18,7 +18,7 @@ >> othervm.dirs=java/awt java/beans javax/accessibility javax/imageio >> javax/sound javax/print javax/management com/sun/awt sun/awt sun/java2d >> sun/pisces javax/xml/jaxp/testng/validation java/lang/ProcessHandle >> >> # Tests that cannot run concurrently >> -exclusiveAccess.dirs=java/rmi/Naming java/util/prefs >> sun/management/jmxremote sun/tools/jstatd sun/security/mscapi >> java/util/stream javax/rmi >> +exclusiveAccess.dirs=java/rmi/Naming java/util/prefs >> sun/management/jmxremote sun/tools/jstatd sun/security/mscapi >> java/util/stream javax/rmi sun/tools/jps >> >> # Group definitions >> groups=TEST.groups [closed/TEST.groups] >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Aug 26 04:46:16 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Aug 2015 14:46:16 +1000 Subject: RFR: JDK-8134458 Make sun/tools/jps tests non-concurrent with other tests In-Reply-To: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> References: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> Message-ID: <55DD4498.70708@oracle.com> Seems reasonable. Thanks, David On 26/08/2015 9:26 AM, Staffan Larsen wrote: > A new test (tools/launcher/ArgFileSyntax.java) was recently added. One > of the things that test does is create command lines with newlines in > them. If this is run at the same time as the jps tests, the regexp in > the jps tests fails on the newlines in the output. A simple workaround > for this is to make sure the jps tests are not being run together with > the other tests. We do this by adding the test path to > exclusiveAccess.dirs in JTREG.ROOT. > > bug: https://bugs.openjdk.java.net/browse/JDK-8134458 > > Thanks, > /Staffan > > > diff --git a/test/TEST.ROOT b/test/TEST.ROOT > --- a/test/TEST.ROOT > +++ b/test/TEST.ROOT > @@ -18,7 +18,7 @@ > othervm.dirs=java/awt java/beans javax/accessibility javax/imageio > javax/sound javax/print javax/management com/sun/awt sun/awt sun/java2d > sun/pisces javax/xml/jaxp/testng/validation java/lang/ProcessHandle > > # Tests that cannot run concurrently > -exclusiveAccess.dirs=java/rmi/Naming java/util/prefs > sun/management/jmxremote sun/tools/jstatd sun/security/mscapi > java/util/stream javax/rmi > +exclusiveAccess.dirs=java/rmi/Naming java/util/prefs > sun/management/jmxremote sun/tools/jstatd sun/security/mscapi > java/util/stream javax/rmi sun/tools/jps > > # Group definitions > groups=TEST.groups [closed/TEST.groups] From staffan.larsen at oracle.com Wed Aug 26 13:47:01 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Wed, 26 Aug 2015 06:47:01 -0700 Subject: RFR: JDK-8134458 Make sun/tools/jps tests non-concurrent with other tests In-Reply-To: References: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> Message-ID: <7D4FDA4C-A773-4DD5-92FB-C3F9543C27DC@oracle.com> Thanks! > On 25 aug 2015, at 21:39, Martin Buchholz wrote: > > Looks ... OK! > > On Tue, Aug 25, 2015 at 9:32 PM, Staffan Larsen > wrote: > >> On 25 aug 2015, at 19:53, Martin Buchholz > wrote: >> >> But it is a workaround that only reduces flakiness, right? If two jtreg invocations are on the same machine running concurrently, we still have a risk of failure? > > Yes, but they have to be run by the same user and we ?never do that?. This isn?t the final fix for the tests, just a quick fix to get the tests to be mor stable in our CI server. > > /Staffan > >> >> On Tue, Aug 25, 2015 at 4:26 PM, Staffan Larsen > wrote: >> A new test (tools/launcher/ArgFileSyntax.java) was recently added. One of the things that test does is create command lines with newlines in them. If this is run at the same time as the jps tests, the regexp in the jps tests fails on the newlines in the output. A simple workaround for this is to make sure the jps tests are not being run together with the other tests. We do this by adding the test path to exclusiveAccess.dirs in JTREG.ROOT. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8134458 >> >> Thanks, >> /Staffan >> >> >> diff --git a/test/TEST.ROOT b/test/TEST.ROOT >> --- a/test/TEST.ROOT >> +++ b/test/TEST.ROOT >> @@ -18,7 +18,7 @@ >> othervm.dirs=java/awt java/beans javax/accessibility javax/imageio javax/sound javax/print javax/management com/sun/awt sun/awt sun/java2d sun/pisces javax/xml/jaxp/testng/validation java/lang/ProcessHandle >> >> # Tests that cannot run concurrently >> -exclusiveAccess.dirs=java/rmi/Naming java/util/prefs sun/management/jmxremote sun/tools/jstatd sun/security/mscapi java/util/stream javax/rmi >> +exclusiveAccess.dirs=java/rmi/Naming java/util/prefs sun/management/jmxremote sun/tools/jstatd sun/security/mscapi java/util/stream javax/rmi sun/tools/jps >> >> # Group definitions >> groups=TEST.groups [closed/TEST.groups] >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Wed Aug 26 13:47:16 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Wed, 26 Aug 2015 06:47:16 -0700 Subject: RFR: JDK-8134458 Make sun/tools/jps tests non-concurrent with other tests In-Reply-To: <55DD4498.70708@oracle.com> References: <560BBBCE-6697-46FF-8638-9960130EDA34@oracle.com> <55DD4498.70708@oracle.com> Message-ID: Thanks! > On 25 aug 2015, at 21:46, David Holmes wrote: > > Seems reasonable. > > Thanks, > David > > On 26/08/2015 9:26 AM, Staffan Larsen wrote: >> A new test (tools/launcher/ArgFileSyntax.java) was recently added. One >> of the things that test does is create command lines with newlines in >> them. If this is run at the same time as the jps tests, the regexp in >> the jps tests fails on the newlines in the output. A simple workaround >> for this is to make sure the jps tests are not being run together with >> the other tests. We do this by adding the test path to >> exclusiveAccess.dirs in JTREG.ROOT. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8134458 >> >> Thanks, >> /Staffan >> >> >> diff --git a/test/TEST.ROOT b/test/TEST.ROOT >> --- a/test/TEST.ROOT >> +++ b/test/TEST.ROOT >> @@ -18,7 +18,7 @@ >> othervm.dirs=java/awt java/beans javax/accessibility javax/imageio >> javax/sound javax/print javax/management com/sun/awt sun/awt sun/java2d >> sun/pisces javax/xml/jaxp/testng/validation java/lang/ProcessHandle >> >> # Tests that cannot run concurrently >> -exclusiveAccess.dirs=java/rmi/Naming java/util/prefs >> sun/management/jmxremote sun/tools/jstatd sun/security/mscapi >> java/util/stream javax/rmi >> +exclusiveAccess.dirs=java/rmi/Naming java/util/prefs >> sun/management/jmxremote sun/tools/jstatd sun/security/mscapi >> java/util/stream javax/rmi sun/tools/jps >> >> # Group definitions >> groups=TEST.groups [closed/TEST.groups] From claes.redestad at oracle.com Wed Aug 26 14:47:59 2015 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 26 Aug 2015 16:47:59 +0200 Subject: Poll: Remove per-compiler thread perf. counters? Message-ID: <55DDD19F.4020003@oracle.com> Hi, I want to raise the question if there are any known users of these per-compiler thread perf. counters, or if they should be removed? sun.ci.compilerThread.#.compiles sun.ci.compilerThread.#.method sun.ci.compilerThread.#.time sun.ci.compilerThread.#.type For detailed information about compilation there are better tools available (JFR, PrintCompilation), whereas the older perf.counters are useful mostly for their aggregated values. /Claes From richard.reingruber at sap.com Wed Aug 26 14:55:56 2015 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 26 Aug 2015 14:55:56 +0000 Subject: assert(_exception_caught == false) failed: _exception_caught is out of phase In-Reply-To: <55DCCBFB.7050201@oracle.com> References: <7F953F1E7B01F54E9F0B51450984507D3993D390@DEWDFEMB19B.global.corp.sap> <55CA4122.3080504@oracle.com> <55CB05A6.1010503@sap.com> <55DCCBFB.7050201@oracle.com> Message-ID: <7F953F1E7B01F54E9F0B51450984507D39940668@DEWDFEMB19B.global.corp.sap> Hi Daniel, > I didn't see any response from the Serviceability Team Me neither. Thanks for filing a bug for the issue. Richard. -----Original Message----- From: Daniel D. Daugherty [mailto:daniel.daugherty at oracle.com] Sent: Dienstag, 25. August 2015 22:12 To: Reingruber, Richard Cc: hotspot-dev at openjdk.java.net; serviceability-dev at openjdk.java.net Subject: Re: assert(_exception_caught == false) failed: _exception_caught is out of phase Richard, I didn't see any response from the Serviceability Team, but maybe I missed it. In any case, I filed: JDK-8134434 JVM_DoPrivileged() fires assert(_exception_caught == false) failed: _exception_caught is out of phase https://bugs.openjdk.java.net/browse/JDK-8134434 for your issue. Dan On 8/12/15 2:36 AM, Richard Reingruber wrote: > > Adding serviceability-dev at ... since this JVM/TI... > Thanks Dan. > > Sorry, forgot to add a stack trace. This is from amd64: > > Stack: [0x00007fe6cdacc000,0x00007fe6cdbcd000], > sp=0x00007fe6cdbc9d30, free space=1015k > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, > C=native code) > V [libjvm.so+0xfce6f1] VMError::report(outputStream*)+0x12fd > V [libjvm.so+0xfcfbed] VMError::report_and_die()+0x3fd > V [libjvm.so+0x86b82d] report_vm_error(char const*, int, char > const*, char const*)+0xb4 > V [libjvm.so+0xbe44ba] > JvmtiThreadState::clear_exception_detected()+0x4e > V [libjvm.so+0xbe3046] > JvmtiExport::clear_detected_exception(JavaThread*)+0x72 > V [libjvm.so+0xb50627] JVM_DoPrivileged+0x7e4 > C [libjava.so+0xd04e] > Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedAction_2+0x42 > j > java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object;+0 > j ExceptionCaughtOutOfPhaseAssertion.main([Ljava/lang/String;)V+59 > v ~StubRoutines::call_stub > [...] > > I can reproduce this on amd64, sparcv9 and ppc64 with dbg builds of > the current openjdk9. > > Richard. > > > On 08/11/2015 08:38 PM, Daniel D. Daugherty wrote: >> Adding serviceability-dev at ... since this JVM/TI... >> >> Dan >> >> >> On 8/11/15 9:06 AM, Reingruber, Richard wrote: >>> Hi, >>> >>> I would like to report that the assertion >>> >>> assert(_exception_caught == false) failed: _exception_caught is >>> out of phase >>> >>> at jvmtiThreadState.hpp:170 fires when running the command >>> >>> ./images/jdk/bin/java >>> -agentlib:jdwp=transport=dt_socket,address=9000,server=y,suspend=n >>> -Xbatch ExceptionCaughtOutOfPhaseAssertion >>> >>> (when analyzing you might want to add -XX:-TieredCompilation >>> -XX:-Inline '-XX:CompileCommand=compileonly *::run') >>> >>> Source Code: >>> >>> import java.security.AccessController; >>> import java.security.PrivilegedAction; >>> public class ExceptionCaughtOutOfPhaseAssertion { >>> public static void main(String[] args) { >>> PrivilegedAction action = new HotThrowingAction(); >>> System.out.println("### Warm-up"); >>> for(int i=0; i<11000; i++) { >>> try { >>> action.run(); // call run() to get it compiled >>> } catch(Throwable t) { /* ignored */ } >>> } >>> System.out.println("### Warm-up done"); >>> System.out.println("### Executing privileged action"); >>> AccessController.doPrivileged(action); >>> } >>> public static class HotThrowingAction implements >>> PrivilegedAction { >>> public Object run() { >>> throw new Error(); >>> } >>> } >>> } >>> My Analysis: >>> >>> * Error is thrown in interpreted activation of run() >>> - JvmtiThreadState::_exception_detected is set to true >>> - JvmtiThreadState::_exception_caught is set to false >>> * Error is caught in main() method >>> - JvmtiThreadState::_exception_detected is set to false >>> - JvmtiThreadState::_exception_caught is set to true >>> * run() method is compiled >>> * PrivilegedAction is executed >>> * compiled activation of run() method >>> - Error object is allocated and initialized >>> - JavaThread::_should_post_on_exceptions_flag is checked and >>> found to be false >>> -> *no* uncommon trap >>> - compiled frame is popped, rethrow stub calls >>> OptoRuntime::rethrow_C() >>> - _exception_detected is *not* set, remains false >>> - _exception_caught is still true >>> * Java call in JVM_DoPrivileged() returns with pending exception >>> * CLEAR_PENDING_EXCEPTION triggers the assertion >>> >>> How to Fix? I'm not really an expert in this area, but here are my >>> two cent: >>> >>> (a) Improve the assertion ...but how?! Toggling JVMTI exception >>> notifications does not seem >>> to be synchronized with java execution. >>> >>> (b) Calling JvmtiExport::post_exception_throw() in >>> OptoRuntime::rethrow_C() is probably not possible, >>> because of the JRT_LEAF comment for rethrow_C(), but >>> _exception_detected = true could be set. >>> >>> (c) Remove the assertion. >>> >>> I guess (b) could be acceptable. >>> >>> What do you think? >>> >>> Best regards, >>> Richard. >> From kim.barrett at oracle.com Wed Aug 26 21:00:04 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 26 Aug 2015 17:00:04 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55DCD94A.30705@oracle.com> References: <55DCD94A.30705@oracle.com> Message-ID: On Aug 25, 2015, at 5:08 PM, Daniel D. Daugherty wrote: > > Greetings, > > I have a "fix" for a long standing race between JVM shutdown and the > JVM statistics subsystem: > > JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() > https://bugs.openjdk.java.net/browse/JDK-8049304 > > Webrev URL: http://cr.openjdk.java.net/~dcubed/8049304-webrev/0-jdk9-hs-rt/ Looking at the webrev, I was initially surprised at the limited number of places that are being changed to check the new predicate for PerfData validity. (The webrev only changes the Java monitor subsystem to call the new predicate, and the RFR even says that, so I really shouldn't have been surprised.) There are lots of places that use PerfData. Don't they all need to be updated? I was hoping the answer would be no, but after reviewing the bug thread and code, I think that hope was in vain. The problem is that exit_globals (and so, ultimately, perfMemory_exit and the proposed modification in PerfDataManager::destroy) is called from a variety of contexts, some of which may have concurrent threads that may touch non-monitor PerfData. A crash in the GC is just one example of the places where this situation could arise. Unfortunately, this puts me back in the position of thinking we should just leak the memory when we're on our way to process exit. That is, callers of exit_globals indicate (via a new flag argument) whether we're on the way to process exit, and that flag gets passed down to perfMemory_exit, which elides the call to PerfDataManager::destroy (but still must call PerfMemory::destroy) when we're exiting the process. Dan, sorry for reversing positions on you, but I'd been so focused on the monitor crashes that we were looking at that I missed the wider scope of the problem. From daniel.daugherty at oracle.com Wed Aug 26 21:15:14 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 26 Aug 2015 15:15:14 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: References: <55DCD94A.30705@oracle.com> Message-ID: <55DE2C62.2@oracle.com> Kim, Thanks for the review! On 8/26/15 3:00 PM, Kim Barrett wrote: > On Aug 25, 2015, at 5:08 PM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a "fix" for a long standing race between JVM shutdown and the >> JVM statistics subsystem: >> >> JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() >> https://bugs.openjdk.java.net/browse/JDK-8049304 >> >> Webrev URL: http://cr.openjdk.java.net/~dcubed/8049304-webrev/0-jdk9-hs-rt/ > Looking at the webrev, I was initially surprised at the limited number > of places that are being changed to check the new predicate for > PerfData validity. (The webrev only changes the Java monitor subsystem > to call the new predicate, and the RFR even says that, so I really > shouldn't have been surprised.) There are lots of places that use > PerfData. Don't they all need to be updated? I was hoping the answer > would be no, but after reviewing the bug thread and code, I think that > hope was in vain. I have only seen sightings of this crash in the monitor subsystem. Do you know of sightings with other PerfData usage? > The problem is that exit_globals (and so, ultimately, perfMemory_exit > and the proposed modification in PerfDataManager::destroy) is called > from a variety of contexts, some of which may have concurrent threads > that may touch non-monitor PerfData. A crash in the GC is just one > example of the places where this situation could arise. > > Unfortunately, this puts me back in the position of thinking we should > just leak the memory when we're on our way to process exit. That is, > callers of exit_globals indicate (via a new flag argument) whether > we're on the way to process exit, and that flag gets passed down to > perfMemory_exit, which elides the call to PerfDataManager::destroy > (but still must call PerfMemory::destroy) when we're exiting the > process. Sorry, I'm not in favor of leaking memory. > Dan, sorry for reversing positions on you, but I'd been so focused on > the monitor crashes that we were looking at that I missed the wider > scope of the problem. It's a little disappointing, but I'll live. :-) Dan From kim.barrett at oracle.com Wed Aug 26 22:02:09 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 26 Aug 2015 18:02:09 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55DE2C62.2@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE2C62.2@oracle.com> Message-ID: On Aug 26, 2015, at 5:15 PM, Daniel D. Daugherty wrote: > > On 8/26/15 3:00 PM, Kim Barrett wrote: >> ...There are lots of places that use >> PerfData. Don't they all need to be updated? I was hoping the answer >> would be no, but after reviewing the bug thread and code, I think that >> hope was in vain. > > I have only seen sightings of this crash in the monitor subsystem. > Do you know of sightings with other PerfData usage? I don't, but I haven't been looking. Also, the monitor subsystem gets hit a lot more heavily than any other PerfData I've looked at, and sightings in the monitor subsystem are pretty rare. I'm pretty sure I could demonstrate one by patching in some sleeps and flag spin-waits in order to achieve the necessary state. >> Unfortunately, this puts me back in the position of thinking we should >> just leak the memory when we're on our way to process exit. ... > > Sorry, I'm not in favor of leaking memory. We're on our way to process exit, where all such sins are forgiven. We already leak like a bucket without a bottom (sieves being too fine to accurately model the situation) when performing an abnormal exit. Actually, a possible upside to this leak would be that the PerfData will be present in a core file. That could maybe even be useful. From daniel.daugherty at oracle.com Wed Aug 26 22:46:39 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 26 Aug 2015 16:46:39 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: References: <55DCD94A.30705@oracle.com> <55DE2C62.2@oracle.com> Message-ID: <55DE41CF.1030002@oracle.com> On 8/26/15 4:02 PM, Kim Barrett wrote: > On Aug 26, 2015, at 5:15 PM, Daniel D. Daugherty wrote: >> On 8/26/15 3:00 PM, Kim Barrett wrote: >>> ...There are lots of places that use >>> PerfData. Don't they all need to be updated? I was hoping the answer >>> would be no, but after reviewing the bug thread and code, I think that >>> hope was in vain. >> I have only seen sightings of this crash in the monitor subsystem. >> Do you know of sightings with other PerfData usage? > I don't, but I haven't been looking. Also, the monitor subsystem gets > hit a lot more heavily than any other PerfData I've looked at, and > sightings in the monitor subsystem are pretty rare. I'm pretty confident we have the monitor case well under control and I'd like to move forward with this fix and see what else shakes out (if anything). The other idea that you had about the SIGSEGV signal handler would be the next place I'd look at if we continue to see issues with this type of race. > I'm pretty sure I could demonstrate one by patching in some sleeps and > flag spin-waits in order to achieve the necessary state. I have no doubt since you came up with the debugging code to make the monitor subsystem race easily reproducible. However, I can't find any other PerfData sighting so I'm wondering if non-monitor subsystem races are more rare. >>> Unfortunately, this puts me back in the position of thinking we should >>> just leak the memory when we're on our way to process exit. ... >> Sorry, I'm not in favor of leaking memory. > We're on our way to process exit, where all such sins are forgiven. I thought the general idea was that we're trying to reduce our memory leaks so that we can eventually reach the stretch goal of being able to restart the VM in the same process. I don't want to add another memory leak to the system without a very compelling reason to do so. So far I'm not convinced especially without any existing bugs pointing to PerfData races with non-monitor subsystem usage. > We already leak like a bucket without a bottom (sieves being too fine > to accurately model the situation) when performing an abnormal exit. But we're not talking about an abnormal exit here. > Actually, a possible upside to this leak would be that the PerfData will > be present in a core file. That could maybe even be useful. I'm pretty sure if you have a PerfData server attached, then we leave the PerfData objects laying around so they can be harvested. However, I'm not really a user of the PerfData stuff so I don't know that for certain. Dan From david.holmes at oracle.com Thu Aug 27 04:26:03 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Aug 2015 14:26:03 +1000 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55DCD94A.30705@oracle.com> References: <55DCD94A.30705@oracle.com> Message-ID: <55DE915B.9020605@oracle.com> Hi Dan, On 26/08/2015 7:08 AM, Daniel D. Daugherty wrote: > Greetings, > > I have a "fix" for a long standing race between JVM shutdown and the > JVM statistics subsystem: > > JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() > https://bugs.openjdk.java.net/browse/JDK-8049304 > > Webrev URL: http://cr.openjdk.java.net/~dcubed/8049304-webrev/0-jdk9-hs-rt/ > > Testing: Aurora Adhoc RT-SVC nightly batch > Aurora Adhoc vm.tmtools batch > Kim's repro sequence for JDK-8049304 > Kim's repro sequence for JDK-8129978 > JPRT -testset hotspot > > This "fix": > > - adds a volatile flag to record whether PerfDataManager is holding > data (PerfData objects) > - adds PerfDataManager::has_PerfData() to return the flag > - changes the Java monitor subsystem's use of PerfData to > check both allocation of the monitor subsystem specific > PerfData object and the new PerfDataManager::has_PerfData() > return value > > If the global 'UsePerfData' option is false, the system works as > it did before. If 'UsePerfData' is true (the default on non-embedded > systems), the Java monitor subsystem will allocate a number of > PerfData objects to record information. The objects will record > information about Java monitor subsystem until the JVM shuts down. > > When the JVM starts to shutdown, the new PerfDataManager flag will > change to false and the Java monitor subsystem will stop using the > PerfData objects. This is the new behavior. As noted in the comments > I added to the code, the race is still present; I'm just changing > the order and the timing to reduce the likelihood of the crash. Right. To sum up: the basic problem is that the PerfData objects are deallocated at the safepoint established for VM termination, but those objects can actually be used by threads that are in a safepoint-safe state: in particular within the low-level synchronization code. As you say this fix narrows the window where a crash can occur, but can not close it. If a thread is descheduled after the check of hasPerfData it can still access the PerfData object when it resumes, which may be after the object was deallocated. There's no true fix here without introducing synchronization (which would have to be even lower-level to avoid reentrant use of the same code we're fixing!) and the overhead of that would be prohibitive for these perf counters. In response to Kim's concern about other code that uses PerfData objects I think you would have to examine those uses to see which, if any, can occur from either a non-JavaThread, or from within the code where a thread is considered safepoint-safe. I'm inclined to agree that given we have not seen issues with such code, either it does not exist or is extremely unlikely to hit this issue. Given the "fix" is itself only narrowing the window it doesn't seem necessary to address code that already has a narrower window. That all said "leaking" the PerfData objects seems no less unpleasant a "fix". There are so many obstacles in the way of being able to unload and re-load the JVM that I do not think this makes the position measurably worse. In fact I can imagine that if we were to allow for such behaviour we would need to be able to terminate threads and reclaim all their resources (like Monitor instances), at which point it would also become easy to deallocate shared memory like PerfData objects. I'll leave it up to you which way to go. As it stands this is Reviewed. Thanks, David > Thanks, in advance, for any comments, questions or suggestions. > > Dan > From claes.redestad at oracle.com Thu Aug 27 12:41:57 2015 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 27 Aug 2015 14:41:57 +0200 Subject: RFR(XS): 8134583: sun.management.HotspotCompilation should handle absence of per-thread perf counters Message-ID: <55DF0595.4020805@oracle.com> Hi, please review this patch to clean up and make sun.management.HotspotCompilation behave nice if the VM would decide to no longer expose per-compiler thread perf counters: webrev: http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8134583 /Claes From jaroslav.bachorik at oracle.com Thu Aug 27 13:04:17 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Thu, 27 Aug 2015 15:04:17 +0200 Subject: RFR(XS): 8134583: sun.management.HotspotCompilation should handle absence of per-thread perf counters In-Reply-To: <55DF0595.4020805@oracle.com> References: <55DF0595.4020805@oracle.com> Message-ID: <55DF0AD1.4080001@oracle.com> Hi, On 27.8.2015 14:41, Claes Redestad wrote: > Hi, > > please review this patch to clean up and make > sun.management.HotspotCompilation > behave nice if the VM would decide to no longer expose per-compiler > thread perf counters: > > webrev: http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8134583 When already changing this wouldn't it be easier to convert the 'threads' variable to List and only add the info for existing compilers threads (eg. not leaving NULL slots in the array). In 'getCompilerThreadStats' method the 'threads' array is converted to a list anyway. -JB- > > /Claes From claes.redestad at oracle.com Thu Aug 27 13:14:42 2015 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 27 Aug 2015 15:14:42 +0200 Subject: RFR(XS): 8134583: sun.management.HotspotCompilation should handle absence of per-thread perf counters In-Reply-To: <55DF0AD1.4080001@oracle.com> References: <55DF0595.4020805@oracle.com> <55DF0AD1.4080001@oracle.com> Message-ID: <55DF0D42.9040803@oracle.com> On 2015-08-27 15:04, Jaroslav Bachorik wrote: > Hi, > > On 27.8.2015 14:41, Claes Redestad wrote: >> Hi, >> >> please review this patch to clean up and make >> sun.management.HotspotCompilation >> behave nice if the VM would decide to no longer expose per-compiler >> thread perf counters: >> >> webrev: http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8134583 > > When already changing this wouldn't it be easier to convert the > 'threads' variable to List and only add the info > for existing compilers threads (eg. not leaving NULL slots in the array). > > In 'getCompilerThreadStats' method the 'threads' array is converted to > a list anyway. The CompilerThreadStat object needs to be created on demand (since it polls the underlying counters), thus we still need to maintain either an array or list of CompilerThreadInfo. Converting CompilerThreadInfo[] to a compact (or empty) List may or may not save a few bytes, but we'd still have to create a new list every time getCompilerThreadStats() is called. /Claes > > -JB- > >> >> /Claes > From jaroslav.bachorik at oracle.com Thu Aug 27 13:21:19 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Thu, 27 Aug 2015 15:21:19 +0200 Subject: RFR(XS): 8134583: sun.management.HotspotCompilation should handle absence of per-thread perf counters In-Reply-To: <55DF0D42.9040803@oracle.com> References: <55DF0595.4020805@oracle.com> <55DF0AD1.4080001@oracle.com> <55DF0D42.9040803@oracle.com> Message-ID: <55DF0ECF.9000908@oracle.com> On 27.8.2015 15:14, Claes Redestad wrote: > > > On 2015-08-27 15:04, Jaroslav Bachorik wrote: >> Hi, >> >> On 27.8.2015 14:41, Claes Redestad wrote: >>> Hi, >>> >>> please review this patch to clean up and make >>> sun.management.HotspotCompilation >>> behave nice if the VM would decide to no longer expose per-compiler >>> thread perf counters: >>> >>> webrev: http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8134583 >> >> When already changing this wouldn't it be easier to convert the >> 'threads' variable to List and only add the info >> for existing compilers threads (eg. not leaving NULL slots in the array). >> >> In 'getCompilerThreadStats' method the 'threads' array is converted to >> a list anyway. > > The CompilerThreadStat object needs to be created on demand (since it > polls the underlying counters), thus we still need to maintain either an > array or list of CompilerThreadInfo. Converting CompilerThreadInfo[] to > a compact (or empty) List may or may not save a few > bytes, but we'd still have to create a new list every time > getCompilerThreadStats() is called. Right. Still could save some null value juggling by storing CompilerThreadInfo instances into a list instead of an array. -JB- > > /Claes > >> >> -JB- >> >>> >>> /Claes >> > From claes.redestad at oracle.com Thu Aug 27 14:42:30 2015 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 27 Aug 2015 16:42:30 +0200 Subject: RFR(XS): 8134583: sun.management.HotspotCompilation should handle absence of per-thread perf counters In-Reply-To: <55DF0ECF.9000908@oracle.com> References: <55DF0595.4020805@oracle.com> <55DF0AD1.4080001@oracle.com> <55DF0D42.9040803@oracle.com> <55DF0ECF.9000908@oracle.com> Message-ID: <55DF21D6.5030407@oracle.com> Updated webrev after comments and discussion with Jaroslav: http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.03 Changes: - convert 'threads' from array to list - simplified further by removing old code dealing with adapterThread /Claes On 2015-08-27 15:21, Jaroslav Bachorik wrote: > On 27.8.2015 15:14, Claes Redestad wrote: >> >> >> On 2015-08-27 15:04, Jaroslav Bachorik wrote: >>> Hi, >>> >>> On 27.8.2015 14:41, Claes Redestad wrote: >>>> Hi, >>>> >>>> please review this patch to clean up and make >>>> sun.management.HotspotCompilation >>>> behave nice if the VM would decide to no longer expose per-compiler >>>> thread perf counters: >>>> >>>> webrev: http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8134583 >>> >>> When already changing this wouldn't it be easier to convert the >>> 'threads' variable to List and only add the info >>> for existing compilers threads (eg. not leaving NULL slots in the >>> array). >>> >>> In 'getCompilerThreadStats' method the 'threads' array is converted to >>> a list anyway. >> >> The CompilerThreadStat object needs to be created on demand (since it >> polls the underlying counters), thus we still need to maintain either an >> array or list of CompilerThreadInfo. Converting CompilerThreadInfo[] to >> a compact (or empty) List may or may not save a few >> bytes, but we'd still have to create a new list every time >> getCompilerThreadStats() is called. > > Right. Still could save some null value juggling by storing > CompilerThreadInfo instances into a list instead of an array. > > -JB- > >> >> /Claes >> >>> >>> -JB- >>> >>>> >>>> /Claes >>> >> > From jaroslav.bachorik at oracle.com Thu Aug 27 15:03:34 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Thu, 27 Aug 2015 17:03:34 +0200 Subject: RFR(XS): 8134583: sun.management.HotspotCompilation should handle absence of per-thread perf counters In-Reply-To: <55DF21D6.5030407@oracle.com> References: <55DF0595.4020805@oracle.com> <55DF0AD1.4080001@oracle.com> <55DF0D42.9040803@oracle.com> <55DF0ECF.9000908@oracle.com> <55DF21D6.5030407@oracle.com> Message-ID: <55DF26C6.2050706@oracle.com> Hi Claes, On 27.8.2015 16:42, Claes Redestad wrote: > Updated webrev after comments and discussion with Jaroslav: > > http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.03 > > Changes: > - convert 'threads' from array to list > - simplified further by removing old code dealing with adapterThread Thanks, I like this simplification. Good to go! -JB- > > /Claes > > On 2015-08-27 15:21, Jaroslav Bachorik wrote: >> On 27.8.2015 15:14, Claes Redestad wrote: >>> >>> >>> On 2015-08-27 15:04, Jaroslav Bachorik wrote: >>>> Hi, >>>> >>>> On 27.8.2015 14:41, Claes Redestad wrote: >>>>> Hi, >>>>> >>>>> please review this patch to clean up and make >>>>> sun.management.HotspotCompilation >>>>> behave nice if the VM would decide to no longer expose per-compiler >>>>> thread perf counters: >>>>> >>>>> webrev: http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.00/ >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8134583 >>>> >>>> When already changing this wouldn't it be easier to convert the >>>> 'threads' variable to List and only add the info >>>> for existing compilers threads (eg. not leaving NULL slots in the >>>> array). >>>> >>>> In 'getCompilerThreadStats' method the 'threads' array is converted to >>>> a list anyway. >>> >>> The CompilerThreadStat object needs to be created on demand (since it >>> polls the underlying counters), thus we still need to maintain either an >>> array or list of CompilerThreadInfo. Converting CompilerThreadInfo[] to >>> a compact (or empty) List may or may not save a few >>> bytes, but we'd still have to create a new list every time >>> getCompilerThreadStats() is called. >> >> Right. Still could save some null value juggling by storing >> CompilerThreadInfo instances into a list instead of an array. >> >> -JB- >> >>> >>> /Claes >>> >>>> >>>> -JB- >>>> >>>>> >>>>> /Claes >>>> >>> >> > From daniel.daugherty at oracle.com Thu Aug 27 15:51:19 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 27 Aug 2015 09:51:19 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55DE915B.9020605@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> Message-ID: <55DF31F7.7050609@oracle.com> Hi David! Thanks for chiming in on this thread! Replies embedded below as usual... On 8/26/15 10:26 PM, David Holmes wrote: > Hi Dan, > > On 26/08/2015 7:08 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a "fix" for a long standing race between JVM shutdown and the >> JVM statistics subsystem: >> >> JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() >> https://bugs.openjdk.java.net/browse/JDK-8049304 >> >> Webrev URL: >> http://cr.openjdk.java.net/~dcubed/8049304-webrev/0-jdk9-hs-rt/ >> >> Testing: Aurora Adhoc RT-SVC nightly batch >> Aurora Adhoc vm.tmtools batch >> Kim's repro sequence for JDK-8049304 >> Kim's repro sequence for JDK-8129978 >> JPRT -testset hotspot >> >> This "fix": >> >> - adds a volatile flag to record whether PerfDataManager is holding >> data (PerfData objects) >> - adds PerfDataManager::has_PerfData() to return the flag >> - changes the Java monitor subsystem's use of PerfData to >> check both allocation of the monitor subsystem specific >> PerfData object and the new PerfDataManager::has_PerfData() >> return value >> >> If the global 'UsePerfData' option is false, the system works as >> it did before. If 'UsePerfData' is true (the default on non-embedded >> systems), the Java monitor subsystem will allocate a number of >> PerfData objects to record information. The objects will record >> information about Java monitor subsystem until the JVM shuts down. >> >> When the JVM starts to shutdown, the new PerfDataManager flag will >> change to false and the Java monitor subsystem will stop using the >> PerfData objects. This is the new behavior. As noted in the comments >> I added to the code, the race is still present; I'm just changing >> the order and the timing to reduce the likelihood of the crash. > > Right. To sum up: the basic problem is that the PerfData objects are > deallocated at the safepoint established for VM termination, but those > objects can actually be used by threads that are in a safepoint-safe > state: in particular within the low-level synchronization code. > > As you say this fix narrows the window where a crash can occur, but > can not close it. If a thread is descheduled after the check of > hasPerfData it can still access the PerfData object when it resumes, > which may be after the object was deallocated. There's no true fix > here without introducing synchronization (which would have to be even > lower-level to avoid reentrant use of the same code we're fixing!) and > the overhead of that would be prohibitive for these perf counters. > > In response to Kim's concern about other code that uses PerfData > objects I think you would have to examine those uses to see which, if > any, can occur from either a non-JavaThread, or from within the code > where a thread is considered safepoint-safe. I'm inclined to agree > that given we have not seen issues with such code, either it does not > exist or is extremely unlikely to hit this issue. Given the "fix" is > itself only narrowing the window it doesn't seem necessary to address > code that already has a narrower window. > > That all said "leaking" the PerfData objects seems no less unpleasant > a "fix". There are so many obstacles in the way of being able to > unload and re-load the JVM that I do not think this makes the position > measurably worse. In fact I can imagine that if we were to allow for > such behaviour we would need to be able to terminate threads and > reclaim all their resources (like Monitor instances), at which point > it would also become easy to deallocate shared memory like PerfData > objects. Here's what I wrote in the bug report before I started this review cycle: Daniel Daugherty added a comment - 2015-08-21 20:40 Continued investigating VM shutdown race: JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() JDK-8129978 SIGSEGV when parsing command line options - Thanks to Kim for providing easy reproduction instructions for both bugs; I've tweaked the repro code a bit - The "correct" solution is to add a locking/memory ordering mechanism to ensure that PerfData is only used when valid. The locking/memory ordering would slow down the PerfData mechanism for every update. Ouch! - The "fast but safe" solution is to leak the PerfData memory and not clean them up at VM shutdown. We're trying to clean up the code base so the idea of intentionally leaking memory makes me cringe. - The solution I'm investigating is between "fast but safe" and "correct". I'm adding a PerfDataManager.has_PerfData() function that returns true when PerfDataManager is holding PerfData objects and false when none have been allocated or when they have been freed at VM shutdown. The flag holding the state is volatile and I use release_store() to change it so that publication is visible more quickly. On the VM shutdown path, I also do a 1ms sleep after setting the flag and before freeing the memory. - The idea is that the 1ms sleep will give any threads that saw PerfDataManager.has_PerfData() == true a chance to do their operation on the PerfData object before VM shutdown thread frees it. So I think we're all roughly on the same page here: 1) We don't like the current system because we keep getting these shutdown race crashes. Of course, a new one came in early this AM: JDK-8134566 java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java crashes in monitor synchronization code https://bugs.openjdk.java.net/browse/JDK-8134566 2) We don't like the "correct" solution because it would slow down the performance counters and possibly skew the very data we are trying to gather. Kim has also pointed out that adding more locking in a subsystem used by higher level locking is risky. 3) We don't like the "fast but safe" solution of leaking the PerfData memory. We try to make ourselves feel better about this by saying there are plenty of other leaks in the VM... slippery slope? 4) We don't like the proposed solution because the race still exists and we could continue to see failures like these. Only they would be more rare and possibly harder to spot. 5) Off-thread Kim and I have been talking about adding logic to the signal handler filters to detect a SIGSEGV that comes from use of a now freed PerfData object. We're mulling on the idea, but have not determined if it is even possible or an acceptable idea... Hopefully, the above accurately sums up our options... > I'll leave it up to you which way to go. As it stands this is Reviewed. Thanks! Here's my proposed plan: 1) I'd like to move forward with this change in order to reduce the occurrences of this crasher. Yes, I'm getting tired of seeing and analyzing them. 2) If we see PerfData crashes in the future with non-monitor subsystem PerfData usage, then we look at adding has_PerfData() calls to that subsystem. 3) If we see PerfData crashes in the future in the monitor subsystem, then that indicates that the theoretical race is real or I missed protecting a PerfData usage with has_PerfData(). If the race is real, then we examine these alternatives: - leak the PerfData objects on the JVM shutdown path, i.e., switch to the "fast but safe" solution - add signal handler support to make PerfData SIGSEGVs benign What do folks think? Dan > > Thanks, > David > >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> From kim.barrett at oracle.com Thu Aug 27 17:47:50 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 27 Aug 2015 13:47:50 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55DF31F7.7050609@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF31F7.7050609@oracle.com> Message-ID: <1457BA32-6B88-4259-8E10-0F4CA93C8856@oracle.com> On Aug 27, 2015, at 11:51 AM, Daniel D. Daugherty wrote: > > 3) We don't like the "fast but safe" solution of leaking the PerfData > memory. We try to make ourselves feel better about this by saying > there are plenty of other leaks in the VM... slippery slope? "Leak" is perhaps misleading and overly perjorative. A better description of what I've suggested is "pass the buck for memory cleanup to the OS". When we're on our way to process exit, we know the OS will deal with our memory resources. We already (and would still) clean up relevant non-memory resources (like shared memory files for the PerfData) - I'm not suggesting any change there. perfMemory_exit already has that separation of memory vs non-memory resource cleanup. I submit that in many situations when we're on our way to process exit, the sleep being introduced by this change may actually have a significant negative impact. If we're on our way to dumping a core file for debugging an error, putting a sleep along the way just allows other threads more time to run further from the state where the error occurred. I wish we were running less code rather than more in that situation. If we're not on our way to process exit, and instead want to achieve a state where we can restart the VM or unload the VM code, we clearly need to make sure that other cleanup has been done, such as bringing all threads to quiescence and eventually tearing them down. But if we don't have other threads running then the problem of some thread trying to touch the PerfData after we've destroyed it simply doesn't happen. And indeed, there is at least the beginnings of code to do that sort of thing; see jni_DestroyJavaVM. And what I've proposed is that we do the PerfData memory cleanup exactly and only when that's the goal state, since in that case it is safe and necessary to do the PerfData memory cleanup. From daniel.daugherty at oracle.com Thu Aug 27 17:59:03 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 27 Aug 2015 11:59:03 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <1457BA32-6B88-4259-8E10-0F4CA93C8856@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF31F7.7050609@oracle.com> <1457BA32-6B88-4259-8E10-0F4CA93C8856@oracle.com> Message-ID: <55DF4FE7.9050509@oracle.com> On 8/27/15 11:47 AM, Kim Barrett wrote: > On Aug 27, 2015, at 11:51 AM, Daniel D. Daugherty wrote: >> 3) We don't like the "fast but safe" solution of leaking the PerfData >> memory. We try to make ourselves feel better about this by saying >> there are plenty of other leaks in the VM... slippery slope? > "Leak" is perhaps misleading and overly perjorative. A better > description of what I've suggested is "pass the buck for memory > cleanup to the OS". When we're on our way to process exit, we know > the OS will deal with our memory resources. We already (and would > still) clean up relevant non-memory resources (like shared memory > files for the PerfData) - I'm not suggesting any change there. > perfMemory_exit already has that separation of memory vs non-memory > resource cleanup. > > I submit that in many situations when we're on our way to process > exit, the sleep being introduced by this change may actually have a > significant negative impact. If we're on our way to dumping a core > file for debugging an error, putting a sleep along the way just allows > other threads more time to run further from the state where the error > occurred. I wish we were running less code rather than more in that > situation. > > If we're not on our way to process exit, and instead want to achieve a > state where we can restart the VM or unload the VM code, we clearly > need to make sure that other cleanup has been done, such as bringing > all threads to quiescence and eventually tearing them down. But if we > don't have other threads running then the problem of some thread > trying to touch the PerfData after we've destroyed it simply doesn't > happen. > > And indeed, there is at least the beginnings of code to do that sort > of thing; see jni_DestroyJavaVM. And what I've proposed is that we do > the PerfData memory cleanup exactly and only when that's the goal > state, since in that case it is safe and necessary to do the PerfData > memory cleanup. > Kim, you keep changing your position. You've gone from acknowledging that this would be a "leak" that is likely detectible by a leak detection tool and saying that we'd have to figure out a way to shut it up to what you have written above. I have no idea what to think anymore. Dan From david.holmes at oracle.com Thu Aug 27 21:26:41 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Aug 2015 07:26:41 +1000 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <1457BA32-6B88-4259-8E10-0F4CA93C8856@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF31F7.7050609@oracle.com> <1457BA32-6B88-4259-8E10-0F4CA93C8856@oracle.com> Message-ID: <55DF8091.8040608@oracle.com> On 28/08/2015 3:47 AM, Kim Barrett wrote: > On Aug 27, 2015, at 11:51 AM, Daniel D. Daugherty wrote: >> >> 3) We don't like the "fast but safe" solution of leaking the PerfData >> memory. We try to make ourselves feel better about this by saying >> there are plenty of other leaks in the VM... slippery slope? > > "Leak" is perhaps misleading and overly perjorative. A better > description of what I've suggested is "pass the buck for memory > cleanup to the OS". When we're on our way to process exit, we know > the OS will deal with our memory resources. We already (and would > still) clean up relevant non-memory resources (like shared memory > files for the PerfData) - I'm not suggesting any change there. > perfMemory_exit already has that separation of memory vs non-memory > resource cleanup. I agree this is not a "leak" in the regular sense it is simply memory not explicitly returned before process exit - of which we already have a lot. Hence I'm not opposed to simply making PerfData objects "permanent". > I submit that in many situations when we're on our way to process > exit, the sleep being introduced by this change may actually have a > significant negative impact. If we're on our way to dumping a core > file for debugging an error, putting a sleep along the way just allows > other threads more time to run further from the state where the error > occurred. I wish we were running less code rather than more in that > situation. If we're on our way to dumping a core file, the error was in the current thread. But point noted other threads will continue longer and potentially hit secondary errors. > If we're not on our way to process exit, and instead want to achieve a > state where we can restart the VM or unload the VM code, we clearly > need to make sure that other cleanup has been done, such as bringing > all threads to quiescence and eventually tearing them down. But if we > don't have other threads running then the problem of some thread > trying to touch the PerfData after we've destroyed it simply doesn't > happen. Right - once we have solved the problem of how to terminate all the existing threads and reclaim their resources, then reclaiming a shared resource like the PerfData objects is "trivial". > And indeed, there is at least the beginnings of code to do that sort > of thing; see jni_DestroyJavaVM. And what I've proposed is that we do I think you misunderstand what jni_DestroyJavaVM is doing. It implements the normal lifecycle management for the JVM - which is that the JVM keeps running as long as there is one non-daemon thread running. So the only waiting in that method is for the non-daemon thread count to hit zero, at which point VM termination is initiated. But there can be a zillion daemon threads still running (application and VM). Cheers, David > the PerfData memory cleanup exactly and only when that's the goal > state, since in that case it is safe and necessary to do the PerfData > memory cleanup. > From daniel.daugherty at oracle.com Thu Aug 27 21:42:06 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 27 Aug 2015 15:42:06 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55DE915B.9020605@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> Message-ID: <55DF842E.1010407@oracle.com> Sorry for starting another e-mail thread fork in an already complicated review... I was mulling on this piece of the thread over lunch (my TZ): > Right. To sum up: the basic problem is that the PerfData objects > are deallocated at the safepoint established for VM termination, > but those objects can actually be used by threads that are in a > safepoint-safe state: in particular within the low-level > synchronization code. and that got me thinking about safepoints and VM exit/shutdown/termination. The problem that I'm trying to solve is the one where we have a test that has passed for all intents and purposes. It has logged success in a message to stdout, stderr, or a log file. The test is exiting with a successful exit code (exit_code == 0 for some, exit_code == 95 for others)... On the way out the door, the test crashes with a SIGSEGV. In these failures, the VM was doing a normal shutdown. In the case where main() has fallen off the end, jni_DestroyJavaVM() is called. In the case where System.exit() is called from Java then vm_exit() is called. src/share/vm/runtime/java.cpp: 503 void vm_exit(int code) { 511 if (VMThread::vm_thread() != NULL) { 512 // Fire off a VM_Exit operation to bring VM to a safepoint and exit 513 VM_Exit op(code); 514 if (thread->is_Java_thread()) 515 ((JavaThread*)thread)->set_thread_state(_thread_in_vm); 516 VMThread::execute(&op); src/share/vm/runtime/vm_operations.cpp: 443 void VM_Exit::doit() { 457 exit_globals(); exit_globals() calls perfMemory_exit() which calls PerfDataManager::destroy() which sets the flag that says there is no more PerfData... So a System.exit() call results in setting the new flag at a safepoint. The jni_DestroyJavaVM() is more complicated: src/share/vm/prims/jni.cpp: 4081 jint JNICALL jni_DestroyJavaVM(JavaVM *vm) { 4105 if (Threads::destroy_vm()) { src/share/vm/runtime/thread.cpp: 3890 bool Threads::destroy_vm() { 3896 // Wait until we are the last non-daemon thread to execute 3897 { MutexLocker nu(Threads_lock); 3898 while (Threads::number_of_non_daemon_threads() > 1) After this while-loop there are no non-daemon JavaThreads. 3926 // Stop VM thread. 3927 { 3928 // 4945125 The vm thread comes to a safepoint during exit. 3929 // GC vm_operations can get caught at the safepoint, and the 3930 // heap is unparseable if they are caught. Grab the Heap_lock 3931 // to prevent this. The GC vm_operations will not be able to 3932 // queue until after the vm thread is dead. After this point, 3933 // we'll never emerge out of the safepoint before the VM exits. 3934 3935 MutexLocker ml(Heap_lock); 3936 3937 VMThread::wait_for_vm_thread_exit(); During VMThread exit, the system was brought to a safepoint. All daemon JavaThreads are at a safepoint. 3966 // exit_globals() will delete tty 3967 exit_globals(); exit_globals() calls perfMemory_exit() which calls PerfDataManager::destroy() which sets the flag that says there is no more PerfData... So a jni_DestroyJavaVM() call results in setting the flag at a safepoint. Whew!!! So the two "normal" exit paths both set the new flag while the system is at a safepoint (as David H said). The remaining question is whether the Java monitor PerfData usage can happen in parallel while the system is at a safepoint. Fortunately, I converted all of the Java monitor subsystem's usage of PerfData to use a new macro called OM_PERFDATA_OP so I can easily visit them all: src/share/vm/runtime/objectMonitor.cpp: ObjectMonitor::enter() OM_PERFDATA_OP(ContendedLockAttempts, inc()) at the end to record a contended monitor enter. The thread is not seen as safepoint-safe at this point so no conflict. src/share/vm/runtime/objectMonitor.cpp: ObjectMonitor::EnterI() OM_PERFDATA_OP(FutileWakeups, inc()) after returning from a park() call and not being able to acquire the lock. The thread is in state _thread_blocked so the thread is seen as safepoint-safe so we may have a conflict. src/share/vm/runtime/objectMonitor.cpp: ObjectMonitor::ReenterI() OM_PERFDATA_OP(FutileWakeups, inc()) after returning from a park() call and not being able to acquire the lock. The thread is in state _thread_blocked so the thread is seen as safepoint-safe so we may have a conflict. src/share/vm/runtime/objectMonitor.cpp: ObjectMonitor::ExitEpilog() OM_PERFDATA_OP(Parks, inc()) at the end of the function after releasing ownership of the monitor. The thread is not seen as safepoint-safe at this point so no conflict. src/share/vm/runtime/objectMonitor.cpp: ObjectMonitor::notify() OM_PERFDATA_OP(Notifications, inc(1)) at the end of the function after internal INotify() has returned. The thread is not seen as safepoint-safe at this point so no conflict. src/share/vm/runtime/objectMonitor.cpp: ObjectMonitor::notifyAll() OM_PERFDATA_OP(Notifications, inc(tally)) at the end of the function after internal INotify() while-loop has finished. The thread is not seen as safepoint-safe at this point so no conflict. src/share/vm/runtime/synchronizer.cpp: ObjectSynchronizer::quick_notify() OM_PERFDATA_OP(Notifications, inc(tally)) after internal INotify() has been called on one or more monitors. The thread is not seen as safepoint-safe at this point so no conflict. src/share/vm/runtime/synchronizer.cpp: ObjectSynchronizer::inflate() OM_PERFDATA_OP(Inflations, inc()) in two places after we've successfully inflated a stack lock into an ObjectMonitor; The thread is not seen as safepoint-safe at this point so no conflict. src/share/vm/runtime/synchronizer.cpp: ObjectSynchronizer::deflate_idle_monitors() OM_PERFDATA_OP(Deflations, inc(nScavenged)) and OM_PERFDATA_OP(MonExtant, set_value(nInCirculation)) after we've scavenged the global list. This function is only called at a safepoint as a periodic cleanup task. No conflict because the periodic task stuff is already disabled. So the only two conflicts that I see are the futile wakeup in EnterI() and ReenterI(). In both of those cases, the thread was in park() and had to be spuriously unpark()'ed or intentionally unpark()'ed after being chosen as the successor for a contended monitor. Additionally, the associated monitor has be held by another thread which is why this is called a futile wakeup. This brings us full circle... The futile wakeup case happens to be the original sighting that motivated me to file this bug back on 2014.07.03. The EnterI() futile wakeup is also the case that Kim's debug repro code tickles for this bug. Kim's repro case for this bug requires a CTRL-C, but I think that results in a mostly orderly shutdown of the VM. Since the test case requires an interactive java session, I tested the fix with delays in place for 50 runs without a failure two different times. So the EnterI() futile wakeup case is tested with delays in place and I can't get the bug to reproduce anymore. Without the fix and with the delays, the crash happens everytime. For the other bug that Kim did repro code for: JDK-8129978, that crash happened in the ObjectMonitor inflation code path and that case is covered by the OM_PERFDATA_OP(Inflations, inc()) notes above. The flag is set at a safepoint, the safepoint ends, the stack lock is inflated into an ObjectMonitor while the system is not at a safepoint, the flag is seen and the PerfData is not used so no crash. I did a 1000 runs with the delays in place to verify that the repro for JDK-8129978 no longer crashes. Short version (if you read this far, you should be laughing at this :-)) I still think this is a good "fix" for this problem. I'm not yet convinced that we need to take the path where we "leak" PerfData objects. Thanks for reading this far... :-) Dan On 8/26/15 10:26 PM, David Holmes wrote: > Hi Dan, > > On 26/08/2015 7:08 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a "fix" for a long standing race between JVM shutdown and the >> JVM statistics subsystem: >> >> JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() >> https://bugs.openjdk.java.net/browse/JDK-8049304 >> >> Webrev URL: >> http://cr.openjdk.java.net/~dcubed/8049304-webrev/0-jdk9-hs-rt/ >> >> Testing: Aurora Adhoc RT-SVC nightly batch >> Aurora Adhoc vm.tmtools batch >> Kim's repro sequence for JDK-8049304 >> Kim's repro sequence for JDK-8129978 >> JPRT -testset hotspot >> >> This "fix": >> >> - adds a volatile flag to record whether PerfDataManager is holding >> data (PerfData objects) >> - adds PerfDataManager::has_PerfData() to return the flag >> - changes the Java monitor subsystem's use of PerfData to >> check both allocation of the monitor subsystem specific >> PerfData object and the new PerfDataManager::has_PerfData() >> return value >> >> If the global 'UsePerfData' option is false, the system works as >> it did before. If 'UsePerfData' is true (the default on non-embedded >> systems), the Java monitor subsystem will allocate a number of >> PerfData objects to record information. The objects will record >> information about Java monitor subsystem until the JVM shuts down. >> >> When the JVM starts to shutdown, the new PerfDataManager flag will >> change to false and the Java monitor subsystem will stop using the >> PerfData objects. This is the new behavior. As noted in the comments >> I added to the code, the race is still present; I'm just changing >> the order and the timing to reduce the likelihood of the crash. > > Right. To sum up: the basic problem is that the PerfData objects are > deallocated at the safepoint established for VM termination, but those > objects can actually be used by threads that are in a safepoint-safe > state: in particular within the low-level synchronization code. > > As you say this fix narrows the window where a crash can occur, but > can not close it. If a thread is descheduled after the check of > hasPerfData it can still access the PerfData object when it resumes, > which may be after the object was deallocated. There's no true fix > here without introducing synchronization (which would have to be even > lower-level to avoid reentrant use of the same code we're fixing!) and > the overhead of that would be prohibitive for these perf counters. > > In response to Kim's concern about other code that uses PerfData > objects I think you would have to examine those uses to see which, if > any, can occur from either a non-JavaThread, or from within the code > where a thread is considered safepoint-safe. I'm inclined to agree > that given we have not seen issues with such code, either it does not > exist or is extremely unlikely to hit this issue. Given the "fix" is > itself only narrowing the window it doesn't seem necessary to address > code that already has a narrower window. > > That all said "leaking" the PerfData objects seems no less unpleasant > a "fix". There are so many obstacles in the way of being able to > unload and re-load the JVM that I do not think this makes the position > measurably worse. In fact I can imagine that if we were to allow for > such behaviour we would need to be able to terminate threads and > reclaim all their resources (like Monitor instances), at which point > it would also become easy to deallocate shared memory like PerfData > objects. > > I'll leave it up to you which way to go. As it stands this is Reviewed. > > Thanks, > David > >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> > > From kim.barrett at oracle.com Fri Aug 28 00:16:39 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 27 Aug 2015 20:16:39 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55DF842E.1010407@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> Message-ID: <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> On Aug 27, 2015, at 5:42 PM, Daniel D. Daugherty wrote: > > Sorry for starting another e-mail thread fork in an already complicated > review... OK, that was fascinating. No, really, I mean it. It made me realize that we've been arguing and talking past each other in part because we're really dealing with two distinct though closely related bugs here. I've been primarily thinking about the case where we're calling vm_abort / os::abort, where the we presently delete the PerfData memory even though there can be arbitrary other threads running. This was the case in JDK-8129978, which is how I got involved here in the first place. In that bug we were in vm_exit_during_initialization and had called perfMemory_exit when some thread attempted to inflate a monitor (which is not one of the conflicting cases discussed by Dan). The problem Dan has been looking at, JDK-8049304, is about a "normal" VM shutdown. In this case, the problem is that we believe it is safe to delete the PerfData, because we've safepointed, and yet some thread unexpectedly runs and attempts to touch the deleted data anyway. I think Dan's proposed fix (mostly) avoids the specific instance of JDK-8129978, but doesn't solve the more general problem of abnormal exit deleting the PerfData while some running thread is touching some non-monitor-related part of that data. My proposal to leave it to the OS to deal with memory cleanup on process exit would deal with this case. I think Dan's proposed fix (mostly) avoids problems like JDK-8049304. And the approach I've been talking about doesn't help at all for this case. But I wonder if Dan's proposed fix can be improved. A "futile wakeup" case doesn't seem to me like one which requires super-high performance. Would it be ok, in the two problematic cases that Dan identified, to use some kind of atomic / locking protocol with the cleanup? Or is the comment for the counter increment in EnterI (and only there) correct that it's important to avoid a lock or atomics here (and presumably in ReenterI too). From claes.redestad at oracle.com Fri Aug 28 11:58:30 2015 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 28 Aug 2015 13:58:30 +0200 Subject: RFR(XS): 8134583: sun.management.HotspotCompilation should handle absence of per-thread perf counters In-Reply-To: <55DF26C6.2050706@oracle.com> References: <55DF0595.4020805@oracle.com> <55DF0AD1.4080001@oracle.com> <55DF0D42.9040803@oracle.com> <55DF0ECF.9000908@oracle.com> <55DF21D6.5030407@oracle.com> <55DF26C6.2050706@oracle.com> Message-ID: <55E04CE6.1040801@oracle.com> Hi Jaroslav, On 2015-08-27 17:03, Jaroslav Bachorik wrote: > Hi Claes, > > On 27.8.2015 16:42, Claes Redestad wrote: >> Updated webrev after comments and discussion with Jaroslav: >> >> http://cr.openjdk.java.net/~redestad/jdk9/8134583/webrev.03 >> >> Changes: >> - convert 'threads' from array to list >> - simplified further by removing old code dealing with adapterThread > > Thanks, I like this simplification. > > Good to go! Thanks for reviewing! I'll push this via jdk9/hs-comp /Claes From tom.benson at oracle.com Fri Aug 28 14:45:08 2015 From: tom.benson at oracle.com (Tom Benson) Date: Fri, 28 Aug 2015 10:45:08 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> Message-ID: <55E073F4.5050906@oracle.com> Hi, One more pair of eyes on this. 8^) On 8/27/2015 8:16 PM, Kim Barrett wrote: > On Aug 27, 2015, at 5:42 PM, Daniel D. Daugherty wrote: >> Sorry for starting another e-mail thread fork in an already complicated >> review... > OK, that was fascinating. No, really, I mean it. > > It made me realize that we've been arguing and talking past each other > in part because we're really dealing with two distinct though closely > related bugs here. > > I've been primarily thinking about the case where we're calling > vm_abort / os::abort, where the we presently delete the PerfData > memory even though there can be arbitrary other threads running. This > was the case in JDK-8129978, which is how I got involved here in the > first place. In that bug we were in vm_exit_during_initialization and > had called perfMemory_exit when some thread attempted to inflate a > monitor (which is not one of the conflicting cases discussed by Dan). > > The problem Dan has been looking at, JDK-8049304, is about a "normal" > VM shutdown. In this case, the problem is that we believe it is safe > to delete the PerfData, because we've safepointed, and yet some thread > unexpectedly runs and attempts to touch the deleted data anyway. > > I think Dan's proposed fix (mostly) avoids the specific instance of > JDK-8129978, but doesn't solve the more general problem of abnormal > exit deleting the PerfData while some running thread is touching some > non-monitor-related part of that data. My proposal to leave it to the > OS to deal with memory cleanup on process exit would deal with this > case. > > I think Dan's proposed fix (mostly) avoids problems like JDK-8049304. > And the approach I've been talking about doesn't help at all for this > case. But I wonder if Dan's proposed fix can be improved. A "futile > wakeup" case doesn't seem to me like one which requires super-high > performance. Would it be ok, in the two problematic cases that Dan > identified, to use some kind of atomic / locking protocol with the > cleanup? Or is the comment for the counter increment in EnterI (and > only there) correct that it's important to avoid a lock or atomics > here (and presumably in ReenterI too). > I notice that EnteriI/ReenterI both end with OrderAccess::fence(). Can the potential update of _sync_FutileWakeups be delayed until that point, to take advantage of the fence to make the sync hole even smaller? You've got a release() (and and short nap!) with the store in PerfDataManager::destroy() to try to close the window somewhat. But I think rather than the release_store() you used, you want a store, followed by a release(). release_store() puts a fence before the store to ensure earlier updates are seen before the current one, no? Also, I think the comment above that release_store() could be clarified. It is fine as is if you're familiar with this bug report and discussion, but... I think it should explicitly say there is still a very small window for the lack of true synchronization to cause a failure. And perhaps that the release_store() (or store/release()) is not half of an acquire/release pair. Tom From daniel.daugherty at oracle.com Fri Aug 28 15:52:33 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2015 09:52:33 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> Message-ID: <55E083C1.8090709@oracle.com> On 8/27/15 6:16 PM, Kim Barrett wrote: > On Aug 27, 2015, at 5:42 PM, Daniel D. Daugherty wrote: >> Sorry for starting another e-mail thread fork in an already complicated >> review... > OK, that was fascinating. No, really, I mean it. :-) > It made me realize that we've been arguing and talking past each other > in part because we're really dealing with two distinct though closely > related bugs here. > > I've been primarily thinking about the case where we're calling > vm_abort / os::abort, where the we presently delete the PerfData > memory even though there can be arbitrary other threads running. This > was the case in JDK-8129978, which is how I got involved here in the > first place. In that bug we were in vm_exit_during_initialization and > had called perfMemory_exit when some thread attempted to inflate a > monitor (which is not one of the conflicting cases discussed by Dan). Right. All abnormal exit cases would be considered conflicting in what I wrote yesterday. > The problem Dan has been looking at, JDK-8049304, is about a "normal" > VM shutdown. In this case, the problem is that we believe it is safe > to delete the PerfData, because we've safepointed, and yet some thread > unexpectedly runs and attempts to touch the deleted data anyway. Right, I was concentrating on "normal". > I think Dan's proposed fix (mostly) avoids the specific instance of > JDK-8129978, but doesn't solve the more general problem of abnormal > exit deleting the PerfData while some running thread is touching some > non-monitor-related part of that data. My proposal to leave it to the > OS to deal with memory cleanup on process exit would deal with this > case. Now I'm starting to see that you are/were focused on a different problem. OK. Definitely should look at what we can do here. If we can avoid crashing a second time while dealing with another crash/ abnormal exit that would be good. > I think Dan's proposed fix (mostly) avoids problems like JDK-8049304. > And the approach I've been talking about doesn't help at all for this > case. But I wonder if Dan's proposed fix can be improved. A "futile > wakeup" case doesn't seem to me like one which requires super-high > performance. Would it be ok, in the two problematic cases that Dan > identified, to use some kind of atomic / locking protocol with the > cleanup? Or is the comment for the counter increment in EnterI (and > only there) correct that it's important to avoid a lock or atomics > here (and presumably in ReenterI too). This comment: 570 // Keep a tally of the # of futile wakeups. 571 // Note that the counter is not protected by a lock or updated by atomics. 572 // That is by design - we trade "lossy" counters which are exposed to 573 // races during updates for a lower probe effect. and this comment: 732 // Keep a tally of the # of futile wakeups. 733 // Note that the counter is not protected by a lock or updated by atomics. 734 // That is by design - we trade "lossy" counters which are exposed to 735 // races during updates for a lower probe effect. are not really specific to the monitor subsystem. I think the comments are generally true about the perf counters. As we discussed earlier in the thread, generally updating the perf counters with syncs or locks will cost and potentially perturb the things we are trying to count. So I think what you're proposing is putting a lock protocol around the setting of the flag and then have the non-safepoint-safe uses grab that lock while the safepoint-safe uses skip the lock because they can rely on the safepoint protocol in the "normal" exit case. Do I have this right? Dan From daniel.daugherty at oracle.com Fri Aug 28 16:27:39 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2015 10:27:39 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E073F4.5050906@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> Message-ID: <55E08BFB.70507@oracle.com> On 8/28/15 8:45 AM, Tom Benson wrote: > Hi, > One more pair of eyes on this. 8^) Hi Tom! Thanks for reviewing and welcome to the party... > > On 8/27/2015 8:16 PM, Kim Barrett wrote: >> On Aug 27, 2015, at 5:42 PM, Daniel D. Daugherty >> wrote: >>> Sorry for starting another e-mail thread fork in an already complicated >>> review... >> OK, that was fascinating. No, really, I mean it. >> >> It made me realize that we've been arguing and talking past each other >> in part because we're really dealing with two distinct though closely >> related bugs here. >> >> I've been primarily thinking about the case where we're calling >> vm_abort / os::abort, where the we presently delete the PerfData >> memory even though there can be arbitrary other threads running. This >> was the case in JDK-8129978, which is how I got involved here in the >> first place. In that bug we were in vm_exit_during_initialization and >> had called perfMemory_exit when some thread attempted to inflate a >> monitor (which is not one of the conflicting cases discussed by Dan). >> >> The problem Dan has been looking at, JDK-8049304, is about a "normal" >> VM shutdown. In this case, the problem is that we believe it is safe >> to delete the PerfData, because we've safepointed, and yet some thread >> unexpectedly runs and attempts to touch the deleted data anyway. >> >> I think Dan's proposed fix (mostly) avoids the specific instance of >> JDK-8129978, but doesn't solve the more general problem of abnormal >> exit deleting the PerfData while some running thread is touching some >> non-monitor-related part of that data. My proposal to leave it to the >> OS to deal with memory cleanup on process exit would deal with this >> case. >> >> I think Dan's proposed fix (mostly) avoids problems like JDK-8049304. >> And the approach I've been talking about doesn't help at all for this >> case. But I wonder if Dan's proposed fix can be improved. A "futile >> wakeup" case doesn't seem to me like one which requires super-high >> performance. Would it be ok, in the two problematic cases that Dan >> identified, to use some kind of atomic / locking protocol with the >> cleanup? Or is the comment for the counter increment in EnterI (and >> only there) correct that it's important to avoid a lock or atomics >> here (and presumably in ReenterI too). >> > > I notice that EnteriI/ReenterI both end with OrderAccess::fence(). Can > the potential update of _sync_FutileWakeups be delayed until that > point, to take advantage of the fence to make the sync hole even smaller? Not easily with EnterI() since there is one optional optimization between the OM_PERFDATA_OP(FutileWakeups, inc()) call and the OrderAccess::fence() call and that would result in lost FutileWakeups increments. Not easily in ReenterI(), the OM_PERFDATA_OP(FutileWakeups, inc()) call is at the bottom of the for-loop and the OrderAccess::fence() call at the end of the function is outside the loop. This would result in lost FutileWakeups increments. So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call immediately follows an OrderAccess::fence() call. Doesn't that make that increment as "safe" as it can be without having a real lock? > You've got a release() (and and short nap!) with the store in > PerfDataManager::destroy() to try to close the window somewhat. Yes, I modeled that after: src/share/vm/runtime/perfMemory.cpp: 83 void PerfMemory::initialize() { 156 OrderAccess::release_store(&_initialized, 1); 157 } > But I think rather than the release_store() you used, you want a > store, followed by a release(). release_store() puts a fence before > the store to ensure earlier updates are seen before the current one, no? Yup, and I see I got my reasoning wrong. The code I modeled is right because you want to flush all the inits and it's OK if the _initialized transition from '0' -> '1' is lazily seen. For my shutdown use, we are transitioning from '1' -> '0' and we need that to be seen proactively so: OrderAccess::release_store(&_has_PerfData, 0); OrderAccess::storeload(); which is modeled after _owner field transitions from non-zeo -> NULL in ObjectMonitor.cpp > > Also, I think the comment above that release_store() could be > clarified. It is fine as is if you're familiar with this bug report > and discussion, but... I think it should explicitly say there is > still a very small window for the lack of true synchronization to > cause a failure. And perhaps that the release_store() (or > store/release()) is not half of an acquire/release pair. Here's the existing comment: 286 // Clear the flag before we free the PerfData counters. Thus begins 287 // the race between this thread and another thread that has just 288 // queried PerfDataManager::has_PerfData() and gotten back 'true'. 289 // The hope is that the other thread will finish its PerfData 290 // manipulation before we free the memory. The two alternatives 291 // are 1) leak the PerfData memory or 2) do some form of ordered 292 // access before every PerfData operation. I think it pretty clearly states that there is still a race here. And I think that option 2 covers that we're not doing completely safe ordered access. I'm not sure how to make this comment more clear, but if you have specific suggestions... Dan > > Tom From tom.benson at oracle.com Fri Aug 28 17:57:39 2015 From: tom.benson at oracle.com (Tom Benson) Date: Fri, 28 Aug 2015 13:57:39 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E08BFB.70507@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> <55E08BFB.70507@oracle.com> Message-ID: <55E0A113.2090106@oracle.com> Hi Dan, On 8/28/2015 12:27 PM, Daniel D. Daugherty wrote: > On 8/28/15 8:45 AM, Tom Benson wrote: >> Hi, >> One more pair of eyes on this. 8^) > > Hi Tom! > > Thanks for reviewing and welcome to the party... > > >> >> On 8/27/2015 8:16 PM, Kim Barrett wrote: >>> On Aug 27, 2015, at 5:42 PM, Daniel D. Daugherty >>> wrote: >>>> Sorry for starting another e-mail thread fork in an already >>>> complicated >>>> review... >>> OK, that was fascinating. No, really, I mean it. >>> >>> It made me realize that we've been arguing and talking past each other >>> in part because we're really dealing with two distinct though closely >>> related bugs here. >>> >>> I've been primarily thinking about the case where we're calling >>> vm_abort / os::abort, where the we presently delete the PerfData >>> memory even though there can be arbitrary other threads running. This >>> was the case in JDK-8129978, which is how I got involved here in the >>> first place. In that bug we were in vm_exit_during_initialization and >>> had called perfMemory_exit when some thread attempted to inflate a >>> monitor (which is not one of the conflicting cases discussed by Dan). >>> >>> The problem Dan has been looking at, JDK-8049304, is about a "normal" >>> VM shutdown. In this case, the problem is that we believe it is safe >>> to delete the PerfData, because we've safepointed, and yet some thread >>> unexpectedly runs and attempts to touch the deleted data anyway. >>> >>> I think Dan's proposed fix (mostly) avoids the specific instance of >>> JDK-8129978, but doesn't solve the more general problem of abnormal >>> exit deleting the PerfData while some running thread is touching some >>> non-monitor-related part of that data. My proposal to leave it to the >>> OS to deal with memory cleanup on process exit would deal with this >>> case. >>> >>> I think Dan's proposed fix (mostly) avoids problems like JDK-8049304. >>> And the approach I've been talking about doesn't help at all for this >>> case. But I wonder if Dan's proposed fix can be improved. A "futile >>> wakeup" case doesn't seem to me like one which requires super-high >>> performance. Would it be ok, in the two problematic cases that Dan >>> identified, to use some kind of atomic / locking protocol with the >>> cleanup? Or is the comment for the counter increment in EnterI (and >>> only there) correct that it's important to avoid a lock or atomics >>> here (and presumably in ReenterI too). >>> >> >> I notice that EnteriI/ReenterI both end with OrderAccess::fence(). >> Can the potential update of _sync_FutileWakeups be delayed until that >> point, to take advantage of the fence to make the sync hole even >> smaller? > > Not easily with EnterI() since there is one optional optimization > between the OM_PERFDATA_OP(FutileWakeups, inc()) call and the > OrderAccess::fence() call and that would result in lost FutileWakeups > increments. > > Not easily in ReenterI(), the OM_PERFDATA_OP(FutileWakeups, inc()) call > is at the bottom of the for-loop and the OrderAccess::fence() call at > the end of the function is outside the loop. This would result in lost > FutileWakeups increments. Yes, you'd have to keep a local count, and then add the total outside the loop for both Enter/Reenter, after the fence. But I see what you mean about the other exit paths in Enter. (The more I look at this code, the more I remember it... BTW, are those knob_ setting defaults ever going to be moved to a platform specific-module? That was my beef (well, one of them) in 2 different ports. Or is the goal to make monitors so well self-tuning that they can go away? Sorry for the digression... 8^)) At any rate, as you say, perhaps it's not worth it to leverage the fences, though it could be done. > > So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call > immediately > follows an OrderAccess::fence() call. Doesn't that make that increment as > "safe" as it can be without having a real lock? > > >> You've got a release() (and and short nap!) with the store in >> PerfDataManager::destroy() to try to close the window somewhat. > > Yes, I modeled that after: > > src/share/vm/runtime/perfMemory.cpp: > > 83 void PerfMemory::initialize() { > > 156 OrderAccess::release_store(&_initialized, 1); > 157 } > > >> But I think rather than the release_store() you used, you want a >> store, followed by a release(). release_store() puts a fence before >> the store to ensure earlier updates are seen before the current one, no? > > Yup, and I see I got my reasoning wrong. The code I modeled > is right because you want to flush all the inits and it's OK > if the _initialized transition from '0' -> '1' is lazily seen. > > For my shutdown use, we are transitioning from '1' -> '0' and > we need that to be seen proactively so: > > OrderAccess::release_store(&_has_PerfData, 0); > OrderAccess::storeload(); > > which is modeled after _owner field transitions from non-zeo > -> NULL in ObjectMonitor.cpp > It's not clear to me why the store needs to be a release_store in this case, as long as the storeload() follows it. You're not protecting any earlier writes. ? > >> >> Also, I think the comment above that release_store() could be >> clarified. It is fine as is if you're familiar with this bug report >> and discussion, but... I think it should explicitly say there is >> still a very small window for the lack of true synchronization to >> cause a failure. And perhaps that the release_store() (or >> store/release()) is not half of an acquire/release pair. > > Here's the existing comment: > > 286 // Clear the flag before we free the PerfData counters. Thus > begins > 287 // the race between this thread and another thread that has > just > 288 // queried PerfDataManager::has_PerfData() and gotten back > 'true'. > 289 // The hope is that the other thread will finish its PerfData > 290 // manipulation before we free the memory. The two alternatives > 291 // are 1) leak the PerfData memory or 2) do some form of > ordered > 292 // access before every PerfData operation. > > I think it pretty clearly states that there is still a race here. > And I think that option 2 covers that we're not doing completely > safe ordered access. I'm not sure how to make this comment more > clear, but if you have specific suggestions... > OK, I guess it's subjective. Tom > Dan > > >> >> Tom > From daniel.daugherty at oracle.com Fri Aug 28 18:12:03 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2015 12:12:03 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E0A113.2090106@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> <55E08BFB.70507@oracle.com> <55E0A113.2090106@oracle.com> Message-ID: <55E0A473.3010900@oracle.com> On 8/28/15 11:57 AM, Tom Benson wrote: > Hi Dan, > > On 8/28/2015 12:27 PM, Daniel D. Daugherty wrote: >> On 8/28/15 8:45 AM, Tom Benson wrote: >>> Hi, >>> One more pair of eyes on this. 8^) >> >> Hi Tom! >> >> Thanks for reviewing and welcome to the party... >> >> >>> >>> On 8/27/2015 8:16 PM, Kim Barrett wrote: >>>> On Aug 27, 2015, at 5:42 PM, Daniel D. Daugherty >>>> wrote: >>>>> Sorry for starting another e-mail thread fork in an already >>>>> complicated >>>>> review... >>>> OK, that was fascinating. No, really, I mean it. >>>> >>>> It made me realize that we've been arguing and talking past each other >>>> in part because we're really dealing with two distinct though closely >>>> related bugs here. >>>> >>>> I've been primarily thinking about the case where we're calling >>>> vm_abort / os::abort, where the we presently delete the PerfData >>>> memory even though there can be arbitrary other threads running. This >>>> was the case in JDK-8129978, which is how I got involved here in the >>>> first place. In that bug we were in vm_exit_during_initialization and >>>> had called perfMemory_exit when some thread attempted to inflate a >>>> monitor (which is not one of the conflicting cases discussed by Dan). >>>> >>>> The problem Dan has been looking at, JDK-8049304, is about a "normal" >>>> VM shutdown. In this case, the problem is that we believe it is safe >>>> to delete the PerfData, because we've safepointed, and yet some thread >>>> unexpectedly runs and attempts to touch the deleted data anyway. >>>> >>>> I think Dan's proposed fix (mostly) avoids the specific instance of >>>> JDK-8129978, but doesn't solve the more general problem of abnormal >>>> exit deleting the PerfData while some running thread is touching some >>>> non-monitor-related part of that data. My proposal to leave it to the >>>> OS to deal with memory cleanup on process exit would deal with this >>>> case. >>>> >>>> I think Dan's proposed fix (mostly) avoids problems like JDK-8049304. >>>> And the approach I've been talking about doesn't help at all for this >>>> case. But I wonder if Dan's proposed fix can be improved. A "futile >>>> wakeup" case doesn't seem to me like one which requires super-high >>>> performance. Would it be ok, in the two problematic cases that Dan >>>> identified, to use some kind of atomic / locking protocol with the >>>> cleanup? Or is the comment for the counter increment in EnterI (and >>>> only there) correct that it's important to avoid a lock or atomics >>>> here (and presumably in ReenterI too). >>>> >>> >>> I notice that EnteriI/ReenterI both end with OrderAccess::fence(). >>> Can the potential update of _sync_FutileWakeups be delayed until >>> that point, to take advantage of the fence to make the sync hole >>> even smaller? >> >> Not easily with EnterI() since there is one optional optimization >> between the OM_PERFDATA_OP(FutileWakeups, inc()) call and the >> OrderAccess::fence() call and that would result in lost FutileWakeups >> increments. >> >> Not easily in ReenterI(), the OM_PERFDATA_OP(FutileWakeups, inc()) call >> is at the bottom of the for-loop and the OrderAccess::fence() call at >> the end of the function is outside the loop. This would result in lost >> FutileWakeups increments. > Yes, you'd have to keep a local count, and then add the total outside > the loop for both Enter/Reenter, after the fence. But I see what you > mean about the other exit paths in Enter. (The more I look at this > code, the more I remember it... I'm sure that is a wonderfully loving memory too! > BTW, are those knob_ setting defaults ever going to be moved to a > platform specific-module? That was my beef (well, one of them) in 2 > different ports. Or is the goal to make monitors so well self-tuning > that they can go away? Sorry for the digression... 8^)) Dice is working on another idea to move tuning to a separate loadable module which is why we deferred the "adaptive spin" and "SpinPause on SPARC" buckets for the Contended Locking project. > At any rate, as you say, perhaps it's not worth it to leverage the > fences, though it could be done. OK so we're agreed on no change here. > >> >> So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call >> immediately >> follows an OrderAccess::fence() call. Doesn't that make that >> increment as >> "safe" as it can be without having a real lock? >> >> >>> You've got a release() (and and short nap!) with the store in >>> PerfDataManager::destroy() to try to close the window somewhat. >> >> Yes, I modeled that after: >> >> src/share/vm/runtime/perfMemory.cpp: >> >> 83 void PerfMemory::initialize() { >> >> 156 OrderAccess::release_store(&_initialized, 1); >> 157 } >> >> >>> But I think rather than the release_store() you used, you want a >>> store, followed by a release(). release_store() puts a fence >>> before the store to ensure earlier updates are seen before the >>> current one, no? >> >> Yup, and I see I got my reasoning wrong. The code I modeled >> is right because you want to flush all the inits and it's OK >> if the _initialized transition from '0' -> '1' is lazily seen. >> >> For my shutdown use, we are transitioning from '1' -> '0' and >> we need that to be seen proactively so: >> >> OrderAccess::release_store(&_has_PerfData, 0); >> OrderAccess::storeload(); >> >> which is modeled after _owner field transitions from non-zero >> -> NULL in ObjectMonitor.cpp >> > > It's not clear to me why the store needs to be a release_store in this > case, as long as the storeload() follows it. You're not protecting > any earlier writes. ? I'm following the model that Dice uses for ObjectMonitors when we change the _owner field from non-NULL -> NULL. There are some long comments in the ObjectMonitor.cpp stuff that talk about why it is done this way. I'm planning to file a cleanup bug for the non-NULL -> NULL transition stuff because the comments and code are not consistent. > >> >>> >>> Also, I think the comment above that release_store() could be >>> clarified. It is fine as is if you're familiar with this bug report >>> and discussion, but... I think it should explicitly say there is >>> still a very small window for the lack of true synchronization to >>> cause a failure. And perhaps that the release_store() (or >>> store/release()) is not half of an acquire/release pair. >> >> Here's the existing comment: >> >> 286 // Clear the flag before we free the PerfData counters. >> Thus begins >> 287 // the race between this thread and another thread that has >> just >> 288 // queried PerfDataManager::has_PerfData() and gotten back >> 'true'. >> 289 // The hope is that the other thread will finish its PerfData >> 290 // manipulation before we free the memory. The two >> alternatives >> 291 // are 1) leak the PerfData memory or 2) do some form of >> ordered >> 292 // access before every PerfData operation. >> >> I think it pretty clearly states that there is still a race here. >> And I think that option 2 covers that we're not doing completely >> safe ordered access. I'm not sure how to make this comment more >> clear, but if you have specific suggestions... >> > OK, I guess it's subjective. I'll always take wording tweaks so if something occurs to you later on... :-) Dan > Tom > >> Dan >> >> >>> >>> Tom >> > From tom.benson at oracle.com Fri Aug 28 18:24:44 2015 From: tom.benson at oracle.com (Tom Benson) Date: Fri, 28 Aug 2015 14:24:44 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E0A473.3010900@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> <55E08BFB.70507@oracle.com> <55E0A113.2090106@oracle.com> <55E0A473.3010900@oracle.com> Message-ID: <55E0A76C.4000101@oracle.com> Hi Dan, On 8/28/2015 2:12 PM, Daniel D. Daugherty wrote: > . . . >>> >>> Not easily in ReenterI(), the OM_PERFDATA_OP(FutileWakeups, inc()) call >>> is at the bottom of the for-loop and the OrderAccess::fence() call at >>> the end of the function is outside the loop. This would result in lost >>> FutileWakeups increments. >> Yes, you'd have to keep a local count, and then add the total outside >> the loop for both Enter/Reenter, after the fence. But I see what you >> mean about the other exit paths in Enter. (The more I look at this >> code, the more I remember it... > > I'm sure that is a wonderfully loving memory too! Absolutely! 8^) > > >> BTW, are those knob_ setting defaults ever going to be moved to a >> platform specific-module? That was my beef (well, one of them) in 2 >> different ports. Or is the goal to make monitors so well self-tuning >> that they can go away? Sorry for the digression... 8^)) > > Dice is working on another idea to move tuning to a separate loadable > module which is why we deferred the "adaptive spin" and "SpinPause on > SPARC" buckets for the Contended Locking project. > > >> At any rate, as you say, perhaps it's not worth it to leverage the >> fences, though it could be done. > > OK so we're agreed on no change here. > > >> >>> >>> So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call >>> immediately >>> follows an OrderAccess::fence() call. Doesn't that make that >>> increment as >>> "safe" as it can be without having a real lock? >>> >>> >>>> You've got a release() (and and short nap!) with the store in >>>> PerfDataManager::destroy() to try to close the window somewhat. >>> >>> Yes, I modeled that after: >>> >>> src/share/vm/runtime/perfMemory.cpp: >>> >>> 83 void PerfMemory::initialize() { >>> >>> 156 OrderAccess::release_store(&_initialized, 1); >>> 157 } >>> >>> >>>> But I think rather than the release_store() you used, you want a >>>> store, followed by a release(). release_store() puts a fence >>>> before the store to ensure earlier updates are seen before the >>>> current one, no? >>> >>> Yup, and I see I got my reasoning wrong. The code I modeled >>> is right because you want to flush all the inits and it's OK >>> if the _initialized transition from '0' -> '1' is lazily seen. >>> >>> For my shutdown use, we are transitioning from '1' -> '0' and >>> we need that to be seen proactively so: >>> >>> OrderAccess::release_store(&_has_PerfData, 0); >>> OrderAccess::storeload(); >>> >>> which is modeled after _owner field transitions from non-zero >>> -> NULL in ObjectMonitor.cpp >>> >> >> It's not clear to me why the store needs to be a release_store in >> this case, as long as the storeload() follows it. You're not >> protecting any earlier writes. ? > > I'm following the model that Dice uses for ObjectMonitors when we > change the _owner field from non-NULL -> NULL. There are some long > comments in the ObjectMonitor.cpp stuff that talk about why it is > done this way. I'm planning to file a cleanup bug for the non-NULL > -> NULL transition stuff because the comments and code are not > consistent. > But that's in lock exit, so the release is needed to ensure all outstanding writes are seen before the owner is set to null. There's nothing analogous in this case. However, since this will be executed once per JVM shutdown, having an extra release() isn't a big deal... Tom From kim.barrett at oracle.com Fri Aug 28 18:27:37 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 28 Aug 2015 14:27:37 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E083C1.8090709@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E083C1.8090709@oracle.com> Message-ID: <6EDED012-FD41-4879-A9E5-3DFA6245FCAD@oracle.com> On Aug 28, 2015, at 11:52 AM, Daniel D. Daugherty wrote: > > This comment: > > 570 // Keep a tally of the # of futile wakeups. > 571 // Note that the counter is not protected by a lock or updated by atomics. > 572 // That is by design - we trade "lossy" counters which are exposed to > 573 // races during updates for a lower probe effect. > > and this comment: > > 732 // Keep a tally of the # of futile wakeups. > 733 // Note that the counter is not protected by a lock or updated by atomics. > 734 // That is by design - we trade "lossy" counters which are exposed to > 735 // races during updates for a lower probe effect. > > are not really specific to the monitor subsystem. I think > the comments are generally true about the perf counters. Yes, but oddly placed here. > As we discussed earlier in the thread, generally updating the perf > counters with syncs or locks will cost and potentially perturb the > things we are trying to count. Yes. > So I think what you're proposing is putting a lock protocol around > the setting of the flag and then have the non-safepoint-safe uses > grab that lock while the safepoint-safe uses skip the lock because > they can rely on the safepoint protocol in the "normal" exit case. > > Do I have this right? Yes. My question is, does the extra overhead matter in these specific cases. And the locking mechanism might be some clever use of atomics rather than any sort of ?standard" mutex. And the safepoint-safe uses not only skip the lock, but don?t need to check the flag at all. From tom.benson at oracle.com Fri Aug 28 18:29:58 2015 From: tom.benson at oracle.com (Tom Benson) Date: Fri, 28 Aug 2015 14:29:58 -0400 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E08BFB.70507@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> <55E08BFB.70507@oracle.com> Message-ID: <55E0A8A6.5040101@oracle.com> Hi again, Just noticed I skipped this question in your reply: > > So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call > immediately > follows an OrderAccess::fence() call. Doesn't that make that increment as > "safe" as it can be without having a real lock? > > Yes - odd that I noticed the fence after the update in that code, and not the one right before it! Thanks, Tom From daniel.daugherty at oracle.com Fri Aug 28 19:46:52 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2015 13:46:52 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <6EDED012-FD41-4879-A9E5-3DFA6245FCAD@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E083C1.8090709@oracle.com> <6EDED012-FD41-4879-A9E5-3DFA6245FCAD@oracle.com> Message-ID: <55E0BAAC.8040300@oracle.com> On 8/28/15 12:27 PM, Kim Barrett wrote: > On Aug 28, 2015, at 11:52 AM, Daniel D. Daugherty wrote: >> This comment: >> >> 570 // Keep a tally of the # of futile wakeups. >> 571 // Note that the counter is not protected by a lock or updated by atomics. >> 572 // That is by design - we trade "lossy" counters which are exposed to >> 573 // races during updates for a lower probe effect. >> >> and this comment: >> >> 732 // Keep a tally of the # of futile wakeups. >> 733 // Note that the counter is not protected by a lock or updated by atomics. >> 734 // That is by design - we trade "lossy" counters which are exposed to >> 735 // races during updates for a lower probe effect. >> >> are not really specific to the monitor subsystem. I think >> the comments are generally true about the perf counters. > Yes, but oddly placed here. > >> As we discussed earlier in the thread, generally updating the perf >> counters with syncs or locks will cost and potentially perturb the >> things we are trying to count. > Yes. > >> So I think what you're proposing is putting a lock protocol around >> the setting of the flag and then have the non-safepoint-safe uses >> grab that lock while the safepoint-safe uses skip the lock because >> they can rely on the safepoint protocol in the "normal" exit case. >> >> Do I have this right? > Yes. My question is, does the extra overhead matter in these specific cases. > And the locking mechanism might be some clever use of atomics rather than > any sort of ?standard" mutex. I figure the lightest I can get away with is an acquire. There's an existing lock for the perf stuff, but I don't want to use a full blow mutex... > And the safepoint-safe uses not only skip the lock, Agreed. > but don?t need to check the > flag at all. I don't agree with this part. Until the VMThread exits and raises the permanent safepoint barrier for remaining daemon threads, I don't think I can guarantee that we won't go to a safepoint, clear the flag & free the memory, and then return from that safepoint... which would allow a daemon thread to access the now freed memory... Dan From daniel.daugherty at oracle.com Fri Aug 28 19:48:08 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2015 13:48:08 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E0A76C.4000101@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> <55E08BFB.70507@oracle.com> <55E0A113.2090106@oracle.com> <55E0A473.3010900@oracle.com> <55E0A76C.4000101@oracle.com> Message-ID: <55E0BAF8.9000303@oracle.com> On 8/28/15 12:24 PM, Tom Benson wrote: > Hi Dan, > > > On 8/28/2015 2:12 PM, Daniel D. Daugherty wrote: >> . . . >>>> >>>> Not easily in ReenterI(), the OM_PERFDATA_OP(FutileWakeups, inc()) >>>> call >>>> is at the bottom of the for-loop and the OrderAccess::fence() call at >>>> the end of the function is outside the loop. This would result in lost >>>> FutileWakeups increments. >>> Yes, you'd have to keep a local count, and then add the total >>> outside the loop for both Enter/Reenter, after the fence. But I see >>> what you mean about the other exit paths in Enter. (The more I look >>> at this code, the more I remember it... >> >> I'm sure that is a wonderfully loving memory too! > > Absolutely! 8^) > >> >> >>> BTW, are those knob_ setting defaults ever going to be moved to a >>> platform specific-module? That was my beef (well, one of them) in >>> 2 different ports. Or is the goal to make monitors so well >>> self-tuning that they can go away? Sorry for the digression... 8^)) >> >> Dice is working on another idea to move tuning to a separate loadable >> module which is why we deferred the "adaptive spin" and "SpinPause on >> SPARC" buckets for the Contended Locking project. >> >> >>> At any rate, as you say, perhaps it's not worth it to leverage the >>> fences, though it could be done. >> >> OK so we're agreed on no change here. >> >> >>> >>>> >>>> So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call >>>> immediately >>>> follows an OrderAccess::fence() call. Doesn't that make that >>>> increment as >>>> "safe" as it can be without having a real lock? >>>> >>>> >>>>> You've got a release() (and and short nap!) with the store in >>>>> PerfDataManager::destroy() to try to close the window somewhat. >>>> >>>> Yes, I modeled that after: >>>> >>>> src/share/vm/runtime/perfMemory.cpp: >>>> >>>> 83 void PerfMemory::initialize() { >>>> >>>> 156 OrderAccess::release_store(&_initialized, 1); >>>> 157 } >>>> >>>> >>>>> But I think rather than the release_store() you used, you want a >>>>> store, followed by a release(). release_store() puts a fence >>>>> before the store to ensure earlier updates are seen before the >>>>> current one, no? >>>> >>>> Yup, and I see I got my reasoning wrong. The code I modeled >>>> is right because you want to flush all the inits and it's OK >>>> if the _initialized transition from '0' -> '1' is lazily seen. >>>> >>>> For my shutdown use, we are transitioning from '1' -> '0' and >>>> we need that to be seen proactively so: >>>> >>>> OrderAccess::release_store(&_has_PerfData, 0); >>>> OrderAccess::storeload(); >>>> >>>> which is modeled after _owner field transitions from non-zero >>>> -> NULL in ObjectMonitor.cpp >>>> >>> >>> It's not clear to me why the store needs to be a release_store in >>> this case, as long as the storeload() follows it. You're not >>> protecting any earlier writes. ? >> >> I'm following the model that Dice uses for ObjectMonitors when we >> change the _owner field from non-NULL -> NULL. There are some long >> comments in the ObjectMonitor.cpp stuff that talk about why it is >> done this way. I'm planning to file a cleanup bug for the non-NULL >> -> NULL transition stuff because the comments and code are not >> consistent. >> > > But that's in lock exit, so the release is needed to ensure all > outstanding writes are seen before the owner is set to null. There's > nothing analogous in this case. True... I'll mull on it since I'm tweaking the code again anyway... Dan > > However, since this will be executed once per JVM shutdown, having an > extra release() isn't a big deal... > Tom > > > From daniel.daugherty at oracle.com Fri Aug 28 19:49:00 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2015 13:49:00 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E0A8A6.5040101@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> <55E08BFB.70507@oracle.com> <55E0A8A6.5040101@oracle.com> Message-ID: <55E0BB2C.2000402@oracle.com> On 8/28/15 12:29 PM, Tom Benson wrote: > Hi again, > Just noticed I skipped this question in your reply: > >> >> So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call >> immediately >> follows an OrderAccess::fence() call. Doesn't that make that >> increment as >> "safe" as it can be without having a real lock? >> >> > Yes - odd that I noticed the fence after the update in that code, and > not the one right before it! That's because you didn't uncross your eyes before reading that part of the monitor code... :-) Dan > Thanks, > Tom > From david.holmes at oracle.com Mon Aug 31 01:30:10 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 31 Aug 2015 11:30:10 +1000 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E08BFB.70507@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> <55E08BFB.70507@oracle.com> Message-ID: <55E3AE22.6050309@oracle.com> Hi Dan, On 29/08/2015 2:27 AM, Daniel D. Daugherty wrote: > On 8/28/15 8:45 AM, Tom Benson wrote: >> Hi, >> One more pair of eyes on this. 8^) > > Hi Tom! > > Thanks for reviewing and welcome to the party... > > >> >> On 8/27/2015 8:16 PM, Kim Barrett wrote: >>> On Aug 27, 2015, at 5:42 PM, Daniel D. Daugherty >>> wrote: >>>> Sorry for starting another e-mail thread fork in an already complicated >>>> review... >>> OK, that was fascinating. No, really, I mean it. >>> >>> It made me realize that we've been arguing and talking past each other >>> in part because we're really dealing with two distinct though closely >>> related bugs here. >>> >>> I've been primarily thinking about the case where we're calling >>> vm_abort / os::abort, where the we presently delete the PerfData >>> memory even though there can be arbitrary other threads running. This >>> was the case in JDK-8129978, which is how I got involved here in the >>> first place. In that bug we were in vm_exit_during_initialization and >>> had called perfMemory_exit when some thread attempted to inflate a >>> monitor (which is not one of the conflicting cases discussed by Dan). >>> >>> The problem Dan has been looking at, JDK-8049304, is about a "normal" >>> VM shutdown. In this case, the problem is that we believe it is safe >>> to delete the PerfData, because we've safepointed, and yet some thread >>> unexpectedly runs and attempts to touch the deleted data anyway. >>> >>> I think Dan's proposed fix (mostly) avoids the specific instance of >>> JDK-8129978, but doesn't solve the more general problem of abnormal >>> exit deleting the PerfData while some running thread is touching some >>> non-monitor-related part of that data. My proposal to leave it to the >>> OS to deal with memory cleanup on process exit would deal with this >>> case. >>> >>> I think Dan's proposed fix (mostly) avoids problems like JDK-8049304. >>> And the approach I've been talking about doesn't help at all for this >>> case. But I wonder if Dan's proposed fix can be improved. A "futile >>> wakeup" case doesn't seem to me like one which requires super-high >>> performance. Would it be ok, in the two problematic cases that Dan >>> identified, to use some kind of atomic / locking protocol with the >>> cleanup? Or is the comment for the counter increment in EnterI (and >>> only there) correct that it's important to avoid a lock or atomics >>> here (and presumably in ReenterI too). >>> >> >> I notice that EnteriI/ReenterI both end with OrderAccess::fence(). Can >> the potential update of _sync_FutileWakeups be delayed until that >> point, to take advantage of the fence to make the sync hole even smaller? > > Not easily with EnterI() since there is one optional optimization > between the OM_PERFDATA_OP(FutileWakeups, inc()) call and the > OrderAccess::fence() call and that would result in lost FutileWakeups > increments. > > Not easily in ReenterI(), the OM_PERFDATA_OP(FutileWakeups, inc()) call > is at the bottom of the for-loop and the OrderAccess::fence() call at > the end of the function is outside the loop. This would result in lost > FutileWakeups increments. > > So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call immediately > follows an OrderAccess::fence() call. Doesn't that make that increment as > "safe" as it can be without having a real lock? > > >> You've got a release() (and and short nap!) with the store in >> PerfDataManager::destroy() to try to close the window somewhat. > > Yes, I modeled that after: > > src/share/vm/runtime/perfMemory.cpp: > > 83 void PerfMemory::initialize() { > > 156 OrderAccess::release_store(&_initialized, 1); > 157 } > > >> But I think rather than the release_store() you used, you want a >> store, followed by a release(). release_store() puts a fence before >> the store to ensure earlier updates are seen before the current one, no? > > Yup, and I see I got my reasoning wrong. The code I modeled > is right because you want to flush all the inits and it's OK > if the _initialized transition from '0' -> '1' is lazily seen. > > For my shutdown use, we are transitioning from '1' -> '0' and > we need that to be seen proactively so: Nit: OrderAccess "barriers" enforce ordering constraints but don't in general provide any guarantees about visibility - ie they are not necessarily "flushes". So while it may be true on some platforms by virtue of the underlying barrier mechanism, in general they don't change when a write becomes visible and so there is nothing "proactive" about them. > OrderAccess::release_store(&_has_PerfData, 0); > OrderAccess::storeload(); I agree with Tom that the release is unnecessary - though harmless. The real ordering constraint here is that we preserve: _has_PerfData = 0; for which a storeload|storestore barrier after the write seems most appropriate. Though with the insertion of the sleep after the write there won't be any reordering anyway so explicit barriers seem redundant. Cheers, David ----- > which is modeled after _owner field transitions from non-zeo > -> NULL in ObjectMonitor.cpp > > >> >> Also, I think the comment above that release_store() could be >> clarified. It is fine as is if you're familiar with this bug report >> and discussion, but... I think it should explicitly say there is >> still a very small window for the lack of true synchronization to >> cause a failure. And perhaps that the release_store() (or >> store/release()) is not half of an acquire/release pair. > > Here's the existing comment: > > 286 // Clear the flag before we free the PerfData counters. Thus > begins > 287 // the race between this thread and another thread that has just > 288 // queried PerfDataManager::has_PerfData() and gotten back > 'true'. > 289 // The hope is that the other thread will finish its PerfData > 290 // manipulation before we free the memory. The two alternatives > 291 // are 1) leak the PerfData memory or 2) do some form of ordered > 292 // access before every PerfData operation. > > I think it pretty clearly states that there is still a race here. > And I think that option 2 covers that we're not doing completely > safe ordered access. I'm not sure how to make this comment more > clear, but if you have specific suggestions... > > Dan > > >> >> Tom > From daniel.daugherty at oracle.com Mon Aug 31 16:58:04 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 31 Aug 2015 10:58:04 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55E3AE22.6050309@oracle.com> References: <55DCD94A.30705@oracle.com> <55DE915B.9020605@oracle.com> <55DF842E.1010407@oracle.com> <6C52FDE5-A4C0-4FC9-9114-83376C97FD88@oracle.com> <55E073F4.5050906@oracle.com> <55E08BFB.70507@oracle.com> <55E3AE22.6050309@oracle.com> Message-ID: <55E4879C.7090203@oracle.com> Hi David, Replies embedded below... On 8/30/15 7:30 PM, David Holmes wrote: > Hi Dan, > > On 29/08/2015 2:27 AM, Daniel D. Daugherty wrote: >> On 8/28/15 8:45 AM, Tom Benson wrote: >>> Hi, >>> One more pair of eyes on this. 8^) >> >> Hi Tom! >> >> Thanks for reviewing and welcome to the party... >> >> >>> >>> On 8/27/2015 8:16 PM, Kim Barrett wrote: >>>> On Aug 27, 2015, at 5:42 PM, Daniel D. Daugherty >>>> wrote: >>>>> Sorry for starting another e-mail thread fork in an already >>>>> complicated >>>>> review... >>>> OK, that was fascinating. No, really, I mean it. >>>> >>>> It made me realize that we've been arguing and talking past each other >>>> in part because we're really dealing with two distinct though closely >>>> related bugs here. >>>> >>>> I've been primarily thinking about the case where we're calling >>>> vm_abort / os::abort, where the we presently delete the PerfData >>>> memory even though there can be arbitrary other threads running. This >>>> was the case in JDK-8129978, which is how I got involved here in the >>>> first place. In that bug we were in vm_exit_during_initialization and >>>> had called perfMemory_exit when some thread attempted to inflate a >>>> monitor (which is not one of the conflicting cases discussed by Dan). >>>> >>>> The problem Dan has been looking at, JDK-8049304, is about a "normal" >>>> VM shutdown. In this case, the problem is that we believe it is safe >>>> to delete the PerfData, because we've safepointed, and yet some thread >>>> unexpectedly runs and attempts to touch the deleted data anyway. >>>> >>>> I think Dan's proposed fix (mostly) avoids the specific instance of >>>> JDK-8129978, but doesn't solve the more general problem of abnormal >>>> exit deleting the PerfData while some running thread is touching some >>>> non-monitor-related part of that data. My proposal to leave it to the >>>> OS to deal with memory cleanup on process exit would deal with this >>>> case. >>>> >>>> I think Dan's proposed fix (mostly) avoids problems like JDK-8049304. >>>> And the approach I've been talking about doesn't help at all for this >>>> case. But I wonder if Dan's proposed fix can be improved. A "futile >>>> wakeup" case doesn't seem to me like one which requires super-high >>>> performance. Would it be ok, in the two problematic cases that Dan >>>> identified, to use some kind of atomic / locking protocol with the >>>> cleanup? Or is the comment for the counter increment in EnterI (and >>>> only there) correct that it's important to avoid a lock or atomics >>>> here (and presumably in ReenterI too). >>>> >>> >>> I notice that EnteriI/ReenterI both end with OrderAccess::fence(). Can >>> the potential update of _sync_FutileWakeups be delayed until that >>> point, to take advantage of the fence to make the sync hole even >>> smaller? >> >> Not easily with EnterI() since there is one optional optimization >> between the OM_PERFDATA_OP(FutileWakeups, inc()) call and the >> OrderAccess::fence() call and that would result in lost FutileWakeups >> increments. >> >> Not easily in ReenterI(), the OM_PERFDATA_OP(FutileWakeups, inc()) call >> is at the bottom of the for-loop and the OrderAccess::fence() call at >> the end of the function is outside the loop. This would result in lost >> FutileWakeups increments. >> >> So in ReenterI() the OM_PERFDATA_OP(FutileWakeups, inc()) call >> immediately >> follows an OrderAccess::fence() call. Doesn't that make that >> increment as >> "safe" as it can be without having a real lock? >> >> >>> You've got a release() (and and short nap!) with the store in >>> PerfDataManager::destroy() to try to close the window somewhat. >> >> Yes, I modeled that after: >> >> src/share/vm/runtime/perfMemory.cpp: >> >> 83 void PerfMemory::initialize() { >> >> 156 OrderAccess::release_store(&_initialized, 1); >> 157 } >> >> >>> But I think rather than the release_store() you used, you want a >>> store, followed by a release(). release_store() puts a fence before >>> the store to ensure earlier updates are seen before the current one, >>> no? >> >> Yup, and I see I got my reasoning wrong. The code I modeled >> is right because you want to flush all the inits and it's OK >> if the _initialized transition from '0' -> '1' is lazily seen. >> >> For my shutdown use, we are transitioning from '1' -> '0' and >> we need that to be seen proactively so: > > Nit: OrderAccess "barriers" enforce ordering constraints but don't in > general provide any guarantees about visibility - ie they are not > necessarily "flushes". So while it may be true on some platforms by > virtue of the underlying barrier mechanism, in general they don't > change when a write becomes visible and so there is nothing > "proactive" about them. My apologies for being sloppy with my wording. What I'm trying to do is put up a barrier so that once the deleting thread has changed the flag from '1' -> '0' another thread trying to read the flag won't see the '1' when it should see the '0'. In order words, I don't want the other thread's read of the flag to float above the setting of the flag to '0'. > >> OrderAccess::release_store(&_has_PerfData, 0); >> OrderAccess::storeload(); > > I agree with Tom that the release is unnecessary - though harmless. I tried to switch the code to: OrderAccess::store(&_has_PerfData, 0) OrderAccess::storeload(); but that wouldn't compile on Solaris X64: Error: static OrderAccess::store(volatile int*, int) is not accessible from static PerfDataManager::destroy(). But then it occurred to me that I was trying too hard to use OrderAccess functions... > The real ordering constraint here is that we preserve: > > _has_PerfData = 0; > I think I can switch to a straight "_has_PerfData = 0;" here and not use either OrderAccess::release_store() or OrderAccess::store(). I also reread OrderAccess.hpp again and convinced myself that an "OrderAccess::fence()" call is the only way to be sure here. I'm pretty sure that achieves 'storeload|storestore' barrier semantics... Yes, I have a headache again... :-) > for which a storeload|storestore barrier after the write seems most > appropriate. Though with the insertion of the sleep after the write > there won't be any reordering anyway so explicit barriers seem redundant. I didn't think that os::naked_short_sleep() would do anything to keep another thread's read from floating above the "_has_PerfData = 0;". What did I miss? My current plan: - switch from "OrderAccess::release_store(&_has_PerfData, 0);" to "_has_PerfData = 0;" - keep "OrderAccess::fence();" - retest and send out for another round of review Dan > > Cheers, > David > ----- > >> which is modeled after _owner field transitions from non-zeo >> -> NULL in ObjectMonitor.cpp >> >> >>> >>> Also, I think the comment above that release_store() could be >>> clarified. It is fine as is if you're familiar with this bug report >>> and discussion, but... I think it should explicitly say there is >>> still a very small window for the lack of true synchronization to >>> cause a failure. And perhaps that the release_store() (or >>> store/release()) is not half of an acquire/release pair. >> >> Here's the existing comment: >> >> 286 // Clear the flag before we free the PerfData counters. Thus >> begins >> 287 // the race between this thread and another thread that >> has just >> 288 // queried PerfDataManager::has_PerfData() and gotten back >> 'true'. >> 289 // The hope is that the other thread will finish its PerfData >> 290 // manipulation before we free the memory. The two >> alternatives >> 291 // are 1) leak the PerfData memory or 2) do some form of >> ordered >> 292 // access before every PerfData operation. >> >> I think it pretty clearly states that there is still a race here. >> And I think that option 2 covers that we're not doing completely >> safe ordered access. I'm not sure how to make this comment more >> clear, but if you have specific suggestions... >> >> Dan >> >> >>> >>> Tom >> From daniel.daugherty at oracle.com Mon Aug 31 22:51:02 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 31 Aug 2015 16:51:02 -0600 Subject: RFR (S) 8049304: race between VM_Exit and _sync_FutileWakeups->inc() In-Reply-To: <55DCD94A.30705@oracle.com> References: <55DCD94A.30705@oracle.com> Message-ID: <55E4DA56.8030002@oracle.com> Greetings, I've updated the "fix" for this bug based on code review comments received in round 0. JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() https://bugs.openjdk.java.net/browse/JDK-8049304 Webrev URL: http://cr.openjdk.java.net/~dcubed/8049304-webrev/1-jdk9-hs-rt/ The easiest way to re-review is to download the two patch files and view them in your favorite file merge tool: http://cr.openjdk.java.net/~dcubed/8049304-webrev/0-jdk9-hs-rt/hotspot.patch http://cr.openjdk.java.net/~dcubed/8049304-webrev/1-jdk9-hs-rt/hotspot.patch Testing: Aurora Adhoc RT-SVC nightly batch (in process) Aurora Adhoc vm.tmtools batch (in process) Kim's repro sequence for JDK-8049304 Kim's repro sequence for JDK-8129978 JPRT -testset hotspot Changes between round 0 and round 1: - add an 'is_safe' parameter to the OM_PERFDATA_OP macro; safepoint-safe callers can access _has_PerfData flag directly; non-safepoint-safe callers use a load-acquire to fetch the current _has_PerfData flag value - change PerfDataManager::destroy() to simply set _has_PerfData to zero (field is volatile) and then use a fence() to prevent any reordering of operations in any direction; it's only done once during VM shutdown so... - change perfMemory_exit() to only call PerfDataManager::destroy() when called at a safepoint and when the StatSample is not running; this means when the VM is aborting, we no longer have a race between the original crash report and this code path. I believe that I've addressed all comments from round 0. Thanks, in advance, for any comments, questions or suggestions. Dan On 8/25/15 3:08 PM, Daniel D. Daugherty wrote: > Greetings, > > I have a "fix" for a long standing race between JVM shutdown and the > JVM statistics subsystem: > > JDK-8049304 race between VM_Exit and _sync_FutileWakeups->inc() > https://bugs.openjdk.java.net/browse/JDK-8049304 > > Webrev URL: > http://cr.openjdk.java.net/~dcubed/8049304-webrev/0-jdk9-hs-rt/ > > Testing: Aurora Adhoc RT-SVC nightly batch > Aurora Adhoc vm.tmtools batch > Kim's repro sequence for JDK-8049304 > Kim's repro sequence for JDK-8129978 > JPRT -testset hotspot > > This "fix": > > - adds a volatile flag to record whether PerfDataManager is holding > data (PerfData objects) > - adds PerfDataManager::has_PerfData() to return the flag > - changes the Java monitor subsystem's use of PerfData to > check both allocation of the monitor subsystem specific > PerfData object and the new PerfDataManager::has_PerfData() > return value > > If the global 'UsePerfData' option is false, the system works as > it did before. If 'UsePerfData' is true (the default on non-embedded > systems), the Java monitor subsystem will allocate a number of > PerfData objects to record information. The objects will record > information about Java monitor subsystem until the JVM shuts down. > > When the JVM starts to shutdown, the new PerfDataManager flag will > change to false and the Java monitor subsystem will stop using the > PerfData objects. This is the new behavior. As noted in the comments > I added to the code, the race is still present; I'm just changing > the order and the timing to reduce the likelihood of the crash. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > >