From erik.gahlin at oracle.com Wed May 1 19:14:46 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 1 May 2019 21:14:46 +0200 Subject: RFR: 8223066: "jfr metadata" output the @Name annotation twice. In-Reply-To: <398c166c-b681-6d32-4f4c-9d43350b622f@oracle.com> References: <398c166c-b681-6d32-4f4c-9d43350b622f@oracle.com> Message-ID: <5CC9F026.4050002@oracle.com> Hi Chihiro, Thanks for reporting the issue, @Name should not be printed twice. Let me give you some background information. I think there are two issues at hand: 1) The Name annotation should probably not be persisted. This can be achieved by removing the MetadataDefinition from the class. I did try that, but I noticed several tests failing, in particular TestName which explicitly checks that @Name is persisted. The purpose @Name is to override the default name, which can then be retrieved from ValueDescriptor::getName, EventType::getName and SettingDescriptor::getName. The annotation is not needed on the consumer side as the name is already persisted elsewhere. I remember discussing this several times during the design, but for some reason we came to the conclusion to persist it, probably to not confuse users if it didn't behave like @Label and @Description. Later, Metadatadefinition was removed from @Threshold, @Periodic, @Enabled and @StackTrace. At that time, it should probably have been removed from @Name as well. Maybe we should go ahead and remove it now, but I will have to think about the implications first. 2) The reason PrettyWriter::printType adds @Name is because there is no good way to include package/namespace in the printout. Events that lack a namespace should not get the annotation, that's why there is a check for "." It's not because the values are primitives. I can see two ways to proceed. Either add a check in printType so @Name is only printed if it doesn't already exists, i.e. if (t.getAnnotation(Name.class) == null) { if (index != -1) { println("@Name(\"" + typeName + "\")"); } } or remove @MetadataDefinition from the Name class and make adjustments to tests etc. If you chose the latter, I like a week to think it over, investigate the implications and discuss it with other people, before I can say if it is the right approach or not. Thanks Erik Hi, > > I fixed an issue that the @Name annotation was printed twice as > followings in "jfr metadata" output. > > @Name("jdk.ActiveSetting") > @Name("jdk.ActiveSetting") > @Label("Recording Setting") > @Category("Flight Recorder") > class ActiveSetting extends jdk.jfr.Event { > > > Could you reveiew this please? > > webrev: > http://cr.openjdk.java.net/~cito/JDK-8223066/webrev.00/ > > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223066 > > I have passed all the tests in test-tier1. > > Best regards, > Chihiro From yasuenag at gmail.com Thu May 2 02:04:42 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 2 May 2019 11:04:42 +0900 Subject: PING: RFR: 8220657: JFR.dump does not work when filename is set In-Reply-To: <6348d8ac-b713-1cb3-39db-859bc833e0ce@gmail.com> References: <5C90E8C9.3010008@oracle.com> <14c04f3a-1fc0-af3b-714f-e35fa18d6a1b@gmail.com> <5C9103DE.4040304@oracle.com> <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> <2dc4e611-bd18-a187-f529-d243af51cf10@gmail.com> <5CBE36B6.6070603@oracle.com> <6348d8ac-b713-1cb3-39db-859bc833e0ce@gmail.com> Message-ID: PING: Could you review it? > http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ Yasumasa On 2019/04/24 21:43, Yasumasa Suenaga wrote: > Hi Erik, > > I uploaded new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ > > I kept to remove setDestination() call in PlatformRecording. > In this webrev, original filename value will be set to temporary > snapshot if DCmdDump does not have recording name and filename. > > Also I added new testcase. It can test JFR.dump with filename. > > > Thanks, > > Yasumasa > > > On 2019/04/23 6:48, Erik Gahlin wrote: >> I have looked further at the issue and while the message to the user >> was incorrect previously ( "Dump failed. No data found in the >> specified interval") it did result in a file with the specified >> filename, i.e "test.jfr" >> >> When "clone.setDestination(this.destination);" is removed, the >> filename passed on command line is ignored, but it does result in a >> message that is consistent with what is happening (a file with an >> autogenerated name is created). It does however change the behavior of >> 'JFR.dump name=1'. Logic needs to be added to DCmdDump so it >> uses an existing filename if it is set, but changes it if a user is >> providing an override. >> >> When comes to testing. I think it would be better to write a separate >> test that checks that things are dumped correctly >> when-XX:StartFlightRecording:filename= is used. >> >> Thanks >> Erik >> >>> PING: Do you have any opinion(s) about this fix? >>> >>> >>> Yasumasa >>> >>> >>> On 2019/04/01 21:46, Yasumasa Suenaga wrote: >>>> Hi, >>>> >>>> Do you have concern(s) about this change? >>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>> >>>> As in the email below, I think it does not affect scenarios except >>>> snapshot cloning. >>>> Please tell me if you have any opinions about this. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2019/03/20 10:37, Yasumasa Suenaga wrote: >>>>> Hi Erik, >>>>> >>>>> I think this change does not affect other scenarios. >>>>> >>>>> :jdk_jfr jtreg tests and submit repo have passed with this change. >>>>> newSnapshotClone() is called by PlatformRecording::dump and >>>>> DCmdDump::newSnapshot. >>>>> Both methods are related to user-requested dump. >>>>> >>>>> What scenarios do you think? I want to check them. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> 2019?3?19?(?) 23:59 Erik Gahlin : >>>>>> >>>>>> Yes, but then you would lose the original destination in other >>>>>> scenarios >>>>>> >>>>>> Maybe I can look at this next week. >>>>>> >>>>>> Erik >>>>>> >>>>>>> Hi Erik, >>>>>>> >>>>>>> Thank you for your reply. >>>>>>> >>>>>>> How about this changeset? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>>>> >>>>>>> I think PlatformRecording#newSnapshotClone() should not dump. >>>>>>> So I removed `setDestination()` from this. >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2019/03/19 22:04, Erik Gahlin wrote: >>>>>>>> This looks a bit hackish. >>>>>>>> >>>>>>>> The real issue seems to be that the method >>>>>>>> PlatformRecording#newSnapshotClone incorrectly sets the >>>>>>>> destination in >>>>>>>> the clone, which then triggers a dump prematurely (when stop is >>>>>>>> called). >>>>>>>> >>>>>>>> Purpose of that method is just to take a snapshot of existing >>>>>>>> chunks in >>>>>>>> the disk repository (or from memory). and then let >>>>>>>> dumpStopped(path) do >>>>>>>> the actual dump (copy out the data from the disk repository). >>>>>>>> >>>>>>>> To make it work, some other changes would be needed as well, but >>>>>>>> I don't >>>>>>>> have time to investigate. >>>>>>>> >>>>>>>> Erik >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Please review this change. >>>>>>>>> >>>>>>>>> ???? JBS: https://bugs.openjdk.java.net/browse/JDK-8220657 >>>>>>>>> ???? webrev: >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.00/ >>>>>>>>> >>>>>>>>> The user might not get flight record via JFR.dump if JFR is >>>>>>>>> started >>>>>>>>> with filename option on target process. >>>>>>>>> >>>>>>>>> Please see JBS if you want to know how to reproduce. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>> >> From erik.gahlin at oracle.com Thu May 2 14:41:12 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 2 May 2019 16:41:12 +0200 Subject: PING: RFR: 8220657: JFR.dump does not work when filename is set In-Reply-To: References: <5C90E8C9.3010008@oracle.com> <14c04f3a-1fc0-af3b-714f-e35fa18d6a1b@gmail.com> <5C9103DE.4040304@oracle.com> <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> <2dc4e611-bd18-a187-f529-d243af51cf10@gmail.com> <5CBE36B6.6070603@oracle.com> <6348d8ac-b713-1cb3-39db-859bc833e0ce@gmail.com> Message-ID: <5CCB0188.7020204@oracle.com> Hi Yasumasa, The test looks complicated. I would prefer something more straight forward with less states and nulls where the expected behavior is easy to see. If the test fails, it's also nice to see from the stack trace which sub test failed. public static void main(String... args) throws Exception { testDumpAll(); testDumpNamed(); testDumpNamedWithFilename(); } private static void testDumpAll() throws Exception { Path p = Path.of("A.jfr").toAbsolutePath(); try (Recording r = new Recording()) { r.setName("A"); r.setDestination(p); JcmdHelper.jcmd("JFR.dump"); Asserts.assertFalse(namedFile(p), "Unexpected file: " + p); Asserts.assertTrue(generatedFile(), "Expected generated file"); } } private static void testDumpNamed() throws Exception { Path p = Path.of("B.jfr").toAbsolutePath(); try (Recording r = new Recording()) { r.setName("B"); r.setDestination(p); JcmdHelper.jcmd("JFR.dump", "name=B"); Asserts.assertTrue(namedFile(p), "Expected file: " + p); Asserts.assertFalse(generatedFile(), "Unexpected generated file"); } } private static void testDumpNamedWithFilename() throws Exception { Path p = Path.of("C.jfr").toAbsolutePath(); Path override = Path.of("override.jfr").toAbsolutePath(); try (Recording r = new Recording()) { r.setName("C"); r.setDestination(p); JcmdHelper.jcmd("JFR.dump", "name=C", "filename=" + override); Asserts.assertFalse(namedFile(p), "Unexpected file: " + p); Asserts.assertTrue(namedFile(p), "Expected file: " + override); Asserts.assertFalse(generatedFile(), "Unexpected generated file"); } } private static boolean namedFile(Path dumpFile) throws IOException { return Files.exists(dumpFile) && Files.size(dumpFile) > 0; } private static boolean generatedFile() throws IOException { long pid = ProcessHandle.current().pid(); return Files.find(Path.of("."), 1, (p, a) -> p.toString() .matches("^.*hotspot-pid-" + pid + ".jfr$") && a.size() > 0L) .findFirst().isPresent(); } I haven't tried it. When it comes to DCmdDump, I would prefer if you would not use "var" as this hides the underlying type (SafePath or WriteableUserPath) which is used to signal if there is a security risk or not. I think it is fine, but if the code gets refactor at a later stage the information could get lost. Thanks Erik > PING: Could you review it? > > > http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ > > > Yasumasa > > > On 2019/04/24 21:43, Yasumasa Suenaga wrote: >> Hi Erik, >> >> I uploaded new webrev: >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ >> >> I kept to remove setDestination() call in PlatformRecording. >> In this webrev, original filename value will be set to temporary >> snapshot if DCmdDump does not have recording name and filename. >> >> Also I added new testcase. It can test JFR.dump with filename. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2019/04/23 6:48, Erik Gahlin wrote: >>> I have looked further at the issue and while the message to the user >>> was incorrect previously ( "Dump failed. No data found in the >>> specified interval") it did result in a file with the specified >>> filename, i.e "test.jfr" >>> >>> When "clone.setDestination(this.destination);" is removed, the >>> filename passed on command line is ignored, but it does result in a >>> message that is consistent with what is happening (a file with an >>> autogenerated name is created). It does however change the behavior >>> of 'JFR.dump name=1'. Logic needs to be added to DCmdDump so >>> it uses an existing filename if it is set, but changes it if a user >>> is providing an override. >>> >>> When comes to testing. I think it would be better to write a >>> separate test that checks that things are dumped correctly >>> when-XX:StartFlightRecording:filename= is used. >>> >>> Thanks >>> Erik >>> >>>> PING: Do you have any opinion(s) about this fix? >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2019/04/01 21:46, Yasumasa Suenaga wrote: >>>>> Hi, >>>>> >>>>> Do you have concern(s) about this change? >>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>> >>>>> As in the email below, I think it does not affect scenarios except >>>>> snapshot cloning. >>>>> Please tell me if you have any opinions about this. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2019/03/20 10:37, Yasumasa Suenaga wrote: >>>>>> Hi Erik, >>>>>> >>>>>> I think this change does not affect other scenarios. >>>>>> >>>>>> :jdk_jfr jtreg tests and submit repo have passed with this change. >>>>>> newSnapshotClone() is called by PlatformRecording::dump and >>>>>> DCmdDump::newSnapshot. >>>>>> Both methods are related to user-requested dump. >>>>>> >>>>>> What scenarios do you think? I want to check them. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> 2019?3?19?(?) 23:59 Erik Gahlin : >>>>>>> >>>>>>> Yes, but then you would lose the original destination in other >>>>>>> scenarios >>>>>>> >>>>>>> Maybe I can look at this next week. >>>>>>> >>>>>>> Erik >>>>>>> >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> Thank you for your reply. >>>>>>>> >>>>>>>> How about this changeset? >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>>>>> >>>>>>>> I think PlatformRecording#newSnapshotClone() should not dump. >>>>>>>> So I removed `setDestination()` from this. >>>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2019/03/19 22:04, Erik Gahlin wrote: >>>>>>>>> This looks a bit hackish. >>>>>>>>> >>>>>>>>> The real issue seems to be that the method >>>>>>>>> PlatformRecording#newSnapshotClone incorrectly sets the >>>>>>>>> destination in >>>>>>>>> the clone, which then triggers a dump prematurely (when stop >>>>>>>>> is called). >>>>>>>>> >>>>>>>>> Purpose of that method is just to take a snapshot of existing >>>>>>>>> chunks in >>>>>>>>> the disk repository (or from memory). and then let >>>>>>>>> dumpStopped(path) do >>>>>>>>> the actual dump (copy out the data from the disk repository). >>>>>>>>> >>>>>>>>> To make it work, some other changes would be needed as well, >>>>>>>>> but I don't >>>>>>>>> have time to investigate. >>>>>>>>> >>>>>>>>> Erik >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Please review this change. >>>>>>>>>> >>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220657 >>>>>>>>>> webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.00/ >>>>>>>>>> >>>>>>>>>> The user might not get flight record via JFR.dump if JFR is >>>>>>>>>> started >>>>>>>>>> with filename option on target process. >>>>>>>>>> >>>>>>>>>> Please see JBS if you want to know how to reproduce. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>> >>> From christoph.langer at sap.com Thu May 2 20:43:10 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 2 May 2019 20:43:10 +0000 Subject: [11u] Questions re. CSR && Backport of JDK-8205516: JFR tool In-Reply-To: References: Message-ID: Hi Jie, Mario, we already ran through backporting an issue with an attached CSR, so I can give you some advice: Generally, you should create a backport JBS item for the issue with target 11-pool. From that one, you'll have to create a CSR, also with version set to 11-pool. In that CSR you can largely paste the original CSR data (as it applies). For your case, I see that there already exists an 11-pool backport item, assigned to Fairoz Matte: https://bugs.openjdk.java.net/browse/JDK-8222896. Maybe you and the colleagues from Oracle could cooperate on this item, at least for JDK11...? Hence, cc-ing hotspot-jfr-dev and Erik and Fairoz. Best regards, Christoph > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Mario Torre > Sent: Donnerstag, 2. Mai 2019 19:53 > To: Jie Kang > Cc: jdk-updates-dev at openjdk.java.net > Subject: Re: [11u] Questions re. CSR && Backport of JDK-8205516: JFR tool > > Since we?re backporting JFR to 8, perhaps makes sense to do that for 8 as > well ;) > > I can help you with the bug and the CSR request, I?ll give a look at it and > let you know. > > Cheers, > Mario > > On Thu 2. May 2019 at 19:28, Jie Kang wrote: > > > Hi all, > > > > I am interested in doing the backport work for JDK-8205516: JFR tool > > [1] to openjdk 11u. A CSR was filed for the original bug so I believe > > I will need to file a CSR for the backport bug [2] as well. However, I > > am not an author or a committer; is this something someone can do in > > my stead? > > > > Also: why should the JFR tool be backported to 11? > > > > JFR is now in openjdk 11+. In order to extract information from flight > > recordings, one would need to write a parser, maybe using existing > > java libraries like jmc-core, download a separate tool (if any exist), > > or use JMC, a full fledged GUI application. A CLI tool that is part of > > the JDK is a very good alternative and I think it makes sense for the > > JFR tool to be available in JDKs that support flight recordings, > > including 11. > > > > > > Regards, > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8205516 > > [2] https://bugs.openjdk.java.net/browse/JDK-8222896 > > > -- > pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF > Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF > > Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens > Proud GNU Classpath developer: http://www.classpath.org/ > OpenJDK: http://openjdk.java.net/projects/caciocavallo/ > > Please, support open standards: > http://endsoftpatents.org/ From neugens at redhat.com Fri May 3 08:26:18 2019 From: neugens at redhat.com (Mario Torre) Date: Fri, 3 May 2019 10:26:18 +0200 Subject: [11u] Questions re. CSR && Backport of JDK-8205516: JFR tool In-Reply-To: References: Message-ID: Hi Christoph, Thanks for the input! Erik, Fairoz, could you please share with us the progress on this? Cheers, Mario On Thu, May 2, 2019 at 10:43 PM Langer, Christoph wrote: > > Hi Jie, Mario, > > we already ran through backporting an issue with an attached CSR, so I can give you some advice: > > Generally, you should create a backport JBS item for the issue with target 11-pool. From that one, you'll have to create a CSR, also with version set to 11-pool. In that CSR you can largely paste the original CSR data (as it applies). > > For your case, I see that there already exists an 11-pool backport item, assigned to Fairoz Matte: https://bugs.openjdk.java.net/browse/JDK-8222896. > > Maybe you and the colleagues from Oracle could cooperate on this item, at least for JDK11...? Hence, cc-ing hotspot-jfr-dev and Erik and Fairoz. > > Best regards, > Christoph > > > > -----Original Message----- > > From: jdk-updates-dev On > > Behalf Of Mario Torre > > Sent: Donnerstag, 2. Mai 2019 19:53 > > To: Jie Kang > > Cc: jdk-updates-dev at openjdk.java.net > > Subject: Re: [11u] Questions re. CSR && Backport of JDK-8205516: JFR tool > > > > Since we?re backporting JFR to 8, perhaps makes sense to do that for 8 as > > well ;) > > > > I can help you with the bug and the CSR request, I?ll give a look at it and > > let you know. > > > > Cheers, > > Mario > > > > On Thu 2. May 2019 at 19:28, Jie Kang wrote: > > > > > Hi all, > > > > > > I am interested in doing the backport work for JDK-8205516: JFR tool > > > [1] to openjdk 11u. A CSR was filed for the original bug so I believe > > > I will need to file a CSR for the backport bug [2] as well. However, I > > > am not an author or a committer; is this something someone can do in > > > my stead? > > > > > > Also: why should the JFR tool be backported to 11? > > > > > > JFR is now in openjdk 11+. In order to extract information from flight > > > recordings, one would need to write a parser, maybe using existing > > > java libraries like jmc-core, download a separate tool (if any exist), > > > or use JMC, a full fledged GUI application. A CLI tool that is part of > > > the JDK is a very good alternative and I think it makes sense for the > > > JFR tool to be available in JDKs that support flight recordings, > > > including 11. > > > > > > > > > Regards, > > > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8205516 > > > [2] https://bugs.openjdk.java.net/browse/JDK-8222896 > > > > > -- > > pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF > > Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF > > > > Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens > > Proud GNU Classpath developer: http://www.classpath.org/ > > OpenJDK: http://openjdk.java.net/projects/caciocavallo/ > > > > Please, support open standards: > > http://endsoftpatents.org/ -- Mario Torre Associate Manager, Software Engineering Red Hat GmbH 9704 A60C B4BE A8B8 0F30 9205 5D7E 4952 3F65 7898 From markus.gronlund at oracle.com Fri May 3 13:00:54 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Fri, 3 May 2019 06:00:54 -0700 (PDT) Subject: RFR(M): 8220293: Deadlock in JFR string pool Message-ID: <5ede3dbd-1f17-489c-abf6-504d1eece7d8@default> Greeting, Please review the following patch to address the following: Bug: https://bugs.openjdk.java.net/browse/JDK-8220293 Webrev: http://cr.openjdk.java.net/~mgronlun/8220293/webrev01/ High level summary: We don't want to release a retired buffer, because this will break the invariant that a successful acquisition implies a valid (non-retired) buffer. This is the reason why this work is, at least prima facie, a little more involved. Tests: test case / reproducer attached to: https://bugs.openjdk.java.net/browse/JDK-8220293 open/test/jdk/:jdk_jfr Thank you Markus From yasuenag at gmail.com Fri May 3 14:05:01 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 3 May 2019 23:05:01 +0900 Subject: PING: RFR: 8220657: JFR.dump does not work when filename is set In-Reply-To: <5CCB0188.7020204@oracle.com> References: <5C90E8C9.3010008@oracle.com> <14c04f3a-1fc0-af3b-714f-e35fa18d6a1b@gmail.com> <5C9103DE.4040304@oracle.com> <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> <2dc4e611-bd18-a187-f529-d243af51cf10@gmail.com> <5CBE36B6.6070603@oracle.com> <6348d8ac-b713-1cb3-39db-859bc833e0ce@gmail.com> <5CCB0188.7020204@oracle.com> Message-ID: Hi Erik, I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.03/ This webrev does not use `var` in any place. And I tweaked testcase based on your idea. Thanks, Yasumasa On 2019/05/02 23:41, Erik Gahlin wrote: > Hi Yasumasa, > > The test looks complicated. > > I would prefer something more straight forward with less states and > nulls where the expected behavior is easy to see. If the test fails, > it's also nice to see from the stack trace which sub test failed. > > ? public static void main(String... args) throws Exception { > ??? testDumpAll(); > ??? testDumpNamed(); > ??? testDumpNamedWithFilename(); > ? } > > ? private static void testDumpAll() throws Exception { > ??? Path p = Path.of("A.jfr").toAbsolutePath(); > ??? try (Recording r = new Recording()) { > ????? r.setName("A"); > ????? r.setDestination(p); > ????? JcmdHelper.jcmd("JFR.dump"); > ????? Asserts.assertFalse(namedFile(p), "Unexpected file: " + p); > ????? Asserts.assertTrue(generatedFile(), "Expected generated file"); > ??? } > ? } > > ? private static void testDumpNamed() throws Exception { > ??? Path p = Path.of("B.jfr").toAbsolutePath(); > ??? try (Recording r = new Recording()) { > ????? r.setName("B"); > ????? r.setDestination(p); > ????? JcmdHelper.jcmd("JFR.dump", "name=B"); > ????? Asserts.assertTrue(namedFile(p), "Expected file: " + p); > ????? Asserts.assertFalse(generatedFile(), "Unexpected generated file"); > ??? } > ? } > > ? private static void testDumpNamedWithFilename() throws Exception { > ??? Path p = Path.of("C.jfr").toAbsolutePath(); > ??? Path override = Path.of("override.jfr").toAbsolutePath(); > ??? try (Recording r = new Recording()) { > ????? r.setName("C"); > ????? r.setDestination(p); > ????? JcmdHelper.jcmd("JFR.dump", "name=C", "filename=" + override); > ????? Asserts.assertFalse(namedFile(p), "Unexpected file: " + p); > ????? Asserts.assertTrue(namedFile(p), "Expected file: " + override); > ????? Asserts.assertFalse(generatedFile(), "Unexpected generated file"); > ??? } > ? } > > ? private static boolean namedFile(Path dumpFile) throws IOException { > ??? return Files.exists(dumpFile) && Files.size(dumpFile) > 0; > ? } > > ? private static boolean generatedFile() throws IOException { > ??? long pid = ProcessHandle.current().pid(); > ??? return Files.find(Path.of("."), 1, (p, a) -> p.toString() > ??????????????? .matches("^.*hotspot-pid-" + pid + ".jfr$") && a.size() > > 0L) > ??????????????? .findFirst().isPresent(); > ? } > > I haven't tried it. > > When it comes to DCmdDump, I would prefer if you would not use "var" as > this hides the underlying type (SafePath or WriteableUserPath) which is > used to signal if there is a security risk or not. I think it is fine, > but if the code gets refactor at a later stage the information could get > lost. > > Thanks > Erik > >> PING: Could you review it? >> >> > http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ >> >> >> Yasumasa >> >> >> On 2019/04/24 21:43, Yasumasa Suenaga wrote: >>> Hi Erik, >>> >>> I uploaded new webrev: >>> >>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ >>> >>> I kept to remove setDestination() call in PlatformRecording. >>> In this webrev, original filename value will be set to temporary >>> snapshot if DCmdDump does not have recording name and filename. >>> >>> Also I added new testcase. It can test JFR.dump with filename. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2019/04/23 6:48, Erik Gahlin wrote: >>>> I have looked further at the issue and while the message to the user >>>> was incorrect previously ( "Dump failed. No data found in the >>>> specified interval") it did result in a file with the specified >>>> filename, i.e "test.jfr" >>>> >>>> When "clone.setDestination(this.destination);" is removed, the >>>> filename passed on command line is ignored, but it does result in a >>>> message that is consistent with what is happening (a file with an >>>> autogenerated name is created). It does however change the behavior >>>> of 'JFR.dump name=1'. Logic needs to be added to DCmdDump so >>>> it uses an existing filename if it is set, but changes it if a user >>>> is providing an override. >>>> >>>> When comes to testing. I think it would be better to write a >>>> separate test that checks that things are dumped correctly >>>> when-XX:StartFlightRecording:filename= is used. >>>> >>>> Thanks >>>> Erik >>>> >>>>> PING: Do you have any opinion(s) about this fix? >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2019/04/01 21:46, Yasumasa Suenaga wrote: >>>>>> Hi, >>>>>> >>>>>> Do you have concern(s) about this change? >>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>>> >>>>>> As in the email below, I think it does not affect scenarios except >>>>>> snapshot cloning. >>>>>> Please tell me if you have any opinions about this. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2019/03/20 10:37, Yasumasa Suenaga wrote: >>>>>>> Hi Erik, >>>>>>> >>>>>>> I think this change does not affect other scenarios. >>>>>>> >>>>>>> :jdk_jfr jtreg tests and submit repo have passed with this change. >>>>>>> newSnapshotClone() is called by PlatformRecording::dump and >>>>>>> DCmdDump::newSnapshot. >>>>>>> Both methods are related to user-requested dump. >>>>>>> >>>>>>> What scenarios do you think? I want to check them. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> 2019?3?19?(?) 23:59 Erik Gahlin : >>>>>>>> >>>>>>>> Yes, but then you would lose the original destination in other >>>>>>>> scenarios >>>>>>>> >>>>>>>> Maybe I can look at this next week. >>>>>>>> >>>>>>>> Erik >>>>>>>> >>>>>>>>> Hi Erik, >>>>>>>>> >>>>>>>>> Thank you for your reply. >>>>>>>>> >>>>>>>>> How about this changeset? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>>>>>> >>>>>>>>> I think PlatformRecording#newSnapshotClone() should not dump. >>>>>>>>> So I removed `setDestination()` from this. >>>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2019/03/19 22:04, Erik Gahlin wrote: >>>>>>>>>> This looks a bit hackish. >>>>>>>>>> >>>>>>>>>> The real issue seems to be that the method >>>>>>>>>> PlatformRecording#newSnapshotClone incorrectly sets the >>>>>>>>>> destination in >>>>>>>>>> the clone, which then triggers a dump prematurely (when stop >>>>>>>>>> is called). >>>>>>>>>> >>>>>>>>>> Purpose of that method is just to take a snapshot of existing >>>>>>>>>> chunks in >>>>>>>>>> the disk repository (or from memory). and then let >>>>>>>>>> dumpStopped(path) do >>>>>>>>>> the actual dump (copy out the data from the disk repository). >>>>>>>>>> >>>>>>>>>> To make it work, some other changes would be needed as well, >>>>>>>>>> but I don't >>>>>>>>>> have time to investigate. >>>>>>>>>> >>>>>>>>>> Erik >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Please review this change. >>>>>>>>>>> >>>>>>>>>>> ???? JBS: https://bugs.openjdk.java.net/browse/JDK-8220657 >>>>>>>>>>> ???? webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> The user might not get flight record via JFR.dump if JFR is >>>>>>>>>>> started >>>>>>>>>>> with filename option on target process. >>>>>>>>>>> >>>>>>>>>>> Please see JBS if you want to know how to reproduce. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>> >>>> > From kdobson at redhat.com Fri May 3 14:44:38 2019 From: kdobson at redhat.com (Ken Dobson) Date: Fri, 3 May 2019 10:44:38 -0400 Subject: RFR 8221507: Implement JFR Events for Shenandoah Message-ID: Hi all, Please review this patch that adds support for two new JFR events ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ The Shenandoah team has also reviewed this patch and approved it from their end. Thanks, Ken Dobson From sgehwolf at redhat.com Fri May 3 15:56:52 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 03 May 2019 17:56:52 +0200 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: <9b790032b583039880db3a7da35f5ebe8ea61e2d.camel@redhat.com> cc'ing hotspot-gc-dev as this also touches GC code (shenandoah). On Fri, 2019-05-03 at 10:44 -0400, Ken Dobson wrote: > Hi all, > > Please review this patch that adds support for two new JFR events > ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > The Shenandoah team has also reviewed this patch and approved it from their > end. > > Thanks, > > Ken Dobson From chihiro.ito at oracle.com Fri May 3 17:48:50 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Sat, 4 May 2019 02:48:50 +0900 Subject: RFR: 8223066: "jfr metadata" output the @Name annotation twice. In-Reply-To: <5CC9F026.4050002@oracle.com> References: <398c166c-b681-6d32-4f4c-9d43350b622f@oracle.com> <5CC9F026.4050002@oracle.com> Message-ID: <82495479-558b-f2da-8ae8-108d07de3e4b@oracle.com> Hi Erik, Thank you for sharing the background. In the former case, I feel that PrettyWriter strongly depends on the name and annotation information of Type. So I think the latter is better. Regard Chihiro On 2019/05/02 4:14, Erik Gahlin wrote: > Hi Chihiro, > > Thanks for reporting the issue, @Name should not be printed twice. > > Let me give you some background information. I think there are two > issues at hand: > > 1) The Name annotation should probably not be persisted. This can be > achieved by removing the MetadataDefinition from the class. I did try > that, but I noticed several tests failing, in particular TestName > which explicitly checks that @Name is persisted. The purpose @Name is > to override the default name, which can then be retrieved from > ValueDescriptor::getName, EventType::getName and > SettingDescriptor::getName. The annotation is not needed on the > consumer side as the name is already persisted elsewhere. > > I remember discussing this several times during the design, but for > some reason we came to the conclusion to persist it, probably to not > confuse users if it didn't behave like @Label and @Description. > Later, Metadatadefinition was removed from @Threshold, @Periodic, > @Enabled and @StackTrace. At that time, it should probably have been > removed from @Name as well. Maybe we should go ahead and remove it > now, but I will have to think about the implications first. > > 2) The reason PrettyWriter::printType adds @Name is because there is > no good way to include package/namespace in the printout. Events that > lack a namespace should not get the annotation, that's why there is a > check for "." It's not because the values are primitives. > > I can see two ways to proceed. Either add a check in printType so > @Name is only printed if it doesn't already exists, i.e. > > if (t.getAnnotation(Name.class) == null) { > if (index != -1) { > println("@Name(\"" + typeName + "\")"); > } > } > > or remove @MetadataDefinition from the Name class and make > adjustments to tests etc. If you chose the latter, I like a week to > think it over, investigate the implications and discuss it with other > people, before I can say if it is the right approach or not. > > Thanks > Erik > > Hi, >> >> I fixed an issue that the @Name annotation was printed twice as >> followings in "jfr metadata" output. >> >> @Name("jdk.ActiveSetting") >> @Name("jdk.ActiveSetting") >> @Label("Recording Setting") >> @Category("Flight Recorder") >> class ActiveSetting extends jdk.jfr.Event { >> >> >> Could you reveiew this please? >> >> webrev: >> http://cr.openjdk.java.net/~cito/JDK-8223066/webrev.00/ >> >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8223066 >> >> I have passed all the tests in test-tier1. >> >> Best regards, >> Chihiro > > From mikhailo.seledtsov at oracle.com Fri May 3 22:19:58 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Fri, 3 May 2019 15:19:58 -0700 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: Hi, ? If possible and feasible, could you please implement tests for new events? Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps create a "shenandoah" subfolder. If the events are too hard to test or not feasible, please add the event names to test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to the hardToTestEvents set. Consider commenting why it is too hard to test. Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ prior to integration. Best regards, Misha On 5/3/19 7:44 AM, Ken Dobson wrote: > Hi all, > > Please review this patch that adds support for two new JFR events > ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > The Shenandoah team has also reviewed this patch and approved it from their > end. > > Thanks, > > Ken Dobson From markus.gronlund at oracle.com Sat May 4 20:23:16 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Sat, 4 May 2019 13:23:16 -0700 (PDT) Subject: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup Message-ID: Greetings, Please review the following patch to improve startup when using JFR: Bug/Enh: https://bugs.openjdk.java.net/browse/JDK-8217089 Webrev: http://cr.openjdk.java.net/~mgronlun/8217089/webrev01/ Summary: Let the periodic event requestor thread lazy install os interface components instead of doing it eagerly as part of startup by the initial thread. Thank you Markus From erik.gahlin at oracle.com Sat May 4 20:43:27 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Sat, 4 May 2019 22:43:27 +0200 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: <5CCDF96F.8070701@oracle.com> Hi Ken, Nice to see some Shenandoah specific events. - You need to update the configuration files for the events (default.jfc and profile.jfc) located in src/jdk.jfr/share/conf/jfr. - I think you need to put INCLUDE_SHENANDOAHGC inside the TRACE_FUNCTION in jfrPeriodic.cpp. A call to the TRACE_FUNCTION will happen regardless if it is built with Shenandoah or not, since the dispatch mechanism is generated from metadata.xml. - If Shenandoah is still an experimental feature(?), the Event element in metadata.xml should have the experimental="true" attribute set [1]. - It would be nice if you could move the Type element in metadata.xml for ShenandoahHeapRegionState below the events so they are next to the other Type elements. Thanks Erik [1] The experimental attribute was created so that JVM developers could add events more freely to HotSpot without the burden to write unit tests or commit to a particular event design. Experimental events are not shown in JMC by default, but can be made visible in the Preference dialog. See the following picture. http://cr.openjdk.java.net/~egahlin/8213966/experimental.png > Hi all, > > Please review this patch that adds support for two new JFR events > ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > The Shenandoah team has also reviewed this patch and approved it from their > end. > > Thanks, > > Ken Dobson From erik.gahlin at oracle.com Sat May 4 21:16:00 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Sat, 4 May 2019 23:16:00 +0200 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: <5CCE0110.7070908@oracle.com> On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: > Hi, > > If possible and feasible, could you please implement tests for new > events? > > Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > create a "shenandoah" subfolder. > > If the events are too hard to test or not feasible, please add the > event names to > test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to the > hardToTestEvents set. Consider commenting why it is too hard to test. If events are are marked experimental, tests are ignored by TestLookForUntestedEvents. java, so we should not add them to the hardToTestEvents set. That may help them pass under the radar once the experimental attribute is removed. That said, tests never hurts, even if they are experimental. I think test/jdk/jdk/jfr/event/gc/detailed would be a good place Erik > > Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ > prior to integration. > > > Best regards, > > Misha > > > > On 5/3/19 7:44 AM, Ken Dobson wrote: >> Hi all, >> >> Please review this patch that adds support for two new JFR events >> ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 >> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ >> >> The Shenandoah team has also reviewed this patch and approved it from >> their >> end. >> >> Thanks, >> >> Ken Dobson > From chihiro.ito at oracle.com Mon May 6 12:19:53 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Mon, 6 May 2019 21:19:53 +0900 Subject: RFR: 8223393: --ids option of "jfr metadata" command doesn't be accepted Message-ID: <39d39ac1-4421-7be1-74ad-539eef73a78a@oracle.com> Hi, Coud you review this tiny fix please? webrev: http://cr.openjdk.java.net/~cito/JDK-8223393/webrev.00/ JBS: https://bugs.openjdk.java.net/browse/JDK-8223393 I have passed all the tests in test/jdk/jdk/jfr. Best regards, Chihiro From mikhailo.seledtsov at oracle.com Mon May 6 15:38:08 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 6 May 2019 08:38:08 -0700 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: <5CCE0110.7070908@oracle.com> References: <5CCE0110.7070908@oracle.com> Message-ID: <2055f245-c722-c61b-2f85-406341cb6668@oracle.com> On 5/4/19 2:16 PM, Erik Gahlin wrote: > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: >> Hi, >> >> ? If possible and feasible, could you please implement tests for new >> events? >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps >> create a "shenandoah" subfolder. >> >> If the events are too hard to test or not feasible, please add the >> event names to >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to >> the hardToTestEvents set. Consider commenting why it is too hard to >> test. > > If events are are marked experimental, tests are ignored by > TestLookForUntestedEvents. java, so we should not? add them to the > hardToTestEvents set. That may help them pass under the radar once the > experimental attribute is removed. Thank you. If the events are "experimental" this approach makes sense. Once the experimental status is removed, the TestLookForUntestedEvents.java should be able to notice if the event is not covered by tests. > That said, tests never hurts, even if they are experimental. I think > test/jdk/jdk/jfr/event/gc/detailed would be a good place +1 Thank you, Misha > > Erik > >> >> Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ >> prior to integration. >> >> >> Best regards, >> >> Misha >> >> >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: >>> Hi all, >>> >>> Please review this patch that adds support for two new JFR events >>> ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ >>> >>> The Shenandoah team has also reviewed this patch and approved it >>> from their >>> end. >>> >>> Thanks, >>> >>> Ken Dobson >> > From ralf.schmelter at sap.com Tue May 7 08:27:53 2019 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Tue, 7 May 2019 08:27:53 +0000 Subject: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup In-Reply-To: References: Message-ID: Hi Markus, I'm not a reviewer, but shouldn't the lazy initialization code have proper memory barriers? Or is there another mechanism which avoids concurrent execution of that code? Best regards, Ralf From markus.gronlund at oracle.com Tue May 7 09:07:52 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 7 May 2019 02:07:52 -0700 (PDT) Subject: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup In-Reply-To: References: Message-ID: <8560f0b5-df9a-41ab-aca1-168d1a0e5557@default> Hi Ralf, Thanks for noticing. It's currently built with the premise that there is only a single thread executing this code, the periodic event requestor thread. Thanks Markus -----Original Message----- From: Schmelter, Ralf Sent: den 7 maj 2019 10:28 To: Markus Gronlund ; Hotspot dev runtime ; hotspot-jfr-dev at openjdk.java.net Subject: RE: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup Hi Markus, I'm not a reviewer, but shouldn't the lazy initialization code have proper memory barriers? Or is there another mechanism which avoids concurrent execution of that code? Best regards, Ralf From chihiro.ito at oracle.com Tue May 7 17:53:27 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Wed, 8 May 2019 02:53:27 +0900 Subject: RFR: 8223498: Add Sum of Duration to the jfr summary command Message-ID: <2109ea3e-0796-e6b0-5f0f-7b8ab6d6d21c@oracle.com> Hi, Could you review this enhancement, please? This enhancement adds the total duration of each event to the jfr summary command to help the user determine which events to display in the jfr output. webrev: http://cr.openjdk.java.net/~cito/JDK-8223498/webrev.00/ jbs: https://bugs.openjdk.java.net/browse/JDK-8223498 I have passed the tests in test/jdk/jdk/jfr. Best regards, Chihiro From yasuenag at gmail.com Wed May 8 13:03:20 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 8 May 2019 22:03:20 +0900 Subject: RFR: 8223498: Add Sum of Duration to the jfr summary command In-Reply-To: <2109ea3e-0796-e6b0-5f0f-7b8ab6d6d21c@oracle.com> References: <2109ea3e-0796-e6b0-5f0f-7b8ab6d6d21c@oracle.com> Message-ID: <8f5b4823-b335-9150-bfcf-0458824bb25b@gmail.com> Hi Chihiro, I think Statistics.sumOfDurationTicks might be overflown. Many thread might be in park and/or sleep event. If so, sum value will be very large number. IMHO we need to set Long.MAX_VALUE to sumOfDurationTicks if it exceeds Long.MAX_VALUE. Then sum value will be shown like "<= 999999999 ns". If sum value exceeds LONG_MAX, sumOfDurationTicks is set to Long.MAX_VALUE, and it is shown console like "<= 9999999999 ns". nanosec might be very large number. So we can use exponent to show. Thanks, Yasumasa On 2019/05/08 2:53, Chihiro Ito wrote: > Hi, > > Could you review this enhancement, please? > > This enhancement adds the total duration of each event to the jfr > summary command to help the user determine which events to display in > the jfr output. > > webrev: > http://cr.openjdk.java.net/~cito/JDK-8223498/webrev.00/ > > > jbs: > https://bugs.openjdk.java.net/browse/JDK-8223498 > > I have passed the tests in test/jdk/jdk/jfr. > > Best regards, > Chihiro > From erik.gahlin at oracle.com Wed May 8 13:55:03 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 8 May 2019 15:55:03 +0200 Subject: RFR: 8223498: Add Sum of Duration to the jfr summary command In-Reply-To: <2109ea3e-0796-e6b0-5f0f-7b8ab6d6d21c@oracle.com> References: <2109ea3e-0796-e6b0-5f0f-7b8ab6d6d21c@oracle.com> Message-ID: <5CD2DFB7.4040307@oracle.com> I don't see a benefit of adding a sum of duration column. Events have thresholds. For example, the jdk.JavaMonitorWait event is only recorded if the event exceeds 20 ms so the sum would not be the actual sum. Also, things often happen in multiple threads which makes it hard to know what the value actually means. The 'jfr' tool tries to not exceed 80 characters (to avoid wrapping), which is hard if another column is added. This especially true if user defined events are added with a longer namespace than "jdk." The longer term plan is to add a more advanced command where users can aggregate data on arbitrary field and not just the sum as an aggregate function. Users would then be able to aggregate duration per event type, if they really wanted it. Erik > Hi, > > Could you review this enhancement, please? > > This enhancement adds the total duration of each event to the jfr > summary command to help the user determine which events to display in > the jfr output. > > webrev: > http://cr.openjdk.java.net/~cito/JDK-8223498/webrev.00/ > > > jbs: > https://bugs.openjdk.java.net/browse/JDK-8223498 > > I have passed the tests in test/jdk/jdk/jfr. > > Best regards, > Chihiro From chihiro.ito at oracle.com Wed May 8 15:21:23 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Thu, 9 May 2019 00:21:23 +0900 Subject: RFR: 8223498: Add Sum of Duration to the jfr summary command In-Reply-To: <5CD2DFB7.4040307@oracle.com> References: <2109ea3e-0796-e6b0-5f0f-7b8ab6d6d21c@oracle.com> <5CD2DFB7.4040307@oracle.com> Message-ID: Until the jfr tool was created, the only way a user could see a jfr file was through JMC. Therefore, when a failure occurs, the user must open all JFRs in the JMC for viewing. Therefore, users want to be able to use the jfr tool to narrow down the jfr files they should see in the JMC. This allows users to see fewer jfr files using JMC. I have used JFR on a cluster of hundreds of Java processes, but it was very hard to see all the jfr files to figure out the cause. JMC had the ability to display the total duration, which was very useful. In my experience, I never use default.jfc or profile.jfc as is. All events indicating that threads cannot be used effectively should be logged without a threshold. I also limit the threads displayed in the JMC to those that are processed by the application. In such a situation, looking at the total duration is very useful for analyzing what is happening in the application. The application may be monitoring by code hidden in the library, or it may be sleeping a thread. I did not consider the 80 character limit. I think limiting the width is very useful for people to see. Instead of displaying nanosecond values, I will devise something. I'm very excited about future enhancements. It would be great if we could provide common requests as standard functions. Regards, Chihiro On 2019/05/08 22:55, Erik Gahlin wrote: > I don't see a benefit of adding a sum of duration column. > > Events have thresholds. For example, the jdk.JavaMonitorWait event is > only recorded if the event exceeds 20 ms so the sum would not be the > actual sum. Also, things often happen in multiple threads which makes > it hard to know what the value actually means. > > The 'jfr' tool tries to not exceed 80 characters (to avoid wrapping), > which is hard if another column is added. This especially true if user > defined events are added with a longer namespace than "jdk." > > The longer term plan is to add a more advanced command where users can > aggregate data on arbitrary field and not just the sum as an aggregate > function. Users would then be able to aggregate duration per event > type, if they really wanted it. > > Erik > >> Hi, >> >> Could you review this enhancement, please? >> >> This enhancement adds the total duration of each event to the jfr >> summary command to help the user determine which events to display in >> the jfr output. >> >> webrev: >> http://cr.openjdk.java.net/~cito/JDK-8223498/webrev.00/ >> >> >> jbs: >> https://bugs.openjdk.java.net/browse/JDK-8223498 >> >> I have passed the tests in test/jdk/jdk/jfr. >> >> Best regards, >> Chihiro > > From david.holmes at oracle.com Fri May 10 02:32:02 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 10 May 2019 12:32:02 +1000 Subject: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup In-Reply-To: References: Message-ID: Hi Markus, On 5/05/2019 6:23 am, Markus Gronlund wrote: > Greetings, > > Please review the following patch to improve startup when using JFR: > > Bug/Enh: https://bugs.openjdk.java.net/browse/JDK-8217089 > Webrev: http://cr.openjdk.java.net/~mgronlun/8217089/webrev01/ > > Summary: > > Let the periodic event requestor thread lazy install os interface components instead of doing it eagerly as part of startup by the initial thread. That all seems fine for single-threaded lazy initialization. Only thing I'm unclear on is the addition of the potential to return OS_ERR. I can't see where these methods get called so can't tell if the callers are prepared to handle the error. Thanks, David ----- > Thank you > Markus > From erik.gahlin at oracle.com Fri May 10 04:53:20 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Fri, 10 May 2019 06:53:20 +0200 Subject: RFR: 8223498: Add Sum of Duration to the jfr summary command In-Reply-To: References: <2109ea3e-0796-e6b0-5f0f-7b8ab6d6d21c@oracle.com> <5CD2DFB7.4040307@oracle.com> Message-ID: > On 8 May 2019, at 17:21, Chihiro Ito wrote: > > Until the jfr tool was created, the only way a user could see a jfr file was through JMC. Therefore, when a failure occurs, the user must open all JFRs in the JMC for viewing. > Therefore, users want to be able to use the jfr tool to narrow down the jfr files they should see in the JMC. This allows users to see fewer jfr files using JMC. > I have used JFR on a cluster of hundreds of Java processes, but it was very hard to see all the jfr files to figure out the cause. Interesting use case. You want to search in a large set of files. Never thought of that. I will keep that in mind going forward. > > JMC had the ability to display the total duration, which was very useful. > In my experience, I never use default.jfc or profile.jfc as is. I think most people use the default configurations (default or profile).Very few would set the threshold to 0 ms for socket och synchronization events, The overhead would be too large. > All events indicating that threads cannot be used effectively should be logged without a threshold. I also limit the threads displayed in the JMC to those that are processed by the application. > In such a situation, looking at the total duration is very useful for analyzing what is happening in the application. The application may be monitoring by code hidden in the library, or it may be sleeping a thread. The purpose of the ?summary' command was not provide a text based version of JMC, but to show low level information that is hard to get otherwise. For example how many bytes does an event use in a file? How much of a chunk consists of metadata or checkpoint data? What is the start time of a chunk? How many chunks are there in a file? What version of the file format is being used? You can not get that kind low level information using the API. For a text based experience, I think it is better to create a new command that is much more powerful. Plan is to revisit this for a later JDK release. For your specific use case, meanwhile, I would just use the API: public static void main(String... args) throws IOException { Map m = new TreeMap<>(); for (var e : RecordingFile.readAllEvents(Path.of(args[0]))) { m.merge(e.getEventType().getName(), 0L, (a, b) -> b + e.getDuration().toNanos()); } for (var h : m.entrySet()) { System.out.println(h.getKey() + " " + h.getValue()); } } } > > I did not consider the 80 character limit. I think limiting the width is very useful for people to see. Instead of displaying nanosecond values, I will devise something. > I'm very excited about future enhancements. It would be great if we could provide common requests as standard functions. I imagine there would be a command to aggregate data similar to SQL query, but also predefined views that will show information for common scenarios, like ?hot-methods?, ?allocation-sites? or ?gc-info'. One such predefined view could be to show statistics per event type. Regards Erik > > Regards, > Chihiro > > > > On 2019/05/08 22:55, Erik Gahlin wrote: >> I don't see a benefit of adding a sum of duration column. >> >> Events have thresholds. For example, the jdk.JavaMonitorWait event is only recorded if the event exceeds 20 ms so the sum would not be the actual sum. Also, things often happen in multiple threads which makes it hard to know what the value actually means. >> >> The 'jfr' tool tries to not exceed 80 characters (to avoid wrapping), which is hard if another column is added. This especially true if user defined events are added with a longer namespace than "jdk." >> >> The longer term plan is to add a more advanced command where users can aggregate data on arbitrary field and not just the sum as an aggregate function. Users would then be able to aggregate duration per event type, if they really wanted it. >> >> Erik >> >>> Hi, >>> >>> Could you review this enhancement, please? >>> >>> This enhancement adds the total duration of each event to the jfr summary command to help the user determine which events to display in the jfr output. >>> >>> webrev: >>> http://cr.openjdk.java.net/~cito/JDK-8223498/webrev.00/ >>> >>> jbs: >>> https://bugs.openjdk.java.net/browse/JDK-8223498 >>> >>> I have passed the tests in test/jdk/jdk/jfr. >>> >>> Best regards, >>> Chihiro >> >> > From kdobson at redhat.com Fri May 10 16:58:00 2019 From: kdobson at redhat.com (Ken Dobson) Date: Fri, 10 May 2019 12:58:00 -0400 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: <2055f245-c722-c61b-2f85-406341cb6668@oracle.com> References: <5CCE0110.7070908@oracle.com> <2055f245-c722-c61b-2f85-406341cb6668@oracle.com> Message-ID: Thanks everyone for their reviews. I've added tests similar to the tests for the G1 events as well as addressed the changes that Erik recommended. Here is the new webrev: http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ Please let me know if there's anything else that should be addressed. Thanks, Ken Dobson On Mon, May 6, 2019 at 11:38 AM wrote: > > > On 5/4/19 2:16 PM, Erik Gahlin wrote: > > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: > >> Hi, > >> > >> If possible and feasible, could you please implement tests for new > >> events? > >> > >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > >> create a "shenandoah" subfolder. > >> > >> If the events are too hard to test or not feasible, please add the > >> event names to > >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to > >> the hardToTestEvents set. Consider commenting why it is too hard to > >> test. > > > > If events are are marked experimental, tests are ignored by > > TestLookForUntestedEvents. java, so we should not add them to the > > hardToTestEvents set. That may help them pass under the radar once the > > experimental attribute is removed. > Thank you. If the events are "experimental" this approach makes sense. > Once the experimental status is removed, the > TestLookForUntestedEvents.java should be able to notice if the event is > not covered by tests. > > That said, tests never hurts, even if they are experimental. I think > > test/jdk/jdk/jfr/event/gc/detailed would be a good place > +1 > > Thank you, > Misha > > > > Erik > > > >> > >> Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ > >> prior to integration. > >> > >> > >> Best regards, > >> > >> Misha > >> > >> > >> > >> On 5/3/19 7:44 AM, Ken Dobson wrote: > >>> Hi all, > >>> > >>> Please review this patch that adds support for two new JFR events > >>> ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. > >>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > >>> > >>> The Shenandoah team has also reviewed this patch and approved it > >>> from their > >>> end. > >>> > >>> Thanks, > >>> > >>> Ken Dobson > >> > > > > From erik.gahlin at oracle.com Mon May 13 00:27:49 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 13 May 2019 02:27:49 +0200 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: <5CCE0110.7070908@oracle.com> <2055f245-c722-c61b-2f85-406341cb6668@oracle.com> Message-ID: <5CD8BA05.5050809@oracle.com> Thanks for fixing! Product code looks good, but I think tests could be improved. There is no need to call dump in a final clause. Events.fromRecording(...) will dump the file into the jtreg scratch directory ( "recording-1-pid-xxx.jfr") by default. I think this is sufficient since there is only one recording per test. If you think the name is cryptic, consider adding another method to Event.fromRecording that takes an additional parameter that becomes a suffix, i.e "recording-1-pid-xxx-test-heap-region-info.jfr" try (Recording r = new recording()) { r.enable(EVENT_NAME); r.start(); r.stop() List events = Events.fromRecording(r); Events.hasEvents(events); for (RecordedEvent event : events) { Events.assertField(event, "index").notEqual(-1); Events.assertField(event, "used").atMost(1024*1024L); String state = Events.assertField(event, "state").getValue(); GCHelper.assertShenandoahHeapRegionState(state)); } The name of the field in metadata.xml is "state", but the tests looks for "type". That seems incorrect. GCHelper: public static assertShenandoahHeapRegionState(String state) { if (!shenandoahHeapRegionStates.contains(state)) { throw new AssertionError("Unknown state '"+ state +"', valid heap region states are " + shenandoahHeapRegionStates); } I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") because you want to backport to JDK 8. Thanks Erik > Thanks everyone for their reviews. I've added tests similar to the > tests for the G1 events as well as addressed the changes that Erik > recommended. > > Here is the new webrev: > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ > > > Please let me know if there's anything else that should be addressed. > > Thanks, > > Ken Dobson > > On Mon, May 6, 2019 at 11:38 AM > wrote: > > > > On 5/4/19 2:16 PM, Erik Gahlin wrote: > > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com > wrote: > >> Hi, > >> > >> If possible and feasible, could you please implement tests > for new > >> events? > >> > >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > >> create a "shenandoah" subfolder. > >> > >> If the events are too hard to test or not feasible, please add the > >> event names to > >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to > >> the hardToTestEvents set. Consider commenting why it is too > hard to > >> test. > > > > If events are are marked experimental, tests are ignored by > > TestLookForUntestedEvents. java, so we should not add them to the > > hardToTestEvents set. That may help them pass under the radar > once the > > experimental attribute is removed. > Thank you. If the events are "experimental" this approach makes > sense. > Once the experimental status is removed, the > TestLookForUntestedEvents.java should be able to notice if the > event is > not covered by tests. > > That said, tests never hurts, even if they are experimental. I > think > > test/jdk/jdk/jfr/event/gc/detailed would be a good place > +1 > > Thank you, > Misha > > > > Erik > > > >> > >> Also, please make sure to run all jfr tests under > test/jdk/jdk/jfr/ > >> prior to integration. > >> > >> > >> Best regards, > >> > >> Misha > >> > >> > >> > >> On 5/3/19 7:44 AM, Ken Dobson wrote: > >>> Hi all, > >>> > >>> Please review this patch that adds support for two new JFR events > >>> ShenandoahHeapRegionStateChange and > ShenandoahHeapRegionInformation. > >>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > >>> > >>> The Shenandoah team has also reviewed this patch and approved it > >>> from their > >>> end. > >>> > >>> Thanks, > >>> > >>> Ken Dobson > >> > > > From yasuenag at gmail.com Mon May 13 10:21:07 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 13 May 2019 19:21:07 +0900 Subject: PING: RFR: 8220657: JFR.dump does not work when filename is set In-Reply-To: References: <5C90E8C9.3010008@oracle.com> <14c04f3a-1fc0-af3b-714f-e35fa18d6a1b@gmail.com> <5C9103DE.4040304@oracle.com> <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> <2dc4e611-bd18-a187-f529-d243af51cf10@gmail.com> <5CBE36B6.6070603@oracle.com> <6348d8ac-b713-1cb3-39db-859bc833e0ce@gmail.com> <5CCB0188.7020204@oracle.com> Message-ID: <3ce13ffd-f795-78ef-ddf6-d56651303341@gmail.com> PING: Could you review this webrev? > http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.03/ Yasumasa On 2019/05/03 23:05, Yasumasa Suenaga wrote: > Hi Erik, > > I uploaded new webrev: > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.03/ > > This webrev does not use `var` in any place. > And I tweaked testcase based on your idea. > > > Thanks, > > Yasumasa > > > On 2019/05/02 23:41, Erik Gahlin wrote: >> Hi Yasumasa, >> >> The test looks complicated. >> >> I would prefer something more straight forward with less states and >> nulls where the expected behavior is easy to see. If the test fails, >> it's also nice to see from the stack trace which sub test failed. >> >> ?? public static void main(String... args) throws Exception { >> ???? testDumpAll(); >> ???? testDumpNamed(); >> ???? testDumpNamedWithFilename(); >> ?? } >> >> ?? private static void testDumpAll() throws Exception { >> ???? Path p = Path.of("A.jfr").toAbsolutePath(); >> ???? try (Recording r = new Recording()) { >> ?????? r.setName("A"); >> ?????? r.setDestination(p); >> ?????? JcmdHelper.jcmd("JFR.dump"); >> ?????? Asserts.assertFalse(namedFile(p), "Unexpected file: " + p); >> ?????? Asserts.assertTrue(generatedFile(), "Expected generated file"); >> ???? } >> ?? } >> >> ?? private static void testDumpNamed() throws Exception { >> ???? Path p = Path.of("B.jfr").toAbsolutePath(); >> ???? try (Recording r = new Recording()) { >> ?????? r.setName("B"); >> ?????? r.setDestination(p); >> ?????? JcmdHelper.jcmd("JFR.dump", "name=B"); >> ?????? Asserts.assertTrue(namedFile(p), "Expected file: " + p); >> ?????? Asserts.assertFalse(generatedFile(), "Unexpected generated file"); >> ???? } >> ?? } >> >> ?? private static void testDumpNamedWithFilename() throws Exception { >> ???? Path p = Path.of("C.jfr").toAbsolutePath(); >> ???? Path override = Path.of("override.jfr").toAbsolutePath(); >> ???? try (Recording r = new Recording()) { >> ?????? r.setName("C"); >> ?????? r.setDestination(p); >> ?????? JcmdHelper.jcmd("JFR.dump", "name=C", "filename=" + override); >> ?????? Asserts.assertFalse(namedFile(p), "Unexpected file: " + p); >> ?????? Asserts.assertTrue(namedFile(p), "Expected file: " + override); >> ?????? Asserts.assertFalse(generatedFile(), "Unexpected generated file"); >> ???? } >> ?? } >> >> ?? private static boolean namedFile(Path dumpFile) throws IOException { >> ???? return Files.exists(dumpFile) && Files.size(dumpFile) > 0; >> ?? } >> >> ?? private static boolean generatedFile() throws IOException { >> ???? long pid = ProcessHandle.current().pid(); >> ???? return Files.find(Path.of("."), 1, (p, a) -> p.toString() >> ???????????????? .matches("^.*hotspot-pid-" + pid + ".jfr$") && >> a.size() ?> 0L) >> ???????????????? .findFirst().isPresent(); >> ?? } >> >> I haven't tried it. >> >> When it comes to DCmdDump, I would prefer if you would not use "var" >> as this hides the underlying type (SafePath or WriteableUserPath) >> which is used to signal if there is a security risk or not. I think it >> is fine, but if the code gets refactor at a later stage the >> information could get lost. >> >> Thanks >> Erik >> >>> PING: Could you review it? >>> >>> > http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ >>> >>> >>> Yasumasa >>> >>> >>> On 2019/04/24 21:43, Yasumasa Suenaga wrote: >>>> Hi Erik, >>>> >>>> I uploaded new webrev: >>>> >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ >>>> >>>> I kept to remove setDestination() call in PlatformRecording. >>>> In this webrev, original filename value will be set to temporary >>>> snapshot if DCmdDump does not have recording name and filename. >>>> >>>> Also I added new testcase. It can test JFR.dump with filename. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2019/04/23 6:48, Erik Gahlin wrote: >>>>> I have looked further at the issue and while the message to the >>>>> user was incorrect previously ( "Dump failed. No data found in the >>>>> specified interval") it did result in a file with the specified >>>>> filename, i.e "test.jfr" >>>>> >>>>> When "clone.setDestination(this.destination);" is removed, the >>>>> filename passed on command line is ignored, but it does result in a >>>>> message that is consistent with what is happening (a file with an >>>>> autogenerated name is created). It does however change the behavior >>>>> of 'JFR.dump name=1'. Logic needs to be added to DCmdDump so >>>>> it uses an existing filename if it is set, but changes it if a user >>>>> is providing an override. >>>>> >>>>> When comes to testing. I think it would be better to write a >>>>> separate test that checks that things are dumped correctly >>>>> when-XX:StartFlightRecording:filename= is used. >>>>> >>>>> Thanks >>>>> Erik >>>>> >>>>>> PING: Do you have any opinion(s) about this fix? >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2019/04/01 21:46, Yasumasa Suenaga wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Do you have concern(s) about this change? >>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>>>> >>>>>>> As in the email below, I think it does not affect scenarios >>>>>>> except snapshot cloning. >>>>>>> Please tell me if you have any opinions about this. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2019/03/20 10:37, Yasumasa Suenaga wrote: >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> I think this change does not affect other scenarios. >>>>>>>> >>>>>>>> :jdk_jfr jtreg tests and submit repo have passed with this change. >>>>>>>> newSnapshotClone() is called by PlatformRecording::dump and >>>>>>>> DCmdDump::newSnapshot. >>>>>>>> Both methods are related to user-requested dump. >>>>>>>> >>>>>>>> What scenarios do you think? I want to check them. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> 2019?3?19?(?) 23:59 Erik Gahlin : >>>>>>>>> >>>>>>>>> Yes, but then you would lose the original destination in other >>>>>>>>> scenarios >>>>>>>>> >>>>>>>>> Maybe I can look at this next week. >>>>>>>>> >>>>>>>>> Erik >>>>>>>>> >>>>>>>>>> Hi Erik, >>>>>>>>>> >>>>>>>>>> Thank you for your reply. >>>>>>>>>> >>>>>>>>>> How about this changeset? >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>>>>>>> >>>>>>>>>> I think PlatformRecording#newSnapshotClone() should not dump. >>>>>>>>>> So I removed `setDestination()` from this. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2019/03/19 22:04, Erik Gahlin wrote: >>>>>>>>>>> This looks a bit hackish. >>>>>>>>>>> >>>>>>>>>>> The real issue seems to be that the method >>>>>>>>>>> PlatformRecording#newSnapshotClone incorrectly sets the >>>>>>>>>>> destination in >>>>>>>>>>> the clone, which then triggers a dump prematurely (when stop >>>>>>>>>>> is called). >>>>>>>>>>> >>>>>>>>>>> Purpose of that method is just to take a snapshot of existing >>>>>>>>>>> chunks in >>>>>>>>>>> the disk repository (or from memory). and then let >>>>>>>>>>> dumpStopped(path) do >>>>>>>>>>> the actual dump (copy out the data from the disk repository). >>>>>>>>>>> >>>>>>>>>>> To make it work, some other changes would be needed as well, >>>>>>>>>>> but I don't >>>>>>>>>>> have time to investigate. >>>>>>>>>>> >>>>>>>>>>> Erik >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Please review this change. >>>>>>>>>>>> >>>>>>>>>>>> ???? JBS: https://bugs.openjdk.java.net/browse/JDK-8220657 >>>>>>>>>>>> ???? webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> The user might not get flight record via JFR.dump if JFR is >>>>>>>>>>>> started >>>>>>>>>>>> with filename option on target process. >>>>>>>>>>>> >>>>>>>>>>>> Please see JBS if you want to know how to reproduce. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>> >>>>> >> From chihiro.ito at oracle.com Tue May 14 18:47:36 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Wed, 15 May 2019 03:47:36 +0900 Subject: RFR: 8223066: "jfr metadata" output the @Name annotation twice. In-Reply-To: <82495479-558b-f2da-8ae8-108d07de3e4b@oracle.com> References: <398c166c-b681-6d32-4f4c-9d43350b622f@oracle.com> <5CC9F026.4050002@oracle.com> <82495479-558b-f2da-8ae8-108d07de3e4b@oracle.com> Message-ID: <72b33e01-cd8e-f44c-727e-a40c30e2b874@oracle.com> Hi Erik, I removed @Name from the annotations. Could you review it? webrev: http://cr.openjdk.java.net/~cito/JDK-8223066/webrev.01/ JBS: https://bugs.openjdk.java.net/browse/JDK-8223066 The output look like this.: @Name("jdk.ActiveSetting") @Label("Recording Setting") @Category("Flight Recorder") class ActiveSetting extends jdk.jfr.Event { Regards, Chihiro On 2019/05/04 2:48, Chihiro Ito wrote: > Hi Erik, > > Thank you for sharing the background. > > In the former case, I feel that PrettyWriter strongly depends on the > name and annotation information of Type. > So I think the latter is better. > > Regard > Chihiro > > On 2019/05/02 4:14, Erik Gahlin wrote: >> Hi Chihiro, >> >> Thanks for reporting the issue, @Name should not be printed twice. >> >> Let me give you some background information. I think there are two >> issues at hand: >> >> 1) The Name annotation should probably not be persisted. This can be >> achieved by removing the MetadataDefinition from the class. I did try >> that, but I noticed several tests failing, in particular TestName >> which explicitly checks that @Name is persisted. The purpose @Name is >> to override the default name, which can then be retrieved from >> ValueDescriptor::getName, EventType::getName and >> SettingDescriptor::getName. The annotation is not needed on the >> consumer side as the name is already persisted elsewhere. >> >> I remember discussing this several times during the design, but for >> some reason we came to the conclusion to persist it, probably to not >> confuse users if it didn't behave like @Label and @Description. >> Later, Metadatadefinition was removed from @Threshold, @Periodic, >> @Enabled and @StackTrace. At that time, it should probably have been >> removed from @Name as well. Maybe we should go ahead and remove it >> now, but I will have to think about the implications first. >> >> 2) The reason PrettyWriter::printType adds @Name is because there is >> no good way to include package/namespace in the printout. Events that >> lack a namespace should not get the annotation, that's why there is a >> check for "." It's not because the values are primitives. >> >> I can see two ways to proceed. Either add a check in printType so >> @Name is only printed if it doesn't already exists, i.e. >> >> if (t.getAnnotation(Name.class) == null) { >> if (index != -1) { >> println("@Name(\"" + typeName + "\")"); >> } >> } >> >> or remove @MetadataDefinition from the Name class and make >> adjustments to tests etc. If you chose the latter, I like a week to >> think it over, investigate the implications and discuss it with other >> people, before I can say if it is the right approach or not. >> >> Thanks >> Erik >> >> Hi, >>> >>> I fixed an issue that the @Name annotation was printed twice as >>> followings in "jfr metadata" output. >>> >>> @Name("jdk.ActiveSetting") >>> @Name("jdk.ActiveSetting") >>> @Label("Recording Setting") >>> @Category("Flight Recorder") >>> class ActiveSetting extends jdk.jfr.Event { >>> >>> >>> Could you reveiew this please? >>> >>> webrev: >>> http://cr.openjdk.java.net/~cito/JDK-8223066/webrev.00/ >>> >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8223066 >>> >>> I have passed all the tests in test-tier1. >>> >>> Best regards, >>> Chihiro >> >> > > From kdobson at redhat.com Tue May 14 21:19:19 2019 From: kdobson at redhat.com (Ken Dobson) Date: Tue, 14 May 2019 17:19:19 -0400 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: <5CD8BA05.5050809@oracle.com> References: <5CCE0110.7070908@oracle.com> <2055f245-c722-c61b-2f85-406341cb6668@oracle.com> <5CD8BA05.5050809@oracle.com> Message-ID: Thank you for the review again. Yes, not sure regarding backporting plans but I imagine it will be in the future so seemed best to avoid anything that cause issues such as Set.of(). I also think the naming of the jfr's is fine as it's stored in a folder named after the test which was easily found now that I'm aware that the jfrs are already dumped. I believe I've addressed the rest of your recommended changes to the tests, let me know if I've covered everything. http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ Thanks, Ken Dobson On Sun, May 12, 2019 at 8:28 PM Erik Gahlin wrote: > Thanks for fixing! > > Product code looks good, but I think tests could be improved. > > There is no need to call dump in a final clause. > > Events.fromRecording(...) will dump the file into the jtreg scratch > directory ( "recording-1-pid-xxx.jfr") by default. I think this is > sufficient since there is only one recording per test. If you think the > name is cryptic, consider adding another method to Event.fromRecording that > takes an additional parameter that becomes a suffix, i.e > "recording-1-pid-xxx-test-heap-region-info.jfr" > > try (Recording r = new recording()) { > r.enable(EVENT_NAME); > r.start(); > r.stop() > List events = Events.fromRecording(r); > Events.hasEvents(events); > for (RecordedEvent event : events) { > Events.assertField(event, "index").notEqual(-1); > Events.assertField(event, "used").atMost(1024*1024L); > String state = Events.assertField(event, "state").getValue(); > GCHelper.assertShenandoahHeapRegionState(state)); > } > > The name of the field in metadata.xml is "state", but the tests looks for > "type". That seems incorrect. > > GCHelper: > > public static assertShenandoahHeapRegionState(String state) { > if (!shenandoahHeapRegionStates.contains(state)) { > throw new AssertionError("Unknown state '"+ state +"', valid heap > region states are " + shenandoahHeapRegionStates); > } > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") > because you want to backport to JDK 8. > > Thanks > Erik > > Thanks everyone for their reviews. I've added tests similar to the tests > for the G1 events as well as addressed the changes that Erik recommended. > > Here is the new webrev: > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ > > Please let me know if there's anything else that should be addressed. > > Thanks, > > Ken Dobson > > On Mon, May 6, 2019 at 11:38 AM wrote: > >> >> >> On 5/4/19 2:16 PM, Erik Gahlin wrote: >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: >> >> Hi, >> >> >> >> If possible and feasible, could you please implement tests for new >> >> events? >> >> >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps >> >> create a "shenandoah" subfolder. >> >> >> >> If the events are too hard to test or not feasible, please add the >> >> event names to >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to >> >> the hardToTestEvents set. Consider commenting why it is too hard to >> >> test. >> > >> > If events are are marked experimental, tests are ignored by >> > TestLookForUntestedEvents. java, so we should not add them to the >> > hardToTestEvents set. That may help them pass under the radar once the >> > experimental attribute is removed. >> Thank you. If the events are "experimental" this approach makes sense. >> Once the experimental status is removed, the >> TestLookForUntestedEvents.java should be able to notice if the event is >> not covered by tests. >> > That said, tests never hurts, even if they are experimental. I think >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place >> +1 >> >> Thank you, >> Misha >> > >> > Erik >> > >> >> >> >> Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ >> >> prior to integration. >> >> >> >> >> >> Best regards, >> >> >> >> Misha >> >> >> >> >> >> >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: >> >>> Hi all, >> >>> >> >>> Please review this patch that adds support for two new JFR events >> >>> ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. >> >>> >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ >> >>> >> >>> The Shenandoah team has also reviewed this patch and approved it >> >>> from their >> >>> end. >> >>> >> >>> Thanks, >> >>> >> >>> Ken Dobson >> >> >> > >> >> > From erik.gahlin at oracle.com Tue May 14 21:37:44 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 14 May 2019 23:37:44 +0200 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: <5CCE0110.7070908@oracle.com> <2055f245-c722-c61b-2f85-406341cb6668@oracle.com> <5CD8BA05.5050809@oracle.com> Message-ID: <5CDB3528.1050200@oracle.com> Looks good. In the tests, you could avoid the null check on the recording and just use try-with-resources. It will close the recording automatically when it runs out of scope. No need to post updated webrev. Thanks Erik > Thank you for the review again. > > Yes, not sure regarding backporting plans but I imagine it will be in > the future so seemed best to avoid anything that cause issues such as > Set.of(). > > I also think the naming of the jfr's is fine as it's stored in a > folder named after the test which was easily found now that I'm aware > that the jfrs are already dumped. > > I believe I've addressed the rest of your recommended changes to the > tests, let me know if I've covered everything. > > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ > > > Thanks, > > Ken Dobson > > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin > wrote: > > Thanks for fixing! > > Product code looks good, but I think tests could be improved. > > There is no need to call dump in a final clause. > > Events.fromRecording(...) will dump the file into the jtreg > scratch directory ( "recording-1-pid-xxx.jfr") by default. I > think this is sufficient since there is only one recording per > test. If you think the name is cryptic, consider adding another > method to Event.fromRecording that takes an additional parameter > that becomes a suffix, i.e > "recording-1-pid-xxx-test-heap-region-info.jfr" > > try (Recording r = new recording()) { > r.enable(EVENT_NAME); > r.start(); > r.stop() > List events = Events.fromRecording(r); > Events.hasEvents(events); > for (RecordedEvent event : events) { > Events.assertField(event, "index").notEqual(-1); > Events.assertField(event, "used").atMost(1024*1024L); > String state = Events.assertField(event, "state").getValue(); > GCHelper.assertShenandoahHeapRegionState(state)); > } > > The name of the field in metadata.xml is "state", but the tests > looks for "type". That seems incorrect. > > GCHelper: > > public static assertShenandoahHeapRegionState(String state) { > if (!shenandoahHeapRegionStates.contains(state)) { > throw new AssertionError("Unknown state '"+ state +"', valid > heap region states are " + shenandoahHeapRegionStates); > } > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") > because you want to backport to JDK 8. > > Thanks > Erik > >> Thanks everyone for their reviews. I've added tests similar to >> the tests for the G1 events as well as addressed the changes that >> Erik recommended. >> >> Here is the new webrev: >> http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ >> >> >> Please let me know if there's anything else that should be addressed. >> >> Thanks, >> >> Ken Dobson >> >> On Mon, May 6, 2019 at 11:38 AM > > wrote: >> >> >> >> On 5/4/19 2:16 PM, Erik Gahlin wrote: >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com >> wrote: >> >> Hi, >> >> >> >> If possible and feasible, could you please implement >> tests for new >> >> events? >> >> >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. >> Perhaps >> >> create a "shenandoah" subfolder. >> >> >> >> If the events are too hard to test or not feasible, please >> add the >> >> event names to >> >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, >> to >> >> the hardToTestEvents set. Consider commenting why it is >> too hard to >> >> test. >> > >> > If events are are marked experimental, tests are ignored by >> > TestLookForUntestedEvents. java, so we should not add them >> to the >> > hardToTestEvents set. That may help them pass under the >> radar once the >> > experimental attribute is removed. >> Thank you. If the events are "experimental" this approach >> makes sense. >> Once the experimental status is removed, the >> TestLookForUntestedEvents.java should be able to notice if >> the event is >> not covered by tests. >> > That said, tests never hurts, even if they are >> experimental. I think >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place >> +1 >> >> Thank you, >> Misha >> > >> > Erik >> > >> >> >> >> Also, please make sure to run all jfr tests under >> test/jdk/jdk/jfr/ >> >> prior to integration. >> >> >> >> >> >> Best regards, >> >> >> >> Misha >> >> >> >> >> >> >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: >> >>> Hi all, >> >>> >> >>> Please review this patch that adds support for two new >> JFR events >> >>> ShenandoahHeapRegionStateChange and >> ShenandoahHeapRegionInformation. >> >>> >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ >> >> >>> >> >>> The Shenandoah team has also reviewed this patch and >> approved it >> >>> from their >> >>> end. >> >>> >> >>> Thanks, >> >>> >> >>> Ken Dobson >> >> >> > >> > From erik.gahlin at oracle.com Tue May 14 21:50:32 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 14 May 2019 23:50:32 +0200 Subject: RFR: 8223066: "jfr metadata" output the @Name annotation twice. In-Reply-To: <72b33e01-cd8e-f44c-727e-a40c30e2b874@oracle.com> References: <398c166c-b681-6d32-4f4c-9d43350b622f@oracle.com> <5CC9F026.4050002@oracle.com> <82495479-558b-f2da-8ae8-108d07de3e4b@oracle.com> <72b33e01-cd8e-f44c-727e-a40c30e2b874@oracle.com> Message-ID: <5CDB3828.20406@oracle.com> Hi Chihiro , I haven't had time to discuss alternative 1, if that is what you want to proceed with. If you want alternative 2, I don't think changes to the "low-level" implementation (MetadataReader) is the correct way to go about it. It will impact other code, not just the 'jfr metadata' command. Erik > Hi Erik, > > I removed @Name from the annotations. > Could you review it? > > webrev: > http://cr.openjdk.java.net/~cito/JDK-8223066/webrev.01/ > > > JBS: > https://bugs.openjdk.java.net/browse/JDK-8223066 > > > The output look like this.: > > @Name("jdk.ActiveSetting") > @Label("Recording Setting") > @Category("Flight Recorder") > class ActiveSetting extends jdk.jfr.Event { > > > Regards, > Chihiro > > On 2019/05/04 2:48, Chihiro Ito wrote: >> Hi Erik, >> >> Thank you for sharing the background. >> >> In the former case, I feel that PrettyWriter strongly depends on the >> name and annotation information of Type. >> So I think the latter is better. >> >> Regard >> Chihiro >> >> On 2019/05/02 4:14, Erik Gahlin wrote: >>> Hi Chihiro, >>> >>> Thanks for reporting the issue, @Name should not be printed twice. >>> >>> Let me give you some background information. I think there are two >>> issues at hand: >>> >>> 1) The Name annotation should probably not be persisted. This can be >>> achieved by removing the MetadataDefinition from the class. I did >>> try that, but I noticed several tests failing, in particular >>> TestName which explicitly checks that @Name is persisted. The >>> purpose @Name is to override the default name, which can then be >>> retrieved from ValueDescriptor::getName, EventType::getName and >>> SettingDescriptor::getName. The annotation is not needed on the >>> consumer side as the name is already persisted elsewhere. >>> >>> I remember discussing this several times during the design, but for >>> some reason we came to the conclusion to persist it, probably to not >>> confuse users if it didn't behave like @Label and @Description. >>> Later, Metadatadefinition was removed from @Threshold, @Periodic, >>> @Enabled and @StackTrace. At that time, it should probably have been >>> removed from @Name as well. Maybe we should go ahead and remove it >>> now, but I will have to think about the implications first. >>> >>> 2) The reason PrettyWriter::printType adds @Name is because there is >>> no good way to include package/namespace in the printout. Events >>> that lack a namespace should not get the annotation, that's why >>> there is a check for "." It's not because the values are primitives. >>> >>> I can see two ways to proceed. Either add a check in printType so >>> @Name is only printed if it doesn't already exists, i.e. >>> >>> if (t.getAnnotation(Name.class) == null) { >>> if (index != -1) { >>> println("@Name(\"" + typeName + "\")"); >>> } >>> } >>> >>> or remove @MetadataDefinition from the Name class and make >>> adjustments to tests etc. If you chose the latter, I like a week to >>> think it over, investigate the implications and discuss it with >>> other people, before I can say if it is the right approach or not. >>> >>> Thanks >>> Erik >>> >>> Hi, >>>> >>>> I fixed an issue that the @Name annotation was printed twice as >>>> followings in "jfr metadata" output. >>>> >>>> @Name("jdk.ActiveSetting") >>>> @Name("jdk.ActiveSetting") >>>> @Label("Recording Setting") >>>> @Category("Flight Recorder") >>>> class ActiveSetting extends jdk.jfr.Event { >>>> >>>> >>>> Could you reveiew this please? >>>> >>>> webrev: >>>> http://cr.openjdk.java.net/~cito/JDK-8223066/webrev.00/ >>>> >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8223066 >>>> >>>> I have passed all the tests in test-tier1. >>>> >>>> Best regards, >>>> Chihiro >>> >>> >> >> > From erik.gahlin at oracle.com Fri May 17 16:27:00 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Fri, 17 May 2019 18:27:00 +0200 Subject: JEP 349: JFR Event Streaming (sandbox preview) Message-ID: <5CDEE0D4.4070502@oracle.com> Hi folks, We have created a sandbox for JEP 349: JFR Event Streaming[1] for people who like to try the feature out. It is work in progress, but plan is to integrate it in the next couple of months, but not for JDK 13. A build can be created like this: $ hg clone -b JEP-349-branch -r 53dccc90a5be http://hg.openjdk.java.net/jdk/sandbox jep-349 $ cd jep-349 $ bash configure $ make images $ ./build/*/images/jdk/bin/java -version See [2] and [3] for more information on how to use the sandbox repository and build the JDK. For testing purposes, and to illustrate how the Event Streaming API can be used, there exists a Java agent[4] that aggregates event data and print the results on standard out. Example, http://cr.openjdk.java.net/~egahlin/health-report-agent/health-report-example.txt =================== HEALTH REPORT === 2019-05-16 23:57:50 ==================== | GC: G1Old/G1New Phys. memory: 28669 MB Alloc Rate: 8 MB/s | | OC Count : 28 Initial Heap: 448 MB Total Alloc: 190 MB | | OC Pause Avg: 40.1 ms Used Heap : 19 MB Thread Count: 20.0 | | OC Pause Max: 48.8 ms Commit. Heap: 47 MB Class Count : 3894.0 | | YC Count : 8 CPU Machine : 20.12 % Safepoints: 335 | | YC Pause Avg: 5.7 ms CPU JVM User : 10.28 % Max Safepoint: 46.4 ms | | YC Pause Max: 22.4 ms CPU JVM System: 1.07 % Max Comp. Time: 728.3 ms | |--- Top Allocation Methods ------------------------------- -----------------| | DataBufferInt.(int) 11.27 % | | Component.size() 9.01 % | | BufferedContext.validate(...) 6.21 % | | Path2D$Double.(...) 5.87 % | | SunGraphics2D.clone() 5.85 % | |--- Hot Methods ------------------------------------------------------------| | DRenderer._endRendering(int, int) 51.11 % | | DRenderer.copyAARow(...) 6.67 % | | Arrays.fill(...) 4.44 % | | StringConcatFactory.doStringConcat(...) 2.22 % | | MarlinTileGenerator.getAlphaNoRLE(...) 2.22 % | ============================================================================== The agent can be started like this: $ java -javaagent:health-report.jar MyApp The interval at which events are flushed can be changed with the interval option. The default interval is 1 second. For example, to flush every ten second, use -javaagent:health-report.jar=interval=10 For quick experimentation, the agent can also be run as a Java program that allocate objects in a loop. $ java Main.java Two new classes, EventStream[5] and RecordingStream[6], have been added to the jdk.jfr.consumer package. A EventFilter class may be added later for more advanced filtering. The JVM implementation is close to finished, but there is work on the consumer side left. There is also an effort to reduce allocation on the client. To get started without using the agent, the CPU load can be printed like this: Configuration c = Configuration.getConfiguration("default"); try (RecordingStream r = new RecordingStream(c)) { r.onEvent("jdk.CPULoad", event -> { float percentage = 100 * event.getFloat("machineTotal"); System.out.printf("CPU: %.2f %% \n\n", percentage); }); r.start(); } Feedback is very much welcomed! Thanks Erik and Markus Outages, known issues and limitations ------------------------------------- - Only start/open one RecordingStream at a time Implementation has not been tested with two RecordingStreams simultaneously. - Exception handling Behavior is undefined if an exception is thrown in a handler. - Parser level filtering disabled Purpose of parser level filtering is to be able to use predefined configurations, for example default.jfc, but only pay the overhead in the parser of the events that the stream is listening to. Now all events are parsed. Workaround is to only enable the events of interest. - Chunk files not removed Chunk files may be left behind in the disk repository when the JVM exits. - EventStream::openFile(...) Not possible to open a file as an event stream. - EventStream:::openRepository(...) Not possible to open a (remote) directory as an event stream. - EventStream.setReuse(...) (cursor mode) Method has not been added / implemented. - EventStream.setParallel(...) Method has not been added / implemented as it will make most sense if EventStream::openFile exists. - EventStream.setOrdered(...) Method has not been added. Need to figure out how to handle events that arrives way out of order. For example, MyEvent event = new MyEvent(); event.begin(); event.end(); // end timestamp taken here Thread.sleep(1000_000); event.commit(); Currently a best effort is made to deliver events in order. [1] https://openjdk.java.net/jeps/349 [2] http://cr.openjdk.java.net/~chegar/docs/sandbox.html [3] http://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/building.html [4] http://cr.openjdk.java.net/~egahlin/health-report-agent/ [5] http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/RecordingStream.html [6] http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/EventStream.html From markus.gronlund at oracle.com Sun May 19 13:00:36 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Sun, 19 May 2019 13:00:36 +0000 (UTC) Subject: RFR(M): 8220293: Deadlock in JFR string pool In-Reply-To: <5ede3dbd-1f17-489c-abf6-504d1eece7d8@default> References: <5ede3dbd-1f17-489c-abf6-504d1eece7d8@default> Message-ID: Hi again, I would appreciate help in reviewing this. Thanks again Markus -----Original Message----- From: Markus Gronlund Sent: den 3 maj 2019 15:01 To: Hotspot dev runtime ; hotspot-jfr-dev at openjdk.java.net Subject: RFR(M): 8220293: Deadlock in JFR string pool Greeting, Please review the following patch to address the following: Bug: https://bugs.openjdk.java.net/browse/JDK-8220293 Webrev: http://cr.openjdk.java.net/~mgronlun/8220293/webrev01/ High level summary: We don't want to release a retired buffer, because this will break the invariant that a successful acquisition implies a valid (non-retired) buffer. This is the reason why this work is, at least prima facie, a little more involved. Tests: test case / reproducer attached to: https://bugs.openjdk.java.net/browse/JDK-8220293 open/test/jdk/:jdk_jfr Thank you Markus From erik.gahlin at oracle.com Mon May 20 14:03:10 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 20 May 2019 16:03:10 +0200 Subject: RFR: 8224217: RecordingInfo should use textual representation of path Message-ID: <5CE2B39E.8000108@oracle.com> Hi, Could I please have a review this fix to RecordingInfo. Webrev: http://cr.openjdk.java.net/~egahlin/8224217/ Bug: https://bugs.openjdk.java.net/browse/JDK-8224217 Testing: test/jdk/jdk/jfr/* Thanks Erik From erik.gahlin at oracle.com Mon May 20 14:30:27 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 20 May 2019 16:30:27 +0200 Subject: RFR: 8219205: JFR file without license header Message-ID: <5CE2BA03.50901@oracle.com> Hi, Could I please have a review this fix to MirrorEvent. Webrev: http://cr.openjdk.java.net/~egahlin/8219205_0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8219205 Thanks Erik From sean.coffey at oracle.com Mon May 20 14:42:58 2019 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 20 May 2019 15:42:58 +0100 Subject: RFR: 8219205: JFR file without license header In-Reply-To: <5CE2BA03.50901@oracle.com> References: <5CE2BA03.50901@oracle.com> Message-ID: Looks fine. Regards, Sean. On 20/05/19 15:30, Erik Gahlin wrote: > Hi, > > Could I please have a review this fix to MirrorEvent. > > Webrev: > http://cr.openjdk.java.net/~egahlin/8219205_0/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8219205 > > Thanks > Erik From markus.gronlund at oracle.com Mon May 20 15:08:56 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Mon, 20 May 2019 15:08:56 +0000 (UTC) Subject: RFR: 8219205: JFR file without license header In-Reply-To: <5CE2BA03.50901@oracle.com> References: <5CE2BA03.50901@oracle.com> Message-ID: Looks good. Markus -----Original Message----- From: Erik Gahlin Sent: den 20 maj 2019 16:30 To: hotspot-jfr-dev at openjdk.java.net Subject: RFR: 8219205: JFR file without license header Hi, Could I please have a review this fix to MirrorEvent. Webrev: http://cr.openjdk.java.net/~egahlin/8219205_0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8219205 Thanks Erik From markus.gronlund at oracle.com Mon May 20 15:58:26 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Mon, 20 May 2019 15:58:26 +0000 (UTC) Subject: RFR: 8224217: RecordingInfo should use textual representation of path In-Reply-To: <5CE2B39E.8000108@oracle.com> References: <5CE2B39E.8000108@oracle.com> Message-ID: <90fdc12d-7e99-436b-871c-be24dd79afd3@default> Looks good. Thanks Markus -----Original Message----- From: Erik Gahlin Sent: den 20 maj 2019 16:03 To: hotspot-jfr-dev at openjdk.java.net Subject: RFR: 8224217: RecordingInfo should use textual representation of path Hi, Could I please have a review this fix to RecordingInfo. Webrev: http://cr.openjdk.java.net/~egahlin/8224217/ Bug: https://bugs.openjdk.java.net/browse/JDK-8224217 Testing: test/jdk/jdk/jfr/* Thanks Erik From chihiro.ito at oracle.com Mon May 20 17:41:53 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Tue, 21 May 2019 02:41:53 +0900 Subject: JEP 349: JFR Event Streaming (sandbox preview) In-Reply-To: <5CDEE0D4.4070502@oracle.com> References: <5CDEE0D4.4070502@oracle.com> Message-ID: <10a1e44a-6596-f375-b6a4-819b20380a25@oracle.com> Hi Erik, I tried JFR Streaming with the following code [1], but occasionally an exception occurred [2]. This is because the repository has not yet been created when JFR Streaming reads the JFR file. I made a patch as attached. How do I handle bug reports? Should I raise the issue as usual? [1] public static void premain(String args, Instrumentation inst) { RecordingStream r = new RecordingStream(); r.startAsync(); Runtime.getRuntime().addShutdownHook(new Thread() { public void run() { r.close(); } }); } public static void main(String... args) throws Exception { TimeUnit.SECONDS.sleep(2); System.out.println("Completed"); } [2] java.nio.file.NoSuchFileException: /tmp/2019_05_20_00_30_44_445 at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:432) at java.base/java.nio.file.Files.newDirectoryStream(Files.java:542) at jdk.jfr/jdk.jfr.consumer.EventSetLocation.updateEventSets(EventSetLocation.java:129) at jdk.jfr/jdk.jfr.consumer.EventSetLocation.acquire(EventSetLocation.java:108) at jdk.jfr/jdk.jfr.consumer.EventSet.next(EventSet.java:210) at jdk.jfr/jdk.jfr.consumer.EventRunner.process(EventRunner.java:132) at jdk.jfr/jdk.jfr.consumer.EventRunner.execute(EventRunner.java:105) at jdk.jfr/jdk.jfr.consumer.EventRunner.lambda$run$0(EventRunner.java:89) at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:96) at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:93) at java.base/java.security.AccessController.doPrivileged(AccessController.java:391) at jdk.jfr/jdk.jfr.consumer.EventRunner.doPriviliged(EventRunner.java:93) at jdk.jfr/jdk.jfr.consumer.EventRunner.run(EventRunner.java:89) at java.base/java.lang.Thread.run(Thread.java:830) Regards, Chihiro On 2019/05/18 1:27, Erik Gahlin wrote: > Hi folks, > > We have created a sandbox for JEP 349: JFR Event Streaming[1] for > people who > like to try the feature out. It is work in progress, but plan is to > integrate it > in the next couple of months, but not for JDK 13. > > A build can be created like this: > > $ hg clone -b JEP-349-branch -r 53dccc90a5be > http://hg.openjdk.java.net/jdk/sandbox jep-349 > $ cd jep-349 > $ bash configure > $ make images > $ ./build/*/images/jdk/bin/java -version > > See [2] and [3] for more information on how to use the sandbox > repository and > build the JDK. > > For testing purposes, and to illustrate how the Event Streaming API > can be > used, there exists a Java agent[4] that aggregates event data and > print the > results on standard out. > > Example, > http://cr.openjdk.java.net/~egahlin/health-report-agent/health-report-example.txt > > =================== HEALTH REPORT === 2019-05-16 23:57:50 > ==================== > | GC: G1Old/G1New Phys. memory: 28669 MB Alloc Rate: 8 > MB/s | > | OC Count : 28 Initial Heap: 448 MB Total Alloc: 190 > MB | > | OC Pause Avg: 40.1 ms Used Heap : 19 MB Thread Count: > 20.0 | > | OC Pause Max: 48.8 ms Commit. Heap: 47 MB Class Count : > 3894.0 | > | YC Count : 8 CPU Machine : 20.12 % Safepoints: > 335 | > | YC Pause Avg: 5.7 ms CPU JVM User : 10.28 % Max Safepoint: > 46.4 ms | > | YC Pause Max: 22.4 ms CPU JVM System: 1.07 % Max Comp. Time: > 728.3 ms | > |--- Top Allocation Methods ------------------------------- > -----------------| > | DataBufferInt.(int) 11.27 % | > | Component.size() 9.01 % | > | BufferedContext.validate(...) 6.21 % | > | Path2D$Double.(...) 5.87 % | > | SunGraphics2D.clone() 5.85 % | > |--- Hot Methods > ------------------------------------------------------------| > | DRenderer._endRendering(int, int) > 51.11 % | > | DRenderer.copyAARow(...) 6.67 % | > | Arrays.fill(...) 4.44 % | > | StringConcatFactory.doStringConcat(...) 2.22 % | > | MarlinTileGenerator.getAlphaNoRLE(...) 2.22 % | > ============================================================================== > > > The agent can be started like this: > > $ java -javaagent:health-report.jar MyApp > > The interval at which events are flushed can be changed with the interval > option. The default interval is 1 second. For example, to flush > every ten second, use -javaagent:health-report.jar=interval=10 > > For quick experimentation, the agent can also be run as a Java program > that > allocate objects in a loop. > > $ java Main.java > > Two new classes, EventStream[5] and RecordingStream[6], have been > added to the > jdk.jfr.consumer package. A EventFilter class may be added later for more > advanced filtering. > > The JVM implementation is close to finished, but there is work on the > consumer > side left. There is also an effort to reduce allocation on the client. > > To get started without using the agent, the CPU load can be printed > like this: > > Configuration c = Configuration.getConfiguration("default"); > try (RecordingStream r = new RecordingStream(c)) { > r.onEvent("jdk.CPULoad", event -> { > float percentage = 100 * event.getFloat("machineTotal"); > System.out.printf("CPU: %.2f %% \n\n", percentage); > }); > r.start(); > } > > Feedback is very much welcomed! > > Thanks > Erik and Markus > > Outages, known issues and limitations > ------------------------------------- > > - Only start/open one RecordingStream at a time > Implementation has not been tested with two RecordingStreams > simultaneously. > > - Exception handling > Behavior is undefined if an exception is thrown in a handler. > > - Parser level filtering disabled > Purpose of parser level filtering is to be able to use predefined > configurations, for example default.jfc, but only pay the overhead in the > parser of the events that the stream is listening to. Now all events are > parsed. Workaround is to only enable the events of interest. > > - Chunk files not removed > Chunk files may be left behind in the disk repository when the JVM exits. > > - EventStream::openFile(...) > Not possible to open a file as an event stream. > > - EventStream:::openRepository(...) > Not possible to open a (remote) directory as an event stream. > > - EventStream.setReuse(...) (cursor mode) > Method has not been added / implemented. > > - EventStream.setParallel(...) > Method has not been added / implemented as it will make most sense if > EventStream::openFile exists. > > - EventStream.setOrdered(...) > Method has not been added. Need to figure out how to handle events > that arrives > way out of order. For example, > > MyEvent event = new MyEvent(); > event.begin(); > event.end(); // end timestamp taken here > Thread.sleep(1000_000); > event.commit(); > > Currently a best effort is made to deliver events in order. > > [1] https://openjdk.java.net/jeps/349 > [2] http://cr.openjdk.java.net/~chegar/docs/sandbox.html > [3] http://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/building.html > [4] http://cr.openjdk.java.net/~egahlin/health-report-agent/ > [5] > http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/RecordingStream.html > [6] > http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/EventStream.html > > -------------- next part -------------- diff --git a/src/jdk.jfr/share/classes/jdk/jfr/consumer/EventSetLocation.java b/src/jdk.jfr/share/classes/jdk/jfr/consumer/EventSetLocation.java --- a/src/jdk.jfr/share/classes/jdk/jfr/consumer/EventSetLocation.java +++ b/src/jdk.jfr/share/classes/jdk/jfr/consumer/EventSetLocation.java @@ -126,14 +126,19 @@ private void updateEventSets(EventSet previousEventSet) throws IOException { List added = new ArrayList<>(); Set current = new HashSet<>(); - try (DirectoryStream dirStream = Files.newDirectoryStream(path, "*.jfr")) { - for (Path p : dirStream) { - if (!lastPaths.containsKey(p)) { - added.add(p); - Logger.log(LogTag.JFR_SYSTEM_STREAMING, LogLevel.DEBUG, "New file found: " + p.toAbsolutePath()); + if(Files.exists(path)) { + Logger.log(LogTag.JFR_SYSTEM_STREAMING, LogLevel.DEBUG, "Repository found: " + path.toAbsolutePath()); + try (DirectoryStream dirStream = Files.newDirectoryStream(path, "*.jfr")) { + for (Path p : dirStream) { + if (!lastPaths.containsKey(p)) { + added.add(p); + Logger.log(LogTag.JFR_SYSTEM_STREAMING, LogLevel.DEBUG, "New file found: " + p.toAbsolutePath()); + } + current.add(p); } - current.add(p); } + } else { + Logger.log(LogTag.JFR_SYSTEM_STREAMING, LogLevel.DEBUG, "Repository does not found: " + path.toAbsolutePath()); } List removed = new ArrayList<>(); for (Path p : lastPaths.keySet()) { From erik.gahlin at oracle.com Mon May 20 17:46:57 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 20 May 2019 19:46:57 +0200 Subject: RFR: 8216283: Allow shorter method sampling interval than 10 ms Message-ID: <5CE2E811.4060904@oracle.com> Hi, Could I please have a review of a fix that allows method sampling to happen at a higher rate. The current limit of 10 ms exists because method sampling used to occur in the VM period task. Method sampling now happens in a separate thread. This was changed in JDK 9. The minimum accepted interval is lowered from 10 ms to 1 ms for both native and Java samples, but configuration files have been changed so native sampling interval can not be set lower than 20 ms using JMC Recording Wizard controls. Manual edit of the event setting allows 1 ms. The reason for this is to avoid excessive number native sample events. Purpose of native sampling is to detect "stuck threads". That is, threads that have called into native, but not returned. Most of the native samples will however be threads waiting in native, for example on a socket. Going below 20 ms adds little value, since a stuck thread is typically in native for a longer period of time. Method sampling in Java on the other hand can be set to 1 ms in a new "mode" called "Extreme". To avoid confusion, I didn't want to use "Maximum" since it means 10 ms in earlier releases. Webrev: http://cr.openjdk.java.net/~egahlin/8216283/ Bug: https://bugs.openjdk.java.net/browse/JDK-8216283 Testing: Tried to set the interval in the JMC Recording Wizard and verified that the correct interval was set in JVM using: $ jcmd JFR.check verbose=true Thanks Erik From erik.gahlin at oracle.com Mon May 20 18:05:54 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 20 May 2019 20:05:54 +0200 Subject: JEP 349: JFR Event Streaming (sandbox preview) In-Reply-To: <10a1e44a-6596-f375-b6a4-819b20380a25@oracle.com> References: <5CDEE0D4.4070502@oracle.com> <10a1e44a-6596-f375-b6a4-819b20380a25@oracle.com> Message-ID: <5CE2EC82.3010501@oracle.com> Hi Chihiro, Thanks for reporting this! The fix looks like a reasonable workaround, but I'm a bit puzzled that this can happen. This is how RecordingStream.startAsync() looks like. PlatformRecording pr = PrivateAccess.getInstance().getPlatformRecording(recording); long startNanos = pr.start(); stream.startAsync(startNanos); and PlatformRecording.start() should not return before RepositoryChunk::newChunk has been called and a repository is created, if it doesn't already exists. What operating system are you using? Thanks Erik > Hi Erik, > > I tried JFR Streaming with the following code [1], but occasionally an > exception occurred [2]. > This is because the repository has not yet been created when JFR > Streaming reads the JFR file. > I made a patch as attached. > > How do I handle bug reports? Should I raise the issue as usual? > > > [1] > public static void premain(String args, Instrumentation inst) { > RecordingStream r = new RecordingStream(); > r.startAsync(); > Runtime.getRuntime().addShutdownHook(new Thread() { > public void run() { > r.close(); > } > }); > } > > public static void main(String... args) throws Exception { > TimeUnit.SECONDS.sleep(2); > System.out.println("Completed"); > } > > [2] > java.nio.file.NoSuchFileException: /tmp/2019_05_20_00_30_44_445 > at > java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) > at > java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:432) > at java.base/java.nio.file.Files.newDirectoryStream(Files.java:542) > at > jdk.jfr/jdk.jfr.consumer.EventSetLocation.updateEventSets(EventSetLocation.java:129) > at > jdk.jfr/jdk.jfr.consumer.EventSetLocation.acquire(EventSetLocation.java:108) > at jdk.jfr/jdk.jfr.consumer.EventSet.next(EventSet.java:210) > at jdk.jfr/jdk.jfr.consumer.EventRunner.process(EventRunner.java:132) > at jdk.jfr/jdk.jfr.consumer.EventRunner.execute(EventRunner.java:105) > at > jdk.jfr/jdk.jfr.consumer.EventRunner.lambda$run$0(EventRunner.java:89) > at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:96) > at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:93) > at > java.base/java.security.AccessController.doPrivileged(AccessController.java:391) > at > jdk.jfr/jdk.jfr.consumer.EventRunner.doPriviliged(EventRunner.java:93) > at jdk.jfr/jdk.jfr.consumer.EventRunner.run(EventRunner.java:89) > at java.base/java.lang.Thread.run(Thread.java:830) > > Regards, > Chihiro > > > > On 2019/05/18 1:27, Erik Gahlin wrote: >> Hi folks, >> >> We have created a sandbox for JEP 349: JFR Event Streaming[1] for >> people who >> like to try the feature out. It is work in progress, but plan is to >> integrate it >> in the next couple of months, but not for JDK 13. >> >> A build can be created like this: >> >> $ hg clone -b JEP-349-branch -r 53dccc90a5be >> http://hg.openjdk.java.net/jdk/sandbox jep-349 >> $ cd jep-349 >> $ bash configure >> $ make images >> $ ./build/*/images/jdk/bin/java -version >> >> See [2] and [3] for more information on how to use the sandbox >> repository and >> build the JDK. >> >> For testing purposes, and to illustrate how the Event Streaming API >> can be >> used, there exists a Java agent[4] that aggregates event data and >> print the >> results on standard out. >> >> Example, >> http://cr.openjdk.java.net/~egahlin/health-report-agent/health-report-example.txt >> >> =================== HEALTH REPORT === 2019-05-16 23:57:50 >> ==================== >> | GC: G1Old/G1New Phys. memory: 28669 MB Alloc Rate: 8 >> MB/s | >> | OC Count : 28 Initial Heap: 448 MB Total Alloc: 190 >> MB | >> | OC Pause Avg: 40.1 ms Used Heap : 19 MB Thread Count: >> 20.0 | >> | OC Pause Max: 48.8 ms Commit. Heap: 47 MB Class Count : >> 3894.0 | >> | YC Count : 8 CPU Machine : 20.12 % Safepoints: >> 335 | >> | YC Pause Avg: 5.7 ms CPU JVM User : 10.28 % Max Safepoint: >> 46.4 ms | >> | YC Pause Max: 22.4 ms CPU JVM System: 1.07 % Max Comp. Time: >> 728.3 ms | >> |--- Top Allocation Methods ------------------------------- >> -----------------| >> | DataBufferInt.(int) 11.27 % | >> | Component.size() 9.01 % | >> | BufferedContext.validate(...) 6.21 % | >> | Path2D$Double.(...) 5.87 % | >> | SunGraphics2D.clone() 5.85 % | >> |--- Hot Methods >> ------------------------------------------------------------| >> | DRenderer._endRendering(int, int) >> 51.11 % | >> | DRenderer.copyAARow(...) 6.67 % | >> | Arrays.fill(...) 4.44 % | >> | StringConcatFactory.doStringConcat(...) 2.22 % | >> | MarlinTileGenerator.getAlphaNoRLE(...) 2.22 % | >> ============================================================================== >> >> >> The agent can be started like this: >> >> $ java -javaagent:health-report.jar MyApp >> >> The interval at which events are flushed can be changed with the >> interval >> option. The default interval is 1 second. For example, to flush >> every ten second, use -javaagent:health-report.jar=interval=10 >> >> For quick experimentation, the agent can also be run as a Java >> program that >> allocate objects in a loop. >> >> $ java Main.java >> >> Two new classes, EventStream[5] and RecordingStream[6], have been >> added to the >> jdk.jfr.consumer package. A EventFilter class may be added later for >> more >> advanced filtering. >> >> The JVM implementation is close to finished, but there is work on the >> consumer >> side left. There is also an effort to reduce allocation on the client. >> >> To get started without using the agent, the CPU load can be printed >> like this: >> >> Configuration c = Configuration.getConfiguration("default"); >> try (RecordingStream r = new RecordingStream(c)) { >> r.onEvent("jdk.CPULoad", event -> { >> float percentage = 100 * event.getFloat("machineTotal"); >> System.out.printf("CPU: %.2f %% \n\n", percentage); >> }); >> r.start(); >> } >> >> Feedback is very much welcomed! >> >> Thanks >> Erik and Markus >> >> Outages, known issues and limitations >> ------------------------------------- >> >> - Only start/open one RecordingStream at a time >> Implementation has not been tested with two RecordingStreams >> simultaneously. >> >> - Exception handling >> Behavior is undefined if an exception is thrown in a handler. >> >> - Parser level filtering disabled >> Purpose of parser level filtering is to be able to use predefined >> configurations, for example default.jfc, but only pay the overhead in >> the >> parser of the events that the stream is listening to. Now all events are >> parsed. Workaround is to only enable the events of interest. >> >> - Chunk files not removed >> Chunk files may be left behind in the disk repository when the JVM >> exits. >> >> - EventStream::openFile(...) >> Not possible to open a file as an event stream. >> >> - EventStream:::openRepository(...) >> Not possible to open a (remote) directory as an event stream. >> >> - EventStream.setReuse(...) (cursor mode) >> Method has not been added / implemented. >> >> - EventStream.setParallel(...) >> Method has not been added / implemented as it will make most sense if >> EventStream::openFile exists. >> >> - EventStream.setOrdered(...) >> Method has not been added. Need to figure out how to handle events >> that arrives >> way out of order. For example, >> >> MyEvent event = new MyEvent(); >> event.begin(); >> event.end(); // end timestamp taken here >> Thread.sleep(1000_000); >> event.commit(); >> >> Currently a best effort is made to deliver events in order. >> >> [1] https://openjdk.java.net/jeps/349 >> [2] http://cr.openjdk.java.net/~chegar/docs/sandbox.html >> [3] http://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/building.html >> [4] http://cr.openjdk.java.net/~egahlin/health-report-agent/ >> [5] >> http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/RecordingStream.html >> [6] >> http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/EventStream.html >> >> > From erik.gahlin at oracle.com Mon May 20 18:09:05 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 20 May 2019 20:09:05 +0200 Subject: JEP 349: JFR Event Streaming (sandbox preview) In-Reply-To: <5CE2EC82.3010501@oracle.com> References: <5CDEE0D4.4070502@oracle.com> <10a1e44a-6596-f375-b6a4-819b20380a25@oracle.com> <5CE2EC82.3010501@oracle.com> Message-ID: <5CE2ED41.4090903@oracle.com> What I think happens is that the JVM is shutting down (removing the repository), before the streaming thread has started to read any files. Erik > Hi Chihiro, > > Thanks for reporting this! > > The fix looks like a reasonable workaround, but I'm a bit puzzled that > this can happen. > > This is how RecordingStream.startAsync() looks like. > > PlatformRecording pr = > PrivateAccess.getInstance().getPlatformRecording(recording); > long startNanos = pr.start(); > stream.startAsync(startNanos); > > and PlatformRecording.start() should not return before > RepositoryChunk::newChunk has been called and a repository is created, > if it doesn't already exists. > > What operating system are you using? > > Thanks > Erik > > >> Hi Erik, >> >> I tried JFR Streaming with the following code [1], but occasionally >> an exception occurred [2]. >> This is because the repository has not yet been created when JFR >> Streaming reads the JFR file. >> I made a patch as attached. >> >> How do I handle bug reports? Should I raise the issue as usual? >> >> >> [1] >> public static void premain(String args, Instrumentation inst) { >> RecordingStream r = new RecordingStream(); >> r.startAsync(); >> Runtime.getRuntime().addShutdownHook(new Thread() { >> public void run() { >> r.close(); >> } >> }); >> } >> >> public static void main(String... args) throws Exception { >> TimeUnit.SECONDS.sleep(2); >> System.out.println("Completed"); >> } >> >> [2] >> java.nio.file.NoSuchFileException: /tmp/2019_05_20_00_30_44_445 >> at >> java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) >> at >> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) >> at >> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) >> at >> java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:432) >> at java.base/java.nio.file.Files.newDirectoryStream(Files.java:542) >> at >> jdk.jfr/jdk.jfr.consumer.EventSetLocation.updateEventSets(EventSetLocation.java:129) >> at >> jdk.jfr/jdk.jfr.consumer.EventSetLocation.acquire(EventSetLocation.java:108) >> at jdk.jfr/jdk.jfr.consumer.EventSet.next(EventSet.java:210) >> at >> jdk.jfr/jdk.jfr.consumer.EventRunner.process(EventRunner.java:132) >> at >> jdk.jfr/jdk.jfr.consumer.EventRunner.execute(EventRunner.java:105) >> at >> jdk.jfr/jdk.jfr.consumer.EventRunner.lambda$run$0(EventRunner.java:89) >> at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:96) >> at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:93) >> at >> java.base/java.security.AccessController.doPrivileged(AccessController.java:391) >> at >> jdk.jfr/jdk.jfr.consumer.EventRunner.doPriviliged(EventRunner.java:93) >> at jdk.jfr/jdk.jfr.consumer.EventRunner.run(EventRunner.java:89) >> at java.base/java.lang.Thread.run(Thread.java:830) >> >> Regards, >> Chihiro >> >> >> >> On 2019/05/18 1:27, Erik Gahlin wrote: >>> Hi folks, >>> >>> We have created a sandbox for JEP 349: JFR Event Streaming[1] for >>> people who >>> like to try the feature out. It is work in progress, but plan is to >>> integrate it >>> in the next couple of months, but not for JDK 13. >>> >>> A build can be created like this: >>> >>> $ hg clone -b JEP-349-branch -r 53dccc90a5be >>> http://hg.openjdk.java.net/jdk/sandbox jep-349 >>> $ cd jep-349 >>> $ bash configure >>> $ make images >>> $ ./build/*/images/jdk/bin/java -version >>> >>> See [2] and [3] for more information on how to use the sandbox >>> repository and >>> build the JDK. >>> >>> For testing purposes, and to illustrate how the Event Streaming API >>> can be >>> used, there exists a Java agent[4] that aggregates event data and >>> print the >>> results on standard out. >>> >>> Example, >>> http://cr.openjdk.java.net/~egahlin/health-report-agent/health-report-example.txt >>> >>> =================== HEALTH REPORT === 2019-05-16 23:57:50 >>> ==================== >>> | GC: G1Old/G1New Phys. memory: 28669 MB Alloc Rate: 8 >>> MB/s | >>> | OC Count : 28 Initial Heap: 448 MB Total Alloc: 190 >>> MB | >>> | OC Pause Avg: 40.1 ms Used Heap : 19 MB Thread Count: >>> 20.0 | >>> | OC Pause Max: 48.8 ms Commit. Heap: 47 MB Class Count : >>> 3894.0 | >>> | YC Count : 8 CPU Machine : 20.12 % Safepoints: >>> 335 | >>> | YC Pause Avg: 5.7 ms CPU JVM User : 10.28 % Max Safepoint: >>> 46.4 ms | >>> | YC Pause Max: 22.4 ms CPU JVM System: 1.07 % Max Comp. Time: >>> 728.3 ms | >>> |--- Top Allocation Methods ------------------------------- >>> -----------------| >>> | DataBufferInt.(int) 11.27 % | >>> | Component.size() 9.01 % | >>> | BufferedContext.validate(...) 6.21 % | >>> | Path2D$Double.(...) 5.87 % | >>> | SunGraphics2D.clone() 5.85 % | >>> |--- Hot Methods >>> ------------------------------------------------------------| >>> | DRenderer._endRendering(int, int) >>> 51.11 % | >>> | DRenderer.copyAARow(...) 6.67 % | >>> | Arrays.fill(...) 4.44 % | >>> | StringConcatFactory.doStringConcat(...) 2.22 % | >>> | MarlinTileGenerator.getAlphaNoRLE(...) 2.22 % | >>> ============================================================================== >>> >>> >>> The agent can be started like this: >>> >>> $ java -javaagent:health-report.jar MyApp >>> >>> The interval at which events are flushed can be changed with the >>> interval >>> option. The default interval is 1 second. For example, to flush >>> every ten second, use -javaagent:health-report.jar=interval=10 >>> >>> For quick experimentation, the agent can also be run as a Java >>> program that >>> allocate objects in a loop. >>> >>> $ java Main.java >>> >>> Two new classes, EventStream[5] and RecordingStream[6], have been >>> added to the >>> jdk.jfr.consumer package. A EventFilter class may be added later for >>> more >>> advanced filtering. >>> >>> The JVM implementation is close to finished, but there is work on >>> the consumer >>> side left. There is also an effort to reduce allocation on the client. >>> >>> To get started without using the agent, the CPU load can be printed >>> like this: >>> >>> Configuration c = Configuration.getConfiguration("default"); >>> try (RecordingStream r = new RecordingStream(c)) { >>> r.onEvent("jdk.CPULoad", event -> { >>> float percentage = 100 * event.getFloat("machineTotal"); >>> System.out.printf("CPU: %.2f %% \n\n", percentage); >>> }); >>> r.start(); >>> } >>> >>> Feedback is very much welcomed! >>> >>> Thanks >>> Erik and Markus >>> >>> Outages, known issues and limitations >>> ------------------------------------- >>> >>> - Only start/open one RecordingStream at a time >>> Implementation has not been tested with two RecordingStreams >>> simultaneously. >>> >>> - Exception handling >>> Behavior is undefined if an exception is thrown in a handler. >>> >>> - Parser level filtering disabled >>> Purpose of parser level filtering is to be able to use predefined >>> configurations, for example default.jfc, but only pay the overhead >>> in the >>> parser of the events that the stream is listening to. Now all events >>> are >>> parsed. Workaround is to only enable the events of interest. >>> >>> - Chunk files not removed >>> Chunk files may be left behind in the disk repository when the JVM >>> exits. >>> >>> - EventStream::openFile(...) >>> Not possible to open a file as an event stream. >>> >>> - EventStream:::openRepository(...) >>> Not possible to open a (remote) directory as an event stream. >>> >>> - EventStream.setReuse(...) (cursor mode) >>> Method has not been added / implemented. >>> >>> - EventStream.setParallel(...) >>> Method has not been added / implemented as it will make most sense if >>> EventStream::openFile exists. >>> >>> - EventStream.setOrdered(...) >>> Method has not been added. Need to figure out how to handle events >>> that arrives >>> way out of order. For example, >>> >>> MyEvent event = new MyEvent(); >>> event.begin(); >>> event.end(); // end timestamp taken here >>> Thread.sleep(1000_000); >>> event.commit(); >>> >>> Currently a best effort is made to deliver events in order. >>> >>> [1] https://openjdk.java.net/jeps/349 >>> [2] http://cr.openjdk.java.net/~chegar/docs/sandbox.html >>> [3] http://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/building.html >>> [4] http://cr.openjdk.java.net/~egahlin/health-report-agent/ >>> [5] >>> http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/RecordingStream.html >>> [6] >>> http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/EventStream.html >>> >>> >> > From chihiro.ito at oracle.com Mon May 20 18:46:22 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Tue, 21 May 2019 03:46:22 +0900 Subject: JEP 349: JFR Event Streaming (sandbox preview) In-Reply-To: <5CE2ED41.4090903@oracle.com> References: <5CDEE0D4.4070502@oracle.com> <10a1e44a-6596-f375-b6a4-819b20380a25@oracle.com> <5CE2EC82.3010501@oracle.com> <5CE2ED41.4090903@oracle.com> Message-ID: <7b268f8c-9db9-b0aa-6e77-19c84aad7cf9@oracle.com> Hi Erik, Thank you for your prompt reply. I did the debugging referring to your advice. As you say, this exception is likely caused by the timing of the Streaming thread and the process of deleting the file. My environment is Oracle Linux 7.5 with kernel 4.1. 12 -112.16. 4. el 7 uke.x 86 _ 64. Regards, Chihiro On 2019/05/21 3:09, Erik Gahlin wrote: > What I think happens is that the JVM is shutting down (removing the > repository), before the streaming thread has started to read any files. > > Erik > >> Hi Chihiro, >> >> Thanks for reporting this! >> >> The fix looks like a reasonable workaround, but I'm a bit puzzled >> that this can happen. >> >> This is how RecordingStream.startAsync() looks like. >> >> PlatformRecording pr = >> PrivateAccess.getInstance().getPlatformRecording(recording); >> long startNanos = pr.start(); >> stream.startAsync(startNanos); >> >> and PlatformRecording.start() should not return before >> RepositoryChunk::newChunk has been called and a repository is >> created, if it doesn't already exists. >> >> What operating system are you using? >> >> Thanks >> Erik >> >> >>> Hi Erik, >>> >>> I tried JFR Streaming with the following code [1], but occasionally >>> an exception occurred [2]. >>> This is because the repository has not yet been created when JFR >>> Streaming reads the JFR file. >>> I made a patch as attached. >>> >>> How do I handle bug reports? Should I raise the issue as usual? >>> >>> >>> [1] >>> public static void premain(String args, Instrumentation inst) { >>> RecordingStream r = new RecordingStream(); >>> r.startAsync(); >>> Runtime.getRuntime().addShutdownHook(new Thread() { >>> public void run() { >>> r.close(); >>> } >>> }); >>> } >>> >>> public static void main(String... args) throws Exception { >>> TimeUnit.SECONDS.sleep(2); >>> System.out.println("Completed"); >>> } >>> >>> [2] >>> java.nio.file.NoSuchFileException: /tmp/2019_05_20_00_30_44_445 >>> at >>> java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) >>> at >>> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) >>> at >>> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) >>> at >>> java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:432) >>> at java.base/java.nio.file.Files.newDirectoryStream(Files.java:542) >>> at >>> jdk.jfr/jdk.jfr.consumer.EventSetLocation.updateEventSets(EventSetLocation.java:129) >>> at >>> jdk.jfr/jdk.jfr.consumer.EventSetLocation.acquire(EventSetLocation.java:108) >>> at jdk.jfr/jdk.jfr.consumer.EventSet.next(EventSet.java:210) >>> at >>> jdk.jfr/jdk.jfr.consumer.EventRunner.process(EventRunner.java:132) >>> at >>> jdk.jfr/jdk.jfr.consumer.EventRunner.execute(EventRunner.java:105) >>> at >>> jdk.jfr/jdk.jfr.consumer.EventRunner.lambda$run$0(EventRunner.java:89) >>> at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:96) >>> at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:93) >>> at >>> java.base/java.security.AccessController.doPrivileged(AccessController.java:391) >>> at >>> jdk.jfr/jdk.jfr.consumer.EventRunner.doPriviliged(EventRunner.java:93) >>> at jdk.jfr/jdk.jfr.consumer.EventRunner.run(EventRunner.java:89) >>> at java.base/java.lang.Thread.run(Thread.java:830) >>> >>> Regards, >>> Chihiro >>> >>> >>> >>> On 2019/05/18 1:27, Erik Gahlin wrote: >>>> Hi folks, >>>> >>>> We have created a sandbox for JEP 349: JFR Event Streaming[1] for >>>> people who >>>> like to try the feature out. It is work in progress, but plan is to >>>> integrate it >>>> in the next couple of months, but not for JDK 13. >>>> >>>> A build can be created like this: >>>> >>>> $ hg clone -b JEP-349-branch -r 53dccc90a5be >>>> http://hg.openjdk.java.net/jdk/sandbox jep-349 >>>> $ cd jep-349 >>>> $ bash configure >>>> $ make images >>>> $ ./build/*/images/jdk/bin/java -version >>>> >>>> See [2] and [3] for more information on how to use the sandbox >>>> repository and >>>> build the JDK. >>>> >>>> For testing purposes, and to illustrate how the Event Streaming API >>>> can be >>>> used, there exists a Java agent[4] that aggregates event data and >>>> print the >>>> results on standard out. >>>> >>>> Example, >>>> http://cr.openjdk.java.net/~egahlin/health-report-agent/health-report-example.txt >>>> >>>> =================== HEALTH REPORT === 2019-05-16 23:57:50 >>>> ==================== >>>> | GC: G1Old/G1New Phys. memory: 28669 MB Alloc Rate: 8 >>>> MB/s | >>>> | OC Count : 28 Initial Heap: 448 MB Total Alloc: >>>> 190 MB | >>>> | OC Pause Avg: 40.1 ms Used Heap : 19 MB Thread Count: >>>> 20.0 | >>>> | OC Pause Max: 48.8 ms Commit. Heap: 47 MB Class Count : >>>> 3894.0 | >>>> | YC Count : 8 CPU Machine : 20.12 % Safepoints: >>>> 335 | >>>> | YC Pause Avg: 5.7 ms CPU JVM User : 10.28 % Max Safepoint: >>>> 46.4 ms | >>>> | YC Pause Max: 22.4 ms CPU JVM System: 1.07 % Max Comp. Time: >>>> 728.3 ms | >>>> |--- Top Allocation Methods ------------------------------- >>>> -----------------| >>>> | DataBufferInt.(int) 11.27 % | >>>> | Component.size() 9.01 % | >>>> | BufferedContext.validate(...) 6.21 % | >>>> | Path2D$Double.(...) 5.87 % | >>>> | SunGraphics2D.clone() 5.85 % | >>>> |--- Hot Methods >>>> ------------------------------------------------------------| >>>> | DRenderer._endRendering(int, >>>> int) 51.11 % | >>>> | DRenderer.copyAARow(...) 6.67 % | >>>> | Arrays.fill(...) 4.44 % | >>>> | StringConcatFactory.doStringConcat(...) 2.22 % | >>>> | MarlinTileGenerator.getAlphaNoRLE(...) 2.22 % | >>>> ============================================================================== >>>> >>>> >>>> The agent can be started like this: >>>> >>>> $ java -javaagent:health-report.jar MyApp >>>> >>>> The interval at which events are flushed can be changed with the >>>> interval >>>> option. The default interval is 1 second. For example, to flush >>>> every ten second, use -javaagent:health-report.jar=interval=10 >>>> >>>> For quick experimentation, the agent can also be run as a Java >>>> program that >>>> allocate objects in a loop. >>>> >>>> $ java Main.java >>>> >>>> Two new classes, EventStream[5] and RecordingStream[6], have been >>>> added to the >>>> jdk.jfr.consumer package. A EventFilter class may be added later >>>> for more >>>> advanced filtering. >>>> >>>> The JVM implementation is close to finished, but there is work on >>>> the consumer >>>> side left. There is also an effort to reduce allocation on the client. >>>> >>>> To get started without using the agent, the CPU load can be printed >>>> like this: >>>> >>>> Configuration c = Configuration.getConfiguration("default"); >>>> try (RecordingStream r = new RecordingStream(c)) { >>>> r.onEvent("jdk.CPULoad", event -> { >>>> float percentage = 100 * event.getFloat("machineTotal"); >>>> System.out.printf("CPU: %.2f %% \n\n", percentage); >>>> }); >>>> r.start(); >>>> } >>>> >>>> Feedback is very much welcomed! >>>> >>>> Thanks >>>> Erik and Markus >>>> >>>> Outages, known issues and limitations >>>> ------------------------------------- >>>> >>>> - Only start/open one RecordingStream at a time >>>> Implementation has not been tested with two RecordingStreams >>>> simultaneously. >>>> >>>> - Exception handling >>>> Behavior is undefined if an exception is thrown in a handler. >>>> >>>> - Parser level filtering disabled >>>> Purpose of parser level filtering is to be able to use predefined >>>> configurations, for example default.jfc, but only pay the overhead >>>> in the >>>> parser of the events that the stream is listening to. Now all >>>> events are >>>> parsed. Workaround is to only enable the events of interest. >>>> >>>> - Chunk files not removed >>>> Chunk files may be left behind in the disk repository when the JVM >>>> exits. >>>> >>>> - EventStream::openFile(...) >>>> Not possible to open a file as an event stream. >>>> >>>> - EventStream:::openRepository(...) >>>> Not possible to open a (remote) directory as an event stream. >>>> >>>> - EventStream.setReuse(...) (cursor mode) >>>> Method has not been added / implemented. >>>> >>>> - EventStream.setParallel(...) >>>> Method has not been added / implemented as it will make most sense if >>>> EventStream::openFile exists. >>>> >>>> - EventStream.setOrdered(...) >>>> Method has not been added. Need to figure out how to handle events >>>> that arrives >>>> way out of order. For example, >>>> >>>> MyEvent event = new MyEvent(); >>>> event.begin(); >>>> event.end(); // end timestamp taken here >>>> Thread.sleep(1000_000); >>>> event.commit(); >>>> >>>> Currently a best effort is made to deliver events in order. >>>> >>>> [1] https://openjdk.java.net/jeps/349 >>>> [2] http://cr.openjdk.java.net/~chegar/docs/sandbox.html >>>> [3] http://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/building.html >>>> [4] http://cr.openjdk.java.net/~egahlin/health-report-agent/ >>>> [5] >>>> http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/RecordingStream.html >>>> [6] >>>> http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/EventStream.html >>>> >>>> >>> >> > > From mikhailo.seledtsov at oracle.com Mon May 20 19:54:09 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 20 May 2019 12:54:09 -0700 Subject: RFR: 8224217: RecordingInfo should use textual representation of path In-Reply-To: <90fdc12d-7e99-436b-871c-be24dd79afd3@default> References: <5CE2B39E.8000108@oracle.com> <90fdc12d-7e99-436b-871c-be24dd79afd3@default> Message-ID: <7f460714-9cc2-6394-7d14-bb7c85cc7fe5@oracle.com> Looks good, Please update the copyright years to 2019 on changed files before the integration (no need for posting new webrev). Thank you, Misha On 5/20/19 8:58 AM, Markus Gronlund wrote: > Looks good. > > Thanks > Markus > > -----Original Message----- > From: Erik Gahlin > Sent: den 20 maj 2019 16:03 > To: hotspot-jfr-dev at openjdk.java.net > Subject: RFR: 8224217: RecordingInfo should use textual representation of path > > Hi, > > Could I please have a review this fix to RecordingInfo. > > Webrev: > http://cr.openjdk.java.net/~egahlin/8224217/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8224217 > > Testing: > test/jdk/jdk/jfr/* > > Thanks > Erik > From robbin.ehn at oracle.com Tue May 21 13:48:40 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 21 May 2019 15:48:40 +0200 Subject: RFR(M): 8220293: Deadlock in JFR string pool In-Reply-To: <5ede3dbd-1f17-489c-abf6-504d1eece7d8@default> References: <5ede3dbd-1f17-489c-abf6-504d1eece7d8@default> Message-ID: Hi Markus, On 2019-05-03 15:00, Markus Gronlund wrote: > Greeting, > > Please review the following patch to address the following: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220293 > Webrev: http://cr.openjdk.java.net/~mgronlun/8220293/webrev01/ > > High level summary: > > We don't want to release a retired buffer, because this will break the invariant that a successful acquisition implies a valid (non-retired) buffer. > This is the reason why this work is, at least prima facie, a little more involved. This seems fine, from my limited understanding of jfr buffer subsystem. Thanks for fixing! /Robbin > > Tests: > test case / reproducer attached to: https://bugs.openjdk.java.net/browse/JDK-8220293 > open/test/jdk/:jdk_jfr > > Thank you > Markus > From markus.gronlund at oracle.com Tue May 21 14:27:06 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 21 May 2019 14:27:06 +0000 (UTC) Subject: RFR(M): 8220293: Deadlock in JFR string pool In-Reply-To: References: <5ede3dbd-1f17-489c-abf6-504d1eece7d8@default> Message-ID: Thank you very much Robbin for taking a look. Cheers Markus -----Original Message----- From: Robbin Ehn Sent: den 21 maj 2019 15:49 To: Markus Gronlund ; Hotspot dev runtime ; hotspot-jfr-dev at openjdk.java.net Subject: Re: RFR(M): 8220293: Deadlock in JFR string pool Hi Markus, On 2019-05-03 15:00, Markus Gronlund wrote: > Greeting, > > Please review the following patch to address the following: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220293 > Webrev: http://cr.openjdk.java.net/~mgronlun/8220293/webrev01/ > > High level summary: > > We don't want to release a retired buffer, because this will break the invariant that a successful acquisition implies a valid (non-retired) buffer. > This is the reason why this work is, at least prima facie, a little more involved. This seems fine, from my limited understanding of jfr buffer subsystem. Thanks for fixing! /Robbin > > Tests: > test case / reproducer attached to: https://bugs.openjdk.java.net/browse/JDK-8220293 > open/test/jdk/:jdk_jfr > > Thank you > Markus > From erik.gahlin at oracle.com Tue May 21 16:46:26 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 21 May 2019 18:46:26 +0200 Subject: RFR(M): 8220293: Deadlock in JFR string pool In-Reply-To: <5ede3dbd-1f17-489c-abf6-504d1eece7d8@default> References: <5ede3dbd-1f17-489c-abf6-504d1eece7d8@default> Message-ID: <5CE42B62.3020303@oracle.com> Looks good. Erik > Greeting, > > Please review the following patch to address the following: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220293 > Webrev: http://cr.openjdk.java.net/~mgronlun/8220293/webrev01/ > > High level summary: > > We don't want to release a retired buffer, because this will break the invariant that a successful acquisition implies a valid (non-retired) buffer. > This is the reason why this work is, at least prima facie, a little more involved. > > Tests: > test case / reproducer attached to: https://bugs.openjdk.java.net/browse/JDK-8220293 > open/test/jdk/:jdk_jfr > > Thank you > Markus From erik.gahlin at oracle.com Tue May 21 18:20:06 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 21 May 2019 20:20:06 +0200 Subject: JEP 349: JFR Event Streaming (sandbox preview) In-Reply-To: <7b268f8c-9db9-b0aa-6e77-19c84aad7cf9@oracle.com> References: <5CDEE0D4.4070502@oracle.com> <10a1e44a-6596-f375-b6a4-819b20380a25@oracle.com> <5CE2EC82.3010501@oracle.com> <5CE2ED41.4090903@oracle.com> <7b268f8c-9db9-b0aa-6e77-19c84aad7cf9@oracle.com> Message-ID: <5CE44156.30807@oracle.com> I will add a Files.exist check file before opening a DirectoryStream. This race should really only happen during shutdown, so I will change so it will cancel the thread, perhaps by throwing an exception. Erik > Hi Erik, > > Thank you for your prompt reply. > I did the debugging referring to your advice. As you say, this > exception is likely caused by the timing of the Streaming thread and > the process of deleting the file. > > My environment is Oracle Linux 7.5 with kernel 4.1. 12 -112.16. 4. el > 7 uke.x 86 _ 64. > > Regards, > Chihiro > > On 2019/05/21 3:09, Erik Gahlin wrote: >> What I think happens is that the JVM is shutting down (removing the >> repository), before the streaming thread has started to read any files. >> >> Erik >> >>> Hi Chihiro, >>> >>> Thanks for reporting this! >>> >>> The fix looks like a reasonable workaround, but I'm a bit puzzled >>> that this can happen. >>> >>> This is how RecordingStream.startAsync() looks like. >>> >>> PlatformRecording pr = >>> PrivateAccess.getInstance().getPlatformRecording(recording); >>> long startNanos = pr.start(); >>> stream.startAsync(startNanos); >>> >>> and PlatformRecording.start() should not return before >>> RepositoryChunk::newChunk has been called and a repository is >>> created, if it doesn't already exists. >>> >>> What operating system are you using? >>> >>> Thanks >>> Erik >>> >>> >>>> Hi Erik, >>>> >>>> I tried JFR Streaming with the following code [1], but occasionally >>>> an exception occurred [2]. >>>> This is because the repository has not yet been created when JFR >>>> Streaming reads the JFR file. >>>> I made a patch as attached. >>>> >>>> How do I handle bug reports? Should I raise the issue as usual? >>>> >>>> >>>> [1] >>>> public static void premain(String args, Instrumentation inst) { >>>> RecordingStream r = new RecordingStream(); >>>> r.startAsync(); >>>> Runtime.getRuntime().addShutdownHook(new Thread() { >>>> public void run() { >>>> r.close(); >>>> } >>>> }); >>>> } >>>> >>>> public static void main(String... args) throws Exception { >>>> TimeUnit.SECONDS.sleep(2); >>>> System.out.println("Completed"); >>>> } >>>> >>>> [2] >>>> java.nio.file.NoSuchFileException: /tmp/2019_05_20_00_30_44_445 >>>> at >>>> java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) >>>> at >>>> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) >>>> at >>>> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) >>>> at >>>> java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:432) >>>> at >>>> java.base/java.nio.file.Files.newDirectoryStream(Files.java:542) >>>> at >>>> jdk.jfr/jdk.jfr.consumer.EventSetLocation.updateEventSets(EventSetLocation.java:129) >>>> at >>>> jdk.jfr/jdk.jfr.consumer.EventSetLocation.acquire(EventSetLocation.java:108) >>>> at jdk.jfr/jdk.jfr.consumer.EventSet.next(EventSet.java:210) >>>> at >>>> jdk.jfr/jdk.jfr.consumer.EventRunner.process(EventRunner.java:132) >>>> at >>>> jdk.jfr/jdk.jfr.consumer.EventRunner.execute(EventRunner.java:105) >>>> at >>>> jdk.jfr/jdk.jfr.consumer.EventRunner.lambda$run$0(EventRunner.java:89) >>>> at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:96) >>>> at jdk.jfr/jdk.jfr.consumer.EventRunner$1.run(EventRunner.java:93) >>>> at >>>> java.base/java.security.AccessController.doPrivileged(AccessController.java:391) >>>> at >>>> jdk.jfr/jdk.jfr.consumer.EventRunner.doPriviliged(EventRunner.java:93) >>>> at jdk.jfr/jdk.jfr.consumer.EventRunner.run(EventRunner.java:89) >>>> at java.base/java.lang.Thread.run(Thread.java:830) >>>> >>>> Regards, >>>> Chihiro >>>> >>>> >>>> >>>> On 2019/05/18 1:27, Erik Gahlin wrote: >>>>> Hi folks, >>>>> >>>>> We have created a sandbox for JEP 349: JFR Event Streaming[1] for >>>>> people who >>>>> like to try the feature out. It is work in progress, but plan is >>>>> to integrate it >>>>> in the next couple of months, but not for JDK 13. >>>>> >>>>> A build can be created like this: >>>>> >>>>> $ hg clone -b JEP-349-branch -r 53dccc90a5be >>>>> http://hg.openjdk.java.net/jdk/sandbox jep-349 >>>>> $ cd jep-349 >>>>> $ bash configure >>>>> $ make images >>>>> $ ./build/*/images/jdk/bin/java -version >>>>> >>>>> See [2] and [3] for more information on how to use the sandbox >>>>> repository and >>>>> build the JDK. >>>>> >>>>> For testing purposes, and to illustrate how the Event Streaming >>>>> API can be >>>>> used, there exists a Java agent[4] that aggregates event data and >>>>> print the >>>>> results on standard out. >>>>> >>>>> Example, >>>>> http://cr.openjdk.java.net/~egahlin/health-report-agent/health-report-example.txt >>>>> >>>>> =================== HEALTH REPORT === 2019-05-16 23:57:50 >>>>> ==================== >>>>> | GC: G1Old/G1New Phys. memory: 28669 MB Alloc Rate: 8 >>>>> MB/s | >>>>> | OC Count : 28 Initial Heap: 448 MB Total Alloc: >>>>> 190 MB | >>>>> | OC Pause Avg: 40.1 ms Used Heap : 19 MB Thread Count: >>>>> 20.0 | >>>>> | OC Pause Max: 48.8 ms Commit. Heap: 47 MB Class Count : >>>>> 3894.0 | >>>>> | YC Count : 8 CPU Machine : 20.12 % Safepoints: >>>>> 335 | >>>>> | YC Pause Avg: 5.7 ms CPU JVM User : 10.28 % Max Safepoint: >>>>> 46.4 ms | >>>>> | YC Pause Max: 22.4 ms CPU JVM System: 1.07 % Max Comp. >>>>> Time: 728.3 ms | >>>>> |--- Top Allocation Methods ------------------------------- >>>>> -----------------| >>>>> | DataBufferInt.(int) 11.27 % | >>>>> | Component.size() 9.01 % | >>>>> | BufferedContext.validate(...) 6.21 % | >>>>> | Path2D$Double.(...) 5.87 % | >>>>> | SunGraphics2D.clone() 5.85 % | >>>>> |--- Hot Methods >>>>> ------------------------------------------------------------| >>>>> | DRenderer._endRendering(int, >>>>> int) 51.11 % | >>>>> | DRenderer.copyAARow(...) 6.67 % | >>>>> | Arrays.fill(...) 4.44 % | >>>>> | StringConcatFactory.doStringConcat(...) 2.22 % | >>>>> | MarlinTileGenerator.getAlphaNoRLE(...) 2.22 % | >>>>> ============================================================================== >>>>> >>>>> >>>>> The agent can be started like this: >>>>> >>>>> $ java -javaagent:health-report.jar MyApp >>>>> >>>>> The interval at which events are flushed can be changed with the >>>>> interval >>>>> option. The default interval is 1 second. For example, to flush >>>>> every ten second, use -javaagent:health-report.jar=interval=10 >>>>> >>>>> For quick experimentation, the agent can also be run as a Java >>>>> program that >>>>> allocate objects in a loop. >>>>> >>>>> $ java Main.java >>>>> >>>>> Two new classes, EventStream[5] and RecordingStream[6], have been >>>>> added to the >>>>> jdk.jfr.consumer package. A EventFilter class may be added later >>>>> for more >>>>> advanced filtering. >>>>> >>>>> The JVM implementation is close to finished, but there is work on >>>>> the consumer >>>>> side left. There is also an effort to reduce allocation on the >>>>> client. >>>>> >>>>> To get started without using the agent, the CPU load can be printed >>>>> like this: >>>>> >>>>> Configuration c = Configuration.getConfiguration("default"); >>>>> try (RecordingStream r = new RecordingStream(c)) { >>>>> r.onEvent("jdk.CPULoad", event -> { >>>>> float percentage = 100 * event.getFloat("machineTotal"); >>>>> System.out.printf("CPU: %.2f %% \n\n", percentage); >>>>> }); >>>>> r.start(); >>>>> } >>>>> >>>>> Feedback is very much welcomed! >>>>> >>>>> Thanks >>>>> Erik and Markus >>>>> >>>>> Outages, known issues and limitations >>>>> ------------------------------------- >>>>> >>>>> - Only start/open one RecordingStream at a time >>>>> Implementation has not been tested with two RecordingStreams >>>>> simultaneously. >>>>> >>>>> - Exception handling >>>>> Behavior is undefined if an exception is thrown in a handler. >>>>> >>>>> - Parser level filtering disabled >>>>> Purpose of parser level filtering is to be able to use predefined >>>>> configurations, for example default.jfc, but only pay the overhead >>>>> in the >>>>> parser of the events that the stream is listening to. Now all >>>>> events are >>>>> parsed. Workaround is to only enable the events of interest. >>>>> >>>>> - Chunk files not removed >>>>> Chunk files may be left behind in the disk repository when the JVM >>>>> exits. >>>>> >>>>> - EventStream::openFile(...) >>>>> Not possible to open a file as an event stream. >>>>> >>>>> - EventStream:::openRepository(...) >>>>> Not possible to open a (remote) directory as an event stream. >>>>> >>>>> - EventStream.setReuse(...) (cursor mode) >>>>> Method has not been added / implemented. >>>>> >>>>> - EventStream.setParallel(...) >>>>> Method has not been added / implemented as it will make most sense if >>>>> EventStream::openFile exists. >>>>> >>>>> - EventStream.setOrdered(...) >>>>> Method has not been added. Need to figure out how to handle events >>>>> that arrives >>>>> way out of order. For example, >>>>> >>>>> MyEvent event = new MyEvent(); >>>>> event.begin(); >>>>> event.end(); // end timestamp taken here >>>>> Thread.sleep(1000_000); >>>>> event.commit(); >>>>> >>>>> Currently a best effort is made to deliver events in order. >>>>> >>>>> [1] https://openjdk.java.net/jeps/349 >>>>> [2] http://cr.openjdk.java.net/~chegar/docs/sandbox.html >>>>> [3] >>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/building.html >>>>> [4] http://cr.openjdk.java.net/~egahlin/health-report-agent/ >>>>> [5] >>>>> http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/RecordingStream.html >>>>> [6] >>>>> http://cr.openjdk.java.net/~egahlin/jep-349-preview/docs/jdk/jfr/consumer/EventStream.html >>>>> >>>>> >>>> >>> >> >> > From christoph.langer at sap.com Tue May 21 20:01:46 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 21 May 2019 20:01:46 +0000 Subject: Build error on Windows after 8221507: Implement JFR Events for Shenandoah Message-ID: Hi Ken, after you pushed JDK-8221507, I see build errors on Windows (enabled warnings as errors): shenandoahHeapRegion.cpp .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): error C2220: warning treated as error - no 'object' file generated .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data ... (rest of output omitted) * For target hotspot_variant-server_libjvm_objs_shenandoahJfrSupport.obj: shenandoahJfrSupport.cpp .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): error C2220: warning treated as error - no 'object' file generated .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data ... (rest of output omitted) I don't know if the fix is to simply add a cast to unsigned int. Can you please check and repair? Thanks Christoph > -----Original Message----- > From: jmc-dev On Behalf Of Ken > Dobson > Sent: Dienstag, 14. Mai 2019 23:19 > To: Erik Gahlin > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; jmc- > dev at openjdk.java.net > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah > > Thank you for the review again. > > Yes, not sure regarding backporting plans but I imagine it will be in the > future so seemed best to avoid anything that cause issues such as Set.of(). > > I also think the naming of the jfr's is fine as it's stored in a folder > named after the test which was easily found now that I'm aware that the > jfrs are already dumped. > > I believe I've addressed the rest of your recommended changes to the tests, > let me know if I've covered everything. > > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ > > Thanks, > > Ken Dobson > > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin wrote: > > > Thanks for fixing! > > > > Product code looks good, but I think tests could be improved. > > > > There is no need to call dump in a final clause. > > > > Events.fromRecording(...) will dump the file into the jtreg scratch > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is > > sufficient since there is only one recording per test. If you think the > > name is cryptic, consider adding another method to Event.fromRecording > that > > takes an additional parameter that becomes a suffix, i.e > > "recording-1-pid-xxx-test-heap-region-info.jfr" > > > > try (Recording r = new recording()) { > > r.enable(EVENT_NAME); > > r.start(); > > r.stop() > > List events = Events.fromRecording(r); > > Events.hasEvents(events); > > for (RecordedEvent event : events) { > > Events.assertField(event, "index").notEqual(-1); > > Events.assertField(event, "used").atMost(1024*1024L); > > String state = Events.assertField(event, "state").getValue(); > > GCHelper.assertShenandoahHeapRegionState(state)); > > } > > > > The name of the field in metadata.xml is "state", but the tests looks for > > "type". That seems incorrect. > > > > GCHelper: > > > > public static assertShenandoahHeapRegionState(String state) { > > if (!shenandoahHeapRegionStates.contains(state)) { > > throw new AssertionError("Unknown state '"+ state +"', valid heap > > region states are " + shenandoahHeapRegionStates); > > } > > > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") > > because you want to backport to JDK 8. > > > > Thanks > > Erik > > > > Thanks everyone for their reviews. I've added tests similar to the tests > > for the G1 events as well as addressed the changes that Erik > recommended. > > > > Here is the new webrev: > > > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ > > > > Please let me know if there's anything else that should be addressed. > > > > Thanks, > > > > Ken Dobson > > > > On Mon, May 6, 2019 at 11:38 AM > wrote: > > > >> > >> > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: > >> >> Hi, > >> >> > >> >> If possible and feasible, could you please implement tests for new > >> >> events? > >> >> > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > >> >> create a "shenandoah" subfolder. > >> >> > >> >> If the events are too hard to test or not feasible, please add the > >> >> event names to > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to > >> >> the hardToTestEvents set. Consider commenting why it is too hard to > >> >> test. > >> > > >> > If events are are marked experimental, tests are ignored by > >> > TestLookForUntestedEvents. java, so we should not add them to the > >> > hardToTestEvents set. That may help them pass under the radar once > the > >> > experimental attribute is removed. > >> Thank you. If the events are "experimental" this approach makes sense. > >> Once the experimental status is removed, the > >> TestLookForUntestedEvents.java should be able to notice if the event is > >> not covered by tests. > >> > That said, tests never hurts, even if they are experimental. I think > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place > >> +1 > >> > >> Thank you, > >> Misha > >> > > >> > Erik > >> > > >> >> > >> >> Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ > >> >> prior to integration. > >> >> > >> >> > >> >> Best regards, > >> >> > >> >> Misha > >> >> > >> >> > >> >> > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: > >> >>> Hi all, > >> >>> > >> >>> Please review this patch that adds support for two new JFR events > >> >>> ShenandoahHeapRegionStateChange and > ShenandoahHeapRegionInformation. > >> >>> > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > >> >>> > >> >>> The Shenandoah team has also reviewed this patch and approved it > >> >>> from their > >> >>> end. > >> >>> > >> >>> Thanks, > >> >>> > >> >>> Ken Dobson > >> >> > >> > > >> > >> > > From mikhailo.seledtsov at oracle.com Tue May 21 23:01:33 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Tue, 21 May 2019 16:01:33 -0700 Subject: RFR(XS): 8224529: [TESTBUG] JFR TestShenandoahHeapRegion* tests fail on build w/o Shenandoah Message-ID: <453a43cb-60e5-6dba-65e6-0cd6f2e6a48a@oracle.com> Please review this small change that fixes at-requires statement for JFR tests that require Shenandoah GC. Change also adds vm.gc.Shenandoah to JDK's TEST.ROOT ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8224529 ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8224529.00/ ??? Testing: ??????? 1. Ran the affected tests on a build w/o Shenandoah: tests skipped as expected ??????? 2. Runnin entire jdk_jfr on a test cluster - in progress Thank you, Misha From thomas.stuefe at gmail.com Wed May 22 05:00:11 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 22 May 2019 07:00:11 +0200 Subject: Build error on Windows after 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: I am confused how this can even happen, should jdk-submit tests not have captured that before pushing? Cheers, Thomas On Tue, May 21, 2019 at 10:02 PM Langer, Christoph wrote: > Hi Ken, > > after you pushed JDK-8221507, I see build errors on Windows (enabled > warnings as errors): > > shenandoahHeapRegion.cpp > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): > error C2220: warning treated as error - no 'object' file generated > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): > warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', > possible loss of data > ... (rest of output omitted) > * For target hotspot_variant-server_libjvm_objs_shenandoahJfrSupport.obj: > shenandoahJfrSupport.cpp > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): > error C2220: warning treated as error - no 'object' file generated > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): > warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', > possible loss of data > ... (rest of output omitted) > > I don't know if the fix is to simply add a cast to unsigned int. Can you > please check and repair? > > Thanks > Christoph > > > > -----Original Message----- > > From: jmc-dev On Behalf Of Ken > > Dobson > > Sent: Dienstag, 14. Mai 2019 23:19 > > To: Erik Gahlin > > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; > jmc- > > dev at openjdk.java.net > > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah > > > > Thank you for the review again. > > > > Yes, not sure regarding backporting plans but I imagine it will be in the > > future so seemed best to avoid anything that cause issues such as > Set.of(). > > > > I also think the naming of the jfr's is fine as it's stored in a folder > > named after the test which was easily found now that I'm aware that the > > jfrs are already dumped. > > > > I believe I've addressed the rest of your recommended changes to the > tests, > > let me know if I've covered everything. > > > > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ > > > > Thanks, > > > > Ken Dobson > > > > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin > wrote: > > > > > Thanks for fixing! > > > > > > Product code looks good, but I think tests could be improved. > > > > > > There is no need to call dump in a final clause. > > > > > > Events.fromRecording(...) will dump the file into the jtreg scratch > > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is > > > sufficient since there is only one recording per test. If you think > the > > > name is cryptic, consider adding another method to Event.fromRecording > > that > > > takes an additional parameter that becomes a suffix, i.e > > > "recording-1-pid-xxx-test-heap-region-info.jfr" > > > > > > try (Recording r = new recording()) { > > > r.enable(EVENT_NAME); > > > r.start(); > > > r.stop() > > > List events = Events.fromRecording(r); > > > Events.hasEvents(events); > > > for (RecordedEvent event : events) { > > > Events.assertField(event, "index").notEqual(-1); > > > Events.assertField(event, "used").atMost(1024*1024L); > > > String state = Events.assertField(event, "state").getValue(); > > > GCHelper.assertShenandoahHeapRegionState(state)); > > > } > > > > > > The name of the field in metadata.xml is "state", but the tests looks > for > > > "type". That seems incorrect. > > > > > > GCHelper: > > > > > > public static assertShenandoahHeapRegionState(String state) { > > > if (!shenandoahHeapRegionStates.contains(state)) { > > > throw new AssertionError("Unknown state '"+ state +"', valid heap > > > region states are " + shenandoahHeapRegionStates); > > > } > > > > > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") > > > because you want to backport to JDK 8. > > > > > > Thanks > > > Erik > > > > > > Thanks everyone for their reviews. I've added tests similar to the > tests > > > for the G1 events as well as addressed the changes that Erik > > recommended. > > > > > > Here is the new webrev: > > > > > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ > > > > > > Please let me know if there's anything else that should be addressed. > > > > > > Thanks, > > > > > > Ken Dobson > > > > > > On Mon, May 6, 2019 at 11:38 AM > > wrote: > > > > > >> > > >> > > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: > > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: > > >> >> Hi, > > >> >> > > >> >> If possible and feasible, could you please implement tests for > new > > >> >> events? > > >> >> > > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > > >> >> create a "shenandoah" subfolder. > > >> >> > > >> >> If the events are too hard to test or not feasible, please add the > > >> >> event names to > > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to > > >> >> the hardToTestEvents set. Consider commenting why it is too hard to > > >> >> test. > > >> > > > >> > If events are are marked experimental, tests are ignored by > > >> > TestLookForUntestedEvents. java, so we should not add them to the > > >> > hardToTestEvents set. That may help them pass under the radar once > > the > > >> > experimental attribute is removed. > > >> Thank you. If the events are "experimental" this approach makes sense. > > >> Once the experimental status is removed, the > > >> TestLookForUntestedEvents.java should be able to notice if the event > is > > >> not covered by tests. > > >> > That said, tests never hurts, even if they are experimental. I think > > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place > > >> +1 > > >> > > >> Thank you, > > >> Misha > > >> > > > >> > Erik > > >> > > > >> >> > > >> >> Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ > > >> >> prior to integration. > > >> >> > > >> >> > > >> >> Best regards, > > >> >> > > >> >> Misha > > >> >> > > >> >> > > >> >> > > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: > > >> >>> Hi all, > > >> >>> > > >> >>> Please review this patch that adds support for two new JFR events > > >> >>> ShenandoahHeapRegionStateChange and > > ShenandoahHeapRegionInformation. > > >> >>> > > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > >> >>> > > >> >>> The Shenandoah team has also reviewed this patch and approved it > > >> >>> from their > > >> >>> end. > > >> >>> > > >> >>> Thanks, > > >> >>> > > >> >>> Ken Dobson > > >> >> > > >> > > > >> > > >> > > > > From christoph.langer at sap.com Wed May 22 06:47:02 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 22 May 2019 06:47:02 +0000 Subject: RFR(XS): 8224573: Fix windows build after JDK-8221507 Message-ID: Hi, please review this build fix for Windows: Bug: https://bugs.openjdk.java.net/browse/JDK-8224573 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8224573.0/ Thanks Christoph > -----Original Message----- > From: hotspot-jfr-dev On > Behalf Of Langer, Christoph > Sent: Dienstag, 21. Mai 2019 22:02 > To: Ken Dobson > Cc: hotspot-jfr-dev at openjdk.java.net > Subject: [CAUTION] Build error on Windows after 8221507: Implement JFR > Events for Shenandoah > > Hi Ken, > > after you pushed JDK-8221507, I see build errors on Windows (enabled > warnings as errors): > > shenandoahHeapRegion.cpp > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687) > : error C2220: warning treated as error - no 'object' file generated > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687) > : warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', > possible loss of data > ... (rest of output omitted) > * For target hotspot_variant- > server_libjvm_objs_shenandoahJfrSupport.obj: > shenandoahJfrSupport.cpp > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): > error C2220: warning treated as error - no 'object' file generated > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): > warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible > loss of data > ... (rest of output omitted) > > I don't know if the fix is to simply add a cast to unsigned int. Can you please > check and repair? > > Thanks > Christoph > > > > -----Original Message----- > > From: jmc-dev On Behalf Of Ken > > Dobson > > Sent: Dienstag, 14. Mai 2019 23:19 > > To: Erik Gahlin > > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; > jmc- > > dev at openjdk.java.net > > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah > > > > Thank you for the review again. > > > > Yes, not sure regarding backporting plans but I imagine it will be in the > > future so seemed best to avoid anything that cause issues such as Set.of(). > > > > I also think the naming of the jfr's is fine as it's stored in a folder > > named after the test which was easily found now that I'm aware that the > > jfrs are already dumped. > > > > I believe I've addressed the rest of your recommended changes to the > tests, > > let me know if I've covered everything. > > > > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ > > > > Thanks, > > > > Ken Dobson > > > > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin > wrote: > > > > > Thanks for fixing! > > > > > > Product code looks good, but I think tests could be improved. > > > > > > There is no need to call dump in a final clause. > > > > > > Events.fromRecording(...) will dump the file into the jtreg scratch > > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is > > > sufficient since there is only one recording per test. If you think the > > > name is cryptic, consider adding another method to Event.fromRecording > > that > > > takes an additional parameter that becomes a suffix, i.e > > > "recording-1-pid-xxx-test-heap-region-info.jfr" > > > > > > try (Recording r = new recording()) { > > > r.enable(EVENT_NAME); > > > r.start(); > > > r.stop() > > > List events = Events.fromRecording(r); > > > Events.hasEvents(events); > > > for (RecordedEvent event : events) { > > > Events.assertField(event, "index").notEqual(-1); > > > Events.assertField(event, "used").atMost(1024*1024L); > > > String state = Events.assertField(event, "state").getValue(); > > > GCHelper.assertShenandoahHeapRegionState(state)); > > > } > > > > > > The name of the field in metadata.xml is "state", but the tests looks for > > > "type". That seems incorrect. > > > > > > GCHelper: > > > > > > public static assertShenandoahHeapRegionState(String state) { > > > if (!shenandoahHeapRegionStates.contains(state)) { > > > throw new AssertionError("Unknown state '"+ state +"', valid heap > > > region states are " + shenandoahHeapRegionStates); > > > } > > > > > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") > > > because you want to backport to JDK 8. > > > > > > Thanks > > > Erik > > > > > > Thanks everyone for their reviews. I've added tests similar to the tests > > > for the G1 events as well as addressed the changes that Erik > > recommended. > > > > > > Here is the new webrev: > > > > > > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ > > > > > > Please let me know if there's anything else that should be addressed. > > > > > > Thanks, > > > > > > Ken Dobson > > > > > > On Mon, May 6, 2019 at 11:38 AM > > wrote: > > > > > >> > > >> > > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: > > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: > > >> >> Hi, > > >> >> > > >> >> If possible and feasible, could you please implement tests for new > > >> >> events? > > >> >> > > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > > >> >> create a "shenandoah" subfolder. > > >> >> > > >> >> If the events are too hard to test or not feasible, please add the > > >> >> event names to > > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, > to > > >> >> the hardToTestEvents set. Consider commenting why it is too hard to > > >> >> test. > > >> > > > >> > If events are are marked experimental, tests are ignored by > > >> > TestLookForUntestedEvents. java, so we should not add them to the > > >> > hardToTestEvents set. That may help them pass under the radar once > > the > > >> > experimental attribute is removed. > > >> Thank you. If the events are "experimental" this approach makes sense. > > >> Once the experimental status is removed, the > > >> TestLookForUntestedEvents.java should be able to notice if the event is > > >> not covered by tests. > > >> > That said, tests never hurts, even if they are experimental. I think > > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place > > >> +1 > > >> > > >> Thank you, > > >> Misha > > >> > > > >> > Erik > > >> > > > >> >> > > >> >> Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ > > >> >> prior to integration. > > >> >> > > >> >> > > >> >> Best regards, > > >> >> > > >> >> Misha > > >> >> > > >> >> > > >> >> > > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: > > >> >>> Hi all, > > >> >>> > > >> >>> Please review this patch that adds support for two new JFR events > > >> >>> ShenandoahHeapRegionStateChange and > > ShenandoahHeapRegionInformation. > > >> >>> > > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > >> >>> > > >> >>> The Shenandoah team has also reviewed this patch and approved it > > >> >>> from their > > >> >>> end. > > >> >>> > > >> >>> Thanks, > > >> >>> > > >> >>> Ken Dobson > > >> >> > > >> > > > >> > > >> > > > From christoph.langer at sap.com Wed May 22 06:53:58 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 22 May 2019 06:53:58 +0000 Subject: RFR(XS): 8224529: [TESTBUG] JFR TestShenandoahHeapRegion* tests fail on build w/o Shenandoah In-Reply-To: <453a43cb-60e5-6dba-65e6-0cd6f2e6a48a@oracle.com> References: <453a43cb-60e5-6dba-65e6-0cd6f2e6a48a@oracle.com> Message-ID: Hi Misha, looks good. Best regards Christoph > -----Original Message----- > From: hotspot-jfr-dev On > Behalf Of mikhailo.seledtsov at oracle.com > Sent: Mittwoch, 22. Mai 2019 01:02 > To: hotspot-jfr-dev at openjdk.java.net; Erik Gahlin > ; Markus Gronlund > > Subject: RFR(XS): 8224529: [TESTBUG] JFR TestShenandoahHeapRegion* > tests fail on build w/o Shenandoah > > Please review this small change that fixes at-requires statement for JFR > tests that require Shenandoah GC. > Change also adds vm.gc.Shenandoah to JDK's TEST.ROOT > > ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8224529 > ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8224529.00/ > ??? Testing: > ??????? 1. Ran the affected tests on a build w/o Shenandoah: tests > skipped as expected > ??????? 2. Runnin entire jdk_jfr on a test cluster - in progress > > > Thank you, > Misha From shade at redhat.com Wed May 22 06:54:21 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 22 May 2019 08:54:21 +0200 Subject: RFR(XS): 8224573: Fix windows build after JDK-8221507 In-Reply-To: References: Message-ID: <5e059d76-63e3-cafc-eea1-eac0371a5193@redhat.com> On 5/22/19 8:47 AM, Langer, Christoph wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8224573 > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8224573.0/ Argh. Casting to unsigned is fine for now. The patch is good and trivial. I would really like to see the types in events match those passed in to event, if possible. Ken, please see to rectify that in the follow up patch? -Aleksey From thomas.stuefe at gmail.com Wed May 22 06:59:12 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 22 May 2019 08:59:12 +0200 Subject: RFR(XS): 8224573: Fix windows build after JDK-8221507 In-Reply-To: References: Message-ID: Looks okay. 32bit should be enough for number of regions I think. If I am not mistaken, they have a min region size of 256K, so this covers a heap of ~1000 tb? ..Thomas On Wed, May 22, 2019 at 8:47 AM Langer, Christoph wrote: > Hi, > > please review this build fix for Windows: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224573 > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8224573.0/ > > Thanks > Christoph > > > > > -----Original Message----- > > From: hotspot-jfr-dev On > > Behalf Of Langer, Christoph > > Sent: Dienstag, 21. Mai 2019 22:02 > > To: Ken Dobson > > Cc: hotspot-jfr-dev at openjdk.java.net > > Subject: [CAUTION] Build error on Windows after 8221507: Implement JFR > > Events for Shenandoah > > > > Hi Ken, > > > > after you pushed JDK-8221507, I see build errors on Windows (enabled > > warnings as errors): > > > > shenandoahHeapRegion.cpp > > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687) > > : error C2220: warning treated as error - no 'object' file generated > > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687) > > : warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', > > possible loss of data > > ... (rest of output omitted) > > * For target hotspot_variant- > > server_libjvm_objs_shenandoahJfrSupport.obj: > > shenandoahJfrSupport.cpp > > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): > > error C2220: warning treated as error - no 'object' file generated > > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): > > warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', > possible > > loss of data > > ... (rest of output omitted) > > > > I don't know if the fix is to simply add a cast to unsigned int. Can you > please > > check and repair? > > > > Thanks > > Christoph > > > > > > > -----Original Message----- > > > From: jmc-dev On Behalf Of Ken > > > Dobson > > > Sent: Dienstag, 14. Mai 2019 23:19 > > > To: Erik Gahlin > > > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; > > jmc- > > > dev at openjdk.java.net > > > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah > > > > > > Thank you for the review again. > > > > > > Yes, not sure regarding backporting plans but I imagine it will be in > the > > > future so seemed best to avoid anything that cause issues such as > Set.of(). > > > > > > I also think the naming of the jfr's is fine as it's stored in a folder > > > named after the test which was easily found now that I'm aware that the > > > jfrs are already dumped. > > > > > > I believe I've addressed the rest of your recommended changes to the > > tests, > > > let me know if I've covered everything. > > > > > > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ > > > > > > Thanks, > > > > > > Ken Dobson > > > > > > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin > > wrote: > > > > > > > Thanks for fixing! > > > > > > > > Product code looks good, but I think tests could be improved. > > > > > > > > There is no need to call dump in a final clause. > > > > > > > > Events.fromRecording(...) will dump the file into the jtreg scratch > > > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is > > > > sufficient since there is only one recording per test. If you think > the > > > > name is cryptic, consider adding another method to > Event.fromRecording > > > that > > > > takes an additional parameter that becomes a suffix, i.e > > > > "recording-1-pid-xxx-test-heap-region-info.jfr" > > > > > > > > try (Recording r = new recording()) { > > > > r.enable(EVENT_NAME); > > > > r.start(); > > > > r.stop() > > > > List events = Events.fromRecording(r); > > > > Events.hasEvents(events); > > > > for (RecordedEvent event : events) { > > > > Events.assertField(event, "index").notEqual(-1); > > > > Events.assertField(event, "used").atMost(1024*1024L); > > > > String state = Events.assertField(event, "state").getValue(); > > > > GCHelper.assertShenandoahHeapRegionState(state)); > > > > } > > > > > > > > The name of the field in metadata.xml is "state", but the tests > looks for > > > > "type". That seems incorrect. > > > > > > > > GCHelper: > > > > > > > > public static assertShenandoahHeapRegionState(String state) { > > > > if (!shenandoahHeapRegionStates.contains(state)) { > > > > throw new AssertionError("Unknown state '"+ state +"', valid heap > > > > region states are " + shenandoahHeapRegionStates); > > > > } > > > > > > > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") > > > > because you want to backport to JDK 8. > > > > > > > > Thanks > > > > Erik > > > > > > > > Thanks everyone for their reviews. I've added tests similar to the > tests > > > > for the G1 events as well as addressed the changes that Erik > > > recommended. > > > > > > > > Here is the new webrev: > > > > > > > > > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ > > > > > > > > Please let me know if there's anything else that should be addressed. > > > > > > > > Thanks, > > > > > > > > Ken Dobson > > > > > > > > On Mon, May 6, 2019 at 11:38 AM > > > wrote: > > > > > > > >> > > > >> > > > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: > > > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: > > > >> >> Hi, > > > >> >> > > > >> >> If possible and feasible, could you please implement tests for > new > > > >> >> events? > > > >> >> > > > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > > > >> >> create a "shenandoah" subfolder. > > > >> >> > > > >> >> If the events are too hard to test or not feasible, please add > the > > > >> >> event names to > > > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, > > to > > > >> >> the hardToTestEvents set. Consider commenting why it is too hard > to > > > >> >> test. > > > >> > > > > >> > If events are are marked experimental, tests are ignored by > > > >> > TestLookForUntestedEvents. java, so we should not add them to the > > > >> > hardToTestEvents set. That may help them pass under the radar once > > > the > > > >> > experimental attribute is removed. > > > >> Thank you. If the events are "experimental" this approach makes > sense. > > > >> Once the experimental status is removed, the > > > >> TestLookForUntestedEvents.java should be able to notice if the > event is > > > >> not covered by tests. > > > >> > That said, tests never hurts, even if they are experimental. I > think > > > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place > > > >> +1 > > > >> > > > >> Thank you, > > > >> Misha > > > >> > > > > >> > Erik > > > >> > > > > >> >> > > > >> >> Also, please make sure to run all jfr tests under > test/jdk/jdk/jfr/ > > > >> >> prior to integration. > > > >> >> > > > >> >> > > > >> >> Best regards, > > > >> >> > > > >> >> Misha > > > >> >> > > > >> >> > > > >> >> > > > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: > > > >> >>> Hi all, > > > >> >>> > > > >> >>> Please review this patch that adds support for two new JFR > events > > > >> >>> ShenandoahHeapRegionStateChange and > > > ShenandoahHeapRegionInformation. > > > >> >>> > > > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > > > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > > >> >>> > > > >> >>> The Shenandoah team has also reviewed this patch and approved it > > > >> >>> from their > > > >> >>> end. > > > >> >>> > > > >> >>> Thanks, > > > >> >>> > > > >> >>> Ken Dobson > > > >> >> > > > >> > > > > >> > > > >> > > > > > From shade at redhat.com Wed May 22 07:02:51 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 22 May 2019 09:02:51 +0200 Subject: RFR(XS): 8224573: Fix windows build after JDK-8221507 In-Reply-To: References: Message-ID: <576c1f88-ba2b-82cc-c604-ebbb0416ca30@redhat.com> On 5/22/19 8:59 AM, Thomas St?fe wrote: > 32bit should be enough for number of regions I think. If I am not mistaken, > they have a min region size of 256K, so this covers a heap of ~1000 tb? Yes, u4 should be enough. That would require some light metadata refactoring and testing, so I am fine with doing that in a separate follow-up fix. -Aleksey From christoph.langer at sap.com Wed May 22 07:17:00 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 22 May 2019 07:17:00 +0000 Subject: RFR(XS): 8224573: Fix windows build after JDK-8221507 In-Reply-To: <576c1f88-ba2b-82cc-c604-ebbb0416ca30@redhat.com> References: <576c1f88-ba2b-82cc-c604-ebbb0416ca30@redhat.com> Message-ID: OK, thanks guys. Will push after my builds are done. > -----Original Message----- > From: Aleksey Shipilev > Sent: Mittwoch, 22. Mai 2019 09:03 > To: Thomas St?fe ; Langer, Christoph > > Cc: hotspot-jfr-dev at openjdk.java.net; Ken Dobson > ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(XS): 8224573: Fix windows build after JDK-8221507 > > On 5/22/19 8:59 AM, Thomas St?fe wrote: > > 32bit should be enough for number of regions I think. If I am not mistaken, > > they have a min region size of 256K, so this covers a heap of ~1000 tb? > > Yes, u4 should be enough. That would require some light metadata > refactoring and testing, so I am > fine with doing that in a separate follow-up fix. > > -Aleksey From christoph.langer at sap.com Wed May 22 07:44:25 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 22 May 2019 07:44:25 +0000 Subject: Build error on Windows after 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: Hi Thomas, I believe it was not caught because Oracle/jdk-submit builds with Shenandoah turned off? /Christoph From: Thomas St?fe Sent: Mittwoch, 22. Mai 2019 07:00 To: Langer, Christoph Cc: Ken Dobson ; hotspot-jfr-dev at openjdk.java.net Subject: Re: Build error on Windows after 8221507: Implement JFR Events for Shenandoah I am confused how this can even happen, should jdk-submit tests not have captured that before pushing? Cheers, Thomas On Tue, May 21, 2019 at 10:02 PM Langer, Christoph > wrote: Hi Ken, after you pushed JDK-8221507, I see build errors on Windows (enabled warnings as errors): shenandoahHeapRegion.cpp .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): error C2220: warning treated as error - no 'object' file generated .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data ... (rest of output omitted) * For target hotspot_variant-server_libjvm_objs_shenandoahJfrSupport.obj: shenandoahJfrSupport.cpp .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): error C2220: warning treated as error - no 'object' file generated .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data ... (rest of output omitted) I don't know if the fix is to simply add a cast to unsigned int. Can you please check and repair? Thanks Christoph > -----Original Message----- > From: jmc-dev > On Behalf Of Ken > Dobson > Sent: Dienstag, 14. Mai 2019 23:19 > To: Erik Gahlin > > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; jmc- > dev at openjdk.java.net > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah > > Thank you for the review again. > > Yes, not sure regarding backporting plans but I imagine it will be in the > future so seemed best to avoid anything that cause issues such as Set.of(). > > I also think the naming of the jfr's is fine as it's stored in a folder > named after the test which was easily found now that I'm aware that the > jfrs are already dumped. > > I believe I've addressed the rest of your recommended changes to the tests, > let me know if I've covered everything. > > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ > > Thanks, > > Ken Dobson > > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin > wrote: > > > Thanks for fixing! > > > > Product code looks good, but I think tests could be improved. > > > > There is no need to call dump in a final clause. > > > > Events.fromRecording(...) will dump the file into the jtreg scratch > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is > > sufficient since there is only one recording per test. If you think the > > name is cryptic, consider adding another method to Event.fromRecording > that > > takes an additional parameter that becomes a suffix, i.e > > "recording-1-pid-xxx-test-heap-region-info.jfr" > > > > try (Recording r = new recording()) { > > r.enable(EVENT_NAME); > > r.start(); > > r.stop() > > List events = Events.fromRecording(r); > > Events.hasEvents(events); > > for (RecordedEvent event : events) { > > Events.assertField(event, "index").notEqual(-1); > > Events.assertField(event, "used").atMost(1024*1024L); > > String state = Events.assertField(event, "state").getValue(); > > GCHelper.assertShenandoahHeapRegionState(state)); > > } > > > > The name of the field in metadata.xml is "state", but the tests looks for > > "type". That seems incorrect. > > > > GCHelper: > > > > public static assertShenandoahHeapRegionState(String state) { > > if (!shenandoahHeapRegionStates.contains(state)) { > > throw new AssertionError("Unknown state '"+ state +"', valid heap > > region states are " + shenandoahHeapRegionStates); > > } > > > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") > > because you want to backport to JDK 8. > > > > Thanks > > Erik > > > > Thanks everyone for their reviews. I've added tests similar to the tests > > for the G1 events as well as addressed the changes that Erik > recommended. > > > > Here is the new webrev: > > > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ > > > > Please let me know if there's anything else that should be addressed. > > > > Thanks, > > > > Ken Dobson > > > > On Mon, May 6, 2019 at 11:38 AM > > wrote: > > > >> > >> > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: > >> >> Hi, > >> >> > >> >> If possible and feasible, could you please implement tests for new > >> >> events? > >> >> > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > >> >> create a "shenandoah" subfolder. > >> >> > >> >> If the events are too hard to test or not feasible, please add the > >> >> event names to > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to > >> >> the hardToTestEvents set. Consider commenting why it is too hard to > >> >> test. > >> > > >> > If events are are marked experimental, tests are ignored by > >> > TestLookForUntestedEvents. java, so we should not add them to the > >> > hardToTestEvents set. That may help them pass under the radar once > the > >> > experimental attribute is removed. > >> Thank you. If the events are "experimental" this approach makes sense. > >> Once the experimental status is removed, the > >> TestLookForUntestedEvents.java should be able to notice if the event is > >> not covered by tests. > >> > That said, tests never hurts, even if they are experimental. I think > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place > >> +1 > >> > >> Thank you, > >> Misha > >> > > >> > Erik > >> > > >> >> > >> >> Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ > >> >> prior to integration. > >> >> > >> >> > >> >> Best regards, > >> >> > >> >> Misha > >> >> > >> >> > >> >> > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: > >> >>> Hi all, > >> >>> > >> >>> Please review this patch that adds support for two new JFR events > >> >>> ShenandoahHeapRegionStateChange and > ShenandoahHeapRegionInformation. > >> >>> > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > >> >>> > >> >>> The Shenandoah team has also reviewed this patch and approved it > >> >>> from their > >> >>> end. > >> >>> > >> >>> Thanks, > >> >>> > >> >>> Ken Dobson > >> >> > >> > > >> > >> > > From thomas.stuefe at gmail.com Wed May 22 07:46:56 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 22 May 2019 09:46:56 +0200 Subject: Build error on Windows after 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: Ah, that explains it. Since I rely on jdk-submit to catch my errors too, I got nervous for a moment. On Wed, May 22, 2019 at 9:44 AM Langer, Christoph wrote: > Hi Thomas, > > > > I believe it was not caught because Oracle/jdk-submit builds with > Shenandoah turned off? > > > > /Christoph > > > > *From:* Thomas St?fe > *Sent:* Mittwoch, 22. Mai 2019 07:00 > *To:* Langer, Christoph > *Cc:* Ken Dobson ; hotspot-jfr-dev at openjdk.java.net > *Subject:* Re: Build error on Windows after 8221507: Implement JFR Events > for Shenandoah > > > > I am confused how this can even happen, should jdk-submit tests not have > captured that before pushing? > > > > Cheers, Thomas > > > > On Tue, May 21, 2019 at 10:02 PM Langer, Christoph < > christoph.langer at sap.com> wrote: > > Hi Ken, > > after you pushed JDK-8221507, I see build errors on Windows (enabled > warnings as errors): > > shenandoahHeapRegion.cpp > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): > error C2220: warning treated as error - no 'object' file generated > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): > warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', > possible loss of data > ... (rest of output omitted) > * For target hotspot_variant-server_libjvm_objs_shenandoahJfrSupport.obj: > shenandoahJfrSupport.cpp > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): > error C2220: warning treated as error - no 'object' file generated > .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): > warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', > possible loss of data > ... (rest of output omitted) > > I don't know if the fix is to simply add a cast to unsigned int. Can you > please check and repair? > > Thanks > Christoph > > > > -----Original Message----- > > From: jmc-dev On Behalf Of Ken > > Dobson > > Sent: Dienstag, 14. Mai 2019 23:19 > > To: Erik Gahlin > > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; > jmc- > > dev at openjdk.java.net > > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah > > > > Thank you for the review again. > > > > Yes, not sure regarding backporting plans but I imagine it will be in the > > future so seemed best to avoid anything that cause issues such as > Set.of(). > > > > I also think the naming of the jfr's is fine as it's stored in a folder > > named after the test which was easily found now that I'm aware that the > > jfrs are already dumped. > > > > I believe I've addressed the rest of your recommended changes to the > tests, > > let me know if I've covered everything. > > > > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ > > > > Thanks, > > > > Ken Dobson > > > > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin > wrote: > > > > > Thanks for fixing! > > > > > > Product code looks good, but I think tests could be improved. > > > > > > There is no need to call dump in a final clause. > > > > > > Events.fromRecording(...) will dump the file into the jtreg scratch > > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is > > > sufficient since there is only one recording per test. If you think > the > > > name is cryptic, consider adding another method to Event.fromRecording > > that > > > takes an additional parameter that becomes a suffix, i.e > > > "recording-1-pid-xxx-test-heap-region-info.jfr" > > > > > > try (Recording r = new recording()) { > > > r.enable(EVENT_NAME); > > > r.start(); > > > r.stop() > > > List events = Events.fromRecording(r); > > > Events.hasEvents(events); > > > for (RecordedEvent event : events) { > > > Events.assertField(event, "index").notEqual(-1); > > > Events.assertField(event, "used").atMost(1024*1024L); > > > String state = Events.assertField(event, "state").getValue(); > > > GCHelper.assertShenandoahHeapRegionState(state)); > > > } > > > > > > The name of the field in metadata.xml is "state", but the tests looks > for > > > "type". That seems incorrect. > > > > > > GCHelper: > > > > > > public static assertShenandoahHeapRegionState(String state) { > > > if (!shenandoahHeapRegionStates.contains(state)) { > > > throw new AssertionError("Unknown state '"+ state +"', valid heap > > > region states are " + shenandoahHeapRegionStates); > > > } > > > > > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") > > > because you want to backport to JDK 8. > > > > > > Thanks > > > Erik > > > > > > Thanks everyone for their reviews. I've added tests similar to the > tests > > > for the G1 events as well as addressed the changes that Erik > > recommended. > > > > > > Here is the new webrev: > > > > > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ > > > > > > Please let me know if there's anything else that should be addressed. > > > > > > Thanks, > > > > > > Ken Dobson > > > > > > On Mon, May 6, 2019 at 11:38 AM > > wrote: > > > > > >> > > >> > > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: > > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: > > >> >> Hi, > > >> >> > > >> >> If possible and feasible, could you please implement tests for > new > > >> >> events? > > >> >> > > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps > > >> >> create a "shenandoah" subfolder. > > >> >> > > >> >> If the events are too hard to test or not feasible, please add the > > >> >> event names to > > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to > > >> >> the hardToTestEvents set. Consider commenting why it is too hard to > > >> >> test. > > >> > > > >> > If events are are marked experimental, tests are ignored by > > >> > TestLookForUntestedEvents. java, so we should not add them to the > > >> > hardToTestEvents set. That may help them pass under the radar once > > the > > >> > experimental attribute is removed. > > >> Thank you. If the events are "experimental" this approach makes sense. > > >> Once the experimental status is removed, the > > >> TestLookForUntestedEvents.java should be able to notice if the event > is > > >> not covered by tests. > > >> > That said, tests never hurts, even if they are experimental. I think > > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place > > >> +1 > > >> > > >> Thank you, > > >> Misha > > >> > > > >> > Erik > > >> > > > >> >> > > >> >> Also, please make sure to run all jfr tests under test/jdk/jdk/jfr/ > > >> >> prior to integration. > > >> >> > > >> >> > > >> >> Best regards, > > >> >> > > >> >> Misha > > >> >> > > >> >> > > >> >> > > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: > > >> >>> Hi all, > > >> >>> > > >> >>> Please review this patch that adds support for two new JFR events > > >> >>> ShenandoahHeapRegionStateChange and > > ShenandoahHeapRegionInformation. > > >> >>> > > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > >> >>> > > >> >>> The Shenandoah team has also reviewed this patch and approved it > > >> >>> from their > > >> >>> end. > > >> >>> > > >> >>> Thanks, > > >> >>> > > >> >>> Ken Dobson > > >> >> > > >> > > > >> > > >> > > > > > From markus.gronlund at oracle.com Wed May 22 09:09:09 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 22 May 2019 09:09:09 +0000 (UTC) Subject: RFR(XS): 8221121: applications/microbenchmarks are encountering crashes in tier5 In-Reply-To: <5CC348AB.7080704@oracle.com> References: <1515916c-6187-46d4-8815-5e824284e04b@default> <5CC348AB.7080704@oracle.com> Message-ID: Thanks Erik, Can I please have yet another reviewer for this change? Thanks in advance Markus -----Original Message----- From: Erik Gahlin Sent: den 26 april 2019 20:07 To: Markus Gronlund ; Hotspot dev runtime ; hotspot-jfr-dev at openjdk.java.net Subject: Re: RFR(XS): 8221121: applications/microbenchmarks are encountering crashes in tier5 Looks good. Erik > Greetings, > > Please review this small patch to address the following: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8221121 > Webrev: http://cr.openjdk.java.net/~mgronlun/8221121/webrev01/ > > Description: > > The applications/microbenchmarks added to tier5 are failing in some instances (debug builds), with, as an example, the following trace: > > # > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error > (/scratch/opt/mach5/mesos/work_dir/slaves/2dd962d0-8988-479b-a804-57ab > 764ada59-S77631/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/e > xecutors/d8f4cb38-0dec-4477-a89a-f62853433c56/runs/cb3f55fa-9e13-481c- > a7d6-9f33d2b8b457/workspace/open/src/hotspot/share/jfr/recorder/storag > e/jfrMemorySpace.inline.hpp:85), pid=5748, tid=5770 # > assert(t->identity() == __null) failed: invariant > > This occurred when JFR was running with the in-memory configuration where buffers are reused FIFO-style. > In the implementation, an "age node" will manage a full buffer for its reclamation, and age nodes provides for a linked "full" (fifo) list. > > Issue: > The age node was not expected to retain an identity after being added to the full list, where the assertion fired during the subsequent discard-reuse processing step. > This situation only manifests with running JFR in-memory configurations and using debug builds. > > Fix is to release the age node before insertion onto full list. > > Thanks > Markus From david.holmes at oracle.com Wed May 22 10:18:41 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 May 2019 20:18:41 +1000 Subject: RFR(XS): 8221121: applications/microbenchmarks are encountering crashes in tier5 In-Reply-To: References: <1515916c-6187-46d4-8815-5e824284e04b@default> <5CC348AB.7080704@oracle.com> Message-ID: Hi Markus, On 22/05/2019 7:09 pm, Markus Gronlund wrote: > Thanks Erik, > > Can I please have yet another reviewer for this change? Seems reasonable based on your description of the issue. Thanks, David > Thanks in advance > Markus > > -----Original Message----- > From: Erik Gahlin > Sent: den 26 april 2019 20:07 > To: Markus Gronlund ; Hotspot dev runtime ; hotspot-jfr-dev at openjdk.java.net > Subject: Re: RFR(XS): 8221121: applications/microbenchmarks are encountering crashes in tier5 > > Looks good. > > Erik > >> Greetings, >> >> Please review this small patch to address the following: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8221121 >> Webrev: http://cr.openjdk.java.net/~mgronlun/8221121/webrev01/ >> >> Description: >> >> The applications/microbenchmarks added to tier5 are failing in some instances (debug builds), with, as an example, the following trace: >> >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> # Internal Error >> (/scratch/opt/mach5/mesos/work_dir/slaves/2dd962d0-8988-479b-a804-57ab >> 764ada59-S77631/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/e >> xecutors/d8f4cb38-0dec-4477-a89a-f62853433c56/runs/cb3f55fa-9e13-481c- >> a7d6-9f33d2b8b457/workspace/open/src/hotspot/share/jfr/recorder/storag >> e/jfrMemorySpace.inline.hpp:85), pid=5748, tid=5770 # >> assert(t->identity() == __null) failed: invariant >> >> This occurred when JFR was running with the in-memory configuration where buffers are reused FIFO-style. >> In the implementation, an "age node" will manage a full buffer for its reclamation, and age nodes provides for a linked "full" (fifo) list. >> >> Issue: >> The age node was not expected to retain an identity after being added to the full list, where the assertion fired during the subsequent discard-reuse processing step. >> This situation only manifests with running JFR in-memory configurations and using debug builds. >> >> Fix is to release the age node before insertion onto full list. >> >> Thanks >> Markus > From markus.gronlund at oracle.com Wed May 22 10:20:50 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 22 May 2019 10:20:50 +0000 (UTC) Subject: RFR(XS): 8221121: applications/microbenchmarks are encountering crashes in tier5 In-Reply-To: References: <1515916c-6187-46d4-8815-5e824284e04b@default> <5CC348AB.7080704@oracle.com> Message-ID: <2ec701bf-f916-4d4c-a1f3-6676b497ba84@default> Thanks David for taking a look, appreciate it. Cheers Markus -----Original Message----- From: David Holmes Sent: den 22 maj 2019 12:19 To: Markus Gronlund ; Erik Gahlin ; Hotspot dev runtime ; hotspot-jfr-dev at openjdk.java.net Subject: Re: RFR(XS): 8221121: applications/microbenchmarks are encountering crashes in tier5 Hi Markus, On 22/05/2019 7:09 pm, Markus Gronlund wrote: > Thanks Erik, > > Can I please have yet another reviewer for this change? Seems reasonable based on your description of the issue. Thanks, David > Thanks in advance > Markus > > -----Original Message----- > From: Erik Gahlin > Sent: den 26 april 2019 20:07 > To: Markus Gronlund ; Hotspot dev runtime > ; > hotspot-jfr-dev at openjdk.java.net > Subject: Re: RFR(XS): 8221121: applications/microbenchmarks are > encountering crashes in tier5 > > Looks good. > > Erik > >> Greetings, >> >> Please review this small patch to address the following: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8221121 >> Webrev: http://cr.openjdk.java.net/~mgronlun/8221121/webrev01/ >> >> Description: >> >> The applications/microbenchmarks added to tier5 are failing in some instances (debug builds), with, as an example, the following trace: >> >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> # Internal Error >> (/scratch/opt/mach5/mesos/work_dir/slaves/2dd962d0-8988-479b-a804-57a >> b >> 764ada59-S77631/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/ >> e >> xecutors/d8f4cb38-0dec-4477-a89a-f62853433c56/runs/cb3f55fa-9e13-481c >> - >> a7d6-9f33d2b8b457/workspace/open/src/hotspot/share/jfr/recorder/stora >> g e/jfrMemorySpace.inline.hpp:85), pid=5748, tid=5770 # >> assert(t->identity() == __null) failed: invariant >> >> This occurred when JFR was running with the in-memory configuration where buffers are reused FIFO-style. >> In the implementation, an "age node" will manage a full buffer for its reclamation, and age nodes provides for a linked "full" (fifo) list. >> >> Issue: >> The age node was not expected to retain an identity after being added to the full list, where the assertion fired during the subsequent discard-reuse processing step. >> This situation only manifests with running JFR in-memory configurations and using debug builds. >> >> Fix is to release the age node before insertion onto full list. >> >> Thanks >> Markus > From kdobson at redhat.com Wed May 22 15:11:36 2019 From: kdobson at redhat.com (Ken Dobson) Date: Wed, 22 May 2019 11:11:36 -0400 Subject: Build error on Windows after 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: Just looking into this now but it appears it's a casting issue and this is unlikely to be an issue unique to Windows. I'll open a bug and post a fix after I've verified. Ken Dobson On Wed, May 22, 2019 at 3:47 AM Thomas St?fe wrote: > Ah, that explains it. Since I rely on jdk-submit to catch my errors too, I > got nervous for a moment. > > On Wed, May 22, 2019 at 9:44 AM Langer, Christoph < > christoph.langer at sap.com> wrote: > >> Hi Thomas, >> >> >> >> I believe it was not caught because Oracle/jdk-submit builds with >> Shenandoah turned off? >> >> >> >> /Christoph >> >> >> >> *From:* Thomas St?fe >> *Sent:* Mittwoch, 22. Mai 2019 07:00 >> *To:* Langer, Christoph >> *Cc:* Ken Dobson ; hotspot-jfr-dev at openjdk.java.net >> *Subject:* Re: Build error on Windows after 8221507: Implement JFR >> Events for Shenandoah >> >> >> >> I am confused how this can even happen, should jdk-submit tests not have >> captured that before pushing? >> >> >> >> Cheers, Thomas >> >> >> >> On Tue, May 21, 2019 at 10:02 PM Langer, Christoph < >> christoph.langer at sap.com> wrote: >> >> Hi Ken, >> >> after you pushed JDK-8221507, I see build errors on Windows (enabled >> warnings as errors): >> >> shenandoahHeapRegion.cpp >> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): >> error C2220: warning treated as error - no 'object' file generated >> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): >> warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', >> possible loss of data >> ... (rest of output omitted) >> * For target hotspot_variant-server_libjvm_objs_shenandoahJfrSupport.obj: >> shenandoahJfrSupport.cpp >> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): >> error C2220: warning treated as error - no 'object' file generated >> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): >> warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', >> possible loss of data >> ... (rest of output omitted) >> >> I don't know if the fix is to simply add a cast to unsigned int. Can you >> please check and repair? >> >> Thanks >> Christoph >> >> >> > -----Original Message----- >> > From: jmc-dev On Behalf Of Ken >> > Dobson >> > Sent: Dienstag, 14. Mai 2019 23:19 >> > To: Erik Gahlin >> > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; >> jmc- >> > dev at openjdk.java.net >> > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah >> > >> > Thank you for the review again. >> > >> > Yes, not sure regarding backporting plans but I imagine it will be in >> the >> > future so seemed best to avoid anything that cause issues such as >> Set.of(). >> > >> > I also think the naming of the jfr's is fine as it's stored in a folder >> > named after the test which was easily found now that I'm aware that the >> > jfrs are already dumped. >> > >> > I believe I've addressed the rest of your recommended changes to the >> tests, >> > let me know if I've covered everything. >> > >> > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ >> > >> > Thanks, >> > >> > Ken Dobson >> > >> > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin >> wrote: >> > >> > > Thanks for fixing! >> > > >> > > Product code looks good, but I think tests could be improved. >> > > >> > > There is no need to call dump in a final clause. >> > > >> > > Events.fromRecording(...) will dump the file into the jtreg scratch >> > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is >> > > sufficient since there is only one recording per test. If you think >> the >> > > name is cryptic, consider adding another method to Event.fromRecording >> > that >> > > takes an additional parameter that becomes a suffix, i.e >> > > "recording-1-pid-xxx-test-heap-region-info.jfr" >> > > >> > > try (Recording r = new recording()) { >> > > r.enable(EVENT_NAME); >> > > r.start(); >> > > r.stop() >> > > List events = Events.fromRecording(r); >> > > Events.hasEvents(events); >> > > for (RecordedEvent event : events) { >> > > Events.assertField(event, "index").notEqual(-1); >> > > Events.assertField(event, "used").atMost(1024*1024L); >> > > String state = Events.assertField(event, "state").getValue(); >> > > GCHelper.assertShenandoahHeapRegionState(state)); >> > > } >> > > >> > > The name of the field in metadata.xml is "state", but the tests looks >> for >> > > "type". That seems incorrect. >> > > >> > > GCHelper: >> > > >> > > public static assertShenandoahHeapRegionState(String state) { >> > > if (!shenandoahHeapRegionStates.contains(state)) { >> > > throw new AssertionError("Unknown state '"+ state +"', valid heap >> > > region states are " + shenandoahHeapRegionStates); >> > > } >> > > >> > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") >> > > because you want to backport to JDK 8. >> > > >> > > Thanks >> > > Erik >> > > >> > > Thanks everyone for their reviews. I've added tests similar to the >> tests >> > > for the G1 events as well as addressed the changes that Erik >> > recommended. >> > > >> > > Here is the new webrev: >> > > >> > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ >> > > >> > > Please let me know if there's anything else that should be addressed. >> > > >> > > Thanks, >> > > >> > > Ken Dobson >> > > >> > > On Mon, May 6, 2019 at 11:38 AM >> > wrote: >> > > >> > >> >> > >> >> > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: >> > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: >> > >> >> Hi, >> > >> >> >> > >> >> If possible and feasible, could you please implement tests for >> new >> > >> >> events? >> > >> >> >> > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps >> > >> >> create a "shenandoah" subfolder. >> > >> >> >> > >> >> If the events are too hard to test or not feasible, please add the >> > >> >> event names to >> > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, to >> > >> >> the hardToTestEvents set. Consider commenting why it is too hard >> to >> > >> >> test. >> > >> > >> > >> > If events are are marked experimental, tests are ignored by >> > >> > TestLookForUntestedEvents. java, so we should not add them to the >> > >> > hardToTestEvents set. That may help them pass under the radar once >> > the >> > >> > experimental attribute is removed. >> > >> Thank you. If the events are "experimental" this approach makes >> sense. >> > >> Once the experimental status is removed, the >> > >> TestLookForUntestedEvents.java should be able to notice if the event >> is >> > >> not covered by tests. >> > >> > That said, tests never hurts, even if they are experimental. I >> think >> > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place >> > >> +1 >> > >> >> > >> Thank you, >> > >> Misha >> > >> > >> > >> > Erik >> > >> > >> > >> >> >> > >> >> Also, please make sure to run all jfr tests under >> test/jdk/jdk/jfr/ >> > >> >> prior to integration. >> > >> >> >> > >> >> >> > >> >> Best regards, >> > >> >> >> > >> >> Misha >> > >> >> >> > >> >> >> > >> >> >> > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: >> > >> >>> Hi all, >> > >> >>> >> > >> >>> Please review this patch that adds support for two new JFR events >> > >> >>> ShenandoahHeapRegionStateChange and >> > ShenandoahHeapRegionInformation. >> > >> >>> >> > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 >> > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ >> > >> >>> >> > >> >>> The Shenandoah team has also reviewed this patch and approved it >> > >> >>> from their >> > >> >>> end. >> > >> >>> >> > >> >>> Thanks, >> > >> >>> >> > >> >>> Ken Dobson >> > >> >> >> > >> > >> > >> >> > >> >> > > >> >> From thomas.stuefe at gmail.com Wed May 22 15:25:36 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 22 May 2019 17:25:36 +0200 Subject: Build error on Windows after 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: Hi Ken, I believe this is already fixed, see 8224573. Regards, Thomas On Wed, May 22, 2019 at 5:11 PM Ken Dobson wrote: > Just looking into this now but it appears it's a casting issue and this is > unlikely to be an issue unique to Windows. I'll open a bug and post a fix > after I've verified. > > Ken Dobson > > On Wed, May 22, 2019 at 3:47 AM Thomas St?fe > wrote: > >> Ah, that explains it. Since I rely on jdk-submit to catch my errors too, >> I got nervous for a moment. >> >> On Wed, May 22, 2019 at 9:44 AM Langer, Christoph < >> christoph.langer at sap.com> wrote: >> >>> Hi Thomas, >>> >>> >>> >>> I believe it was not caught because Oracle/jdk-submit builds with >>> Shenandoah turned off? >>> >>> >>> >>> /Christoph >>> >>> >>> >>> *From:* Thomas St?fe >>> *Sent:* Mittwoch, 22. Mai 2019 07:00 >>> *To:* Langer, Christoph >>> *Cc:* Ken Dobson ; hotspot-jfr-dev at openjdk.java.net >>> *Subject:* Re: Build error on Windows after 8221507: Implement JFR >>> Events for Shenandoah >>> >>> >>> >>> I am confused how this can even happen, should jdk-submit tests not have >>> captured that before pushing? >>> >>> >>> >>> Cheers, Thomas >>> >>> >>> >>> On Tue, May 21, 2019 at 10:02 PM Langer, Christoph < >>> christoph.langer at sap.com> wrote: >>> >>> Hi Ken, >>> >>> after you pushed JDK-8221507, I see build errors on Windows (enabled >>> warnings as errors): >>> >>> shenandoahHeapRegion.cpp >>> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): >>> error C2220: warning treated as error - no 'object' file generated >>> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): >>> warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', >>> possible loss of data >>> ... (rest of output omitted) >>> * For target hotspot_variant-server_libjvm_objs_shenandoahJfrSupport.obj: >>> shenandoahJfrSupport.cpp >>> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): >>> error C2220: warning treated as error - no 'object' file generated >>> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): >>> warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', >>> possible loss of data >>> ... (rest of output omitted) >>> >>> I don't know if the fix is to simply add a cast to unsigned int. Can you >>> please check and repair? >>> >>> Thanks >>> Christoph >>> >>> >>> > -----Original Message----- >>> > From: jmc-dev On Behalf Of Ken >>> > Dobson >>> > Sent: Dienstag, 14. Mai 2019 23:19 >>> > To: Erik Gahlin >>> > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; >>> jmc- >>> > dev at openjdk.java.net >>> > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah >>> > >>> > Thank you for the review again. >>> > >>> > Yes, not sure regarding backporting plans but I imagine it will be in >>> the >>> > future so seemed best to avoid anything that cause issues such as >>> Set.of(). >>> > >>> > I also think the naming of the jfr's is fine as it's stored in a folder >>> > named after the test which was easily found now that I'm aware that the >>> > jfrs are already dumped. >>> > >>> > I believe I've addressed the rest of your recommended changes to the >>> tests, >>> > let me know if I've covered everything. >>> > >>> > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ >>> > >>> > Thanks, >>> > >>> > Ken Dobson >>> > >>> > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin >>> wrote: >>> > >>> > > Thanks for fixing! >>> > > >>> > > Product code looks good, but I think tests could be improved. >>> > > >>> > > There is no need to call dump in a final clause. >>> > > >>> > > Events.fromRecording(...) will dump the file into the jtreg scratch >>> > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is >>> > > sufficient since there is only one recording per test. If you think >>> the >>> > > name is cryptic, consider adding another method to >>> Event.fromRecording >>> > that >>> > > takes an additional parameter that becomes a suffix, i.e >>> > > "recording-1-pid-xxx-test-heap-region-info.jfr" >>> > > >>> > > try (Recording r = new recording()) { >>> > > r.enable(EVENT_NAME); >>> > > r.start(); >>> > > r.stop() >>> > > List events = Events.fromRecording(r); >>> > > Events.hasEvents(events); >>> > > for (RecordedEvent event : events) { >>> > > Events.assertField(event, "index").notEqual(-1); >>> > > Events.assertField(event, "used").atMost(1024*1024L); >>> > > String state = Events.assertField(event, "state").getValue(); >>> > > GCHelper.assertShenandoahHeapRegionState(state)); >>> > > } >>> > > >>> > > The name of the field in metadata.xml is "state", but the tests >>> looks for >>> > > "type". That seems incorrect. >>> > > >>> > > GCHelper: >>> > > >>> > > public static assertShenandoahHeapRegionState(String state) { >>> > > if (!shenandoahHeapRegionStates.contains(state)) { >>> > > throw new AssertionError("Unknown state '"+ state +"', valid heap >>> > > region states are " + shenandoahHeapRegionStates); >>> > > } >>> > > >>> > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") >>> > > because you want to backport to JDK 8. >>> > > >>> > > Thanks >>> > > Erik >>> > > >>> > > Thanks everyone for their reviews. I've added tests similar to the >>> tests >>> > > for the G1 events as well as addressed the changes that Erik >>> > recommended. >>> > > >>> > > Here is the new webrev: >>> > > >>> > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ >>> > > >>> > > Please let me know if there's anything else that should be addressed. >>> > > >>> > > Thanks, >>> > > >>> > > Ken Dobson >>> > > >>> > > On Mon, May 6, 2019 at 11:38 AM >>> > wrote: >>> > > >>> > >> >>> > >> >>> > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: >>> > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: >>> > >> >> Hi, >>> > >> >> >>> > >> >> If possible and feasible, could you please implement tests for >>> new >>> > >> >> events? >>> > >> >> >>> > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps >>> > >> >> create a "shenandoah" subfolder. >>> > >> >> >>> > >> >> If the events are too hard to test or not feasible, please add >>> the >>> > >> >> event names to >>> > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, >>> to >>> > >> >> the hardToTestEvents set. Consider commenting why it is too hard >>> to >>> > >> >> test. >>> > >> > >>> > >> > If events are are marked experimental, tests are ignored by >>> > >> > TestLookForUntestedEvents. java, so we should not add them to the >>> > >> > hardToTestEvents set. That may help them pass under the radar once >>> > the >>> > >> > experimental attribute is removed. >>> > >> Thank you. If the events are "experimental" this approach makes >>> sense. >>> > >> Once the experimental status is removed, the >>> > >> TestLookForUntestedEvents.java should be able to notice if the >>> event is >>> > >> not covered by tests. >>> > >> > That said, tests never hurts, even if they are experimental. I >>> think >>> > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place >>> > >> +1 >>> > >> >>> > >> Thank you, >>> > >> Misha >>> > >> > >>> > >> > Erik >>> > >> > >>> > >> >> >>> > >> >> Also, please make sure to run all jfr tests under >>> test/jdk/jdk/jfr/ >>> > >> >> prior to integration. >>> > >> >> >>> > >> >> >>> > >> >> Best regards, >>> > >> >> >>> > >> >> Misha >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: >>> > >> >>> Hi all, >>> > >> >>> >>> > >> >>> Please review this patch that adds support for two new JFR >>> events >>> > >> >>> ShenandoahHeapRegionStateChange and >>> > ShenandoahHeapRegionInformation. >>> > >> >>> >>> > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 >>> > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ >>> > >> >>> >>> > >> >>> The Shenandoah team has also reviewed this patch and approved it >>> > >> >>> from their >>> > >> >>> end. >>> > >> >>> >>> > >> >>> Thanks, >>> > >> >>> >>> > >> >>> Ken Dobson >>> > >> >> >>> > >> > >>> > >> >>> > >> >>> > > >>> >>> From kdobson at redhat.com Wed May 22 15:44:16 2019 From: kdobson at redhat.com (Ken Dobson) Date: Wed, 22 May 2019 11:44:16 -0400 Subject: Build error on Windows after 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: Well that's easy then, thanks Christoph. Ken Dobson On Wed, May 22, 2019 at 11:26 AM Thomas St?fe wrote: > Hi Ken, I believe this is already fixed, see 8224573. > > Regards, Thomas > > On Wed, May 22, 2019 at 5:11 PM Ken Dobson wrote: > >> Just looking into this now but it appears it's a casting issue and this >> is unlikely to be an issue unique to Windows. I'll open a bug and post a >> fix after I've verified. >> >> Ken Dobson >> >> On Wed, May 22, 2019 at 3:47 AM Thomas St?fe >> wrote: >> >>> Ah, that explains it. Since I rely on jdk-submit to catch my errors too, >>> I got nervous for a moment. >>> >>> On Wed, May 22, 2019 at 9:44 AM Langer, Christoph < >>> christoph.langer at sap.com> wrote: >>> >>>> Hi Thomas, >>>> >>>> >>>> >>>> I believe it was not caught because Oracle/jdk-submit builds with >>>> Shenandoah turned off? >>>> >>>> >>>> >>>> /Christoph >>>> >>>> >>>> >>>> *From:* Thomas St?fe >>>> *Sent:* Mittwoch, 22. Mai 2019 07:00 >>>> *To:* Langer, Christoph >>>> *Cc:* Ken Dobson ; hotspot-jfr-dev at openjdk.java.net >>>> *Subject:* Re: Build error on Windows after 8221507: Implement JFR >>>> Events for Shenandoah >>>> >>>> >>>> >>>> I am confused how this can even happen, should jdk-submit tests not >>>> have captured that before pushing? >>>> >>>> >>>> >>>> Cheers, Thomas >>>> >>>> >>>> >>>> On Tue, May 21, 2019 at 10:02 PM Langer, Christoph < >>>> christoph.langer at sap.com> wrote: >>>> >>>> Hi Ken, >>>> >>>> after you pushed JDK-8221507, I see build errors on Windows (enabled >>>> warnings as errors): >>>> >>>> shenandoahHeapRegion.cpp >>>> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): >>>> error C2220: warning treated as error - no 'object' file generated >>>> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp(687): >>>> warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', >>>> possible loss of data >>>> ... (rest of output omitted) >>>> * For target >>>> hotspot_variant-server_libjvm_objs_shenandoahJfrSupport.obj: >>>> shenandoahJfrSupport.cpp >>>> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): >>>> error C2220: warning treated as error - no 'object' file generated >>>> .../jdk/src/hotspot/share/gc/shenandoah/shenandoahJfrSupport.cpp(60): >>>> warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', >>>> possible loss of data >>>> ... (rest of output omitted) >>>> >>>> I don't know if the fix is to simply add a cast to unsigned int. Can >>>> you please check and repair? >>>> >>>> Thanks >>>> Christoph >>>> >>>> >>>> > -----Original Message----- >>>> > From: jmc-dev On Behalf Of Ken >>>> > Dobson >>>> > Sent: Dienstag, 14. Mai 2019 23:19 >>>> > To: Erik Gahlin >>>> > Cc: hotspot-jfr-dev at openjdk.java.net; mikhailo.seledtsov at oracle.com; >>>> jmc- >>>> > dev at openjdk.java.net >>>> > Subject: Re: RFR 8221507: Implement JFR Events for Shenandoah >>>> > >>>> > Thank you for the review again. >>>> > >>>> > Yes, not sure regarding backporting plans but I imagine it will be in >>>> the >>>> > future so seemed best to avoid anything that cause issues such as >>>> Set.of(). >>>> > >>>> > I also think the naming of the jfr's is fine as it's stored in a >>>> folder >>>> > named after the test which was easily found now that I'm aware that >>>> the >>>> > jfrs are already dumped. >>>> > >>>> > I believe I've addressed the rest of your recommended changes to the >>>> tests, >>>> > let me know if I've covered everything. >>>> > >>>> > http://cr.openjdk.java.net/~kdobson/eventswithtests/webrev/ >>>> > >>>> > Thanks, >>>> > >>>> > Ken Dobson >>>> > >>>> > On Sun, May 12, 2019 at 8:28 PM Erik Gahlin >>>> wrote: >>>> > >>>> > > Thanks for fixing! >>>> > > >>>> > > Product code looks good, but I think tests could be improved. >>>> > > >>>> > > There is no need to call dump in a final clause. >>>> > > >>>> > > Events.fromRecording(...) will dump the file into the jtreg scratch >>>> > > directory ( "recording-1-pid-xxx.jfr") by default. I think this is >>>> > > sufficient since there is only one recording per test. If you >>>> think the >>>> > > name is cryptic, consider adding another method to >>>> Event.fromRecording >>>> > that >>>> > > takes an additional parameter that becomes a suffix, i.e >>>> > > "recording-1-pid-xxx-test-heap-region-info.jfr" >>>> > > >>>> > > try (Recording r = new recording()) { >>>> > > r.enable(EVENT_NAME); >>>> > > r.start(); >>>> > > r.stop() >>>> > > List events = Events.fromRecording(r); >>>> > > Events.hasEvents(events); >>>> > > for (RecordedEvent event : events) { >>>> > > Events.assertField(event, "index").notEqual(-1); >>>> > > Events.assertField(event, "used").atMost(1024*1024L); >>>> > > String state = Events.assertField(event, "state").getValue(); >>>> > > GCHelper.assertShenandoahHeapRegionState(state)); >>>> > > } >>>> > > >>>> > > The name of the field in metadata.xml is "state", but the tests >>>> looks for >>>> > > "type". That seems incorrect. >>>> > > >>>> > > GCHelper: >>>> > > >>>> > > public static assertShenandoahHeapRegionState(String state) { >>>> > > if (!shenandoahHeapRegionStates.contains(state)) { >>>> > > throw new AssertionError("Unknown state '"+ state +"', valid >>>> heap >>>> > > region states are " + shenandoahHeapRegionStates); >>>> > > } >>>> > > >>>> > > I assume you don't use Set.of("Empty Uncommitted" , ... , "Trash") >>>> > > because you want to backport to JDK 8. >>>> > > >>>> > > Thanks >>>> > > Erik >>>> > > >>>> > > Thanks everyone for their reviews. I've added tests similar to the >>>> tests >>>> > > for the G1 events as well as addressed the changes that Erik >>>> > recommended. >>>> > > >>>> > > Here is the new webrev: >>>> > > >>>> > http://cr.openjdk.java.net/~kdobson/shenandoaheventswithtests/webrev/ >>>> > > >>>> > > Please let me know if there's anything else that should be >>>> addressed. >>>> > > >>>> > > Thanks, >>>> > > >>>> > > Ken Dobson >>>> > > >>>> > > On Mon, May 6, 2019 at 11:38 AM >>>> > wrote: >>>> > > >>>> > >> >>>> > >> >>>> > >> On 5/4/19 2:16 PM, Erik Gahlin wrote: >>>> > >> > On 2019-05-04 00:19, mikhailo.seledtsov at oracle.com wrote: >>>> > >> >> Hi, >>>> > >> >> >>>> > >> >> If possible and feasible, could you please implement tests >>>> for new >>>> > >> >> events? >>>> > >> >> >>>> > >> >> Please place the tests under test/jdk/jdk/jfr/event/gc/. Perhaps >>>> > >> >> create a "shenandoah" subfolder. >>>> > >> >> >>>> > >> >> If the events are too hard to test or not feasible, please add >>>> the >>>> > >> >> event names to >>>> > >> >> test/jdk/jdk/jfr/event/metadata/TestLookForUntestedEvents.java, >>>> to >>>> > >> >> the hardToTestEvents set. Consider commenting why it is too >>>> hard to >>>> > >> >> test. >>>> > >> > >>>> > >> > If events are are marked experimental, tests are ignored by >>>> > >> > TestLookForUntestedEvents. java, so we should not add them to >>>> the >>>> > >> > hardToTestEvents set. That may help them pass under the radar >>>> once >>>> > the >>>> > >> > experimental attribute is removed. >>>> > >> Thank you. If the events are "experimental" this approach makes >>>> sense. >>>> > >> Once the experimental status is removed, the >>>> > >> TestLookForUntestedEvents.java should be able to notice if the >>>> event is >>>> > >> not covered by tests. >>>> > >> > That said, tests never hurts, even if they are experimental. I >>>> think >>>> > >> > test/jdk/jdk/jfr/event/gc/detailed would be a good place >>>> > >> +1 >>>> > >> >>>> > >> Thank you, >>>> > >> Misha >>>> > >> > >>>> > >> > Erik >>>> > >> > >>>> > >> >> >>>> > >> >> Also, please make sure to run all jfr tests under >>>> test/jdk/jdk/jfr/ >>>> > >> >> prior to integration. >>>> > >> >> >>>> > >> >> >>>> > >> >> Best regards, >>>> > >> >> >>>> > >> >> Misha >>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> > >> >> On 5/3/19 7:44 AM, Ken Dobson wrote: >>>> > >> >>> Hi all, >>>> > >> >>> >>>> > >> >>> Please review this patch that adds support for two new JFR >>>> events >>>> > >> >>> ShenandoahHeapRegionStateChange and >>>> > ShenandoahHeapRegionInformation. >>>> > >> >>> >>>> > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 >>>> > >> >>> Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ >>>> > >> >>> >>>> > >> >>> The Shenandoah team has also reviewed this patch and approved >>>> it >>>> > >> >>> from their >>>> > >> >>> end. >>>> > >> >>> >>>> > >> >>> Thanks, >>>> > >> >>> >>>> > >> >>> Ken Dobson >>>> > >> >> >>>> > >> > >>>> > >> >>>> > >> >>>> > > >>>> >>>> From erik.gahlin at oracle.com Wed May 22 17:12:21 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 22 May 2019 19:12:21 +0200 Subject: RFR(XS): 8224529: [TESTBUG] JFR TestShenandoahHeapRegion* tests fail on build w/o Shenandoah In-Reply-To: <453a43cb-60e5-6dba-65e6-0cd6f2e6a48a@oracle.com> References: <453a43cb-60e5-6dba-65e6-0cd6f2e6a48a@oracle.com> Message-ID: <5CE582F5.7020702@oracle.com> Looks good. Erik > Please review this small change that fixes at-requires statement for > JFR tests that require Shenandoah GC. > Change also adds vm.gc.Shenandoah to JDK's TEST.ROOT > > JBS: https://bugs.openjdk.java.net/browse/JDK-8224529 > Webrev: http://cr.openjdk.java.net/~mseledtsov/8224529.00/ > Testing: > 1. Ran the affected tests on a build w/o Shenandoah: tests > skipped as expected > 2. Runnin entire jdk_jfr on a test cluster - in progress > > > Thank you, > Misha > From mikhailo.seledtsov at oracle.com Wed May 22 17:32:21 2019 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Wed, 22 May 2019 10:32:21 -0700 Subject: RFR(XS): 8224529: [TESTBUG] JFR TestShenandoahHeapRegion* tests fail on build w/o Shenandoah In-Reply-To: <5CE582F5.7020702@oracle.com> References: <453a43cb-60e5-6dba-65e6-0cd6f2e6a48a@oracle.com> <5CE582F5.7020702@oracle.com> Message-ID: <5CE587A5.9080502@oracle.com> Christoph, Erik, Thank you for review, Misha On 5/22/19, 10:12 AM, Erik Gahlin wrote: > Looks good. > > Erik > >> Please review this small change that fixes at-requires statement for >> JFR tests that require Shenandoah GC. >> Change also adds vm.gc.Shenandoah to JDK's TEST.ROOT >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8224529 >> Webrev: http://cr.openjdk.java.net/~mseledtsov/8224529.00/ >> Testing: >> 1. Ran the affected tests on a build w/o Shenandoah: tests >> skipped as expected >> 2. Runnin entire jdk_jfr on a test cluster - in progress >> >> >> Thank you, >> Misha >> > From erik.gahlin at oracle.com Tue May 28 18:42:14 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 28 May 2019 20:42:14 +0200 Subject: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup In-Reply-To: References: Message-ID: <5CED8106.1040702@oracle.com> Looks good. Erik > Greetings, > > Please review the following patch to improve startup when using JFR: > > Bug/Enh: https://bugs.openjdk.java.net/browse/JDK-8217089 > Webrev: http://cr.openjdk.java.net/~mgronlun/8217089/webrev01/ > > Summary: > > Let the periodic event requestor thread lazy install os interface components instead of doing it eagerly as part of startup by the initial thread. > > Thank you > Markus From markus.gronlund at oracle.com Tue May 28 20:37:50 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 28 May 2019 13:37:50 -0700 (PDT) Subject: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup In-Reply-To: References: Message-ID: <256a3874-6eca-4821-96f4-f1a35d33e676@default> Hi David, Sorry for the delay in getting back to you on this, thank you for taking a look. The existing callers are all located in jfrPeriodic.cpp and handles the OS_ERR return codes already (with logging). Thanks again Markus -----Original Message----- From: David Holmes Sent: den 10 maj 2019 04:32 To: Markus Gronlund ; Hotspot dev runtime ; hotspot-jfr-dev at openjdk.java.net Subject: Re: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup Hi Markus, On 5/05/2019 6:23 am, Markus Gronlund wrote: > Greetings, > > Please review the following patch to improve startup when using JFR: > > Bug/Enh: https://bugs.openjdk.java.net/browse/JDK-8217089 > Webrev: http://cr.openjdk.java.net/~mgronlun/8217089/webrev01/ > > Summary: > > Let the periodic event requestor thread lazy install os interface components instead of doing it eagerly as part of startup by the initial thread. That all seems fine for single-threaded lazy initialization. Only thing I'm unclear on is the addition of the potential to return OS_ERR. I can't see where these methods get called so can't tell if the callers are prepared to handle the error. Thanks, David ----- > Thank you > Markus > From david.holmes at oracle.com Wed May 29 07:34:07 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 29 May 2019 17:34:07 +1000 Subject: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup In-Reply-To: <256a3874-6eca-4821-96f4-f1a35d33e676@default> References: <256a3874-6eca-4821-96f4-f1a35d33e676@default> Message-ID: <84c3f691-5606-597f-51dc-377f6a1fbf4a@oracle.com> Hi Markus, Thanks for clarifying. David On 29/05/2019 6:37 am, Markus Gronlund wrote: > Hi David, > > Sorry for the delay in getting back to you on this, thank you for taking a look. > > The existing callers are all located in jfrPeriodic.cpp and handles the OS_ERR return codes already (with logging). > > Thanks again > Markus > > -----Original Message----- > From: David Holmes > Sent: den 10 maj 2019 04:32 > To: Markus Gronlund ; Hotspot dev runtime ; hotspot-jfr-dev at openjdk.java.net > Subject: Re: RFR(M): 8217089: JFR: Lazy install os interface components for improved startup > > Hi Markus, > > On 5/05/2019 6:23 am, Markus Gronlund wrote: >> Greetings, >> >> Please review the following patch to improve startup when using JFR: >> >> Bug/Enh: https://bugs.openjdk.java.net/browse/JDK-8217089 >> Webrev: http://cr.openjdk.java.net/~mgronlun/8217089/webrev01/ >> >> Summary: >> >> Let the periodic event requestor thread lazy install os interface components instead of doing it eagerly as part of startup by the initial thread. > > That all seems fine for single-threaded lazy initialization. > > Only thing I'm unclear on is the addition of the potential to return OS_ERR. I can't see where these methods get called so can't tell if the callers are prepared to handle the error. > > Thanks, > David > ----- > >> Thank you >> Markus >> From markus.gronlund at oracle.com Wed May 29 17:59:06 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 29 May 2019 10:59:06 -0700 (PDT) Subject: RFR(XXS): 8225004: Remove invalid assertion in jfr_conditional_flush() Message-ID: <85e6c70a-596f-451f-b15f-540636116ad3@default> Greetings, Please review this small change to remove an invalid assertion. Bug: https://bugs.openjdk.java.net/browse/JDK-8225004 Change: diff -r c1ad2862d0dd -r 0c17f2ea252e src/hotspot/share/jfr/support/jfrFlush.cpp --- a/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 09:53:28 2019 -0700 +++ b/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 19:04:34 2019 +0200 @@ -62,7 +62,6 @@ } void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { - assert(jfr_is_event_enabled(id), "invariant"); if (t->jfr_thread_local()->has_native_buffer()) { JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); if (LessThanSize::evaluate(buffer, size)) { Description: JfrConditionalFlush(Thread* t) { if (jfr_is_event_enabled(Event::eventId)) { <<--------------- evaluates to true jfr_conditional_flush(Event::eventId, sizeof(Event), t); } } void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { assert(jfr_is_event_enabled(id), "invariant"); <<-------------------- could evaluate to false if (t->jfr_thread_local()->has_native_buffer()) { JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); if (LessThanSize::evaluate(buffer, size)) { JfrFlush f(buffer, 0, 0, t); } } } Event settings are updated asynchronously, where event disablement happens after the recording has stopped. A thread could have evaluated the initial check to true to enter jfr_conditional_flush(). Depending on visibility, context switching and preemption, another thread could have disabled the event before the thread runs jfr_conditional_flush(). There is no validity to be asserted at this location, so the assertion needs to be removed. Thanks Markus From daniel.daugherty at oracle.com Wed May 29 18:39:04 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 29 May 2019 14:39:04 -0400 Subject: RFR(XXS): 8225004: Remove invalid assertion in jfr_conditional_flush() In-Reply-To: <85e6c70a-596f-451f-b15f-540636116ad3@default> References: <85e6c70a-596f-451f-b15f-540636116ad3@default> Message-ID: On 5/29/19 1:59 PM, Markus Gronlund wrote: > Greetings, > > Please review this small change to remove an invalid assertion. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8225004 > > Change: > > diff -r c1ad2862d0dd -r 0c17f2ea252e src/hotspot/share/jfr/support/jfrFlush.cpp > --- a/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 09:53:28 2019 -0700 > +++ b/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 19:04:34 2019 +0200 > @@ -62,7 +62,6 @@ > } > > void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { > - assert(jfr_is_event_enabled(id), "invariant"); > if (t->jfr_thread_local()->has_native_buffer()) { > JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); > if (LessThanSize::evaluate(buffer, size)) { Thumbs up. And thanks for the analysis below. If I remember correctly, JVM/TI async events and event handlers run into the same issue. :-) Dan > > > Description: > > JfrConditionalFlush(Thread* t) { > if (jfr_is_event_enabled(Event::eventId)) { <<--------------- evaluates to true > jfr_conditional_flush(Event::eventId, sizeof(Event), t); > } > } > > void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { > assert(jfr_is_event_enabled(id), "invariant"); <<-------------------- could evaluate to false > if (t->jfr_thread_local()->has_native_buffer()) { > JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); > if (LessThanSize::evaluate(buffer, size)) { > JfrFlush f(buffer, 0, 0, t); > } > } > } > > Event settings are updated asynchronously, where event disablement happens after the recording has stopped. > > A thread could have evaluated the initial check to true to enter jfr_conditional_flush(). Depending on visibility, context switching and preemption, another thread could have disabled the event before the thread runs jfr_conditional_flush(). > > There is no validity to be asserted at this location, so the assertion needs to be removed. > > Thanks > Markus From markus.gronlund at oracle.com Wed May 29 19:43:45 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 29 May 2019 12:43:45 -0700 (PDT) Subject: RFR(XXS): 8225004: Remove invalid assertion in jfr_conditional_flush() In-Reply-To: References: <85e6c70a-596f-451f-b15f-540636116ad3@default> Message-ID: <3fa5104a-8477-4134-9ff7-ae24e968fdc6@default> Thank you Dan, Appreciate you taking a look. Thanks again Markus -----Original Message----- From: Daniel D. Daugherty Sent: den 29 maj 2019 20:39 To: Markus Gronlund ; hotspot-jfr-dev at openjdk.java.net; Hotspot dev runtime Subject: Re: RFR(XXS): 8225004: Remove invalid assertion in jfr_conditional_flush() On 5/29/19 1:59 PM, Markus Gronlund wrote: > Greetings, > > Please review this small change to remove an invalid assertion. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8225004 > > Change: > > diff -r c1ad2862d0dd -r 0c17f2ea252e src/hotspot/share/jfr/support/jfrFlush.cpp > --- a/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 09:53:28 2019 -0700 > +++ b/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 19:04:34 2019 +0200 > @@ -62,7 +62,6 @@ > } > > void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { > - assert(jfr_is_event_enabled(id), "invariant"); > if (t->jfr_thread_local()->has_native_buffer()) { > JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); > if (LessThanSize::evaluate(buffer, size)) { Thumbs up. And thanks for the analysis below. If I remember correctly, JVM/TI async events and event handlers run into the same issue. :-) Dan > > > Description: > > JfrConditionalFlush(Thread* t) { > if (jfr_is_event_enabled(Event::eventId)) { <<--------------- evaluates to true > jfr_conditional_flush(Event::eventId, sizeof(Event), t); > } > } > > void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { > assert(jfr_is_event_enabled(id), "invariant"); <<-------------------- could evaluate to false > if (t->jfr_thread_local()->has_native_buffer()) { > JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); > if (LessThanSize::evaluate(buffer, size)) { > JfrFlush f(buffer, 0, 0, t); > } > } > } > > Event settings are updated asynchronously, where event disablement happens after the recording has stopped. > > A thread could have evaluated the initial check to true to enter jfr_conditional_flush(). Depending on visibility, context switching and preemption, another thread could have disabled the event before the thread runs jfr_conditional_flush(). > > There is no validity to be asserted at this location, so the assertion needs to be removed. > > Thanks > Markus From david.holmes at oracle.com Thu May 30 05:16:30 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 30 May 2019 15:16:30 +1000 Subject: RFR(XXS): 8225004: Remove invalid assertion in jfr_conditional_flush() In-Reply-To: <85e6c70a-596f-451f-b15f-540636116ad3@default> References: <85e6c70a-596f-451f-b15f-540636116ad3@default> Message-ID: <7999c0bb-f671-b3a4-4c97-3af53caaf0a9@oracle.com> Looks good Markus. Thanks for the clear explanation. David On 30/05/2019 3:59 am, Markus Gronlund wrote: > Greetings, > > Please review this small change to remove an invalid assertion. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8225004 > > Change: > > diff -r c1ad2862d0dd -r 0c17f2ea252e src/hotspot/share/jfr/support/jfrFlush.cpp > --- a/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 09:53:28 2019 -0700 > +++ b/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 19:04:34 2019 +0200 > @@ -62,7 +62,6 @@ > } > > void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { > - assert(jfr_is_event_enabled(id), "invariant"); > if (t->jfr_thread_local()->has_native_buffer()) { > JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); > if (LessThanSize::evaluate(buffer, size)) { > > > Description: > > JfrConditionalFlush(Thread* t) { > if (jfr_is_event_enabled(Event::eventId)) { <<--------------- evaluates to true > jfr_conditional_flush(Event::eventId, sizeof(Event), t); > } > } > > void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { > assert(jfr_is_event_enabled(id), "invariant"); <<-------------------- could evaluate to false > if (t->jfr_thread_local()->has_native_buffer()) { > JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); > if (LessThanSize::evaluate(buffer, size)) { > JfrFlush f(buffer, 0, 0, t); > } > } > } > > Event settings are updated asynchronously, where event disablement happens after the recording has stopped. > > A thread could have evaluated the initial check to true to enter jfr_conditional_flush(). Depending on visibility, context switching and preemption, another thread could have disabled the event before the thread runs jfr_conditional_flush(). > > There is no validity to be asserted at this location, so the assertion needs to be removed. > > Thanks > Markus > From markus.gronlund at oracle.com Thu May 30 09:23:45 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 30 May 2019 02:23:45 -0700 (PDT) Subject: RFR(XXS): 8225004: Remove invalid assertion in jfr_conditional_flush() In-Reply-To: <7999c0bb-f671-b3a4-4c97-3af53caaf0a9@oracle.com> References: <85e6c70a-596f-451f-b15f-540636116ad3@default> <7999c0bb-f671-b3a4-4c97-3af53caaf0a9@oracle.com> Message-ID: Thanks David! Cheers Markus -----Original Message----- From: David Holmes Sent: den 30 maj 2019 07:17 To: Markus Gronlund ; hotspot-jfr-dev at openjdk.java.net; Hotspot dev runtime Subject: Re: RFR(XXS): 8225004: Remove invalid assertion in jfr_conditional_flush() Looks good Markus. Thanks for the clear explanation. David On 30/05/2019 3:59 am, Markus Gronlund wrote: > Greetings, > > Please review this small change to remove an invalid assertion. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8225004 > > Change: > > diff -r c1ad2862d0dd -r 0c17f2ea252e src/hotspot/share/jfr/support/jfrFlush.cpp > --- a/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 09:53:28 2019 -0700 > +++ b/src/hotspot/share/jfr/support/jfrFlush.cpp Wed May 29 19:04:34 2019 +0200 > @@ -62,7 +62,6 @@ > } > > void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { > - assert(jfr_is_event_enabled(id), "invariant"); > if (t->jfr_thread_local()->has_native_buffer()) { > JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); > if (LessThanSize::evaluate(buffer, size)) { > > > Description: > > JfrConditionalFlush(Thread* t) { > if (jfr_is_event_enabled(Event::eventId)) { <<--------------- evaluates to true > jfr_conditional_flush(Event::eventId, sizeof(Event), t); > } > } > > void jfr_conditional_flush(JfrEventId id, size_t size, Thread* t) { > assert(jfr_is_event_enabled(id), "invariant"); <<-------------------- could evaluate to false > if (t->jfr_thread_local()->has_native_buffer()) { > JfrStorage::Buffer* const buffer = t->jfr_thread_local()->native_buffer(); > if (LessThanSize::evaluate(buffer, size)) { > JfrFlush f(buffer, 0, 0, t); > } > } > } > > Event settings are updated asynchronously, where event disablement happens after the recording has stopped. > > A thread could have evaluated the initial check to true to enter jfr_conditional_flush(). Depending on visibility, context switching and preemption, another thread could have disabled the event before the thread runs jfr_conditional_flush(). > > There is no validity to be asserted at this location, so the assertion needs to be removed. > > Thanks > Markus > From mikhailo.seledtsov at oracle.com Fri May 31 20:47:28 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Fri, 31 May 2019 13:47:28 -0700 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp Message-ID: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> Please review this change that converts all uses of Files.createTempFile/createTempDirecotry in JFR tests to use corresponding test library Utils methods. Using test.lib.Utils for this purpose is a recommended practice for JDK tests. I have also create a new method in jdk.test.lib.Utils: createTempDirectory() ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ ??? Testing: ??????? 1. Locally: ran the affected tests (Mac) - All PASS ??????? 2. test cluster: running jdk_jfr and tier1 - in Progress Note: to be safe, I will not integrate this until Monday morning. Thank you, Misha From harold.seigel at oracle.com Fri May 31 20:59:22 2019 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 31 May 2019 16:59:22 -0400 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp In-Reply-To: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> References: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> Message-ID: Hi Misha, This change looks good! One small question, why change "dump-path" to "dump-path-" at line 66 of TestJcmdConfigure.java ? Thanks, Harold On 5/31/2019 4:47 PM, mikhailo.seledtsov at oracle.com wrote: > Please review this change that converts all uses of > Files.createTempFile/createTempDirecotry in JFR tests to > use corresponding test library Utils methods. Using test.lib.Utils for > this purpose is a recommended practice > for JDK tests. > > I have also create a new method in jdk.test.lib.Utils: > createTempDirectory() > > ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 > ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ > ??? Testing: > ??????? 1. Locally: ran the affected tests (Mac) - All PASS > ??????? 2. test cluster: running jdk_jfr and tier1 - in Progress > > Note: to be safe, I will not integrate this until Monday morning. > > > Thank you, > Misha > From mikhailo.seledtsov at oracle.com Fri May 31 21:32:12 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Fri, 31 May 2019 14:32:12 -0700 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp In-Reply-To: References: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> Message-ID: <017022ce-b335-f253-d9f9-0951fe6a42b8@oracle.com> Hi Harold, ? Thank you for Review. On 5/31/19 1:59 PM, Harold Seigel wrote: > Hi Misha, > > This change looks good! > > One small question, why change "dump-path" to "dump-path-" at line 66 > of TestJcmdConfigure.java ? Purely cosmetic change: the file name looks better this way: dump-path927834 vs dump-path-927834. Misha > > Thanks, Harold > > On 5/31/2019 4:47 PM, mikhailo.seledtsov at oracle.com wrote: >> Please review this change that converts all uses of >> Files.createTempFile/createTempDirecotry in JFR tests to >> use corresponding test library Utils methods. Using test.lib.Utils >> for this purpose is a recommended practice >> for JDK tests. >> >> I have also create a new method in jdk.test.lib.Utils: >> createTempDirectory() >> >> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 >> ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ >> ??? Testing: >> ??????? 1. Locally: ran the affected tests (Mac) - All PASS >> ??????? 2. test cluster: running jdk_jfr and tier1 - in Progress >> >> Note: to be safe, I will not integrate this until Monday morning. >> >> >> Thank you, >> Misha >> From mikhailo.seledtsov at oracle.com Fri May 31 22:50:57 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Fri, 31 May 2019 15:50:57 -0700 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp In-Reply-To: <2555EFDD-3DC0-455A-8AC2-B0B419191123@oracle.com> References: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> <2555EFDD-3DC0-455A-8AC2-B0B419191123@oracle.com> Message-ID: <6c3041ab-9ced-8b60-6b7e-8515d19915c2@oracle.com> Hi Leonid, ? Thank you for this suggestion. No, I did not consider that. With my current fix I simply addressed the specific test set and a specific bug. The change described in JDK-8213214 is much larger in scope and impact, touches build infra. The RFE is assigned to Erik, who is an expert in make and build system. Best regards, Misha On 5/31/19 3:33 PM, Leonid Mesnik wrote: > Hi > > Have you considered fixing makefiles to set 'java.io.tmpdir' described > the https://bugs.openjdk.java.net/browse/JDK-8213214 > rather than adding new method and fixing individual tests? > > Leonid >> On May 31, 2019, at 1:47 PM, mikhailo.seledtsov at oracle.com >> wrote: >> >> Please review this change that converts all uses of >> Files.createTempFile/createTempDirecotry in JFR tests to >> use corresponding test library Utils methods. Using test.lib.Utils >> for this purpose is a recommended practice >> for JDK tests. >> >> I have also create a new method in jdk.test.lib.Utils: >> createTempDirectory() >> >> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 >> ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ >> ??? Testing: >> ??????? 1. Locally: ran the affected tests (Mac) - All PASS >> ??????? 2. test cluster: running jdk_jfr and tier1 - in Progress >> >> Note: to be safe, I will not integrate this until Monday morning. >> >> >> Thank you, >> Misha >> > From erik.joelsson at oracle.com Fri May 31 23:05:24 2019 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 31 May 2019 16:05:24 -0700 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp In-Reply-To: <2555EFDD-3DC0-455A-8AC2-B0B419191123@oracle.com> References: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> <2555EFDD-3DC0-455A-8AC2-B0B419191123@oracle.com> Message-ID: <5d761e9c-f81e-e083-e4f4-5ffc8241ab3f@oracle.com> Unfortunately, JDK-8213214 is blocked as some tests cannot handle how we would then set java.io.tmpdir. In general, I think Misha's approach is much better. Jtreg is providing a scratch dir where tests can create files. There should be a very good reason to leaving droppings elsewhere. These files are also saved if anything goes wrong, which can aid debugging. /Erik On 2019-05-31 15:33, Leonid Mesnik wrote: > Hi > > Have you considered fixing makefiles to set 'java.io.tmpdir' described > the https://bugs.openjdk.java.net/browse/JDK-8213214 > rather than adding new method and fixing individual tests? > > Leonid >> On May 31, 2019, at 1:47 PM, mikhailo.seledtsov at oracle.com >> wrote: >> >> Please review this change that converts all uses of >> Files.createTempFile/createTempDirecotry in JFR tests to >> use corresponding test library Utils methods. Using test.lib.Utils >> for this purpose is a recommended practice >> for JDK tests. >> >> I have also create a new method in jdk.test.lib.Utils: >> createTempDirectory() >> >> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 >> ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ >> ??? Testing: >> ??????? 1. Locally: ran the affected tests (Mac) - All PASS >> ??????? 2. test cluster: running jdk_jfr and tier1 - in Progress >> >> Note: to be safe, I will not integrate this until Monday morning. >> >> >> Thank you, >> Misha >> > From mikhailo.seledtsov at oracle.com Fri May 31 23:08:13 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Fri, 31 May 2019 16:08:13 -0700 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp In-Reply-To: <5d761e9c-f81e-e083-e4f4-5ffc8241ab3f@oracle.com> References: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> <2555EFDD-3DC0-455A-8AC2-B0B419191123@oracle.com> <5d761e9c-f81e-e083-e4f4-5ffc8241ab3f@oracle.com> Message-ID: <5872e603-9a92-73dd-aca7-34b8f9c78a92@oracle.com> Thank you Harold, Leonid and Erik, Misha On 5/31/19 4:05 PM, Erik Joelsson wrote: > > Unfortunately, JDK-8213214 is blocked as some tests cannot handle how > we would then set java.io.tmpdir. > > In general, I think Misha's approach is much better. Jtreg is > providing a scratch dir where tests can create files. There should be > a very good reason to leaving droppings elsewhere. These files are > also saved if anything goes wrong, which can aid debugging. > > /Erik > > On 2019-05-31 15:33, Leonid Mesnik wrote: >> Hi >> >> Have you considered fixing makefiles to set 'java.io.tmpdir' >> described the https://bugs.openjdk.java.net/browse/JDK-8213214 >> rather than adding new method and fixing individual tests? >> >> Leonid >>> On May 31, 2019, at 1:47 PM, mikhailo.seledtsov at oracle.com >>> wrote: >>> >>> Please review this change that converts all uses of >>> Files.createTempFile/createTempDirecotry in JFR tests to >>> use corresponding test library Utils methods. Using test.lib.Utils >>> for this purpose is a recommended practice >>> for JDK tests. >>> >>> I have also create a new method in jdk.test.lib.Utils: >>> createTempDirectory() >>> >>> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 >>> ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ >>> ??? Testing: >>> ??????? 1. Locally: ran the affected tests (Mac) - All PASS >>> ??????? 2. test cluster: running jdk_jfr and tier1 - in Progress >>> >>> Note: to be safe, I will not integrate this until Monday morning. >>> >>> >>> Thank you, >>> Misha >>> >> From leonid.mesnik at oracle.com Fri May 31 22:33:59 2019 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 31 May 2019 15:33:59 -0700 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp In-Reply-To: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> References: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> Message-ID: <2555EFDD-3DC0-455A-8AC2-B0B419191123@oracle.com> Hi Have you considered fixing makefiles to set 'java.io.tmpdir' described the https://bugs.openjdk.java.net/browse/JDK-8213214 rather than adding new method and fixing individual tests? Leonid > On May 31, 2019, at 1:47 PM, mikhailo.seledtsov at oracle.com wrote: > > Please review this change that converts all uses of Files.createTempFile/createTempDirecotry in JFR tests to > use corresponding test library Utils methods. Using test.lib.Utils for this purpose is a recommended practice > for JDK tests. > > I have also create a new method in jdk.test.lib.Utils: createTempDirectory() > > JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 > Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ > Testing: > 1. Locally: ran the affected tests (Mac) - All PASS > 2. test cluster: running jdk_jfr and tier1 - in Progress > > Note: to be safe, I will not integrate this until Monday morning. > > > Thank you, > Misha > From leonid.mesnik at oracle.com Fri May 31 23:02:52 2019 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 31 May 2019 16:02:52 -0700 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp In-Reply-To: <6c3041ab-9ced-8b60-6b7e-8515d19915c2@oracle.com> References: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> <2555EFDD-3DC0-455A-8AC2-B0B419191123@oracle.com> <6c3041ab-9ced-8b60-6b7e-8515d19915c2@oracle.com> Message-ID: <96B16B19-18B0-4312-AB87-72EEA987EC57@oracle.com> Thanks for checking this. As I understand bug JDK-8213214 is not going to be fixed soon. So your fix looks good. Leonid > On May 31, 2019, at 3:50 PM, mikhailo.seledtsov at oracle.com wrote: > > Hi Leonid, > > Thank you for this suggestion. > > No, I did not consider that. With my current fix I simply addressed the specific test set and a specific bug. > > The change described in JDK-8213214 is much larger in scope and impact, touches build infra. The RFE is assigned to Erik, who is an expert in make and build system. > > > > Best regards, > > Misha > > On 5/31/19 3:33 PM, Leonid Mesnik wrote: >> Hi >> >> Have you considered fixing makefiles to set 'java.io.tmpdir' described the https://bugs.openjdk.java.net/browse/JDK-8213214 >> rather than adding new method and fixing individual tests? >> >> Leonid >>> On May 31, 2019, at 1:47 PM, mikhailo.seledtsov at oracle.com wrote: >>> >>> Please review this change that converts all uses of Files.createTempFile/createTempDirecotry in JFR tests to >>> use corresponding test library Utils methods. Using test.lib.Utils for this purpose is a recommended practice >>> for JDK tests. >>> >>> I have also create a new method in jdk.test.lib.Utils: createTempDirectory() >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 >>> Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ >>> Testing: >>> 1. Locally: ran the affected tests (Mac) - All PASS >>> 2. test cluster: running jdk_jfr and tier1 - in Progress >>> >>> Note: to be safe, I will not integrate this until Monday morning. >>> >>> >>> Thank you, >>> Misha >>> >> From leonid.mesnik at oracle.com Fri May 31 23:11:34 2019 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 31 May 2019 16:11:34 -0700 Subject: RFR(S): 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp In-Reply-To: <5d761e9c-f81e-e083-e4f4-5ffc8241ab3f@oracle.com> References: <44e3b505-a39f-17ff-276e-b6879ac5dbae@oracle.com> <2555EFDD-3DC0-455A-8AC2-B0B419191123@oracle.com> <5d761e9c-f81e-e083-e4f4-5ffc8241ab3f@oracle.com> Message-ID: <999FDAC4-40FE-4F50-A78B-508DA3806683@oracle.com> Thanks for explanation. I think that in such case JDK-8213214 might be updated to cover fixing tests to use Utils.createTempFile instead of File.createTempFile. Leonid > On May 31, 2019, at 4:05 PM, Erik Joelsson wrote: > > Unfortunately, JDK-8213214 is blocked as some tests cannot handle how we would then set java.io.tmpdir. > > In general, I think Misha's approach is much better. Jtreg is providing a scratch dir where tests can create files. There should be a very good reason to leaving droppings elsewhere. These files are also saved if anything goes wrong, which can aid debugging. > > /Erik > > On 2019-05-31 15:33, Leonid Mesnik wrote: >> Hi >> >> Have you considered fixing makefiles to set 'java.io.tmpdir' described the https://bugs.openjdk.java.net/browse/JDK-8213214 >> rather than adding new method and fixing individual tests? >> >> Leonid >>> On May 31, 2019, at 1:47 PM, mikhailo.seledtsov at oracle.com wrote: >>> >>> Please review this change that converts all uses of Files.createTempFile/createTempDirecotry in JFR tests to >>> use corresponding test library Utils methods. Using test.lib.Utils for this purpose is a recommended practice >>> for JDK tests. >>> >>> I have also create a new method in jdk.test.lib.Utils: createTempDirectory() >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223396 >>> Webrev: http://cr.openjdk.java.net/~mseledtsov/8223396.00/ >>> Testing: >>> 1. Locally: ran the affected tests (Mac) - All PASS >>> 2. test cluster: running jdk_jfr and tier1 - in Progress >>> >>> Note: to be safe, I will not integrate this until Monday morning. >>> >>> >>> Thank you, >>> Misha >>> >>