From Leonid.Kuskov at Oracle.com Mon Oct 8 20:48:05 2018 From: Leonid.Kuskov at Oracle.com (Leonid Kuskov) Date: Mon, 8 Oct 2018 13:48:05 -0700 Subject: RFR: CODETOOLS-7902332: jasm should support nested static arguments of bootstrap methods Message-ID: <7b95119a-5902-5b8d-ad3b-df01907294f9@Oracle.com> Hi, The changeset http://hg.openjdk.java.net/code-tools/asmtools/rev/6f5ba7fbaa0b causes a noticeable regression in processing nested static arguments of invokedynamic instruction. jasm is able to process only ldc(*) instructions and fails to work with invokedynamic. The trivial fix is to reuse the method written for ldc in invokedynamic instruction. Bug: https://bugs.openjdk.java.net/browse/CODETOOLS-7902332 Webrev: http://cr.openjdk.java.net/~lkuskov/7902332/webrev.00 Thanks, Leonid From Leonid.Kuskov at Oracle.com Tue Oct 9 16:52:57 2018 From: Leonid.Kuskov at Oracle.com (Leonid Kuskov) Date: Tue, 9 Oct 2018 09:52:57 -0700 Subject: RFR [jcov, unfilled]: jcov does not instrument files with NestHost/NestMembers attributes In-Reply-To: <5B69B9D2.5010404@oracle.com> References: <5B69B9D2.5010404@oracle.com> Message-ID: <7e4188c9-f594-eed8-d19d-25264b715f38@Oracle.com> Hi Jan, Could you please file the bug and apply your changes to http://hg.openjdk.java.net/code-tools/jcov/. I need to switch jcov to the next version of asm and upload couple fixes that needed for JCK. This is best done after your fixes. Thanks, Leonid On 8/7/18 08:25, Jan Lahoda wrote: > Hi, > > It seems that jcov cannot instrument files with NestHost/NestMembers > attributes (JDK 11 classfiles), and silently uses the original > version. The reason appears to be that even though it uses ASM 6.2, > which knows about those attributes, the visitors are using "ASM6" > compatibility flag, which does not support these attributes. > > My proposal here is to upgrade the compatibility flag to > "ASM7_EXPERIMENTAL" compatibility level. It might be better to define > a constant for the compatibility level constant, so that on next > upgrade, we don't need to modify so many files. The proposed patch is > here: > http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.allow.nesthost.00/ > > > Also, we could enhance the build script to allow running tests: > http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.test.00/ > > If these would seem to go in a reasonable direction, I'll file bugs. > It may be necessary to do a few more changes to support JDK 12 > classfiles. > > Any feedback is welcome! > > Thanks, > ??? Jan From jan.lahoda at oracle.com Wed Oct 10 12:19:40 2018 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Wed, 10 Oct 2018 14:19:40 +0200 Subject: RFR [jcov, unfilled]: jcov does not instrument files with NestHost/NestMembers attributes In-Reply-To: <7e4188c9-f594-eed8-d19d-25264b715f38@Oracle.com> References: <5B69B9D2.5010404@oracle.com> <7e4188c9-f594-eed8-d19d-25264b715f38@Oracle.com> Message-ID: <5BBDEE5C.20302@oracle.com> Hello, I apologize, I forgot to do this. I've filled: https://bugs.openjdk.java.net/browse/CODETOOLS-7902334 (for the NestHost problem) https://bugs.openjdk.java.net/browse/CODETOOLS-7902335 (for enhancing the build script with tests) And put patches that might be suitable for commit here, respectivelly: http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.allow.nesthost.00/7902334 http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.test.00/7902335 I am not a committer on the code-tools project, so I'd like to ask if Jon could take a look whether it is fine to apply these commits. Thanks, Jan On 9.10.2018 18:52, Leonid Kuskov wrote: > Hi Jan, > > Could you please file the bug and apply your changes to > http://hg.openjdk.java.net/code-tools/jcov/. > I need to switch jcov to the next version of asm and upload couple fixes > that needed for JCK. > This is best done after your fixes. > > Thanks, > Leonid > > > On 8/7/18 08:25, Jan Lahoda wrote: >> Hi, >> >> It seems that jcov cannot instrument files with NestHost/NestMembers >> attributes (JDK 11 classfiles), and silently uses the original >> version. The reason appears to be that even though it uses ASM 6.2, >> which knows about those attributes, the visitors are using "ASM6" >> compatibility flag, which does not support these attributes. >> >> My proposal here is to upgrade the compatibility flag to >> "ASM7_EXPERIMENTAL" compatibility level. It might be better to define >> a constant for the compatibility level constant, so that on next >> upgrade, we don't need to modify so many files. The proposed patch is >> here: >> http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.allow.nesthost.00/ >> >> >> Also, we could enhance the build script to allow running tests: >> http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.test.00/ >> >> If these would seem to go in a reasonable direction, I'll file bugs. >> It may be necessary to do a few more changes to support JDK 12 >> classfiles. >> >> Any feedback is welcome! >> >> Thanks, >> Jan > From jonathan.gibbons at oracle.com Wed Oct 10 16:15:53 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 10 Oct 2018 09:15:53 -0700 Subject: RFR [jcov, unfilled]: jcov does not instrument files with NestHost/NestMembers attributes In-Reply-To: <5BBDEE5C.20302@oracle.com> References: <5B69B9D2.5010404@oracle.com> <7e4188c9-f594-eed8-d19d-25264b715f38@Oracle.com> <5BBDEE5C.20302@oracle.com> Message-ID: <8a0c96e1-8217-7630-6441-9c6e0eb8ef10@oracle.com> Jan, Leonid, Given that Leonid has requested that these changes should be pushed, I will push them later today. -- Jon On 10/10/18 5:19 AM, Jan Lahoda wrote: > Hello, > > I apologize, I forgot to do this. I've filled: > https://bugs.openjdk.java.net/browse/CODETOOLS-7902334 > (for the NestHost problem) > https://bugs.openjdk.java.net/browse/CODETOOLS-7902335 > (for enhancing the build script with tests) > > And put patches that might be suitable for commit here, respectivelly: > http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.allow.nesthost.00/7902334 > > > http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.test.00/7902335 > > I am not a committer on the code-tools project, so I'd like to ask if > Jon could take a look whether it is fine to apply these commits. > > Thanks, > ??? Jan > > On 9.10.2018 18:52, Leonid Kuskov wrote: >> Hi Jan, >> >> Could you please file the bug and apply your changes to >> http://hg.openjdk.java.net/code-tools/jcov/. >> I need to switch jcov to the next version of asm and upload couple fixes >> that needed for JCK. >> This is best done after your fixes. >> >> Thanks, >> Leonid >> >> >> On 8/7/18 08:25, Jan Lahoda wrote: >>> Hi, >>> >>> It seems that jcov cannot instrument files with NestHost/NestMembers >>> attributes (JDK 11 classfiles), and silently uses the original >>> version. The reason appears to be that even though it uses ASM 6.2, >>> which knows about those attributes, the visitors are using "ASM6" >>> compatibility flag, which does not support these attributes. >>> >>> My proposal here is to upgrade the compatibility flag to >>> "ASM7_EXPERIMENTAL" compatibility level. It might be better to define >>> a constant for the compatibility level constant, so that on next >>> upgrade, we don't need to modify so many files. The proposed patch is >>> here: >>> http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.allow.nesthost.00/ >>> >>> >>> >>> Also, we could enhance the build script to allow running tests: >>> http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.test.00/ >>> >>> If these would seem to go in a reasonable direction, I'll file bugs. >>> It may be necessary to do a few more changes to support JDK 12 >>> classfiles. >>> >>> Any feedback is welcome! >>> >>> Thanks, >>> ??? Jan >> From volker.simonis at gmail.com Thu Oct 11 17:01:02 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 11 Oct 2018 19:01:02 +0200 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* Message-ID: Hi, the change "CODETOOLS-7902290: introduce error state of @requires properties" [1] which was recently pushed to the codetools repo, breaks all JTreg tests which have a require clause that queries a non final (i.e. no "vm.opt.final") VM option. For example the test "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: test result: Error. Error evaluating expression: name not defined: vm.opt.DeoptimizeALot This is because com.sun.javatest.regtest.config.RegressionContext.get() was changed to return null instead of the string "null" for unknown properties. Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" is indeed not defined (as many other "vm.opt" properties, see below). So it turns out that all the tests which @required a "vm.opt" option did run in a faulty configuration until now. I could for example easily fix "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding the corresponding option in VMProps.java: $ hg diff diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 2018 -0400 +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 2018 +0200 @@ -266,6 +266,8 @@ vmOptFinalFlag(map, "ClassUnloading"); vmOptFinalFlag(map, "UseCompressedOops"); vmOptFinalFlag(map, "EnableJVMCI"); + String value = String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); + map.put("vm.opt.DeoptimizeALot", value); } Following is a list of all "vm.opt" options used in @requires clauses without being defined in VMProps.java: vm.opt.AggressiveOpts vm.opt.ClassUnloading vm.opt.ClassUnloadingWithConcurrentMark vm.opt.DeoptimizeALot vm.opt.DisableExplicitGC vm.opt.ExplicitGCInvokesConcurrent vm.opt.FlightRecorder vm.opt.G1HeapRegionSize vm.opt.Inline vm.opt.MaxGCPauseMillis vm.opt.StartFlightRecording vm.opt.SurvivorAlignmentInBytes vm.opt.TieredCompilation vm.opt.TieredStopAtLevel vm.opt.TieredStopAtLevel==4) vm.opt.TieredStopAtLevel==null vm.opt.UseJVMCICompiler What's the current plan? Do we want to add all these options in VMProps.java? Or should we better add all the -XX VM options together to VMProps such that they can all be freely used within @requires clauses? Thank you and best regards, Volker PS: I've noticed that there's "8209807: improve handling exception in requires.VMProps" [2] to apparently adapt VMProps.java to CODETOOLS-7902290 but I've tried the attached webrev and it doesn't help with the described problem. [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 [2] https://bugs.openjdk.java.net/browse/JDK-8209807 From jonathan.gibbons at oracle.com Thu Oct 11 17:13:20 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 11 Oct 2018 10:13:20 -0700 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: Message-ID: Volker, Thanks for the report.? I'll take a look shortly. -- Jon On 10/11/18 10:01 AM, Volker Simonis wrote: > Hi, > > the change "CODETOOLS-7902290: introduce error state of @requires > properties" [1] which was recently pushed to the codetools repo, > breaks all JTreg tests which have a require clause that queries a non > final (i.e. no "vm.opt.final") VM option. > > For example the test > "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which > uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: > > test result: Error. Error evaluating expression: name not defined: > vm.opt.DeoptimizeALot > > This is because > com.sun.javatest.regtest.config.RegressionContext.get() was changed to > return null instead of the string "null" for unknown properties. > Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" > is indeed not defined (as many other "vm.opt" properties, see below). > So it turns out that all the tests which @required a "vm.opt" option > did run in a faulty configuration until now. > > I could for example easily fix > "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding > the corresponding option in VMProps.java: > > $ hg diff > diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java > --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 2018 -0400 > +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 2018 +0200 > @@ -266,6 +266,8 @@ > vmOptFinalFlag(map, "ClassUnloading"); > vmOptFinalFlag(map, "UseCompressedOops"); > vmOptFinalFlag(map, "EnableJVMCI"); > + String value = String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); > + map.put("vm.opt.DeoptimizeALot", value); > } > > Following is a list of all "vm.opt" options used in @requires clauses > without being defined in VMProps.java: > > vm.opt.AggressiveOpts > vm.opt.ClassUnloading > vm.opt.ClassUnloadingWithConcurrentMark > vm.opt.DeoptimizeALot > vm.opt.DisableExplicitGC > vm.opt.ExplicitGCInvokesConcurrent > vm.opt.FlightRecorder > vm.opt.G1HeapRegionSize > vm.opt.Inline > vm.opt.MaxGCPauseMillis > vm.opt.StartFlightRecording > vm.opt.SurvivorAlignmentInBytes > vm.opt.TieredCompilation > vm.opt.TieredStopAtLevel > vm.opt.TieredStopAtLevel==4) > vm.opt.TieredStopAtLevel==null > vm.opt.UseJVMCICompiler > > What's the current plan? Do we want to add all these options in > VMProps.java? Or should we better add all the -XX VM options together > to VMProps such that they can all be freely used within @requires > clauses? > > Thank you and best regards, > Volker > > PS: I've noticed that there's "8209807: improve handling exception in > requires.VMProps" [2] to apparently adapt VMProps.java to > CODETOOLS-7902290 but I've tried the attached webrev and it doesn't > help with the described problem. > > [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 > [2] https://bugs.openjdk.java.net/browse/JDK-8209807 From jonathan.gibbons at oracle.com Thu Oct 11 17:16:08 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 11 Oct 2018 10:16:08 -0700 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: Message-ID: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Volker, It's a typo/oops on my part.? I'll fix it. -- Jon On 10/11/18 10:13 AM, Jonathan Gibbons wrote: > Volker, > > Thanks for the report.? I'll take a look shortly. > > -- Jon > > > On 10/11/18 10:01 AM, Volker Simonis wrote: >> Hi, >> >> the change "CODETOOLS-7902290: introduce error state of @requires >> properties" [1] which was recently pushed to the codetools repo, >> breaks all JTreg tests which have a require clause that queries a non >> final (i.e. no "vm.opt.final") VM option. >> >> For example the test >> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which >> uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: >> >> test result: Error. Error evaluating expression: name not defined: >> vm.opt.DeoptimizeALot >> >> This is because >> com.sun.javatest.regtest.config.RegressionContext.get() was changed to >> return null instead of the string "null" for unknown properties. >> Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" >> is indeed not defined (as many other "vm.opt" properties, see below). >> So it turns out that all the tests which @required a "vm.opt" option >> did run in a faulty configuration until now. >> >> I could for example easily fix >> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding >> the corresponding option in VMProps.java: >> >> $ hg diff >> diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java >> --- a/test/jtreg-ext/requires/VMProps.java????? Thu Oct 11 10:11:18 >> 2018 -0400 >> +++ b/test/jtreg-ext/requires/VMProps.java????? Thu Oct 11 18:51:37 >> 2018 +0200 >> @@ -266,6 +266,8 @@ >> ????????? vmOptFinalFlag(map, "ClassUnloading"); >> ????????? vmOptFinalFlag(map, "UseCompressedOops"); >> ????????? vmOptFinalFlag(map, "EnableJVMCI"); >> +??????? String value = >> String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); >> +??????? map.put("vm.opt.DeoptimizeALot", value); >> ????? } >> >> Following is a list of all "vm.opt" options used in @requires clauses >> without being defined in VMProps.java: >> >> vm.opt.AggressiveOpts >> vm.opt.ClassUnloading >> vm.opt.ClassUnloadingWithConcurrentMark >> vm.opt.DeoptimizeALot >> vm.opt.DisableExplicitGC >> vm.opt.ExplicitGCInvokesConcurrent >> vm.opt.FlightRecorder >> vm.opt.G1HeapRegionSize >> vm.opt.Inline >> vm.opt.MaxGCPauseMillis >> vm.opt.StartFlightRecording >> vm.opt.SurvivorAlignmentInBytes >> vm.opt.TieredCompilation >> vm.opt.TieredStopAtLevel >> vm.opt.TieredStopAtLevel==4) >> vm.opt.TieredStopAtLevel==null >> vm.opt.UseJVMCICompiler >> >> What's the current plan? Do we want to add all these options in >> VMProps.java? Or should we better add all the -XX VM options together >> to VMProps such that they can all be freely used within @requires >> clauses? >> >> Thank you and best regards, >> Volker >> >> PS: I've noticed that there's "8209807: improve handling exception in >> requires.VMProps" [2] to apparently adapt VMProps.java to >> CODETOOLS-7902290 but I've tried the attached webrev and it doesn't >> help with the described problem. >> >> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 >> [2] https://bugs.openjdk.java.net/browse/JDK-8209807 > From volker.simonis at gmail.com Thu Oct 11 20:59:49 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 11 Oct 2018 22:59:49 +0200 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> References: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Message-ID: Jonathan Gibbons schrieb am Do. 11. Okt. 2018 um 19:16: > Volker, > > It's a typo/oops on my part. I'll fix it. > Hi John, thanks for looking at this issue! What do you mean with ? typo?? Returning null instead of the string ?null?? That would ?fix? the corresponding tests in the sense that they would run again, however still with bogus results for all the @requires guards which check the ?vm.opt.*? options. Shouldn?t we fix that as well? Regards, Volker > -- Jon > > > On 10/11/18 10:13 AM, Jonathan Gibbons wrote: > > Volker, > > > > Thanks for the report. I'll take a look shortly. > > > > -- Jon > > > > > > On 10/11/18 10:01 AM, Volker Simonis wrote: > >> Hi, > >> > >> the change "CODETOOLS-7902290: introduce error state of @requires > >> properties" [1] which was recently pushed to the codetools repo, > >> breaks all JTreg tests which have a require clause that queries a non > >> final (i.e. no "vm.opt.final") VM option. > >> > >> For example the test > >> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which > >> uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: > >> > >> test result: Error. Error evaluating expression: name not defined: > >> vm.opt.DeoptimizeALot > >> > >> This is because > >> com.sun.javatest.regtest.config.RegressionContext.get() was changed to > >> return null instead of the string "null" for unknown properties. > >> Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" > >> is indeed not defined (as many other "vm.opt" properties, see below). > >> So it turns out that all the tests which @required a "vm.opt" option > >> did run in a faulty configuration until now. > >> > >> I could for example easily fix > >> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding > >> the corresponding option in VMProps.java: > >> > >> $ hg diff > >> diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java > >> --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 > >> 2018 -0400 > >> +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 > >> 2018 +0200 > >> @@ -266,6 +266,8 @@ > >> vmOptFinalFlag(map, "ClassUnloading"); > >> vmOptFinalFlag(map, "UseCompressedOops"); > >> vmOptFinalFlag(map, "EnableJVMCI"); > >> + String value = > >> String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); > >> + map.put("vm.opt.DeoptimizeALot", value); > >> } > >> > >> Following is a list of all "vm.opt" options used in @requires clauses > >> without being defined in VMProps.java: > >> > >> vm.opt.AggressiveOpts > >> vm.opt.ClassUnloading > >> vm.opt.ClassUnloadingWithConcurrentMark > >> vm.opt.DeoptimizeALot > >> vm.opt.DisableExplicitGC > >> vm.opt.ExplicitGCInvokesConcurrent > >> vm.opt.FlightRecorder > >> vm.opt.G1HeapRegionSize > >> vm.opt.Inline > >> vm.opt.MaxGCPauseMillis > >> vm.opt.StartFlightRecording > >> vm.opt.SurvivorAlignmentInBytes > >> vm.opt.TieredCompilation > >> vm.opt.TieredStopAtLevel > >> vm.opt.TieredStopAtLevel==4) > >> vm.opt.TieredStopAtLevel==null > >> vm.opt.UseJVMCICompiler > >> > >> What's the current plan? Do we want to add all these options in > >> VMProps.java? Or should we better add all the -XX VM options together > >> to VMProps such that they can all be freely used within @requires > >> clauses? > >> > >> Thank you and best regards, > >> Volker > >> > >> PS: I've noticed that there's "8209807: improve handling exception in > >> requires.VMProps" [2] to apparently adapt VMProps.java to > >> CODETOOLS-7902290 but I've tried the attached webrev and it doesn't > >> help with the described problem. > >> > >> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 > >> [2] https://bugs.openjdk.java.net/browse/JDK-8209807 > > > > From volker.simonis at gmail.com Fri Oct 12 15:15:21 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 12 Oct 2018 17:15:21 +0200 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Message-ID: On Thu, Oct 11, 2018 at 11:09 PM Igor Ignatyev wrote: > > Hi Volker, > > vm.opt.* options are set by jtreg (RegressionContext::processVMOptions), and their value is expected to be "null" if a flag hasn't been passed to JDK under tests via -javaoptions or -vmoptions. I don't think there is something to fix (at least not in jtreg). > Well, if "vm.opt.*" options are indeed only options which are taken from the command line (i.e. "-javaoption/-vmoption"), than there's definitely something we have to fix! Because, as I described before, after CODETOOLS-7902290, all test which check such an option in a @requires clause (negatively or positively) will now fail if that option is not given explicitly on the command line. If "vm.opt.*" options are really only the options given on the command line (please confirm this and please document it somewhere prominently!) I think many tests which use them are broken because they simply check such options like this: @requires vm.opt.TieredCompilation == true This would be wrong and would have to be replaced by: @requires vm.opt.TieredCompilation == null | vm.opt.TieredCompilation == true to also account for the case where -XX:TieredCompilation was not explicitly set on the command line. Notice that in in such a case, the value of "TieredCompilation" in the VM could be either "true" or "false" (depending on the platform and VM build configuration), but it wouldn't be possible to test that by using "vm.opt.TieredCompilation". That said, there are three "vm.opt.final.*" options which are set by VMProps: vmOptFinalFlag(map, "ClassUnloading"); vmOptFinalFlag(map, "UseCompressedOops"); vmOptFinalFlag(map, "EnableJVMCI"); Notice that these options are set to the values of the corresponding flags in the agent VM . For tests which run in "othervm" mode (or which start new VMs with ProcessBuilder or jdk.test.lib.process.ProcessTools), that doesn't necessarily corresponds to the values of these flags in the testee VM. Finally, options specified with "-javaoption/-vmoption" and reflected in the corresponding "vm.opt.*" flags, may be over-ridden by command line options specified for tests which run in "othervm" mode (so checking them with a @requires clause is of limited usefulness as well). All this is quite complex and I think we should *really* document it in a prominent place like the "jtreg: Command Line Options" page [1] AND the "The JDK Test Framework: Tag Language Specification" page [2]. Regards, Volker [1] http://openjdk.java.net/jtreg/command-help.html [2] http://openjdk.java.net/jtreg/tag-spec.html > -- Igor > > > > On Oct 11, 2018, at 1:59 PM, Volker Simonis wrote: > > > > Jonathan Gibbons schrieb am Do. 11. Okt. 2018 > > um 19:16: > > > >> Volker, > >> > >> It's a typo/oops on my part. I'll fix it. > >> > > > > Hi John, > > > > thanks for looking at this issue! > > > > What do you mean with ? typo?? Returning null instead of the string ?null?? > > > > That would ?fix? the corresponding tests in the sense that they would run > > again, however still with bogus results for all the @requires guards which > > check the ?vm.opt.*? options. Shouldn?t we fix that as well? > > > > Regards, > > Volker > > > > > > > >> -- Jon > >> > >> > >> On 10/11/18 10:13 AM, Jonathan Gibbons wrote: > >>> Volker, > >>> > >>> Thanks for the report. I'll take a look shortly. > >>> > >>> -- Jon > >>> > >>> > >>> On 10/11/18 10:01 AM, Volker Simonis wrote: > >>>> Hi, > >>>> > >>>> the change "CODETOOLS-7902290: introduce error state of @requires > >>>> properties" [1] which was recently pushed to the codetools repo, > >>>> breaks all JTreg tests which have a require clause that queries a non > >>>> final (i.e. no "vm.opt.final") VM option. > >>>> > >>>> For example the test > >>>> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" which > >>>> uses "@requires vm.opt.DeoptimizeALot != true" will now fail with: > >>>> > >>>> test result: Error. Error evaluating expression: name not defined: > >>>> vm.opt.DeoptimizeALot > >>>> > >>>> This is because > >>>> com.sun.javatest.regtest.config.RegressionContext.get() was changed to > >>>> return null instead of the string "null" for unknown properties. > >>>> Looking at requires.VMProps, I realized that "vm.opt.DeoptimizeALot" > >>>> is indeed not defined (as many other "vm.opt" properties, see below). > >>>> So it turns out that all the tests which @required a "vm.opt" option > >>>> did run in a faulty configuration until now. > >>>> > >>>> I could for example easily fix > >>>> "hotspot/jtreg/runtime/ReservedStack/ReservedStackTest.java" by adding > >>>> the corresponding option in VMProps.java: > >>>> > >>>> $ hg diff > >>>> diff -r 7a1e2d7ac55a test/jtreg-ext/requires/VMProps.java > >>>> --- a/test/jtreg-ext/requires/VMProps.java Thu Oct 11 10:11:18 > >>>> 2018 -0400 > >>>> +++ b/test/jtreg-ext/requires/VMProps.java Thu Oct 11 18:51:37 > >>>> 2018 +0200 > >>>> @@ -266,6 +266,8 @@ > >>>> vmOptFinalFlag(map, "ClassUnloading"); > >>>> vmOptFinalFlag(map, "UseCompressedOops"); > >>>> vmOptFinalFlag(map, "EnableJVMCI"); > >>>> + String value = > >>>> String.valueOf(WB.getBooleanVMFlag("DeoptimizeALot")); > >>>> + map.put("vm.opt.DeoptimizeALot", value); > >>>> } > >>>> > >>>> Following is a list of all "vm.opt" options used in @requires clauses > >>>> without being defined in VMProps.java: > >>>> > >>>> vm.opt.AggressiveOpts > >>>> vm.opt.ClassUnloading > >>>> vm.opt.ClassUnloadingWithConcurrentMark > >>>> vm.opt.DeoptimizeALot > >>>> vm.opt.DisableExplicitGC > >>>> vm.opt.ExplicitGCInvokesConcurrent > >>>> vm.opt.FlightRecorder > >>>> vm.opt.G1HeapRegionSize > >>>> vm.opt.Inline > >>>> vm.opt.MaxGCPauseMillis > >>>> vm.opt.StartFlightRecording > >>>> vm.opt.SurvivorAlignmentInBytes > >>>> vm.opt.TieredCompilation > >>>> vm.opt.TieredStopAtLevel > >>>> vm.opt.TieredStopAtLevel==4) > >>>> vm.opt.TieredStopAtLevel==null > >>>> vm.opt.UseJVMCICompiler > >>>> > >>>> What's the current plan? Do we want to add all these options in > >>>> VMProps.java? Or should we better add all the -XX VM options together > >>>> to VMProps such that they can all be freely used within @requires > >>>> clauses? > >>>> > >>>> Thank you and best regards, > >>>> Volker > >>>> > >>>> PS: I've noticed that there's "8209807: improve handling exception in > >>>> requires.VMProps" [2] to apparently adapt VMProps.java to > >>>> CODETOOLS-7902290 but I've tried the attached webrev and it doesn't > >>>> help with the described problem. > >>>> > >>>> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7902290 > >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8209807 > >>> > >> > >> > From jonathan.gibbons at oracle.com Fri Oct 12 15:42:13 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 12 Oct 2018 08:42:13 -0700 Subject: CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.* In-Reply-To: References: <1025735f-94ca-12a3-78f8-b895fb1b4b2e@oracle.com> Message-ID: <7ea8c652-bcbd-154a-a02c-0a55cc4c6478@oracle.com> On 10/12/18 8:15 AM, Volker Simonis wrote: > On Thu, Oct 11, 2018 at 11:09 PM Igor Ignatyev wrote: >> Hi Volker, >> >> vm.opt.* options are set by jtreg (RegressionContext::processVMOptions), and their value is expected to be "null" if a flag hasn't been passed to JDK under tests via -javaoptions or -vmoptions. I don't think there is something to fix (at least not in jtreg). >> > Well, if "vm.opt.*" options are indeed only options which are taken > from the command line (i.e. "-javaoption/-vmoption"), than there's > definitely something we have to fix! Because, as I described before, > after CODETOOLS-7902290, all test which check such an option in a > @requires clause (negatively or positively) will now fail if that > option is not given explicitly on the command line. > > If "vm.opt.*" options are really only the options given on the command > line (please confirm this and please document it somewhere > prominently!) I think many tests which use them are broken because > they simply check such options like this: > > @requires vm.opt.TieredCompilation == true > > This would be wrong and would have to be replaced by: > > @requires vm.opt.TieredCompilation == null | vm.opt.TieredCompilation == true > > to also account for the case where -XX:TieredCompilation was not > explicitly set on the command line. Notice that in in such a case, the > value of "TieredCompilation" in the VM could be either "true" or > "false" (depending on the platform and VM build configuration), but it > wouldn't be possible to test that by using "vm.opt.TieredCompilation". > > That said, there are three "vm.opt.final.*" options which are set by VMProps: > > vmOptFinalFlag(map, "ClassUnloading"); > vmOptFinalFlag(map, "UseCompressedOops"); > vmOptFinalFlag(map, "EnableJVMCI"); > > Notice that these options are set to the values of the corresponding > flags in the agent VM . For tests which run in "othervm" mode (or > which start new VMs with ProcessBuilder or > jdk.test.lib.process.ProcessTools), that doesn't necessarily > corresponds to the values of these flags in the testee VM. > > Finally, options specified with "-javaoption/-vmoption" and reflected > in the corresponding "vm.opt.*" flags, may be over-ridden by command > line options specified for tests which run in "othervm" mode (so > checking them with a @requires clause is of limited usefulness as > well). > > All this is quite complex and I think we should*really* document it > in a prominent place like the "jtreg: Command Line Options" page [1] > AND the > "The JDK Test Framework: Tag Language Specification" page [2]. > > Regards, > Volker > > [1]http://openjdk.java.net/jtreg/command-help.html > [2]http://openjdk.java.net/jtreg/tag-spec.html > Volker, I will look? at what can be done to improve the documentation, especially with regard to documenting the limitations of the @requires mechanism. That being said, most of the mechanism is provided outside of jtreg, by the extraPropDefns? extension mechanism specified in TEST.ROOT.? It was a deliberate design choice to decouple these classes from jtreg, so that they can evolve faster and separately from jtreg promotions, according to the needs of the test suite. So, without diminishing the need for additional documentation, I'll note that detailed documentation may not belong in the jtreg pages you specified. Finally, you are right to observe the inadequacies of the vm.opt.* mechanism. IMO, it is better to use custom properties defined by the extraPropDefns mechanism that provide the result of analyzing the options, as compared to checking the options themselves. My prime example of this, when I was writing the jtreg support, was the "-server" and "-client" options, which are so-called supported options on all system, but may be no-ops on some platforms. Therefore, it is more important to check the internal settings in the VM than to check the options given to the VM. -- Jon From jonathan.gibbons at oracle.com Thu Oct 11 17:41:05 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 11 Oct 2018 10:41:05 -0700 Subject: RFR [jcov, unfilled]: jcov does not instrument files with NestHost/NestMembers attributes In-Reply-To: <5BBDEE5C.20302@oracle.com> References: <5B69B9D2.5010404@oracle.com> <7e4188c9-f594-eed8-d19d-25264b715f38@Oracle.com> <5BBDEE5C.20302@oracle.com> Message-ID: <31a4b04a-d299-db6c-eb18-13bab6868671@oracle.com> I've pushed these changes. I had to apply an intermediate patch to build/build.properties to convert it from Windows newlines to Unix newlines, before the second patch would apply. This was done with the following command: sed --in-place -e 's/^M//' build/build.properties In addition, I edited the changesets to include "Contributed-by: jan.lahoda at oracle.com" -- Jon On 10/10/2018 05:19 AM, Jan Lahoda wrote: > Hello, > > I apologize, I forgot to do this. I've filled: > https://bugs.openjdk.java.net/browse/CODETOOLS-7902334 > (for the NestHost problem) > https://bugs.openjdk.java.net/browse/CODETOOLS-7902335 > (for enhancing the build script with tests) > > And put patches that might be suitable for commit here, respectivelly: > http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.allow.nesthost.00/7902334 > > > http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.test.00/7902335 > > I am not a committer on the code-tools project, so I'd like to ask if > Jon could take a look whether it is fine to apply these commits. > > Thanks, > ??? Jan > > On 9.10.2018 18:52, Leonid Kuskov wrote: >> Hi Jan, >> >> Could you please file the bug and apply your changes to >> http://hg.openjdk.java.net/code-tools/jcov/. >> I need to switch jcov to the next version of asm and upload couple fixes >> that needed for JCK. >> This is best done after your fixes. >> >> Thanks, >> Leonid >> >> >> On 8/7/18 08:25, Jan Lahoda wrote: >>> Hi, >>> >>> It seems that jcov cannot instrument files with NestHost/NestMembers >>> attributes (JDK 11 classfiles), and silently uses the original >>> version. The reason appears to be that even though it uses ASM 6.2, >>> which knows about those attributes, the visitors are using "ASM6" >>> compatibility flag, which does not support these attributes. >>> >>> My proposal here is to upgrade the compatibility flag to >>> "ASM7_EXPERIMENTAL" compatibility level. It might be better to define >>> a constant for the compatibility level constant, so that on next >>> upgrade, we don't need to modify so many files. The proposed patch is >>> here: >>> http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.allow.nesthost.00/ >>> >>> >>> >>> Also, we could enhance the build script to allow running tests: >>> http://cr.openjdk.java.net/~jlahoda/jcov-nesthost/webrev.test.00/ >>> >>> If these would seem to go in a reasonable direction, I'll file bugs. >>> It may be necessary to do a few more changes to support JDK 12 >>> classfiles. >>> >>> Any feedback is welcome! >>> >>> Thanks, >>> ??? Jan >> From alexandre.iline at oracle.com Mon Oct 15 20:57:32 2018 From: alexandre.iline at oracle.com (Alexandre (Shura) Iline) Date: Mon, 15 Oct 2018 13:57:32 -0700 Subject: RFR: CODETOOLS-7902332: jasm should support nested static arguments of bootstrap methods In-Reply-To: <7b95119a-5902-5b8d-ad3b-df01907294f9@Oracle.com> References: <7b95119a-5902-5b8d-ad3b-df01907294f9@Oracle.com> Message-ID: <5620E7EE-2D12-4076-94CD-C4AFD5C9A4C5@oracle.com> Looks good, Leonid. > On Oct 8, 2018, at 1:48 PM, Leonid Kuskov wrote: > > Hi, > > The changeset http://hg.openjdk.java.net/code-tools/asmtools/rev/6f5ba7fbaa0b causes a noticeable regression in processing nested static arguments of invokedynamic instruction. > jasm is able to process only ldc(*) instructions and fails to work with invokedynamic. The trivial fix is to reuse the method written for ldc in invokedynamic instruction. > > Bug: https://bugs.openjdk.java.net/browse/CODETOOLS-7902332 > Webrev: http://cr.openjdk.java.net/~lkuskov/7902332/webrev.00 > > Thanks, > Leonid > From simon at dancingcloudservices.com Tue Oct 23 22:26:26 2018 From: simon at dancingcloudservices.com (Simon Roberts) Date: Tue, 23 Oct 2018 16:26:26 -0600 Subject: hprof format question Message-ID: Hi all, I'm hoping this is the correct list for a question on the hprof file format (1.0.2)? I found this information: http://hg.openjdk.java.net/jdk6/jdk6/jdk/raw-file/tip/src/share/demo/jvmti/hprof/manual.html and have been working on a small project to read these files. (Yes, I know that NetBeans/VisualVM and Eclipse both have such libraries, and a number of other tools have been derived from those, but so far as I can tell, they all are fundamentally built on the notion of fully decoding everything, and creating memory representations of the entire heap. I want to pull out only certain pieces of information--specifically object counts by class--from a large, ~20Gb, dump file, and those tools just give up the ghost on my systems.) Anyway, my code reads the file pretty well so far, except that the file I want to analyze seems to contradict the specifications of the document mentioned above. Specifically, after processing about five HEAP_DUMP_SEGMENTS with around 1.5 million sub-records in each, I come across some ROOT_JNI_LOCAL records. The first 15 follow the format specified in the above document (one 8 byte "ID" and two four byte values.) But the 16th omits the two four byte records (well, it might simply have more, but visual analysis shows that after the 8 byte ID, I have a new block tag, and a believable structure. My current intention is to write a "smart" parser that investigates whether processing will succeed or not if the "normal" format is read, or will use the alternate format if it realizes that it would result in the next block starting with an illegal tag value. Clearly this is not ideal! Is there any more detailed, newer, better, information? Or anything else I should know to pursue this tool (or indeed a simple object frequency by classname result) from an hprof 1.0.2 format file? (And yes, I'm pursuing a putative memory leak :) Thanks for any input (including "dude, this is the wrong list!") Cheers, Simon -- Simon Roberts (303) 249 3613 From magnus.ihse.bursie at oracle.com Wed Oct 31 08:41:17 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 31 Oct 2018 09:41:17 +0100 Subject: hprof format question In-Reply-To: References: Message-ID: <5e9255d2-4cb3-4390-6254-f4bd3d5d66e1@oracle.com> Hi Simon, On 2018-10-24 00:26, Simon Roberts wrote: > Hi all, I'm hoping this is the correct list for a question on the hprof > file format (1.0.2)? I don't think it is. You might get better response from serviceability-dev at openjdk.java.net. /Magnus > > I found this information: > http://hg.openjdk.java.net/jdk6/jdk6/jdk/raw-file/tip/src/share/demo/jvmti/hprof/manual.html > > and have been working on a small project to read these files. (Yes, I know > that NetBeans/VisualVM and Eclipse both have such libraries, and a number > of other tools have been derived from those, but so far as I can tell, they > all are fundamentally built on the notion of fully decoding everything, and > creating memory representations of the entire heap. I want to pull out only > certain pieces of information--specifically object counts by class--from a > large, ~20Gb, dump file, and those tools just give up the ghost on my > systems.) > > Anyway, my code reads the file pretty well so far, except that the file I > want to analyze seems to contradict the specifications of the document > mentioned above. Specifically, after processing about five > HEAP_DUMP_SEGMENTS with around 1.5 million sub-records in each, I come > across some ROOT_JNI_LOCAL records. The first 15 follow the format > specified in the above document (one 8 byte "ID" and two four byte values.) > But the 16th omits the two four byte records (well, it might simply have > more, but visual analysis shows that after the 8 byte ID, I have a new > block tag, and a believable structure. > > My current intention is to write a "smart" parser that investigates whether > processing will succeed or not if the "normal" format is read, or will use > the alternate format if it realizes that it would result in the next block > starting with an illegal tag value. Clearly this is not ideal! > > Is there any more detailed, newer, better, information? Or anything else I > should know to pursue this tool (or indeed a simple object frequency by > classname result) from an hprof 1.0.2 format file? > > (And yes, I'm pursuing a putative memory leak :) > > Thanks for any input (including "dude, this is the wrong list!") > Cheers, > Simon > > > From simon at dancingcloudservices.com Wed Oct 31 09:16:25 2018 From: simon at dancingcloudservices.com (Simon Roberts) Date: Wed, 31 Oct 2018 11:16:25 +0200 Subject: hprof format question In-Reply-To: <5e9255d2-4cb3-4390-6254-f4bd3d5d66e1@oracle.com> References: <5e9255d2-4cb3-4390-6254-f4bd3d5d66e1@oracle.com> Message-ID: Ah, my apologies, many thanks for the guidance. I'll try there. On Wed, Oct 31, 2018 at 10:41 AM Magnus Ihse Bursie < magnus.ihse.bursie at oracle.com> wrote: > Hi Simon, > > On 2018-10-24 00:26, Simon Roberts wrote: > > Hi all, I'm hoping this is the correct list for a question on the hprof > > file format (1.0.2)? > I don't think it is. You might get better response from > serviceability-dev at openjdk.java.net. > > /Magnus > > > > > I found this information: > > > http://hg.openjdk.java.net/jdk6/jdk6/jdk/raw-file/tip/src/share/demo/jvmti/hprof/manual.html > > > > and have been working on a small project to read these files. (Yes, I > know > > that NetBeans/VisualVM and Eclipse both have such libraries, and a number > > of other tools have been derived from those, but so far as I can tell, > they > > all are fundamentally built on the notion of fully decoding everything, > and > > creating memory representations of the entire heap. I want to pull out > only > > certain pieces of information--specifically object counts by class--from > a > > large, ~20Gb, dump file, and those tools just give up the ghost on my > > systems.) > > > > Anyway, my code reads the file pretty well so far, except that the file I > > want to analyze seems to contradict the specifications of the document > > mentioned above. Specifically, after processing about five > > HEAP_DUMP_SEGMENTS with around 1.5 million sub-records in each, I come > > across some ROOT_JNI_LOCAL records. The first 15 follow the format > > specified in the above document (one 8 byte "ID" and two four byte > values.) > > But the 16th omits the two four byte records (well, it might simply have > > more, but visual analysis shows that after the 8 byte ID, I have a new > > block tag, and a believable structure. > > > > My current intention is to write a "smart" parser that investigates > whether > > processing will succeed or not if the "normal" format is read, or will > use > > the alternate format if it realizes that it would result in the next > block > > starting with an illegal tag value. Clearly this is not ideal! > > > > Is there any more detailed, newer, better, information? Or anything else > I > > should know to pursue this tool (or indeed a simple object frequency by > > classname result) from an hprof 1.0.2 format file? > > > > (And yes, I'm pursuing a putative memory leak :) > > > > Thanks for any input (including "dude, this is the wrong list!") > > Cheers, > > Simon > > > > > > > > -- Simon Roberts (303) 249 3613