From huizhe.wang at oracle.com Fri Dec 1 06:11:55 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 30 Nov 2017 22:11:55 -0800 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter Message-ID: <5A20F2AB.3050608@oracle.com> Hi, Adding convenient methods that take a Charset to several classes that have already had methods with a Charset/Encoding name as a parameter. To avoid any impact on compatibility and JCK tests, we've kept the existing methods virtually untouched except making a reference to the new ones to encourage the use of these new methods going forward. The javadocs of the new methods however, may be more complete with details on handling edge cases / Exceptions. JBS: https://bugs.openjdk.java.net/browse/JDK-8183743 webrev: http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html Thanks, Joe From huizhe.wang at oracle.com Fri Dec 1 06:23:28 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 30 Nov 2017 22:23:28 -0800 Subject: RFR (JDK10/JAXP) 8191938: Fix lint warnings in JAXP repo: a few Deprecation warrnings and enable -Xlint:all In-Reply-To: <5A205B0E.80409@oracle.com> References: <5A1DA6EC.6080702@oracle.com> <5A1F035B.7060900@oracle.com> <5A1F307B.6080601@oracle.com> <5A205B0E.80409@oracle.com> Message-ID: <5A20F560.5060607@oracle.com> Hi all, The webrev is updated with a javac flag that includes a set of public APIs in java.xml that we care about with regards to doclint. http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/ Thanks, Joe On 11/30/17, 11:25 AM, Joe Wang wrote: > Hi Jon, all, > > For the LastModified tag, yes, "make docs" was fine. Note that it will > appear only in impl classes. > > For the webrev, I also updated the javac flag from doclint:none to a > set of java.xml public packages that we care about. > > http://cr.openjdk.java.net/%7Ejoehw/jdk10/8191938/webrev/ > > Thanks, > Joe > > On 11/29/17, 2:11 PM, Jonathan Gibbons wrote: >> Joe, >> >> I presume javadoc is OK with this not-quite-a-doc-comment-tag, when >> you do "make docs" ... >> >> -- Jon >> >> >> On 11/29/2017 10:58 AM, Joe Wang wrote: >>> Hi Joe, >>> >>> I moved the LastModified to the bottom of the class comment block. >>> Please let me know what you think: >>> http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/index.html >>> >>> Thanks, >>> Joe >>> >>> On 11/28/17, 10:19 AM, joe darcy wrote: >>>> Hi Joe, >>>> >>>> The code changes look fine, but the copyright blocks should *not* >>>> be updated to include a "@LastModified: Nov 2017" comment. >>>> >>>> Cheers, >>>> >>>> -Joe >>>> >>>> >>>> On 11/28/2017 10:11 AM, Joe Wang wrote: >>>>> Hi, >>>>> >>>>> Please review a fix for a few more deprecation warnings. Compiling >>>>> with -Xlint:all showed that these were the last few warnings. We >>>>> can then enable -Xlint:all for the java.xml module. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8191938 >>>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/ >>>>> >>>>> Thanks, >>>>> Joe >>>> >> From joe.darcy at oracle.com Fri Dec 1 06:43:00 2017 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 30 Nov 2017 22:43:00 -0800 Subject: RFR (JDK10/JAXP) 8191938: Fix lint warnings in JAXP repo: a few Deprecation warrnings and enable -Xlint:all In-Reply-To: <5A20F560.5060607@oracle.com> References: <5A1DA6EC.6080702@oracle.com> <5A1F035B.7060900@oracle.com> <5A1F307B.6080601@oracle.com> <5A205B0E.80409@oracle.com> <5A20F560.5060607@oracle.com> Message-ID: Looks fine Joe; good to see this work get in :-) Thanks, -Joe On 11/30/2017 10:23 PM, Joe Wang wrote: > Hi all, > > The webrev is updated with a javac flag that includes a set of public > APIs in java.xml that we care about with regards to doclint. > > http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/ > > Thanks, > Joe > > On 11/30/17, 11:25 AM, Joe Wang wrote: >> Hi Jon, all, >> >> For the LastModified tag, yes, "make docs" was fine. Note that it >> will appear only in impl classes. >> >> For the webrev, I also updated the javac flag from doclint:none to a >> set of java.xml public packages that we care about. >> >> http://cr.openjdk.java.net/%7Ejoehw/jdk10/8191938/webrev/ >> >> Thanks, >> Joe >> >> On 11/29/17, 2:11 PM, Jonathan Gibbons wrote: >>> Joe, >>> >>> I presume javadoc is OK with this not-quite-a-doc-comment-tag, when >>> you do "make docs" ... >>> >>> -- Jon >>> >>> >>> On 11/29/2017 10:58 AM, Joe Wang wrote: >>>> Hi Joe, >>>> >>>> I moved the LastModified to the bottom of the class comment block. >>>> Please let me know what you think: >>>> http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/index.html >>>> >>>> Thanks, >>>> Joe >>>> >>>> On 11/28/17, 10:19 AM, joe darcy wrote: >>>>> Hi Joe, >>>>> >>>>> The code changes look fine, but the copyright blocks should *not* >>>>> be updated to include a "@LastModified: Nov 2017" comment. >>>>> >>>>> Cheers, >>>>> >>>>> -Joe >>>>> >>>>> >>>>> On 11/28/2017 10:11 AM, Joe Wang wrote: >>>>>> Hi, >>>>>> >>>>>> Please review a fix for a few more deprecation warnings. >>>>>> Compiling with -Xlint:all showed that these were the last few >>>>>> warnings. We can then enable -Xlint:all for the java.xml module. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8191938 >>>>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/ >>>>>> >>>>>> Thanks, >>>>>> Joe >>>>> >>> From nishit.jain at oracle.com Fri Dec 1 08:17:16 2017 From: nishit.jain at oracle.com (Nishit Jain) Date: Fri, 1 Dec 2017 13:47:16 +0530 Subject: [10] RFR 8187551: MessageFormat.setFormat(int, Format) AIOOBE not thrown when documented Message-ID: <53f445ff-a4b3-5029-77f0-45125514a662@oracle.com> Hi, Please review the fix for JDK-8187551. Bug: https://bugs.openjdk.java.net/browse/JDK-8187551 Webrev: http://cr.openjdk.java.net/~nishjain/8187551/webrev.02/ Fix: As documented, updated MessageFormat.setFormat() to throw AIOOBE with invalid format element index. Thanks Martin Buchholz for filing the issue and suggesting the fix. Regards, Nishit Jain From Alan.Bateman at oracle.com Fri Dec 1 10:32:56 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 1 Dec 2017 10:32:56 +0000 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> Message-ID: <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> On 30/11/2017 22:57, mandy chung wrote: > : > > This is indeed a bug that should throw ISE when > ClassLoader.getSystemClassLoader is called during the initialization > of the system class loader, as the spec states. > > line 1921: I suggest to revise the message to make it clearer: > ? ?? "getSystemClassLoader cannot be called during the system class > loader instantiation" > > In ClassLoader::initSystemClassLoader (line 1971), when the exception > is thrown via Constructor::newInstance, it would be good to the cause > of an InvocationTargetException like: > > +??????????????? Throwable t = e; > +??????????????? if (e instanceof InvocationTargetException) { > +??????????????????? t = e.getCause(); > +??????????????? } > +??????????????? throw new Error(t.getMessage(), t); Better still might be for initSystemClassLoader to re-throw the cause so that it appears immediately after the "Error occurred during initialization of VM" message that the VM will fail with. -Alan From Alan.Bateman at oracle.com Fri Dec 1 11:16:52 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 1 Dec 2017 11:16:52 +0000 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: <5A20F2AB.3050608@oracle.com> References: <5A20F2AB.3050608@oracle.com> Message-ID: <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> On 01/12/2017 06:11, Joe Wang wrote: > Hi, > > Adding convenient methods that take a Charset to several classes that > have already had methods with a Charset/Encoding name as a parameter. > To avoid any impact on compatibility and JCK tests, we've kept the > existing methods virtually untouched except making a reference to the > new ones to encourage the use of these new methods going forward. The > javadocs of the new methods however, may be more complete with details > on handling edge cases / Exceptions. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8183743 > webrev: http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html I looked through the javadoc for the updated/new methods and it mostly looks good. I think URLDecoder.decode methods should have @throws IllegalArgumentException. I realize it's implementation specific as to whether IAE is thrown with bad input but given that the RI does throw IAE then consumers of the API should be prepared for it. The @implNote can stay and probably should be copied into the legacy decode method too. One of the new constructors on Scanner needs @since 10. -Alan From kumar.x.srinivasan at oracle.com Fri Dec 1 14:41:44 2017 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Fri, 01 Dec 2017 06:41:44 -0800 Subject: RFR: 8189102: All tools should support -?, -h and --help In-Reply-To: References: <4a021bc072f24c7b9d1bb3b00b774867@sap.com> <5A1C4081.2090107@oracle.com> Message-ID: <5A216A28.3030702@oracle.com> Hi, Besides my private comments to you, there are 2-3 internal tests which fail. Have you run all the langtools tests ? There are 4 Windows tests that fail with langtools: jdk/javadoc/doclet/testHelpOption/TestHelpOption.java jdk/javadoc/tool/CheckResourceKeys.java jdk/javadoc/tool/ToolProviderTest.java tools/javap/InvalidOptions.java tools/jdeps/MultiReleaseJar.java This changeset needs to be vetted thoroughly using internal build and test systems. Any Oracle engineer willing to sponsor this ? Kumar On 11/28/2017 3:16 AM, Lindenmaier, Goetz wrote: > Hi, > > I uploaded a new webrev: > http://cr.openjdk.java.net/~goetz/wr17/8189102-helpMessage/webrev.04/ > > This includes the changes > - to jshell requested by Robert > - to the test as requested by Kumar. > > See also incremental patch and the test output including all the > help messages referenced in the webrev. > > Best regards, > Goetz. > >> -----Original Message----- >> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] >> Sent: Montag, 27. November 2017 17:43 >> To: Lindenmaier, Goetz >> Cc: core-libs-dev at openjdk.java.net; 'compiler-dev at openjdk.java.net' >> ; serviceability-dev (serviceability- >> dev at openjdk.java.net) >> Subject: Re: RFR: 8189102: All tools should support -?, -h and --help >> >> Hi, >> >> I looked at some of the components I maintain, and they look good. >> >> Wrt. the test: >> >> 1. The local variables/fields don't comply with the coding conventions, >> we have been >> following in the JDK. >> >> 2. Hmm, someone parked a .ini file in bin directory, later removed, >> perhaps on windows simply check for .exe ? Should a check exist >> to verify if file has executable permissions ?"File.canExecute". >> >> 171 // Returns true if the file is not a tool. >> 172 static boolean notATool(String file) { >> 173 file = file.toLowerCase(); >> 174 if (file.endsWith(".dll") || >> 175 file.endsWith(".map") || >> 176 file.endsWith(".pdb") || >> 177 file.equals("server")) { // server subdir on windows. >> 178 return true; >> 179 } >> 180 return false; >> 181 } >> >> >> 3. Typo in comment >> >> 201 // Check whether 'flag' appears in 'line' as a word of itself. I must not >> >> s/I must/It must/ >> >> >> 4. There is a method to check for isEnglishLocale in TestHelper, perhaps >> use it >> to have the test bale out in non english locales as early as possible ? >> >> 5. Has this been tested on all platforms ? do you need help testing it ? >> >> Kumar >> >> >> On 11/17/2017 3:23 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> please review this change. I also filed a CSR for this: >>> http://cr.openjdk.java.net/~goetz/wr17/8189102- >> helpMessage/webrev.02/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8189102 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8191477 >>> >>> See the webrev for a detailed description of the changes. >>> >>> If required, I'll make break-out changes to be reviewed separately. >>> >>> I had posted a RFR before, but improved the change to >>> give a more complete overview of currently supported flags >>> for the CSR: >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017- >> October/028615.html >>> Best regards, >>> Goetz. >>> From Roger.Riggs at Oracle.com Fri Dec 1 15:10:55 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 1 Dec 2017 10:10:55 -0500 Subject: [10] RFR 8187551: MessageFormat.setFormat(int, Format) AIOOBE not thrown when documented In-Reply-To: <53f445ff-a4b3-5029-77f0-45125514a662@oracle.com> References: <53f445ff-a4b3-5029-77f0-45125514a662@oracle.com> Message-ID: Hi Nishit, Please add the new test to test/jdk/java/text/Format/MessageFormat/MessageRegression.java instead of creating a new test. Also the convention for test names should be a functional description of the test; bug numbers are uninformative. (I'm a bit surprised there are few/no regression tests for setFormat). Thanks, Roger On 12/1/2017 3:17 AM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8187551. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8187551 > Webrev: http://cr.openjdk.java.net/~nishjain/8187551/webrev.02/ > > Fix: As documented, updated MessageFormat.setFormat() to throw AIOOBE > with invalid format element index. > > Thanks Martin Buchholz for filing the issue and suggesting the fix. > > Regards, > Nishit Jain From goetz.lindenmaier at sap.com Fri Dec 1 15:16:47 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 1 Dec 2017 15:16:47 +0000 Subject: RFR: 8189102: All tools should support -?, -h and --help In-Reply-To: <5A216A28.3030702@oracle.com> References: <4a021bc072f24c7b9d1bb3b00b774867@sap.com> <5A1C4081.2090107@oracle.com> <5A216A28.3030702@oracle.com> Message-ID: <2ac9c1e37d8e493aa5b581d1a69862e8@sap.com> Hi Kumar, I'm already looking at the issues from your other mail. Detailed comments will follow. I'll also check and fix the new tests you report. I think we don't run the langtool tests, i.e. I know we didn't run them before the repos were merged. Obviously we should add them to our testing :) Thanks for further testing and best regards, Goetz. > -----Original Message----- > From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] > Sent: Freitag, 1. Dezember 2017 15:42 > To: Lindenmaier, Goetz > Cc: core-libs-dev at openjdk.java.net; 'compiler-dev at openjdk.java.net' > ; serviceability-dev (serviceability- > dev at openjdk.java.net) > Subject: Re: RFR: 8189102: All tools should support -?, -h and --help > > Hi, > > Besides my private comments to you, there are 2-3 internal tests > which fail. > > Have you run all the langtools tests ? There are 4 Windows tests > that fail with langtools: > > jdk/javadoc/doclet/testHelpOption/TestHelpOption.java > jdk/javadoc/tool/CheckResourceKeys.java > jdk/javadoc/tool/ToolProviderTest.java > tools/javap/InvalidOptions.java > tools/jdeps/MultiReleaseJar.java > > This changeset needs to be vetted thoroughly using internal build and test > systems. > Any Oracle engineer willing to sponsor this ? > > Kumar > > > On 11/28/2017 3:16 AM, Lindenmaier, Goetz wrote: > > > Hi, > > I uploaded a new webrev: > http://cr.openjdk.java.net/~goetz/wr17/8189102- > helpMessage/webrev.04/ > > This includes the changes > - to jshell requested by Robert > - to the test as requested by Kumar. > > See also incremental patch and the test output including all the > help messages referenced in the webrev. > > Best regards, > Goetz. > > > -----Original Message----- > From: Kumar Srinivasan > [mailto:kumar.x.srinivasan at oracle.com] > Sent: Montag, 27. November 2017 17:43 > To: Lindenmaier, Goetz > > Cc: core-libs-dev at openjdk.java.net dev at openjdk.java.net> ; 'compiler-dev at openjdk.java.net > ' > dev at openjdk.java.net> ; serviceability-dev (serviceability- > dev at openjdk.java.net ) > dev at openjdk.java.net> > Subject: Re: RFR: 8189102: All tools should support -?, -h and - > -help > > Hi, > > I looked at some of the components I maintain, and they look > good. > > Wrt. the test: > > 1. The local variables/fields don't comply with the coding > conventions, > we have been > following in the JDK. > > 2. Hmm, someone parked a .ini file in bin directory, later > removed, > perhaps on windows simply check for .exe ? Should a > check exist > to verify if file has executable permissions > ?"File.canExecute". > > 171 // Returns true if the file is not a tool. > 172 static boolean notATool(String file) { > 173 file = file.toLowerCase(); > 174 if (file.endsWith(".dll") || > 175 file.endsWith(".map") || > 176 file.endsWith(".pdb") || > 177 file.equals("server")) { // server subdir on > windows. > 178 return true; > 179 } > 180 return false; > 181 } > > > 3. Typo in comment > > 201 // Check whether 'flag' appears in 'line' as a word of > itself. I must not > > s/I must/It must/ > > > 4. There is a method to check for isEnglishLocale in > TestHelper, perhaps > use it > to have the test bale out in non english locales as early as > possible ? > > 5. Has this been tested on all platforms ? do you need help > testing it ? > > Kumar > > > On 11/17/2017 3:23 AM, Lindenmaier, Goetz wrote: > > Hi, > > please review this change. I also filed a CSR for this: > http://cr.openjdk.java.net/~goetz/wr17/8189102- > > helpMessage/webrev.02/ > > Bug: https://bugs.openjdk.java.net/browse/JDK- > 8189102 > CSR: https://bugs.openjdk.java.net/browse/JDK- > 8191477 > > See the webrev for a detailed description of the > changes. > > If required, I'll make break-out changes to be > reviewed separately. > > I had posted a RFR before, but improved the change > to > give a more complete overview of currently > supported flags > for the CSR: > http://mail.openjdk.java.net/pipermail/hotspot- > dev/2017- > > October/028615.html > > > Best regards, > Goetz. > > > > From mandy.chung at oracle.com Fri Dec 1 16:33:28 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Dec 2017 08:33:28 -0800 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> Message-ID: On 12/1/17 2:32 AM, Alan Bateman wrote: > On 30/11/2017 22:57, mandy chung wrote: >> : >> >> This is indeed a bug that should throw ISE when >> ClassLoader.getSystemClassLoader is called during the initialization >> of the system class loader, as the spec states. >> >> line 1921: I suggest to revise the message to make it clearer: >> ? ?? "getSystemClassLoader cannot be called during the system class >> loader instantiation" >> >> In ClassLoader::initSystemClassLoader (line 1971), when the exception >> is thrown via Constructor::newInstance, it would be good to the cause >> of an InvocationTargetException like: >> >> +??????????????? Throwable t = e; >> +??????????????? if (e instanceof InvocationTargetException) { >> +??????????????????? t = e.getCause(); >> +??????????????? } >> +??????????????? throw new Error(t.getMessage(), t); > Better still might be for initSystemClassLoader to re-throw the cause > so that it appears immediately after the "Error occurred during > initialization of VM" message that the VM will fail with. Yes that would be better. Mandy From huizhe.wang at oracle.com Fri Dec 1 18:08:01 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 01 Dec 2017 10:08:01 -0800 Subject: RFR (JDK10/JAXP) 8191938: Fix lint warnings in JAXP repo: a few Deprecation warrnings and enable -Xlint:all In-Reply-To: References: <5A1DA6EC.6080702@oracle.com> <5A1F035B.7060900@oracle.com> <5A1F307B.6080601@oracle.com> <5A205B0E.80409@oracle.com> <5A20F560.5060607@oracle.com> Message-ID: <5A219A81.6060009@oracle.com> Thanks Joe! Pushed. Thanks for helping me getting this work done, finally! From 5230 warnings in your original count, plus some new warnings introduced by JDK 9, down to zero, it's been two years already. I'm happy we join in the big family to be lint free :-) -Joe On 11/30/17, 10:43 PM, joe darcy wrote: > Looks fine Joe; good to see this work get in :-) > > Thanks, > > -Joe > > > On 11/30/2017 10:23 PM, Joe Wang wrote: >> Hi all, >> >> The webrev is updated with a javac flag that includes a set of public >> APIs in java.xml that we care about with regards to doclint. >> >> http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/ >> >> Thanks, >> Joe >> >> On 11/30/17, 11:25 AM, Joe Wang wrote: >>> Hi Jon, all, >>> >>> For the LastModified tag, yes, "make docs" was fine. Note that it >>> will appear only in impl classes. >>> >>> For the webrev, I also updated the javac flag from doclint:none to a >>> set of java.xml public packages that we care about. >>> >>> http://cr.openjdk.java.net/%7Ejoehw/jdk10/8191938/webrev/ >>> >>> Thanks, >>> Joe >>> >>> On 11/29/17, 2:11 PM, Jonathan Gibbons wrote: >>>> Joe, >>>> >>>> I presume javadoc is OK with this not-quite-a-doc-comment-tag, when >>>> you do "make docs" ... >>>> >>>> -- Jon >>>> >>>> >>>> On 11/29/2017 10:58 AM, Joe Wang wrote: >>>>> Hi Joe, >>>>> >>>>> I moved the LastModified to the bottom of the class comment block. >>>>> Please let me know what you think: >>>>> http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/index.html >>>>> >>>>> Thanks, >>>>> Joe >>>>> >>>>> On 11/28/17, 10:19 AM, joe darcy wrote: >>>>>> Hi Joe, >>>>>> >>>>>> The code changes look fine, but the copyright blocks should *not* >>>>>> be updated to include a "@LastModified: Nov 2017" comment. >>>>>> >>>>>> Cheers, >>>>>> >>>>>> -Joe >>>>>> >>>>>> >>>>>> On 11/28/2017 10:11 AM, Joe Wang wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review a fix for a few more deprecation warnings. >>>>>>> Compiling with -Xlint:all showed that these were the last few >>>>>>> warnings. We can then enable -Xlint:all for the java.xml module. >>>>>>> >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8191938 >>>>>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8191938/webrev/ >>>>>>> >>>>>>> Thanks, >>>>>>> Joe >>>>>> >>>> > From brent.christian at oracle.com Fri Dec 1 19:40:16 2017 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 1 Dec 2017 11:40:16 -0800 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> Message-ID: On 12/1/17 8:33 AM, mandy chung wrote: >> Better still might be for initSystemClassLoader to re-throw the cause >> so that it appears immediately after the "Error occurred during >> initialization of VM" message that the VM will fail with. > > Yes that would be better. So would I do this for RuntimeException and Error, and then package checked types of causes in an Error, as Mandy suggested? (We don't want to throw a checked exception from initSystemClassLoader(), I don't think.) Thanks, -Brent From mandy.chung at oracle.com Fri Dec 1 20:16:52 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Dec 2017 12:16:52 -0800 Subject: RFR 8186961 Class.getFields() does not return fields of previously visited super interfaces/classes. In-Reply-To: References: <929496D6-A4B3-4EF2-BDCB-D2504C1BECE3@oracle.com> Message-ID: On 11/30/17 12:17 PM, Paul Sandoz wrote: > Here is the updated webrev: > > http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8186961-iface-static-fields-get/webrev/ > > I opted for the simple solution using a LinkedHashSet. This fix looks good except a typo in the test: 94 "Class %s does not have extracly one field: %d", c.getName(), nfs)); s/extracly/exactly/ I am wondering if these @run should run with both othervm and agentvm mode since it currently depends on the jtreg command-line (I believe our test target uses agentvm as the default). > It?s possible to heavily optimize (avoiding the production of a linked hash set until required [*]) but the resulting code is harder to understand. I was tempted to come up with optimization too when first reading the patch but I am with you.? I like the resulting code simple and clear.?? The difference is an array list vs linked hash set which we should wait until this is really a performance issue.? FWIW getMethods implementation also creates a linked hash set if not cached. Mandy > Paul. > > [*] when there are two or more super interface with fields, or one or more super interfaces and a super classes with fields. > > >> On 30 Nov 2017, at 08:41, Paul Sandoz wrote: >> >> Hi Caes, >> >> As we discussed off list the post loop logic is easily missed. >> >> However, i found another obvious issue i missed with two classes (super/sub) extending from the same interface that declares a field. See updated test case in the webrev. >> >> I have an idea to retain Field[] arrays and then process ?em all at the end of the method to produce the final array. That should hopefully make the logic clearer too. >> >> Paul. >> >> >>> On 29 Nov 2017, at 16:00, Claes Redestad wrote: >>> >>> Hi Paul, >>> >>> it seems you'll effectively skip processing of the last interface of c in the new code - is this intentional? >>> >>> 3049 Field[] iFields = null; >>> 3050 for (Class c : getInterfaces()) { >>> 3051 if (iFields != null && iFields.length > 0) { >>> ... >>> 3060 } >>> 3061 iFields = c.privateGetPublicFields(); >>> 3062 } >>> >>> ifaces is unused: >>> >>> 3047 Class[] ifaces = getInterfaces(); >>> >>> Nit: You could probably simplify code by replacing List fields with the LinkedHashSet directly, removing the need to create it conditionally. >>> >>> /Claes >>> >>> >>> On 2017-11-29 21:15, Paul Sandoz wrote: >>>> Hi, >>>> >>>> Please review: >>>> >>>> http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8186961-iface-static-fields-get/webrev/ >>>> >>>> There is a bug lurking, perhaps for a while, where diamond patterns for interface hierarchies can result in the incorrect reporting of fields when using reflection, see the test case. >>>> >>>> Since reflection data is cached i don?t see an advantage to retaining a set of of traversed interfaces, which is the root cause of the issue. >>>> >>>> The code is optimized for common cases and will for less common cases collect fields of interfaces into a temporary linked hash set (to preserve the order, perhaps not strictly necessary but i did not want to change that behaviour). >>>> >>>> Thanks, >>>> Paul. From huizhe.wang at oracle.com Fri Dec 1 20:35:36 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 01 Dec 2017 12:35:36 -0800 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> References: <5A20F2AB.3050608@oracle.com> <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> Message-ID: <5A21BD18.2050201@oracle.com> On 12/1/17, 3:16 AM, Alan Bateman wrote: > > > On 01/12/2017 06:11, Joe Wang wrote: >> Hi, >> >> Adding convenient methods that take a Charset to several classes that >> have already had methods with a Charset/Encoding name as a parameter. >> To avoid any impact on compatibility and JCK tests, we've kept the >> existing methods virtually untouched except making a reference to the >> new ones to encourage the use of these new methods going forward. The >> javadocs of the new methods however, may be more complete with >> details on handling edge cases / Exceptions. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8183743 >> webrev: >> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html > I looked through the javadoc for the updated/new methods and it mostly > looks good. Thanks for all of the reviews! > > I think URLDecoder.decode methods should have @throws > IllegalArgumentException. I realize it's implementation specific as to > whether IAE is thrown with bad input but given that the RI does throw > IAE then consumers of the API should be prepared for it. The @implNote > can stay and probably should be copied into the legacy decode method too. I added @throws IAE. On a 2nd thought, would that give no flexibility to alternative impls as the general (class) spec had given? With this addition, it becomes a requirement. > > One of the new constructors on Scanner needs @since 10. Fixed. -Joe > > -Alan From paul.sandoz at oracle.com Fri Dec 1 22:52:59 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 1 Dec 2017 14:52:59 -0800 Subject: RFR 8186961 Class.getFields() does not return fields of previously visited super interfaces/classes. In-Reply-To: References: <929496D6-A4B3-4EF2-BDCB-D2504C1BECE3@oracle.com> Message-ID: > On 1 Dec 2017, at 12:16, mandy chung wrote: > > > > On 11/30/17 12:17 PM, Paul Sandoz wrote: >> Here is the updated webrev: >> >> >> http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8186961-iface-static-fields-get/webrev/ >> >> >> I opted for the simple solution using a LinkedHashSet. >> > > This fix looks good except a typo in the test: > > 94 "Class %s does not have extracly one field: %d", c.getName(), nfs)); > > s/extracly/exactly/ > Fixed. > I am wondering if these @run should run with both othervm and agentvm mode since > it currently depends on the jtreg command-line > (I believe our test target uses > agentvm as the default). > It does. * @run main/othervm StaticFieldsOnInterface C * @run main/othervm StaticFieldsOnInterface D * @run main/othervm StaticFieldsOnInterface Y This ok? >> It?s possible to heavily optimize (avoiding the production of a linked hash set until required [*]) but the resulting code is harder to understand. >> > > I was tempted to come up with optimization too when first reading the patch but I am with you. I like the resulting code simple and clear. The difference is an array list vs linked hash set which we should wait until this is really a performance issue. FWIW getMethods implementation also creates a linked hash set if not cached. > Ok, thanks, Paul. From mandy.chung at oracle.com Fri Dec 1 23:27:39 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 1 Dec 2017 15:27:39 -0800 Subject: RFR 8186961 Class.getFields() does not return fields of previously visited super interfaces/classes. In-Reply-To: References: <929496D6-A4B3-4EF2-BDCB-D2504C1BECE3@oracle.com> Message-ID: <7114D370-91AF-440A-9252-0F24FC5701B1@oracle.com> > On Dec 1, 2017, at 2:52 PM, Paul Sandoz wrote: > > > >> On 1 Dec 2017, at 12:16, mandy chung wrote: >> >> >> >>> On 11/30/17 12:17 PM, Paul Sandoz wrote: >>> Here is the updated webrev: >>> >>> >>> http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8186961-iface-static-fields-get/webrev/ >>> >>> >>> I opted for the simple solution using a LinkedHashSet. >>> >> >> This fix looks good except a typo in the test: >> >> 94 "Class %s does not have extracly one field: %d", c.getName(), nfs)); >> >> s/extracly/exactly/ >> > > Fixed. > > >> I am wondering if these @run should run with both othervm and agentvm mode since >> it currently depends on the jtreg command-line > > >> (I believe our test target uses >> agentvm as the default). >> > > It does. > > * @run main/othervm StaticFieldsOnInterface C > * @run main/othervm StaticFieldsOnInterface D > * @run main/othervm StaticFieldsOnInterface Y > > This ok? > Yes. Mandy > >>> It?s possible to heavily optimize (avoiding the production of a linked hash set until required [*]) but the resulting code is harder to understand. >>> >> >> I was tempted to come up with optimization too when first reading the patch but I am with you. I like the resulting code simple and clear. The difference is an array list vs linked hash set which we should wait until this is really a performance issue. FWIW getMethods implementation also creates a linked hash set if not cached. >> > > Ok, thanks, > Paul. From paul.sandoz at oracle.com Fri Dec 1 23:40:55 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 1 Dec 2017 15:40:55 -0800 Subject: RFR JDK-8191918: tomcat gzip-compressed response bodies appear to be broken in update 151 In-Reply-To: <5A208A37.3030109@oracle.com> References: <5A208A37.3030109@oracle.com> Message-ID: <40D2F6A6-95C9-40C1-8F9B-E2C416BFCD58@oracle.com> > On 30 Nov 2017, at 14:46, Xueming Shen wrote: > > Hi, > > Please help review the change for JDK-8191918: > > issue: https://bugs.openjdk.java.net/browse/JDK-8191918 > webrev: http://cr.openjdk.java.net/~sherman/8191918/webrev > InflateIn_DeflateOut.java ? 174 GZIPInputStream gzis = new GZIPInputStream(byteInStream); 175 ByteArrayOutputStream baos = new ByteArrayOutputStream(); 176 int numRead; 177 byte[] b = new byte[4 * 1024]; 178 try { 179 while ((numRead = gzis.read(buf)) >= 0) { 180 baos.write(b, 0, numRead); 181 } 182 } finally { 183 baos.close(); 184 } Can you use gzis.transferTo(baos)? Paul. > This is the backport/identical change we have already putback into earlier update > releases for JDK-8189789. It includes two changes > > (1) to replace the change we put into jdk9 for #8184306 with the "official" > change currently in zlib github repo (https://github.com/madler/zlib/issues/275) > > http://cr.openjdk.java.net/~sherman/8184306 > https://bugs.openjdk.java.net/browse/JDK-8184306 > > (2) to change the way we handle the returned result Z_BUF_ERROR. Now the > implementation expects that there might be bytes read from the input and bytes > written to the output buffer when the deflateParams()/deflate() returns > Z_BUF_ERROR. It is "interpreted as there is no enough buffer space for all output, > but the deflater has decoded input bytes and write out output bytes as many as > possible, come back with more available output space". > > See related github/zlib discussions at #275 [1] and #305[2] > > Thanks, > Sherman > > [1] https://github.com/madler/zlib/issues/275 > [2] https://github.com/madler/zlib/issues/305 From martinrb at google.com Sat Dec 2 00:42:24 2017 From: martinrb at google.com (Martin Buchholz) Date: Fri, 1 Dec 2017 16:42:24 -0800 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc Message-ID: 1. JDK-8192935 http://cr.openjdk.java.net/~martin/webrevs/openjdk10/EnumSet-SerializationProxy/EnumSet-SerializationProxy.patch From brent.christian at oracle.com Sat Dec 2 00:46:10 2017 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 1 Dec 2017 16:46:10 -0800 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> Message-ID: <08ea805e-4da5-3bd9-904c-9a56e7899d03@oracle.com> On 12/1/17 11:40 AM, Brent Christian wrote: > On 12/1/17 8:33 AM, mandy chung wrote: >>> Better still might be for initSystemClassLoader to re-throw the cause >>> so that it appears immediately after the "Error occurred during >>> initialization of VM" message that the VM will fail with. >> >> Yes that would be better. > > So would I do this for RuntimeException and Error, and then package > checked types of causes in an Error, as Mandy suggested?? (We don't want > to throw a checked exception from initSystemClassLoader(), I don't think.) Anyway, this is what that looks like (I did RuntimeException, but didn't bother with Error). I also made the test changes that Mandy suggested. http://cr.openjdk.java.net/~bchristi/8187222/webrev.01/index.html Thanks, -Brent From xueming.shen at oracle.com Sat Dec 2 01:04:06 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 01 Dec 2017 17:04:06 -0800 Subject: RFR JDK-8191918: tomcat gzip-compressed response bodies appear to be broken in update 151 In-Reply-To: <40D2F6A6-95C9-40C1-8F9B-E2C416BFCD58@oracle.com> References: <5A208A37.3030109@oracle.com> <40D2F6A6-95C9-40C1-8F9B-E2C416BFCD58@oracle.com> Message-ID: <5A21FC06.5060408@oracle.com> On 12/1/17, 3:40 PM, Paul Sandoz wrote: > >> On 30 Nov 2017, at 14:46, Xueming Shen wrote: >> >> Hi, >> >> Please help review the change for JDK-8191918: >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8191918 >> webrev: http://cr.openjdk.java.net/~sherman/8191918/webrev >> > InflateIn_DeflateOut.java > ? > > 174 GZIPInputStream gzis = new GZIPInputStream(byteInStream); > 175 ByteArrayOutputStream baos = new ByteArrayOutputStream(); > 176 int numRead; > 177 byte[] b = new byte[4 * 1024]; > 178 try { > 179 while ((numRead = gzis.read(buf))>= 0) { > 180 baos.write(b, 0, numRead); > 181 } > 182 } finally { > 183 baos.close(); > 184 } > > Can you use gzis.transferTo(baos)? > > Paul. > > Thanks Paul! webrev has been updated as suggested. http://cr.openjdk.java.net/~sherman/8191918/webrev -Sherman From paul.sandoz at oracle.com Sat Dec 2 01:11:35 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 1 Dec 2017 17:11:35 -0800 Subject: RFR JDK-8191918: tomcat gzip-compressed response bodies appear to be broken in update 151 In-Reply-To: <5A21FC06.5060408@oracle.com> References: <5A208A37.3030109@oracle.com> <40D2F6A6-95C9-40C1-8F9B-E2C416BFCD58@oracle.com> <5A21FC06.5060408@oracle.com> Message-ID: <7206C05A-0219-4CBD-99EC-ACBAF5428DB3@oracle.com> +1 This is a good example of where ?var? can really reduce the verbosity, up to you. Also ByteArrayOutputStream.close is a no-op, you don?t need the finally block. Paul. > On 1 Dec 2017, at 17:04, Xueming Shen wrote: > > On 12/1/17, 3:40 PM, Paul Sandoz wrote: >> >>> On 30 Nov 2017, at 14:46, Xueming Shen wrote: >>> >>> Hi, >>> >>> Please help review the change for JDK-8191918: >>> >>> issue: https://bugs.openjdk.java.net/browse/JDK-8191918 >>> webrev: http://cr.openjdk.java.net/~sherman/8191918/webrev >>> >> InflateIn_DeflateOut.java >> ? >> >> 174 GZIPInputStream gzis = new GZIPInputStream(byteInStream); >> 175 ByteArrayOutputStream baos = new ByteArrayOutputStream(); >> 176 int numRead; >> 177 byte[] b = new byte[4 * 1024]; >> 178 try { >> 179 while ((numRead = gzis.read(buf))>= 0) { >> 180 baos.write(b, 0, numRead); >> 181 } >> 182 } finally { >> 183 baos.close(); >> 184 } >> >> Can you use gzis.transferTo(baos)? >> >> Paul. >> >> > > Thanks Paul! > > webrev has been updated as suggested. > > http://cr.openjdk.java.net/~sherman/8191918/webrev > > -Sherman From mandy.chung at oracle.com Sat Dec 2 01:17:09 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 1 Dec 2017 17:17:09 -0800 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: <08ea805e-4da5-3bd9-904c-9a56e7899d03@oracle.com> References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> <08ea805e-4da5-3bd9-904c-9a56e7899d03@oracle.com> Message-ID: <79EEA360-45C5-47AA-B48B-D9418F1A56B3@oracle.com> > On Dec 1, 2017, at 4:46 PM, Brent Christian wrote: > >> On 12/1/17 11:40 AM, Brent Christian wrote: >> On 12/1/17 8:33 AM, mandy chung wrote: >>>> Better still might be for initSystemClassLoader to re-throw the cause so that it appears immediately after the "Error occurred during initialization of VM" message that the VM will fail with. >>> >>> Yes that would be better. >> So would I do this for RuntimeException and Error, and then package checked types of causes in an Error, as Mandy suggested? (We don't want to throw a checked exception from initSystemClassLoader(), I don't think.) > > Anyway, this is what that looks like (I did RuntimeException, but didn't bother with Error). I also made the test changes that Mandy suggested. > > http://cr.openjdk.java.net/~bchristi/8187222/webrev.01/index.html > Yes and I think should also rethrow the cause if the cause is an Error (why not) Mandy From stuart.marks at oracle.com Sat Dec 2 01:20:45 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 1 Dec 2017 17:20:45 -0800 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: References: Message-ID: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> On 12/1/17 4:42 PM, Martin Buchholz wrote: > 1. JDK-8192935 > > http://cr.openjdk.java.net/~martin/webrevs/openjdk10/EnumSet-SerializationProxy/EnumSet-SerializationProxy.patch --- a/src/java.base/share/classes/java/util/EnumSet.java +++ b/src/java.base/share/classes/java/util/EnumSet.java @@ -75,7 +75,6 @@ * @author Josh Bloch * @since 1.5 * @see EnumMap - * @serial exclude */ I suspect you're following other examples in the JDK that include the serial form documentation for a class that uses a serialization proxy, but I think this is a mistake. It's a mistake because this will cause EnumSet to appear in serialized-form.html, but EnumSet actually should *never* appear in any serialized byte stream. This is quite confusing. I think those other places should be fixed, instead. Instead of including EnumSet itself in the serialized-form.html, how about restoring its "@serial exclude" and then putting a link directly from the EnumSet class doc to the EnumSet.SerializationProxy serial form doc? Something like *

When an instance of this class is serialized, it is replaced with the * serial form of an instance of * * {@code EnumSet.SerializationProxy}. The changes to EnumSet.SerializationProxy class are good. s'marks From mandy.chung at oracle.com Sat Dec 2 01:22:01 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 1 Dec 2017 17:22:01 -0800 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: <79EEA360-45C5-47AA-B48B-D9418F1A56B3@oracle.com> References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> <08ea805e-4da5-3bd9-904c-9a56e7899d03@oracle.com> <79EEA360-45C5-47AA-B48B-D9418F1A56B3@oracle.com> Message-ID: > On Dec 1, 2017, at 5:17 PM, Mandy Chung wrote: > > >>> On Dec 1, 2017, at 4:46 PM, Brent Christian wrote: >>> >>> On 12/1/17 11:40 AM, Brent Christian wrote: >>> On 12/1/17 8:33 AM, mandy chung wrote: >>>>> Better still might be for initSystemClassLoader to re-throw the cause so that it appears immediately after the "Error occurred during initialization of VM" message that the VM will fail with. >>>> >>>> Yes that would be better. >>> So would I do this for RuntimeException and Error, and then package checked types of causes in an Error, as Mandy suggested? (We don't want to throw a checked exception from initSystemClassLoader(), I don't think.) >> >> Anyway, this is what that looks like (I did RuntimeException, but didn't bother with Error). I also made the test changes that Mandy suggested. >> >> http://cr.openjdk.java.net/~bchristi/8187222/webrev.01/index.html >> > > Yes and I think should also rethrow the cause if the cause is an Error (why not) The instanceof RuntimeException check should be moved outside to the if-statement when it?s an instance of InvocationTargetException. Mandy From Roger.Riggs at oracle.com Sat Dec 2 02:38:43 2017 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 1 Dec 2017 21:38:43 -0500 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved Message-ID: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> Please review a revision to the work on remove a dependency on finalization from FileInputStream, FileOutputStream, and add cleanup of closing file descriptors in FileChannel. The previous version was too aggressive in removing the finalize method and was considered to be insufficiently backward compatible. For compatibility with previous versions, the close method will be called when the FIS/FOS is unreachable only when FIS/FOS has been subclassed to override close() or finalize().? In other cases, the cleaner is used to make sure the FileDescriptor is closed. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ Issue: https://bugs.openjdk.java.net/browse/JDK-8080225 CSR: Relax FileInputStream/FileOutputStream requirement to use finalize https://bugs.openjdk.java.net/browse/JDK-8187325 Thanks, Roger From Alan.Bateman at oracle.com Sun Dec 3 14:21:15 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 3 Dec 2017 14:21:15 +0000 Subject: MR JarFile was: Scanning multi version jars? In-Reply-To: References: Message-ID: <9b8ac72a-4766-c6af-31c4-ae3fe71c45a6@oracle.com> On 15/09/2017 11:37, Alan Bateman wrote: > On 15/09/2017 05:43, Greg Wilkins wrote: >> >> : >> >> The issue I have are that sometimes the class is trying to hide the >> MR aspects, yet other times it is not.? ?Specifically when iterating >> it returns JarEntry instances that ignore versions and always return >> the content to which they refer, yet if you obtain a JarEntry from >> the getJarEntry API, it behaves differently and may return the >> versioned content even if it has the un-versioned path. > The entries and stream methods have been discussed here several times. > The conclusion was that they would continue to enumerate or return a > stream over all entries in the JAR file.? A new versionedStream() > method was proposed and I agree is needed, it just didn't make it into > Java SE 9. Now might be the time to put it back on the table. Just to follow up from on this thread from September. jdk-10+34 has the JarFile::versionedStream and JarEntry::getRealName methods that we've been discussing here to complete the API support for MR JARs. It would be good to try them out and report back any issues that you find. -Alan From ralph.goers at dslextreme.com Sun Dec 3 18:25:08 2017 From: ralph.goers at dslextreme.com (Ralph Goers) Date: Sun, 3 Dec 2017 11:25:08 -0700 Subject: Android and Log4j Message-ID: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> Log4j added support for Java 9 by: Converting the Log4j-API jar to a multi-release jar that includes support for StackWalker and the new Process Id support. Adding a module-info.jar to the Log4j API jar. We are now getting complaints from Android users (as well as a few others) that their tools no longer work with log4j. During development I ran into problems with OSGi. The problems seem to mostly revolve around the fact that they can?t deal with the classes for Java 9. I was surprised that Android is failing on the classes in META-INF/versions/9 as I had assumed that would be an invalid location for a class file prior to Java 9, but that seems not to be the case. The fact that module-info.java turns into a class file also seems to be a problem since the various tools are seeing it and having problems with it. We have been discussing various ways to handle this in https://issues.apache.org/jira/browse/LOG4J2-2133 . There seems to be a strong push to just remove the support for Java 9 since it is breaking so many things. It seems impossible to have a module-info.java file in a jar that is going to be included in Android. If this had been a json file that was interpreted by the classloader or had a different file extension other than .class we wouldn?t be in this mess. We also need another mechanism to bring in our code that uses StackWalker as calling it via Reflection and emulating lambda expressions seems like it would be painful and slow. Do you have any recommendations on how we can resolve this impasse? Ralph From forax at univ-mlv.fr Sun Dec 3 19:02:19 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 3 Dec 2017 20:02:19 +0100 (CET) Subject: Android and Log4j In-Reply-To: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> Message-ID: <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> Ask Google to fix dx, dx should ignore the module-info.class and everything inside META-INF/versions (at least it's a first simple patch). cheers, R?mi ----- Mail original ----- > De: "Ralph Goers" > ?: "core-libs-dev" > Envoy?: Dimanche 3 D?cembre 2017 19:25:08 > Objet: Android and Log4j > Log4j added support for Java 9 by: > Converting the Log4j-API jar to a multi-release jar that includes support for > StackWalker and the new Process Id support. > Adding a module-info.jar to the Log4j API jar. > > We are now getting complaints from Android users (as well as a few others) that > their tools no longer work with log4j. During development I ran into problems > with OSGi. The problems seem to mostly revolve around the fact that they can?t > deal with the classes for Java 9. I was surprised that Android is failing on > the classes in META-INF/versions/9 as I had assumed that would be an invalid > location for a class file prior to Java 9, but that seems not to be the case. > The fact that module-info.java turns into a class file also seems to be a > problem since the various tools are seeing it and having problems with it. > > We have been discussing various ways to handle this in > https://issues.apache.org/jira/browse/LOG4J2-2133 > . There seems to be a strong > push to just remove the support for Java 9 since it is breaking so many things. > > It seems impossible to have a module-info.java file in a jar that is going to be > included in Android. If this had been a json file that was interpreted by the > classloader or had a different file extension other than .class we wouldn?t be > in this mess. We also need another mechanism to bring in our code that uses > StackWalker as calling it via Reflection and emulating lambda expressions seems > like it would be painful and slow. > > Do you have any recommendations on how we can resolve this impasse? > > Ralph From Alan.Bateman at oracle.com Sun Dec 3 19:19:34 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 3 Dec 2017 19:19:34 +0000 Subject: Android and Log4j In-Reply-To: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> Message-ID: <23ad45ce-811e-20ba-0e74-31d1862fc90f@oracle.com> On 03/12/2017 18:25, Ralph Goers wrote: > Log4j added support for Java 9 by: > Converting the Log4j-API jar to a multi-release jar that includes support for StackWalker and the new Process Id support. > Adding a module-info.jar to the Log4j API jar. > > We are now getting complaints from Android users (as well as a few others) that their tools no longer work with log4j. During development I ran into problems with OSGi. The problems seem to mostly revolve around the fact that they can?t deal with the classes for Java 9. I was surprised that Android is failing on the classes in META-INF/versions/9 as I had assumed that would be an invalid location for a class file prior to Java 9, but that seems not to be the case. The fact that module-info.java turns into a class file also seems to be a problem since the various tools are seeing it and having problems with it. > > : > > Do you have any recommendations on how we can resolve this impasse? > The Android tools should ignore class files in META/**, at least until they are updated to work with multi-release JARs. Have you created a bug for them to look at this issue? -Alan From Alan.Bateman at oracle.com Sun Dec 3 19:34:08 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 3 Dec 2017 19:34:08 +0000 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: <5A21BD18.2050201@oracle.com> References: <5A20F2AB.3050608@oracle.com> <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> <5A21BD18.2050201@oracle.com> Message-ID: <34dacec0-7e2f-65db-1262-30c0c4a279c1@oracle.com> On 01/12/2017 20:35, Joe Wang wrote: > : > >> >> I think URLDecoder.decode methods should have @throws >> IllegalArgumentException. I realize it's implementation specific as >> to whether IAE is thrown with bad input but given that the RI does >> throw IAE then consumers of the API should be prepared for it. The >> @implNote can stay and probably should be copied into the legacy >> decode method too. > > I added @throws IAE. On a 2nd thought, would that give no flexibility > to alternative impls as the general (class) spec had given? With this > addition, it becomes a requirement. If the @throws description starts with "if the implementation ..." then it should be clear that it is optional. -Alan From claes.redestad at oracle.com Sun Dec 3 19:53:01 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 3 Dec 2017 20:53:01 +0100 Subject: RFR: 8176188: jdk/internal/misc/JavaLangAccess/NewUnsafeString.java failing since 9-b93 Message-ID: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> Hi, the compact strings JEP changed semantics of the package-private String(char[], boolean) constructor to do the same as the public String(char[]) constructor. Previously the former was used in trusted, internal code to avoid copying the given char[], but since the char[] now has to be converted to a byte[] that optimization is no longer possible via this method[1], and tests that checked that the returned string shared the given char[] naturally stopped working. To fix this bug I propose the following clean-up: - change all uses of JavaLangAccess.newUnsafeString(char[]) to new String(char[]) - remove the package-private String(char[], boolean) constructor - remove the newUnsafeString from JavaLangAccess - remove the now unnecessary NewUnsafeString test Patch: http://cr.openjdk.java.net/~redestad/8176188/open.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8176188 Thanks! /Claes [1] For some of the usages here we could improve somewhat by exposingthe String(byte[], byte) constructor, but I think that's out of scope here and I think we'd best avoid leaking the coder byte implementation detail outside of java.lang. From ralph.goers at dslextreme.com Sun Dec 3 20:08:56 2017 From: ralph.goers at dslextreme.com (Ralph Goers) Date: Sun, 3 Dec 2017 13:08:56 -0700 Subject: Android and Log4j In-Reply-To: <23ad45ce-811e-20ba-0e74-31d1862fc90f@oracle.com> References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <23ad45ce-811e-20ba-0e74-31d1862fc90f@oracle.com> Message-ID: > On Dec 3, 2017, at 12:19 PM, Alan Bateman wrote: > > On 03/12/2017 18:25, Ralph Goers wrote: >> Log4j added support for Java 9 by: >> Converting the Log4j-API jar to a multi-release jar that includes support for StackWalker and the new Process Id support. >> Adding a module-info.jar to the Log4j API jar. >> >> We are now getting complaints from Android users (as well as a few others) that their tools no longer work with log4j. During development I ran into problems with OSGi. The problems seem to mostly revolve around the fact that they can?t deal with the classes for Java 9. I was surprised that Android is failing on the classes in META-INF/versions/9 as I had assumed that would be an invalid location for a class file prior to Java 9, but that seems not to be the case. The fact that module-info.java turns into a class file also seems to be a problem since the various tools are seeing it and having problems with it. >> >> : >> >> Do you have any recommendations on how we can resolve this impasse? >> > The Android tools should ignore class files in META/**, at least until they are updated to work with multi-release JARs. Have you created a bug for them to look at this issue? Yes, although I don?t develop for Android so I am not certain I created the issue correctly. https://issuetracker.google.com/u/1/issues/70118537 . I also created https://issues.apache.org/jira/browse/FELIX-5592 which relates to https://issues.apache.org/jira/browse/FELIX-5527 . No activity has occurred on either of these issues. Ralph From abdul.kolarkunnu at oracle.com Mon Dec 4 06:30:46 2017 From: abdul.kolarkunnu at oracle.com (Muneer Kolarkunnu) Date: Sun, 3 Dec 2017 22:30:46 -0800 (PST) Subject: RFR 8192958 [JDK10] : TEST.groups: group jdk_util_other: file not found: jdk/internal/util Message-ID: Hi, Please review the fix for JDK- 8192958. Bug: https://bugs.openjdk.java.net/browse/JDK-8192958 Webrev: http://cr.openjdk.java.net/~akolarkunnu/8192958/webrev.00/ Issue: There was only one test case under the folder "jdk/internal/util" which was jdk/internal/util/jar/VersionedStream.java. This test case has removed by task JDK-8192879. Fix: Remove entry "jdk/internal/util" from test group jdk_util_other Regards, Muneer From nishit.jain at oracle.com Mon Dec 4 06:36:11 2017 From: nishit.jain at oracle.com (Nishit Jain) Date: Mon, 4 Dec 2017 12:06:11 +0530 Subject: [10] RFR 8187551: MessageFormat.setFormat(int, Format) AIOOBE not thrown when documented In-Reply-To: References: <53f445ff-a4b3-5029-77f0-45125514a662@oracle.com> Message-ID: Thanks Roger, Updated the webrevto add the new test case in MessageRegression.java http://cr.openjdk.java.net/~nishjain/8187551/webrev.03/ Regards, Nishit Jain On 01-12-2017 20:40, Roger Riggs wrote: > Hi Nishit, > > Please add the new test to > test/jdk/java/text/Format/MessageFormat/MessageRegression.java instead > of creating a new test. > > Also the convention for test names should be a functional description > of the test; bug numbers are uninformative. > > (I'm a bit surprised there are few/no regression tests for setFormat). > > Thanks, Roger > > > On 12/1/2017 3:17 AM, Nishit Jain wrote: >> Hi, >> >> Please review the fix for JDK-8187551. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8187551 >> Webrev: http://cr.openjdk.java.net/~nishjain/8187551/webrev.02/ >> >> Fix: As documented, updated MessageFormat.setFormat() to throw AIOOBE >> with invalid format element index. >> >> Thanks Martin Buchholz for filing the issue and suggesting the fix. >> >> Regards, >> Nishit Jain > From david.holmes at oracle.com Mon Dec 4 06:38:59 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 Dec 2017 16:38:59 +1000 Subject: RFR 8192958 [JDK10] : TEST.groups: group jdk_util_other: file not found: jdk/internal/util In-Reply-To: References: Message-ID: Looks good! Thanks, David On 4/12/2017 4:30 PM, Muneer Kolarkunnu wrote: > Hi, > > > > Please review the fix for JDK- 8192958. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8192958 > > Webrev: http://cr.openjdk.java.net/~akolarkunnu/8192958/webrev.00/ > > > > Issue: There was only one test case under the folder "jdk/internal/util" which was jdk/internal/util/jar/VersionedStream.java. This test case has removed by task JDK-8192879. > > Fix: Remove entry "jdk/internal/util" from test group jdk_util_other > > > > Regards, > > Muneer > From christoph.langer at sap.com Mon Dec 4 08:47:51 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 4 Dec 2017 08:47:51 +0000 Subject: RFR (XXS): 8192961: Remove some double semicolons Message-ID: Hi, can I please have a review for this small cleanup. I spotted some double semicolons at a few import statements. Bug: https://bugs.openjdk.java.net/browse/JDK-8192961 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8192961.0/ Thanks Christoph From claes.redestad at oracle.com Mon Dec 4 08:54:04 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 4 Dec 2017 09:54:04 +0100 Subject: RFR (XXS): 8192961: Remove some double semicolons In-Reply-To: References: Message-ID: <87f120e5-b9c8-6939-f1bd-e6c18265cf00@oracle.com> On 2017-12-04 09:47, Langer, Christoph wrote: > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8192961.0/ Looks good. /Claes From peter.levart at gmail.com Mon Dec 4 13:25:37 2017 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 4 Dec 2017 14:25:37 +0100 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> Message-ID: Hi Rogger, On 12/04/2017 02:17 PM, Peter Levart wrote: > Hi Rogger, > > Interesting approach. Conditional finalization. You use finalization > to support cases where user overrides finalize() and/or close() and > Cleaner when he doesn't. > > I wonder if it is the right thing to use AltFinalizer when user > overrides finalize() method. In that case the method is probably not > empty and calls super.finalize() (if it is empty or doesn't call > super, user probably doesn't want the finalization to close the > stream) and so normal finalization applies. If you register > AltFinalizer for such case, close() will be called twice. Ah, scrap that. I forgot that XXXStream.finalize() is now empty, so user overriding it and calling super does not in fact close the stream. You have to register AltFinalizer in that case. But now I wonder if the logic should still be 3-state and do the following: - if user overrides finalize() - use AltFinalizer to call both: first finalize() and then close(); else - if user overrides close() - use AltFinalizer to call close(); else - use Cleaner What do you think? Regards, Peter From peter.levart at gmail.com Mon Dec 4 13:29:38 2017 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 4 Dec 2017 14:29:38 +0100 Subject: Fwd: Re: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: References: Message-ID: Sending previous message to the list also (forgot to mention it in CC)... Hi Rogger, Interesting approach. Conditional finalization. You use finalization to support cases where user overrides finalize() and/or close() and Cleaner when he doesn't. I wonder if it is the right thing to use AltFinalizer when user overrides finalize() method. In that case the method is probably not empty and calls super.finalize() (if it is empty or doesn't call super, user probably doesn't want the finalization to close the stream) and so normal finalization applies. If you register AltFinalizer for such case, close() will be called twice. The logic should probably be 3-state: - if user overrides finalize() - do nothing; else - if user overrides close() - use AltFinalizer; else - use Cleaner Some additional comments: - FileInputStream.AltFinalizer#get could be written to not use exceptions for flow control. For example: ??? static AltFinalizer get(FileInputStream fis) { ??????? return Stream.>iterate(fis.getClass(), ??????????????????????????????????????? cl -> cl != FileInputStream.class, ??????????????????????????????????????? Class::getSuperclass) ??????????? .flatMap(clazz -> Stream.of(clazz.getDeclaredMethods())) ??????????? .filter(m -> m.getParameterCount() == 0 && ???????????????????????? (m.getName().equals("close") || ????????????????????????? m.getName().equals("finalize"))) ??????????? .findFirst() ??????????? .map(m -> new AltFinalizer(fis)).orElse(null); ??? } or, to support 3-state logic: ??? static Optional get(FileInputStream fis) { ??????? int flag = Stream. ??????????? >iterate(fis.getClass(), ????????????????????????????? cl -> cl != FileInputStream.class, ????????????????????????????? Class::getSuperclass) ??????????? .flatMap(clazz -> Stream.of(clazz.getDeclaredMethods())) ??????????? .mapToInt(m -> m.getParameterCount() == 0 ?????????????????????????? ? m.getName().equals("close") ???????????????????????????? ? 1 ???????????????????????????? : (m.getName().equals("finalize") ? 2 : 0) ?????????????????????????? : 0) ??????????? .reduce(0, (i, j) -> i | j); ??????? if ((flag & 2) != 0) { // overrides finalize() ??????????? return Optional.empty(); ??????? } else if ((flag & 1) != 0) { // overrides close() ??????????? return Optional.of(new AltFinalizer(fis)); ??????? } else { // overrides none ??????????? return null; ??????? } ??? } - similar for FileOutputStream.AltFinalizer That's all for now. Will try to check the rest later. Regards, Peter On 12/02/2017 03:38 AM, Roger Riggs wrote: > Please review a revision to the work on remove a dependency on > finalization from > FileInputStream, FileOutputStream, and add cleanup of closing file > descriptors in FileChannel. > > The previous version was too aggressive in removing the finalize > method and was considered to be insufficiently > backward compatible. > > For compatibility with previous versions, the close method will be > called when the FIS/FOS is unreachable > only when FIS/FOS has been subclassed to override close() or > finalize().? In other cases, the cleaner is used > to make sure the FileDescriptor is closed. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8080225 > > CSR: > Relax FileInputStream/FileOutputStream requirement to use finalize > https://bugs.openjdk.java.net/browse/JDK-8187325 > > Thanks, Roger > > > > From peter.levart at gmail.com Mon Dec 4 13:47:35 2017 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 4 Dec 2017 14:47:35 +0100 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> Message-ID: <0d7dd754-fedd-b497-2dbc-572473a136fb@gmail.com> This is a nice exercise in compatible migration from finalize() to Cleaner. The compatibility would be even better with a little tweak to the VM. Currently VM ignores empty finalize() method(s) and does not register the instance for finalization if it has empty finalize() (just return bytecode). There could be a special method-level runtime annotation, say @IgnoreFinalize that would be used to annotate no-empty finalize() methods and would have the same effect as empty method on the VM behavior. XXXStream could then just annotate the finalize() and leave it as is so that potential overriders of finalize() could call super. Current approach (with AltFinalizer) is an approximation since it can only call finalize() and close() in succession. If overridden finalize() callse super.finalize() in the middle of the method, it may expect the stream to be already closed for the rest of the method: @Override protected finalize() { ??? ... ??? ... pre-close logic ??? ... ??? super.finalize(); ??? ... ??? ... post-close logic (may expect stream to be already closed) ??? ... } ...bust since super.close() is empty, the stream is not closed yet and will be closed by AltFinalizer after this.finalize() returns. I don't know what impact does such order have on the compatibility. Probably not big. Regards, Peter On 12/04/2017 02:25 PM, Peter Levart wrote: > Hi Rogger, > > On 12/04/2017 02:17 PM, Peter Levart wrote: >> Hi Rogger, >> >> Interesting approach. Conditional finalization. You use finalization >> to support cases where user overrides finalize() and/or close() and >> Cleaner when he doesn't. >> >> I wonder if it is the right thing to use AltFinalizer when user >> overrides finalize() method. In that case the method is probably not >> empty and calls super.finalize() (if it is empty or doesn't call >> super, user probably doesn't want the finalization to close the >> stream) and so normal finalization applies. If you register >> AltFinalizer for such case, close() will be called twice. > > Ah, scrap that. I forgot that XXXStream.finalize() is now empty, so > user overriding it and calling super does not in fact close the > stream. You have to register AltFinalizer in that case. But now I > wonder if the logic should still be 3-state and do the following: > > - if user overrides finalize() - use AltFinalizer to call both: first > finalize() and then close(); else > - if user overrides close() - use AltFinalizer to call close(); else > - use Cleaner > > What do you think? > > Regards, Peter > From peter.levart at gmail.com Mon Dec 4 14:09:52 2017 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 4 Dec 2017 15:09:52 +0100 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> Message-ID: <6e172668-f13a-faf4-48bb-cdebd3a802d8@gmail.com> On 12/04/2017 02:25 PM, Peter Levart wrote: > Hi Rogger, > > On 12/04/2017 02:17 PM, Peter Levart wrote: >> Hi Rogger, >> >> Interesting approach. Conditional finalization. You use finalization >> to support cases where user overrides finalize() and/or close() and >> Cleaner when he doesn't. >> >> I wonder if it is the right thing to use AltFinalizer when user >> overrides finalize() method. In that case the method is probably not >> empty and calls super.finalize() (if it is empty or doesn't call >> super, user probably doesn't want the finalization to close the >> stream) and so normal finalization applies. If you register >> AltFinalizer for such case, close() will be called twice. > > Ah, scrap that. I forgot that XXXStream.finalize() is now empty, so > user overriding it and calling super does not in fact close the > stream. You have to register AltFinalizer in that case. But now I > wonder if the logic should still be 3-state and do the following: > > - if user overrides finalize() - use AltFinalizer to call both: first > finalize() and then close(); else > - if user overrides close() - use AltFinalizer to call close(); else > - use Cleaner > > What do you think? > > Regards, Peter > I just realized that in the above case when finalize is overridden, it would be called twice. once by finalization and once by AltFinalizer. So your logic is as correct as it can be for that case (to just call close() with AltFinalizer). The only problem is order which is arbitrary, so it may happen that AltFinalizer calls close() 1st and then finalization calls overridden finalize() method which might expect the stream to still be open until it calls super.finalize(). Regards, Peter From peter.levart at gmail.com Mon Dec 4 14:25:28 2017 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 4 Dec 2017 15:25:28 +0100 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: <6e172668-f13a-faf4-48bb-cdebd3a802d8@gmail.com> References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> <6e172668-f13a-faf4-48bb-cdebd3a802d8@gmail.com> Message-ID: <4b1a6556-d998-3945-2ea7-fa785a889680@gmail.com> Hi Roger, On 12/04/2017 03:09 PM, Peter Levart wrote: > > > On 12/04/2017 02:25 PM, Peter Levart wrote: >> Hi Rogger, >> >> On 12/04/2017 02:17 PM, Peter Levart wrote: >>> Hi Rogger, >>> >>> Interesting approach. Conditional finalization. You use finalization >>> to support cases where user overrides finalize() and/or close() and >>> Cleaner when he doesn't. >>> >>> I wonder if it is the right thing to use AltFinalizer when user >>> overrides finalize() method. In that case the method is probably not >>> empty and calls super.finalize() (if it is empty or doesn't call >>> super, user probably doesn't want the finalization to close the >>> stream) and so normal finalization applies. If you register >>> AltFinalizer for such case, close() will be called twice. >> >> Ah, scrap that. I forgot that XXXStream.finalize() is now empty, so >> user overriding it and calling super does not in fact close the >> stream. You have to register AltFinalizer in that case. But now I >> wonder if the logic should still be 3-state and do the following: >> >> - if user overrides finalize() - use AltFinalizer to call both: first >> finalize() and then close(); else >> - if user overrides close() - use AltFinalizer to call close(); else >> - use Cleaner >> >> What do you think? >> >> Regards, Peter >> > > I just realized that in the above case when finalize is overridden, it > would be called twice. once by finalization and once by AltFinalizer. > So your logic is as correct as it can be for that case (to just call > close() with AltFinalizer). The only problem is order which is > arbitrary, so it may happen that AltFinalizer calls close() 1st and > then finalization calls overridden finalize() method which might > expect the stream to still be open until it calls super.finalize(). > > Regards, Peter > Final refinement... (hopefully!) If user overrides just finalize() and does not override close(), then it might be best to employ Cleaner to close the stream. Cleaner is Phantom based and will get fired after finalization invokes overridden finalize(), enabling the finalize() method to still access the stream in that case. So this is the final logic (I think): - if user overrides close(), use AltFinalizer to call close(); else - use Cleaner to close the stream The above logic in action: finalize()?? ??? ??? ?? close()??? ??? ??? ??? ??? ??? action not overridden??? not overridden?????????? Cleaner closes stream not overridden??? overridden???????????????? AltFinalizer calls close() overridden??? ??? ? not overridden?????????? finalization calls finalize() then Cleaner closes stream overridden??? ??? ? overridden???????????????? finalization calls finalize() and AltFinalizer calls close() (in arbitrary order) Regards, Peter From Roger.Riggs at Oracle.com Mon Dec 4 14:57:33 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Dec 2017 09:57:33 -0500 Subject: [10] RFR 8187551: MessageFormat.setFormat(int, Format) AIOOBE not thrown when documented In-Reply-To: References: <53f445ff-a4b3-5029-77f0-45125514a662@oracle.com> Message-ID: <8b4c1d9b-95e2-a8ac-1879-e3352a668107@Oracle.com> Hi Nishit, Looks good, Thanks, Roger On 12/4/2017 1:36 AM, Nishit Jain wrote: > Thanks Roger, > > Updated the webrevto add the new test case in MessageRegression.java > > http://cr.openjdk.java.net/~nishjain/8187551/webrev.03/ > > Regards, > Nishit Jain > On 01-12-2017 20:40, Roger Riggs wrote: >> Hi Nishit, >> >> Please add the new test to >> test/jdk/java/text/Format/MessageFormat/MessageRegression.java instead >> of creating a new test. >> >> Also the convention for test names should be a functional description >> of the test; bug numbers are uninformative. >> >> (I'm a bit surprised there are few/no regression tests for setFormat). >> >> Thanks, Roger >> >> >> On 12/1/2017 3:17 AM, Nishit Jain wrote: >>> Hi, >>> >>> Please review the fix for JDK-8187551. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8187551 >>> Webrev: http://cr.openjdk.java.net/~nishjain/8187551/webrev.02/ >>> >>> Fix: As documented, updated MessageFormat.setFormat() to throw >>> AIOOBE with invalid format element index. >>> >>> Thanks Martin Buchholz for filing the issue and suggesting the fix. >>> >>> Regards, >>> Nishit Jain >> > From sean.coffey at oracle.com Mon Dec 4 16:03:48 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 4 Dec 2017 16:03:48 +0000 Subject: RFR JDK-8191918: tomcat gzip-compressed response bodies appear to be broken in update 151 In-Reply-To: <5A21FC06.5060408@oracle.com> References: <5A208A37.3030109@oracle.com> <40D2F6A6-95C9-40C1-8F9B-E2C416BFCD58@oracle.com> <5A21FC06.5060408@oracle.com> Message-ID: <0ce6113e-46b3-b835-120e-d4e09c586d07@oracle.com> Looks good to me. Regards, Sean. On 02/12/17 01:04, Xueming Shen wrote: > On 12/1/17, 3:40 PM, Paul Sandoz wrote: >> >>> On 30 Nov 2017, at 14:46, Xueming Shen wrote: >>> >>> Hi, >>> >>> Please help review the change for JDK-8191918: >>> >>> issue: https://bugs.openjdk.java.net/browse/JDK-8191918 >>> webrev: http://cr.openjdk.java.net/~sherman/8191918/webrev >>> >> InflateIn_DeflateOut.java >> ? >> >> 174 GZIPInputStream gzis = new GZIPInputStream(byteInStream); >> 175 ByteArrayOutputStream baos = new ByteArrayOutputStream(); >> 176 int numRead; >> 177 byte[] b = new byte[4 * 1024]; >> 178 try { >> 179 while ((numRead = gzis.read(buf))>= 0) { >> 180 baos.write(b, 0, numRead); >> 181 } >> 182 } finally { >> 183 baos.close(); >> 184 } >> >> Can you use gzis.transferTo(baos)? >> >> Paul. >> >> > > Thanks Paul! > > webrev has been updated as suggested. > > http://cr.openjdk.java.net/~sherman/8191918/webrev > > -Sherman From Roger.Riggs at Oracle.com Mon Dec 4 16:54:53 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Dec 2017 11:54:53 -0500 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: <4b1a6556-d998-3945-2ea7-fa785a889680@gmail.com> References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> <6e172668-f13a-faf4-48bb-cdebd3a802d8@gmail.com> <4b1a6556-d998-3945-2ea7-fa785a889680@gmail.com> Message-ID: Hi Peter, Thanks for reviewing. This is a transition step to removing the finalize method completely while giving subclasses notice to upgrade their cleanup activities and yet gain the performance benefits sooner. Later, finalize() and related compatibility mechanisms will be removed. Simpler code, even if sometimes less than optimal is preferred to maintain compatibility in the interim. ... On 12/4/2017 9:25 AM, Peter Levart wrote: > Hi Roger, > > On 12/04/2017 03:09 PM, Peter Levart wrote: >> >> >> On 12/04/2017 02:25 PM, Peter Levart wrote: >>> Hi Rogger, >>> >>> On 12/04/2017 02:17 PM, Peter Levart wrote: >>>> Hi Rogger, >>>> >>>> Interesting approach. Conditional finalization. You use >>>> finalization to support cases where user overrides finalize() >>>> and/or close() and Cleaner when he doesn't. >>>> >>>> I wonder if it is the right thing to use AltFinalizer when user >>>> overrides finalize() method. In that case the method is probably >>>> not empty and calls super.finalize() (if it is empty or doesn't >>>> call super, user probably doesn't want the finalization to close >>>> the stream) and so normal finalization applies. If you register >>>> AltFinalizer for such case, close() will be called twice. >>> >>> Ah, scrap that. I forgot that XXXStream.finalize() is now empty, so >>> user overriding it and calling super does not in fact close the >>> stream. You have to register AltFinalizer in that case. But now I >>> wonder if the logic should still be 3-state and do the following: >>> >>> - if user overrides finalize() - use AltFinalizer to call both: >>> first finalize() and then close(); else >>> - if user overrides close() - use AltFinalizer to call close(); else >>> - use Cleaner >>> >>> What do you think? >>> >>> Regards, Peter >>> >> >> I just realized that in the above case when finalize is overridden, >> it would be called twice. once by finalization and once by >> AltFinalizer. So your logic is as correct as it can be for that case >> (to just call close() with AltFinalizer). The only problem is order >> which is arbitrary, so it may happen that AltFinalizer calls close() >> 1st and then finalization calls overridden finalize() method which >> might expect the stream to still be open until it calls >> super.finalize(). >> >> Regards, Peter >> > > Final refinement... (hopefully!) > > If user overrides just finalize() and does not override close(), then > it might be best to employ Cleaner to close the stream. Cleaner is > Phantom based and will get fired after finalization invokes overridden > finalize(), enabling the finalize() method to still access the stream > in that case. So this is the final logic (I think): > > - if user overrides close(), use AltFinalizer to call close(); else > - use Cleaner to close the stream > > The above logic in action: > > finalize()?? ??? ??? ?? close()??? ??? ??? ??? ??? ??? action > > not overridden??? not overridden?????????? Cleaner closes stream > not overridden??? overridden???????????????? AltFinalizer calls close() > overridden??? ??? ? not overridden?????????? finalization calls > finalize() then Cleaner closes stream > overridden??? ??? ? overridden???????????????? finalization calls > finalize() and AltFinalizer calls close() (in arbitrary order) Right, if close() is not overridden the behavior of close() is known and the Cleaner can be used. The contents of an overridden finalize() method are unknowable, it may or may not call close() itself and may or may not call super.finalize().? If? close() is called, then the Cleaner will be removed; otherwise it will release the resources. The close method should be idempotent, calling it twice should not be a problem. This does simplify the logic. Thanks, Roger > > > Regards, Peter > From Roger.Riggs at Oracle.com Mon Dec 4 16:58:21 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Dec 2017 11:58:21 -0500 Subject: Fwd: Re: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: References: Message-ID: <9045b291-fb06-b331-900f-d6b221fe9a3b@Oracle.com> Hi Peter, I'd like to keep the code recognizably simple; (in the absence of a known performance issue). It would be interesting to know how the streams versions compares to the exception versions. Something for a rainy day... Roger On 12/4/2017 8:29 AM, Peter Levart wrote: > Sending previous message to the list also (forgot to mention it in CC)... > > Hi Rogger, > > Interesting approach. Conditional finalization. You use finalization to > support cases where user overrides finalize() and/or close() and Cleaner > when he doesn't. > > I wonder if it is the right thing to use AltFinalizer when user > overrides finalize() method. In that case the method is probably not > empty and calls super.finalize() (if it is empty or doesn't call super, > user probably doesn't want the finalization to close the stream) and so > normal finalization applies. If you register AltFinalizer for such case, > close() will be called twice. > > The logic should probably be 3-state: > > - if user overrides finalize() - do nothing; else > - if user overrides close() - use AltFinalizer; else > - use Cleaner > > Some additional comments: > > - FileInputStream.AltFinalizer#get could be written to not use > exceptions for flow control. For example: > > ??? static AltFinalizer get(FileInputStream fis) { > ??????? return Stream.>iterate(fis.getClass(), > ??????????????????????????????????????? cl -> cl != > FileInputStream.class, > ??????????????????????????????????????? Class::getSuperclass) > ??????????? .flatMap(clazz -> Stream.of(clazz.getDeclaredMethods())) > ??????????? .filter(m -> m.getParameterCount() == 0 && > ???????????????????????? (m.getName().equals("close") || > ????????????????????????? m.getName().equals("finalize"))) > ??????????? .findFirst() > ??????????? .map(m -> new AltFinalizer(fis)).orElse(null); > ??? } > > or, to support 3-state logic: > > ??? static Optional get(FileInputStream fis) { > ??????? int flag = Stream. > ??????????? >iterate(fis.getClass(), > ????????????????????????????? cl -> cl != FileInputStream.class, > ????????????????????????????? Class::getSuperclass) > ??????????? .flatMap(clazz -> Stream.of(clazz.getDeclaredMethods())) > ??????????? .mapToInt(m -> m.getParameterCount() == 0 > ?????????????????????????? ? m.getName().equals("close") > ???????????????????????????? ? 1 > ???????????????????????????? : (m.getName().equals("finalize") ? 2 : 0) > ?????????????????????????? : 0) > ??????????? .reduce(0, (i, j) -> i | j); > > ??????? if ((flag & 2) != 0) { // overrides finalize() > ??????????? return Optional.empty(); > ??????? } else if ((flag & 1) != 0) { // overrides close() > ??????????? return Optional.of(new AltFinalizer(fis)); > ??????? } else { // overrides none > ??????????? return null; > ??????? } > ??? } > > - similar for FileOutputStream.AltFinalizer > > > That's all for now. Will try to check the rest later. > > Regards, Peter > > On 12/02/2017 03:38 AM, Roger Riggs wrote: >> Please review a revision to the work on remove a dependency on >> finalization from >> FileInputStream, FileOutputStream, and add cleanup of closing file >> descriptors in FileChannel. >> >> The previous version was too aggressive in removing the finalize >> method and was considered to be insufficiently >> backward compatible. >> >> For compatibility with previous versions, the close method will be >> called when the FIS/FOS is unreachable >> only when FIS/FOS has been subclassed to override close() or >> finalize().? In other cases, the cleaner is used >> to make sure the FileDescriptor is closed. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8080225 >> >> CSR: >> Relax FileInputStream/FileOutputStream requirement to use finalize >> https://bugs.openjdk.java.net/browse/JDK-8187325 >> >> Thanks, Roger >> >> >> >> > From naoto.sato at oracle.com Mon Dec 4 17:06:36 2017 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 4 Dec 2017 09:06:36 -0800 Subject: [10] RFR 8187551: MessageFormat.setFormat(int, Format) AIOOBE not thrown when documented In-Reply-To: References: <53f445ff-a4b3-5029-77f0-45125514a662@oracle.com> Message-ID: <7dde3e8f-cfe2-5776-16e7-d4da9d676620@oracle.com> +1 Naoto On 12/3/17 10:36 PM, Nishit Jain wrote: > Thanks Roger, > > Updated the webrevto add the new test case in MessageRegression.java > > http://cr.openjdk.java.net/~nishjain/8187551/webrev.03/ > > Regards, > Nishit Jain > On 01-12-2017 20:40, Roger Riggs wrote: >> Hi Nishit, >> >> Please add the new test to >> test/jdk/java/text/Format/MessageFormat/MessageRegression.java instead >> of creating a new test. >> >> Also the convention for test names should be a functional description >> of the test; bug numbers are uninformative. >> >> (I'm a bit surprised there are few/no regression tests for setFormat). >> >> Thanks, Roger >> >> >> On 12/1/2017 3:17 AM, Nishit Jain wrote: >>> Hi, >>> >>> Please review the fix for JDK-8187551. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8187551 >>> Webrev: http://cr.openjdk.java.net/~nishjain/8187551/webrev.02/ >>> >>> Fix: As documented, updated MessageFormat.setFormat() to throw AIOOBE >>> with invalid format element index. >>> >>> Thanks Martin Buchholz for filing the issue and suggesting the fix. >>> >>> Regards, >>> Nishit Jain >> > From mandy.chung at oracle.com Mon Dec 4 17:20:51 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 4 Dec 2017 09:20:51 -0800 Subject: RFR 8192958 [JDK10] : TEST.groups: group jdk_util_other: file not found: jdk/internal/util In-Reply-To: References: Message-ID: <9216973d-49e0-7dc8-063e-89523ecb6eb5@oracle.com> Looks good.? Thanks for cleaning it up. The test case was actually removed by JDK-8189611 whereas the class was removed by JDK-8192879. Mandy On 12/3/17 10:30 PM, Muneer Kolarkunnu wrote: > > Hi, > > Please review the fix for JDK- 8192958. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8192958 > > Webrev: http://cr.openjdk.java.net/~akolarkunnu/8192958/webrev.00/ > > > Issue: There was only one test case under the folder > "jdk/internal/util" which was > jdk/internal/util/jar/VersionedStream.java. This test case has removed > by task JDK-8192879. > > Fix: Remove entry "jdk/internal/util" from test group jdk_util_other > > Regards, > > Muneer > From uwe at thetaphi.de Sun Dec 3 19:49:46 2017 From: uwe at thetaphi.de (Uwe Schindler) Date: Sun, 03 Dec 2017 19:49:46 +0000 Subject: Android and Log4j In-Reply-To: <23ad45ce-811e-20ba-0e74-31d1862fc90f@oracle.com> References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <23ad45ce-811e-20ba-0e74-31d1862fc90f@oracle.com> Message-ID: <4243612D-1C34-4862-B95B-09165AC2C32D@thetaphi.de> Hi, I'd also suggest to place the module-info.class inside the versions folder. So it should be ignored once the meta-inf bug is fixed. Java 9 should also read the module info from the versioned folder. Uwe Am 3. Dezember 2017 20:19:34 MEZ schrieb Alan Bateman : >On 03/12/2017 18:25, Ralph Goers wrote: >> Log4j added support for Java 9 by: >> Converting the Log4j-API jar to a multi-release jar that includes >support for StackWalker and the new Process Id support. >> Adding a module-info.jar to the Log4j API jar. >> >> We are now getting complaints from Android users (as well as a few >others) that their tools no longer work with log4j. During development >I ran into problems with OSGi. The problems seem to mostly revolve >around the fact that they can?t deal with the classes for Java 9. I was >surprised that Android is failing on the classes in META-INF/versions/9 >as I had assumed that would be an invalid location for a class file >prior to Java 9, but that seems not to be the case. The fact that >module-info.java turns into a class file also seems to be a problem >since the various tools are seeing it and having problems with it. >> >> : >> >> Do you have any recommendations on how we can resolve this impasse? >> >The Android tools should ignore class files in META/**, at least until >they are updated to work with multi-release JARs. Have you created a >bug >for them to look at this issue? > >-Alan -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de From martinrb at google.com Mon Dec 4 19:45:40 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 11:45:40 -0800 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> Message-ID: Interesting. You are trying to define a new Best Practice for the Serialization Proxy Pattern. Serialization is weird/broken in many ways - one of its weirdnesses is documenting behavior indirectly by publishing javadoc for private (!) methods (and fixing that would be a huge project (and I'm definitely not signing up to fix serialization!)). So the writeReplace and readResolve methods together describe how to serialize and deserialize these classes. But the methods are in different classes! Further, there's the serial spec for readObject which ensures that bogus serial forms are rejected, which is another reason to publish a serial form spec for EnumSet. There's also a reasonable expectation that public Serializable classes have an entry in the serialized form page. So I'd like to go with what I've got. http://cr.openjdk.java.net/~martin/webrevs/jdk/EnumSet-SerializationProxy/ (I had to modify the BogusEnumSet test, probably due to my added "transient") On Fri, Dec 1, 2017 at 5:20 PM, Stuart Marks wrote: > On 12/1/17 4:42 PM, Martin Buchholz wrote: > >> 1. JDK-8192935 >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk10/EnumSet >> -SerializationProxy/EnumSet-SerializationProxy.patch >> > > --- a/src/java.base/share/classes/java/util/EnumSet.java > +++ b/src/java.base/share/classes/java/util/EnumSet.java > @@ -75,7 +75,6 @@ > * @author Josh Bloch > * @since 1.5 > * @see EnumMap > - * @serial exclude > */ > > I suspect you're following other examples in the JDK that include the > serial form documentation for a class that uses a serialization proxy, but > I think this is a mistake. It's a mistake because this will cause EnumSet > to appear in serialized-form.html, but EnumSet actually should *never* > appear in any serialized byte stream. This is quite confusing. I think > those other places should be fixed, instead. > > Instead of including EnumSet itself in the serialized-form.html, how about > restoring its "@serial exclude" and then putting a link directly from the > EnumSet class doc to the EnumSet.SerializationProxy serial form doc? > Something like > > *

When an instance of this class is serialized, it is replaced with > the > * serial form of an instance of > * > > * {@code EnumSet.SerializationProxy}. > > The changes to EnumSet.SerializationProxy class are good. > > s'marks > From martinrb at google.com Mon Dec 4 20:03:23 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 12:03:23 -0800 Subject: Android and Log4j In-Reply-To: <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> Message-ID: On Sun, Dec 3, 2017 at 11:02 AM, Remi Forax wrote: > Ask Google to fix dx, > dx should ignore the module-info.class and everything inside > META-INF/versions (at least it's a first simple patch). > Hi, I'm "Google", sort of. Friends tell me that dx is getting fixed. From brent.christian at oracle.com Mon Dec 4 20:11:04 2017 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 4 Dec 2017 12:11:04 -0800 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> <08ea805e-4da5-3bd9-904c-9a56e7899d03@oracle.com> <79EEA360-45C5-47AA-B48B-D9418F1A56B3@oracle.com> Message-ID: <7982a98f-c56e-b8b7-0f1f-abf75fb4e056@oracle.com> On 12/1/17 5:22 PM, Mandy Chung wrote: > > I think should also rethrow the cause if the cause is an Error > (why not) Alright. > The instanceof RuntimeException check should be moved outside to the > if-statement when it?s an instance of InvocationTargetException. So rethrow RuntimeExceptions directly, whether they're the cause of an InvocationTargetException or not. Updated webrev: http://cr.openjdk.java.net/~bchristi/8187222/webrev.02/ Thanks, -Brent From Roger.Riggs at Oracle.com Mon Dec 4 20:12:32 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Dec 2017 15:12:32 -0500 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> Message-ID: <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> Hi Martin, The java.time APIs refined the pattern used for Serialization proxies to document the relationship between the original class and its serialization proxies methods. It is important the EnumSet appear in the serialized form to document the behavior that it should no appear in the stream and to provide links to the expected serial proxy type and its behaviors.? The SerializationProxy (though a private class) is part of the public API for serialization. EnumSet should have a serialVersion uid anyway; to avoid the kind of touchup needed in the test. (And you would not need to suppress the warning). There is no harm in it having a fixed suid. 452: SerializationProxy.readResolve doesn't need the comment about elementType.cast since EnumSet.add method checks each element that is added. Regards, Roger On 12/4/2017 2:45 PM, Martin Buchholz wrote: > Interesting. You are trying to define a new Best Practice for the > Serialization Proxy Pattern. > > Serialization is weird/broken in many ways - one of its weirdnesses is > documenting behavior indirectly by publishing javadoc for private (!) > methods (and fixing that would be a huge project (and I'm definitely not > signing up to fix serialization!)). So the writeReplace and readResolve > methods together describe how to serialize and deserialize these classes. > But the methods are in different classes! Further, there's the serial spec > for readObject which ensures that bogus serial forms are rejected, which is > another reason to publish a serial form spec for EnumSet. There's also a > reasonable expectation that public Serializable classes have an entry in > the serialized form page. So I'd like to go with what I've got. > > http://cr.openjdk.java.net/~martin/webrevs/jdk/EnumSet-SerializationProxy/ > > (I had to modify the BogusEnumSet test, probably due to my added > "transient") > > > On Fri, Dec 1, 2017 at 5:20 PM, Stuart Marks > wrote: > >> On 12/1/17 4:42 PM, Martin Buchholz wrote: >> >>> 1. JDK-8192935 >>> >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk10/EnumSet >>> -SerializationProxy/EnumSet-SerializationProxy.patch >>> >> --- a/src/java.base/share/classes/java/util/EnumSet.java >> +++ b/src/java.base/share/classes/java/util/EnumSet.java >> @@ -75,7 +75,6 @@ >> * @author Josh Bloch >> * @since 1.5 >> * @see EnumMap >> - * @serial exclude >> */ >> >> I suspect you're following other examples in the JDK that include the >> serial form documentation for a class that uses a serialization proxy, but >> I think this is a mistake. It's a mistake because this will cause EnumSet >> to appear in serialized-form.html, but EnumSet actually should *never* >> appear in any serialized byte stream. This is quite confusing. I think >> those other places should be fixed, instead. >> >> Instead of including EnumSet itself in the serialized-form.html, how about >> restoring its "@serial exclude" and then putting a link directly from the >> EnumSet class doc to the EnumSet.SerializationProxy serial form doc? >> Something like >> >> *

When an instance of this class is serialized, it is replaced with >> the >> * serial form of an instance of >> * >> >> * {@code EnumSet.SerializationProxy}. >> >> The changes to EnumSet.SerializationProxy class are good. >> >> s'marks >> From Roger.Riggs at Oracle.com Mon Dec 4 20:16:38 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Dec 2017 15:16:38 -0500 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> Message-ID: Correction below. On 12/4/2017 3:12 PM, Roger Riggs wrote: > Hi Martin, > > The java.time APIs refined the pattern used for Serialization proxies > to document the relationship between > the original class and its serialization proxies methods. > > It is important the EnumSet appear in the serialized form to document > the behavior > that it should no appear in the stream and to provide links to the > expected serial proxy type > and its behaviors.? The SerializationProxy (though a private class) is > part of the public API for serialization. > > EnumSet should have a serialVersion uid anyway; to avoid the kind of > touchup needed in the test. > (And you would not need to suppress the warning). There is no harm in > it having a fixed suid. Never mind, the proxy's suid has nothing to do with the BogusEnumSet suid. > > 452: SerializationProxy.readResolve doesn't need the comment about > elementType.cast > since EnumSet.add method checks each element that is added. > > Regards, Roger > > > On 12/4/2017 2:45 PM, Martin Buchholz wrote: >> Interesting.? You are trying to define a new Best Practice for the >> Serialization Proxy Pattern. >> >> Serialization is weird/broken in many ways - one of its weirdnesses is >> documenting behavior indirectly by publishing javadoc for private (!) >> methods (and fixing that would be a huge project (and I'm definitely not >> signing up to fix serialization!)).? So the writeReplace and readResolve >> methods together describe how to serialize and deserialize these >> classes. >> But the methods are in different classes!? Further, there's the >> serial spec >> for readObject which ensures that bogus serial forms are rejected, >> which is >> another reason to publish a serial form spec for EnumSet. There's also a >> reasonable expectation that public Serializable classes have an entry in >> the serialized form page.? So I'd like to go with what I've got. >> >> http://cr.openjdk.java.net/~martin/webrevs/jdk/EnumSet-SerializationProxy/ >> >> >> (I had to modify the BogusEnumSet test, probably due to my added >> "transient") >> >> >> On Fri, Dec 1, 2017 at 5:20 PM, Stuart Marks >> wrote: >> >>> On 12/1/17 4:42 PM, Martin Buchholz wrote: >>> >>>> ???? 1. JDK-8192935 >>>> >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk10/EnumSet >>>> -SerializationProxy/EnumSet-SerializationProxy.patch >>>> >>> --- a/src/java.base/share/classes/java/util/EnumSet.java >>> +++ b/src/java.base/share/classes/java/util/EnumSet.java >>> @@ -75,7 +75,6 @@ >>> ?? * @author Josh Bloch >>> ?? * @since 1.5 >>> ?? * @see EnumMap >>> - * @serial exclude >>> ?? */ >>> >>> I suspect you're following other examples in the JDK that include the >>> serial form documentation for a class that uses a serialization >>> proxy, but >>> I think this is a mistake. It's a mistake because this will cause >>> EnumSet >>> to appear in serialized-form.html, but EnumSet actually should *never* >>> appear in any serialized byte stream. This is quite confusing. I think >>> those other places should be fixed, instead. >>> >>> Instead of including EnumSet itself in the serialized-form.html, how >>> about >>> restoring its "@serial exclude" and then putting a link directly >>> from the >>> EnumSet class doc to the EnumSet.SerializationProxy serial form doc? >>> Something like >>> >>> ? *

When an instance of this class is serialized, it is replaced >>> with >>> the >>> ? * serial form of an instance of >>> ? * >> href="../../serialized-form.html#java.util.EnumSet.SerializationProxy"> >>> >>> ? * {@code EnumSet.SerializationProxy}. >>> >>> The changes to EnumSet.SerializationProxy class are good. >>> >>> s'marks >>> > From mandy.chung at oracle.com Mon Dec 4 20:18:47 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 4 Dec 2017 12:18:47 -0800 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: <7982a98f-c56e-b8b7-0f1f-abf75fb4e056@oracle.com> References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> <08ea805e-4da5-3bd9-904c-9a56e7899d03@oracle.com> <79EEA360-45C5-47AA-B48B-D9418F1A56B3@oracle.com> <7982a98f-c56e-b8b7-0f1f-abf75fb4e056@oracle.com> Message-ID: <381991d7-544a-37ba-8bb5-92c4c0144596@oracle.com> On 12/4/17 12:11 PM, Brent Christian wrote: > So rethrow RuntimeExceptions directly, whether they're the cause of an > InvocationTargetException or not. > > Updated webrev: > http://cr.openjdk.java.net/~bchristi/8187222/webrev.02/ > Looks good. Thanks Mandy From martinrb at google.com Mon Dec 4 20:36:33 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 12:36:33 -0800 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> Message-ID: Hi Roger, On Mon, Dec 4, 2017 at 12:12 PM, Roger Riggs wrote: > Hi Martin, > > The java.time APIs refined the pattern used for Serialization proxies to > document the relationship between > the original class and its serialization proxies methods. > Right. I was aware of the effort that java.time people put into their serialization code. They did a good job, and my proposed change makes it more like theirs (but even more like the ones in j.u.c.a.) > 452: SerializationProxy.readResolve doesn't need the comment about > elementType.cast > since EnumSet.add method checks each element that is added. > You may be right, but I was only preserving the original comment, and removing it should be a separate change (I leave it to you!). From paul.sandoz at oracle.com Mon Dec 4 20:38:08 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 4 Dec 2017 12:38:08 -0800 Subject: RFR: 8176188: jdk/internal/misc/JavaLangAccess/NewUnsafeString.java failing since 9-b93 In-Reply-To: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> References: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> Message-ID: <4466D108-D234-4B7A-B2FA-0403E74DB89C@oracle.com> +1 Paul. > On 3 Dec 2017, at 11:53, Claes Redestad wrote: > > Hi, > > the compact strings JEP changed semantics of the package-private String(char[], boolean) > constructor to do the same as the public String(char[]) constructor. > > Previously the former was used in trusted, internal code to avoid copying the given char[], > but since the char[] now has to be converted to a byte[] that optimization is no longer > possible via this method[1], and tests that checked that the returned string shared the > given char[] naturally stopped working. > > To fix this bug I propose the following clean-up: > - change all uses of JavaLangAccess.newUnsafeString(char[]) to new String(char[]) > - remove the package-private String(char[], boolean) constructor > - remove the newUnsafeString from JavaLangAccess > - remove the now unnecessary NewUnsafeString test > > Patch: http://cr.openjdk.java.net/~redestad/8176188/open.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8176188 > > Thanks! > > /Claes > > [1] For some of the usages here we could improve somewhat by exposingthe String(byte[], byte) > constructor, but I think that's out of scope here and I think we'd best avoid leaking the > coder byte implementation detail outside of java.lang. From Roger.Riggs at Oracle.com Mon Dec 4 20:38:54 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Dec 2017 15:38:54 -0500 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> Message-ID: Keep the comment, I didn't notice it had only been relocated. Thanks, On 12/4/2017 3:36 PM, Martin Buchholz wrote: > Hi Roger, > > On Mon, Dec 4, 2017 at 12:12 PM, Roger Riggs > wrote: > > Hi Martin, > > The java.time APIs refined the pattern used for Serialization > proxies to document the relationship between > the original class and its serialization proxies methods. > > > Right.? I was aware of the effort that java.time people put into their > serialization code.? They did a good job, and my proposed change makes > it more like theirs (but even more like the ones in j.u.c.a.) :) > > 452: SerializationProxy.readResolve doesn't need the comment about > elementType.cast > since EnumSet.add method checks each element that is added. > > > You may be right, but I was only preserving the original comment, and > removing it should be a separate change (I leave it to you!). From martinrb at google.com Mon Dec 4 21:06:28 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 13:06:28 -0800 Subject: RFR: 8176188: jdk/internal/misc/JavaLangAccess/NewUnsafeString.java failing since 9-b93 In-Reply-To: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> References: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> Message-ID: I'm rather sad about what happened to our non-copying String constructions for trusted code. This is a performance regression with the change in String representation that should have blocked that change IMO. I think we should have a plan for moving in the opposite direction. I don't think we can implement something as ambitious as Rust's ownership tracking, so have to restrict ourselves to trusted code. The use case that keeps coming up is constructing zip entry names, which are much more likely to be pure ASCII than their file contents. I don't have a good design for how one could do that, and who the trusted set of callers is (at least java.base, not java.lang), but I think we should set a direction. On Sun, Dec 3, 2017 at 11:53 AM, Claes Redestad wrote: > Hi, > > the compact strings JEP changed semantics of the package-private > String(char[], boolean) > constructor to do the same as the public String(char[]) constructor. > > Previously the former was used in trusted, internal code to avoid copying > the given char[], > but since the char[] now has to be converted to a byte[] that optimization > is no longer > possible via this method[1], and tests that checked that the returned > string shared the > given char[] naturally stopped working. > > To fix this bug I propose the following clean-up: > - change all uses of JavaLangAccess.newUnsafeString(char[]) to new > String(char[]) > - remove the package-private String(char[], boolean) constructor > - remove the newUnsafeString from JavaLangAccess > - remove the now unnecessary NewUnsafeString test > > Patch: http://cr.openjdk.java.net/~redestad/8176188/open.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8176188 > > Thanks! > > /Claes > > [1] For some of the usages here we could improve somewhat by exposingthe > String(byte[], byte) > constructor, but I think that's out of scope here and I think we'd best > avoid leaking the > coder byte implementation detail outside of java.lang. > From stuart.marks at oracle.com Mon Dec 4 21:23:49 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Dec 2017 13:23:49 -0800 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> Message-ID: <2e1ff97f-ce42-50f3-4f11-105af0159a6e@oracle.com> On 12/4/17 12:36 PM, Martin Buchholz wrote: > On Mon, Dec 4, 2017 at 12:12 PM, Roger Riggs wrote: >> The java.time APIs refined the pattern used for Serialization proxies to >> document the relationship between >> the original class and its serialization proxies methods. >> > > Right. I was aware of the effort that java.time people put into their > serialization code. They did a good job, and my proposed change makes it > more like theirs (but even more like the ones in j.u.c.a.) Hi Martin, Roger, I disagree that the java.time approach is a Best Practice (capitalized), as Martin said, for dealing with documentation of serial proxies. It's a workaround for lack of support for this pattern in the javadoc tool. Having the redeclare all the fields of EnumSet as transient is evidence of this. (IMO the java.time classes are worse, as they include the fields of objects that are replaced by the proxy.) Having the original class appear in the serialized form documentation is a bug, since it never appears in the serial form. It sort-of helps define the relationship between the original class and the proxy, but in a confusing, roundabout way. If you don't like my alternative, fine; it has its own set of tradeoffs that might be net positive or negative. If you want to proceed with the current approach, then I won't stand in the way. At the very least there should be some boilerplate added to EnumSet that makes it clear that EnumSet itself never appears in the serial form. s'marks From Roger.Riggs at Oracle.com Mon Dec 4 21:46:24 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Dec 2017 16:46:24 -0500 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: <2e1ff97f-ce42-50f3-4f11-105af0159a6e@oracle.com> References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> <2e1ff97f-ce42-50f3-4f11-105af0159a6e@oracle.com> Message-ID: Hi Stuart, On 12/4/2017 4:23 PM, Stuart Marks wrote: > > > On 12/4/17 12:36 PM, Martin Buchholz wrote: >> On Mon, Dec 4, 2017 at 12:12 PM, Roger Riggs >> wrote: >>> The java.time APIs refined the pattern used for Serialization >>> proxies to >>> document the relationship between >>> the original class and its serialization proxies methods. >>> >> >> Right.? I was aware of the effort that java.time people put into their >> serialization code.? They did a good job, and my proposed change >> makes it >> more like theirs (but even more like the ones in j.u.c.a.) > > Hi Martin, Roger, > > I disagree that the java.time approach is a Best Practice > (capitalized), as Martin said, for dealing with documentation of > serial proxies. It's a workaround for lack of support for this pattern > in the javadoc tool. I wouldn't argue with javadoc tool being improved, but with the tools available... > > Having the redeclare all the fields of EnumSet as transient is > evidence of this. (IMO the java.time classes are worse, as they > include the fields of objects that are replaced by the proxy.) Another alternative is to define ObjectStreamField[] serialPersistentFields = {}; > > Having the original class appear in the serialized form documentation > is a bug, since it never appears in the serial form. It sort-of helps > define the relationship between the original class and the proxy, but > in a confusing, roundabout way. I disagree here since it implements Serializable and needs to be indexed as such. > > If you don't like my alternative, fine; it has its own set of > tradeoffs that might be net positive or negative. If you want to > proceed with the current approach, then I won't stand in the way. At > the very least there should be some boilerplate added to EnumSet that > makes it clear that EnumSet itself never appears in the serial form. I don't disagree, there are many things that could be improved. Roger From claes.redestad at oracle.com Mon Dec 4 21:47:20 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 4 Dec 2017 22:47:20 +0100 Subject: RFR: 8176188: jdk/internal/misc/JavaLangAccess/NewUnsafeString.java failing since 9-b93 In-Reply-To: References: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> Message-ID: Hi Martin, On 2017-12-04 22:06, Martin Buchholz wrote: > I'm rather sad about what happened to our non-copying String > constructions for trusted code.? This is a performance regression with > the change in String representation that should have blocked that > change IMO.? I think we should have a plan for moving in the opposite > direction.? I don't think we can implement something as ambitious as > Rust's ownership tracking, so have to restrict ourselves to trusted > code.? The use case that keeps coming up is constructing zip entry > names, which are much more likely to be pure ASCII than their file > contents. > > I don't have a good design for how one could do that, and who the > trusted set of callers is (at least java.base, not java.lang), but I > think we should set a direction. as I alluded to in a footnote there exists a non-copying String(byte[] value, byte coder) constructor - the problem is that it's somewhat cumbersome to use: - first off, the caller needs to be aware about the value of String.COMPACT_STRINGS: if false, all strings needs to be UTF-16 encoded and the coder byte always set to String.UTF16 - secondly, the caller needs to know if the byte[] you're constructing needs to be LATIN-1 or UTF-16 encoded up front and act accordingly Some of the more performance sensitive uses outside of java.lang was addressed by the Compact Strings update, for example the implementation backing java.util.UUID was somewhat surprisingly moved into java.lang.Long::fastUUID[1]. Something similar is doable for the java.sql types, but further complicated by those classes being in a different module, and ultimately questionable since their implementations in JDK 9 are quite a bit more performant than in any previous release (thus not technically a regression). That leaves StringJoiner as the one case that stands out. And the fact that existing uses of String(byte[], byte) are a bit of an eye-sore[1!!1!]. One idea I'm tinkering with here is to have a trusted, package-private SharedStringBuilder added to java.lang and exposed via SharedSecrets. It'd more or less mimic StringBuilder (including deal with inflating the byte[] when necessary, encapsulate the awkward String.COMPACT_STRINGS checks etc) but enable calling String(byte[], byte) in the toString() call.? To be effective it'll only have a single constructor taking the capacity, and should probably throw IOOBE rather than resize the internal buffer. Some cases like Long::fastUUID could probably be much simplified by using such a builder instead (for a very minimal overhead). Does that sound reasonable? At any rate I think of this as a possible follow-up RFE, and not an alternative to the cleanup/"bugfix" at hand. Thanks! /Claes [1] http://hg.openjdk.java.net/jdk/jdk/file/532cdc178e42/src/java.base/share/classes/java/lang/Long.java#l427 From martinrb at google.com Mon Dec 4 22:23:43 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 14:23:43 -0800 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> <2e1ff97f-ce42-50f3-4f11-105af0159a6e@oracle.com> Message-ID: On Mon, Dec 4, 2017 at 1:46 PM, Roger Riggs wrote: > > If you don't like my alternative, fine; it has its own set of tradeoffs > that might be net positive or negative. If you want to proceed with the > current approach, then I won't stand in the way. At the very least there > should be some boilerplate added to EnumSet that makes it clear that > EnumSet itself never appears in the serial form. > > I don't disagree, there are many things that could be improved. > I only volunteered to bring EnumSet (as the poster child for the Serialization Proxy Pattern) into a no-worse state than other classes implementing the pattern. The doc of the writeReplace and readObject methods is pretty good implicit documentation that the pattern applies here. Serialization overall remains as deeply flawed as ever. I still plan to submit what I have now. From xueming.shen at oracle.com Mon Dec 4 23:14:06 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 04 Dec 2017 15:14:06 -0800 Subject: RFR JDK-8187485: Update Zip implementation to use Cleaner, not finalizers Message-ID: <5A25D6BE.60203@oracle.com> Hi Please review the revision to the change for JDK-8187485: Update Zip implementation to use Cleaner, not finalizers For compatibility with previous jdk releases, in this proposed revision the corresponding close()/end() methods will be called when the ZipFile/Inflater/Deflater object has become unreachable, if the ZipFile/Inflater/Deflater has been subclassed and its close()/end() method has been overridden. In this case, the finalization mechanism is used to clean up the underlying resources. If the ZipFile/Inflater/Deflater has not been subclassed OR the corresponding close()/end() method has not been overridden, then the Cleaner mechanism is used to do the cleanup. issue: https://bugs.openjdk.java.net/browse/JDK-8185582 webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev csr: https://bugs.openjdk.java.net/browse/JDK-8187485 Thanks, Sherman From martinrb at google.com Mon Dec 4 23:46:18 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 15:46:18 -0800 Subject: RFR: 8176188: jdk/internal/misc/JavaLangAccess/NewUnsafeString.java failing since 9-b93 In-Reply-To: References: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> Message-ID: Thanks, Claes. I think we're in agreement! I did that shared String optimization for StringJoiner. It's a lot harder to justify in the new String world because we have to handle non-ASCII, and the non-ASCII case is actually fairly likely. Does it make sense to keep COMPACT_STRINGS as an option in openjdk? If essentially every user runs with that flag turned on, bit rot is likely to set in, and even if not, we have to remain ever vigilant to not break anyone turning it off. The code in Long.fastUUID is indeed ugly. I've never heard of UUID creation being a bottleneck. At Google it sometimes seems all our java performance problems are with zip file manipulation. I might have avoided the code duplication in Long.fastUUID by: - build the all-ASCII byte[] unconditionally - if COMPACT_STRINGS, then use the secret LATIN1 constructor, else use the public constructor(byte[], ISO_8859_1) One idea for a public performant API is to add something like Charset.toString(byte[]) or Charset.toString(byte[], int start, int length) via default method on Charset. Then; override that in ISO_8859_1 to do only a single copy (and can we cheat somehow to do no copies?) On Mon, Dec 4, 2017 at 1:47 PM, Claes Redestad wrote: > Hi Martin, > > On 2017-12-04 22:06, Martin Buchholz wrote: > >> I'm rather sad about what happened to our non-copying String >> constructions for trusted code. This is a performance regression with the >> change in String representation that should have blocked that change IMO. >> I think we should have a plan for moving in the opposite direction. I >> don't think we can implement something as ambitious as Rust's ownership >> tracking, so have to restrict ourselves to trusted code. The use case that >> keeps coming up is constructing zip entry names, which are much more likely >> to be pure ASCII than their file contents. >> >> I don't have a good design for how one could do that, and who the trusted >> set of callers is (at least java.base, not java.lang), but I think we >> should set a direction. >> > > as I alluded to in a footnote there exists a non-copying String(byte[] > value, byte coder) constructor - the problem is that it's somewhat > cumbersome to use: > > - first off, the caller needs to be aware about the value of > String.COMPACT_STRINGS: if false, all strings needs to be UTF-16 encoded > and the coder byte always set to String.UTF16 > - secondly, the caller needs to know if the byte[] you're constructing > needs to be LATIN-1 or UTF-16 encoded up front and act accordingly > > Some of the more performance sensitive uses outside of java.lang was > addressed by the Compact Strings update, for example the implementation > backing java.util.UUID was somewhat surprisingly moved into > java.lang.Long::fastUUID[1]. Something similar is doable for the java.sql > types, but further complicated by those classes being in a different > module, and ultimately questionable since their implementations in JDK 9 > are quite a bit more performant than in any previous release (thus not > technically a regression). > > That leaves StringJoiner as the one case that stands out. And the fact > that existing uses of String(byte[], byte) are a bit of an eye-sore[1!!1!]. > > One idea I'm tinkering with here is to have a trusted, package-private > SharedStringBuilder added to java.lang and exposed via SharedSecrets. It'd > more or less mimic StringBuilder (including deal with inflating the byte[] > when necessary, encapsulate the awkward String.COMPACT_STRINGS checks etc) > but enable calling String(byte[], byte) in the toString() call. To be > effective it'll only have a single constructor taking the capacity, and > should probably throw IOOBE rather than resize the internal buffer. Some > cases like Long::fastUUID could probably be much simplified by using such a > builder instead (for a very minimal overhead). Does that sound reasonable? > At any rate I think of this as a possible follow-up RFE, and not an > alternative to the cleanup/"bugfix" at hand. > > Thanks! > > /Claes > > [1] http://hg.openjdk.java.net/jdk/jdk/file/532cdc178e42/src/jav > a.base/share/classes/java/lang/Long.java#l427 > > From xueming.shen at oracle.com Mon Dec 4 23:59:52 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 04 Dec 2017 15:59:52 -0800 Subject: RFR: 8176188: jdk/internal/misc/JavaLangAccess/NewUnsafeString.java failing since 9-b93 In-Reply-To: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> References: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> Message-ID: <5A25E178.2010707@oracle.com> +1 On 12/3/17, 11:53 AM, Claes Redestad wrote: > Hi, > > the compact strings JEP changed semantics of the package-private > String(char[], boolean) > constructor to do the same as the public String(char[]) constructor. > > Previously the former was used in trusted, internal code to avoid > copying the given char[], > but since the char[] now has to be converted to a byte[] that > optimization is no longer > possible via this method[1], and tests that checked that the returned > string shared the > given char[] naturally stopped working. > > To fix this bug I propose the following clean-up: > - change all uses of JavaLangAccess.newUnsafeString(char[]) to new > String(char[]) > - remove the package-private String(char[], boolean) constructor > - remove the newUnsafeString from JavaLangAccess > - remove the now unnecessary NewUnsafeString test > > Patch: http://cr.openjdk.java.net/~redestad/8176188/open.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8176188 > > Thanks! > > /Claes > > [1] For some of the usages here we could improve somewhat by > exposingthe String(byte[], byte) > constructor, but I think that's out of scope here and I think we'd > best avoid leaking the > coder byte implementation detail outside of java.lang. From stuart.marks at oracle.com Tue Dec 5 00:20:24 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Dec 2017 16:20:24 -0800 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> <2e1ff97f-ce42-50f3-4f11-105af0159a6e@oracle.com> Message-ID: <30f47f0d-2c0c-3d46-1cdf-bcf0238da1a3@oracle.com> >> If you don't like my alternative, fine; it has its own set of tradeoffs >> that might be net positive or negative. If you want to proceed with the >> current approach, then I won't stand in the way. At the very least there >> should be some boilerplate added to EnumSet that makes it clear that >> EnumSet itself never appears in the serial form. > I don't disagree, there are many things that could be improved. > > I only volunteered to bring EnumSet (as the poster child for the Serialization > Proxy Pattern) into a no-worse state than other classes implementing the > pattern.? The doc of the writeReplace and readObject methods is pretty good > implicit documentation that the pattern applies here.? Serialization overall > remains as deeply flawed as ever. > > I still plan to submit what I have now. Thanks for volunteering. It goes to show that no good deed goes unpunished. :-) To close the loop on this, I think what you have is acceptable. I also think that "no-worse state" is a better characterization than "Best Practice," which seems to imply that no further improvement is possible or necessary. And finally, Jon Gibbons has filed JDK-8193019 to cover future javadoc enhancements to better support serialization. s'marks From stevenschlansker at gmail.com Tue Dec 5 00:24:54 2017 From: stevenschlansker at gmail.com (Steven Schlansker) Date: Mon, 4 Dec 2017 16:24:54 -0800 Subject: RFR: 8176188: jdk/internal/misc/JavaLangAccess/NewUnsafeString.java failing since 9-b93 In-Reply-To: References: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> Message-ID: > On Dec 4, 2017, at 3:46 PM, Martin Buchholz wrote: > > The code in Long.fastUUID is indeed ugly. I've never heard of UUID > creation being a bottleneck. At Google it sometimes seems all our java > performance problems are with zip file manipulation. At $MY_WORK, we never use Zip files, but indeed ran into some really nasty performance bottlenecks in UUID, and cared enough to whine about it and get it fixed ;) http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-January/013494.html http://bugs.java.com/view_bug.do?bug_id=JDK-8006627 While I doubt a single string allocation is make or break for us (by far the bigger problem was the truly awful use of regex and tons of intermediate arrays) but UUID is a core data type and people expect it to be reasonably performant. From claes.redestad at oracle.com Tue Dec 5 01:10:31 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 5 Dec 2017 02:10:31 +0100 Subject: RFR: 8176188: jdk/internal/misc/JavaLangAccess/NewUnsafeString.java failing since 9-b93 In-Reply-To: References: <07b19aea-7e33-b441-509d-c15f6e1fab92@oracle.com> Message-ID: On 2017-12-05 00:46, Martin Buchholz wrote: > Thanks, Claes. > > I think we're in agreement! Great! > > I did that shared String optimization for StringJoiner. It's a lot > harder to justify in the new String world because we have to handle > non-ASCII, and the non-ASCII case is actually fairly likely. Let's sleep on it, at least. :-) > Does it make sense to keep COMPACT_STRINGS as an option in openjdk?? > If essentially every user runs with that flag turned on, bit rot is > likely to set in, and even if not, we have to remain ever vigilant to > not break anyone turning it off. I think it was only kept as a safeguard in case anyone would run into serious trouble with the implementation (unlikely), so it might be reasonable to deprecate the flag in an upcoming release. Maybe survey if anyone ever had to opt-out of it first. > > The code in Long.fastUUID is indeed ugly.? I've never heard of UUID > creation being a bottleneck.? At Google it sometimes seems all our > java performance problems are with zip file manipulation. I see Steven already said Hi. I helped him get this optimization in, and it did look a bit nicer originally. :-) > > I might have avoided the code duplication in Long.fastUUID by: > - build the all-ASCII byte[] unconditionally > - if COMPACT_STRINGS, then use the secret LATIN1 constructor, else use > the public constructor(byte[], ISO_8859_1) I think the implementors felt it would be unfair to penalize those who had to opt-out of it for some unlikely reason. > > One idea for a public performant API is to add something like > Charset.toString(byte[]) or Charset.toString(byte[], int start, int > length) via default method on Charset.? Then; override that > in?ISO_8859_1 to do only a single copy (and can we cheat somehow to do > no copies?) I think these would need to clone the byte[] most of the time, so I think it'd not be that much different from new String(byte[], ISO_8859_1). The variant with Charset.toString(byte[], int, int) could optimize away one copy when start > 0 or length < bytes.length, but this seems like a corner case gain that doesn't really motivate adding two new methods to every Charset class. I guess a SharedStringBuilder (InlineStringBuilder?) could be made public if we used $suitable_concurrency_trick to ensure the toString() is callable at most once and that once it's been called the byte[] can't be written to again. Maybe a ThreadLocal-based implementation could suffice if we disallow nested InlineStringBuilder use (shouldn't be too harsh of a limitation). /Claes > > On Mon, Dec 4, 2017 at 1:47 PM, Claes Redestad > > wrote: > > Hi Martin, > > On 2017-12-04 22:06, Martin Buchholz wrote: > > I'm rather sad about what happened to our non-copying String > constructions for trusted code.? This is a performance > regression with the change in String representation that > should have blocked that change IMO.? I think we should have a > plan for moving in the opposite direction.? I don't think we > can implement something as ambitious as Rust's ownership > tracking, so have to restrict ourselves to trusted code.? The > use case that keeps coming up is constructing zip entry names, > which are much more likely to be pure ASCII than their file > contents. > > I don't have a good design for how one could do that, and who > the trusted set of callers is (at least java.base, not > java.lang), but I think we should set a direction. > > > as I alluded to in a footnote there exists a non-copying > String(byte[] value, byte coder) constructor - the problem is that > it's somewhat cumbersome to use: > > - first off, the caller needs to be aware about the value of > String.COMPACT_STRINGS: if false, all strings needs to be UTF-16 > encoded and the coder byte always set to String.UTF16 > - secondly, the caller needs to know if the byte[] you're > constructing needs to be LATIN-1 or UTF-16 encoded up front and > act accordingly > > Some of the more performance sensitive uses outside of java.lang > was addressed by the Compact Strings update, for example the > implementation backing java.util.UUID was somewhat surprisingly > moved into java.lang.Long::fastUUID[1]. Something similar is > doable for the java.sql types, but further complicated by those > classes being in a different module, and ultimately questionable > since their implementations in JDK 9 are quite a bit more > performant than in any previous release (thus not technically a > regression). > > That leaves StringJoiner as the one case that stands out. And the > fact that existing uses of String(byte[], byte) are a bit of an > eye-sore[1!!1!]. > > One idea I'm tinkering with here is to have a trusted, > package-private SharedStringBuilder added to java.lang and exposed > via SharedSecrets. It'd more or less mimic StringBuilder > (including deal with inflating the byte[] when necessary, > encapsulate the awkward String.COMPACT_STRINGS checks etc) but > enable calling String(byte[], byte) in the toString() call.? To be > effective it'll only have a single constructor taking the > capacity, and should probably throw IOOBE rather than resize the > internal buffer. Some cases like Long::fastUUID could probably be > much simplified by using such a builder instead (for a very > minimal overhead). Does that sound reasonable? At any rate I think > of this as a possible follow-up RFE, and not an alternative to the > cleanup/"bugfix" at hand. > > Thanks! > > /Claes > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/532cdc178e42/src/java.base/share/classes/java/lang/Long.java#l427 > > > From stuart.marks at oracle.com Tue Dec 5 03:20:17 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Dec 2017 19:20:17 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) Message-ID: Hi all, Please review this small enhancement to add a default method Collection.toArray(generator) that takes a function that creates the destination array. This is analogous to Stream.toArray(generator). Bug: https://bugs.openjdk.java.net/browse/JDK-8060192 Webrev: http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.0/ Thanks, s'marks From martinrb at google.com Tue Dec 5 03:50:31 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 19:50:31 -0800 Subject: RFR: 8193031: Collections.addAll is likely to perform worse than Collection.addAll Message-ID: https://bugs.openjdk.java.net/browse/JDK-8193031 http://cr.openjdk.java.net/~martin/webrevs/jdk/Collections-addAll/ From huizhe.wang at oracle.com Tue Dec 5 03:52:18 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 04 Dec 2017 19:52:18 -0800 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: <34dacec0-7e2f-65db-1262-30c0c4a279c1@oracle.com> References: <5A20F2AB.3050608@oracle.com> <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> <5A21BD18.2050201@oracle.com> <34dacec0-7e2f-65db-1262-30c0c4a279c1@oracle.com> Message-ID: <5A2617F2.6030005@oracle.com> On 12/3/17, 11:34 AM, Alan Bateman wrote: > > > On 01/12/2017 20:35, Joe Wang wrote: >> : >> >>> >>> I think URLDecoder.decode methods should have @throws >>> IllegalArgumentException. I realize it's implementation specific as >>> to whether IAE is thrown with bad input but given that the RI does >>> throw IAE then consumers of the API should be prepared for it. The >>> @implNote can stay and probably should be copied into the legacy >>> decode method too. >> >> I added @throws IAE. On a 2nd thought, would that give no flexibility >> to alternative impls as the general (class) spec had given? With this >> addition, it becomes a requirement. > If the @throws description starts with "if the implementation ..." > then it should be clear that it is optional. Thanks Alan. I made the change accordingly. http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html -Joe > > -Alan From martinrb at google.com Tue Dec 5 04:05:41 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 20:05:41 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: Message-ID: + // no spec changes relative to supertype + public T[] toArray(IntFunction generator) { You probably at least need @inheritDoc for the unchecked exception throws (as I've forgotten many times) On Mon, Dec 4, 2017 at 7:20 PM, Stuart Marks wrote: > Hi all, > > Please review this small enhancement to add a default method > Collection.toArray(generator) that takes a function that creates the > destination array. This is analogous to Stream.toArray(generator). > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8060192 > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.0/ > > Thanks, > > s'marks > From martinrb at google.com Tue Dec 5 04:26:02 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 20:26:02 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: Message-ID: The needToWorkAround6260652 changes ought to be in a separate changeset. The biggest question is whether Collection.toArray(generator) pulls its weight, especially in view of https://shipilev.net/blog/2016/arrays-wisdom-ancients. I rarely want to dump elements into a typed array. Dumping into Object[] with toArray() is just fine for me (but I'm a biased core library developer). From martinrb at google.com Tue Dec 5 04:36:31 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Dec 2017 20:36:31 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: Message-ID: On Mon, Dec 4, 2017 at 8:26 PM, Martin Buchholz wrote: > > The biggest question is whether Collection.toArray(generator) pulls its > weight, especially in view of https://shipilev.net/blog/ > 2016/arrays-wisdom-ancients. > Oh wait - Aleksey actually links to this bug! Sounds like he would be the most qualified reviewer! From forax at univ-mlv.fr Tue Dec 5 07:03:47 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 5 Dec 2017 08:03:47 +0100 (CET) Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: Message-ID: <1134719756.620531.1512457427706.JavaMail.zimbra@u-pem.fr> Hi Martin, ----- Mail original ----- > De: "Martin Buchholz" > ?: "Stuart Marks" > Cc: "core-libs-dev" > Envoy?: Mardi 5 D?cembre 2017 05:26:02 > Objet: Re: RFR(s): 8060192: Add default method Collection.toArray(generator) > The needToWorkAround6260652 changes ought to be in a separate changeset. > > The biggest question is whether Collection.toArray(generator) pulls its > weight, especially in view of > https://shipilev.net/blog/2016/arrays-wisdom-ancients. > > I rarely want to dump elements into a typed array. Dumping into Object[] > with toArray() is just fine for me (but I'm a biased core library > developer). Dumping an ArrayList into an array of String is fairly frequent, i think. The main issue with the current API, T[] toArray(T[] array), is that T can be unrelated to E (the type of the element in the collection) so one can write ArrayList list = ... String[] array = list.toArray(new String[0]); it may even works if the list is empty. It's getting worst if E and T can be a primitive type (with valhalla), because toArray(T[]) as to support all combinations. So in my opinion, introducing toArray(generator) is a step in the right direction. cheers, R?mi From stuart.marks at oracle.com Tue Dec 5 07:27:41 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Dec 2017 23:27:41 -0800 Subject: RFR: 8193031: Collections.addAll is likely to perform worse than Collection.addAll In-Reply-To: References: Message-ID: <354d9037-95e8-97bd-bd9d-5f6bebae9f48@oracle.com> On 12/4/17 7:50 PM, Martin Buchholz wrote: > https://bugs.openjdk.java.net/browse/JDK-8193031 > http://cr.openjdk.java.net/~martin/webrevs/jdk/Collections-addAll/ * Adds all of the specified elements to the specified collection. * Elements to be added may be specified individually or as an array. * The behavior of this convenience method is identical to that of - * {@code c.addAll(Arrays.asList(elements))}, but this method is likely - * to run significantly faster under most implementations. + * {@code c.addAll(Arrays.asList(elements))}. * *

When elements are specified individually, this method provides a * convenient way to add a few elements to an existing collection: I agree with the removal of "significantly faster." In some real cases, it's definitely not! Usually the behavior wording isn't "is identical to". I'd prefer something like "the behavior is as if...." public static boolean addAll(Collection c, T... elements) { - boolean result = false; - for (T element : elements) - result |= c.add(element); - return result; + return c.addAll(Arrays.asList(elements)); } So, this seems sensible, but the tradeoffs aren't obvious. If the destination is an ArrayList, the first thing that addAll() does is call toArray() on the argument. And Arrays$ArrayList.toArray() makes a copy, as required by spec. So this redundantly copies every element, compared to the for-loop. But I did some quick benchmarks and I found that bulk copies are so fast that even doing two of them is quite a bit faster than a single copy in a for-loop, ranging from 4x-6x faster over a variety of sizes. This also requires temp space the size of the input array. This gives me (and the garbage collector) a bit of a pause. Turning to other collections, the AbstractCollection.addAll() implementation runs essentially the same loop as here, except that it runs it over its Collection argument instead of an array. Thus it uses an Iterator to loop instead of indexing over an array. Here, converting to a list has a slight performance disadvantage, around 10-15% (for HashSet, which uses AbstractCollection.addAll). You point out in the bug report that the implementation of Collections.addAll() misses out on optimizations that implementations can provide for bulk updates. This is true, but my feeling is that loses something by converting the array into a Collection before calling the virtual addAll(Collection) method. How about this alternative: do the same thing we did with sort(). Add a new default method Collection.addEach(T...) whose default implementation is either the original implementation of Collections.addAll() or your proposed modification. (We should use a name other than addAll, since overloading with a varargs method is a bad idea.) Add overrides for key implementations like ArrayList. Then, have Collections.addAll(c, T... elements) delegate to c.addEach(elements). For ArrayList, at least, this would avoid the redundant copy and temporary allocation, which seems nice. s'marks From forax at univ-mlv.fr Tue Dec 5 07:36:34 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 5 Dec 2017 08:36:34 +0100 (CET) Subject: RFR: 8193031: Collections.addAll is likely to perform worse than Collection.addAll In-Reply-To: <354d9037-95e8-97bd-bd9d-5f6bebae9f48@oracle.com> References: <354d9037-95e8-97bd-bd9d-5f6bebae9f48@oracle.com> Message-ID: <526183162.632690.1512459394491.JavaMail.zimbra@u-pem.fr> +1 being also able to write, List l = ... l.addEach("foo", "bar", "baz"); is a nice addition. R?mi ----- Mail original ----- > De: "Stuart Marks" > ?: "Martin Buchholz" > Cc: "core-libs-dev" > Envoy?: Mardi 5 D?cembre 2017 08:27:41 > Objet: Re: RFR: 8193031: Collections.addAll is likely to perform worse than Collection.addAll > On 12/4/17 7:50 PM, Martin Buchholz wrote: >> https://bugs.openjdk.java.net/browse/JDK-8193031 >> http://cr.openjdk.java.net/~martin/webrevs/jdk/Collections-addAll/ > > * Adds all of the specified elements to the specified collection. > * Elements to be added may be specified individually or as an array. > * The behavior of this convenience method is identical to that of > - * {@code c.addAll(Arrays.asList(elements))}, but this method is likely > - * to run significantly faster under most implementations. > + * {@code c.addAll(Arrays.asList(elements))}. > * > *

When elements are specified individually, this method provides a > * convenient way to add a few elements to an existing collection: > > I agree with the removal of "significantly faster." In some real cases, it's > definitely not! Usually the behavior wording isn't "is identical to". I'd prefer > something like "the behavior is as if...." > > public static boolean addAll(Collection c, T... elements) { > - boolean result = false; > - for (T element : elements) > - result |= c.add(element); > - return result; > + return c.addAll(Arrays.asList(elements)); > } > > So, this seems sensible, but the tradeoffs aren't obvious. > > If the destination is an ArrayList, the first thing that addAll() does is call > toArray() on the argument. And Arrays$ArrayList.toArray() makes a copy, as > required by spec. So this redundantly copies every element, compared to the > for-loop. > > But I did some quick benchmarks and I found that bulk copies are so fast that > even doing two of them is quite a bit faster than a single copy in a for-loop, > ranging from 4x-6x faster over a variety of sizes. > > This also requires temp space the size of the input array. This gives me (and > the garbage collector) a bit of a pause. > > Turning to other collections, the AbstractCollection.addAll() implementation > runs essentially the same loop as here, except that it runs it over its > Collection argument instead of an array. Thus it uses an Iterator to loop > instead of indexing over an array. Here, converting to a list has a slight > performance disadvantage, around 10-15% (for HashSet, which uses > AbstractCollection.addAll). > > You point out in the bug report that the implementation of Collections.addAll() > misses out on optimizations that implementations can provide for bulk updates. > This is true, but my feeling is that loses something by converting the array > into a Collection before calling the virtual addAll(Collection) method. > > How about this alternative: do the same thing we did with sort(). > > Add a new default method Collection.addEach(T...) whose default implementation > is either the original implementation of Collections.addAll() or your proposed > modification. (We should use a name other than addAll, since overloading with a > varargs method is a bad idea.) Add overrides for key implementations like > ArrayList. Then, have Collections.addAll(c, T... elements) delegate to > c.addEach(elements). > > For ArrayList, at least, this would avoid the redundant copy and temporary > allocation, which seems nice. > > s'marks From stuart.marks at oracle.com Tue Dec 5 07:36:56 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Dec 2017 23:36:56 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: Message-ID: > > + // no spec changes relative to supertype > + public T[] toArray(IntFunction generator) { > > You probably at least need @inheritDoc for the unchecked exception throws (as I've forgotten many times) > There's a new javadoc option "--override-methods summary" that was recently added that we're using now. If an overriding method has no doc comment, it's simply listed in the "Methods inherited from" section at the bottom of the summary table. (It's now called "Methods declared in".) See it in action here: http://download.java.net/java/jdk10/docs/api/java/util/Properties.html Compare it to the JDK 9 version of Properties. (This suggests a big cleanup in collections, and probably many other areas, where docs were copied into subclasses for no good reason, or because they wanted to disinherit stuff that properly belongs in @implSpec.) > The needToWorkAround6260652 changes ought to be in a separate changeset. OK, I can pull these out. > The biggest question is whether Collection.toArray(generator) pulls its weight, especially in view of https://shipilev.net/blog/2016/arrays-wisdom-ancients. > > I rarely want to dump elements into a typed array. Dumping into Object[] with toArray() is just fine for me (but I'm a biased core library developer). Yeah, most collections want to use Object[]. But when you want a typed array, this is really helpful. > Oh wait - Aleksey actually links to this bug!? Sounds like he would be the most > qualified reviewer! :-) s'marks From david.lloyd at redhat.com Tue Dec 5 13:59:33 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 5 Dec 2017 07:59:33 -0600 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <1134719756.620531.1512457427706.JavaMail.zimbra@u-pem.fr> References: <1134719756.620531.1512457427706.JavaMail.zimbra@u-pem.fr> Message-ID: On Tue, Dec 5, 2017 at 1:03 AM, Remi Forax wrote: > Dumping an ArrayList into an array of String is fairly frequent, i think. > > The main issue with the current API, T[] toArray(T[] array), is that T can be unrelated to E (the type of the element in the collection) so one can write > ArrayList list = ... > String[] array = list.toArray(new String[0]); > it may even works if the list is empty. > > It's getting worst if E and T can be a primitive type (with valhalla), because toArray(T[]) as to support all combinations. > > So in my opinion, introducing toArray(generator) is a step in the right direction. The signature of the proposed generator is > public T[] toArray(IntFunction generator) So I don't think that anything has really changed in this regard; T can still be unrelated to E. That said, sending in a constant String[0] is probably just as good as a generator, or better (in my naive estimation the tradeoff is a >0 comparison versus an interface method dispatch), than an i -> new String[i] unless some kind of benchmark would appear which disproves that hypothesis. -- - DML From Alan.Bateman at oracle.com Tue Dec 5 14:07:04 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Dec 2017 14:07:04 +0000 Subject: RFR 8187222 : ClassLoader.getSystemClassLoader not clear if recursive initialization leads to ISE or unspecified error In-Reply-To: <381991d7-544a-37ba-8bb5-92c4c0144596@oracle.com> References: <6d4e7e94-9e83-f215-0a2a-10f0dd3b69f4@oracle.com> <93aa28c5-3d83-7536-e574-60517342e481@oracle.com> <2e121d61-5f9c-3a4a-b3ef-a416b98318c3@oracle.com> <08ea805e-4da5-3bd9-904c-9a56e7899d03@oracle.com> <79EEA360-45C5-47AA-B48B-D9418F1A56B3@oracle.com> <7982a98f-c56e-b8b7-0f1f-abf75fb4e056@oracle.com> <381991d7-544a-37ba-8bb5-92c4c0144596@oracle.com> Message-ID: On 04/12/2017 20:18, mandy chung wrote: > > > On 12/4/17 12:11 PM, Brent Christian wrote: >> So rethrow RuntimeExceptions directly, whether they're the cause of >> an InvocationTargetException or not. >> >> Updated webrev: >> http://cr.openjdk.java.net/~bchristi/8187222/webrev.02/ >> > > Looks good. Yes, I think this looks okay (although future maintainers might wonder about the additionally unwrapping). -Alan From claes.redestad at oracle.com Tue Dec 5 14:51:48 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 5 Dec 2017 15:51:48 +0100 Subject: RFR: 8193064: JarFile::getEntry0 method reference use cause for startup regression Message-ID: <50ee2e9d-71ef-ba8b-baef-77722827c6e4@oracle.com> Hi, desugaring the JarFileEntry::new in JarFile::getEntry0 avoids a hefty startup regression on our smallest startup tests: http://cr.openjdk.java.net/~redestad/8193064/open.00/ I think the implementation could be improved further by using static non-capturing functions rather than Jar-/ZipFileEntry::new (which, since they reference a non static class constructor, are implicitly capturing) and filed: https://bugs.openjdk.java.net/browse/JDK-8193066 Thanks! /Claes From xueming.shen at oracle.com Tue Dec 5 16:13:02 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 05 Dec 2017 08:13:02 -0800 Subject: RFR: 8193064: JarFile::getEntry0 method reference use cause for startup regression In-Reply-To: <50ee2e9d-71ef-ba8b-baef-77722827c6e4@oracle.com> References: <50ee2e9d-71ef-ba8b-baef-77722827c6e4@oracle.com> Message-ID: <5A26C58E.3030303@oracle.com> Hi Claes, I'm OK with this. It would be help to add a simple comment to remind why we have to do this here, before there is better alternative in place for the startup. Thanks! Sherman On 12/5/17, 6:51 AM, Claes Redestad wrote: > Hi, > > desugaring the JarFileEntry::new in JarFile::getEntry0 avoids a hefty > startup regression on our smallest startup tests: > > http://cr.openjdk.java.net/~redestad/8193064/open.00/ > > I think the implementation could be improved further by using static > non-capturing functions rather than Jar-/ZipFileEntry::new (which, > since they reference a non static class constructor, are implicitly > capturing) and filed: https://bugs.openjdk.java.net/browse/JDK-8193066 > > Thanks! > > /Claes From mandy.chung at oracle.com Tue Dec 5 16:14:19 2017 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 5 Dec 2017 08:14:19 -0800 Subject: RFR: 8193064: JarFile::getEntry0 method reference use cause for startup regression In-Reply-To: <50ee2e9d-71ef-ba8b-baef-77722827c6e4@oracle.com> References: <50ee2e9d-71ef-ba8b-baef-77722827c6e4@oracle.com> Message-ID: <3741ab55-e8a5-3157-0c0a-4c2c612b5ffd@oracle.com> +1 nit: s/jarFileEntryFunction/newJarFileEntryFn/ Mandy On 12/5/17 6:51 AM, Claes Redestad wrote: > Hi, > > desugaring the JarFileEntry::new in JarFile::getEntry0 avoids a hefty > startup regression on our smallest startup tests: > > http://cr.openjdk.java.net/~redestad/8193064/open.00/ > > I think the implementation could be improved further by using static > non-capturing functions rather than Jar-/ZipFileEntry::new (which, > since they reference a non static class constructor, are implicitly > capturing) and filed: https://bugs.openjdk.java.net/browse/JDK-8193066 > > Thanks! > > /Claes From claes.redestad at oracle.com Tue Dec 5 16:25:31 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 5 Dec 2017 17:25:31 +0100 Subject: RFR: 8193064: JarFile::getEntry0 method reference use cause for startup regression In-Reply-To: <5A26C58E.3030303@oracle.com> References: <50ee2e9d-71ef-ba8b-baef-77722827c6e4@oracle.com> <5A26C58E.3030303@oracle.com> Message-ID: <2e27c96d-f184-7e43-2e52-ee62e6e42e0c@oracle.com> Hi Sherman, sure, I'll add a small comment to this effect. /Claes On 2017-12-05 17:13, Xueming Shen wrote: > Hi Claes, > > I'm OK with this. It would be help to add a simple comment to remind > why we have to do this > here, before there is better alternative in place for the startup. > > Thanks! > Sherman > > On 12/5/17, 6:51 AM, Claes Redestad wrote: >> Hi, >> >> desugaring the JarFileEntry::new in JarFile::getEntry0 avoids a hefty >> startup regression on our smallest startup tests: >> >> http://cr.openjdk.java.net/~redestad/8193064/open.00/ >> >> I think the implementation could be improved further by using static >> non-capturing functions rather than Jar-/ZipFileEntry::new (which, >> since they reference a non static class constructor, are implicitly >> capturing) and filed: https://bugs.openjdk.java.net/browse/JDK-8193066 >> >> Thanks! >> >> /Claes > From claes.redestad at oracle.com Tue Dec 5 17:04:32 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 5 Dec 2017 18:04:32 +0100 Subject: RFR: 8193064: JarFile::getEntry0 method reference use cause for startup regression In-Reply-To: <3741ab55-e8a5-3157-0c0a-4c2c612b5ffd@oracle.com> References: <50ee2e9d-71ef-ba8b-baef-77722827c6e4@oracle.com> <3741ab55-e8a5-3157-0c0a-4c2c612b5ffd@oracle.com> Message-ID: Thanks Mandy, On 2017-12-05 17:14, mandy chung wrote: > +1 > > nit: s/jarFileEntryFunction/newJarFileEntryFn/ updated in-place along with a comment that this is deliberately not a method ref for startup reasons, as requested by Sherman. /Claes From rob.mckenna at oracle.com Tue Dec 5 17:48:24 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Tue, 5 Dec 2017 17:48:24 +0000 Subject: [RFR] 8160768: Add capability to custom resolve host/domain names within the default JDNI LDAP provider Message-ID: <20171205174824.GA3515@vimes> As this enhancement is limited to JDK10 this frees us up to explore a different approach. http://cr.openjdk.java.net/~robm/8160768/webrev.06/ In this webrev we're using the Service Provider Interface to load an implementation of the new LdapDnsProvider class. Implementations of this class are responsible for resolving DNS endpoints for a given ldap url should the default implementation be insufficient in a particular environment. Note: if a security manager is installed the ldapDnsProvider RuntimePermission must be granted. -Rob From paul.sandoz at oracle.com Tue Dec 5 18:34:20 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 5 Dec 2017 10:34:20 -0800 Subject: RFR 8015667 Stream.toArray(IntFunction) ArrayStoreException should refer to component type of array Message-ID: <89B96080-7E89-4C32-A741-FD1DC2A9C831@oracle.com> Hi, Since Stuart is cleaning up Collection.toArray etc we might as well do the same for Stream.toArray. diff -r df95bd1fd4b1 src/java.base/share/classes/java/util/stream/Stream.java --- a/src/java.base/share/classes/java/util/stream/Stream.java Tue Dec 05 09:44:32 2017 -0800 +++ b/src/java.base/share/classes/java/util/stream/Stream.java Tue Dec 05 10:32:27 2017 -0800 @@ -671,7 +671,8 @@ *

This is a terminal * operation. * - * @return an array containing the elements of this stream + * @return an array, whose {@linkplain Class#getComponentType runtime component + * type} is {@code Object}, containing the elements of this stream */ Object[] toArray(); @@ -694,13 +695,13 @@ * .toArray(Person[]::new); * } * - * @param the element type of the resulting array + * @param the component type of the resulting array * @param generator a function which produces a new array of the desired * type and the provided length * @return an array containing the elements in this stream - * @throws ArrayStoreException if the runtime type of the array returned - * from the array generator is not a supertype of the runtime type of every - * element in this stream + * @throws ArrayStoreException if the runtime type of any element of this + * stream is not assignable to the {@linkplain Class#getComponentType + * runtime component type} of the generated array */ A[] toArray(IntFunction generator); Paul. From paul.sandoz at oracle.com Tue Dec 5 18:47:19 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 5 Dec 2017 10:47:19 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: Message-ID: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> > On 4 Dec 2017, at 19:20, Stuart Marks wrote: > > Hi all, > > Please review this small enhancement to add a default method Collection.toArray(generator) that takes a function that creates the destination array. This is analogous to Stream.toArray(generator). > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8060192 > 345 *

Suppose {@code x} is a collection known to contain only strings. 346 * The following code can be used to dump the collection into a newly 347 * allocated array of {@code String}: Make it an API note? (same for toArray(T[]) 352 * @implSpec 353 * The default implementation calls the generator function with zero 354 * and then passes the resulting array to {@link #toArray(T[])}. That?s reasonable. I pondered about being vague and specifying a value in the range if [0, size()), to allow wiggle room for using 0 or size(). 360 * @throws ArrayStoreException if the runtime type of any element in this 361 * collection is not assignable to the {@linkplain Class#getComponentType 362 * runtime component type} of array returned by the generator function s/of array returned by the generator function/of the generated array/ ? Paul. > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.0/ > > Thanks, > > s'marks From stuart.marks at oracle.com Tue Dec 5 19:07:34 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 5 Dec 2017 11:07:34 -0800 Subject: RFR 8015667 Stream.toArray(IntFunction) ArrayStoreException should refer to component type of array In-Reply-To: <89B96080-7E89-4C32-A741-FD1DC2A9C831@oracle.com> References: <89B96080-7E89-4C32-A741-FD1DC2A9C831@oracle.com> Message-ID: Hi Paul, Thanks for picking this up. The change looks good. s'marks On 12/5/17 10:34 AM, Paul Sandoz wrote: > Hi, > > Since Stuart is cleaning up Collection.toArray etc we might as well do the same for Stream.toArray. > > diff -r df95bd1fd4b1 src/java.base/share/classes/java/util/stream/Stream.java > --- a/src/java.base/share/classes/java/util/stream/Stream.java Tue Dec 05 09:44:32 2017 -0800 > +++ b/src/java.base/share/classes/java/util/stream/Stream.java Tue Dec 05 10:32:27 2017 -0800 > @@ -671,7 +671,8 @@ > *

This is a terminal > * operation. > * > - * @return an array containing the elements of this stream > + * @return an array, whose {@linkplain Class#getComponentType runtime component > + * type} is {@code Object}, containing the elements of this stream > */ > Object[] toArray(); > > @@ -694,13 +695,13 @@ > * .toArray(Person[]::new); > * } > * > - * @param the element type of the resulting array > + * @param the component type of the resulting array > * @param generator a function which produces a new array of the desired > * type and the provided length > * @return an array containing the elements in this stream > - * @throws ArrayStoreException if the runtime type of the array returned > - * from the array generator is not a supertype of the runtime type of every > - * element in this stream > + * @throws ArrayStoreException if the runtime type of any element of this > + * stream is not assignable to the {@linkplain Class#getComponentType > + * runtime component type} of the generated array > */ > A[] toArray(IntFunction generator); > > Paul. > > From stuart.marks at oracle.com Tue Dec 5 19:19:07 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 5 Dec 2017 11:19:07 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> Message-ID: On 12/5/17 10:47 AM, Paul Sandoz wrote: > > 345 *

Suppose {@code x} is a collection known to contain only strings. > 346 * The following code can be used to dump the collection into a newly > 347 * allocated array of {@code String}: > > Make it an API note? (same for toArray(T[]) Will do. > 352 * @implSpec > 353 * The default implementation calls the generator function with zero > 354 * and then passes the resulting array to {@link #toArray(T[])}. > > That?s reasonable. I pondered about being vague and specifying a value in the range if [0, size()), to allow wiggle room for using 0 or size(). No, I really think it has to be exactly zero. If the default were to choose some value K < size(), the collection size could change to be smaller than K after allocation but before the call to toArray(T[]), resulting in an array that is unexpectedly large. > 360 * @throws ArrayStoreException if the runtime type of any element in this > 361 * collection is not assignable to the {@linkplain Class#getComponentType > 362 * runtime component type} of array returned by the generator function > > s/of array returned by the generator function/of the generated array/ ? Will do. s'marks From mandy.chung at oracle.com Tue Dec 5 19:27:10 2017 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 5 Dec 2017 11:27:10 -0800 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> Message-ID: <053ce75c-88d5-63c7-9c0f-b49a7a5cdb15@oracle.com> On 12/1/17 6:38 PM, Roger Riggs wrote: > Please review a revision to the work on remove a dependency on > finalization from > FileInputStream, FileOutputStream, and add cleanup of closing file > descriptors in FileChannel. This revised approach sounds reasonable that minimizes the compatibility risk in JDK 10 while giving an advanced notice to existing applications to prepare for the removal of FIS/FOS finalizer method and existing subclasses that override the close method should make appropriate change to prepare the incompatibility risk when the FIS/FOS finalizer is removed in a future release. Thanks for doing this. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ > Looks good overall.?? I agree with Peter's suggestion that subclasses overriding the close method will use AltFinalizer and subclasses overriding the finalizer but not close method will get invoked anyway and it can employ the cleaner to close the stream in that case.? I see the webrev has been updated to reflect that. Minor comments: Nit: I wonder if {@link #close} should be used instead of {@linkplain #close} in the javadoc (as I read that referring to the method rather than the action).?? Just to mention it. The javadoc of the close method: 343 * @implNote 344 * Overriding {@linkplain #close} to perform cleanup actions is reliable 345 * only when called directly or when called by try-with-resources. I think this can be @apiNote, like the @apiNote added in the finalize method.? Maybe you want to promote this to @apiNote when the finalize method is removed. test/jdk/sun/security/provider/FileInputStreamPool/FileInputStreamPoolTest.java Nit: @modules would be good to keep it sorted. Formatting nit: line 84-106: it reads to me that the indentation is 5-space thanks Mandy From paul.sandoz at oracle.com Tue Dec 5 19:35:37 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 5 Dec 2017 11:35:37 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> Message-ID: > On 5 Dec 2017, at 11:19, Stuart Marks wrote: > > > > On 12/5/17 10:47 AM, Paul Sandoz wrote: >> 345 *

Suppose {@code x} is a collection known to contain only strings. >> 346 * The following code can be used to dump the collection into a newly >> 347 * allocated array of {@code String}: >> Make it an API note? (same for toArray(T[]) > > Will do. > >> 352 * @implSpec >> 353 * The default implementation calls the generator function with zero >> 354 * and then passes the resulting array to {@link #toArray(T[])}. >> That?s reasonable. I pondered about being vague and specifying a value in the range if [0, size()), to allow wiggle room for using 0 or size(). > > No, I really think it has to be exactly zero. If the default were to choose some value K < size(), the collection size could change to be smaller than K after allocation but before the call to toArray(T[]), resulting in an array that is unexpectedly large. Ah yes, concurrent collections. Paul. > >> 360 * @throws ArrayStoreException if the runtime type of any element in this >> 361 * collection is not assignable to the {@linkplain Class#getComponentType >> 362 * runtime component type} of array returned by the generator function >> s/of array returned by the generator function/of the generated array/ ? > > Will do. > > s'marks From Roger.Riggs at Oracle.com Tue Dec 5 20:01:27 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Dec 2017 15:01:27 -0500 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: <053ce75c-88d5-63c7-9c0f-b49a7a5cdb15@oracle.com> References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> <053ce75c-88d5-63c7-9c0f-b49a7a5cdb15@oracle.com> Message-ID: Hi Mandy, Updated the webrev in place: http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ On 12/5/2017 2:27 PM, mandy chung wrote: > > > On 12/1/17 6:38 PM, Roger Riggs wrote: >> Please review a revision to the work on remove a dependency on >> finalization from >> FileInputStream, FileOutputStream, and add cleanup of closing file >> descriptors in FileChannel. > > This revised approach sounds reasonable that minimizes the > compatibility risk in JDK 10 while giving an advanced notice to > existing applications to prepare for the removal of FIS/FOS finalizer > method and existing subclasses that override the close method should > make appropriate change to prepare the incompatibility risk when the > FIS/FOS finalizer is removed in a future release. > > Thanks for doing this. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ >> > > Looks good overall.?? I agree with Peter's suggestion that subclasses > overriding the close method will use AltFinalizer and subclasses > overriding the finalizer but not close method will get invoked anyway > and it can employ the cleaner to close the stream in that case.? I see > the webrev has been updated to reflect that. Thanks for confirming > > Minor comments: > > Nit: I wonder if {@link #close} should be used instead of {@linkplain > #close} in the javadoc (as I read that referring to the method rather > than the action).?? Just to mention it. > done > The javadoc of the close method: > 343 * @implNote > 344 * Overriding {@linkplain #close} to perform cleanup actions is > reliable > 345 * only when called directly or when called by try-with-resources. > I think this can be @apiNote, like the @apiNote added in the finalize > method.? Maybe you want to promote this to @apiNote when the finalize > method is removed. > Yes, it is more like an apiNote > test/jdk/sun/security/provider/FileInputStreamPool/FileInputStreamPoolTest.java > Nit: @modules would be good to keep it sorted. Formatting nit: line > 84-106: it reads to me that the indentation is 5-space fixed Thanks, Roger From mandy.chung at oracle.com Tue Dec 5 21:23:26 2017 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 5 Dec 2017 13:23:26 -0800 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> <053ce75c-88d5-63c7-9c0f-b49a7a5cdb15@oracle.com> Message-ID: <6a834630-e000-9aee-1713-113f6472f170@oracle.com> On 12/5/17 12:01 PM, Roger Riggs wrote: > Hi Mandy, > > Updated the webrev in place: > http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ > Thanks.? Looks good. Mandy From brian.burkhalter at oracle.com Tue Dec 5 22:20:42 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 5 Dec 2017 14:20:42 -0800 Subject: RFR 8080225 FileInput/OutputStream/FileChannel cleanup should be improved In-Reply-To: <6a834630-e000-9aee-1713-113f6472f170@oracle.com> References: <2c4a2e07-6b6c-0c9d-a4b1-0d732ae40cba@oracle.com> <053ce75c-88d5-63c7-9c0f-b49a7a5cdb15@oracle.com> <6a834630-e000-9aee-1713-113f6472f170@oracle.com> Message-ID: On Dec 5, 2017, at 1:23 PM, mandy chung wrote: > On 12/5/17 12:01 PM, Roger Riggs wrote: >> Hi Mandy, >> >> Updated the webrev in place: http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ >> > > Thanks. Looks good. +1 Brian From david.holmes at oracle.com Wed Dec 6 01:07:57 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Dec 2017 11:07:57 +1000 Subject: Fw: Question about JEP 306. In-Reply-To: References: Message-ID: <2a493b34-a717-852b-23a8-d8827be0f61e@oracle.com> Adding core-libs-dev as both mailing lists are named in this JEP. David On 6/12/2017 11:02 AM, A Z wrote: > -I have been wondering what interest, focus or progress > > is happening around OpenJDK JEP 306: > > > http://openjdk.java.net/jeps/306 > > > -As a Java feature request, could I request feature 306? > Underflow and overflow need to be eliminated > from double and float. > > There could be an associativefp option. It should also be the case > that there should be a round half up for the decimal place > beyond the last one in the float or double range, to help support > all notions of > > (pow(sqrt(3),2) == 3) //true > > > -The other thing that there should be is a StrictMath > that works for BigInteger and BigDecimal, allowing: > > pow(BigDecimal,BigDecimal) > pi(...) > e(...) > sin(Bigdecimal) > asin(Bigdecimal) ... > > which would allow for any power at all, as well as the nth root. > It is also a good idea to include a radians and degrees version > of all the trig functions. > From jeremymanson at google.com Wed Dec 6 01:17:18 2017 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 5 Dec 2017 17:17:18 -0800 Subject: Change in properties for logging: deliberate? In-Reply-To: <628fd19f-b93b-ca66-b3b3-d65817bc8171@oracle.com> References: <71981898-fa64-e69c-5e56-246eaed548f7@oracle.com> <9adc01cf-fe46-f1be-6d92-182cc9ddd3ef@oracle.com> <59156c8a-8622-99b9-7a2f-b0c161868b4f@oracle.com> <5d3ffed1-6c0d-d2f2-a541-a82d5ed8ebd5@oracle.com> <9d326eca-e94f-cfb2-6daf-e9db456050ff@oracle.com> <628fd19f-b93b-ca66-b3b3-d65817bc8171@oracle.com> Message-ID: Hey folks, Any thoughts on a timeline for this? We're just having to decide what to do internally. If a patch is likely to arrive in the next month or so, then we'll probably wait, but if not, we should probably figure out a workaround. (I'm not trying to be too pushy - we can certainly figure out a workaround.) Jeremy On Mon, Nov 13, 2017 at 7:30 AM, Daniel Fuchs wrote: > Hi Jason, > > On 13/11/2017 15:14, Jason Mehrens wrote: > >> Hi Daniel, >> >> Sorry for the late reply I was offline for the long weekend. >> >> Hot reloads of the LogManager have always been a problem. I think you >> are running into https://bugs.openjdk.java.net/browse/JDK-8033661 in >> your testing and that is going to give you troubling results on what is >> recreated after the call. Make sure you test updateConfiguration which is >> the replacement everyone is to use going forward. >> > > Yes - I know - I fixed that one ;-) > > I think you'll want to make it so that "handlers" is just an alias name >> ".handlers". That way empty string is just name of the root logger which >> enables consistent use of other properties like ".level" and ".filter". >> If both are defined in the logging.properties, then install the union of >> the two line. >> > > That's precisely where I didn't want to go. > > When I fixed JDK-8033661 I choose to use "handlers" for the root logger > instead of ".handlers" when implementing updateConfiguration() because > "handlers" is explicitly documented in LogManager API documentation > and conf/logging.properties. > > So for the root logger the mapping function will only consider > "handlers" but not ".handlers". Trying to change that would > add too much complexity IMHO. > > best regards, > > -- daniel > > > >> Jason >> >> ________________________________________ >> From: Daniel Fuchs >> Sent: Friday, November 10, 2017 10:04 AM >> To: Jason Mehrens; mandy chung >> Cc: core-libs-dev at openjdk.java.net >> Subject: Re: Change in properties for logging: deliberate? >> >> Hi Jason, >> >> I have done a few tests with JDK 8 & 7. >> >> I have created custom handlers and added some >> debug traces in their constructors and debug methods. >> >> Then I have added these two lines to my logging.properties: >> >> handlers = custom.Handler >> .handlers = custom.DotHandler >> >> What I see is this: >> >> - the first time the configuration is read, two handlers >> are added to the root logger: >> - an instance of DotHandler (first), then an instance >> of Handler (second). >> >> Then if you call LogManager.readConfiguration() again, >> both handlers are closed, and this time only one >> instance of Handler is added to the root logger. >> No instance of DotHandler is added. >> From now on the property is ignored. >> >> This is because the root logger is a special beast: >> it will not be removed (like all other loggers) when >> LogManager.readConfiguration() is called. >> >> And as it happens, handlers are added to loggers >> when the loggers are added to the LogManager. >> As it happens, the ".handlers" property is only parsed >> and read when the root logger is added to the LogManager, >> and thus only once. >> >> The "handlers" property on the other hand is parsed >> every time LogManager.readConfiguration() is called. >> >> Given that, I suspect we should deprecate the use of >> ".handlers" for the root logger, as it appears that >> it has never worked properly. I could work on a patch >> for 10 (possibly backport it to 9 update) to preserve >> the strange behavior of 7 & 8, but is it worth it? >> >> What are your thoughts? >> >> best regards, >> >> -- daniel >> >> >> >> >> On 09/11/2017 19:50, Jason Mehrens wrote: >> >>> Daniel, >>> >>> I would assume you would fix since it is advertised as a feature over >>> here: https://docs.oracle.com/javase/1.5.0/docs/guide/logging/chan >>> ges.html >>> >>> If it helps, I've dug up a lot of the history on this over here a while >>> back: >>> https://stackoverflow.com/questions/36726431/in-a-java-util- >>> logging-logging-properties-file-whats-the-difference-between-h >>> >>> I've updated that to include the links to this new issue. Now that I've >>> linked this message thread to that message thread that should crash the >>> internet. :) >>> >>> Jason >>> >>> ________________________________________ >>> From: core-libs-dev on behalf >>> of Daniel Fuchs >>> Sent: Thursday, November 9, 2017 1:29 PM >>> To: mandy chung >>> Cc: core-libs-dev at openjdk.java.net >>> Subject: Re: Change in properties for logging: deliberate? >>> >>> On 09/11/2017 19:16, mandy chung wrote: >>> >>>> Daniel - we should add this known issue in the release note and document >>>> the workaround. >>>> >>> >>> Hi Mandy, >>> >>> Right, it either need to be fixed, or documented in the release >>> notes. Let me first have a look at the issue though. >>> >>> best regards, >>> >>> -- daniel >>> >>> >> > From stuart.marks at oracle.com Wed Dec 6 01:27:44 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 5 Dec 2017 17:27:44 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> Message-ID: <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> Revised webrev based on comments from Martin Buchholz and Paul Sandoz: http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.1/ I've done a bit of rewriting of the @apiNote sections to clarify the intended use of the various toArray() methods. s'marks From stuart.marks at oracle.com Wed Dec 6 01:56:52 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 5 Dec 2017 17:56:52 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <1134719756.620531.1512457427706.JavaMail.zimbra@u-pem.fr> Message-ID: > The signature of the proposed generator is > >> public T[] toArray(IntFunction generator) > > So I don't think that anything has really changed in this regard; T > can still be unrelated to E. Right, the new method doesn't offer any safety compared to toArray(T[]). You can still get ArrayStoreException if T is incompatible with E. That's in part what led me to update the @throws ArrayStoreException text, since the previous text was basically wrong. But fundamentally the problem is unchanged. There's a related enhancement request https://bugs.openjdk.java.net/browse/JDK-7023484 add typesafe static Collections.toArray(Collection, IntFunction) that would provide static type safety for this operation. Unclear whether it's worthwhile to add something like this, though. > That said, sending in a constant String[0] is probably just as good as > a generator, or better (in my naive estimation the tradeoff is a >0 > comparison versus an interface method dispatch), than an i -> new > String[i] unless some kind of benchmark would appear which disproves > that hypothesis. I did some quick benchmarking, and it looks like String[]::new is a shade faster (heh) than new String[size], but not quite as fast as new String[0]. But take these results with a grain of salt. I was doing the benchmarking on my laptop with my entire environment running, and I was also measuring my development fastdebug build. I wasn't able to see the same difference between new String[size] and new String[0] that Shipil?v observed.[1] It would be interesting to get some rigorous benchmarks to compare these, but I haven't yet taken the time to do this. I don't think the main point of this is performance, though. I think the new preferred way to create an array of a particular type should be toArray(Foo[]::new), which is preferable to toArray(new Foo[0]) or toArray(EMPTY_FOO_ARRAY). I suppose those of us "in the know" would recognize toArray(new Foo[0]) to be idiomatic. But to understand it, you have to fully understand the semantics of toArray(T[]), and you have to have read Shipil?v's article to understand why creating an apparently wasted zero-sized array is (usually) the right thing. That seems pretty obscure. s'marks [1] https://shipilev.net/blog/2016/arrays-wisdom-ancients/ From amaembo at gmail.com Wed Dec 6 02:54:25 2017 From: amaembo at gmail.com (Tagir Valeev) Date: Wed, 6 Dec 2017 09:54:25 +0700 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> Message-ID: Hello! I think, CopyOnWriteArrayList and a List returned by Arrays.asList deserve specialized implementation as well. With best regards, Tagir Valeev. On Wed, Dec 6, 2017 at 8:27 AM, Stuart Marks wrote: > Revised webrev based on comments from Martin Buchholz and Paul Sandoz: > > http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.1/ > > I've done a bit of rewriting of the @apiNote sections to clarify the > intended use of the various toArray() methods. > > s'marks > From joe.darcy at oracle.com Wed Dec 6 02:55:57 2017 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 05 Dec 2017 18:55:57 -0800 Subject: Fw: Question about JEP 306. In-Reply-To: <2a493b34-a717-852b-23a8-d8827be0f61e@oracle.com> References: <2a493b34-a717-852b-23a8-d8827be0f61e@oracle.com> Message-ID: <5A275C3D.4060208@oracle.com> Hello, On 12/5/2017 5:07 PM, David Holmes wrote: > Adding core-libs-dev as both mailing lists are named in this JEP. > > David > > On 6/12/2017 11:02 AM, A Z wrote: >> -I have been wondering what interest, focus or progress >> >> is happening around OpenJDK JEP 306: The rampdown phase for JDK 10 is beginning later this month. Since JEP 306 is not in the process of being targeted for JDK 10, it will not be part of that release. However, it may be included in a subsequent JDK release; once we have sufficient clarity of the work needed and commitment to do the work, we target a JEP to a particular release. >> >> >> http://openjdk.java.net/jeps/306 >> >> >> -As a Java feature request, could I request feature 306? >> Underflow and overflow need to be eliminated >> from double and float. That is outside of the stated description of what the JEP aims to do. IEEE 754 arithmetic has rules about underflow and overflow, rules also generally followed by the Java language and JVM. There are no plans to adopt a system like unums that avoid imprecise overflows to infinity by losing precision at values extremely large in magnitude. >> >> There could be an associativefp option. Such an option is explicitly called out as a non-goal for JEP 306: "Defining any sort of "fast-fp" or "loose-fp" (c.f. JSR 84: Floating Point Extensions) is out of scope for this JEP." However, such an option could be done under another project. With suitable caveats about forward looking statements, no promises, etc. the possibility of a fast-fp option along with some of the technical issues that would need to be addressed is discussed in the last few minutes of my JVMLS talk from earlier this year: "Forward to the Past: The Case for Uniformly Strict Floating Point Arithmetic on the JVM" https://www.youtube.com/watch?v=qTKeU_3rhk4 The bulk of the talk is a justification for JEP 306. >> It should also be the case >> that there should be a round half up for the decimal place >> beyond the last one in the float or double range, to help support >> all notions of >> >> (pow(sqrt(3),2) == 3) //true Putting aside non-finite values like NaN and infinities, this identity is difficult to have hold in any fixed-precision system, including IEEE-style floating-point. The mathematical square root function in general returns irrational results for rational inputs so the result is fundamentally approximated. If my rusty calculus is correct, for x >= 0.25, the derivative of sqrt of x has magnitude less than one and is positive, meaning there exists some set of multiple input values that must get mapped to the same output value. Therefore, the information to do an exact inverse operation based on the output is not present just from a pigeon hole principle argument. >> >> >> -The other thing that there should be is a StrictMath >> that works for BigInteger and BigDecimal, allowing: >> >> pow(BigDecimal,BigDecimal) >> pi(...) >> e(...) >> sin(Bigdecimal) >> asin(Bigdecimal) ... >> Such methods would have utility and we've had enhancements request for those features JDK-4297957: Add algebraic and transcendental functions operating on BigDecimal but we've closed them as will not fix. >> which would allow for any power at all, as well as the nth root. >> It is also a good idea to include a radians and degrees version >> of all the trig functions. See RFE JDK-6341878: add degree-based trigonometric methods to the Math class https://bugs.openjdk.java.net/browse/JDK-6341878 Cheers, -Joe From ecki at zusammenkunft.net Wed Dec 6 04:41:21 2017 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Wed, 6 Dec 2017 04:41:21 +0000 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> , <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> Message-ID: Should the test also check for the case the generator returns under- and oversized arrays? The default impl looks less efficient than (new T[0]), should it really be removed as a major Javadoc example? (Or maybe add more specific implementations besides ArrayList?) Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: core-libs-dev on behalf of Stuart Marks Sent: Wednesday, December 6, 2017 2:27:44 AM To: core-libs-dev Subject: Re: RFR(s): 8060192: Add default method Collection.toArray(generator) Revised webrev based on comments from Martin Buchholz and Paul Sandoz: http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.1/ I've done a bit of rewriting of the @apiNote sections to clarify the intended use of the various toArray() methods. s'marks From peter.levart at gmail.com Wed Dec 6 11:08:02 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Dec 2017 12:08:02 +0100 Subject: RFR 8191216: SimpleTimeZone.clone() has a data race on cache fields In-Reply-To: References: <53644ce5-7c7c-18ff-4da9-7dfacd5b678d@linux.vnet.ibm.com> <576cd10d-c75e-c7e2-ecb9-e28d09e059cf@linux.vnet.ibm.com> <84ca56e0-67c8-3fbb-d558-4ebe797d513c@oracle.com> <8894f619-3510-acfa-3987-f013ca3638f3@gmail.com> <72a7efe3-ac34-1534-3a9b-146c647e1292@linux.vnet.ibm.com> <017c44ce-a3e2-a03d-d437-98e08655a6a1@gmail.com> Message-ID: Hi David, Can I consider your comment as a Review? I'd like to get this patch into JDK10 if possible. Regards, Peter On 11/28/2017 08:17 AM, David Holmes wrote: > Hi Peter, > > I like what you have done here. That said the general > thread-unsafeness of the code in SimpleTimeZone still causes me > concern - but what you are doing is not breaking anything more than it > is already broken. > > David > > On 25/11/2017 9:32 AM, Peter Levart wrote: >> Hi, >> >> @Venkat: Sorry for late response, but I had to try something 1st. >> >> This is an official request for reviewing a patch for fixing a data >> race between cloning a SimpleTimeZone object and lazily initializing >> its 3 cache fields which may produce a clone with inconsistent cache >> state. Here's Jira issue with details: >> >> https://bugs.openjdk.java.net/browse/JDK-8191216 >> >> Venkat has proposed to simply call invalidateCache() on the clone >> before returning it from SimpleTimeZone.clone() method: >> >> ???? public Object clone() >> ???? { >> ???????? SimpleTimeZone clone = (SimpleTimeZone) super.clone(); >> ???????? clone.invalidateCache(); >> ???????? return clone; >> ???? } >> >> This fixes the issue and for the case of TimeZone.getDefault() which >> is called from ZoneId.systemDefault() even elides synchronization in >> clone.invalidateCache() and allocation of the clone, so JITed >> ZoneId.systemDefault() is unaffected by the patch. Initially I was >> satisfied with the fix, but then I tested something. Suppose someone >> sets default zone to SimpleTimeZone: >> >> ???????? TimeZone.setDefault( >> ???????????? new SimpleTimeZone(3600000, >> ??????????????????????????????? "Europe/Paris", >> ??????????????????????????????? Calendar.MARCH, -1, Calendar.SUNDAY, >> ??????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, >> ??????????????????????????????? Calendar.OCTOBER, -1, Calendar.SUNDAY, >> ??????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, >> ??????????????????????????????? 3600000) >> ???????? ); >> >> And then calls some methods that initialize the cache of the internal >> shared TimeZone.defaultTimeZone instance: >> >> ???? new Date().toString(); >> >> The code which after that tries to obtain default time zone and >> calculate the offset from UTC at some current point in time, for >> example: >> >> ???? TimeZone.getDefault().getOffset(now) >> >> can't use the cached state because it has been invalidated in the >> returned clone. Such code has to re-compute the offset every time it >> gets new clone. I measured this with a JMH benchmark and got the >> following result: >> >> Default: >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? >> ns/op >> >> Venkat's patch - invalidateCache(): >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 179,476 ? 1,942? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,538 ? 0,073? >> ns/op >> >> We see that ZoneId.systemDefault() is unaffected, but >> TimeZone.getDefault().getOffset(now) becomes 3x slower. >> >> This is not good, so I tried an alternative fix for the issue - >> simply making the SimpleTimeZone.clone() synchronized. Access to >> cache fields is already synchronized, so this should fix the race. >> Here's the JMH result: >> >> Default: >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? >> ns/op >> >> Synchronized clone(): >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 58,103 ? 0,936? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 4,154 ? 0,034? >> ns/op >> >> We see that caching works again, but synchronization has some >> overhead which is not big for single-threaded access, but might get >> bigger when multiple threads try to get default zone concurrently. >> >> So I created a 3rd variant of the fix which I'm presenting here and >> requesting a review for: >> >> http://cr.openjdk.java.net/~plevart/jdk10-dev/8191216_SimpleTimeZone_clone_race/webrev.01/ >> >> >> The JMH benchmark shows this: >> >> Default: >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? >> ns/op >> >> Cache object: >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 42,860 ? 0,274? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,545 ? 0,057? >> ns/op >> >> Not only does the fix not affect ZoneId.systemDefault() which is not >> surprising, but it also speeds-up cache lookup in single-threaded >> benchmark and certainly eliminates possible contention among threads >> looking up the shared instance. >> >> I have run the test/jdk/java/util/TimeZone and >> test/jdk/java/util/Calendar jtreg tests and there were 7 failures >> caused by compilation errors (package sun.util.locale.provider is not >> visible), while 59 other tests pass. >> >> So, what do you think? >> >> Regards, Peter >> >> >> Venkateswara R Chintala je 21. 11. 2017 ob 10:14?napisal: >>> Thanks Peter for sponsoring this patch. Is there anything else that >>> needs to be done from my end for this patch to be integrated? Please >>> let me know. >>> >>> -Venkat >>> >>> >>> On 14/11/17 8:46 PM, Peter Levart wrote: >>>> Hi Venkat, >>>> >>>> I created the following issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8191216 >>>> >>>> I can also sponsor this patch and push it for JDK 10. >>>> >>>> The patch that you are proposing looks good to me. Does anybody >>>> have anything else to say? >>>> >>>> Regards, Peter >>>> >>>> >>>> On 11/13/2017 11:28 AM, Venkateswara R Chintala wrote: >>>>> Thanks David, Peter for your review and comments. As I am new to >>>>> the community, can you please help me to open a bug and integrate >>>>> the changes into code base? >>>>> >>>>> -Venkat >>>>> >>>>> On 12/11/17 2:19 AM, Peter Levart wrote: >>>>>> Hi David, Venkat, >>>>>> >>>>>> On 11/11/17 21:15, Peter Levart wrote: >>>>>>> For example, take the following method: >>>>>>> >>>>>>> String defaultTZID() { >>>>>>> ??? return TimeZone.getDefault().getID(); >>>>>>> } >>>>>>> >>>>>>> When JIT compiles it and inlines invocations to other methods >>>>>>> within it, it can prove that cloned TimeZone instance never >>>>>>> escapes the call to defaultTZID() and can therefore skip >>>>>>> allocating the instance on heap. >>>>>>> >>>>>>> But this is fragile. If code in invoked methods changes, they >>>>>>> may not get inlined or EA may not be able to prove that the >>>>>>> cloned instance can't escape and allocation may be introduced. >>>>>>> ZoneId.systemDefault() is a hot method and it would be nice if >>>>>>> we manage to keep it allocation free. >>>>>> >>>>>> Well, I tried the following variant of SimpleTimeZone.clone() patch: >>>>>> >>>>>> ??? public Object clone() >>>>>> ??? { >>>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>>> ??????? // like tz.invalidateCache() but without holding a lock >>>>>> on clone >>>>>> ??????? tz.cacheYear = tz.startYear - 1; >>>>>> ??????? tz.cacheStart = tz.cacheEnd = 0; >>>>>> ??????? return tz; >>>>>> ??? } >>>>>> >>>>>> >>>>>> ...and the JMH benchmark with gc profiling shows that >>>>>> ZoneId.systemDefault() still manages to get JIT-compiled without >>>>>> introducing allocation. >>>>>> >>>>>> Even the following (original Venkat's) patch: >>>>>> >>>>>> ??? public Object clone() >>>>>> ??? { >>>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>>> ??????? tz.invalidateCache(); >>>>>> ??????? return tz; >>>>>> ??? } >>>>>> >>>>>> ...does the same and the locking in invalidateCache() is elided >>>>>> too. Allocation and lock-elision go hand-in-hand. When object >>>>>> doesn't escape, allocation on heap may be eliminated and locks on >>>>>> that instance elided. >>>>>> >>>>>> So this is better than synchronizing on the original instance >>>>>> during .clone() execution as it has potential to avoid locking >>>>>> overhead. >>>>>> >>>>>> So Venkat, go ahead. My fear was unjustified. >>>>>> >>>>>> Regards, Peter >>>>>> >>>>> >>>> >>>> >>> >> From david.holmes at oracle.com Wed Dec 6 11:32:55 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Dec 2017 21:32:55 +1000 Subject: RFR 8191216: SimpleTimeZone.clone() has a data race on cache fields In-Reply-To: References: <53644ce5-7c7c-18ff-4da9-7dfacd5b678d@linux.vnet.ibm.com> <576cd10d-c75e-c7e2-ecb9-e28d09e059cf@linux.vnet.ibm.com> <84ca56e0-67c8-3fbb-d558-4ebe797d513c@oracle.com> <8894f619-3510-acfa-3987-f013ca3638f3@gmail.com> <72a7efe3-ac34-1534-3a9b-146c647e1292@linux.vnet.ibm.com> <017c44ce-a3e2-a03d-d437-98e08655a6a1@gmail.com> Message-ID: <3bf0ce49-cb3d-f70b-717d-03680b001d3b@oracle.com> Hi Peter, On 6/12/2017 9:08 PM, Peter Levart wrote: > Hi David, > > Can I consider your comment as a Review? I'd like to get this patch into > JDK10 if possible. No sorry. I see what you're doing and I think it is okay but the regular owners/maintainers of this code need to have the say on any changes here. David > Regards, Peter > > > On 11/28/2017 08:17 AM, David Holmes wrote: >> Hi Peter, >> >> I like what you have done here. That said the general >> thread-unsafeness of the code in SimpleTimeZone still causes me >> concern - but what you are doing is not breaking anything more than it >> is already broken. >> >> David >> >> On 25/11/2017 9:32 AM, Peter Levart wrote: >>> Hi, >>> >>> @Venkat: Sorry for late response, but I had to try something 1st. >>> >>> This is an official request for reviewing a patch for fixing a data >>> race between cloning a SimpleTimeZone object and lazily initializing >>> its 3 cache fields which may produce a clone with inconsistent cache >>> state. Here's Jira issue with details: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8191216 >>> >>> Venkat has proposed to simply call invalidateCache() on the clone >>> before returning it from SimpleTimeZone.clone() method: >>> >>> ???? public Object clone() >>> ???? { >>> ???????? SimpleTimeZone clone = (SimpleTimeZone) super.clone(); >>> ???????? clone.invalidateCache(); >>> ???????? return clone; >>> ???? } >>> >>> This fixes the issue and for the case of TimeZone.getDefault() which >>> is called from ZoneId.systemDefault() even elides synchronization in >>> clone.invalidateCache() and allocation of the clone, so JITed >>> ZoneId.systemDefault() is unaffected by the patch. Initially I was >>> satisfied with the fix, but then I tested something. Suppose someone >>> sets default zone to SimpleTimeZone: >>> >>> ???????? TimeZone.setDefault( >>> ???????????? new SimpleTimeZone(3600000, >>> ??????????????????????????????? "Europe/Paris", >>> ??????????????????????????????? Calendar.MARCH, -1, Calendar.SUNDAY, >>> ??????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, >>> ??????????????????????????????? Calendar.OCTOBER, -1, Calendar.SUNDAY, >>> ??????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, >>> ??????????????????????????????? 3600000) >>> ???????? ); >>> >>> And then calls some methods that initialize the cache of the internal >>> shared TimeZone.defaultTimeZone instance: >>> >>> ???? new Date().toString(); >>> >>> The code which after that tries to obtain default time zone and >>> calculate the offset from UTC at some current point in time, for >>> example: >>> >>> ???? TimeZone.getDefault().getOffset(now) >>> >>> can't use the cached state because it has been invalidated in the >>> returned clone. Such code has to re-compute the offset every time it >>> gets new clone. I measured this with a JMH benchmark and got the >>> following result: >>> >>> Default: >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040 ns/op >>> >>> Venkat's patch - invalidateCache(): >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 179,476 ? 1,942 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,538 ? 0,073 ns/op >>> >>> We see that ZoneId.systemDefault() is unaffected, but >>> TimeZone.getDefault().getOffset(now) becomes 3x slower. >>> >>> This is not good, so I tried an alternative fix for the issue - >>> simply making the SimpleTimeZone.clone() synchronized. Access to >>> cache fields is already synchronized, so this should fix the race. >>> Here's the JMH result: >>> >>> Default: >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040 ns/op >>> >>> Synchronized clone(): >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 58,103 ? 0,936 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 4,154 ? 0,034 ns/op >>> >>> We see that caching works again, but synchronization has some >>> overhead which is not big for single-threaded access, but might get >>> bigger when multiple threads try to get default zone concurrently. >>> >>> So I created a 3rd variant of the fix which I'm presenting here and >>> requesting a review for: >>> >>> http://cr.openjdk.java.net/~plevart/jdk10-dev/8191216_SimpleTimeZone_clone_race/webrev.01/ >>> >>> >>> The JMH benchmark shows this: >>> >>> Default: >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040 ns/op >>> >>> Cache object: >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 42,860 ? 0,274 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,545 ? 0,057 ns/op >>> >>> Not only does the fix not affect ZoneId.systemDefault() which is not >>> surprising, but it also speeds-up cache lookup in single-threaded >>> benchmark and certainly eliminates possible contention among threads >>> looking up the shared instance. >>> >>> I have run the test/jdk/java/util/TimeZone and >>> test/jdk/java/util/Calendar jtreg tests and there were 7 failures >>> caused by compilation errors (package sun.util.locale.provider is not >>> visible), while 59 other tests pass. >>> >>> So, what do you think? >>> >>> Regards, Peter >>> >>> >>> Venkateswara R Chintala je 21. 11. 2017 ob 10:14?napisal: >>>> Thanks Peter for sponsoring this patch. Is there anything else that >>>> needs to be done from my end for this patch to be integrated? Please >>>> let me know. >>>> >>>> -Venkat >>>> >>>> >>>> On 14/11/17 8:46 PM, Peter Levart wrote: >>>>> Hi Venkat, >>>>> >>>>> I created the following issue: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8191216 >>>>> >>>>> I can also sponsor this patch and push it for JDK 10. >>>>> >>>>> The patch that you are proposing looks good to me. Does anybody >>>>> have anything else to say? >>>>> >>>>> Regards, Peter >>>>> >>>>> >>>>> On 11/13/2017 11:28 AM, Venkateswara R Chintala wrote: >>>>>> Thanks David, Peter for your review and comments. As I am new to >>>>>> the community, can you please help me to open a bug and integrate >>>>>> the changes into code base? >>>>>> >>>>>> -Venkat >>>>>> >>>>>> On 12/11/17 2:19 AM, Peter Levart wrote: >>>>>>> Hi David, Venkat, >>>>>>> >>>>>>> On 11/11/17 21:15, Peter Levart wrote: >>>>>>>> For example, take the following method: >>>>>>>> >>>>>>>> String defaultTZID() { >>>>>>>> ??? return TimeZone.getDefault().getID(); >>>>>>>> } >>>>>>>> >>>>>>>> When JIT compiles it and inlines invocations to other methods >>>>>>>> within it, it can prove that cloned TimeZone instance never >>>>>>>> escapes the call to defaultTZID() and can therefore skip >>>>>>>> allocating the instance on heap. >>>>>>>> >>>>>>>> But this is fragile. If code in invoked methods changes, they >>>>>>>> may not get inlined or EA may not be able to prove that the >>>>>>>> cloned instance can't escape and allocation may be introduced. >>>>>>>> ZoneId.systemDefault() is a hot method and it would be nice if >>>>>>>> we manage to keep it allocation free. >>>>>>> >>>>>>> Well, I tried the following variant of SimpleTimeZone.clone() patch: >>>>>>> >>>>>>> ??? public Object clone() >>>>>>> ??? { >>>>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>>>> ??????? // like tz.invalidateCache() but without holding a lock >>>>>>> on clone >>>>>>> ??????? tz.cacheYear = tz.startYear - 1; >>>>>>> ??????? tz.cacheStart = tz.cacheEnd = 0; >>>>>>> ??????? return tz; >>>>>>> ??? } >>>>>>> >>>>>>> >>>>>>> ...and the JMH benchmark with gc profiling shows that >>>>>>> ZoneId.systemDefault() still manages to get JIT-compiled without >>>>>>> introducing allocation. >>>>>>> >>>>>>> Even the following (original Venkat's) patch: >>>>>>> >>>>>>> ??? public Object clone() >>>>>>> ??? { >>>>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>>>> ??????? tz.invalidateCache(); >>>>>>> ??????? return tz; >>>>>>> ??? } >>>>>>> >>>>>>> ...does the same and the locking in invalidateCache() is elided >>>>>>> too. Allocation and lock-elision go hand-in-hand. When object >>>>>>> doesn't escape, allocation on heap may be eliminated and locks on >>>>>>> that instance elided. >>>>>>> >>>>>>> So this is better than synchronizing on the original instance >>>>>>> during .clone() execution as it has potential to avoid locking >>>>>>> overhead. >>>>>>> >>>>>>> So Venkat, go ahead. My fear was unjustified. >>>>>>> >>>>>>> Regards, Peter >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> > From adam.farley at uk.ibm.com Wed Dec 6 11:53:39 2017 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 6 Dec 2017 11:53:39 +0000 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java In-Reply-To: References: Message-ID: Hi All, We have a bug in OpenJDK where if you pass an info-only option (like -agentlib:jdwp=help) in through the JNI interface, it can exit your code with RC 0. I think this is a bug because if you planned to do anything after starting a Java VM, you have to do it in an exit hook. If an info-only option is used, exit(0) should not happen. Instead, the user's code should be told the VM cannot be used. Bug Link: https://bugs.openjdk.java.net/browse/JDK-8190187 I have proposed the creation of a new JNI return code indicating a "silent exit", where the vm encountered no error, but that it is not in a fit state to be used. See the link for an implementation of this, along with a handy test. In short, if you agree that this is a problem, we need a champion to: 1) Complete the CSR process for the new JNI Return code. 2) Commit the changes that contain (a) the new return code, and (b) the non-Hotspot code that handles the new code. Optionally, if you want to commit the code containing an example of the fix, you will need to: 3) Commit the Hotspot code that channels the new return code. 4) Complete the CSR process for the jdwp behaviour change. 5) Commit the jdwp code. Note that all of this code has *already been written*. This should be an easy commit once the CSR is done with. Best Regards Adam Farley Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Alan.Bateman at oracle.com Wed Dec 6 13:30:15 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Dec 2017 13:30:15 +0000 Subject: RFR 8191216: SimpleTimeZone.clone() has a data race on cache fields In-Reply-To: <3bf0ce49-cb3d-f70b-717d-03680b001d3b@oracle.com> References: <53644ce5-7c7c-18ff-4da9-7dfacd5b678d@linux.vnet.ibm.com> <576cd10d-c75e-c7e2-ecb9-e28d09e059cf@linux.vnet.ibm.com> <84ca56e0-67c8-3fbb-d558-4ebe797d513c@oracle.com> <8894f619-3510-acfa-3987-f013ca3638f3@gmail.com> <72a7efe3-ac34-1534-3a9b-146c647e1292@linux.vnet.ibm.com> <017c44ce-a3e2-a03d-d437-98e08655a6a1@gmail.com> <3bf0ce49-cb3d-f70b-717d-03680b001d3b@oracle.com> Message-ID: <9dbdf9bc-e93a-6ac4-d3ae-5168fd365403@oracle.com> On 06/12/2017 11:32, David Holmes wrote: > Hi Peter, > > On 6/12/2017 9:08 PM, Peter Levart wrote: >> Hi David, >> >> Can I consider your comment as a Review? I'd like to get this patch >> into JDK10 if possible. > > No sorry. I see what you're doing and I think it is okay but the > regular owners/maintainers of this code need to have the say on any > changes here. I think this class is normally maintained on i18n-dev but I think introducing the Cache object looks good and making this much easier to understand. -Alan From peter.levart at gmail.com Wed Dec 6 14:10:04 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Dec 2017 15:10:04 +0100 Subject: RFR 8191216: SimpleTimeZone.clone() has a data race on cache fields In-Reply-To: <9dbdf9bc-e93a-6ac4-d3ae-5168fd365403@oracle.com> References: <53644ce5-7c7c-18ff-4da9-7dfacd5b678d@linux.vnet.ibm.com> <576cd10d-c75e-c7e2-ecb9-e28d09e059cf@linux.vnet.ibm.com> <84ca56e0-67c8-3fbb-d558-4ebe797d513c@oracle.com> <8894f619-3510-acfa-3987-f013ca3638f3@gmail.com> <72a7efe3-ac34-1534-3a9b-146c647e1292@linux.vnet.ibm.com> <017c44ce-a3e2-a03d-d437-98e08655a6a1@gmail.com> <3bf0ce49-cb3d-f70b-717d-03680b001d3b@oracle.com> <9dbdf9bc-e93a-6ac4-d3ae-5168fd365403@oracle.com> Message-ID: Hi, On 12/06/2017 02:30 PM, Alan Bateman wrote: > I think this class is normally maintained on i18n-dev but I think > introducing the Cache object looks good and making this much easier to > understand. > > -Alan Thanks Alan, I'm forwarding to i18n-dev to see if maintainers of that part of JDK have any comments... This is an official request for reviewing a patch for fixing a data race between cloning a SimpleTimeZone object and lazily initializing its 3 cache fields which may produce a clone with inconsistent cache state. Here's Jira issue with details: https://bugs.openjdk.java.net/browse/JDK-8191216 Venkat has proposed to simply call invalidateCache() on the clone before returning it from SimpleTimeZone.clone() method: ??? public Object clone() ??? { ??????? SimpleTimeZone clone = (SimpleTimeZone) super.clone(); ??????? clone.invalidateCache(); ??????? return clone; ??? } This fixes the issue and for the case of TimeZone.getDefault() which is called from ZoneId.systemDefault() even elides synchronization in clone.invalidateCache() and allocation of the clone, so JITed ZoneId.systemDefault() is unaffected by the patch. Initially I was satisfied with the fix, but then I tested something. Suppose someone sets default zone to SimpleTimeZone: ??????? TimeZone.setDefault( ??????????? new SimpleTimeZone(3600000, ?????????????????????????????? "Europe/Paris", ?????????????????????????????? Calendar.MARCH, -1, Calendar.SUNDAY, ?????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, ?????????????????????????????? Calendar.OCTOBER, -1, Calendar.SUNDAY, ?????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, ?????????????????????????????? 3600000) ??????? ); And then calls some methods that initialize the cache of the internal shared TimeZone.defaultTimeZone instance: ??? new Date().toString(); The code which after that tries to obtain default time zone and calculate the offset from UTC at some current point in time, for example: ??? TimeZone.getDefault().getOffset(now) can't use the cached state because it has been invalidated in the returned clone. Such code has to re-compute the offset every time it gets new clone. I measured this with a JMH benchmark and got the following result: Default: Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? ns/op ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? ns/op Venkat's patch - invalidateCache(): Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 179,476 ? 1,942? ns/op ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,538 ? 0,073? ns/op We see that ZoneId.systemDefault() is unaffected, but TimeZone.getDefault().getOffset(now) becomes 3x slower. This is not good, so I tried an alternative fix for the issue - simply making the SimpleTimeZone.clone() synchronized. Access to cache fields is already synchronized, so this should fix the race. Here's the JMH result: Default: Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? ns/op ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? ns/op Synchronized clone(): Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 58,103 ? 0,936? ns/op ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 4,154 ? 0,034? ns/op We see that caching works again, but synchronization has some overhead which is not big for single-threaded access, but might get bigger when multiple threads try to get default zone concurrently. So I created a 3rd variant of the fix which I'm presenting here and requesting a review for: http://cr.openjdk.java.net/~plevart/jdk10-dev/8191216_SimpleTimeZone_clone_race/webrev.01/ The JMH benchmark shows this: Default: Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? ns/op ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? ns/op Cache object: Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 42,860 ? 0,274? ns/op ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,545 ? 0,057? ns/op Not only does the fix not affect ZoneId.systemDefault() which is not surprising, but it also speeds-up cache lookup in single-threaded benchmark and certainly eliminates possible contention among threads looking up the shared instance. I have run the test/jdk/java/util/TimeZone and test/jdk/java/util/Calendar jtreg tests and there were 7 failures caused by compilation errors (package sun.util.locale.provider is not visible), while 59 other tests pass. So, what do you think? Regards, Peter Venkateswara R Chintala je 21. 11. 2017 ob 10:14?napisal: > Thanks Peter for sponsoring this patch. Is there anything else that > needs to be done from my end for this patch to be integrated? Please > let me know. > > -Venkat > > > On 14/11/17 8:46 PM, Peter Levart wrote: >> Hi Venkat, >> >> I created the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8191216 >> >> I can also sponsor this patch and push it for JDK 10. >> >> The patch that you are proposing looks good to me. Does anybody have >> anything else to say? >> >> Regards, Peter >> >> >> On 11/13/2017 11:28 AM, Venkateswara R Chintala wrote: >>> Thanks David, Peter for your review and comments. As I am new to the >>> community, can you please help me to open a bug and integrate the >>> changes into code base? >>> >>> -Venkat >>> >>> On 12/11/17 2:19 AM, Peter Levart wrote: >>>> Hi David, Venkat, >>>> >>>> On 11/11/17 21:15, Peter Levart wrote: >>>>> For example, take the following method: >>>>> >>>>> String defaultTZID() { >>>>> ??? return TimeZone.getDefault().getID(); >>>>> } >>>>> >>>>> When JIT compiles it and inlines invocations to other methods >>>>> within it, it can prove that cloned TimeZone instance never >>>>> escapes the call to defaultTZID() and can therefore skip >>>>> allocating the instance on heap. >>>>> >>>>> But this is fragile. If code in invoked methods changes, they may >>>>> not get inlined or EA may not be able to prove that the cloned >>>>> instance can't escape and allocation may be introduced. >>>>> ZoneId.systemDefault() is a hot method and it would be nice if we >>>>> manage to keep it allocation free. >>>> >>>> Well, I tried the following variant of SimpleTimeZone.clone() patch: >>>> >>>> ??? public Object clone() >>>> ??? { >>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>> ??????? // like tz.invalidateCache() but without holding a lock on >>>> clone >>>> ??????? tz.cacheYear = tz.startYear - 1; >>>> ??????? tz.cacheStart = tz.cacheEnd = 0; >>>> ??????? return tz; >>>> ??? } >>>> >>>> >>>> ...and the JMH benchmark with gc profiling shows that >>>> ZoneId.systemDefault() still manages to get JIT-compiled without >>>> introducing allocation. >>>> >>>> Even the following (original Venkat's) patch: >>>> >>>> ??? public Object clone() >>>> ??? { >>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>> ??????? tz.invalidateCache(); >>>> ??????? return tz; >>>> ??? } >>>> >>>> ...does the same and the locking in invalidateCache() is elided >>>> too. Allocation and lock-elision go hand-in-hand. When object >>>> doesn't escape, allocation on heap may be eliminated and locks on >>>> that instance elided. >>>> >>>> So this is better than synchronizing on the original instance >>>> during .clone() execution as it has potential to avoid locking >>>> overhead. >>>> >>>> So Venkat, go ahead. My fear was unjustified. >>>> >>>> Regards, Peter >>>> >>> >> >> > From adam.farley at uk.ibm.com Wed Dec 6 16:38:19 2017 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 6 Dec 2017 16:38:19 +0000 Subject: Proposal for New Functionality: Allow module-info merging in GenModuleInfoSource.java Message-ID: Hi All, Currently, GenModuleInfoSource.java does not allow you to merge extra module-info files into the primary module-info file (for a given module) at build time. Put simply; I think it should have this functionality. Can committers please review and opine? You can already see this code change in action while compiling OpenJDK with OpenJ9, so we know it works. I've included the "diff -u" output for the proposed code change in this email's P.S. Best Regards Adam Farley P.S. Code diff: --- original_GenModuleInfoSource.java 2017-12-06 15:46:43.000000000 +0000 +++ improved_GenModuleInfoSource.java 2017-12-06 15:49:04.000000000 +0000 @@ -1,4 +1,10 @@ /* + * =========================================================================== + * (c) Copyright IBM Corp. 2015, 2017 All Rights Reserved + * =========================================================================== + */ + +/* * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * @@ -156,12 +162,12 @@ } // requires - for (String l : lines) { - if (l.trim().startsWith("requires")) - writer.println(l); - } + //for (String l : lines) { + // if (l.trim().startsWith("requires")) + // writer.println(l); + //} - // write exports, opens, uses, and provides + // write requires, exports, opens, uses, and provides moduleInfo.print(writer); // close @@ -171,12 +177,13 @@ class ModuleInfo { + final Map requires = new HashMap<>(); final Map exports = new HashMap<>(); final Map opens = new HashMap<>(); final Map uses = new HashMap<>(); final Map provides = new HashMap<>(); - Statement getStatement(String directive, String name) { + Statement getStatement(String directive, String qualifier, String name) { switch (directive) { case "exports": if (moduleInfo.exports.containsKey(name) && @@ -205,6 +212,10 @@ return uses.computeIfAbsent(name, _n -> new Statement("uses", "", name)); + case "requires": + return uses.computeIfAbsent(name, + _n -> new Statement("requires", qualifier, name)); + case "provides": return provides.computeIfAbsent(name, _n -> new Statement("provides", "with", name, true)); @@ -233,7 +244,7 @@ .stream() .filter(e -> !exports.containsKey(e.getKey()) && e.getValue().filter(modules)) - .forEach(e -> addTargets(getStatement("exports", e.getKey()), + .forEach(e -> addTargets(getStatement("exports", null, e.getKey()), e.getValue(), modules)); @@ -251,7 +262,7 @@ .stream() .filter(e -> !opens.containsKey(e.getKey()) && e.getValue().filter(modules)) - .forEach(e -> addTargets(getStatement("opens", e.getKey()), + .forEach(e -> addTargets(getStatement("opens", null, e.getKey()), e.getValue(), modules)); @@ -272,6 +283,12 @@ .stream() .filter(service -> !uses.containsKey(service)) .forEach(service -> uses.put(service, extraFiles.uses.get(service))); + + // requires + extraFiles.requires.keySet() + .stream() + .filter(require -> !requires.containsKey(require)) + .forEach(require -> requires.put(require, extraFiles.requires.get(require))); } // add qualified exports or opens to known modules only @@ -324,6 +341,11 @@ void print(PrintWriter writer) { + // requires + requires.entrySet().stream() + .sorted(Map.Entry.comparingByKey()) + .forEach(e -> writer.println(e.getValue())); + // print unqualified exports exports.entrySet().stream() .filter(e -> e.getValue().targets.isEmpty()) @@ -418,27 +440,45 @@ String keyword = s[0].trim(); String name = s.length > 1 ? s[1].trim() : null; + String requiresModifiedName1 = s.length > 2 ? s[2].trim() : null; + String requiresModifiedName2 = s.length > 3 ? s[3].trim() : null; trace("%d: %s index=%d len=%d%n", lineNumber, l, index, l.length()); switch (keyword) { case "module": - case "requires": case "}": index = l.length(); // skip to the end continue; + case "requires": case "exports": case "opens": case "provides": case "uses": + String requiresQualifier = null; // assume name immediately after exports, opens, provides, uses - statement = getStatement(keyword, name); + if (keyword.equals("requires")) { + if (requiresModifiedName2 != null) { // 2 modifiers specified + if ((name.equals("transitive") || name.equals("static")) && + (requiresModifiedName1.equals("transitive") || requiresModifiedName1.equals("static"))) { + requiresQualifier = "static transitive"; + name = requiresModifiedName2; + } + } else if (requiresModifiedName1 != null) { // 1 modifier specified + if (name.equals("transitive") || name.equals("static")) { + requiresQualifier = name; + name = requiresModifiedName1; + } + } + } + + statement = getStatement(keyword, requiresQualifier, name); hasTargets = false; int i = l.indexOf(name, keyword.length()+1) + name.length() + 1; l = i < l.length() ? l.substring(i, l.length()).trim() : ""; index = 0; - if (s.length >= 3) { + if (!keyword.equals("requires") && s.length >= 3) { if (!s[2].trim().equals(statement.qualifier)) { throw new RuntimeException(sourcefile + ", line " + lineNumber + ", is malformed: " + s[2]); @@ -594,17 +634,25 @@ @Override public String toString() { StringBuilder sb = new StringBuilder(" "); - sb.append(directive).append(" ").append(name); - if (targets.isEmpty()) { - sb.append(";"); - } else if (targets.size() == 1) { - sb.append(" ").append(qualifier) - .append(orderedTargets().collect(joining(",", " ", ";"))); + if (directive.equals("requires")) { + if (qualifier != null) { + sb.append(directive).append(" ").append(qualifier).append(" ").append(name).append(";"); + } else { + sb.append(directive).append(" ").append(name).append(";"); + } } else { - sb.append(" ").append(qualifier) - .append(orderedTargets() - .map(target -> String.format(" %s", target)) - .collect(joining(",\n", "\n", ";"))); + sb.append(directive).append(" ").append(name); + if (targets.isEmpty()) { + sb.append(";"); + } else if (targets.size() == 1) { + sb.append(" ").append(qualifier) + .append(orderedTargets().collect(joining(",", " ", ";"))); + } else { + sb.append(" ").append(qualifier) + .append(orderedTargets() + .map(target -> String.format(" %s", target)) + .collect(joining(",\n", "\n", ";"))); + } } return sb.toString(); } @@ -620,4 +668,4 @@ System.out.format(fmt, params); } } -} +} \ No newline at end of file Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From vyom.tewari at oracle.com Wed Dec 6 16:44:58 2017 From: vyom.tewari at oracle.com (vyom tewari) Date: Wed, 6 Dec 2017 22:14:58 +0530 Subject: [RFR] 8160768: Add capability to custom resolve host/domain names within the default JDNI LDAP provider In-Reply-To: <20171205174824.GA3515@vimes> References: <20171205174824.GA3515@vimes> Message-ID: Hi Rob, Please find below comments. DefaultLdapDnsProvider.java ?line 35: ??? convert it to for-each code will be more readable. LdapDnsProviderService.java ?line 21: constant name does not follow Java naming convention, there are other places as well please fix it. getInstance() is already synchronized why you need another "synchronized" block ? Line 70: Please use result.getEndpoints().isEmpty() in place of result.getEndpoints().size() == 0 LdapDnsProvider.java constructor LdapDnsProvider() calls LdapDnsProvider(Void unused) which does nothing, can you explain why LdapDnsProvider()? calls LdapDnsProvider(Void unused) ? LdapDnsProviderResult.java ? Private field? domainName & endpoints both can be final. LdapDnsProviderTest.java Fix the tag Order it is not correct. correct order is as follows. /** ?* @test ?* @bug 8160768 ?* @summary ctx provider tests for ldap ?* @modules java.naming/com.sun.jndi.ldap ?* @compile dnsprovider/TestDnsProvider.java ?* @run main/othervm LdapDnsProviderTest ?* @run main/othervm LdapDnsProviderTest nosm ?* @run main/othervm LdapDnsProviderTest smnodns ?* @run main/othervm LdapDnsProviderTest smdns ?*/ Line 82,83 you don't need System.out.println(e); e.printStackTrace(); Line 70: you don't need this try block Line 96 : constant name does not follow Java naming convention Line 105: you can use try-with-resources this will avoid some boilerplate. Thanks, Vyom On Tuesday 05 December 2017 11:18 PM, Rob McKenna wrote: > As this enhancement is limited to JDK10 this frees us up to explore a > different approach. > > http://cr.openjdk.java.net/~robm/8160768/webrev.06/ > > In this webrev we're using the Service Provider Interface to load an > implementation of the new LdapDnsProvider class. Implementations of this > class are responsible for resolving DNS endpoints for a given ldap url > should the default implementation be insufficient in a particular > environment. > > Note: if a security manager is installed the ldapDnsProvider > RuntimePermission must be granted. > > -Rob > From rob.mckenna at oracle.com Wed Dec 6 18:24:34 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Wed, 6 Dec 2017 18:24:34 +0000 Subject: [RFR] 8160768: Add capability to custom resolve host/domain names within the default JDNI LDAP provider In-Reply-To: References: <20171205174824.GA3515@vimes> Message-ID: <20171206182434.GB4409@vimes> Thanks Vyom, these should be fixed in: http://cr.openjdk.java.net/~robm/8160768/webrev.07/ Comments inline: On 06/12/17 22:14, vyom tewari wrote: > Hi Rob, > > Please find below comments. > > DefaultLdapDnsProvider.java > > ?line 35: ??? convert it to for-each code will be more readable. > > LdapDnsProviderService.java > > ?line 21: constant name does not follow Java naming convention, there are > other places as well please fix it. > > getInstance() is already synchronized why you need another "synchronized" > block ? Ah, neglected to remove the outer synchronized keyword, thanks. > > Line 70: Please use result.getEndpoints().isEmpty() in place of > result.getEndpoints().size() == 0 > > LdapDnsProvider.java > > constructor LdapDnsProvider() calls LdapDnsProvider(Void unused) which does > nothing, can you explain why LdapDnsProvider()? calls LdapDnsProvider(Void > unused) ? I've copied this pattern from the System.LoggerFinder api and I had forgotten what it was for too: http://www.oracle.com/technetwork/java/seccodeguide-139067.html#7 Section 7.3. > > LdapDnsProviderResult.java > > ? Private field? domainName & endpoints both can be final. > > LdapDnsProviderTest.java > > Fix the tag Order it is not correct. correct order is as follows. > > /** > ?* @test > ?* @bug 8160768 > ?* @summary ctx provider tests for ldap > ?* @modules java.naming/com.sun.jndi.ldap > ?* @compile dnsprovider/TestDnsProvider.java > ?* @run main/othervm LdapDnsProviderTest > ?* @run main/othervm LdapDnsProviderTest nosm > ?* @run main/othervm LdapDnsProviderTest smnodns > ?* @run main/othervm LdapDnsProviderTest smdns > ?*/ > > Line 82,83 you don't need System.out.println(e); e.printStackTrace(); I'm going to leave these in as I've had a few failing tests in the past where I've needed to push diagnostic changes like this to determine the root cause. -Rob > > Line 70: you don't need this try block > > Line 96 : constant name does not follow Java naming convention > > Line 105: you can use try-with-resources this will avoid some boilerplate. > > Thanks, > Vyom > > On Tuesday 05 December 2017 11:18 PM, Rob McKenna wrote: > >As this enhancement is limited to JDK10 this frees us up to explore a > >different approach. > > > >http://cr.openjdk.java.net/~robm/8160768/webrev.06/ > > > >In this webrev we're using the Service Provider Interface to load an > >implementation of the new LdapDnsProvider class. Implementations of this > >class are responsible for resolving DNS endpoints for a given ldap url > >should the default implementation be insufficient in a particular > >environment. > > > >Note: if a security manager is installed the ldapDnsProvider > >RuntimePermission must be granted. > > > > -Rob > > > From brian.burkhalter at oracle.com Wed Dec 6 19:00:40 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Dec 2017 11:00:40 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream Message-ID: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> https://bugs.openjdk.java.net/browse/JDK-4358774 http://cr.openjdk.java.net/~bpb/4358774/webrev.00/ Add nullStream() method to each of InputStream and OutputStream. Thanks, Brian From jonathan.gibbons at oracle.com Wed Dec 6 19:11:41 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 6 Dec 2017 11:11:41 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> Message-ID: "null" is a significant term in the Java ecosystem, and the relationship here, to /dev/null or NUL seems somewhat tenuous. Have any other names been considered?? At least for the InputStream, calling it an "empty stream" seems more intuitive than a "null stream". -- Jon On 12/6/17 11:00 AM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-4358774 > http://cr.openjdk.java.net/~bpb/4358774/webrev.00/ > > Add nullStream() method to each of InputStream and OutputStream. > > Thanks, > > Brian From brian.burkhalter at oracle.com Wed Dec 6 19:28:10 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Dec 2017 11:28:10 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> Message-ID: <6C0A7D9C-CE3D-4B9C-8E30-E54AC0B0CD88@oracle.com> The name ?emptyStream()? was considered for InputStream and ?discardingStream()? for OutputStream. It was thought that ?null? or ?empty? would be more likely to be found by developers due to familiarity. FWIW there is precedent in third party libraries for the ?null? names. Brian On Dec 6, 2017, at 11:11 AM, Jonathan Gibbons wrote: > "null" is a significant term in the Java ecosystem, and the relationship here, to /dev/null or NUL seems somewhat tenuous. > > Have any other names been considered? At least for the InputStream, calling it an "empty stream" seems more intuitive than a "null stream". From stuart.marks at oracle.com Wed Dec 6 19:39:11 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Dec 2017 11:39:11 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> Message-ID: <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> On 12/5/17 6:54 PM, Tagir Valeev wrote: > I think, CopyOnWriteArrayList and a List returned by Arrays.asList > deserve specialized implementation as well. Sure, I can add one to Array.asList right away (as part of this changeset). It's even covered by tests already. I'll work with Martin to update COWAL. Depending on how he wants to handle it, this might be done separately. s'marks From naoto.sato at oracle.com Wed Dec 6 19:41:41 2017 From: naoto.sato at oracle.com (Naoto Sato) Date: Wed, 6 Dec 2017 11:41:41 -0800 Subject: RFR 8191216: SimpleTimeZone.clone() has a data race on cache fields In-Reply-To: References: <53644ce5-7c7c-18ff-4da9-7dfacd5b678d@linux.vnet.ibm.com> <576cd10d-c75e-c7e2-ecb9-e28d09e059cf@linux.vnet.ibm.com> <84ca56e0-67c8-3fbb-d558-4ebe797d513c@oracle.com> <8894f619-3510-acfa-3987-f013ca3638f3@gmail.com> <72a7efe3-ac34-1534-3a9b-146c647e1292@linux.vnet.ibm.com> <017c44ce-a3e2-a03d-d437-98e08655a6a1@gmail.com> <3bf0ce49-cb3d-f70b-717d-03680b001d3b@oracle.com> <9dbdf9bc-e93a-6ac4-d3ae-5168fd365403@oracle.com> Message-ID: <86b95ce3-2acd-8fda-77cf-222f957ac687@oracle.com> Hi Peter, Venkat, Thank you for the fix. It looks good to me. Improved performance is a nice bonus! Would you be able to provide with a regression test? Naoto On 12/6/17 6:10 AM, Peter Levart wrote: > Hi, > > On 12/06/2017 02:30 PM, Alan Bateman wrote: >> I think this class is normally maintained on i18n-dev but I think >> introducing the Cache object looks good and making this much easier to >> understand. >> >> -Alan > > Thanks Alan, I'm forwarding to i18n-dev to see if maintainers of that > part of JDK have any comments... > > This is an official request for reviewing a patch for fixing a data race > between cloning a SimpleTimeZone object and lazily initializing its 3 > cache fields which may produce a clone with inconsistent cache state. > Here's Jira issue with details: > > https://bugs.openjdk.java.net/browse/JDK-8191216 > > Venkat has proposed to simply call invalidateCache() on the clone before > returning it from SimpleTimeZone.clone() method: > > ??? public Object clone() > ??? { > ??????? SimpleTimeZone clone = (SimpleTimeZone) super.clone(); > ??????? clone.invalidateCache(); > ??????? return clone; > ??? } > > This fixes the issue and for the case of TimeZone.getDefault() which is > called from ZoneId.systemDefault() even elides synchronization in > clone.invalidateCache() and allocation of the clone, so JITed > ZoneId.systemDefault() is unaffected by the patch. Initially I was > satisfied with the fix, but then I tested something. Suppose someone > sets default zone to SimpleTimeZone: > > ??????? TimeZone.setDefault( > ??????????? new SimpleTimeZone(3600000, > ?????????????????????????????? "Europe/Paris", > ?????????????????????????????? Calendar.MARCH, -1, Calendar.SUNDAY, > ?????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, > ?????????????????????????????? Calendar.OCTOBER, -1, Calendar.SUNDAY, > ?????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, > ?????????????????????????????? 3600000) > ??????? ); > > And then calls some methods that initialize the cache of the internal > shared TimeZone.defaultTimeZone instance: > > ??? new Date().toString(); > > The code which after that tries to obtain default time zone and > calculate the offset from UTC at some current point in time, for example: > > ??? TimeZone.getDefault().getOffset(now) > > can't use the cached state because it has been invalidated in the > returned clone. Such code has to re-compute the offset every time it > gets new clone. I measured this with a JMH benchmark and got the > following result: > > Default: > > Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units > ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? ns/op > ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? ns/op > > Venkat's patch - invalidateCache(): > > Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units > ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 179,476 ? 1,942? ns/op > ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,538 ? 0,073? ns/op > > We see that ZoneId.systemDefault() is unaffected, but > TimeZone.getDefault().getOffset(now) becomes 3x slower. > > This is not good, so I tried an alternative fix for the issue - simply > making the SimpleTimeZone.clone() synchronized. Access to cache fields > is already synchronized, so this should fix the race. Here's the JMH > result: > > Default: > > Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units > ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? ns/op > ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? ns/op > > Synchronized clone(): > > Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units > ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 58,103 ? 0,936? ns/op > ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 4,154 ? 0,034? ns/op > > We see that caching works again, but synchronization has some overhead > which is not big for single-threaded access, but might get bigger when > multiple threads try to get default zone concurrently. > > So I created a 3rd variant of the fix which I'm presenting here and > requesting a review for: > > http://cr.openjdk.java.net/~plevart/jdk10-dev/8191216_SimpleTimeZone_clone_race/webrev.01/ > > > The JMH benchmark shows this: > > Default: > > Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units > ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? ns/op > ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? ns/op > > Cache object: > > Benchmark????????????????????????????????? Mode? Cnt Score?? Error? Units > ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 42,860 ? 0,274? ns/op > ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,545 ? 0,057? ns/op > > Not only does the fix not affect ZoneId.systemDefault() which is not > surprising, but it also speeds-up cache lookup in single-threaded > benchmark and certainly eliminates possible contention among threads > looking up the shared instance. > > I have run the test/jdk/java/util/TimeZone and > test/jdk/java/util/Calendar jtreg tests and there were 7 failures caused > by compilation errors (package sun.util.locale.provider is not visible), > while 59 other tests pass. > > So, what do you think? > > Regards, Peter > > > Venkateswara R Chintala je 21. 11. 2017 ob 10:14?napisal: >> Thanks Peter for sponsoring this patch. Is there anything else that >> needs to be done from my end for this patch to be integrated? Please >> let me know. >> >> -Venkat >> >> >> On 14/11/17 8:46 PM, Peter Levart wrote: >>> Hi Venkat, >>> >>> I created the following issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8191216 >>> >>> I can also sponsor this patch and push it for JDK 10. >>> >>> The patch that you are proposing looks good to me. Does anybody have >>> anything else to say? >>> >>> Regards, Peter >>> >>> >>> On 11/13/2017 11:28 AM, Venkateswara R Chintala wrote: >>>> Thanks David, Peter for your review and comments. As I am new to the >>>> community, can you please help me to open a bug and integrate the >>>> changes into code base? >>>> >>>> -Venkat >>>> >>>> On 12/11/17 2:19 AM, Peter Levart wrote: >>>>> Hi David, Venkat, >>>>> >>>>> On 11/11/17 21:15, Peter Levart wrote: >>>>>> For example, take the following method: >>>>>> >>>>>> String defaultTZID() { >>>>>> ??? return TimeZone.getDefault().getID(); >>>>>> } >>>>>> >>>>>> When JIT compiles it and inlines invocations to other methods >>>>>> within it, it can prove that cloned TimeZone instance never >>>>>> escapes the call to defaultTZID() and can therefore skip >>>>>> allocating the instance on heap. >>>>>> >>>>>> But this is fragile. If code in invoked methods changes, they may >>>>>> not get inlined or EA may not be able to prove that the cloned >>>>>> instance can't escape and allocation may be introduced. >>>>>> ZoneId.systemDefault() is a hot method and it would be nice if we >>>>>> manage to keep it allocation free. >>>>> >>>>> Well, I tried the following variant of SimpleTimeZone.clone() patch: >>>>> >>>>> ??? public Object clone() >>>>> ??? { >>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>> ??????? // like tz.invalidateCache() but without holding a lock on >>>>> clone >>>>> ??????? tz.cacheYear = tz.startYear - 1; >>>>> ??????? tz.cacheStart = tz.cacheEnd = 0; >>>>> ??????? return tz; >>>>> ??? } >>>>> >>>>> >>>>> ...and the JMH benchmark with gc profiling shows that >>>>> ZoneId.systemDefault() still manages to get JIT-compiled without >>>>> introducing allocation. >>>>> >>>>> Even the following (original Venkat's) patch: >>>>> >>>>> ??? public Object clone() >>>>> ??? { >>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>> ??????? tz.invalidateCache(); >>>>> ??????? return tz; >>>>> ??? } >>>>> >>>>> ...does the same and the locking in invalidateCache() is elided >>>>> too. Allocation and lock-elision go hand-in-hand. When object >>>>> doesn't escape, allocation on heap may be eliminated and locks on >>>>> that instance elided. >>>>> >>>>> So this is better than synchronizing on the original instance >>>>> during .clone() execution as it has potential to avoid locking >>>>> overhead. >>>>> >>>>> So Venkat, go ahead. My fear was unjustified. >>>>> >>>>> Regards, Peter >>>>> >>>> >>> >>> >> From jason_mehrens at hotmail.com Wed Dec 6 19:54:13 2017 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Wed, 6 Dec 2017 19:54:13 +0000 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> Message-ID: Brian, For nullInputStream would it make any sense to use the ByteArrayInputStream with a (private static) empty byte array? Maybe 'return new ByteArrayInputStream("".getBytes());'? One side effect is that mark support returns true. Does it make sense to follow the advice in https://bugs.openjdk.java.net/browse/JDK-6533165 with regards to streams being closed? Thanks, Jason ________________________________________ From: core-libs-dev on behalf of Brian Burkhalter Sent: Wednesday, December 6, 2017 1:00 PM To: core-libs-dev Subject: RFR 4358774: Add null InputStream and OutputStream https://bugs.openjdk.java.net/browse/JDK-4358774 http://cr.openjdk.java.net/~bpb/4358774/webrev.00/ Add nullStream() method to each of InputStream and OutputStream. Thanks, Brian From brian.burkhalter at oracle.com Wed Dec 6 20:05:56 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Dec 2017 12:05:56 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> Message-ID: <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> Jason, On Dec 6, 2017, at 11:54 AM, Jason Mehrens wrote: > For nullInputStream would it make any sense to use the ByteArrayInputStream with a (private static) empty byte array? Maybe 'return new ByteArrayInputStream("".getBytes());'? One side effect is that mark support returns true. As we are trying to retain the behavior of closed streams throwing IOExceptions I don?t think that BAIS would work here. > Does it make sense to follow the advice inhttps://bugs.openjdk.java.net/browse/JDK-6533165 with regards to streams being closed? I don?t know exactly what you intend here. In the linked issue there is information to impart, namely the index and the size. Here there is only the indication that the stream is closed. It?s not clear to me what could be done here. Thanks, Brian From claes.redestad at oracle.com Wed Dec 6 20:21:55 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 6 Dec 2017 21:21:55 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() Message-ID: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> Hi, please help review this patch to consolidate the number of implementation classes returned by the static collection factories: http://cr.openjdk.java.net/~redestad/8193128/open.00/ I set out to explore options for addressing small inefficiencies we've been running into, the latest after replacing use of @Stable arrays in java.lang.invoke with List.of() (JDK-8184777): - List.indexOf deferred to the iterator in AbstractList, which check for concurrent modification - Warmup takes a bit longer due to having to warm up four different classes and associated methods - Slowdowns in certain code paths attributable to certain calls becoming megamorphic Microbenchmark: http://cr.openjdk.java.net/~redestad/scratch/less_immutables/ListMorphism.java http://cr.openjdk.java.net/~redestad/scratch/less_immutables/benchmarks.jar The benchmark explores how call-sites behave when performing some naive operations on a few different Lists. For every benchmark using List.of() there's a variant using ArrayList for comparison: Baseline: Benchmark???????????????????????????? Mode? Cnt??? Score Error?? Units ListMorphism.finalGetFromArrayList?? thrpt?? 25?? 92.659 ?? 3.058 ops/us ListMorphism.finalGetFromList??????? thrpt?? 25? 335.245 ?? 0.244 ops/us # 3.6x ListMorphism.finalSumSizesArrayList? thrpt?? 25? 245.020 ?? 0.106 ops/us ListMorphism.finalSumSizesList?????? thrpt?? 25? 335.107 ?? 0.439 ops/us # 1.4x ListMorphism.getFromArrayList??????? thrpt?? 25?? 70.343 ?? 0.972 ops/us ListMorphism.getFromList???????????? thrpt?? 25?? 37.121 ?? 0.135 ops/us # 0.53x ListMorphism.getFromArrayList12????? thrpt?? 25 109.890 ?? 0.286? ops/us ListMorphism.getFromList12?????????? thrpt?? 25? 109.552 ? 6.972? ops/us # 1.0x ListMorphism.sumSizesArrayList?????? thrpt?? 25? 131.467 ?? 4.672 ops/us ListMorphism.sumSizesList??????????? thrpt?? 25?? 57.877 ?? 0.102 ops/us # 0.45x ListMorphism.sumSizesArrayList12???? thrpt?? 25 208.652 ? 11.294? ops/us ListMorphism.sumSizesList12????????? thrpt?? 25? 227.269 ? 0.961? ops/us # 1.1x The good: When dealing with List literals (the final* benchmarks), List.of() allows really nice speed-ups compared to ArrayList. The bad: When not used as a literal, List.of() implementations at call-sites can cause a substantial slowdown (megamorphic dispatch) The ugly: After some explorations[1], I narrowed in on the following experiment: http://cr.openjdk.java.net/~redestad/scratch/less_immutables/webrev/ Basically: Merge List1 and List2 into a single class, List12, merge List0 into ListN (List0 has a singleton instance, so might as well be an instance of ListN). Same for Set0,1,2,N. Map0 is merged into MapN. This strikes a balance between throughput, footprint and slightly better startup/warmup behavior. According to jol estimates[2] the size of List12 is the same as both List1 and List2 in the current JDK implementation. Set12 is footprint neutral compared to Set2 on all platforms but lose a few bytes on 64-bit VMs compared to Set1. Benchmark???????????????????????????? Mode? Cnt??? Score?? Error Units ListMorphism.finalGetFromArrayList?? thrpt?? 25?? 93.046 ? 0.569 ops/us ListMorphism.finalGetFromList??????? thrpt?? 25? 335.280 ? 0.154 ops/us # 3.6x ListMorphism.finalSumSizesArrayList? thrpt?? 25? 244.595 ? 1.085 ops/us ListMorphism.finalSumSizesList?????? thrpt?? 25? 303.351 ? 0.182 ops/us # 1.2x ListMorphism.getFromArrayList??????? thrpt?? 25?? 70.631 ? 0.070 ops/us ListMorphism.getFromList???????????? thrpt?? 25?? 73.258 ? 2.955 ops/us # 1.04x ListMorphism.getFromArrayList12????? thrpt?? 25 109.921 ? 0.096? ops/us ListMorphism.getFromList12?????????? thrpt?? 25? 127.392 ? 0.088? ops/us # 1.16x ListMorphism.sumSizesArrayList?????? thrpt?? 25? 131.393 ? 4.882 ops/us ListMorphism.sumSizesList??????????? thrpt?? 25? 107.686 ? 5.286 ops/us # 0.82x ListMorphism.sumSizesArrayList12???? thrpt?? 25? 212.350 ? 0.134? ops/us ListMorphism.sumSizesList12????????? thrpt?? 25? 198.778 ? 0.479? ops/us # 0.94x The experiment has a flag to change number of specialized List/Set/Map classes (-Djdk.ImmutableCollections.specializations=0|1|2, default=2). 1 specialization (List1 + ListN, Set1 + SetN) is more or less the same as 2, some ~1-2% improvements, mainly in sumSizes micros. 0 specializations (List-, Set, MapN only) achieves a small increase in throughput on some micros by ensuring callsites stay monomorphic, but it's not very substantial by my measures (~5%, but mostly in sumSizes micros). Keeping the footprint more or less the same, while evening out a few rough edges and improving startup and static footprint seems like the overall best option. An alternative would be to keep the experimental flag, but I don't think a 5% gain on micros warrants the extra maintenance burden and testing that entails. The proposed patch is more or less this experiment with 2 specializations, but having removed the flag and code movement needed to support it removed (along with a fix in the writeReplace methods of List12/Set12) Thanks! /Claes [1] Older experiments: http://cr.openjdk.java.net/~redestad/immutable/list12N.0/ ?-- simply merge L0 into LN (still have L1, L2 and LN) - nothing really changed, though http://cr.openjdk.java.net/~redestad/immutable/list1N.0/ ?-- L0 merged into LN, drop L2. Substantial performance boost on micros. Footprint drop for 2-element lists. http://cr.openjdk.java.net/~redestad/immutable/listNdouble.0/ ?-- L0+L1+L2+LN merged into one implementation. Same footprint with a single class, but loses a lot on throughput in micros. http://cr.openjdk.java.net/~redestad/immutable/listNsingle.0/ ?-- L0+L1+LN merged, drop L2. Simplification of the previous. Like the list1N.0 experiment in footprint, but a loss in throughput on all measures. No approach seemed a win across the board here: we either had to accept a footprint reduction, or see throughput suffer dramatically. [2] http://openjdk.java.net/projects/code-tools/jol/ From patrick at reini.net Wed Dec 6 20:36:00 2017 From: patrick at reini.net (Patrick Reinhart) Date: Wed, 6 Dec 2017 21:36:00 +0100 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> Message-ID: <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> Sounds great, perfect to remove some more own code? Is there also a issue for the same kind of methods for Reader and Writer? -Patrick > Am 06.12.2017 um 20:00 schrieb Brian Burkhalter : > > https://bugs.openjdk.java.net/browse/JDK-4358774 > http://cr.openjdk.java.net/~bpb/4358774/webrev.00/ > > Add nullStream() method to each of InputStream and OutputStream. > > Thanks, > > Brian From brian.burkhalter at oracle.com Wed Dec 6 20:43:19 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Dec 2017 12:43:19 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> Message-ID: <14347E03-D0B8-450B-BCB8-2B0475F166BE@oracle.com> On Dec 6, 2017, at 12:36 PM, Patrick Reinhart wrote: > Sounds great, perfect to remove some more own code? Yes I still need to look through the JDK source base for some code to replace with this, assuming this is approved. > Is there also a issue for the same kind of methods for Reader and Writer? Nothing officially on file yet but it?s ?on the list? somewhere. Brian From patrick at reini.net Wed Dec 6 20:46:30 2017 From: patrick at reini.net (Patrick Reinhart) Date: Wed, 6 Dec 2017 21:46:30 +0100 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <14347E03-D0B8-450B-BCB8-2B0475F166BE@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> <14347E03-D0B8-450B-BCB8-2B0475F166BE@oracle.com> Message-ID: <24201C5F-4B41-49D5-8AB1-0AF03BD24F2A@reini.net> > Am 06.12.2017 um 21:43 schrieb Brian Burkhalter : > > On Dec 6, 2017, at 12:36 PM, Patrick Reinhart > wrote: > >> Sounds great, perfect to remove some more own code? > > Yes I still need to look through the JDK source base for some code to replace with this, assuming this is approved. You got min - even I?m not eligible to give you an official one :-) > >> Is there also a issue for the same kind of methods for Reader and Writer? > > Nothing officially on file yet but it?s ?on the list? somewhere. Would be a something that I could work on as soon you got the approval? -Patrick From brian.burkhalter at oracle.com Wed Dec 6 20:49:20 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Dec 2017 12:49:20 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <24201C5F-4B41-49D5-8AB1-0AF03BD24F2A@reini.net> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> <14347E03-D0B8-450B-BCB8-2B0475F166BE@oracle.com> <24201C5F-4B41-49D5-8AB1-0AF03BD24F2A@reini.net> Message-ID: <674DAED3-023A-4B94-8EA1-6AE253E5E199@oracle.com> On Dec 6, 2017, at 12:46 PM, Patrick Reinhart wrote: >> Yes I still need to look through the JDK source base for some code to replace with this, assuming this is approved. > > You got min - even I?m not eligible to give you an official one :-) Thanks. >> Is there also a issue for the same kind of methods for Reader and Writer? >> >> Nothing officially on file yet but it?s ?on the list? somewhere. > > Would be a something that I could work on as soon you got the approval? Sure. Thanks, Brian From jbluettduncan at gmail.com Wed Dec 6 20:58:29 2017 From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan) Date: Wed, 6 Dec 2017 20:58:29 +0000 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> Message-ID: Hi Claes, Looking at http://cr.openjdk.java.net/~redestad/8193128/open.00/src/java.base/share/classes/java/util/ImmutableCollections.java.cdiff.html, there are sections labelled --- 646,657 ---- and --- 834,845 ---- where lines like `Objects.requireNonNull(0 /* zero */);` are written. I believe that they were supposed to be either removed or made to be written like `Objects.requireNonNull(o /* lowercase o */)`. Is my belief/understanding correct? Cheers, Jonathan On 6 December 2017 at 20:21, Claes Redestad wrote: > Hi, > > please help review this patch to consolidate the number of implementation > classes returned by the static collection factories: > > http://cr.openjdk.java.net/~redestad/8193128/open.00/ > > I set out to explore options for addressing small inefficiencies we've > been running into, the latest after replacing use of @Stable arrays in > java.lang.invoke with List.of() (JDK-8184777): > > - List.indexOf deferred to the iterator in AbstractList, which check for > concurrent modification > - Warmup takes a bit longer due to having to warm up four different > classes and associated methods > - Slowdowns in certain code paths attributable to certain calls becoming > megamorphic > > Microbenchmark: http://cr.openjdk.java.net/~re > destad/scratch/less_immutables/ListMorphism.java > http://cr.openjdk.java.net/~redestad/scratch/less_immutables > /benchmarks.jar > > The benchmark explores how call-sites behave when performing some naive > operations on a few different Lists. > > For every benchmark using List.of() there's a variant using ArrayList for > comparison: > > Baseline: > > Benchmark Mode Cnt Score Error Units > ListMorphism.finalGetFromArrayList thrpt 25 92.659 ? 3.058 ops/us > ListMorphism.finalGetFromList thrpt 25 335.245 ? 0.244 ops/us > # 3.6x > ListMorphism.finalSumSizesArrayList thrpt 25 245.020 ? 0.106 ops/us > ListMorphism.finalSumSizesList thrpt 25 335.107 ? 0.439 ops/us > # 1.4x > ListMorphism.getFromArrayList thrpt 25 70.343 ? 0.972 ops/us > ListMorphism.getFromList thrpt 25 37.121 ? 0.135 ops/us > # 0.53x > ListMorphism.getFromArrayList12 thrpt 25 109.890 ? 0.286 ops/us > ListMorphism.getFromList12 thrpt 25 109.552 ? 6.972 ops/us > # 1.0x > ListMorphism.sumSizesArrayList thrpt 25 131.467 ? 4.672 ops/us > ListMorphism.sumSizesList thrpt 25 57.877 ? 0.102 ops/us > # 0.45x > ListMorphism.sumSizesArrayList12 thrpt 25 208.652 ? 11.294 ops/us > ListMorphism.sumSizesList12 thrpt 25 227.269 ? 0.961 ops/us > # 1.1x > > The good: When dealing with List literals (the final* benchmarks), > List.of() allows really nice speed-ups compared to ArrayList. > > The bad: When not used as a literal, List.of() implementations at > call-sites can cause a substantial slowdown (megamorphic dispatch) > > The ugly: > > After some explorations[1], I narrowed in on the following experiment: > http://cr.openjdk.java.net/~redestad/scratch/less_immutables/webrev/ > > Basically: Merge List1 and List2 into a single class, List12, merge List0 > into ListN (List0 has a singleton instance, so might as well be an instance > of ListN). Same for Set0,1,2,N. Map0 is merged into MapN. > > This strikes a balance between throughput, footprint and slightly better > startup/warmup behavior. > > According to jol estimates[2] the size of List12 is the same as both List1 > and List2 in the current JDK implementation. Set12 is footprint neutral > compared to Set2 on all platforms but lose a few bytes on 64-bit VMs > compared to Set1. > > Benchmark Mode Cnt Score Error Units > ListMorphism.finalGetFromArrayList thrpt 25 93.046 ? 0.569 ops/us > ListMorphism.finalGetFromList thrpt 25 335.280 ? 0.154 ops/us # > 3.6x > ListMorphism.finalSumSizesArrayList thrpt 25 244.595 ? 1.085 ops/us > ListMorphism.finalSumSizesList thrpt 25 303.351 ? 0.182 ops/us # > 1.2x > ListMorphism.getFromArrayList thrpt 25 70.631 ? 0.070 ops/us > ListMorphism.getFromList thrpt 25 73.258 ? 2.955 ops/us # > 1.04x > ListMorphism.getFromArrayList12 thrpt 25 109.921 ? 0.096 ops/us > ListMorphism.getFromList12 thrpt 25 127.392 ? 0.088 ops/us > # 1.16x > ListMorphism.sumSizesArrayList thrpt 25 131.393 ? 4.882 ops/us > ListMorphism.sumSizesList thrpt 25 107.686 ? 5.286 ops/us # > 0.82x > ListMorphism.sumSizesArrayList12 thrpt 25 212.350 ? 0.134 ops/us > ListMorphism.sumSizesList12 thrpt 25 198.778 ? 0.479 ops/us > # 0.94x > > The experiment has a flag to change number of specialized List/Set/Map > classes (-Djdk.ImmutableCollections.specializations=0|1|2, default=2). > > 1 specialization (List1 + ListN, Set1 + SetN) is more or less the same as > 2, some ~1-2% improvements, mainly in sumSizes micros. > > 0 specializations (List-, Set, MapN only) achieves a small increase in > throughput on some micros by ensuring callsites stay monomorphic, but it's > not very substantial by my measures (~5%, but mostly in sumSizes micros). > > Keeping the footprint more or less the same, while evening out a few rough > edges and improving startup and static footprint seems like the overall > best option. An alternative would be to keep the experimental flag, but I > don't think a 5% gain on micros warrants the extra maintenance burden and > testing that entails. > > The proposed patch is more or less this experiment with 2 specializations, > but having removed the flag and code movement needed to support it removed > (along with a fix in the writeReplace methods of List12/Set12) > > Thanks! > > /Claes > > [1] Older experiments: > http://cr.openjdk.java.net/~redestad/immutable/list12N.0/ > -- simply merge L0 into LN (still have L1, L2 and LN) - nothing really > changed, though > > http://cr.openjdk.java.net/~redestad/immutable/list1N.0/ > -- L0 merged into LN, drop L2. Substantial performance boost on micros. > Footprint drop for 2-element lists. > > http://cr.openjdk.java.net/~redestad/immutable/listNdouble.0/ > -- L0+L1+L2+LN merged into one implementation. Same footprint with a > single class, but loses a lot on throughput in micros. > > http://cr.openjdk.java.net/~redestad/immutable/listNsingle.0/ > -- L0+L1+LN merged, drop L2. Simplification of the previous. Like the > list1N.0 experiment in footprint, but a loss in throughput on all measures. > > No approach seemed a win across the board here: we either had to accept a > footprint reduction, or see throughput suffer dramatically. > > [2] http://openjdk.java.net/projects/code-tools/jol/ > From jason_mehrens at hotmail.com Wed Dec 6 21:01:03 2017 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Wed, 6 Dec 2017 21:01:03 +0000 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> , <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> Message-ID: Brian, My understand is JDK-6533165 is moving the bytes that are rarely invoked to a cold method. Therefore: ==== if (closed) { throw new IOException("Stream closed"); } ==becomes=== if (closed) { throw sc(); } private static IOException sc() { return new IOException("Stream closed"); } ================================ Since there is no string concatenation in the IOE message there are only a few bytes that can be shaved off. I didn't benchmark it either case but I would assume it would matter for the nullOutputStream since the write methods could be invoked multiple times. Jason ________________________________________ From: Brian Burkhalter Sent: Wednesday, December 6, 2017 2:05 PM To: Jason Mehrens Cc: core-libs-dev Subject: Re: RFR 4358774: Add null InputStream and OutputStream Jason, On Dec 6, 2017, at 11:54 AM, Jason Mehrens > wrote: For nullInputStream would it make any sense to use the ByteArrayInputStream with a (private static) empty byte array? Maybe 'return new ByteArrayInputStream("".getBytes());'? One side effect is that mark support returns true. As we are trying to retain the behavior of closed streams throwing IOExceptions I don?t think that BAIS would work here. Does it make sense to follow the advice inhttps://bugs.openjdk.java.net/browse/JDK-6533165 with regards to streams being closed? I don?t know exactly what you intend here. In the linked issue there is information to impart, namely the index and the size. Here there is only the indication that the stream is closed. It?s not clear to me what could be done here. Thanks, Brian From martinrb at google.com Wed Dec 6 21:20:15 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Dec 2017 13:20:15 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> Message-ID: Guava struggled with this as well with their immutable collections. You could look at their revision history (I haven't). Maybe they got rid of SingletonImmutableList for the same reason? On Wed, Dec 6, 2017 at 12:21 PM, Claes Redestad wrote: > Hi, > > please help review this patch to consolidate the number of implementation > classes returned by the static collection factories: > > http://cr.openjdk.java.net/~redestad/8193128/open.00/ > > I set out to explore options for addressing small inefficiencies we've > been running into, the latest after replacing use of @Stable arrays in > java.lang.invoke with List.of() (JDK-8184777): > > - List.indexOf deferred to the iterator in AbstractList, which check for > concurrent modification > - Warmup takes a bit longer due to having to warm up four different > classes and associated methods > - Slowdowns in certain code paths attributable to certain calls becoming > megamorphic > > Microbenchmark: http://cr.openjdk.java.net/~re > destad/scratch/less_immutables/ListMorphism.java > http://cr.openjdk.java.net/~redestad/scratch/less_immutables > /benchmarks.jar > > The benchmark explores how call-sites behave when performing some naive > operations on a few different Lists. > > For every benchmark using List.of() there's a variant using ArrayList for > comparison: > > Baseline: > > Benchmark Mode Cnt Score Error Units > ListMorphism.finalGetFromArrayList thrpt 25 92.659 ? 3.058 ops/us > ListMorphism.finalGetFromList thrpt 25 335.245 ? 0.244 ops/us > # 3.6x > ListMorphism.finalSumSizesArrayList thrpt 25 245.020 ? 0.106 ops/us > ListMorphism.finalSumSizesList thrpt 25 335.107 ? 0.439 ops/us > # 1.4x > ListMorphism.getFromArrayList thrpt 25 70.343 ? 0.972 ops/us > ListMorphism.getFromList thrpt 25 37.121 ? 0.135 ops/us > # 0.53x > ListMorphism.getFromArrayList12 thrpt 25 109.890 ? 0.286 ops/us > ListMorphism.getFromList12 thrpt 25 109.552 ? 6.972 ops/us > # 1.0x > ListMorphism.sumSizesArrayList thrpt 25 131.467 ? 4.672 ops/us > ListMorphism.sumSizesList thrpt 25 57.877 ? 0.102 ops/us > # 0.45x > ListMorphism.sumSizesArrayList12 thrpt 25 208.652 ? 11.294 ops/us > ListMorphism.sumSizesList12 thrpt 25 227.269 ? 0.961 ops/us > # 1.1x > > The good: When dealing with List literals (the final* benchmarks), > List.of() allows really nice speed-ups compared to ArrayList. > > The bad: When not used as a literal, List.of() implementations at > call-sites can cause a substantial slowdown (megamorphic dispatch) > > The ugly: > > After some explorations[1], I narrowed in on the following experiment: > http://cr.openjdk.java.net/~redestad/scratch/less_immutables/webrev/ > > Basically: Merge List1 and List2 into a single class, List12, merge List0 > into ListN (List0 has a singleton instance, so might as well be an instance > of ListN). Same for Set0,1,2,N. Map0 is merged into MapN. > > This strikes a balance between throughput, footprint and slightly better > startup/warmup behavior. > > According to jol estimates[2] the size of List12 is the same as both List1 > and List2 in the current JDK implementation. Set12 is footprint neutral > compared to Set2 on all platforms but lose a few bytes on 64-bit VMs > compared to Set1. > > Benchmark Mode Cnt Score Error Units > ListMorphism.finalGetFromArrayList thrpt 25 93.046 ? 0.569 ops/us > ListMorphism.finalGetFromList thrpt 25 335.280 ? 0.154 ops/us # > 3.6x > ListMorphism.finalSumSizesArrayList thrpt 25 244.595 ? 1.085 ops/us > ListMorphism.finalSumSizesList thrpt 25 303.351 ? 0.182 ops/us # > 1.2x > ListMorphism.getFromArrayList thrpt 25 70.631 ? 0.070 ops/us > ListMorphism.getFromList thrpt 25 73.258 ? 2.955 ops/us # > 1.04x > ListMorphism.getFromArrayList12 thrpt 25 109.921 ? 0.096 ops/us > ListMorphism.getFromList12 thrpt 25 127.392 ? 0.088 ops/us > # 1.16x > ListMorphism.sumSizesArrayList thrpt 25 131.393 ? 4.882 ops/us > ListMorphism.sumSizesList thrpt 25 107.686 ? 5.286 ops/us # > 0.82x > ListMorphism.sumSizesArrayList12 thrpt 25 212.350 ? 0.134 ops/us > ListMorphism.sumSizesList12 thrpt 25 198.778 ? 0.479 ops/us > # 0.94x > > The experiment has a flag to change number of specialized List/Set/Map > classes (-Djdk.ImmutableCollections.specializations=0|1|2, default=2). > > 1 specialization (List1 + ListN, Set1 + SetN) is more or less the same as > 2, some ~1-2% improvements, mainly in sumSizes micros. > > 0 specializations (List-, Set, MapN only) achieves a small increase in > throughput on some micros by ensuring callsites stay monomorphic, but it's > not very substantial by my measures (~5%, but mostly in sumSizes micros). > > Keeping the footprint more or less the same, while evening out a few rough > edges and improving startup and static footprint seems like the overall > best option. An alternative would be to keep the experimental flag, but I > don't think a 5% gain on micros warrants the extra maintenance burden and > testing that entails. > > The proposed patch is more or less this experiment with 2 specializations, > but having removed the flag and code movement needed to support it removed > (along with a fix in the writeReplace methods of List12/Set12) > > Thanks! > > /Claes > > [1] Older experiments: > http://cr.openjdk.java.net/~redestad/immutable/list12N.0/ > -- simply merge L0 into LN (still have L1, L2 and LN) - nothing really > changed, though > > http://cr.openjdk.java.net/~redestad/immutable/list1N.0/ > -- L0 merged into LN, drop L2. Substantial performance boost on micros. > Footprint drop for 2-element lists. > > http://cr.openjdk.java.net/~redestad/immutable/listNdouble.0/ > -- L0+L1+L2+LN merged into one implementation. Same footprint with a > single class, but loses a lot on throughput in micros. > > http://cr.openjdk.java.net/~redestad/immutable/listNsingle.0/ > -- L0+L1+LN merged, drop L2. Simplification of the previous. Like the > list1N.0 experiment in footprint, but a loss in throughput on all measures. > > No approach seemed a win across the board here: we either had to accept a > footprint reduction, or see throughput suffer dramatically. > > [2] http://openjdk.java.net/projects/code-tools/jol/ > From martinrb at google.com Wed Dec 6 21:35:42 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Dec 2017 13:35:42 -0800 Subject: Android and Log4j In-Reply-To: References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> Message-ID: Very latest Android Studio comes with two compilers, dx and d8. https://developer.android.com/studio/preview/features/index.html I hear that both compilers know to skip module-info files. On Mon, Dec 4, 2017 at 12:03 PM, Martin Buchholz wrote: > > > On Sun, Dec 3, 2017 at 11:02 AM, Remi Forax wrote: > >> Ask Google to fix dx, >> dx should ignore the module-info.class and everything inside >> META-INF/versions (at least it's a first simple patch). >> > > Hi, I'm "Google", sort of. Friends tell me that dx is getting fixed. > > From stuart.marks at oracle.com Wed Dec 6 21:42:27 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Dec 2017 13:42:27 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> Message-ID: <16fff744-84e2-b0f3-a2c5-9679f6efaf3b@oracle.com> On 12/5/17 8:41 PM, Bernd Eckenfels wrote: > Should the test also check for the case the generator returns under- and oversized arrays? Did you mean that the default implementation (and various overriding implementations) of toArray(generator) should do checks on the array that's returned from the generator? If so, I'm skeptical of the utility of adding such checks. If the array is too short, ArrayIndexOutOfBoundsException will occur, so no check is necessary. If the array is too long, it will work, but the application might have trouble ascertaining the number of elements copied. Presumably it wouldn't create a too-long array unless it can deal with this. In any case, it doesn't necessarily seem like this should be an error. > The default impl looks less efficient than (new T[0]), should it really be removed as a major Javadoc example? (Or maybe add more specific implementations besides ArrayList?) Although my benchmarks were inconclusive, it doesn't seem like the extra call to the generator with zero should be a performance problem. Even if this is a problem, it will disappear as more overrides are added to the system. s'marks From forax at univ-mlv.fr Wed Dec 6 21:59:23 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 6 Dec 2017 22:59:23 +0100 (CET) Subject: Android and Log4j In-Reply-To: References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> Message-ID: <1419863824.1566406.1512597563495.JavaMail.zimbra@u-pem.fr> Hi Google :) great news. R?mi > De: "Martin Buchholz" > ?: "Remi Forax" > Cc: "Ralph Goers" , "core-libs-dev" > > Envoy?: Lundi 4 D?cembre 2017 21:03:23 > Objet: Re: Android and Log4j > On Sun, Dec 3, 2017 at 11:02 AM, Remi Forax < [ mailto:forax at univ-mlv.fr | > forax at univ-mlv.fr ] > wrote: >> Ask Google to fix dx, >> dx should ignore the module-info.class and everything inside META-INF/versions >> (at least it's a first simple patch). > Hi, I'm "Google", sort of. Friends tell me that dx is getting fixed. From forax at univ-mlv.fr Wed Dec 6 22:03:28 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 6 Dec 2017 23:03:28 +0100 (CET) Subject: Android and Log4j In-Reply-To: References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> Message-ID: <1045760895.1566792.1512597808721.JavaMail.zimbra@u-pem.fr> > De: "Martin Buchholz" > ?: "Remi Forax" > Cc: "Ralph Goers" , "core-libs-dev" > > Envoy?: Mercredi 6 D?cembre 2017 22:35:42 > Objet: Re: Android and Log4j > Very latest Android Studio comes with two compilers, dx and d8. > [ https://developer.android.com/studio/preview/features/index.html | > https://developer.android.com/studio/preview/features/index.html ] > I hear that both compilers know to skip module-info files. I hope dx will die soon, i had to patch it once and i was messy, a register allocator is usually not a fun code but in case of dx, it is hidden by several layers of abstraction which made me cringe. R?mi > On Mon, Dec 4, 2017 at 12:03 PM, Martin Buchholz < [ mailto:martinrb at google.com > | martinrb at google.com ] > wrote: >> On Sun, Dec 3, 2017 at 11:02 AM, Remi Forax < [ mailto:forax at univ-mlv.fr | >> forax at univ-mlv.fr ] > wrote: >>> Ask Google to fix dx, >>> dx should ignore the module-info.class and everything inside META-INF/versions >>> (at least it's a first simple patch). >> Hi, I'm "Google", sort of. Friends tell me that dx is getting fixed. From stuart.marks at oracle.com Wed Dec 6 22:33:36 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Dec 2017 14:33:36 -0800 Subject: RFR(s): 8177681: Remove methods Runtime.getLocalized{Input,Output}Stream Message-ID: <418e5290-d0d5-c21c-9fde-258ef53c75fd@oracle.com> Hi all, Please review the removal of these methods, which were part of an obsolete internationalization mechanism. They were deprecated in JDK 1.1 and deprecated for removal in JDK 9. As far as I can see, there is no usage of these methods anywhere. Bug link: https://bugs.openjdk.java.net/browse/JDK-8177681 Diffs below. s'marks diff -r 6ee80cd217e0 -r 3443fda73ab6 src/java.base/share/classes/java/lang/Runtime.java --- a/src/java.base/share/classes/java/lang/Runtime.java Mon Dec 04 11:50:04 2017 -0800 +++ b/src/java.base/share/classes/java/lang/Runtime.java Wed Dec 06 13:55:35 2017 -0800 @@ -877,62 +877,6 @@ } /** - * Creates a localized version of an input stream. This method takes - * an {@code InputStream} and returns an {@code InputStream} - * equivalent to the argument in all respects except that it is - * localized: as characters in the local character set are read from - * the stream, they are automatically converted from the local - * character set to Unicode. - *

- * If the argument is already a localized stream, it may be returned - * as the result. - * - * @param in InputStream to localize - * @return a localized input stream - * @see java.io.InputStream - * @see java.io.BufferedReader#BufferedReader(java.io.Reader) - * @see java.io.InputStreamReader#InputStreamReader(java.io.InputStream) - * @deprecated As of JDK 1.1, the preferred way to translate a byte - * stream in the local encoding into a character stream in Unicode is via - * the {@code InputStreamReader} and {@code BufferedReader} - * classes. - * This method is subject to removal in a future version of Java SE. - */ - @Deprecated(since="1.1", forRemoval=true) - public InputStream getLocalizedInputStream(InputStream in) { - return in; - } - - /** - * Creates a localized version of an output stream. This method - * takes an {@code OutputStream} and returns an - * {@code OutputStream} equivalent to the argument in all respects - * except that it is localized: as Unicode characters are written to - * the stream, they are automatically converted to the local - * character set. - *

- * If the argument is already a localized stream, it may be returned - * as the result. - * - * @deprecated As of JDK 1.1, the preferred way to translate a - * Unicode character stream into a byte stream in the local encoding is via - * the {@code OutputStreamWriter}, {@code BufferedWriter}, and - * {@code PrintWriter} classes. - * This method is subject to removal in a future version of Java SE. - * - * @param out OutputStream to localize - * @return a localized output stream - * @see java.io.OutputStream - * @see java.io.BufferedWriter#BufferedWriter(java.io.Writer) - * @see java.io.OutputStreamWriter#OutputStreamWriter(java.io.OutputStream) - * @see java.io.PrintWriter#PrintWriter(java.io.OutputStream) - */ - @Deprecated(since="1.1", forRemoval=true) - public OutputStream getLocalizedOutputStream(OutputStream out) { - return out; - } - - /** * Returns the version of the Java Runtime Environment as a {@link Version}. * * @return the {@link Version} of the Java Runtime Environment From Roger.Riggs at Oracle.com Wed Dec 6 22:45:26 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Dec 2017 17:45:26 -0500 Subject: RFR JDK-8187485: Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <5A25D6BE.60203@oracle.com> References: <5A25D6BE.60203@oracle.com> Message-ID: <545536b4-f981-7611-5269-e056e3c6a1f0@Oracle.com> Hi Sherman, Looking good. Deflater(573) and Inflater(398), and ZipFile(890): ?? For consistency, where is says "phantom-reachable" in the finalize methods it can say "unreachable". ZStreamRef: ?- line 92: FinalizableZStreamRef? - owner should be final Regards,? Roger On 12/4/2017 6:14 PM, Xueming Shen wrote: > Hi > > Please review the revision to the change for > > JDK-8187485: Update Zip implementation to use Cleaner, not finalizers > > For compatibility with previous jdk releases, in this proposed > revision the corresponding > close()/end() methods will be called when the > ZipFile/Inflater/Deflater object has become > unreachable, if the ZipFile/Inflater/Deflater has been subclassed and > its close()/end() method > has been overridden. In this case, the finalization mechanism is used > to clean up the underlying > resources.? If the ZipFile/Inflater/Deflater has not been subclassed > OR the corresponding > close()/end() method has not been overridden, then the Cleaner > mechanism is used to do > the cleanup. > > > issue: https://bugs.openjdk.java.net/browse/JDK-8185582 > webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev > csr: https://bugs.openjdk.java.net/browse/JDK-8187485 > > Thanks, > Sherman From brian.burkhalter at oracle.com Wed Dec 6 22:46:16 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Dec 2017 14:46:16 -0800 Subject: RFR(s): 8177681: Remove methods Runtime.getLocalized{Input, Output}Stream In-Reply-To: <418e5290-d0d5-c21c-9fde-258ef53c75fd@oracle.com> References: <418e5290-d0d5-c21c-9fde-258ef53c75fd@oracle.com> Message-ID: <411A91B3-E4F3-4E56-B98D-6FABED5EECA7@oracle.com> +1 Brian On Dec 6, 2017, at 2:33 PM, Stuart Marks wrote: > Please review the removal of these methods, which were part of an obsolete internationalization mechanism. They were deprecated in JDK 1.1 and deprecated for removal in JDK 9. As far as I can see, there is no usage of these methods anywhere. > > Bug link: > > https://bugs.openjdk.java.net/browse/JDK-8177681 From Roger.Riggs at Oracle.com Wed Dec 6 22:47:28 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Dec 2017 17:47:28 -0500 Subject: RFR(s): 8177681: Remove methods Runtime.getLocalized{Input,Output}Stream In-Reply-To: <418e5290-d0d5-c21c-9fde-258ef53c75fd@oracle.com> References: <418e5290-d0d5-c21c-9fde-258ef53c75fd@oracle.com> Message-ID: <4db0a73a-097d-de99-6e4e-2f576f2a9955@Oracle.com> +1,? About time! On 12/6/2017 5:33 PM, Stuart Marks wrote: > Hi all, > > Please review the removal of these methods, which were part of an > obsolete internationalization mechanism. They were deprecated in JDK > 1.1 and deprecated for removal in JDK 9. As far as I can see, there is > no usage of these methods anywhere. > > Bug link: > > ??? https://bugs.openjdk.java.net/browse/JDK-8177681 > > Diffs below. > > s'marks > > diff -r 6ee80cd217e0 -r 3443fda73ab6 > src/java.base/share/classes/java/lang/Runtime.java > --- a/src/java.base/share/classes/java/lang/Runtime.java??? Mon Dec 04 > 11:50:04 2017 -0800 > +++ b/src/java.base/share/classes/java/lang/Runtime.java??? Wed Dec 06 > 13:55:35 2017 -0800 > @@ -877,62 +877,6 @@ > ???? } > > ???? /** > -???? * Creates a localized version of an input stream. This method takes > -???? * an {@code InputStream} and returns an {@code InputStream} > -???? * equivalent to the argument in all respects except that it is > -???? * localized: as characters in the local character set are read from > -???? * the stream, they are automatically converted from the local > -???? * character set to Unicode. > -???? *

> -???? * If the argument is already a localized stream, it may be returned > -???? * as the result. > -???? * > -???? * @param????? in InputStream to localize > -???? * @return???? a localized input stream > -???? * @see??????? java.io.InputStream > -???? * @see java.io.BufferedReader#BufferedReader(java.io.Reader) > -???? * @see > java.io.InputStreamReader#InputStreamReader(java.io.InputStream) > -???? * @deprecated As of JDK 1.1, the preferred way to translate > a byte > -???? * stream in the local encoding into a character stream in > Unicode is via > -???? * the {@code InputStreamReader} and {@code BufferedReader} > -???? * classes. > -???? * This method is subject to removal in a future version of Java SE. > -???? */ > -??? @Deprecated(since="1.1", forRemoval=true) > -??? public InputStream getLocalizedInputStream(InputStream in) { > -??????? return in; > -??? } > - > -??? /** > -???? * Creates a localized version of an output stream. This method > -???? * takes an {@code OutputStream} and returns an > -???? * {@code OutputStream} equivalent to the argument in all respects > -???? * except that it is localized: as Unicode characters are written to > -???? * the stream, they are automatically converted to the local > -???? * character set. > -???? *

> -???? * If the argument is already a localized stream, it may be returned > -???? * as the result. > -???? * > -???? * @deprecated As of JDK 1.1, the preferred way to translate a > -???? * Unicode character stream into a byte stream in the local > encoding is via > -???? * the {@code OutputStreamWriter}, {@code BufferedWriter}, and > -???? * {@code PrintWriter} classes. > -???? * This method is subject to removal in a future version of Java SE. > -???? * > -???? * @param????? out OutputStream to localize > -???? * @return???? a localized output stream > -???? * @see??????? java.io.OutputStream > -???? * @see java.io.BufferedWriter#BufferedWriter(java.io.Writer) > -???? * @see > java.io.OutputStreamWriter#OutputStreamWriter(java.io.OutputStream) > -???? * @see java.io.PrintWriter#PrintWriter(java.io.OutputStream) > -???? */ > -??? @Deprecated(since="1.1", forRemoval=true) > -??? public OutputStream getLocalizedOutputStream(OutputStream out) { > -??????? return out; > -??? } > - > -??? /** > ????? * Returns the version of the Java Runtime Environment as a > {@link Version}. > ????? * > ????? * @return? the {@link Version} of the Java Runtime Environment From claes.redestad at oracle.com Wed Dec 6 22:54:22 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 6 Dec 2017 23:54:22 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> Message-ID: <5aa0208f-3fcf-6923-f645-a56ada1c2045@oracle.com> Hi Jonathan, On 2017-12-06 21:58, Jonathan Bluett-Duncan wrote: > Hi Claes, > > Looking at > http://cr.openjdk.java.net/~redestad/8193128/open.00/src/java.base/share/classes/java/util/ImmutableCollections.java.cdiff.html > , > there are sections labelled --- 646,657 ----?and --- 834,845 > ----?where lines like `Objects.requireNonNull(0 /* zero */);` are > written. I believe that they were supposed to be either removed or > made to be written like `Objects.requireNonNull(o /* lowercase o */)`. > Is my belief/understanding correct? nice catch! That was two fun little typos on my part (now why did we ever make 0 box to an Object...). Fixing in-place. Thanks! /Claes From forax at univ-mlv.fr Wed Dec 6 22:57:22 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 6 Dec 2017 23:57:22 +0100 (CET) Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> Message-ID: <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> Hi Claes, - both constructors of SubList should be package private, - in listIterator, i can be declared outside of the ListIterator as a local variable that would be captured by the anonymous class, so index is not used inside the anonymous class. Also you can use the diamond syntax for anonymous class now. public ListIterator listIterator(int index) { rangeCheck(index); ListIterator i = root.listIterator(offset + index); return new ListIterator<>() { ... - you can move "new IndexOutOfBoundsException" out of rangeCheck into outOfBoundsMsg, so the bytecode size of rangeCheck() will be smaller. - in Itr, please declare the constructor after the declaration of the fields, and assigning the cursor to zero is useless. - in equals, the code is weirdly asymmetric because e1 is typed as an Iterator, declaring it as an Iterator will make the code more symmetric. - in List12, you have a lot of useless @SuppressWarnings("unchecked") ?? in size(), get(), contains(), hashCode(), writeReplace(). - in ListN, again some useless @SuppressWarnings("unchecked") ?? in size(), get(), contains(), hashCode(), writeReplace() - the constructor of MapN(K,V) can be made a little more efficient (less getfield opcodes) if written like this MapN(K key, V value) { table = new Object[] { key, value }; size = 1; } and i do not understand why the field size is not declared @Stable anymore, ok, it can be equals to zero, but in that case the JIT will emit a move so it's better than always asking for a move (or i do not understand the semantics of @Stable ?) cheers, R?mi ----- Mail original ----- > De: "Claes Redestad" > ?: "core-libs-dev" , "Stuart Marks" > Envoy?: Mercredi 6 D?cembre 2017 21:21:55 > Objet: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() > Hi, > > please help review this patch to consolidate the number of > implementation classes returned by the static collection factories: > > http://cr.openjdk.java.net/~redestad/8193128/open.00/ > > I set out to explore options for addressing small inefficiencies we've > been running into, the latest after replacing use of @Stable arrays in > java.lang.invoke with List.of() (JDK-8184777): > > - List.indexOf deferred to the iterator in AbstractList, which check for > concurrent modification > - Warmup takes a bit longer due to having to warm up four different > classes and associated methods > - Slowdowns in certain code paths attributable to certain calls becoming > megamorphic > > Microbenchmark: > http://cr.openjdk.java.net/~redestad/scratch/less_immutables/ListMorphism.java > http://cr.openjdk.java.net/~redestad/scratch/less_immutables/benchmarks.jar > > The benchmark explores how call-sites behave when performing some naive > operations on a few different Lists. > > For every benchmark using List.of() there's a variant using ArrayList > for comparison: > > Baseline: > > Benchmark???????????????????????????? Mode? Cnt??? Score Error?? Units > ListMorphism.finalGetFromArrayList?? thrpt?? 25?? 92.659 ?? 3.058 ops/us > ListMorphism.finalGetFromList??????? thrpt?? 25? 335.245 ?? 0.244 ops/us > # 3.6x > ListMorphism.finalSumSizesArrayList? thrpt?? 25? 245.020 ?? 0.106 ops/us > ListMorphism.finalSumSizesList?????? thrpt?? 25? 335.107 ?? 0.439 ops/us > # 1.4x > ListMorphism.getFromArrayList??????? thrpt?? 25?? 70.343 ?? 0.972 ops/us > ListMorphism.getFromList???????????? thrpt?? 25?? 37.121 ?? 0.135 ops/us > # 0.53x > ListMorphism.getFromArrayList12????? thrpt?? 25 109.890 ?? 0.286? ops/us > ListMorphism.getFromList12?????????? thrpt?? 25? 109.552 ? 6.972? ops/us > # 1.0x > ListMorphism.sumSizesArrayList?????? thrpt?? 25? 131.467 ?? 4.672 ops/us > ListMorphism.sumSizesList??????????? thrpt?? 25?? 57.877 ?? 0.102 ops/us > # 0.45x > ListMorphism.sumSizesArrayList12???? thrpt?? 25 208.652 ? 11.294? ops/us > ListMorphism.sumSizesList12????????? thrpt?? 25? 227.269 ? 0.961? ops/us > # 1.1x > > The good: When dealing with List literals (the final* benchmarks), > List.of() allows really nice speed-ups compared to ArrayList. > > The bad: When not used as a literal, List.of() implementations at > call-sites can cause a substantial slowdown (megamorphic dispatch) > > The ugly: > > After some explorations[1], I narrowed in on the following experiment: > http://cr.openjdk.java.net/~redestad/scratch/less_immutables/webrev/ > > Basically: Merge List1 and List2 into a single class, List12, merge > List0 into ListN (List0 has a singleton instance, so might as well be an > instance of ListN). Same for Set0,1,2,N. Map0 is merged into MapN. > > This strikes a balance between throughput, footprint and slightly better > startup/warmup behavior. > > According to jol estimates[2] the size of List12 is the same as both > List1 and List2 in the current JDK implementation. Set12 is footprint > neutral compared to Set2 on all platforms but lose a few bytes on 64-bit > VMs compared to Set1. > > Benchmark???????????????????????????? Mode? Cnt??? Score?? Error Units > ListMorphism.finalGetFromArrayList?? thrpt?? 25?? 93.046 ? 0.569 ops/us > ListMorphism.finalGetFromList??????? thrpt?? 25? 335.280 ? 0.154 ops/us > # 3.6x > ListMorphism.finalSumSizesArrayList? thrpt?? 25? 244.595 ? 1.085 ops/us > ListMorphism.finalSumSizesList?????? thrpt?? 25? 303.351 ? 0.182 ops/us > # 1.2x > ListMorphism.getFromArrayList??????? thrpt?? 25?? 70.631 ? 0.070 ops/us > ListMorphism.getFromList???????????? thrpt?? 25?? 73.258 ? 2.955 ops/us > # 1.04x > ListMorphism.getFromArrayList12????? thrpt?? 25 109.921 ? 0.096? ops/us > ListMorphism.getFromList12?????????? thrpt?? 25? 127.392 ? 0.088? ops/us > # 1.16x > ListMorphism.sumSizesArrayList?????? thrpt?? 25? 131.393 ? 4.882 ops/us > ListMorphism.sumSizesList??????????? thrpt?? 25? 107.686 ? 5.286 ops/us > # 0.82x > ListMorphism.sumSizesArrayList12???? thrpt?? 25? 212.350 ? 0.134? ops/us > ListMorphism.sumSizesList12????????? thrpt?? 25? 198.778 ? 0.479? ops/us > # 0.94x > > The experiment has a flag to change number of specialized List/Set/Map > classes (-Djdk.ImmutableCollections.specializations=0|1|2, default=2). > > 1 specialization (List1 + ListN, Set1 + SetN) is more or less the same > as 2, some ~1-2% improvements, mainly in sumSizes micros. > > 0 specializations (List-, Set, MapN only) achieves a small increase in > throughput on some micros by ensuring callsites stay monomorphic, but > it's not very substantial by my measures (~5%, but mostly in sumSizes > micros). > > Keeping the footprint more or less the same, while evening out a few > rough edges and improving startup and static footprint seems like the > overall best option. An alternative would be to keep the experimental > flag, but I don't think a 5% gain on micros warrants the extra > maintenance burden and testing that entails. > > The proposed patch is more or less this experiment with 2 > specializations, but having removed the flag and code movement needed to > support it removed (along with a fix in the writeReplace methods of > List12/Set12) > > Thanks! > > /Claes > > [1] Older experiments: > http://cr.openjdk.java.net/~redestad/immutable/list12N.0/ > ?-- simply merge L0 into LN (still have L1, L2 and LN) - nothing really > changed, though > > http://cr.openjdk.java.net/~redestad/immutable/list1N.0/ > ?-- L0 merged into LN, drop L2. Substantial performance boost on > micros. Footprint drop for 2-element lists. > > http://cr.openjdk.java.net/~redestad/immutable/listNdouble.0/ > ?-- L0+L1+L2+LN merged into one implementation. Same footprint with a > single class, but loses a lot on throughput in micros. > > http://cr.openjdk.java.net/~redestad/immutable/listNsingle.0/ > ?-- L0+L1+LN merged, drop L2. Simplification of the previous. Like the > list1N.0 experiment in footprint, but a loss in throughput on all measures. > > No approach seemed a win across the board here: we either had to accept > a footprint reduction, or see throughput suffer dramatically. > > [2] http://openjdk.java.net/projects/code-tools/jol/ From claes.redestad at oracle.com Wed Dec 6 23:08:07 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 7 Dec 2017 00:08:07 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> Message-ID: Hi, On 2017-12-06 22:20, Martin Buchholz wrote: > Guava struggled with this as well with their immutable collections.? > You could look at their revision history (I haven't). Maybe they got > rid of SingletonImmutableList for the same reason? I've not looked at guava sources, but I've seen comments alluding to similar issues there, yes. There's definitely different trade-offs here, and I'm making the conservative choice of taking the one that is in effect and trying to make it just a bit better. /Claes From joe.darcy at oracle.com Wed Dec 6 23:15:57 2017 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 6 Dec 2017 15:15:57 -0800 Subject: Fw: Question about JEP 306. In-Reply-To: <5A275C3D.4060208@oracle.com> References: <2a493b34-a717-852b-23a8-d8827be0f61e@oracle.com> <5A275C3D.4060208@oracle.com> Message-ID: <30fb449a-63de-63ee-b7e9-b8b86aa63301@oracle.com> PS With a more concrete example below.... On 12/5/2017 6:55 PM, Joseph D. Darcy wrote: > Hello, > > On 12/5/2017 5:07 PM, David Holmes wrote: >> Adding core-libs-dev as both mailing lists are named in this JEP. > [snip] >>> It should also be the case >>> that there should be a round half up for the decimal place >>> beyond the last one in the float or double range, to help support >>> all notions of >>> >>> (pow(sqrt(3),2) == 3) //true > > Putting aside non-finite values like NaN and infinities, this identity > is difficult to have hold in any fixed-precision system, including > IEEE-style floating-point. The mathematical square root function in > general returns irrational results for rational inputs so the result > is fundamentally approximated. If my rusty calculus is correct, for x > >= 0.25, the derivative of sqrt of x has magnitude less than one and > is positive, meaning there exists some set of multiple input values > that must get mapped to the same output value. Therefore, the > information to do an exact inverse operation based on the output is > not present just from a pigeon hole principle argument. Consider the square root function on the interval [1, 2), that is the half-open region that includes 1 but does not include 2. This corresponds to the floating-point numbers with an exponent of 0. There are 2^52 floating-point values with this exponent. Mathematically, sqrt(1) = 1 and sqrt(2) ~= 1.414... Therefore, in floating-point, for values 1.0 <= x < 2.0, the result of sqrt(x) also has an exponent of 0. Now we see that there are 2^52 floating-point values with an exponent of 0 and taking a square root of those values will only yield about 0.414 * 2^52 distinct answers. Therefore, we see that using a fixed floating-point precision, the square root function on [1.0, 2.0) *cannot* be inverted losslessly since only a minority of the input elements have distinct answers. HTH, -Joe From martinrb at google.com Wed Dec 6 23:17:45 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Dec 2017 15:17:45 -0800 Subject: Android and Log4j In-Reply-To: <1045760895.1566792.1512597808721.JavaMail.zimbra@u-pem.fr> References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> <1045760895.1566792.1512597808721.JavaMail.zimbra@u-pem.fr> Message-ID: I can neither confirm nor deny that dx is going to die, but it sure looks like "Google" is replacing it with "D8". https://android-developers.googleblog.com/2017/08/next-generation-dex-compiler-now-in.html On Wed, Dec 6, 2017 at 2:03 PM, wrote: > > > ------------------------------ > > *De: *"Martin Buchholz" > *?: *"Remi Forax" > *Cc: *"Ralph Goers" , "core-libs-dev" < > core-libs-dev at openjdk.java.net> > *Envoy?: *Mercredi 6 D?cembre 2017 22:35:42 > *Objet: *Re: Android and Log4j > > Very latest Android Studio comes with two compilers, dx and d8. > https://developer.android.com/studio/preview/features/index.html > I hear that both compilers know to skip module-info files. > > > I hope dx will die soon, i had to patch it once and i was messy, a > register allocator is usually not a fun code but in case of dx, it is > hidden by several layers of abstraction which made me cringe. > > R?mi > > > > On Mon, Dec 4, 2017 at 12:03 PM, Martin Buchholz > wrote: > >> >> >> On Sun, Dec 3, 2017 at 11:02 AM, Remi Forax wrote: >> >>> Ask Google to fix dx, >>> dx should ignore the module-info.class and everything inside >>> META-INF/versions (at least it's a first simple patch). >>> >> >> Hi, I'm "Google", sort of. Friends tell me that dx is getting fixed. >> >> > From claes.redestad at oracle.com Thu Dec 7 00:12:24 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 7 Dec 2017 01:12:24 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> Message-ID: <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> Hi R?mi, On 2017-12-06 23:57, Remi Forax wrote: > Hi Claes, > - both constructors of SubList should be package private, deal! > - in listIterator, i can be declared outside of the ListIterator as a local variable that would be captured by the anonymous class, > so index is not used inside the anonymous class. Also you can use the diamond syntax for anonymous class now. > > public ListIterator listIterator(int index) { > rangeCheck(index); > ListIterator i = root.listIterator(offset + index); > return new ListIterator<>() { > ... Sure. > - you can move "new IndexOutOfBoundsException" out of rangeCheck into outOfBoundsMsg, so the bytecode size of rangeCheck() will be smaller. I had already done so for the outer class, and realized I had forgotten to clean this part up: SubList is now final, inherits from AbstractImmutableList instead of AbstractList and reuses more of the shared code. I also moved 'implements Serializable' from AbstractImmutableList to List12 and ListN respectively to not change the behavior that List.of(0,1) is serializable while List.of(0,1).subList(0,1) isn't. > - in Itr, please declare the constructor after the declaration of the fields, > and assigning the cursor to zero is useless. Done. > > - in equals, the code is weirdly asymmetric because e1 is typed as an Iterator, declaring it as an Iterator will make the code more symmetric. I also realized I had missed an opportunity to take advantage of the fact that elements returned from e1 are guaranteed to be non-null. Might as well. > > - in List12, you have a lot of useless @SuppressWarnings("unchecked") ?? > in size(), get(), contains(), hashCode(), writeReplace(). > > - in ListN, again some useless @SuppressWarnings("unchecked") ?? > in size(), get(), contains(), hashCode(), writeReplace() I don't know how these snuck in, but my IDE was pleasantly hiding these away. I think I cleaned out all the useless ones. > - the constructor of MapN(K,V) can be made a little more efficient (less getfield opcodes) if written like this > > MapN(K key, V value) { > table = new Object[] { key, value }; > size = 1; > } Oops, this was a leftover from my experiment where I adapted MapN to implement Map1 when setting specializers=0. Removed. > > and i do not understand why the field size is not declared @Stable anymore, > ok, it can be equals to zero, but in that case the JIT will emit a move > so it's better than always asking for a move (or i do not understand the semantics of @Stable ?) Hmm, I'm under the impression @Stable brings no additional value to a final non-array fields (definitely important for arrays). I might have been guilty of adding @Stable to more fields than necessary in these implementations in the first place. I've reverted this removal and will add a note to investigate separately if we can more systematically clean them up. http://cr.openjdk.java.net/~redestad/8193128/open.01/ Thanks! /Claes From mandy.chung at oracle.com Thu Dec 7 00:33:51 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 6 Dec 2017 16:33:51 -0800 Subject: Review Request: JDK-8193159: Reduce the number of classes loaded due to NativeLibrary Message-ID: A tiny startup fix - useArrayDeque instead of LinkedList for ClassLoader.NativeLibrary which is typically loaded at startup for example when loading a JAR file. Thanks Mandy diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java --- a/src/java.base/share/classes/java/lang/ClassLoader.java +++ b/src/java.base/share/classes/java/lang/ClassLoader.java @@ -38,6 +38,7 @@ ?import java.security.PrivilegedAction; ?import java.security.ProtectionDomain; ?import java.security.cert.Certificate; +import java.util.ArrayDeque; ?import java.util.Arrays; ?import java.util.Collections; ?import java.util.Deque; @@ -45,7 +46,6 @@ ?import java.util.HashMap; ?import java.util.HashSet; ?import java.util.Hashtable; -import java.util.LinkedList; ?import java.util.Map; ?import java.util.NoSuchElementException; ?import java.util.Objects; @@ -2496,7 +2496,7 @@ ???????? } ???????? // native libraries being loaded -??????? static Deque nativeLibraryContext = new LinkedList<>(); +??????? static Deque nativeLibraryContext = new ArrayDeque<>(); ???????? /* ????????? * The run() method will be invoked when this class loader becomes From igor.ignatyev at oracle.com Thu Dec 7 00:35:46 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 6 Dec 2017 16:35:46 -0800 Subject: RFR(XS) : 8181118 : update java/time tests to use RandomFactory from the top level testlibrary Message-ID: http://cr.openjdk.java.net/~iignatyev/8181118/webrev.00 > 115 lines changed: 0 ins; 108 del; 7 mod; Hi all, could you please review this small fix for java/time tests? RandomFactory has been moved to the top level testlibrary by 8180805[1], but java/time weren't updated when b/c jtreg didn't use external root for libraries listed in lib.dirs test property file[2]. it has been fixed in jtreg4.2b09, now java/time tests can be updated to use top level testlibrary and @deprecated RandomFactory (from test/jdk/lib/testlibrary) can be removed. webrev: http://cr.openjdk.java.net/~iignatyev/8181118/webrev.00 JBS: https://bugs.openjdk.java.net/browse/JDK-8181118 testing: java/time/jck and java/time/tests [1] https://bugs.openjdk.java.net/browse/JDK-8180805 [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7901987 Thanks, -- Igor From hboehm at google.com Thu Dec 7 00:38:09 2017 From: hboehm at google.com (Hans Boehm) Date: Wed, 6 Dec 2017 16:38:09 -0800 Subject: Finalization and dead references: another proposal Message-ID: We're still trying to deal with a fair amount of code that implicitly assumes that finalization or similar clean-up will not occur while a pointer to the affected object is in scope. Which is of course not true. As a reminder, the canonical offending usage (anti-)pattern with (deprecated, but easier to write) finalizers is class Foo { private long mPtrToNativeFoo; private static native void nativeDeallocate(long nativePtr); private static native void nativeDoSomething(long nativePtr, long anotherNativePtr); protected void finalize() { ... nativeDeallocate(mPtrToNativeFoo); ... } public void doSomething(Foo another) { ... nativeDoSomething(mPtrToNativeFoo, another.mPtrToNativeFoo) ... } ... } This is subtly incorrect in that, while executing the final call to doSomething() on a particular object, just after retrieving mPtrToNativeFoo and another.mPtrToNativeFoo, but before invoking nativeDoSomething(), the garbage collector may run, and "this" and "another" may be finalized, deallocating the native objects their mPtrToNativeFoos refer to. When nativeDoSomething() finally does run, it may see dangling pointers. Examples using java.lang.ref or Cleaners (or even WeakHashMap, if you must) instead of finalize() are as easy to construct, but a bit longer. (The finalizer version is also arguably incorrect in other ways that are not relevant here. Pretend this were written in terms of PhantomReferences.) It is easily possible to construct 100% Java code with the same problem. Instead of mPtrToNativeFoo, each object stores an integer handle that is used to access additional Java state logically associated with the object. But the native pointer case seems to dominate in practice. Various solutions to this have been proposed, but none seem quite attractive enough that I actually feel comfortable asking people to update their code to use them. Noteworthy proposals include: 0) Explicitly call close() on such objects. Great idea when it works. In general it doesn't, since the code needs to know when the enclosing Java object is no longer needed. If we always knew that we wouldn't need a GC. 1) Various hacks to keep variables live, e.g. the one based on synchronized blocks I advocated in my 2004 JavaOne talk. These are slow and ugly, as we've always admitted. Nobody used them. Incorrect won over slow, ugly, and complicated ~100% of the time. 2) Java 9's reachabilityFence(). This is better. It can be implemented so that it's no longer slow. But in many common cases, it's still quite ugly. For example, the call to nativeDoSomething() above must be followed by two reachabilityFences, one on this and one on another. And to do this really correctly, the whole thing would often need to be in a try...finally block. And in reality code following this pattern usually doesn't have just a single doSomething method that needs this treatment, but may easily have dozens. And the rules for placing reachabilityFences can become quite subtle if there are e.g. locally allocated objects involved. My assessment is that this isn't good enough. People may no longer write incorrect code 100% of the time, but I'd bet on 70%+. 3) JNI functions can be rewritten, so that the containing Java object is passed in addition to the native pointers. Somewhat accidentally, this happens to be roughly free for single argument functions. (Delete "static".) It adds overhead in other cases, like the one above, and the rewriting can get somewhat convoluted. In some cases, it doesn't work at all. AFAIK, it's never actually guaranteed to be correct; it works because standard implementations don't optimize across the language boundary. That's not too likely to change. Maybe. 4) We could change the language spec to always prohibit premature finalization/cleaning in cases like the above. I could personally live with that solution, and proposed it internally here in the past. But it doesn't seem to go over well among implementers. And AFAICT, doing it well requires significant tooling changes, in that we do want to reliably treat local variables as dead once they go out of scope, a piece of information that doesn't seem to be reliably preserved in class files. One could argue that the current class file design implicitly assumes that we can do dead variable elimination. After going back and forth on this, my conclusion is that we need a linguistic mechanism for identifying the case in which the garbage collector is being used to managed external resources associated with a field. A (still slowly evolving) proposal to add an annotation to do so is at https://docs.google.com/document/d/1yMC4VzZobMQTrVAraMy7xBdWPCJ0p7G5r_43yaqsj-s/edit?usp=sharing In many ways, this is inherently a compromise. But in the vast majority of cases, it greatly reduces the required source code changes over all but (4) above. And I think I could usually explain to an author of currently broken code in under 5 minutes exactly what they need to do to fix it. And I wouldn't have to lie much to do so. I don't think (0)-(3) above share that property. This has already benefited from comments provided by a few people. (Thanks to Doug, Jeremy, and Martin, among others.) But I would really like more feedback, including suggestions about how to proceed. Hans From paul.sandoz at oracle.com Thu Dec 7 01:16:18 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 6 Dec 2017 17:16:18 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> Message-ID: <8FE723F7-9127-4B6C-A09F-2E4AFDE0AE5F@oracle.com> > On 6 Dec 2017, at 16:12, Claes Redestad wrote: >> >> and i do not understand why the field size is not declared @Stable anymore, >> ok, it can be equals to zero, but in that case the JIT will emit a move >> so it's better than always asking for a move (or i do not understand the semantics of @Stable ?) > > Hmm, I'm under the impression @Stable brings no additional value to a final non-array fields (definitely important for arrays). > It can, since final fields are not treated as really final (unless in java.lang.invoke package, where it?s as if all such fields are annotated with @Stable). C2 will treat the field value a constant value if the collection is held in a static final field. Paul. > I might have been guilty of adding @Stable to more fields than necessary in these implementations in the first place. I've > reverted this removal and will add a note to investigate separately if we can more systematically clean them up. > > http://cr.openjdk.java.net/~redestad/8193128/open.01/ > > Thanks! > > /Claes From david.holmes at oracle.com Thu Dec 7 02:02:15 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 7 Dec 2017 12:02:15 +1000 Subject: Java SE Maths questions. In-Reply-To: References: Message-ID: <5883ca2a-f0a6-703a-0658-f2846a4a96d6@oracle.com> cc'ing core-libs-dev. Despite the notation on JEP 306 I don't think there's anyone from hotspot active in this area and most of the below is about SE library classes. David On 7/12/2017 11:39 AM, A Z wrote: > These are Java SE questions. > > -Is it likely that JEP 306 could be updated in a Java update, earlier? > Apart from the assertion 'will not fix', accurate floating point > is a massive requirement. It isn't very possible to do within-range > float or double multiplies, divides, powers, roots, or even things > in other classes like obtaining the distances between two points > (in Java2D and Java3D). > > -Could there be a BigDecimal version of StrictMath > being put in to the standard libraries, under Java's own umbrella, > and not as someone else's separate library? > From martinrb at google.com Thu Dec 7 02:08:49 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Dec 2017 18:08:49 -0800 Subject: Review Request: JDK-8193159: Reduce the number of classes loaded due to NativeLibrary In-Reply-To: References: Message-ID: Google decided that LinkedList is almost never the best choice, so any use of LinkedList is discouraged. ArrayDeque's backing array never shrinks, so you might want to give it an explicit initial size. On Wed, Dec 6, 2017 at 4:33 PM, mandy chung wrote: > A tiny startup fix - useArrayDeque instead of LinkedList for > ClassLoader.NativeLibrary which is typically loaded at startup for example > when loading a JAR file. > > Thanks > Mandy > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java > b/src/java.base/share/classes/java/lang/ClassLoader.java > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > @@ -38,6 +38,7 @@ > import java.security.PrivilegedAction; > import java.security.ProtectionDomain; > import java.security.cert.Certificate; > +import java.util.ArrayDeque; > import java.util.Arrays; > import java.util.Collections; > import java.util.Deque; > @@ -45,7 +46,6 @@ > import java.util.HashMap; > import java.util.HashSet; > import java.util.Hashtable; > -import java.util.LinkedList; > import java.util.Map; > import java.util.NoSuchElementException; > import java.util.Objects; > @@ -2496,7 +2496,7 @@ > } > > // native libraries being loaded > - static Deque nativeLibraryContext = new > LinkedList<>(); > + static Deque nativeLibraryContext = new > ArrayDeque<>(); > > /* > * The run() method will be invoked when this class loader becomes > From joe.darcy at oracle.com Thu Dec 7 02:15:29 2017 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Wed, 06 Dec 2017 18:15:29 -0800 Subject: Java SE Maths questions. In-Reply-To: <5883ca2a-f0a6-703a-0658-f2846a4a96d6@oracle.com> References: <5883ca2a-f0a6-703a-0658-f2846a4a96d6@oracle.com> Message-ID: <5A28A441.6080901@oracle.com> On 12/6/2017 6:02 PM, David Holmes wrote: > cc'ing core-libs-dev. Despite the notation on JEP 306 I don't think > there's anyone from hotspot active in this area and most of the below > is about SE library classes. > > David > > On 7/12/2017 11:39 AM, A Z wrote: >> These are Java SE questions. >> >> -Is it likely that JEP 306 could be updated in a Java update, earlier? >> Apart from the assertion 'will not fix', accurate floating point >> is a massive requirement. It isn't very possible to do within-range >> float or double multiplies, divides, powers, roots, or even things >> in other classes like obtaining the distances between two points >> (in Java2D and Java3D). >> >> -Could there be a BigDecimal version of StrictMath >> being put in to the standard libraries, under Java's own umbrella, >> and not as someone else's separate library? >> The answer to this question hasn't changed since I answered it yesterday [1] or from when the same question was asked and answered by via a web bug a month ago. [2] If the same question is asked again tomorrow in this or another venue the answer will be the same and I will not respond again explicitly. -Joe [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050357.html [2] https://bugs.openjdk.java.net/browse/JDK-8190947 From vitalyd at gmail.com Thu Dec 7 02:27:12 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 07 Dec 2017 02:27:12 +0000 Subject: Finalization and dead references: another proposal In-Reply-To: References: Message-ID: On Wed, Dec 6, 2017 at 7:38 PM Hans Boehm wrote: > We're still trying to deal with a fair amount of code that implicitly > assumes that finalization or similar clean-up will not occur while a > pointer to the affected object is in scope. Which is of course not true. > > As a reminder, the canonical offending usage (anti-)pattern with > (deprecated, but easier to write) finalizers is > > class Foo { > private long mPtrToNativeFoo; > > private static native void nativeDeallocate(long nativePtr); > private static native void nativeDoSomething(long nativePtr, long > anotherNativePtr); > > protected void finalize() { ... nativeDeallocate(mPtrToNativeFoo); ... > } > > public void doSomething(Foo another) { ... > nativeDoSomething(mPtrToNativeFoo, another.mPtrToNativeFoo) ... } > ... > } > > This is subtly incorrect in that, while executing the final call to > doSomething() on a particular object, just after retrieving mPtrToNativeFoo > and another.mPtrToNativeFoo, but before invoking nativeDoSomething(), the > garbage collector may run, and "this" and "another" may be finalized, > deallocating the native objects their mPtrToNativeFoos refer to. > When nativeDoSomething() finally does run, it may see dangling pointers. > > Examples using java.lang.ref or Cleaners (or even WeakHashMap, if you must) > instead of finalize() are as easy to construct, but a bit longer. (The > finalizer version is also arguably incorrect in other ways that are not > relevant here. Pretend this were written in terms of PhantomReferences.) > > It is easily possible to construct 100% Java code with the same problem. > Instead of mPtrToNativeFoo, each object stores an integer handle that is > used to access additional Java state logically associated with the object. > But the native pointer case seems to dominate in practice. > > Various solutions to this have been proposed, but none seem quite > attractive enough that I actually feel comfortable asking people to update > their code to use them. Noteworthy proposals include: > > 0) Explicitly call close() on such objects. Great idea when it works. In > general it doesn't, since the code needs to know when the enclosing Java > object is no longer needed. If we always knew that we wouldn't need a GC. > 1) Various hacks to keep variables live, e.g. the one based on synchronized > blocks I advocated in my 2004 JavaOne talk. These are slow and ugly, as > we've always admitted. Nobody used them. Incorrect won over slow, ugly, and > complicated ~100% of the time. > 2) Java 9's reachabilityFence(). This is better. It can be implemented so > that it's no longer slow. But in many common cases, it's still quite ugly. > For example, the call to nativeDoSomething() above must be followed by two > reachabilityFences, one on this and one on another. And to do this really > correctly, the whole thing would often need to be in a try...finally block. > And in reality code following this pattern usually doesn't have just a > single doSomething method that needs this treatment, but may easily have > dozens. And the rules for placing reachabilityFences can become quite > subtle if there are e.g. locally allocated objects involved. My assessment > is that this isn't good enough. People may no longer write incorrect code > 100% of the time, but I'd bet on 70%+. > 3) JNI functions can be rewritten, so that the containing Java object is > passed in addition to the native pointers. Somewhat accidentally, this > happens to be roughly free for single argument functions. (Delete > "static".) It adds overhead in other cases, like the one above, and the > rewriting can get somewhat convoluted. In some cases, it doesn't work at > all. AFAIK, it's never actually guaranteed to be correct; it works because > standard implementations don't optimize across the language boundary. > That's not too likely to change. Maybe. > 4) We could change the language spec to always prohibit premature > finalization/cleaning in cases like the above. I could personally live with > that solution, and proposed it internally here in the past. But it doesn't > seem to go over well among implementers. And AFAICT, doing it well requires > significant tooling changes, in that we do want to reliably treat local > variables as dead once they go out of scope, a piece of information that > doesn't seem to be reliably preserved in class files. One could argue that > the current class file design implicitly assumes that we can do dead > variable elimination. > > After going back and forth on this, my conclusion is that we need a > linguistic mechanism for identifying the case in which the garbage > collector is being used to managed external resources associated with a > field. So kind of the opposite of WeakReference - a SuperStrongReference :). Kidding aside, it seems like the way you?d want to encapsulate this at the language level is via a type that the JVM intrinsically knows about; in this way it?s similar to the reference types today. An annotation probably does the trick when the value doesn?t escape from the enclosing instance but I?ve no idea if that assumption covers enough code to warrant this approach. AFAICT, if the value escapes into an instance of another type that doesn?t annotate its field then all bets are off. Having a wrapper type would at least make it harder to leak the native handle vs the annotation approach. But of course the wrapper comes with footprint and indirection costs. Maybe Valhalla could allow exposing some magic value type that?s a zero-cost wrapper but preserves the type information the JIT can track? > A (still slowly evolving) proposal to add an annotation to do so is > at > > > https://docs.google.com/document/d/1yMC4VzZobMQTrVAraMy7xBdWPCJ0p7G5r_43yaqsj-s/edit?usp=sharing > > In many ways, this is inherently a compromise. But in the vast majority of > cases, it greatly reduces the required source code changes over all but (4) > above. And I think I could usually explain to an author of currently broken > code in under 5 minutes exactly what they need to do to fix it. And I > wouldn't have to lie much to do so. I don't think (0)-(3) above share that > property. > > This has already benefited from comments provided by a few people. (Thanks > to Doug, Jeremy, and Martin, among others.) But I would really like more > feedback, including suggestions about how to proceed. > > Hans > -- Sent from my phone From vitalyd at gmail.com Thu Dec 7 03:03:30 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 6 Dec 2017 22:03:30 -0500 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> Message-ID: On Wed, Dec 6, 2017 at 4:01 PM, Jason Mehrens wrote: > Brian, > > My understand is JDK-6533165 is moving the bytes that are rarely invoked > to a cold method. > > Therefore: > ==== > if (closed) { > throw new IOException("Stream closed"); > } > ==becomes=== > if (closed) { > throw sc(); > } > > private static IOException sc() { > return new IOException("Stream closed"); > } > ================================ > > Since there is no string concatenation in the IOE message there are only a > few bytes that can be shaved off. I didn't benchmark it either case but I > would assume it would matter for the nullOutputStream since the write > methods could be invoked multiple times. > >From a performance angle, I'd be more concerned with the calls to Objects.xyz() methods there. Unless something has changed in the JIT recently, those are susceptible to profile pollution and can cause missed optimizations. I'd inline those methods manually to give these methods their own profiles. > > Jason > ________________________________________ > From: Brian Burkhalter > Sent: Wednesday, December 6, 2017 2:05 PM > To: Jason Mehrens > Cc: core-libs-dev > Subject: Re: RFR 4358774: Add null InputStream and OutputStream > > Jason, > > On Dec 6, 2017, at 11:54 AM, Jason Mehrens mailto:jason_mehrens at hotmail.com>> wrote: > > For nullInputStream would it make any sense to use the > ByteArrayInputStream with a (private static) empty byte array? Maybe > 'return new ByteArrayInputStream("".getBytes());'? One side effect is > that mark support returns true. > > As we are trying to retain the behavior of closed streams throwing > IOExceptions I don?t think that BAIS would work here. > > Does it make sense to follow the advice inhttps://bugs.openjdk.java. > net/browse/JDK-6533165 with regards to streams being closed? > > I don?t know exactly what you intend here. In the linked issue there is > information to impart, namely the index and the size. Here there is only > the indication that the stream is closed. It?s not clear to me what could > be done here. > > Thanks, > > Brian > From mark.reinhold at oracle.com Thu Dec 7 03:35:34 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 06 Dec 2017 22:35:34 -0500 Subject: RFR(s): 8177681: Remove methods Runtime.getLocalized{Input, Output}Stream In-Reply-To: <418e5290-d0d5-c21c-9fde-258ef53c75fd@oracle.com> References: <418e5290-d0d5-c21c-9fde-258ef53c75fd@oracle.com> Message-ID: <20171206223534.921941454@eggemoggin.niobe.net> 2017/12/6 17:33:36 -0500, stuart.marks at oracle.com: > Please review the removal of these methods, which were part of an obsolete > internationalization mechanism. They were deprecated in JDK 1.1 and deprecated > for removal in JDK 9. As far as I can see, there is no usage of these methods > anywhere. As the developer responsible for deprecating these methods in JDK 1.1 -- way back in 1996 --- I heartily approve of this change! - Mark From mandy.chung at oracle.com Thu Dec 7 05:04:09 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 6 Dec 2017 21:04:09 -0800 Subject: Review Request: JDK-8193159: Reduce the number of classes loaded due to NativeLibrary In-Reply-To: References: Message-ID: <57df1268-51f4-32ad-6c9d-0ecef465335a@oracle.com> On 12/6/17 6:08 PM, Martin Buchholz wrote: > Google decided that LinkedList is almost never the best choice, so any > use of LinkedList is discouraged. > > ArrayDeque's backing array never shrinks, so you might want to give it > an explicit initial size. > The initial size is 16 which is okay.? I can make it smaller instead: -??????? static Deque nativeLibraryContext = new LinkedList<>(); +??????? static Deque nativeLibraryContext = new ArrayDeque<>(8); Mandy > On Wed, Dec 6, 2017 at 4:33 PM, mandy chung > wrote: > > A tiny startup fix - useArrayDeque instead of LinkedList for > ClassLoader.NativeLibrary which is typically loaded at startup for > example when loading a JAR file. > > Thanks > Mandy > > diff --git > a/src/java.base/share/classes/java/lang/ClassLoader.java > b/src/java.base/share/classes/java/lang/ClassLoader.java > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > @@ -38,6 +38,7 @@ > ?import java.security.PrivilegedAction; > ?import java.security.ProtectionDomain; > ?import java.security.cert.Certificate; > +import java.util.ArrayDeque; > ?import java.util.Arrays; > ?import java.util.Collections; > ?import java.util.Deque; > @@ -45,7 +46,6 @@ > ?import java.util.HashMap; > ?import java.util.HashSet; > ?import java.util.Hashtable; > -import java.util.LinkedList; > ?import java.util.Map; > ?import java.util.NoSuchElementException; > ?import java.util.Objects; > @@ -2496,7 +2496,7 @@ > ???????? } > > ???????? // native libraries being loaded > -??????? static Deque nativeLibraryContext = new > LinkedList<>(); > +??????? static Deque nativeLibraryContext = new > ArrayDeque<>(); > > ???????? /* > ????????? * The run() method will be invoked when this class > loader becomes > > From ralph.goers at dslextreme.com Thu Dec 7 05:25:30 2017 From: ralph.goers at dslextreme.com (Ralph Goers) Date: Wed, 6 Dec 2017 22:25:30 -0700 Subject: Android and Log4j In-Reply-To: References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> Message-ID: Martin, Do they also ignore the class files in META-INF/versions? Ralph > On Dec 6, 2017, at 2:35 PM, Martin Buchholz wrote: > > Very latest Android Studio comes with two compilers, dx and d8. > https://developer.android.com/studio/preview/features/index.html > I hear that both compilers know to skip module-info files. > > On Mon, Dec 4, 2017 at 12:03 PM, Martin Buchholz > wrote: > > > On Sun, Dec 3, 2017 at 11:02 AM, Remi Forax > wrote: > Ask Google to fix dx, > dx should ignore the module-info.class and everything inside META-INF/versions (at least it's a first simple patch). > > Hi, I'm "Google", sort of. Friends tell me that dx is getting fixed. > > From goetz.lindenmaier at sap.com Thu Dec 7 11:20:40 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 7 Dec 2017 11:20:40 +0000 Subject: FW: RFR(M): 8189102: All tools should support -?, -h and --help In-Reply-To: <8aa25976d496465fac00a4ae3628e533@sap.com> References: <8aa25976d496465fac00a4ae3628e533@sap.com> Message-ID: <456c694ec25a42658af1b8600aaa136d@sap.com> Hi, ... missed some lists in my first post ... I prepared a fifth webrev for this change. Please review. It incorporates the changes requested by the CSR reviewers (not to remove docuemtation of '-help' where is was documented before) and the changes proposed by Kumar: http://cr.openjdk.java.net/~goetz/wr17/8189102-helpMessage/webrev.05/ See also the information in the webrev itself, there are also patch files with the incremental builds. This change contains fixes for some langtool tests. I ran the following test suites on it: hotspot, jdk, langtools, nashorn, jaxp, most of them on all the platforms we build. Best regards, Goetz. > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Mittwoch, 11. Oktober 2017 22:07 > To: hotspot-dev developers > Subject: RFR(M): 8189102: All tools should support -?, -h and --help > > Hi > > The tools in jdk should all show the same behavior wrt. help flags. > This change normalizes the help flags of a row of the tools in the jdk. > Java accepts -?, -h and --help, thus I changed the tools to support > these, too. Some tools exited with '1' after displaying the help message, > I turned this to '0'. > > Maybe this is not the right mailing list for this, please advise. > > Please review this change. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/wr17/8189102-helpMessage/webrev.01/ > > In detail, this fixes the help message of the following tools: > jar -? -h --help; added -?. > jarsigner -? -h --help; added --help. -help accepted but not documented. > javac -? --help; added -?. Removed -help. -h is taken for other purpose > javadoc -? -h --help; added -h -?. Removed -help > javap -? -h --help; added -h. -help accepted but no more documented. > jcmd -? -h --help; added -? --help. -help accepted but no more > documented. Changed return value to '0' > jdb -? -h --help; added -? -h --help. -help accepted but no more > documented. > jdeprscan -? -h --help; added -? > jinfo -? -h --help; added -? --help. -help accepted but no more > documented. > jjs -h --help; Replaced -help by --help. Adding more not straight > forward. > jps -? -h --help; added -? --help. -help accepted but no more > documented. > jshell -? -h --help; added -? > jstat -? -h --help; added -h --help. -help accepted but no more > documented. > > Best regards, > Goetz. From andrej.golovnin at gmail.com Thu Dec 7 11:40:21 2017 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Thu, 7 Dec 2017 12:40:21 +0100 Subject: Add EnumMap.keyType() and EnumSet.elementType() Message-ID: Hi all, it would be nice if we would have access to the class of the enum type used to create an EnumMap or an EnumSet. This is usefull when you write a custom serialization library and would like to serialize/deserialize an empty EnumMap or an empty EnumSet. For the empty EnumSet there is a workaround to get the enum class: EnumSet empty = EnumSet.noneOf(MyEnum.class); EnumSet tmp = EnumSet.complementOf(empty); Class elementType = tmp.iterator().next().getClass(); But this only works when the enum class has at least one enum. For EnumMap there is no such workaround at all and we have to use the Reflection API. And you know the warnings since Java 9 :-) : WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by c.r.r.k.EnumMapSerializer to field java.util.EnumMap.keyType WARNING: Please consider reporting this to the maintainers of c.r.r.k.EnumMapSerializer WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release So to avoid this warning and to avoid that our application stops to work with a future release of Java I would like to propose to add the following two methods: EnumMap: /** * Returns the {@code Class} object of the key type for this enum map. * * @return the {@code Class} object of the key type for this enum map. * * @since 10 */ public Class keyType() EnumSet: /** * Returns the {@code Class} object of all the elements of this set. * * @return the {@code Class} object of all the elements of this set. * * @since 10 */ public Class elementType() The suggested change is attached as diff. Best reagrds, Andrej Golovnin -------------- next part -------------- diff --git a/src/java.base/share/classes/java/util/EnumMap.java b/src/java.base/share/classes/java/util/EnumMap.java --- a/src/java.base/share/classes/java/util/EnumMap.java +++ b/src/java.base/share/classes/java/util/EnumMap.java @@ -179,6 +179,17 @@ } } + /** + * Returns the {@code Class} object of the key type for this enum map. + * + * @return the {@code Class} object of the key type for this enum map. + * + * @since 10 + */ + public Class keyType() { + return keyType; + } + // Query Operations /** diff --git a/src/java.base/share/classes/java/util/EnumSet.java b/src/java.base/share/classes/java/util/EnumSet.java --- a/src/java.base/share/classes/java/util/EnumSet.java +++ b/src/java.base/share/classes/java/util/EnumSet.java @@ -365,6 +365,17 @@ } /** + * Returns the {@code Class} object of all the elements of this set. + * + * @return the {@code Class} object of all the elements of this set. + * + * @since 10 + */ + public Class elementType() { + return elementType; + } + + /** * Adds the specified range to this enum set, which is empty prior * to the call. */ From scolebourne at joda.org Thu Dec 7 12:09:40 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 7 Dec 2017 12:09:40 +0000 Subject: Add EnumMap.keyType() and EnumSet.elementType() In-Reply-To: References: Message-ID: I'm surprised I've never run into this. This seems like a simple and useful change. Stephen On 7 December 2017 at 11:40, Andrej Golovnin wrote: > Hi all, > > it would be nice if we would have access to the class of the enum type > used to create an EnumMap or an EnumSet. > > This is usefull when you write a custom serialization library and > would like to serialize/deserialize an empty EnumMap or an empty > EnumSet. For the empty EnumSet there is a workaround to get the enum > class: > > EnumSet empty = EnumSet.noneOf(MyEnum.class); > EnumSet tmp = EnumSet.complementOf(empty); > Class elementType = tmp.iterator().next().getClass(); > > But this only works when the enum class has at least one enum. For > EnumMap there is no such workaround at all and we have to use the > Reflection API. And you know the warnings since Java 9 :-) : > > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by c.r.r.k.EnumMapSerializer to > field java.util.EnumMap.keyType > WARNING: Please consider reporting this to the maintainers of > c.r.r.k.EnumMapSerializer > WARNING: Use --illegal-access=warn to enable warnings of further > illegal reflective access operations > WARNING: All illegal access operations will be denied in a future release > > So to avoid this warning and to avoid that our application stops to > work with a future release of Java I would like to propose to add the > following two methods: > > EnumMap: > /** > * Returns the {@code Class} object of the key type for this enum map. > * > * @return the {@code Class} object of the key type for this enum map. > * > * @since 10 > */ > public Class keyType() > > > EnumSet: > /** > * Returns the {@code Class} object of all the elements of this set. > * > * @return the {@code Class} object of all the elements of this set. > * > * @since 10 > */ > public Class elementType() > > The suggested change is attached as diff. > > Best reagrds, > Andrej Golovnin From Alan.Bateman at oracle.com Thu Dec 7 13:39:36 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 7 Dec 2017 13:39:36 +0000 Subject: Review Request: JDK-8193159: Reduce the number of classes loaded due to NativeLibrary In-Reply-To: <57df1268-51f4-32ad-6c9d-0ecef465335a@oracle.com> References: <57df1268-51f4-32ad-6c9d-0ecef465335a@oracle.com> Message-ID: On 07/12/2017 05:04, mandy chung wrote: > > > On 12/6/17 6:08 PM, Martin Buchholz wrote: >> Google decided that LinkedList is almost never the best choice, so >> any use of LinkedList is discouraged. >> >> ArrayDeque's backing array never shrinks, so you might want to give >> it an explicit initial size. >> > > The initial size is 16 which is okay.? I can make it smaller instead: > > -??????? static Deque nativeLibraryContext = new > LinkedList<>(); > +??????? static Deque nativeLibraryContext = new > ArrayDeque<>(8); This looks okay. It seems unlikely that hundreds of native libraries will be loaded, then unloaded, and the backing array being a memory concern. -Alan From claes.redestad at oracle.com Thu Dec 7 13:50:30 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 7 Dec 2017 14:50:30 +0100 Subject: Review Request: JDK-8193159: Reduce the number of classes loaded due to NativeLibrary In-Reply-To: References: <57df1268-51f4-32ad-6c9d-0ecef465335a@oracle.com> Message-ID: On 2017-12-07 14:39, Alan Bateman wrote: > > > On 07/12/2017 05:04, mandy chung wrote: >> >> >> On 12/6/17 6:08 PM, Martin Buchholz wrote: >>> Google decided that LinkedList is almost never the best choice, so >>> any use of LinkedList is discouraged. >>> >>> ArrayDeque's backing array never shrinks, so you might want to give >>> it an explicit initial size. >>> >> >> The initial size is 16 which is okay.? I can make it smaller instead: >> >> -??????? static Deque nativeLibraryContext = new >> LinkedList<>(); >> +??????? static Deque nativeLibraryContext = new >> ArrayDeque<>(8); +1 > This looks okay. It seems unlikely that hundreds of native libraries > will be loaded, then unloaded, and the backing array being a memory > concern. It also needs to load said hundreds of native libraries in parallel for this context queue to grow in the first place, right? So I'm not concerned, but on that tangent I have been wondering for some time why ArrayDeque doesn't have a trimToSize method like ArrayList does. Theoretically it'd be nice to have some means to allow periodically taking a look at persistent queues like this one and trim the backing arrays down if they ever outgrow their purpose. I guess it's just never become a real problem... /Claes From Alan.Bateman at oracle.com Thu Dec 7 14:08:55 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 7 Dec 2017 14:08:55 +0000 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> Message-ID: <9e7c216e-9fc7-88e6-0dec-37a59adbdb4f@oracle.com> On 06/12/2017 21:01, Jason Mehrens wrote: > Brian, > > My understand is JDK-6533165 is moving the bytes that are rarely invoked to a cold method. > > Therefore: > ==== > if (closed) { > throw new IOException("Stream closed"); > } > ==becomes=== > if (closed) { > throw sc(); > } > > private static IOException sc() { > return new IOException("Stream closed"); > } > ================================ > If nothing else, a private ensureOpen method would make it easier to read so that the "if (closed) throw ..." isn't needed in every method. -Alan From daniel.fuchs at oracle.com Thu Dec 7 14:38:59 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 7 Dec 2017 14:38:59 +0000 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works Message-ID: Hi, Please find below a patch for: 8191033 Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works https://bugs.openjdk.java.net/browse/JDK-8191033 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8191033/webrev.00/ The patch restores the behavior that was observed for JDK 8. The test fails with jdk 9.0.1, but passes with jdk1.8.0_131 and with this patch applied to a clone of http://hg.openjdk.java.net/jdk/jdk best regards, -- daniel From Roger.Riggs at Oracle.com Thu Dec 7 14:40:10 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 7 Dec 2017 09:40:10 -0500 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> Message-ID: <7a545feb-858f-5007-a361-a08e2d1c0eba@Oracle.com> Hmm... Seems like a lot of fine tuning for a function that has a low profile and no performance data... $.02, Roger On 12/6/2017 10:03 PM, Vitaly Davidovich wrote: > On Wed, Dec 6, 2017 at 4:01 PM, Jason Mehrens > wrote: > >> Brian, >> >> My understand is JDK-6533165 is moving the bytes that are rarely invoked >> to a cold method. >> >> Therefore: >> ==== >> if (closed) { >> throw new IOException("Stream closed"); >> } >> ==becomes=== >> if (closed) { >> throw sc(); >> } >> >> private static IOException sc() { >> return new IOException("Stream closed"); >> } >> ================================ >> >> Since there is no string concatenation in the IOE message there are only a >> few bytes that can be shaved off. I didn't benchmark it either case but I >> would assume it would matter for the nullOutputStream since the write >> methods could be invoked multiple times. >> > From a performance angle, I'd be more concerned with the calls to > Objects.xyz() methods there. Unless something has changed in the JIT > recently, those are susceptible to profile pollution and can cause missed > optimizations. I'd inline those methods manually to give these methods > their own profiles. > >> Jason >> ________________________________________ >> From: Brian Burkhalter >> Sent: Wednesday, December 6, 2017 2:05 PM >> To: Jason Mehrens >> Cc: core-libs-dev >> Subject: Re: RFR 4358774: Add null InputStream and OutputStream >> >> Jason, >> >> On Dec 6, 2017, at 11:54 AM, Jason Mehrens > mailto:jason_mehrens at hotmail.com>> wrote: >> >> For nullInputStream would it make any sense to use the >> ByteArrayInputStream with a (private static) empty byte array? Maybe >> 'return new ByteArrayInputStream("".getBytes());'? One side effect is >> that mark support returns true. >> >> As we are trying to retain the behavior of closed streams throwing >> IOExceptions I don?t think that BAIS would work here. >> >> Does it make sense to follow the advice inhttps://bugs.openjdk.java. >> net/browse/JDK-6533165 with regards to streams being closed? >> >> I don?t know exactly what you intend here. In the linked issue there is >> information to impart, namely the index and the size. Here there is only >> the indication that the stream is closed. It?s not clear to me what could >> be done here. >> >> Thanks, >> >> Brian >> From daniel.fuchs at oracle.com Thu Dec 7 14:41:57 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 7 Dec 2017 14:41:57 +0000 Subject: Change in properties for logging: deliberate? In-Reply-To: References: <71981898-fa64-e69c-5e56-246eaed548f7@oracle.com> <9adc01cf-fe46-f1be-6d92-182cc9ddd3ef@oracle.com> <59156c8a-8622-99b9-7a2f-b0c161868b4f@oracle.com> <5d3ffed1-6c0d-d2f2-a541-a82d5ed8ebd5@oracle.com> <9d326eca-e94f-cfb2-6daf-e9db456050ff@oracle.com> <628fd19f-b93b-ca66-b3b3-d65817bc8171@oracle.com> Message-ID: <87eae799-b99c-608d-fc49-54101459ef21@oracle.com> Hi Jeremy, I just proposed a patch on core-libs-dev. best regards, -- daniel On 06/12/2017 01:17, Jeremy Manson wrote: > Hey folks, > > Any thoughts on a timeline for this? We're just having to decide what to > do internally.? If a patch is likely to arrive in the next month or so, > then we'll probably wait, but if not, we should probably figure out a > workaround. > > (I'm not trying to be too pushy - we can certainly figure out a workaround.) > > Jeremy > From vitalyd at gmail.com Thu Dec 7 14:54:08 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 07 Dec 2017 14:54:08 +0000 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <7a545feb-858f-5007-a361-a08e2d1c0eba@Oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> <7a545feb-858f-5007-a361-a08e2d1c0eba@Oracle.com> Message-ID: On Thu, Dec 7, 2017 at 9:40 AM Roger Riggs wrote: > Hmm... > > Seems like a lot of fine tuning for a function that has a low profile and > no > performance data... These are known issues in the current JIT optimizer. We all wish we didn?t have to do this stuff. If we want the best chance of these noop impls to actually be close to noop then it warrants consideration. If that?s not a concern then sure, nothing to discuss. That?s all :). I also wouldn?t say ?it?s a lot of fine tuning? - these are simple mechanical things. > > > $.02, Roger > > On 12/6/2017 10:03 PM, Vitaly Davidovich wrote: > > On Wed, Dec 6, 2017 at 4:01 PM, Jason Mehrens > > > wrote: > > > >> Brian, > >> > >> My understand is JDK-6533165 is moving the bytes that are rarely invoked > >> to a cold method. > >> > >> Therefore: > >> ==== > >> if (closed) { > >> throw new IOException("Stream closed"); > >> } > >> ==becomes=== > >> if (closed) { > >> throw sc(); > >> } > >> > >> private static IOException sc() { > >> return new IOException("Stream closed"); > >> } > >> ================================ > >> > >> Since there is no string concatenation in the IOE message there are > only a > >> few bytes that can be shaved off. I didn't benchmark it either case > but I > >> would assume it would matter for the nullOutputStream since the write > >> methods could be invoked multiple times. > >> > > From a performance angle, I'd be more concerned with the calls to > > Objects.xyz() methods there. Unless something has changed in the JIT > > recently, those are susceptible to profile pollution and can cause missed > > optimizations. I'd inline those methods manually to give these methods > > their own profiles. > > > >> Jason > >> ________________________________________ > >> From: Brian Burkhalter > >> Sent: Wednesday, December 6, 2017 2:05 PM > >> To: Jason Mehrens > >> Cc: core-libs-dev > >> Subject: Re: RFR 4358774: Add null InputStream and OutputStream > >> > >> Jason, > >> > >> On Dec 6, 2017, at 11:54 AM, Jason Mehrens >> mailto:jason_mehrens at hotmail.com>> wrote: > >> > >> For nullInputStream would it make any sense to use the > >> ByteArrayInputStream with a (private static) empty byte array? Maybe > >> 'return new ByteArrayInputStream("".getBytes());'? One side effect is > >> that mark support returns true. > >> > >> As we are trying to retain the behavior of closed streams throwing > >> IOExceptions I don?t think that BAIS would work here. > >> > >> Does it make sense to follow the advice inhttps://bugs.openjdk.java. > >> net/browse/JDK-6533165 with regards to streams being closed? > >> > >> I don?t know exactly what you intend here. In the linked issue there is > >> information to impart, namely the index and the size. Here there is only > >> the indication that the stream is closed. It?s not clear to me what > could > >> be done here. > >> > >> Thanks, > >> > >> Brian > >> > > -- Sent from my phone From Roger.Riggs at Oracle.com Thu Dec 7 15:27:31 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 7 Dec 2017 10:27:31 -0500 Subject: RFR(XS) : 8181118 : update java/time tests to use RandomFactory from the top level testlibrary In-Reply-To: References: Message-ID: <50d55936-7fa3-60ec-da6e-01266b2f0fa0@Oracle.com> Hi Igor, Looks fine. Is the current jtreg b09? Also, please break the line (w/ '\`) for exclusiveAccess so side-by-side diffs fit on the screen. Thanks, Roger On 12/6/2017 7:35 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8181118/webrev.00 >> 115 lines changed: 0 ins; 108 del; 7 mod; > Hi all, > > could you please review this small fix for java/time tests? RandomFactory has been moved to the top level testlibrary by 8180805[1], but java/time weren't updated when b/c jtreg didn't use external root for libraries listed in lib.dirs test property file[2]. it has been fixed in jtreg4.2b09, now java/time tests can be updated to use top level testlibrary and @deprecated RandomFactory (from test/jdk/lib/testlibrary) can be removed. > > webrev: http://cr.openjdk.java.net/~iignatyev/8181118/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8181118 > testing: java/time/jck and java/time/tests > > [1] https://bugs.openjdk.java.net/browse/JDK-8180805 > [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7901987 > > Thanks, > -- Igor From vyom.tewari at oracle.com Thu Dec 7 15:44:19 2017 From: vyom.tewari at oracle.com (vyom tewari) Date: Thu, 7 Dec 2017 21:14:19 +0530 Subject: [RFR] 8160768: Add capability to custom resolve host/domain names within the default JDNI LDAP provider In-Reply-To: <20171206182434.GB4409@vimes> References: <20171205174824.GA3515@vimes> <20171206182434.GB4409@vimes> Message-ID: Hi Rob, Latest code looks good to me. minor bit. LdapDnsProviderService.java Line: 71 , while loop condition is bit complex, we can simplify it little bit as follows.This will make code more readable and easy to understand. while (iterator.hasNext()) ??????????? { ??????????????? LdapDnsProvider p = iterator.next(); ??????????????? result = p.lookupEndpoints(url, env); ??????????????? if(result != null && !result.getEndpoints().isEmpty()){ ??????????????????? break; ??????????????? } ??????????? } It is just a personal preference you can ignore it if you don't like it. Thanks, Vyom On Wednesday 06 December 2017 11:54 PM, Rob McKenna wrote: > Thanks Vyom, these should be fixed in: > > http://cr.openjdk.java.net/~robm/8160768/webrev.07/ > > Comments inline: > > On 06/12/17 22:14, vyom tewari wrote: >> Hi Rob, >> >> Please find below comments. >> >> DefaultLdapDnsProvider.java >> >> ?line 35: ??? convert it to for-each code will be more readable. >> >> LdapDnsProviderService.java >> >> ?line 21: constant name does not follow Java naming convention, there are >> other places as well please fix it. >> >> getInstance() is already synchronized why you need another "synchronized" >> block ? > Ah, neglected to remove the outer synchronized keyword, thanks. > >> Line 70: Please use result.getEndpoints().isEmpty() in place of >> result.getEndpoints().size() == 0 >> >> LdapDnsProvider.java >> >> constructor LdapDnsProvider() calls LdapDnsProvider(Void unused) which does >> nothing, can you explain why LdapDnsProvider()? calls LdapDnsProvider(Void >> unused) ? > I've copied this pattern from the System.LoggerFinder api and I had > forgotten what it was for too: > > http://www.oracle.com/technetwork/java/seccodeguide-139067.html#7 > > Section 7.3. > >> LdapDnsProviderResult.java >> >> ? Private field? domainName & endpoints both can be final. >> >> LdapDnsProviderTest.java >> >> Fix the tag Order it is not correct. correct order is as follows. >> >> /** >> ?* @test >> ?* @bug 8160768 >> ?* @summary ctx provider tests for ldap >> ?* @modules java.naming/com.sun.jndi.ldap >> ?* @compile dnsprovider/TestDnsProvider.java >> ?* @run main/othervm LdapDnsProviderTest >> ?* @run main/othervm LdapDnsProviderTest nosm >> ?* @run main/othervm LdapDnsProviderTest smnodns >> ?* @run main/othervm LdapDnsProviderTest smdns >> ?*/ >> >> Line 82,83 you don't need System.out.println(e); e.printStackTrace(); > I'm going to leave these in as I've had a few failing tests in the past > where I've needed to push diagnostic changes like this to determine the > root cause. > > -Rob > >> Line 70: you don't need this try block >> >> Line 96 : constant name does not follow Java naming convention >> >> Line 105: you can use try-with-resources this will avoid some boilerplate. >> >> Thanks, >> Vyom >> >> On Tuesday 05 December 2017 11:18 PM, Rob McKenna wrote: >>> As this enhancement is limited to JDK10 this frees us up to explore a >>> different approach. >>> >>> http://cr.openjdk.java.net/~robm/8160768/webrev.06/ >>> >>> In this webrev we're using the Service Provider Interface to load an >>> implementation of the new LdapDnsProvider class. Implementations of this >>> class are responsible for resolving DNS endpoints for a given ldap url >>> should the default implementation be insufficient in a particular >>> environment. >>> >>> Note: if a security manager is installed the ldapDnsProvider >>> RuntimePermission must be granted. >>> >>> -Rob >>> From peter.levart at gmail.com Thu Dec 7 15:49:15 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 7 Dec 2017 16:49:15 +0100 Subject: Finalization and dead references: another proposal In-Reply-To: References: Message-ID: <44204441-d6a5-b13f-355b-003aa4f80171@gmail.com> Hi, I'm trying to read the rules and imagine cases where they would not work... /A field declared to be reachability-sensitive is reachability-safe. An access a to a reachability-safe field of object p inside a (possibly static) method in p?s class, results in the introduction of reachabilityFence()s according to the following rules:// // //For every local reference variable v declared immediately inside lexical scope s, if s contains such an access a, where p is obtained by dereferencing v, then Reference.reachabilityFence(v) will be executed just before either (1) the exit of the scope s, or (2) just before any assignment to v. For present purposes, ?this? is treated as a variable declared at method scope, as if it were an explicit parameter.// //If the object containing the field f is allocated in the same full-expression as the access a, then Reference.reachabilityFence(p), where p is a reference to the object containing f, is executed at the end of the full expression.// // //The second case covers expressions like// // //??? nativeDoSomething(new Foo(...).nativeHandle)// // //where again the newly allocated object could otherwise be cleaned before nativeHandle is used.// / 1st case that comes to mind and is a real case in the JDK is the existence of sun.nio.ch.DirectBuffer internal (not exported) interface implemented by NIO direct buffer(s) with method: ??? long address(); Utility method like the following: ??? static void erase(ByteBuffer bb) { ??????? unsafe.setMemory(((DirectBuffer)bb).address(), bb.capacity(), (byte)0); ??? } ...won't get reachabilityFence(bb) before return, will it? Calling a method on a bb doesn't count as a dereference of a reachablity-sensitive field of bb. Unless methods could also be annotated with the same annotation. This would only work for instance methods that return a native resource kept in the same instance or some instance that is reachable from this instance. Native resources are typically encapsulated, but there are various levels of encapsulation. DirectBuffer is an example of module level encapsulation by non-exported public interface implemented by package-private classes. 2nd case is artificial. But, since we have streams, lambdas, optionals... It's easy to fool above rules even if the annotation is put on the DirectBuffer#address() method ... static void doSomethingWith1stNonNull(DirectBuffer... dbs) { ??? Stream.of(dbs) ??? ??? .filter(Objects::nonNull) ??? ??? .mapToLong(DirectBuffer::address) ??? ??? .findFirst() ??? ??? .ifPresent(addr -> { ??? ??? ??? ... do something with addr ... ??? ??? }); } Even with imperative programming, one can play with scopes: static void doSomethingWith1stNonNull(DirectBuffer... dbs) { ??? long addr = 0; ??? for (DirectBuffer db : dbs) { ??? ??? if (db != null) { ??? ??? ??? addr = db.address(); ??? ??? ??? break; ??? ??? } ??? } ??? if (addr != 0) { ??? ??? ... do something with addr ... ??? } } Or, hope that this would work (but the above rules don't cover it): static void doSomethingWithEither(DirectBuffer db1, DirectBuffer db2) { ??? long addr = ((some condition) ? db1 : db2).address(); ??? ... do something with addr ... } But I can imagine that teaching users to not do such foolish things would be easy. The rules are simple: - always dereference reachablity-sensitive resources directly through local reference variables. - make sure that the local reference variable that you extracted reachablity-sensitive resource from, is in scope while you perform operations on the resource. Regards, Peter On 12/07/2017 01:38 AM, Hans Boehm wrote: > We're still trying to deal with a fair amount of code that implicitly > assumes that finalization or similar clean-up will not occur while a > pointer to the affected object is in scope. Which is of course not true. > > As a reminder, the canonical offending usage (anti-)pattern with > (deprecated, but easier to write) finalizers is > > class Foo { > private long mPtrToNativeFoo; > > private static native void nativeDeallocate(long nativePtr); > private static native void nativeDoSomething(long nativePtr, long > anotherNativePtr); > > protected void finalize() { ... nativeDeallocate(mPtrToNativeFoo); ... } > > public void doSomething(Foo another) { ... > nativeDoSomething(mPtrToNativeFoo, another.mPtrToNativeFoo) ... } > ... > } > > This is subtly incorrect in that, while executing the final call to > doSomething() on a particular object, just after retrieving mPtrToNativeFoo > and another.mPtrToNativeFoo, but before invoking nativeDoSomething(), the > garbage collector may run, and "this" and "another" may be finalized, > deallocating the native objects their mPtrToNativeFoos refer to. > When nativeDoSomething() finally does run, it may see dangling pointers. > > Examples using java.lang.ref or Cleaners (or even WeakHashMap, if you must) > instead of finalize() are as easy to construct, but a bit longer. (The > finalizer version is also arguably incorrect in other ways that are not > relevant here. Pretend this were written in terms of PhantomReferences.) > > It is easily possible to construct 100% Java code with the same problem. > Instead of mPtrToNativeFoo, each object stores an integer handle that is > used to access additional Java state logically associated with the object. > But the native pointer case seems to dominate in practice. > > Various solutions to this have been proposed, but none seem quite > attractive enough that I actually feel comfortable asking people to update > their code to use them. Noteworthy proposals include: > > 0) Explicitly call close() on such objects. Great idea when it works. In > general it doesn't, since the code needs to know when the enclosing Java > object is no longer needed. If we always knew that we wouldn't need a GC. > 1) Various hacks to keep variables live, e.g. the one based on synchronized > blocks I advocated in my 2004 JavaOne talk. These are slow and ugly, as > we've always admitted. Nobody used them. Incorrect won over slow, ugly, and > complicated ~100% of the time. > 2) Java 9's reachabilityFence(). This is better. It can be implemented so > that it's no longer slow. But in many common cases, it's still quite ugly. > For example, the call to nativeDoSomething() above must be followed by two > reachabilityFences, one on this and one on another. And to do this really > correctly, the whole thing would often need to be in a try...finally block. > And in reality code following this pattern usually doesn't have just a > single doSomething method that needs this treatment, but may easily have > dozens. And the rules for placing reachabilityFences can become quite > subtle if there are e.g. locally allocated objects involved. My assessment > is that this isn't good enough. People may no longer write incorrect code > 100% of the time, but I'd bet on 70%+. > 3) JNI functions can be rewritten, so that the containing Java object is > passed in addition to the native pointers. Somewhat accidentally, this > happens to be roughly free for single argument functions. (Delete > "static".) It adds overhead in other cases, like the one above, and the > rewriting can get somewhat convoluted. In some cases, it doesn't work at > all. AFAIK, it's never actually guaranteed to be correct; it works because > standard implementations don't optimize across the language boundary. > That's not too likely to change. Maybe. > 4) We could change the language spec to always prohibit premature > finalization/cleaning in cases like the above. I could personally live with > that solution, and proposed it internally here in the past. But it doesn't > seem to go over well among implementers. And AFAICT, doing it well requires > significant tooling changes, in that we do want to reliably treat local > variables as dead once they go out of scope, a piece of information that > doesn't seem to be reliably preserved in class files. One could argue that > the current class file design implicitly assumes that we can do dead > variable elimination. > > After going back and forth on this, my conclusion is that we need a > linguistic mechanism for identifying the case in which the garbage > collector is being used to managed external resources associated with a > field. A (still slowly evolving) proposal to add an annotation to do so is > at > > https://docs.google.com/document/d/1yMC4VzZobMQTrVAraMy7xBdWPCJ0p7G5r_43yaqsj-s/edit?usp=sharing > > In many ways, this is inherently a compromise. But in the vast majority of > cases, it greatly reduces the required source code changes over all but (4) > above. And I think I could usually explain to an author of currently broken > code in under 5 minutes exactly what they need to do to fix it. And I > wouldn't have to lie much to do so. I don't think (0)-(3) above share that > property. > > This has already benefited from comments provided by a few people. (Thanks > to Doug, Jeremy, and Martin, among others.) But I would really like more > feedback, including suggestions about how to proceed. > > Hans From mandy.chung at oracle.com Thu Dec 7 16:13:15 2017 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Dec 2017 08:13:15 -0800 Subject: Review Request: JDK-8193159: Reduce the number of classes loaded due to NativeLibrary In-Reply-To: References: <57df1268-51f4-32ad-6c9d-0ecef465335a@oracle.com> Message-ID: <5511d16b-e44a-7a7d-f668-c2bf92de9c5a@oracle.com> On 12/7/17 5:39 AM, Alan Bateman wrote: > > > On 07/12/2017 05:04, mandy chung wrote: >> >> >> On 12/6/17 6:08 PM, Martin Buchholz wrote: >>> Google decided that LinkedList is almost never the best choice, so >>> any use of LinkedList is discouraged. >>> >>> ArrayDeque's backing array never shrinks, so you might want to give >>> it an explicit initial size. >>> >> >> The initial size is 16 which is okay.? I can make it smaller instead: >> >> -??????? static Deque nativeLibraryContext = new >> LinkedList<>(); >> +??????? static Deque nativeLibraryContext = new >> ArrayDeque<>(8); > This looks okay. It seems unlikely that hundreds of native libraries > will be loaded, then unloaded, and the backing array being a memory > concern. Yup, it's not a big concern. ? I believe the number of elements in this deque is typically very small number.?? This deque is used to keep the native libraries are being loaded and its context used by JNI FindClass when called from JNI_OnLoad. Mandy From igor.ignatyev at oracle.com Thu Dec 7 16:18:45 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 7 Dec 2017 08:18:45 -0800 Subject: RFR(XS) : 8181118 : update java/time tests to use RandomFactory from the top level testlibrary In-Reply-To: <50d55936-7fa3-60ec-da6e-01266b2f0fa0@Oracle.com> References: <50d55936-7fa3-60ec-da6e-01266b2f0fa0@Oracle.com> Message-ID: Hi Roger, > On Dec 7, 2017, at 7:27 AM, Roger Riggs wrote: > > Hi Igor, > > Looks fine. > > Is the current jtreg b09? no, the latest the greatest jtreg is 4.2-b10. usually we specify the minimal required version in TEST.ROOT, not the latest available. > > Also, please break the line (w/ '\`) for exclusiveAccess so side-by-side diffs fit on the screen. it's not really related to this patch, so if you don't mind of course, I'd prefer to make that refactoring in a separate RFE. > > Thanks, Roger > > > On 12/6/2017 7:35 PM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8181118/webrev.00 >>> 115 lines changed: 0 ins; 108 del; 7 mod; >> Hi all, >> >> could you please review this small fix for java/time tests? RandomFactory has been moved to the top level testlibrary by 8180805[1], but java/time weren't updated when b/c jtreg didn't use external root for libraries listed in lib.dirs test property file[2]. it has been fixed in jtreg4.2b09, now java/time tests can be updated to use top level testlibrary and @deprecated RandomFactory (from test/jdk/lib/testlibrary) can be removed. >> >> webrev: http://cr.openjdk.java.net/~iignatyev/8181118/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8181118 >> testing: java/time/jck and java/time/tests >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8180805 >> [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7901987 >> >> Thanks, >> -- Igor > From mandy.chung at oracle.com Thu Dec 7 16:31:14 2017 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Dec 2017 08:31:14 -0800 Subject: RFR JDK-8187485: Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <5A25D6BE.60203@oracle.com> References: <5A25D6BE.60203@oracle.com> Message-ID: <30f9111b-933d-c5e6-39dc-f45261549068@oracle.com> On 12/4/17 3:14 PM, Xueming Shen wrote: > > issue: https://bugs.openjdk.java.net/browse/JDK-8185582 > webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev > ZStreamRef.java 79 static ZStreamRef get(Object owner, LongSupplier addr, LongConsumer end) { It may be better to have two factory methods that take the specific type (Deflater & Inflater) instead of this single method taking Object owner. ZipFile.java 701 // List of cached Inflater objects for decompression 702 Deque inflaterCache;should this be a volatile field? TestCleaner.java 27 * @summary Check the resources of Inflater, Deflater and ZipFile are always 28 * cleaned/released when the instane is not unreachable typo: s/instane/instance and s/not unreachable/unreachable Otherwise looks good. Mandy From martinrb at google.com Thu Dec 7 16:40:27 2017 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Dec 2017 08:40:27 -0800 Subject: Review Request: JDK-8193159: Reduce the number of classes loaded due to NativeLibrary In-Reply-To: <5511d16b-e44a-7a7d-f668-c2bf92de9c5a@oracle.com> References: <57df1268-51f4-32ad-6c9d-0ecef465335a@oracle.com> <5511d16b-e44a-7a7d-f668-c2bf92de9c5a@oracle.com> Message-ID: +1 From Roger.Riggs at Oracle.com Thu Dec 7 16:52:39 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 7 Dec 2017 11:52:39 -0500 Subject: RFR(XS) : 8181118 : update java/time tests to use RandomFactory from the top level testlibrary In-Reply-To: References: <50d55936-7fa3-60ec-da6e-01266b2f0fa0@Oracle.com> Message-ID: <4e4840a7-1064-8ff0-83ce-1e51c4840f78@Oracle.com> Hi Igor, On 12/7/2017 11:18 AM, Igor Ignatyev wrote: > >> Also, please break the line (w/ '\`) for exclusiveAccess so side-by-side diffs fit on the screen. > it's not really related to this patch, so if you don't mind of course, I'd prefer to make that refactoring in a separate RFE. True, but it is incidental here and is not worth doing as a separate RFE.? Just trying to keep the overhead low. if you're not going to do it now; don't bother creating an RFE; it just more noise. Thanks, Roger >> Thanks, Roger >> >> >> On 12/6/2017 7:35 PM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev/8181118/webrev.00 >>> From daniel.daugherty at oracle.com Thu Dec 7 16:55:57 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 7 Dec 2017 11:55:57 -0500 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> Message-ID: <0a36c975-8bca-c618-9133-66348f145436@oracle.com> Adding core-libs-dev at ... since this is RMI code. Thanks Alan! Folks, this review spans three OpenJDK aliases so Thunderbird's reply-to-list feature won't get it right. This is one of the few times that reply-to-all is the right thing to do... Dan On 12/7/17 11:38 AM, Daniel D. Daugherty wrote: > Greetings, > > I have a small fix for a very intermittent ServerSocket related test > failure: > > ??? JDK-8182307: Error during JRMP connection establishment > ??? https://bugs.openjdk.java.net/browse/JDK-8182307 > > The fix is copied from Jerry's work on the following bugs: > > ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris > ??? https://bugs.openjdk.java.net/browse/JDK-8182757 > > ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with Exception > ??? https://bugs.openjdk.java.net/browse/JDK-8178676 > > For the gory details of the reasons for this fix please see > Jerry's bugs. > > Webrev URL: http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ > > > I observed this test failure one time in my Thread-SMR work and have > been including this fix in months of Thread-SMR stress testing. It was > also included in both JPRT and Mach5 tier[1-5] testing of the Thread-SMR > changes. > > Thanks, in advance, for any comments, suggestions or questions. > > Dan > From martinrb at google.com Thu Dec 7 16:58:41 2017 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Dec 2017 08:58:41 -0800 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works In-Reply-To: References: Message-ID: configure => configured + // For backward compatibility: add any handlers configure using From gerald.thornbrugh at oracle.com Thu Dec 7 17:00:43 2017 From: gerald.thornbrugh at oracle.com (Gerald Thornbrugh) Date: Thu, 7 Dec 2017 10:00:43 -0700 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: <0a36c975-8bca-c618-9133-66348f145436@oracle.com> References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> Message-ID: Hi Dan, Your fix looks good. Jerry > Adding core-libs-dev at ... since this is RMI code. Thanks Alan! > > Folks, this review spans three OpenJDK aliases so Thunderbird's > reply-to-list feature won't get it right. This is one of the > few times that reply-to-all is the right thing to do... > > Dan > > On 12/7/17 11:38 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a small fix for a very intermittent ServerSocket related test >> failure: >> >> ??? JDK-8182307: Error during JRMP connection establishment >> ??? https://bugs.openjdk.java.net/browse/JDK-8182307 >> >> The fix is copied from Jerry's work on the following bugs: >> >> ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris >> ??? https://bugs.openjdk.java.net/browse/JDK-8182757 >> >> ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with Exception >> ??? https://bugs.openjdk.java.net/browse/JDK-8178676 >> >> For the gory details of the reasons for this fix please see >> Jerry's bugs. >> >> Webrev URL: http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >> >> >> I observed this test failure one time in my Thread-SMR work and have >> been including this fix in months of Thread-SMR stress testing. It was >> also included in both JPRT and Mach5 tier[1-5] testing of the Thread-SMR >> changes. >> >> Thanks, in advance, for any comments, suggestions or questions. >> >> Dan >> > From daniel.daugherty at oracle.com Thu Dec 7 17:05:52 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 7 Dec 2017 12:05:52 -0500 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> Message-ID: <3c719cde-b764-5582-67c9-33ff7cf95661@oracle.com> Jerry, Thanks for the review! Dan On 12/7/17 12:00 PM, Gerald Thornbrugh wrote: > Hi Dan, > > Your fix looks good. > > Jerry > >> Adding core-libs-dev at ... since this is RMI code. Thanks Alan! >> >> Folks, this review spans three OpenJDK aliases so Thunderbird's >> reply-to-list feature won't get it right. This is one of the >> few times that reply-to-all is the right thing to do... >> >> Dan >> >> On 12/7/17 11:38 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> I have a small fix for a very intermittent ServerSocket related test >>> failure: >>> >>> ??? JDK-8182307: Error during JRMP connection establishment >>> ??? https://bugs.openjdk.java.net/browse/JDK-8182307 >>> >>> The fix is copied from Jerry's work on the following bugs: >>> >>> ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris >>> ??? https://bugs.openjdk.java.net/browse/JDK-8182757 >>> >>> ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with Exception >>> ??? https://bugs.openjdk.java.net/browse/JDK-8178676 >>> >>> For the gory details of the reasons for this fix please see >>> Jerry's bugs. >>> >>> Webrev URL: http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >>> >>> >>> I observed this test failure one time in my Thread-SMR work and have >>> been including this fix in months of Thread-SMR stress testing. It was >>> also included in both JPRT and Mach5 tier[1-5] testing of the >>> Thread-SMR >>> changes. >>> >>> Thanks, in advance, for any comments, suggestions or questions. >>> >>> Dan >>> >> > From Roger.Riggs at Oracle.com Thu Dec 7 17:07:13 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 7 Dec 2017 12:07:13 -0500 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> Message-ID: +1 On 12/7/2017 12:00 PM, Gerald Thornbrugh wrote: > Hi Dan, > > Your fix looks good. > > Jerry > >> Adding core-libs-dev at ... since this is RMI code. Thanks Alan! >> >> Folks, this review spans three OpenJDK aliases so Thunderbird's >> reply-to-list feature won't get it right. This is one of the >> few times that reply-to-all is the right thing to do... >> >> Dan >> >> On 12/7/17 11:38 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> I have a small fix for a very intermittent ServerSocket related test >>> failure: >>> >>> ??? JDK-8182307: Error during JRMP connection establishment >>> ??? https://bugs.openjdk.java.net/browse/JDK-8182307 >>> >>> The fix is copied from Jerry's work on the following bugs: >>> >>> ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris >>> ??? https://bugs.openjdk.java.net/browse/JDK-8182757 >>> >>> ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with Exception >>> ??? https://bugs.openjdk.java.net/browse/JDK-8178676 >>> >>> For the gory details of the reasons for this fix please see >>> Jerry's bugs. >>> >>> Webrev URL: http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >>> >>> >>> I observed this test failure one time in my Thread-SMR work and have >>> been including this fix in months of Thread-SMR stress testing. It was >>> also included in both JPRT and Mach5 tier[1-5] testing of the >>> Thread-SMR >>> changes. >>> >>> Thanks, in advance, for any comments, suggestions or questions. >>> >>> Dan >>> >> > From daniel.daugherty at oracle.com Thu Dec 7 17:09:19 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 7 Dec 2017 12:09:19 -0500 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> Message-ID: <35163e84-b100-1b85-3179-a58699163fc5@oracle.com> Roger, Thanks for the review! Dan P.S. I'm planning to push this fix to jdk/hs since the only sightings have been in jdk/hs testing or in projects that are parented to jdk/hs repos... Hope that's okay... On 12/7/17 12:07 PM, Roger Riggs wrote: > +1 > > On 12/7/2017 12:00 PM, Gerald Thornbrugh wrote: >> Hi Dan, >> >> Your fix looks good. >> >> Jerry >> >>> Adding core-libs-dev at ... since this is RMI code. Thanks Alan! >>> >>> Folks, this review spans three OpenJDK aliases so Thunderbird's >>> reply-to-list feature won't get it right. This is one of the >>> few times that reply-to-all is the right thing to do... >>> >>> Dan >>> >>> On 12/7/17 11:38 AM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> I have a small fix for a very intermittent ServerSocket related test >>>> failure: >>>> >>>> ??? JDK-8182307: Error during JRMP connection establishment >>>> https://bugs.openjdk.java.net/browse/JDK-8182307 >>>> >>>> The fix is copied from Jerry's work on the following bugs: >>>> >>>> ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris >>>> https://bugs.openjdk.java.net/browse/JDK-8182757 >>>> >>>> ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with >>>> Exception >>>> https://bugs.openjdk.java.net/browse/JDK-8178676 >>>> >>>> For the gory details of the reasons for this fix please see >>>> Jerry's bugs. >>>> >>>> Webrev URL: http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >>>> >>>> >>>> I observed this test failure one time in my Thread-SMR work and have >>>> been including this fix in months of Thread-SMR stress testing. It was >>>> also included in both JPRT and Mach5 tier[1-5] testing of the >>>> Thread-SMR >>>> changes. >>>> >>>> Thanks, in advance, for any comments, suggestions or questions. >>>> >>>> Dan >>>> >>> >> > From martinrb at google.com Thu Dec 7 17:14:29 2017 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Dec 2017 09:14:29 -0800 Subject: RFR: jsr166 jdk integration 2017-12-XX Message-ID: http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html With the change to the openjdk release model, jsr166 integrations will simply be "jdk integrations" instead of e.g. "jdk10 integrations" from now on. (Nevertheless, we try to get in important bug fixes before jdk release gates shut) 8193174: SubmissionPublisher invokes the Subscriber's onComplete before all of its submitted items have been published http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/SubmissionPublisher-race/index.html https://bugs.openjdk.java.net/browse/JDK-8193174 8192943: Optimize atomic accumulators using VarHandle getAndSet http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/getAndSet/index.html https://bugs.openjdk.java.net/browse/JDK-8192943 8192944: Miscellaneous changes imported from jsr166 CVS 2017-12-XX http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html https://bugs.openjdk.java.net/browse/JDK-8192944 From mandy.chung at oracle.com Thu Dec 7 17:17:14 2017 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Dec 2017 09:17:14 -0800 Subject: Review Request: JDK-8193192: jdeps --generate-module-info does not look at module path Message-ID: jdeps currently finds modules from the module path for analysis based on the value specified via --add-modules option.? When the input classes depend on a module on the given module path, it will report missing dependences until the module is specified in --add-modules option.? This fixes this issue and jdeps should look at the modules on the module path when analyzing classes (except jdeps -m option). http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8193192/webrev.00 thanks Mandy From peter.levart at gmail.com Thu Dec 7 17:28:50 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 7 Dec 2017 18:28:50 +0100 Subject: Finalization and dead references: another proposal In-Reply-To: References: Message-ID: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> Hi, On 12/07/2017 03:27 AM, Vitaly Davidovich wrote: > So kind of the opposite of WeakReference - a SuperStrongReference :). > > Kidding aside, it seems like the way you?d want to encapsulate this at the > language level is via a type that the JVM intrinsically knows about; in > this way it?s similar to the reference types today. > > An annotation probably does the trick when the value doesn?t escape from > the enclosing instance but I?ve no idea if that assumption covers enough > code to warrant this approach. AFAICT, if the value escapes into an > instance of another type that doesn?t annotate its field then all bets are > off. > > Having a wrapper type would at least make it harder to leak the native > handle vs the annotation approach. But of course the wrapper comes with > footprint and indirection costs. Maybe Valhalla could allow exposing some > magic value type that?s a zero-cost wrapper but preserves the type > information the JIT can track? There is a middle-ground. By (ab)using value types only and no special JIT magic. DirectBuffer(s) for example, could return the address not in a long, but in a value type like the following (using valhalla MVT speak): public __ByValue final class Address { ??? private final long address; ??? private final Object referent; ??? public _ValueFactory static Address create(long address, Object referent) { ??? ??? Address a = __MakeDefault Address(); ??? ??? a.address = address; ??? ??? a.referent = referent; ??? ??? return a; ??? } } DirectByteBuffer for example, would have the following address() method: private long address; public Address address() { ??? return Address.create(address, this); } Notice that 'address' field of Address value is encapsulated, so Java code can't access it directly nor it needs to. Native / Unsafe methods would be taking Address values instead of long(s), making sure the embedded referent is kept reachable. This is equivalent to passing the DirectBuffer reference to the native method(s) together with the address value, but enforced by the API so the programmer can not make a mistake. There's a lot of arithmetic going on with addresses, inside and outside of DirectBuffer implementations. Such arithmetic would have to be performed by the Address value type itself, making sure the results of operations on Address values are Address values that maintain the same referent. For example: public __ByValue final class Address { ... ??? public _ValueFactory Address plus(long offset) { ??????? Address a = __MakeDefault Address(); ??? ??? a.address = address + offset; ??? ??? a.referent = referent; ??? ??? return a; ??? } ... So no magic here. Just API. Regards, Peter From pavel.rappo at oracle.com Thu Dec 7 17:38:44 2017 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 7 Dec 2017 20:38:44 +0300 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <6C0A7D9C-CE3D-4B9C-8E30-E54AC0B0CD88@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <6C0A7D9C-CE3D-4B9C-8E30-E54AC0B0CD88@oracle.com> Message-ID: > On Dec 6, 2017, at 11:11 AM, Jonathan Gibbons wrote: > >> "null" is a significant term in the Java ecosystem, and the relationship here, to /dev/null or NUL seems somewhat tenuous. >> >> Have any other names been considered? At least for the InputStream, calling it an "empty stream" seems more intuitive than a "null stream". Jon, I think there's an established name for this kind of objects: https://en.wikipedia.org/wiki/Null_object_pattern > On 6 Dec 2017, at 22:28, Brian Burkhalter wrote: > > The name ?emptyStream()? was considered for InputStream and ?discardingStream()? for OutputStream. It was thought that ?null? or ?empty? would be more likely to be found by developers due to familiarity. FWIW there is precedent in third party libraries for the ?null? names. > > Brian +1 From vitalyd at gmail.com Thu Dec 7 17:46:41 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 07 Dec 2017 17:46:41 +0000 Subject: Finalization and dead references: another proposal In-Reply-To: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> References: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> Message-ID: On Thu, Dec 7, 2017 at 12:28 PM Peter Levart wrote: > Hi, > > On 12/07/2017 03:27 AM, Vitaly Davidovich wrote: > > So kind of the opposite of WeakReference - a SuperStrongReference :). > > > > Kidding aside, it seems like the way you?d want to encapsulate this at > the > > language level is via a type that the JVM intrinsically knows about; in > > this way it?s similar to the reference types today. > > > > An annotation probably does the trick when the value doesn?t escape from > > the enclosing instance but I?ve no idea if that assumption covers enough > > code to warrant this approach. AFAICT, if the value escapes into an > > instance of another type that doesn?t annotate its field then all bets > are > > off. > > > > Having a wrapper type would at least make it harder to leak the native > > handle vs the annotation approach. But of course the wrapper comes with > > footprint and indirection costs. Maybe Valhalla could allow exposing > some > > magic value type that?s a zero-cost wrapper but preserves the type > > information the JIT can track? > > There is a middle-ground. By (ab)using value types only and no special > JIT magic. > > DirectBuffer(s) for example, could return the address not in a long, but > in a value type like the following (using valhalla MVT speak): > > public __ByValue final class Address { > private final long address; > private final Object referent; > > public _ValueFactory static Address create(long address, Object > referent) { > Address a = __MakeDefault Address(); > a.address = address; > a.referent = referent; > return a; > } > } > > > DirectByteBuffer for example, would have the following address() method: > > private long address; > > public Address address() { > return Address.create(address, this); > } > > Notice that 'address' field of Address value is encapsulated, so Java > code can't access it directly nor it needs to. Native / Unsafe methods > would be taking Address values instead of long(s), making sure the > embedded referent is kept reachable. > > This is equivalent to passing the DirectBuffer reference to the native > method(s) together with the address value, but enforced by the API so > the programmer can not make a mistake. There's a lot of arithmetic going > on with addresses, inside and outside of DirectBuffer implementations. > Such arithmetic would have to be performed by the Address value type > itself, making sure the results of operations on Address values are > Address values that maintain the same referent. For example: > > public __ByValue final class Address { > ... > public _ValueFactory Address plus(long offset) { > Address a = __MakeDefault Address(); > a.address = address + offset; > a.referent = referent; > return a; > } > ... > > So no magic here. Just API. This is an API version of Hans?s #3 approach. As he said, there?s performance overhead and nothing guarantees that the referent is kept alive - that?s an implementation artifact. I think without the VM knowing about these things intrinsically it?s not a 100% reliable solution because it?s not concretely requesting a certain behavior. > > > Regards, Peter > > > -- Sent from my phone From paul.sandoz at oracle.com Thu Dec 7 18:03:08 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 7 Dec 2017 10:03:08 -0800 Subject: RFR: jsr166 jdk integration 2017-12-XX In-Reply-To: References: Message-ID: <9F6FC09C-7DE1-47EB-9162-FAAA23624A58@oracle.com> +1 Paul. > On 7 Dec 2017, at 09:14, Martin Buchholz wrote: > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > > With the change to the openjdk release model, jsr166 integrations will simply be "jdk integrations" instead of e.g. "jdk10 integrations" from now on. (Nevertheless, we try to get in important bug fixes before jdk release gates shut) > > 8193174: SubmissionPublisher invokes the Subscriber's onComplete before all of its submitted items have been published > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/SubmissionPublisher-race/index.html > https://bugs.openjdk.java.net/browse/JDK-8193174 > > 8192943: Optimize atomic accumulators using VarHandle getAndSet > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/getAndSet/index.html > https://bugs.openjdk.java.net/browse/JDK-8192943 > > 8192944: Miscellaneous changes imported from jsr166 CVS 2017-12-XX > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html > https://bugs.openjdk.java.net/browse/JDK-8192944 > From serguei.spitsyn at oracle.com Thu Dec 7 18:08:07 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 7 Dec 2017 10:08:07 -0800 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: <35163e84-b100-1b85-3179-a58699163fc5@oracle.com> References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> <35163e84-b100-1b85-3179-a58699163fc5@oracle.com> Message-ID: Hi Dan, The fix looks good to me. Nice, you have caught it. Do you want this fixed in 10 or 11? I thought that the jdk/hs is for 11 now. Is it correct? Thanks, Serguei On 12/7/17 09:09, Daniel D. Daugherty wrote: > Roger, > > Thanks for the review! > > Dan > > P.S. > I'm planning to push this fix to jdk/hs since the only sightings > have been in jdk/hs testing or in projects that are parented to > jdk/hs repos... Hope that's okay... > > > On 12/7/17 12:07 PM, Roger Riggs wrote: >> +1 >> >> On 12/7/2017 12:00 PM, Gerald Thornbrugh wrote: >>> Hi Dan, >>> >>> Your fix looks good. >>> >>> Jerry >>> >>>> Adding core-libs-dev at ... since this is RMI code. Thanks Alan! >>>> >>>> Folks, this review spans three OpenJDK aliases so Thunderbird's >>>> reply-to-list feature won't get it right. This is one of the >>>> few times that reply-to-all is the right thing to do... >>>> >>>> Dan >>>> >>>> On 12/7/17 11:38 AM, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> I have a small fix for a very intermittent ServerSocket related test >>>>> failure: >>>>> >>>>> ??? JDK-8182307: Error during JRMP connection establishment >>>>> https://bugs.openjdk.java.net/browse/JDK-8182307 >>>>> >>>>> The fix is copied from Jerry's work on the following bugs: >>>>> >>>>> ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris >>>>> https://bugs.openjdk.java.net/browse/JDK-8182757 >>>>> >>>>> ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with >>>>> Exception >>>>> https://bugs.openjdk.java.net/browse/JDK-8178676 >>>>> >>>>> For the gory details of the reasons for this fix please see >>>>> Jerry's bugs. >>>>> >>>>> Webrev URL: >>>>> http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >>>>> >>>>> >>>>> I observed this test failure one time in my Thread-SMR work and have >>>>> been including this fix in months of Thread-SMR stress testing. It >>>>> was >>>>> also included in both JPRT and Mach5 tier[1-5] testing of the >>>>> Thread-SMR >>>>> changes. >>>>> >>>>> Thanks, in advance, for any comments, suggestions or questions. >>>>> >>>>> Dan >>>>> >>>> >>> >> > From forax at univ-mlv.fr Thu Dec 7 18:12:33 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 7 Dec 2017 19:12:33 +0100 (CET) Subject: Finalization and dead references: another proposal In-Reply-To: References: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> Message-ID: <2057035983.1969283.1512670353891.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Vitaly Davidovich" > ?: "Peter Levart" > Cc: "core-libs-dev" > Envoy?: Jeudi 7 D?cembre 2017 18:46:41 > Objet: Re: Finalization and dead references: another proposal > On Thu, Dec 7, 2017 at 12:28 PM Peter Levart wrote: > >> Hi, >> >> On 12/07/2017 03:27 AM, Vitaly Davidovich wrote: >> > So kind of the opposite of WeakReference - a SuperStrongReference :). >> > >> > Kidding aside, it seems like the way you?d want to encapsulate this at >> the >> > language level is via a type that the JVM intrinsically knows about; in >> > this way it?s similar to the reference types today. >> > >> > An annotation probably does the trick when the value doesn?t escape from >> > the enclosing instance but I?ve no idea if that assumption covers enough >> > code to warrant this approach. AFAICT, if the value escapes into an >> > instance of another type that doesn?t annotate its field then all bets >> are >> > off. >> > >> > Having a wrapper type would at least make it harder to leak the native >> > handle vs the annotation approach. But of course the wrapper comes with >> > footprint and indirection costs. Maybe Valhalla could allow exposing >> some >> > magic value type that?s a zero-cost wrapper but preserves the type >> > information the JIT can track? >> >> There is a middle-ground. By (ab)using value types only and no special >> JIT magic. >> >> DirectBuffer(s) for example, could return the address not in a long, but >> in a value type like the following (using valhalla MVT speak): >> >> public __ByValue final class Address { >> private final long address; >> private final Object referent; >> >> public _ValueFactory static Address create(long address, Object >> referent) { >> Address a = __MakeDefault Address(); >> a.address = address; >> a.referent = referent; >> return a; >> } >> } >> >> >> DirectByteBuffer for example, would have the following address() method: >> >> private long address; >> >> public Address address() { >> return Address.create(address, this); >> } >> >> Notice that 'address' field of Address value is encapsulated, so Java >> code can't access it directly nor it needs to. Native / Unsafe methods >> would be taking Address values instead of long(s), making sure the >> embedded referent is kept reachable. >> >> This is equivalent to passing the DirectBuffer reference to the native >> method(s) together with the address value, but enforced by the API so >> the programmer can not make a mistake. There's a lot of arithmetic going >> on with addresses, inside and outside of DirectBuffer implementations. >> Such arithmetic would have to be performed by the Address value type >> itself, making sure the results of operations on Address values are >> Address values that maintain the same referent. For example: >> >> public __ByValue final class Address { >> ... >> public _ValueFactory Address plus(long offset) { >> Address a = __MakeDefault Address(); >> a.address = address + offset; >> a.referent = referent; >> return a; >> } >> ... >> >> So no magic here. Just API. > > This is an API version of Hans?s #3 approach. As he said, there?s > performance overhead and nothing guarantees that the referent is kept alive > - that?s an implementation artifact. it's not even true, the implementation do not even guarantee that the referent field will be kept on stack, a value type can disappear completely if its field are not used, so without a reachabilityFence somewhere, it will not work. > > I think without the VM knowing about these things intrinsically it?s not a > 100% reliable solution because it?s not concretely requesting a certain > behavior. here is another approach based on the Peter idea, what we want is that in order to access to an address, you have to send an object that will be kept until the end of the call because the is a call to reachabilityFence at the end, so class DirectByteBuffer { private long address; void compute(LongConsumer consumer) { consumer.accept(address); Reference.reachabilityFence(this); } } so with the example of Hans, private static native void nativeDoSomething(long nativePtr, long anotherNativePtr); a call to nativeDoSomething, is buffer1.compute(address1 -> buffer2.compute(address2 -> nativeDoSomething(address1, address2))); so at least in term of API it's doable, the question is how to be sure that the JIT will inline the whole thing. > >> >> >> Regards, Peter >> >> >> -- > Sent from my phone cheers, R?mi From vitalyd at gmail.com Thu Dec 7 18:21:00 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 7 Dec 2017 13:21:00 -0500 Subject: Finalization and dead references: another proposal In-Reply-To: <2057035983.1969283.1512670353891.JavaMail.zimbra@u-pem.fr> References: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> <2057035983.1969283.1512670353891.JavaMail.zimbra@u-pem.fr> Message-ID: On Thu, Dec 7, 2017 at 1:12 PM, Remi Forax wrote: > ----- Mail original ----- > > De: "Vitaly Davidovich" > > ?: "Peter Levart" > > Cc: "core-libs-dev" > > Envoy?: Jeudi 7 D?cembre 2017 18:46:41 > > Objet: Re: Finalization and dead references: another proposal > > > On Thu, Dec 7, 2017 at 12:28 PM Peter Levart > wrote: > > > >> Hi, > >> > >> On 12/07/2017 03:27 AM, Vitaly Davidovich wrote: > >> > So kind of the opposite of WeakReference - a SuperStrongReference :). > >> > > >> > Kidding aside, it seems like the way you?d want to encapsulate this at > >> the > >> > language level is via a type that the JVM intrinsically knows about; > in > >> > this way it?s similar to the reference types today. > >> > > >> > An annotation probably does the trick when the value doesn?t escape > from > >> > the enclosing instance but I?ve no idea if that assumption covers > enough > >> > code to warrant this approach. AFAICT, if the value escapes into an > >> > instance of another type that doesn?t annotate its field then all bets > >> are > >> > off. > >> > > >> > Having a wrapper type would at least make it harder to leak the > native > >> > handle vs the annotation approach. But of course the wrapper comes > with > >> > footprint and indirection costs. Maybe Valhalla could allow exposing > >> some > >> > magic value type that?s a zero-cost wrapper but preserves the type > >> > information the JIT can track? > >> > >> There is a middle-ground. By (ab)using value types only and no special > >> JIT magic. > >> > >> DirectBuffer(s) for example, could return the address not in a long, but > >> in a value type like the following (using valhalla MVT speak): > >> > >> public __ByValue final class Address { > >> private final long address; > >> private final Object referent; > >> > >> public _ValueFactory static Address create(long address, Object > >> referent) { > >> Address a = __MakeDefault Address(); > >> a.address = address; > >> a.referent = referent; > >> return a; > >> } > >> } > >> > >> > >> DirectByteBuffer for example, would have the following address() method: > >> > >> private long address; > >> > >> public Address address() { > >> return Address.create(address, this); > >> } > >> > >> Notice that 'address' field of Address value is encapsulated, so Java > >> code can't access it directly nor it needs to. Native / Unsafe methods > >> would be taking Address values instead of long(s), making sure the > >> embedded referent is kept reachable. > >> > >> This is equivalent to passing the DirectBuffer reference to the native > >> method(s) together with the address value, but enforced by the API so > >> the programmer can not make a mistake. There's a lot of arithmetic going > >> on with addresses, inside and outside of DirectBuffer implementations. > >> Such arithmetic would have to be performed by the Address value type > >> itself, making sure the results of operations on Address values are > >> Address values that maintain the same referent. For example: > >> > >> public __ByValue final class Address { > >> ... > >> public _ValueFactory Address plus(long offset) { > >> Address a = __MakeDefault Address(); > >> a.address = address + offset; > >> a.referent = referent; > >> return a; > >> } > >> ... > >> > >> So no magic here. Just API. > > > > This is an API version of Hans?s #3 approach. As he said, there?s > > performance overhead and nothing guarantees that the referent is kept > alive > > - that?s an implementation artifact. > > it's not even true, the implementation do not even guarantee that the > referent field will be kept on stack, > a value type can disappear completely if its field are not used, > so without a reachabilityFence somewhere, it will not work. > Not sure what you mean here. Even if the value type is scalar replaced (which I'd actually hope to be the case for something like this, but that's an aside), it doesn't change anything - that referent is passed into a black hole (i.e. the JNI call) that, as of today, the JIT cannot reason around. As such, the referent is kept alive across the call. That's #3 that Hans mentioned. The value type aspect doesn't change anything other than not requiring a separate heap object to be the wrapper. > > > > > I think without the VM knowing about these things intrinsically it?s not > a > > 100% reliable solution because it?s not concretely requesting a certain > > behavior. > > here is another approach based on the Peter idea, > what we want is that in order to access to an address, you have to send an > object that will be kept until the end of the call because the is a call to > reachabilityFence at the end, so > > class DirectByteBuffer { > private long address; > > void compute(LongConsumer consumer) { > consumer.accept(address); > Reference.reachabilityFence(this); > } > } > > > so with the example of Hans, > private static native void nativeDoSomething(long nativePtr, long > anotherNativePtr); > > a call to nativeDoSomething, is > buffer1.compute(address1 -> buffer2.compute(address2 -> > nativeDoSomething(address1, address2))); > > so at least in term of API it's doable, the question is how to be sure > that the JIT will inline the whole thing. > It's not just inlining, but escape analysis as well (for this particular example). And this particular API will be very much a victim of profile pollution, and so inlining will stop once this goes megamorphic (and it probably will). Once inlining stops, EA will most likely fall apart as well. > > > > >> > >> > >> Regards, Peter > >> > >> > >> -- > > Sent from my phone > > cheers, > R?mi > From peter.levart at gmail.com Thu Dec 7 18:22:15 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 7 Dec 2017 19:22:15 +0100 Subject: Finalization and dead references: another proposal In-Reply-To: References: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> Message-ID: <1231ce93-3602-7a7d-5a2a-fbca314c8715@gmail.com> On 12/07/2017 06:46 PM, Vitaly Davidovich wrote: > > So no magic here. Just API. > > This is an API version of Hans?s #3 approach.? As he said, there?s > performance overhead and nothing guarantees that the referent is kept > alive - that?s an implementation artifact. > > I think without the VM knowing about these things intrinsically it?s > not a 100% reliable solution because it?s not concretely requesting a > certain behavior. I would say that for JNI "there?s performance overhead XOR nothing guarantees that the referent is kept alive", because the overhead is caused by reference parameter management that guarantees that the referents are kept alive while the native method executes and may access them. Can we live with such overhead? I don't know. Would have to measure if it really presents a problem. The magic would have to be performed by intrinsified method(s) (Unsafe for example), but they are just few and developed by select group of developers. The main problem I think is not the overhead of passing referents together with addresses to JNI functions and their overheads, but lack of enforcement for this to be performed consistently. It's just too easy to split the native resource from the referent that keeps it into two parts with each having separate lifetime. API with value types could enforce such pairs (address, referent) to be kept together until they hit the final leaf operation which is typically a native method where the referent(s) are either automatically kept alive or would have to be manually, but such methods are in minority. Regards, Peter From vitalyd at gmail.com Thu Dec 7 18:35:52 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 7 Dec 2017 13:35:52 -0500 Subject: Finalization and dead references: another proposal In-Reply-To: <1231ce93-3602-7a7d-5a2a-fbca314c8715@gmail.com> References: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> <1231ce93-3602-7a7d-5a2a-fbca314c8715@gmail.com> Message-ID: On Thu, Dec 7, 2017 at 1:22 PM, Peter Levart wrote: > > > On 12/07/2017 06:46 PM, Vitaly Davidovich wrote: > > So no magic here. Just API. > > This is an API version of Hans?s #3 approach. As he said, there?s > performance overhead and nothing guarantees that the referent is kept alive > - that?s an implementation artifact. > > I think without the VM knowing about these things intrinsically it?s not a > 100% reliable solution because it?s not concretely requesting a certain > behavior. > > > I would say that for JNI "there?s performance overhead XOR nothing > guarantees that the referent is kept alive", because the overhead is caused > by reference parameter management that guarantees that the referents are > kept alive while the native method executes and may access them. Can we > live with such overhead? I don't know. Would have to measure if it really > presents a problem. > I'd say the bigger overhead is in JNI argument marshaling. But JNI is already costly so I don't know. > > The magic would have to be performed by intrinsified method(s) (Unsafe for > example), but they are just few and developed by select group of developers. > > The main problem I think is not the overhead of passing referents together > with addresses to JNI functions and their overheads, but lack of > enforcement for this to be performed consistently. It's just too easy to > split the native resource from the referent that keeps it into two parts > with each having separate lifetime. API with value types could enforce such > pairs (address, referent) to be kept together until they hit the final leaf > operation which is typically a native method where the referent(s) are > either automatically kept alive or would have to be manually, but such > methods are in minority. > I agree that the main problem is how to not "detach" these things by accident. To that end, you want to bundle them together as tightly as possible, and prevent de-encapsulation by the users (or at least make it harder and more explicit). This is why I threw value types out there since I think they might provide a better solution than an annotation, which I suspect will be very easy to accidentally circumvent. But ultimately, you really want the JVM to understand about "foreign resources/pointers" in a 1st class/principled manner, which suggests a type based approach (I'm assuming a language approach is off the table :)). Since this type would be requesting special handling by the JVM, it's hard to make it library/API only. Or rather, it would be hard, if not impossible, to make it library/API only but then not suffer performance consequences. You also don't want to accidentally omit or put the reachabilityFence() in the wrong place or on the wrong object, both things possible with one careless refactoring (gone wrong). > > Regards, Peter > > From serguei.spitsyn at oracle.com Thu Dec 7 18:36:35 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 7 Dec 2017 10:36:35 -0800 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> <35163e84-b100-1b85-3179-a58699163fc5@oracle.com> Message-ID: <968040bb-0923-a930-cab6-fce3e6d718a7@oracle.com> On 12/7/17 10:08, serguei.spitsyn at oracle.com wrote: > Hi Dan, > > The fix looks good to me. > Nice, you have caught it. > > Do you want this fixed in 10 or 11? > I thought that the jdk/hs is for 11 now. > Is it correct? Never mind. I've just found a message from Jesper the jdk/hs is used for 10 pushes for one more week. Thanks, Serguei > > Thanks, > Serguei > > > On 12/7/17 09:09, Daniel D. Daugherty wrote: >> Roger, >> >> Thanks for the review! >> >> Dan >> >> P.S. >> I'm planning to push this fix to jdk/hs since the only sightings >> have been in jdk/hs testing or in projects that are parented to >> jdk/hs repos... Hope that's okay... >> >> >> On 12/7/17 12:07 PM, Roger Riggs wrote: >>> +1 >>> >>> On 12/7/2017 12:00 PM, Gerald Thornbrugh wrote: >>>> Hi Dan, >>>> >>>> Your fix looks good. >>>> >>>> Jerry >>>> >>>>> Adding core-libs-dev at ... since this is RMI code. Thanks Alan! >>>>> >>>>> Folks, this review spans three OpenJDK aliases so Thunderbird's >>>>> reply-to-list feature won't get it right. This is one of the >>>>> few times that reply-to-all is the right thing to do... >>>>> >>>>> Dan >>>>> >>>>> On 12/7/17 11:38 AM, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> I have a small fix for a very intermittent ServerSocket related test >>>>>> failure: >>>>>> >>>>>> ??? JDK-8182307: Error during JRMP connection establishment >>>>>> https://bugs.openjdk.java.net/browse/JDK-8182307 >>>>>> >>>>>> The fix is copied from Jerry's work on the following bugs: >>>>>> >>>>>> ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris >>>>>> https://bugs.openjdk.java.net/browse/JDK-8182757 >>>>>> >>>>>> ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with >>>>>> Exception >>>>>> https://bugs.openjdk.java.net/browse/JDK-8178676 >>>>>> >>>>>> For the gory details of the reasons for this fix please see >>>>>> Jerry's bugs. >>>>>> >>>>>> Webrev URL: >>>>>> http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >>>>>> >>>>>> >>>>>> I observed this test failure one time in my Thread-SMR work and have >>>>>> been including this fix in months of Thread-SMR stress testing. >>>>>> It was >>>>>> also included in both JPRT and Mach5 tier[1-5] testing of the >>>>>> Thread-SMR >>>>>> changes. >>>>>> >>>>>> Thanks, in advance, for any comments, suggestions or questions. >>>>>> >>>>>> Dan >>>>>> >>>>> >>>> >>> >> > From daniel.daugherty at oracle.com Thu Dec 7 18:46:27 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 7 Dec 2017 13:46:27 -0500 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> <35163e84-b100-1b85-3179-a58699163fc5@oracle.com> Message-ID: On 12/7/17 1:08 PM, serguei.spitsyn at oracle.com wrote: > Hi Dan, > > The fix looks good to me. > Nice, you have caught it. Thanks for the review! > Do you want this fixed in 10 or 11? > I thought that the jdk/hs is for 11 now. > Is it correct? Please see Mark R's e-mail with the subject line of "JDK 10 enters Rampdown Phase One in one week". In short, the RDP1 deadline applies to jdk/jdk, jdk/hs and jdk/client all at the same time. Dan > > Thanks, > Serguei > > > On 12/7/17 09:09, Daniel D. Daugherty wrote: >> Roger, >> >> Thanks for the review! >> >> Dan >> >> P.S. >> I'm planning to push this fix to jdk/hs since the only sightings >> have been in jdk/hs testing or in projects that are parented to >> jdk/hs repos... Hope that's okay... >> >> >> On 12/7/17 12:07 PM, Roger Riggs wrote: >>> +1 >>> >>> On 12/7/2017 12:00 PM, Gerald Thornbrugh wrote: >>>> Hi Dan, >>>> >>>> Your fix looks good. >>>> >>>> Jerry >>>> >>>>> Adding core-libs-dev at ... since this is RMI code. Thanks Alan! >>>>> >>>>> Folks, this review spans three OpenJDK aliases so Thunderbird's >>>>> reply-to-list feature won't get it right. This is one of the >>>>> few times that reply-to-all is the right thing to do... >>>>> >>>>> Dan >>>>> >>>>> On 12/7/17 11:38 AM, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> I have a small fix for a very intermittent ServerSocket related test >>>>>> failure: >>>>>> >>>>>> ??? JDK-8182307: Error during JRMP connection establishment >>>>>> https://bugs.openjdk.java.net/browse/JDK-8182307 >>>>>> >>>>>> The fix is copied from Jerry's work on the following bugs: >>>>>> >>>>>> ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris >>>>>> https://bugs.openjdk.java.net/browse/JDK-8182757 >>>>>> >>>>>> ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with >>>>>> Exception >>>>>> https://bugs.openjdk.java.net/browse/JDK-8178676 >>>>>> >>>>>> For the gory details of the reasons for this fix please see >>>>>> Jerry's bugs. >>>>>> >>>>>> Webrev URL: >>>>>> http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >>>>>> >>>>>> >>>>>> I observed this test failure one time in my Thread-SMR work and have >>>>>> been including this fix in months of Thread-SMR stress testing. >>>>>> It was >>>>>> also included in both JPRT and Mach5 tier[1-5] testing of the >>>>>> Thread-SMR >>>>>> changes. >>>>>> >>>>>> Thanks, in advance, for any comments, suggestions or questions. >>>>>> >>>>>> Dan >>>>>> >>>>> >>>> >>> >> > From peter.levart at gmail.com Thu Dec 7 18:46:47 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 7 Dec 2017 19:46:47 +0100 Subject: Finalization and dead references: another proposal In-Reply-To: <2057035983.1969283.1512670353891.JavaMail.zimbra@u-pem.fr> References: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> <2057035983.1969283.1512670353891.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Remi, On 12/07/2017 07:12 PM, Remi Forax wrote: >>> So no magic here. Just API. >> This is an API version of Hans?s #3 approach. As he said, there?s >> performance overhead and nothing guarantees that the referent is kept alive >> - that?s an implementation artifact. > it's not even true, the implementation do not even guarantee that the referent field will be kept on stack, > a value type can disappear completely if its field are not used, > so without a reachabilityFence somewhere, it will not work. > What do you mean by saying "the implementation do not even guarantee that the referent field will be kept on stack". If a referent is passed through a reference typed parameter to the JNI method (there's no other way actually), then such referent must be kept alive for the entire time the native method executes, because native method may call methods or access fields on such referent. Because there's no cross-native-baundary optimization going on currently, JNI must guarantee that. If/when there will be cross-native-baundary optimization, I suppose native side would have to have an equivalent to reachabilityFence too ... Regards, Peter From martinrb at google.com Thu Dec 7 19:02:16 2017 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Dec 2017 11:02:16 -0800 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works In-Reply-To: References: Message-ID: On Thu, Dec 7, 2017 at 11:00 AM, Martin Buchholz wrote: > You went to the trouble of checking test.src. Shouldn't CONFIG_FILE be > found relative to SRC_DIR? > > Oh never mind, I now see you did + Path initialProps = SRC_DIR.resolve(CONFIG_FILE); + Path loggingProps = USER_DIR.resolve(CONFIG_FILE); > + public static final Path SRC_DIR = > + Paths.get(System.getProperty("test.src", "src")); > + public static final Path USER_DIR = > + Paths.get(System.getProperty("user.dir", ".")); > + public static final Path CONFIG_FILE = Paths.get("logging.properties" > ); > > From peter.levart at gmail.com Thu Dec 7 19:03:31 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 7 Dec 2017 20:03:31 +0100 Subject: Finalization and dead references: another proposal In-Reply-To: <1231ce93-3602-7a7d-5a2a-fbca314c8715@gmail.com> References: <97b6b137-da14-bba4-717f-8c08911c10d2@gmail.com> <1231ce93-3602-7a7d-5a2a-fbca314c8715@gmail.com> Message-ID: <38a77a6b-ad25-ecf7-254a-4e42f2c26c96@gmail.com> On 12/07/2017 07:22 PM, Peter Levart wrote: > > > On 12/07/2017 06:46 PM, Vitaly Davidovich wrote: >> >> So no magic here. Just API. >> >> This is an API version of Hans?s #3 approach. As he said, there?s >> performance overhead and nothing guarantees that the referent is kept >> alive - that?s an implementation artifact. >> >> I think without the VM knowing about these things intrinsically it?s >> not a 100% reliable solution because it?s not concretely requesting a >> certain behavior. > > I would say that for JNI "there?s performance overhead XOR nothing > guarantees that the referent is kept alive", because the overhead is > caused by reference parameter management that guarantees that the > referents are kept alive while the native method executes and may > access them. Can we live with such overhead? I don't know. Would have > to measure if it really presents a problem. > > The magic would have to be performed by intrinsified method(s) (Unsafe > for example), but they are just few and developed by select group of > developers. > > The main problem I think is not the overhead of passing referents > together with addresses to JNI functions and their overheads, but lack > of enforcement for this to be performed consistently. It's just too > easy to split the native resource from the referent that keeps it into > two parts with each having separate lifetime. API with value types > could enforce such pairs (address, referent) to be kept together until > they hit the final leaf operation which is typically a native method > where the referent(s) are either automatically kept alive or would > have to be manually, but such methods are in minority. > > Regards, Peter > There's a quick solution to get rid of that overhead. Each native method (or intrinsified leaf method such as those on Unsafe) can have a java front end. They are in minority and kept under control. Similar to argument validation front-ends for intrinsified methods. For example: public class Unsafe { ??? public Address allocateMemory(long bytes, Object referent) { ??? ??? return Address.create(allocateMemory(bytes), referent); ??? } ??? public void setMemory(Address address, long bytes, byte value) { ??? ??? setMemory(/* access check absent equivalent of */ address.address, bytes, value); ??? ??? Reference.reachabilityFence(/* access check absent equivalent of */ address.referent); ??? } ??? ... Regards, Peter From martinrb at google.com Thu Dec 7 19:00:38 2017 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Dec 2017 11:00:38 -0800 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works In-Reply-To: References: Message-ID: You went to the trouble of checking test.src. Shouldn't CONFIG_FILE be found relative to SRC_DIR? + public static final Path SRC_DIR = + Paths.get(System.getProperty("test.src", "src")); + public static final Path USER_DIR = + Paths.get(System.getProperty("user.dir", ".")); + public static final Path CONFIG_FILE = Paths.get("logging.properties"); From martinrb at google.com Thu Dec 7 19:15:54 2017 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Dec 2017 11:15:54 -0800 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works In-Reply-To: References: Message-ID: I'm not a logging expert, but this change Looks Good To Me. --- A bunch of "handler" should be changed to "handlers" + // Verify that exactly one of the two handler is a custom.Handler + // Verify that exactly one of the two handler is a custom.DotHandler + // Verify that the two handler have an id of '1' -- This code makes me think you use the identity of the Longs, but it seems you don't, so I would just get rid of these and use e.g. 3L instead of THREE. + public static final Long ONE = 1L; + public static final Long TWO = 2L; + public static final Long THREE = 3L; From brian.burkhalter at oracle.com Thu Dec 7 19:27:49 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 7 Dec 2017 11:27:49 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <9e7c216e-9fc7-88e6-0dec-37a59adbdb4f@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> <9e7c216e-9fc7-88e6-0dec-37a59adbdb4f@oracle.com> Message-ID: Updated patch: http://cr.openjdk.java.net/~bpb/4358774/webrev.01/ On Dec 7, 2017, at 6:08 AM, Alan Bateman wrote: > If nothing else, a private ensureOpen method would make it easier to read so that the "if (closed) throw ..." isn't needed in every method. Done. On Dec 6, 2017, at 7:03 PM, Vitaly Davidovich wrote: > From a performance angle, I'd be more concerned with the calls to Objects.xyz() methods there. Unless something has changed in the JIT recently, those are susceptible to profile pollution and can cause missed optimizations. I'd inline those methods manually to give these methods their own profiles. Done. Thanks, Brian From john.r.rose at oracle.com Thu Dec 7 19:50:01 2017 From: john.r.rose at oracle.com (John Rose) Date: Thu, 7 Dec 2017 11:50:01 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <8FE723F7-9127-4B6C-A09F-2E4AFDE0AE5F@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <8FE723F7-9127-4B6C-A09F-2E4AFDE0AE5F@oracle.com> Message-ID: > On Dec 6, 2017, at 5:16 PM, Paul Sandoz wrote: > > It can, since final fields are not treated as really final (unless in java.lang.invoke package, where it?s as if all such fields are annotated with @Stable). C2 will treat the field value a constant value if the collection is held in a static final field. We could tweak the semantics of @Stable to omit the zero corner case for a final field. Then the annotation on a non-array field would mean ?trust this final?. It would be yet another stop gap before that far-off day when we fix finals (and arrays). Such new uses of @Stable would have to be evaluated for any interaction with deserialization, so it?s not a simple decision. From guy.steele at oracle.com Thu Dec 7 19:39:29 2017 From: guy.steele at oracle.com (Guy Steele) Date: Thu, 7 Dec 2017 14:39:29 -0500 Subject: Draft JEP: Enhanced pseudo-random number generators Message-ID: <7D8C726E-BD86-4FF0-8BB9-0FB0893C73E6@oracle.com> Please check out a new draft JEP https://bugs.openjdk.java.net/browse/JDK-8193209 that proposes new interface types and implementations for pseudo-random number generators. ?Guy From forax at univ-mlv.fr Thu Dec 7 20:05:07 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 7 Dec 2017 21:05:07 +0100 (CET) Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <8FE723F7-9127-4B6C-A09F-2E4AFDE0AE5F@oracle.com> Message-ID: <2135054645.1984917.1512677107908.JavaMail.zimbra@u-pem.fr> You have also the problem of people doing a lot of things in the constructor, by example class Foo { @Stable final int x; Foo() { m(); x = 3; m(); } void m() { for(...) { // use x here } } } here, m() can be JITed with x equals to 0 as a constant and the code of m() be re-use for the second call even if the value of x has changed in the middle. in that case, either you need to maintain dependency between the JITed code and the field 'x' or have a bit saying if an object is fully initialized or not, this bit will be true at the end of the constructor and can be set for the de-serialization too. R?mi ----- Mail original ----- > De: "John Rose" > ?: "Paul Sandoz" > Cc: "core-libs-dev" > Envoy?: Jeudi 7 D?cembre 2017 20:50:01 > Objet: Re: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() >> On Dec 6, 2017, at 5:16 PM, Paul Sandoz wrote: >> >> It can, since final fields are not treated as really final (unless in >> java.lang.invoke package, where it?s as if all such fields are annotated with >> @Stable). C2 will treat the field value a constant value if the collection is >> held in a static final field. > > We could tweak the semantics of @Stable to omit the zero corner case for a final > field. Then the annotation on a non-array field would mean ?trust this final?. > > It would be yet another stop gap before that far-off day when we fix finals (and > arrays). Such new uses of @Stable would have to be evaluated for any > interaction with deserialization, so it?s not a simple decision. From john.r.rose at oracle.com Thu Dec 7 20:16:50 2017 From: john.r.rose at oracle.com (John Rose) Date: Thu, 7 Dec 2017 12:16:50 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <2135054645.1984917.1512677107908.JavaMail.zimbra@u-pem.fr> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <8FE723F7-9127-4B6C-A09F-2E4AFDE0AE5F@oracle.com> <2135054645.1984917.1512677107908.JavaMail.zimbra@u-pem.fr> Message-ID: <6461E0B3-FC1D-4CC7-80E2-34DBAD8A2DA3@oracle.com> On Dec 7, 2017, at 12:05 PM, Remi Forax wrote: > > have a bit saying if an object is fully initialized or not, this bit will be true at the end of the constructor and can be set for the de-serialization too Yes, that's the hard part of fixing final. Sometimes we call this the "slushy bit". It would almost always be false, but would be true for those unusual objects which are in the process of deserialization, and also (perhaps) objects which may escape into the optimizer while undergoing normal construction. The latter version of "slushy" is probably a statically computable condition, unlike "slushy during deserialization". From Roger.Riggs at Oracle.com Thu Dec 7 22:04:33 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 7 Dec 2017 17:04:33 -0500 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <750AD53B-016C-4FC2-8B5C-43356562A5FC@oracle.com> <9e7c216e-9fc7-88e6-0dec-37a59adbdb4f@oracle.com> Message-ID: <3c3e9704-650c-ee6b-3b6d-fbfbacb7dd84@Oracle.com> Hi Brian, For checking indices, I think you should leverage the work done for java.util.Objects.checkFromIndexSize(...) as optimized for this purpose in 8135248.? That was extensively designed and reviewed for optimal performance. Regards, Roger [1] https://bugs.openjdk.java.net/browse/JDK-8135248 On 12/7/2017 2:27 PM, Brian Burkhalter wrote: > Updated patch: > > http://cr.openjdk.java.net/~bpb/4358774/webrev.01/ > > > On Dec 7, 2017, at 6:08 AM, Alan Bateman > wrote: > >> If nothing else, a private ensureOpen method would make it easier to >> read so that the "if (closed) throw ..." isn't needed in every method. > > Done. > > On Dec 6, 2017, at 7:03 PM, Vitaly Davidovich > wrote: > >> From a performance angle, I'd be more concerned with the calls to >> Objects.xyz() methods there. Unless something has changed in the JIT >> recently, those are susceptible to profile pollution and can cause >> missed optimizations.? I'd inline those methods manually to give >> these methods their own profiles. > > Done. > > Thanks, > > Brian From brian.burkhalter at oracle.com Thu Dec 7 22:55:19 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 7 Dec 2017 14:55:19 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream References: Message-ID: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> Hi Roger, I agree. It does not seem that whatever performance improvement might accrue from not using the Objects methods is offset by the increased code readability in addition to mitigating the risks mentioned in [1]. I have reinstated this approach in http://cr.openjdk.java.net/~bpb/4358774/webrev.02/. Thanks, Brian On Dec 7, 2017, at 2:04 PM, Roger Riggs wrote: > For checking indices, I think you should leverage the work done for java.util.Objects.checkFromIndexSize(...) > as optimized for this purpose in 8135248. That was extensively designed and reviewed for optimal performance. > > Regards, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8135248 From stuart.marks at oracle.com Thu Dec 7 22:58:35 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 7 Dec 2017 14:58:35 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> Message-ID: <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> [Martin: please review changes to the JSR 166 TCK.] Another updated webrev for this changeset: http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.2/ This includes an override of toArray(IntFunction) in the implementation of Arrays.asList(), as requested by Tagir Valeev. Overrides are also added for various wrapper classes in j.u.Collections. Turns out there's a test test/jdk/java/util/Collections/Wrappers.java that checks to ensure that the wrappers don't inherit any default methods. (This has been a source of bugs in the past.) Significantly, this webrev also includes changes to several tests in the JSR 166 TCK. The issue is that these tests have cases for null handling, where they call coll.toArray(null) to ensure that NPE is thrown. Given that this method is now overloaded: toArray(T[]) toArray(IntFunction) passing null is now ambiguous and thus generates a compiler error. I've modified the tests to call toArray((Object[])null) which is a bit of a stopgap. I can't think of anything obviously better to do, though. s'marks From Sergey.Bylokhov at oracle.com Thu Dec 7 23:14:25 2017 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Thu, 7 Dec 2017 15:14:25 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> Message-ID: On 07/12/2017 14:55, Brian Burkhalter wrote: > Hi Roger, > > I agree. It does not seem that whatever performance improvement might accrue from not using the Objects methods is offset by the increased code readability in addition to mitigating the risks mentioned in [1]. I have reinstated this approach in http://cr.openjdk.java.net/~bpb/4358774/webrev.02/. Why the Objects.requireNonNull(b); should be called before Objects.checkFromIndexSize(off, len, b.length); since the second call will gen NPW at b.length anyway? > > Thanks, > > Brian > > On Dec 7, 2017, at 2:04 PM, Roger Riggs wrote: > > >> For checking indices, I think you should leverage the work done for java.util.Objects.checkFromIndexSize(...) >> as optimized for this purpose in 8135248. That was extensively designed and reviewed for optimal performance. >> >> Regards, Roger >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8135248 > -- Best regards, Sergey. From brian.burkhalter at oracle.com Thu Dec 7 23:26:36 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 7 Dec 2017 15:26:36 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> Message-ID: Hi Sergey, On Dec 7, 2017, at 3:14 PM, Sergey Bylokhov wrote: >> http://cr.openjdk.java.net/~bpb/4358774/webrev.02/. > > Why the > Objects.requireNonNull(b); > should be called before > Objects.checkFromIndexSize(off, len, b.length); > since the second call will gen NPW at b.length anyway? Good point. I?ll update after I see whether any other updates will be needed. I think this situation probably obtains in some other files as well. Thanks, Brian From claes.redestad at oracle.com Thu Dec 7 23:41:55 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 8 Dec 2017 00:41:55 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> Message-ID: <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> On 2017-12-07 01:12, Claes Redestad wrote: > SubList is now final, inherits from AbstractImmutableList instead of > AbstractList and reuses more of the shared code. > > I also moved 'implements Serializable' from AbstractImmutableList to > List12 and ListN respectively to not > change the behavior that List.of(0,1) is serializable while > List.of(0,1).subList(0,1) isn't. While doing this, I realized a few issues with my patch around the subList mechanics: - I had "optimized" AbstractImmutable::subList to return self or emptyList if appropriate, which sounded nice, but is in fact an incompatible change (some subLists become Serializable). - the ListFactories test didn't explicitly verify that sublists retrieved from various List.of() impls aren't Serializable. Added tests to check no sub-list is Serializable, and as a bonus this test now verifies that List.of(...).subList(...) behave like their List.of(..) counterparts in most other ways. - fixed range check in AbstractImmutableList.listIterator(0) to not throw IOOBE if the list is empty http://cr.openjdk.java.net/~redestad/8193128/open.02/ Thanks! /Claes From jbluettduncan at gmail.com Thu Dec 7 23:50:17 2017 From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan) Date: Thu, 7 Dec 2017 23:50:17 +0000 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: Hi Stuart, Looking over http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.2/src/java.base/share/classes/java/util/ArrayList.java.cdiff.html, I'm wondering if the method `ArrayList.toArray(IntFunction)` should have an `@Override` to make it clear that it's overriding the default method in Collection. What do you think? Cheers, Jonathan On 7 December 2017 at 22:58, Stuart Marks wrote: > [Martin: please review changes to the JSR 166 TCK.] > > Another updated webrev for this changeset: > > http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.2/ > > This includes an override of toArray(IntFunction) in the implementation of > Arrays.asList(), as requested by Tagir Valeev. > > Overrides are also added for various wrapper classes in j.u.Collections. > Turns out there's a test test/jdk/java/util/Collections/Wrappers.java > that checks to ensure that the wrappers don't inherit any default methods. > (This has been a source of bugs in the past.) > > Significantly, this webrev also includes changes to several tests in the > JSR 166 TCK. The issue is that these tests have cases for null handling, > where they call > > coll.toArray(null) > > to ensure that NPE is thrown. Given that this method is now overloaded: > > toArray(T[]) > toArray(IntFunction) > > passing null is now ambiguous and thus generates a compiler error. I've > modified the tests to call toArray((Object[])null) which is a bit of a > stopgap. I can't think of anything obviously better to do, though. > > s'marks > From chris.hegarty at oracle.com Thu Dec 7 23:59:08 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 7 Dec 2017 23:59:08 +0000 Subject: RFR: jsr166 jdk integration 2017-12-XX In-Reply-To: <9F6FC09C-7DE1-47EB-9162-FAAA23624A58@oracle.com> References: <9F6FC09C-7DE1-47EB-9162-FAAA23624A58@oracle.com> Message-ID: <0FF94615-FC9D-47B1-B8A9-B6B13B461C64@oracle.com> ++1 -Chris > On 7 Dec 2017, at 18:03, Paul Sandoz wrote: > > +1 > > Paul. > >> On 7 Dec 2017, at 09:14, Martin Buchholz wrote: >> >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html >> >> With the change to the openjdk release model, jsr166 integrations will simply be "jdk integrations" instead of e.g. "jdk10 integrations" from now on. (Nevertheless, we try to get in important bug fixes before jdk release gates shut) >> >> 8193174: SubmissionPublisher invokes the Subscriber's onComplete before all of its submitted items have been published >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/SubmissionPublisher-race/index.html >> https://bugs.openjdk.java.net/browse/JDK-8193174 >> >> 8192943: Optimize atomic accumulators using VarHandle getAndSet >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/getAndSet/index.html >> https://bugs.openjdk.java.net/browse/JDK-8192943 >> >> 8192944: Miscellaneous changes imported from jsr166 CVS 2017-12-XX >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html >> https://bugs.openjdk.java.net/browse/JDK-8192944 >> > From mandy.chung at oracle.com Thu Dec 7 23:59:59 2017 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Dec 2017 15:59:59 -0800 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works In-Reply-To: References: Message-ID: <1825e86c-ad8f-c4c5-9fc2-fbe8892afeda@oracle.com> On 12/7/17 6:38 AM, Daniel Fuchs wrote: > Hi, > > Please find below a patch for: > > 8191033 Regression in logging.properties: specifying .handlers= for > ??????? root logger (instead of handlers=) no longer works > https://bugs.openjdk.java.net/browse/JDK-8191033 > > webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8191033/webrev.00/ Looks good to me. Mandy From hboehm at google.com Fri Dec 8 00:01:14 2017 From: hboehm at google.com (Hans Boehm) Date: Thu, 7 Dec 2017 16:01:14 -0800 Subject: Finalization and dead references: another proposal In-Reply-To: References: Message-ID: Some sort of zero cost wrapper type for native pointers, with semantics similar to the annotation, would certainly be better than just the annotation. My best guess is that it would cover something like 90% of use cases. I certainly don't want to preclude it, once we have such a mechanism. But I'm not yet convinced that a "native pointer" type covers enough of the use cases that it can serve as the only mechanism. Not all handles requiring some sort of deletion are represented as native pointers. Sometimes this problem does arise in pure Java contexts, e.g. FinalizableDelegatedExecutorService. And there are cases in which the Java object being cleaned owns the native pointer indirectly, via another Java object. On Wed, Dec 6, 2017 at 6:27 PM, Vitaly Davidovich wrote: > On Wed, Dec 6, 2017 at 7:38 PM Hans Boehm wrote: > >> We're still trying to deal with a fair amount of code that implicitly >> assumes that finalization or similar clean-up will not occur while a >> pointer to the affected object is in scope. Which is of course not true. >> >> As a reminder, the canonical offending usage (anti-)pattern with >> (deprecated, but easier to write) finalizers is >> >> class Foo { >> private long mPtrToNativeFoo; >> >> private static native void nativeDeallocate(long nativePtr); >> private static native void nativeDoSomething(long nativePtr, long >> anotherNativePtr); >> >> protected void finalize() { ... nativeDeallocate(mPtrToNativeFoo); >> ... } >> >> public void doSomething(Foo another) { ... >> nativeDoSomething(mPtrToNativeFoo, another.mPtrToNativeFoo) ... } >> ... >> } >> >> This is subtly incorrect in that, while executing the final call to >> doSomething() on a particular object, just after retrieving >> mPtrToNativeFoo >> and another.mPtrToNativeFoo, but before invoking nativeDoSomething(), the >> garbage collector may run, and "this" and "another" may be finalized, >> deallocating the native objects their mPtrToNativeFoos refer to. >> When nativeDoSomething() finally does run, it may see dangling pointers. >> >> Examples using java.lang.ref or Cleaners (or even WeakHashMap, if you >> must) >> instead of finalize() are as easy to construct, but a bit longer. (The >> finalizer version is also arguably incorrect in other ways that are not >> relevant here. Pretend this were written in terms of PhantomReferences.) >> >> It is easily possible to construct 100% Java code with the same problem. >> Instead of mPtrToNativeFoo, each object stores an integer handle that is >> used to access additional Java state logically associated with the object. >> But the native pointer case seems to dominate in practice. >> >> Various solutions to this have been proposed, but none seem quite >> attractive enough that I actually feel comfortable asking people to update >> their code to use them. Noteworthy proposals include: >> >> 0) Explicitly call close() on such objects. Great idea when it works. In >> general it doesn't, since the code needs to know when the enclosing Java >> object is no longer needed. If we always knew that we wouldn't need a GC. >> 1) Various hacks to keep variables live, e.g. the one based on >> synchronized >> blocks I advocated in my 2004 JavaOne talk. These are slow and ugly, as >> we've always admitted. Nobody used them. Incorrect won over slow, ugly, >> and >> complicated ~100% of the time. >> 2) Java 9's reachabilityFence(). This is better. It can be implemented so >> that it's no longer slow. But in many common cases, it's still quite ugly. >> For example, the call to nativeDoSomething() above must be followed by two >> reachabilityFences, one on this and one on another. And to do this really >> correctly, the whole thing would often need to be in a try...finally >> block. >> And in reality code following this pattern usually doesn't have just a >> single doSomething method that needs this treatment, but may easily have >> dozens. And the rules for placing reachabilityFences can become quite >> subtle if there are e.g. locally allocated objects involved. My assessment >> is that this isn't good enough. People may no longer write incorrect code >> 100% of the time, but I'd bet on 70%+. >> 3) JNI functions can be rewritten, so that the containing Java object is >> passed in addition to the native pointers. Somewhat accidentally, this >> happens to be roughly free for single argument functions. (Delete >> "static".) It adds overhead in other cases, like the one above, and the >> rewriting can get somewhat convoluted. In some cases, it doesn't work at >> all. AFAIK, it's never actually guaranteed to be correct; it works because >> standard implementations don't optimize across the language boundary. >> That's not too likely to change. Maybe. >> 4) We could change the language spec to always prohibit premature >> finalization/cleaning in cases like the above. I could personally live >> with >> that solution, and proposed it internally here in the past. But it doesn't >> seem to go over well among implementers. And AFAICT, doing it well >> requires >> significant tooling changes, in that we do want to reliably treat local >> variables as dead once they go out of scope, a piece of information that >> doesn't seem to be reliably preserved in class files. One could argue that >> the current class file design implicitly assumes that we can do dead >> variable elimination. >> >> After going back and forth on this, my conclusion is that we need a >> linguistic mechanism for identifying the case in which the garbage >> collector is being used to managed external resources associated with a >> field. > > So kind of the opposite of WeakReference - a SuperStrongReference :). > > Kidding aside, it seems like the way you?d want to encapsulate this at the > language level is via a type that the JVM intrinsically knows about; in > this way it?s similar to the reference types today. > > An annotation probably does the trick when the value doesn?t escape from > the enclosing instance but I?ve no idea if that assumption covers enough > code to warrant this approach. AFAICT, if the value escapes into an > instance of another type that doesn?t annotate its field then all bets are > off. > > Having a wrapper type would at least make it harder to leak the native > handle vs the annotation approach. But of course the wrapper comes with > footprint and indirection costs. Maybe Valhalla could allow exposing some > magic value type that?s a zero-cost wrapper but preserves the type > information the JIT can track? > >> A (still slowly evolving) proposal to add an annotation to do so is >> at >> >> https://docs.google.com/document/d/1yMC4VzZobMQTrVAraMy7xBdWPCJ0p >> 7G5r_43yaqsj-s/edit?usp=sharing >> >> In many ways, this is inherently a compromise. But in the vast majority of >> cases, it greatly reduces the required source code changes over all but >> (4) >> above. And I think I could usually explain to an author of currently >> broken >> code in under 5 minutes exactly what they need to do to fix it. And I >> wouldn't have to lie much to do so. I don't think (0)-(3) above share that >> property. >> >> This has already benefited from comments provided by a few people. (Thanks >> to Doug, Jeremy, and Martin, among others.) But I would really like more >> feedback, including suggestions about how to proceed. >> >> Hans >> > -- > Sent from my phone > From stuart.marks at oracle.com Fri Dec 8 00:33:41 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 7 Dec 2017 16:33:41 -0800 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() Message-ID: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Hi all, Please review this changeset that introduces a new no-arg method orElseThrow() to Optional, as a preferred alternative to the get() method. Corresponding methods are also added to OptionalDouble, Int, and Long. The orElseThrow() method is functionally identical to get(). It has some affinity with the existing orElseThrow(exceptionSupplier) method, being equivalent to orElseThrow(NoSuchElementException::new). The get() method is functionally unchanged. It is NOT being deprecated, although some wording in the doc has been added to point to orElseThrow() as the "preferred alternative." This is part of a (slow) migration away from Optional.get(), which has an obvious, attractive name that is often misused. These issues have been discussed on this list previously: http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-April/040484.html Please note that much of that discussion was about the then-proposed deprecation of Optional.get(), which is NOT part of this proposal. There are no plans to deprecate Optional.get() at this time. This proposal ONLY introduces a new method orElseThrow() that is identical in function to get(). Bug report: https://bugs.openjdk.java.net/browse/JDK-8140281 Webrev: http://cr.openjdk.java.net/~smarks/reviews/8140281/webrev.1/ Thanks, s'marks From stuart.marks at oracle.com Fri Dec 8 00:47:03 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 7 Dec 2017 16:47:03 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: <3a77f8c0-0c8b-3735-a0a6-f39e2d0f5eb9@oracle.com> On 12/7/17 3:50 PM, Jonathan Bluett-Duncan wrote: > Looking over > http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.2/src/java.base/share/classes/java/util/ArrayList.java.cdiff.html, > I'm wondering if the method `ArrayList.toArray(IntFunction)` should have an > `@Override` to make it clear that it's overriding the default method in > Collection. What do you think? I guess it wouldn't hurt. I had thought about adding it, but most of the methods in ArrayList that override implementations from supertypes *don't* have @Override. Looking more closely, it appears that the ones that override interface default methods *do* have @Override. I don't think there's a rule that says that @Override should be used when overriding interface default methods but not when overriding implementations from a superclass or when implementing an abstract interface method. Frankly, I think the JDK is just sloppy here. Most of the existing implementations didn't use @Override -- possibly because they were introduced before annotations existed -- and later on, whoever implemented the overrides of the Java 8 default methods decided to add @Override. s'marks From hboehm at google.com Fri Dec 8 00:57:51 2017 From: hboehm at google.com (Hans Boehm) Date: Thu, 7 Dec 2017 16:57:51 -0800 Subject: Finalization and dead references: another proposal In-Reply-To: <44204441-d6a5-b13f-355b-003aa4f80171@gmail.com> References: <44204441-d6a5-b13f-355b-003aa4f80171@gmail.com> Message-ID: On Thu, Dec 7, 2017 at 7:49 AM, Peter Levart wrote: > ... > 1st case that comes to mind and is a real case in the JDK is the existence > of sun.nio.ch.DirectBuffer internal (not exported) interface implemented by > NIO direct buffer(s) with method: > > long address(); > > Utility method like the following: > > static void erase(ByteBuffer bb) { > unsafe.setMemory(((DirectBuffer)bb).address(), bb.capacity(), > (byte)0); > } > > ...won't get reachabilityFence(bb) before return, will it? Calling a > method on a bb doesn't count as a dereference of a reachablity-sensitive > field of bb. Unless methods could also be annotated with the same > annotation. This would only work for instance methods that return a native > resource kept in the same instance or some instance that is reachable from > this instance. > > Native resources are typically encapsulated, but there are various levels > of encapsulation. DirectBuffer is an example of module level encapsulation > by non-exported public interface implemented by package-private classes. > I think there are two issues here. The first one is the use of a getter to retrieve the value of the address field. This yields a value whose logical lifetime is limited by the lifetime of a separate garbage-collected Java object. That feels very un-Java-like and dangerous. My immediate inclination is to strongly discourage this idiom. A second issue is a possible access to an @ReachabilitySensitive field from outside the declaring class. I've gone back and forth as to whether they should be covered, i.e. result in a logical reachabilityFence(bb). It's easy enough to specify either way. The major difference is whether an implementation that simply avoids doing dead reference elimination on any class containing an @ReachabilitySensitive annotation is conforming. If you want this to work, the criterion becomes "accesses an @ReachabilitySensitive field". Which may be OK? > 2nd case is artificial. But, since we have streams, lambdas, optionals... > It's easy to fool above rules even if the annotation is put on the > DirectBuffer#address() method ... > > static void doSomethingWith1stNonNull(DirectBuffer... dbs) { > Stream.of(dbs) > .filter(Objects::nonNull) > .mapToLong(DirectBuffer::address) > .findFirst() > .ifPresent(addr -> { > ... do something with addr ... > }); > } > > Even with imperative programming, one can play with scopes: > > static void doSomethingWith1stNonNull(DirectBuffer... dbs) { > long addr = 0; > for (DirectBuffer db : dbs) { > if (db != null) { > addr = db.address(); > break; > } > } > if (addr != 0) { > ... do something with addr ... > } > } > Aside from the above issues, the intent was for these to work. A rechabilityFence(dbs) should be generated at the end of either method (possibly among others). > > Or, hope that this would work (but the above rules don't cover it): > > static void doSomethingWithEither(DirectBuffer db1, DirectBuffer db2) { > long addr = ((some condition) ? db1 : db2).address(); > ... do something with addr ... > } > > Again, this should be covered, aside from the issues discussed above. If this were an access to an @ReachabilitySensitive field, a reachaibilityFence for the appropriate dbX is required at the end. A real compiler would just generate one each for both. > But I can imagine that teaching users to not do such foolish things would > be easy. The rules are simple: > - always dereference reachablity-sensitive resources directly through > local reference variables. > - make sure that the local reference variable that you extracted > reachablity-sensitive resource from, is in scope while you perform > operations on the resource. > > Regards, Peter > > > From martinrb at google.com Fri Dec 8 01:03:50 2017 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Dec 2017 17:03:50 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: (I'm still trying to love this new API) The changes to the jsr166 tck are fine. I'm not convinced that the new implementation for ArrayList is progress. The current implementation of toArray(T[]) does // Make a new array of a's runtime type, but my contents: return (T[]) Arrays.copyOf(elementData, size, a.getClass()); which does not have to pay the cost of zeroing the array, so is potentially faster. (depends on whether the VM provides cheap pre-zeroed memory?! what does jmh say?) If we're not getting type safety and not getting significantly better performance, all we have is a more natural API. But toArray(IntFunction) also seems not very natural to me, and we'll have to live with the toArray(new String[0]) wart forever anyways. (But is it really so bad?) (Maybe toArray(Class) is actually more natural?) On Thu, Dec 7, 2017 at 2:58 PM, Stuart Marks wrote: > [Martin: please review changes to the JSR 166 TCK.] > > Another updated webrev for this changeset: > > http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.2/ > > This includes an override of toArray(IntFunction) in the implementation of > Arrays.asList(), as requested by Tagir Valeev. > > Overrides are also added for various wrapper classes in j.u.Collections. > Turns out there's a test test/jdk/java/util/Collections/Wrappers.java > that checks to ensure that the wrappers don't inherit any default methods. > (This has been a source of bugs in the past.) > > Significantly, this webrev also includes changes to several tests in the > JSR 166 TCK. The issue is that these tests have cases for null handling, > where they call > > coll.toArray(null) > > to ensure that NPE is thrown. Given that this method is now overloaded: > > toArray(T[]) > toArray(IntFunction) > > passing null is now ambiguous and thus generates a compiler error. I've > modified the tests to call toArray((Object[])null) which is a bit of a > stopgap. I can't think of anything obviously better to do, though. > > s'marks > From brian.burkhalter at oracle.com Fri Dec 8 01:47:30 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 7 Dec 2017 17:47:30 -0800 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Message-ID: On Dec 7, 2017, at 4:33 PM, Stuart Marks wrote: > Please review this changeset that introduces a new no-arg method orElseThrow() to Optional, as a preferred alternative to the get() method. Looks OK to me. > Corresponding methods are also added to OptionalDouble, Int, and Long. > > The orElseThrow() method is functionally identical to get(). It has some affinity with the existing orElseThrow(exceptionSupplier) method, being equivalent to orElseThrow(NoSuchElementException::new). > > The get() method is functionally unchanged. It is NOT being deprecated, although some wording in the doc has been added to point to orElseThrow() as the "preferred alternative." This is part of a (slow) migration away from Optional.get(), which has an obvious, attractive name that is often misused. These issues have been discussed on this list previously: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-April/040484.html Quite a discussion. Good thing you opted not to name it ?Optional.dudeCallIsPresentBeforeCallingMe().? Brian From david.lloyd at redhat.com Fri Dec 8 03:09:13 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Thu, 7 Dec 2017 21:09:13 -0600 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: Yes! It would be even better for the "toArray(Class)" case if Class had a "getArrayClass()" method which returned the Class, which would allow: return (T[]) Arrays.copyOf(elementData, size, componentType.getArrayClass()); and similar for other array-backed collections. I never understood why that method doesn't exist; in fact, am I wrong in understanding that "Array.newInstance(clazz, 0).getClass()" is effectively the only way to do this today? On Thu, Dec 7, 2017 at 7:03 PM, Martin Buchholz wrote: > (I'm still trying to love this new API) > > The changes to the jsr166 tck are fine. > > I'm not convinced that the new implementation for ArrayList is progress. > The current implementation of toArray(T[]) does > > // Make a new array of a's runtime type, but my contents: > return (T[]) Arrays.copyOf(elementData, size, a.getClass()); > > which does not have to pay the cost of zeroing the array, so is potentially > faster. (depends on whether the VM provides cheap pre-zeroed memory?! what > does jmh say?) > > If we're not getting type safety and not getting significantly better > performance, all we have is a more natural API. But toArray(IntFunction) > also seems not very natural to me, and we'll have to live with the > toArray(new String[0]) wart forever anyways. (But is it really so bad?) > (Maybe toArray(Class) is actually more natural?) > > On Thu, Dec 7, 2017 at 2:58 PM, Stuart Marks > wrote: > >> [Martin: please review changes to the JSR 166 TCK.] >> >> Another updated webrev for this changeset: >> >> http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.2/ >> >> This includes an override of toArray(IntFunction) in the implementation of >> Arrays.asList(), as requested by Tagir Valeev. >> >> Overrides are also added for various wrapper classes in j.u.Collections. >> Turns out there's a test test/jdk/java/util/Collections/Wrappers.java >> that checks to ensure that the wrappers don't inherit any default methods. >> (This has been a source of bugs in the past.) >> >> Significantly, this webrev also includes changes to several tests in the >> JSR 166 TCK. The issue is that these tests have cases for null handling, >> where they call >> >> coll.toArray(null) >> >> to ensure that NPE is thrown. Given that this method is now overloaded: >> >> toArray(T[]) >> toArray(IntFunction) >> >> passing null is now ambiguous and thus generates a compiler error. I've >> modified the tests to call toArray((Object[])null) which is a bit of a >> stopgap. I can't think of anything obviously better to do, though. >> >> s'marks >> -- - DML From martinrb at google.com Fri Dec 8 03:25:59 2017 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Dec 2017 19:25:59 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <3a77f8c0-0c8b-3735-a0a6-f39e2d0f5eb9@oracle.com> References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> <3a77f8c0-0c8b-3735-a0a6-f39e2d0f5eb9@oracle.com> Message-ID: On Thu, Dec 7, 2017 at 4:47 PM, Stuart Marks wrote: > > > On 12/7/17 3:50 PM, Jonathan Bluett-Duncan wrote: > > Looking over http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.2/ >> src/java.base/share/classes/java/util/ArrayList.java.cdiff.html, I'm >> wondering if the method `ArrayList.toArray(IntFunction)` should have an >> `@Override` to make it clear that it's overriding the default method in >> Collection. What do you think? >> > > I guess it wouldn't hurt. I had thought about adding it, but most of the > methods in ArrayList that override implementations from supertypes *don't* > have @Override. Looking more closely, it appears that the ones that > override interface default methods *do* have @Override. > > I don't think there's a rule that says that @Override should be used when > overriding interface default methods but not when overriding > implementations from a superclass or when implementing an abstract > interface method. Frankly, I think the JDK is just sloppy here. Most of the > existing implementations didn't use @Override -- possibly because they were > introduced before annotations existed -- and later on, whoever implemented > the overrides of the Java 8 default methods decided to add @Override. > Part of it is that Doug and I are not huge fans of @Override. Is the gain in safety really worth the added verbosity? If we do decide to mandate the use of @Override in all of openjdk, then the errorprone project has a bug pattern check for you, which will provide more safety yet. From david.lloyd at redhat.com Fri Dec 8 04:19:41 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Thu, 7 Dec 2017 22:19:41 -0600 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? Message-ID: I was doing some research related to AccessController, and I noted this code [1] in StackFrameInfo#getByteCodeIndex(): @Override public int getByteCodeIndex() { // bci not available for native methods if (isNativeMethod()) return -1; return bci; } Now bci is of type short, and given the spec of the method, should the return not be: return bci & 0xffff; instead? Else it would return wrong values for methods with more than 32767 bytecodes in them. [1] http://hg.openjdk.java.net/jdk/jdk/file/e3b6cb90d7ce/src/java.base/share/classes/java/lang/StackFrameInfo.java#l96 -- - DML From andrej.golovnin at gmail.com Fri Dec 8 06:54:25 2017 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Fri, 8 Dec 2017 07:54:25 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: Hi Claes, > http://cr.openjdk.java.net/~redestad/8193128/open.02/ I think you should provide specialised implementations of the #indexOf() and #lastIndexOf() methods in the classes List12 and ListN. Using an iterator in List12 is an overkill. But even in ListN using a simple for loop would be much better. In any case please take look at the implementation of the #lastIndexOf() method in the AbstractImmutableList class. It looks like a copy of AbstractImmutableList#indexOf() and this is wrong. Best regards, Andrej Golovnin From Alan.Bateman at oracle.com Fri Dec 8 09:10:15 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 8 Dec 2017 09:10:15 +0000 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Message-ID: On 08/12/2017 00:33, Stuart Marks wrote: > Hi all, > > Please review this changeset that introduces a new no-arg method > orElseThrow() to Optional, as a preferred alternative to the get() > method. > This looks good. The naming is consistent and I think better than the "getWhenPresent" proposed in the original thread. -Alan From Alan.Bateman at oracle.com Fri Dec 8 09:28:01 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 8 Dec 2017 09:28:01 +0000 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: <0a36c975-8bca-c618-9133-66348f145436@oracle.com> References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> Message-ID: <6cadd346-f826-55ce-0276-cfd2f7bef7f4@oracle.com> On 07/12/2017 16:55, Daniel D. Daugherty wrote: > : >> Greetings, >> >> I have a small fix for a very intermittent ServerSocket related test >> failure: >> >> ??? JDK-8182307: Error during JRMP connection establishment >> ??? https://bugs.openjdk.java.net/browse/JDK-8182307 >> >> : >> >> For the gory details of the reasons for this fix please see >> Jerry's bugs. >> >> Webrev URL: http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >> It's not clear to me how this change solves the issue.? It's a "read timeout" so this means the connection has been established. The client will not care if the server has enabled SO_REUSEADDR or whether it initially bound to a fixed or ephemeral port. Is this issue Solaris only? I ask because there is an awkward issue on Solaris where the kernel will accept a pending connection when the process is at its file descriptor limit. We've seen this periodically, esp. with tests that leave connections or files open. An unsuspecting tests runs later, establishes a connection but gets timeouts as there isn't no code at the application level has accepted the connection. -Alan From forax at univ-mlv.fr Fri Dec 8 09:35:27 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 8 Dec 2017 10:35:27 +0100 (CET) Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Message-ID: <967605459.2165732.1512725727144.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Alan Bateman" > ?: "Stuart Marks" , "core-libs-dev" > Envoy?: Vendredi 8 D?cembre 2017 10:10:15 > Objet: Re: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() > On 08/12/2017 00:33, Stuart Marks wrote: >> Hi all, >> >> Please review this changeset that introduces a new no-arg method >> orElseThrow() to Optional, as a preferred alternative to the get() >> method. >> > This looks good. The naming is consistent and I think better than the > "getWhenPresent" proposed in the original thread. i agree with Alan. Having a name that starts with "get" as the great advantage as to be visible in the IDE completion box just after the method get() so it will help people to transition from get() to getWhenPresent() BTW, i do not see the point to not deprecate get() at the same time. > > -Alan R?mi From forax at univ-mlv.fr Fri Dec 8 09:45:35 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 8 Dec 2017 10:45:35 +0100 (CET) Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Message-ID: <1088674183.2171176.1512726335648.JavaMail.zimbra@u-pem.fr> Let's gently derail this thread, the is another pain point with the current optional API, it seems that are no simple way to transform an Optional to an OptionalInt and vice-versa. It's painful because we start to see interface that returns by example OptionalInt while the implementation you want to connect with return an Optional. The only workaround seems to be to use a Stream/IntStream: Optional -> OptionalInt optional.stream().mapToInt(x -> x).findFirst() OptionalInt -> Optional optionalInt.stream().boxed().findFirst(); I think, Optional should have the method mapTo*/flatMapTo* and Optional[Primitive] the method boxed(). regards, R?mi ----- Mail original ----- > De: "Stuart Marks" > ?: "core-libs-dev" > Envoy?: Vendredi 8 D?cembre 2017 01:33:41 > Objet: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() > Hi all, > > Please review this changeset that introduces a new no-arg method orElseThrow() > to Optional, as a preferred alternative to the get() method. > > Corresponding methods are also added to OptionalDouble, Int, and Long. > > The orElseThrow() method is functionally identical to get(). It has some > affinity with the existing orElseThrow(exceptionSupplier) method, being > equivalent to orElseThrow(NoSuchElementException::new). > > The get() method is functionally unchanged. It is NOT being deprecated, although > some wording in the doc has been added to point to orElseThrow() as the > "preferred alternative." This is part of a (slow) migration away from > Optional.get(), which has an obvious, attractive name that is often misused. > These issues have been discussed on this list previously: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-April/040484.html > > Please note that much of that discussion was about the then-proposed deprecation > of Optional.get(), which is NOT part of this proposal. There are no plans to > deprecate Optional.get() at this time. This proposal ONLY introduces a new > method orElseThrow() that is identical in function to get(). > > Bug report: > > https://bugs.openjdk.java.net/browse/JDK-8140281 > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8140281/webrev.1/ > > Thanks, > > s'marks From daniel.fuchs at oracle.com Fri Dec 8 10:29:48 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 8 Dec 2017 10:29:48 +0000 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works In-Reply-To: References: Message-ID: <5256803f-83a9-2fc0-591a-f50baab212dc@oracle.com> On 07/12/2017 19:15, Martin Buchholz wrote: > I'm not a logging expert, but this change Looks Good To Me. Thanks Martin! > > configure => configured > + // For backward compatibility: add any handlers configure using > Fixed. Thanks for spotting that! > --- > > A bunch of "handler" should be changed to "handlers" Oh :-( I am ashamed. I should have proof read it. Fixed. > + ? ? ? ?// Verify that exactly one of the two handler is a custom.Handler > + ? ? ? ?// Verify that exactly one of the two handler is a > custom.DotHandler > + ? ? ? ?// Verify that the two handler have an id of '1' > > -- > > This code makes me think you use the identity of the Longs, but it seems > you don't, so I would just get rid of these and use e.g. 3L instead of > THREE. Done. > > + ? ?public static final Long ONE = 1L; > + ? ?public static final Long TWO = 2L; > + ? ?public static final Long THREE = 3L; > I will push shortly - but for the record I have copied the final webrev here: http://cr.openjdk.java.net/~dfuchs/webrev_8191033/webrev.01 best regards, -- daniel From daniel.fuchs at oracle.com Fri Dec 8 10:30:27 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 8 Dec 2017 10:30:27 +0000 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works In-Reply-To: <1825e86c-ad8f-c4c5-9fc2-fbe8892afeda@oracle.com> References: <1825e86c-ad8f-c4c5-9fc2-fbe8892afeda@oracle.com> Message-ID: Thanks Mandy! -- daniel On 07/12/2017 23:59, mandy chung wrote: > > > On 12/7/17 6:38 AM, Daniel Fuchs wrote: >> Hi, >> >> Please find below a patch for: >> >> 8191033 Regression in logging.properties: specifying .handlers= for >> ??????? root logger (instead of handlers=) no longer works >> https://bugs.openjdk.java.net/browse/JDK-8191033 >> >> webrev: >> http://cr.openjdk.java.net/~dfuchs/webrev_8191033/webrev.00/ > > Looks good to me. > > Mandy From peter.levart at gmail.com Fri Dec 8 12:04:48 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 8 Dec 2017 13:04:48 +0100 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: On 12/08/2017 04:09 AM, David Lloyd wrote: > Yes! It would be even better for the "toArray(Class)" case if > Class had a "getArrayClass()" method which returned the Class, > which would allow: > > return (T[]) Arrays.copyOf(elementData, size, componentType.getArrayClass()); > > and similar for other array-backed collections. I never understood > why that method doesn't exist; in fact, am I wrong in understanding > that "Array.newInstance(clazz, 0).getClass()" is effectively the only > way to do this today? I think there is a reason for Arrays.copyOf() to take an array type, not the component type. If it was taking component type, you could pass in primitive types too with no compile-time type checking, since int.class is of Class static type. So why not the following: public interface Collection extends Iterable { ??? @SuppressWarnings("unchecked") ??? default T[] toArray(Class arrayType) { ??????? return toArray((T[]) Array.newInstance(arrayType.getComponentType(), size())); ??? } ArrayList override could then simply be this: ??? @Override ??? public T[] toArray(Class arrayType) { ??????? return Arrays.copyOf(elementData, size, arrayType); ??? } Regards, Peter > > On Thu, Dec 7, 2017 at 7:03 PM, Martin Buchholz wrote: >> (I'm still trying to love this new API) >> >> The changes to the jsr166 tck are fine. >> >> I'm not convinced that the new implementation for ArrayList is progress. >> The current implementation of toArray(T[]) does >> >> // Make a new array of a's runtime type, but my contents: >> return (T[]) Arrays.copyOf(elementData, size, a.getClass()); >> >> which does not have to pay the cost of zeroing the array, so is potentially >> faster. (depends on whether the VM provides cheap pre-zeroed memory?! what >> does jmh say?) >> >> If we're not getting type safety and not getting significantly better >> performance, all we have is a more natural API. But toArray(IntFunction) >> also seems not very natural to me, and we'll have to live with the >> toArray(new String[0]) wart forever anyways. (But is it really so bad?) >> (Maybe toArray(Class) is actually more natural?) >> >> On Thu, Dec 7, 2017 at 2:58 PM, Stuart Marks >> wrote: >> >>> [Martin: please review changes to the JSR 166 TCK.] >>> >>> Another updated webrev for this changeset: >>> >>> http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.2/ >>> >>> This includes an override of toArray(IntFunction) in the implementation of >>> Arrays.asList(), as requested by Tagir Valeev. >>> >>> Overrides are also added for various wrapper classes in j.u.Collections. >>> Turns out there's a test test/jdk/java/util/Collections/Wrappers.java >>> that checks to ensure that the wrappers don't inherit any default methods. >>> (This has been a source of bugs in the past.) >>> >>> Significantly, this webrev also includes changes to several tests in the >>> JSR 166 TCK. The issue is that these tests have cases for null handling, >>> where they call >>> >>> coll.toArray(null) >>> >>> to ensure that NPE is thrown. Given that this method is now overloaded: >>> >>> toArray(T[]) >>> toArray(IntFunction) >>> >>> passing null is now ambiguous and thus generates a compiler error. I've >>> modified the tests to call toArray((Object[])null) which is a bit of a >>> stopgap. I can't think of anything obviously better to do, though. >>> >>> s'marks >>> > > From Alan.Bateman at oracle.com Fri Dec 8 12:39:19 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 8 Dec 2017 12:39:19 +0000 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> Message-ID: On 07/12/2017 22:55, Brian Burkhalter wrote: > Hi Roger, > > I agree. It does not seem that whatever performance improvement might accrue from not using the Objects methods is offset by the increased code readability in addition to mitigating the risks mentioned in [1]. I have reinstated this approach in http://cr.openjdk.java.net/~bpb/4358774/webrev.02/. > I read through the javadoc again and I think it looks good. On the implementation then Sergey has a point, the requireNonNull isn't needed to check b when b.length is used on the next line. One other nit is that the "overridden for efficiency" comment between @Override and the method declaration is a distraction. I don't think the comment is needed or just put it on the same line or above the @Overview so it doesn't get in the way. The NullXXX tests can be converted to TestNG unit tests if you have cycles. -Alan From daniel.daugherty at oracle.com Fri Dec 8 14:36:59 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 8 Dec 2017 09:36:59 -0500 Subject: RFR(XXS): 8182307 - Error during JRMP connection establishment In-Reply-To: <6cadd346-f826-55ce-0276-cfd2f7bef7f4@oracle.com> References: <58075a8f-1732-7b9e-74b1-3e36189f87fd@oracle.com> <0a36c975-8bca-c618-9133-66348f145436@oracle.com> <6cadd346-f826-55ce-0276-cfd2f7bef7f4@oracle.com> Message-ID: <56b505b7-4294-69e0-b501-708ea2c7a81d@oracle.com> On 12/8/17 4:28 AM, Alan Bateman wrote: > On 07/12/2017 16:55, Daniel D. Daugherty wrote: >> : >>> Greetings, >>> >>> I have a small fix for a very intermittent ServerSocket related test >>> failure: >>> >>> ??? JDK-8182307: Error during JRMP connection establishment >>> ??? https://bugs.openjdk.java.net/browse/JDK-8182307 >>> >>> : >>> >>> For the gory details of the reasons for this fix please see >>> Jerry's bugs. >>> >>> Webrev URL: http://cr.openjdk.java.net/~dcubed/8182307-webrev/jdk10-0/ >>> > It's not clear to me how this change solves the issue.? It's a "read > timeout" so this means the connection has been established. Yes, the connection has been established, but it has been established to the wrong ServerSocket. The ServerSocket port that was picked by the test with its "return new ServerSocket(port)" call was also picked by another "interloper" process. It's the SO_REUSEADDR attribute that allows these two processes to both think that they have the same random port. We have only observed proven sightings of this bug on Solaris SPARC and Solaris X64 machines. So the interloper and the server side of the test both did accept() calls on the same port. The interloper won the race in this case so it is matched up with the test's client side connect(). The test's client side starts doing its protocol reads, but the interloper does not send what the test's client side expects so the test's client side times out in read(). Here's Jerry's eval note from https://bugs.openjdk.java.net/browse/JDK-8182757: > gthornbr Gerald Thornbrugh > > added a comment - 2017-07-27 11:33 > If a socket is being setup without a fixed port using the SO_REUSEADDR > flag can lead to other processes interfering with the poll/receive > process of a debugger/debuggee configuring a socket for communication. > When SO_REUSEADDR is used other processes can attempt a listen() on > the same port and receive a connect from the debuggee. This causes the > debugger to stay in poll() waiting for a connect and the debuggee > stays in recv() waiting to receive data from the "rogue" process that > will never send it. > > This can also lead to connections being terminated early on the > debuggee side when the "rogue" process terminates the connection > because it does not receive what it expected from the client process > (i.e. the debuggee). > > The fix is to not use the SO_REUSEADDR flag for non-fixed port > sockets. This keeps "rogue" processes from reusing the port address > and from stealing the connects sent by from the debuggee. In the hunt for JDK-8182757 we were fortunate that the tests were configured for the server side accept() call to _not_ timeout. That allowed us to capture stacks from both the debuggee and debugger sides. We were also able to capture debug info from different points in the protocol stack in various repro attemps. The only thing we didn't do was add debugging info in the kernel to try and chase the race enabled by SO_REUSEADDR to ground. This bug's (JDK-8182307) failure mode is more like the other failure that Jerry fixed: https://bugs.openjdk.java.net/browse/JDK-8178676 The server side accept() is configured to timeout so we don't have a stack from the server side hang point to prove that the JDK-8178676 failure is the same as the JDK-8182757. With the fixes for JDK-8182757 and JDK-8178676 in place, we have not seen these failure modes reproduce. The fix for JDK-8182757 was pushed on 2017-08-03 and the fix for JDK-8178676 was pushed on 2017-08-14. It is not proof, but it is a strong indicator that these instances of this failure mode are fixed. > The client will not care if the server has enabled SO_REUSEADDR or > whether it initially bound to a fixed or ephemeral port. True, but the client has been connected to the interloper process which is why the read() times out. It is the SO_REUSEADDR attribute that allows the interloper to accept() the test's server side port and that does break the client side of the test. > Is this issue Solaris only? I ask because there is an awkward issue on > Solaris where the kernel will accept a pending connection when the > process is at its file descriptor limit. We've seen this periodically, > esp. with tests that leave connections or files open. An unsuspecting > tests runs later, establishes a connection but gets timeouts as there > isn't no code at the application level has accepted the connection. We have only seen provable sightings of this failure mode on Solaris SPARC and Solaris X64 machines. Folks have added sightings on other platforms to the older bug that was tracking the original issue: ??? JDK-6303969 JDWP: Socket Transport handshake fails rarely on InstancesTest.java ??? https://bugs.openjdk.java.net/browse/JDK-6303969 but Jerry and I were never able to prove a sighting on anything other than Solaris. This "file descriptor limit" issue is new to me. Do you have a pointer to it? It's entirely possible that there is more than one bug at play here... Dan > > -Alan > > From christoph.langer at sap.com Fri Dec 8 14:39:31 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 8 Dec 2017 14:39:31 +0000 Subject: [RFR] 8160768: Add capability to custom resolve host/domain names within the default JDNI LDAP provider In-Reply-To: <20171206182434.GB4409@vimes> References: <20171205174824.GA3515@vimes> <20171206182434.GB4409@vimes> Message-ID: <20b1bc146f8a41838efe27c4412dc2bd@sap.com> Hi Rob, a little style nit: Is it really a good idea to use import java.util.*; in src/java.naming/share/classes/com/sun/jndi/ldap/LdapCtxFactory.java? I thought one is supposed to only use qualified exports nowadays (with all the IDE support). Best regards Christoph -----Original Message----- From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Rob McKenna Sent: Mittwoch, 6. Dezember 2017 19:25 To: vyom tewari Cc: Osipov, Michael ; core-libs-dev at openjdk.java.net Subject: Re: [RFR] 8160768: Add capability to custom resolve host/domain names within the default JDNI LDAP provider Thanks Vyom, these should be fixed in: http://cr.openjdk.java.net/~robm/8160768/webrev.07/ Comments inline: On 06/12/17 22:14, vyom tewari wrote: > Hi Rob, > > Please find below comments. > > DefaultLdapDnsProvider.java > > ?line 35: ??? convert it to for-each code will be more readable. > > LdapDnsProviderService.java > > ?line 21: constant name does not follow Java naming convention, there are > other places as well please fix it. > > getInstance() is already synchronized why you need another "synchronized" > block ? Ah, neglected to remove the outer synchronized keyword, thanks. > > Line 70: Please use result.getEndpoints().isEmpty() in place of > result.getEndpoints().size() == 0 > > LdapDnsProvider.java > > constructor LdapDnsProvider() calls LdapDnsProvider(Void unused) which does > nothing, can you explain why LdapDnsProvider()? calls LdapDnsProvider(Void > unused) ? I've copied this pattern from the System.LoggerFinder api and I had forgotten what it was for too: http://www.oracle.com/technetwork/java/seccodeguide-139067.html#7 Section 7.3. > > LdapDnsProviderResult.java > > ? Private field? domainName & endpoints both can be final. > > LdapDnsProviderTest.java > > Fix the tag Order it is not correct. correct order is as follows. > > /** > ?* @test > ?* @bug 8160768 > ?* @summary ctx provider tests for ldap > ?* @modules java.naming/com.sun.jndi.ldap > ?* @compile dnsprovider/TestDnsProvider.java > ?* @run main/othervm LdapDnsProviderTest > ?* @run main/othervm LdapDnsProviderTest nosm > ?* @run main/othervm LdapDnsProviderTest smnodns > ?* @run main/othervm LdapDnsProviderTest smdns > ?*/ > > Line 82,83 you don't need System.out.println(e); e.printStackTrace(); I'm going to leave these in as I've had a few failing tests in the past where I've needed to push diagnostic changes like this to determine the root cause. -Rob > > Line 70: you don't need this try block > > Line 96 : constant name does not follow Java naming convention > > Line 105: you can use try-with-resources this will avoid some boilerplate. > > Thanks, > Vyom > > On Tuesday 05 December 2017 11:18 PM, Rob McKenna wrote: > >As this enhancement is limited to JDK10 this frees us up to explore a > >different approach. > > > >http://cr.openjdk.java.net/~robm/8160768/webrev.06/ > > > >In this webrev we're using the Service Provider Interface to load an > >implementation of the new LdapDnsProvider class. Implementations of this > >class are responsible for resolving DNS endpoints for a given ldap url > >should the default implementation be insufficient in a particular > >environment. > > > >Note: if a security manager is installed the ldapDnsProvider > >RuntimePermission must be granted. > > > > -Rob > > > From claes.redestad at oracle.com Fri Dec 8 15:13:06 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 8 Dec 2017 16:13:06 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: Hi, On 2017-12-08 07:54, Andrej Golovnin wrote: > Hi Claes, > >> http://cr.openjdk.java.net/~redestad/8193128/open.02/ > I think you should provide specialised implementations of the > #indexOf() and #lastIndexOf() methods in the classes List12 and ListN. > Using an iterator in List12 is an overkill. But even in ListN using a > simple for loop would be much better. it's somewhat ironic that I started looking at this *because* indexOf was slow due use of iterators[1], but then never got around to specialize them in this patch. > In any case please take look at > the implementation of the #lastIndexOf() method in the > AbstractImmutableList class. It looks like a copy of > AbstractImmutableList#indexOf() and this is wrong. Nice catch! Quite the embarrassing copy-paste that... - Specialized the index/lastIndexOf methods for List12, ListN - Fixed implementation of lastIndexOf in AbstractImmutableList. - As AbstractImmutableList.indexOf/lastIndexOf are now only used by AbstractImmutableList.SubList, I moved them there with some commentary since it's not clear JDK-8191418 should add null checkson the input for SubList or not. - Added sanity tests of indexOf/lastIndexOf that touches all the new implementations: http://cr.openjdk.java.net/~redestad/8193128/open.03/ Thanks! /Claes [1] https://bugs.openjdk.java.net/browse/JDK-8191442 From daniel.fuchs at oracle.com Fri Dec 8 15:56:23 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 8 Dec 2017 15:56:23 +0000 Subject: RFR: 8187073: The java.util.logging.Level.findLevel() will not correctly find a Level by it's int value Message-ID: <12c621ce-3591-8311-c0bf-e6fd79a0c441@oracle.com> Hi, Please find below a straightforward fix for: 8187073: The java.util.logging.Level.findLevel() will not correctly find a Level by it's int value https://bugs.openjdk.java.net/browse/JDK-8187073 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8187073/webrev.00/ As expected, the test fails without the fix and passes with it. best regards, -- daniel From Roger.Riggs at Oracle.com Fri Dec 8 16:09:39 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 8 Dec 2017 11:09:39 -0500 Subject: RFR: 8187073: The java.util.logging.Level.findLevel() will not correctly find a Level by it's int value In-Reply-To: <12c621ce-3591-8311-c0bf-e6fd79a0c441@oracle.com> References: <12c621ce-3591-8311-c0bf-e6fd79a0c441@oracle.com> Message-ID: +1 On 12/8/2017 10:56 AM, Daniel Fuchs wrote: > Hi, > > Please find below a straightforward fix for: > > 8187073: The java.util.logging.Level.findLevel() will not correctly > ???????? find a Level by it's int value > https://bugs.openjdk.java.net/browse/JDK-8187073 > > webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8187073/webrev.00/ > > As expected, the test fails without the fix and passes with it. > > best regards, > > -- daniel From brian.burkhalter at oracle.com Fri Dec 8 16:29:03 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Dec 2017 08:29:03 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> Message-ID: <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> On Dec 8, 2017, at 4:39 AM, Alan Bateman wrote: >> http://cr.openjdk.java.net/~bpb/4358774/webrev.02/. >> > I read through the javadoc again and I think it looks good. > > On the implementation then Sergey has a point, the requireNonNull isn't needed to check b when b.length is used on the next line. Already made this change locally. > One other nit is that the "overridden for efficiency" comment between @Override and the method declaration is a distraction. I don't think the comment is needed or just put it on the same line or above the @Overview so it doesn't get in the way. > > The NullXXX tests can be converted to TestNG unit tests if you have cycles. I?ll make the above two changes and re-post later today. Thanks, Brian From jeremymanson at google.com Fri Dec 8 17:37:53 2017 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 8 Dec 2017 09:37:53 -0800 Subject: RFR: 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works In-Reply-To: References: <1825e86c-ad8f-c4c5-9fc2-fbe8892afeda@oracle.com> Message-ID: Thanks for looking at this, Daniel! Jeremy On Fri, Dec 8, 2017 at 2:30 AM, Daniel Fuchs wrote: > Thanks Mandy! > > -- daniel > > > On 07/12/2017 23:59, mandy chung wrote: > >> >> >> On 12/7/17 6:38 AM, Daniel Fuchs wrote: >> >>> Hi, >>> >>> Please find below a patch for: >>> >>> 8191033 Regression in logging.properties: specifying .handlers= for >>> root logger (instead of handlers=) no longer works >>> https://bugs.openjdk.java.net/browse/JDK-8191033 >>> >>> webrev: >>> http://cr.openjdk.java.net/~dfuchs/webrev_8191033/webrev.00/ >>> >> >> Looks good to me. >> >> Mandy >> > > From martinrb at google.com Fri Dec 8 17:44:39 2017 From: martinrb at google.com (Martin Buchholz) Date: Fri, 8 Dec 2017 09:44:39 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: Should there be changes to general purpose collection testing classes like test/jdk/java/util/concurrent/tck/CollectionTest.java test/jdk/java/util/Collection/MOAT.java that would have caught this bug? (although those are mostly designed for mutable collections, but I think some immutable collections (nCopies?) get tested somewhere. On Fri, Dec 8, 2017 at 7:13 AM, Claes Redestad wrote: > Hi, > > On 2017-12-08 07:54, Andrej Golovnin wrote: > >> Hi Claes, >> >> http://cr.openjdk.java.net/~redestad/8193128/open.02/ >>> >> I think you should provide specialised implementations of the >> #indexOf() and #lastIndexOf() methods in the classes List12 and ListN. >> Using an iterator in List12 is an overkill. But even in ListN using a >> simple for loop would be much better. >> > > it's somewhat ironic that I started looking at this *because* > indexOf was slow due use of iterators[1], but then never got > around to specialize them in this patch. > > In any case please take look at >> the implementation of the #lastIndexOf() method in the >> AbstractImmutableList class. It looks like a copy of >> AbstractImmutableList#indexOf() and this is wrong. >> > > Nice catch! Quite the embarrassing copy-paste that... > > - Specialized the index/lastIndexOf methods for List12, ListN > - Fixed implementation of lastIndexOf in AbstractImmutableList. > - As AbstractImmutableList.indexOf/lastIndexOf are now only used > by AbstractImmutableList.SubList, I moved them there with some > commentary since it's not clear JDK-8191418 should add null > checkson the input for SubList or not. > - Added sanity tests of indexOf/lastIndexOf that touches all > the new implementations: > > http://cr.openjdk.java.net/~redestad/8193128/open.03/ > > Thanks! > > /Claes > > [1] https://bugs.openjdk.java.net/browse/JDK-8191442 > From brian.burkhalter at oracle.com Fri Dec 8 18:38:29 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Dec 2017 10:38:29 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> Message-ID: <99FA6355-05A8-4CDB-AE54-40F6A09D68F9@oracle.com> Patrick?s comment made us think again about the naming here as ?nullStream()? would not fit for eventual equivalent methods on Reader and Writer. It might be better to go with something like InputStream InputStream.nullSource(); OutputStream.nullSink(); and later Reader.nullSource(); Writer.nullSink(); Another alternative would be simply to reflect the class names in the methods: InputStream InputStream.nullInputStream(); OutputStream.nullOutputStream(); and later Reader.nullReader(); Writer.nullWriter(); Comments? Thanks, Brian On Dec 6, 2017, at 12:36 PM, Patrick Reinhart wrote: > Is there also a issue for the same kind of methods for Reader and Writer? From david.lloyd at redhat.com Fri Dec 8 18:52:37 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 8 Dec 2017 12:52:37 -0600 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <99FA6355-05A8-4CDB-AE54-40F6A09D68F9@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> <99FA6355-05A8-4CDB-AE54-40F6A09D68F9@oracle.com> Message-ID: On Fri, Dec 8, 2017 at 12:38 PM, Brian Burkhalter wrote: > Patrick?s comment made us think again about the naming here as ?nullStream()? would not fit for eventual equivalent methods on Reader and Writer. It might be better to go with something like > > InputStream InputStream.nullSource(); > OutputStream.nullSink(); > > and later > > Reader.nullSource(); > Writer.nullSink(); > > Another alternative would be simply to reflect the class names in the methods: > > InputStream InputStream.nullInputStream(); > OutputStream.nullOutputStream(); > > and later > > Reader.nullReader(); > Writer.nullWriter(); I for one prefer this alternative; it's very clear and unambiguous. -- - DML From brian.burkhalter at oracle.com Fri Dec 8 19:03:04 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Dec 2017 11:03:04 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream References: <1C790930-2DD0-4451-8104-6A58C6443B88@oracle.com> Message-ID: <79C1C875-D7AA-45F5-8DE4-FFCC089311B5@oracle.com> On Dec 8, 2017, at 10:52 AM, David Lloyd wrote: > On Fri, Dec 8, 2017 at 12:38 PM, Brian Burkhalter > wrote: >> Patrick?s comment made us think again about the naming here as ?nullStream()? would not fit for eventual equivalent methods on Reader and Writer. It might be better to go with something like >> >> InputStream InputStream.nullSource(); >> OutputStream.nullSink(); >> >> and later >> >> Reader.nullSource(); >> Writer.nullSink(); >> >> Another alternative would be simply to reflect the class names in the methods: >> >> InputStream InputStream.nullInputStream(); >> OutputStream.nullOutputStream(); >> >> and later >> >> Reader.nullReader(); >> Writer.nullWriter(); > > I for one prefer this alternative; it's very clear and unambiguous. The second one? Thanks, Brian From jbluettduncan at gmail.com Fri Dec 8 19:07:39 2017 From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan) Date: Fri, 8 Dec 2017 19:07:39 +0000 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <79C1C875-D7AA-45F5-8DE4-FFCC089311B5@oracle.com> References: <1C790930-2DD0-4451-8104-6A58C6443B88@oracle.com> <79C1C875-D7AA-45F5-8DE4-FFCC089311B5@oracle.com> Message-ID: Like David, I prefer the `null{$ClassName}` alternative to `null{Source|Sink}`. (I assume from his wording that he prefers `null{$ClassName}`.) I prefer `null{$ClassName}` not only because it's less ambiguous, but also because Guava has existing types `{Byte|Char}Source` and `{Byte|Char}Sink`, so I'd find it a bit confusing to see `nullSource()` at first (especially if it were statically-imported) as I would initially expect it to have a return type of `*Source`. Cheers, Jonathan On 8 December 2017 at 19:03, Brian Burkhalter wrote: > On Dec 8, 2017, at 10:52 AM, David Lloyd wrote: > > > On Fri, Dec 8, 2017 at 12:38 PM, Brian Burkhalter > > wrote: > >> Patrick?s comment made us think again about the naming here as > ?nullStream()? would not fit for eventual equivalent methods on Reader and > Writer. It might be better to go with something like > >> > >> InputStream InputStream.nullSource(); > >> OutputStream.nullSink(); > >> > >> and later > >> > >> Reader.nullSource(); > >> Writer.nullSink(); > >> > >> Another alternative would be simply to reflect the class names in the > methods: > >> > >> InputStream InputStream.nullInputStream(); > >> OutputStream.nullOutputStream(); > >> > >> and later > >> > >> Reader.nullReader(); > >> Writer.nullWriter(); > > > > I for one prefer this alternative; it's very clear and unambiguous. > The second one? > > Thanks, > > Brian > > From david.lloyd at redhat.com Fri Dec 8 19:18:16 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 8 Dec 2017 13:18:16 -0600 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <79C1C875-D7AA-45F5-8DE4-FFCC089311B5@oracle.com> References: <1C790930-2DD0-4451-8104-6A58C6443B88@oracle.com> <79C1C875-D7AA-45F5-8DE4-FFCC089311B5@oracle.com> Message-ID: On Fri, Dec 8, 2017 at 1:03 PM, Brian Burkhalter wrote: > On Dec 8, 2017, at 10:52 AM, David Lloyd wrote: > >> On Fri, Dec 8, 2017 at 12:38 PM, Brian Burkhalter >> wrote: >>> Patrick?s comment made us think again about the naming here as ?nullStream()? would not fit for eventual equivalent methods on Reader and Writer. It might be better to go with something like >>> >>> InputStream InputStream.nullSource(); >>> OutputStream.nullSink(); >>> >>> and later >>> >>> Reader.nullSource(); >>> Writer.nullSink(); >>> >>> Another alternative would be simply to reflect the class names in the methods: >>> >>> InputStream InputStream.nullInputStream(); >>> OutputStream.nullOutputStream(); >>> >>> and later >>> >>> Reader.nullReader(); >>> Writer.nullWriter(); >> >> I for one prefer this alternative; it's very clear and unambiguous. > The second one? Yes, sorry. -- - DML From mandy.chung at oracle.com Wed Dec 6 18:51:28 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 6 Dec 2017 10:51:28 -0800 Subject: Proposal for New Functionality: Allow module-info merging in GenModuleInfoSource.java In-Reply-To: References: Message-ID: <9155b0eb-5c6b-0339-a6a9-20633accd477@oracle.com> Moving this to jigsaw-dev.... On 12/6/17 8:38 AM, Adam Farley8 wrote: > Hi All, > > Currently, GenModuleInfoSource.java does not allow you to merge extra > module-info files into the primary module-info file (for a given module) > at build time. This tool intends to augment platform-specific exports/opens/uses/provides but not requires.? It was a design choice we made that JDK modules are expected to have the same dependences for all platforms. > Put simply; I think it should have this functionality. Can committers > please review and opine? Can you explain why you want the module dependences be different on different platform?? Is it an option to add the requires src//share/classes/module-info.java ? The build generates the target dependences based on the requires from module-info.java.?? At the moment it does not take module-info.java.extra into account AFAIU. Mandy From brent.christian at oracle.com Fri Dec 8 19:44:26 2017 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 8 Dec 2017 11:44:26 -0800 Subject: RFR 8193271 : ProblemList tools/launcher/TestXcheckJNIWarnings.java Message-ID: <6329b70e-4cb2-6934-f443-30184a267407@oracle.com> Hi, Please review my change to add the intermittently-failing tools/launcher/TestXcheckJNIWarnings.java to the problem list. diff -r a1f88c937a77 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt Tue Nov 28 10:15:47 2017 -0800 +++ b/test/jdk/ProblemList.txt Fri Dec 08 11:42:55 2017 -0800 @@ -261,6 +261,7 @@ tools/pack200/CommandLineTests.java 8059906 generic-all tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all +tools/launcher/TestXcheckJNIWarnings.java 8190984 solaris-all tools/jimage/JImageExtractTest.java 8170120 generic-all Thanks! -Brent From joe.darcy at oracle.com Fri Dec 8 19:45:51 2017 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 8 Dec 2017 11:45:51 -0800 Subject: RFR 8193271 : ProblemList tools/launcher/TestXcheckJNIWarnings.java In-Reply-To: <6329b70e-4cb2-6934-f443-30184a267407@oracle.com> References: <6329b70e-4cb2-6934-f443-30184a267407@oracle.com> Message-ID: <4e03a2f4-916d-d4be-54cd-5e0228dd83db@oracle.com> +1 Cheers, -Joe On 12/8/2017 11:44 AM, Brent Christian wrote: > Hi, > > Please review my change to add the intermittently-failing > tools/launcher/TestXcheckJNIWarnings.java to the problem list. > > diff -r a1f88c937a77 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt Tue Nov 28 10:15:47 2017 -0800 > +++ b/test/jdk/ProblemList.txt Fri Dec 08 11:42:55 2017 -0800 > @@ -261,6 +261,7 @@ > tools/pack200/CommandLineTests.java 8059906 generic-all > > tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all > +tools/launcher/TestXcheckJNIWarnings.java 8190984 solaris-all > > tools/jimage/JImageExtractTest.java 8170120 generic-all > > > Thanks! > -Brent From brent.christian at oracle.com Fri Dec 8 21:06:22 2017 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 8 Dec 2017 13:06:22 -0800 Subject: RFR 8193271 : ProblemList tools/launcher/TestXcheckJNIWarnings.java In-Reply-To: <4e03a2f4-916d-d4be-54cd-5e0228dd83db@oracle.com> References: <6329b70e-4cb2-6934-f443-30184a267407@oracle.com> <4e03a2f4-916d-d4be-54cd-5e0228dd83db@oracle.com> Message-ID: <0f1e798e-4b06-7940-d033-c1a39b28fedb@oracle.com> Thanks, Joe -B On 12/8/17 11:45 AM, joe darcy wrote: > +1 > > Cheers, > > -Joe > > > On 12/8/2017 11:44 AM, Brent Christian wrote: >> Hi, >> >> Please review my change to add the intermittently-failing >> tools/launcher/TestXcheckJNIWarnings.java to the problem list. >> >> diff -r a1f88c937a77 test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt??? Tue Nov 28 10:15:47 2017 -0800 >> +++ b/test/jdk/ProblemList.txt??? Fri Dec 08 11:42:55 2017 -0800 >> @@ -261,6 +261,7 @@ >> ?tools/pack200/CommandLineTests.java 8059906 generic-all >> >> ?tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all >> +tools/launcher/TestXcheckJNIWarnings.java 8190984 solaris-all >> >> ?tools/jimage/JImageExtractTest.java 8170120 generic-all >> >> >> Thanks! >> -Brent > From patrick at reini.net Fri Dec 8 21:18:47 2017 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 8 Dec 2017 22:18:47 +0100 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <99FA6355-05A8-4CDB-AE54-40F6A09D68F9@oracle.com> References: <6FD5C365-CF2E-4FC6-ABE4-1D2359C95E71@oracle.com> <539CE455-7369-4EF6-AC26-732293C35DAC@reini.net> <99FA6355-05A8-4CDB-AE54-40F6A09D68F9@oracle.com> Message-ID: <58736D9A-9EB7-49A6-A842-ABD8D6570525@reini.net> I like the nullInputStream() nullOutputStream() as I would first search for those names, also nullReader() / nullWriter() seem to fit more than Sink/Source or Stream -Patrick > Am 08.12.2017 um 19:38 schrieb Brian Burkhalter : > > Patrick?s comment made us think again about the naming here as ?nullStream()? would not fit for eventual equivalent methods on Reader and Writer. It might be better to go with something like > > InputStream InputStream.nullSource(); > OutputStream.nullSink(); > > and later > > Reader.nullSource(); > Writer.nullSink(); > > Another alternative would be simply to reflect the class names in the methods: > > InputStream InputStream.nullInputStream(); > OutputStream.nullOutputStream(); > > and later > > Reader.nullReader(); > Writer.nullWriter(); > > Comments? > > Thanks, > > Brian > > On Dec 6, 2017, at 12:36 PM, Patrick Reinhart wrote: > >> Is there also a issue for the same kind of methods for Reader and Writer? > From philipp.kunz at paratix.ch Fri Dec 8 22:43:13 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Fri, 8 Dec 2017 23:43:13 +0100 Subject: RFR JDK-6372077: Manifest should handle manifest attribute names up to 70 bytes In-Reply-To: References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> <02abcb68-2294-c733-5048-b1f81ee99435@oracle.com> <88b89995-cfe0-d439-d63d-699c628e2315@paratix.ch> Message-ID: <481d7e57-e3d7-b0ca-f308-d81a450378b6@paratix.ch> Hi everyone, After I haven't found a sponsor so far, I try to make it even easier this time by providing a hopefully ready patch this time. See http://files.paratix.ch/jdk/6372077/webrev.01/ or the attached 6372077.patch. In short, this patch increases the manifest files line width by two bytes in order to provide enough space for up to 70 bytes of a header name and the following two bytes of the ": " delimiter in order to fulfill the manifest specification that does not allow header names and the delimiter to be broken across lines in Manifest#make72Safe(StringBuffer). It also changes that make72Safe is now not anymore invoked with the trailing line break in the StringBuffer argument but appended only after that call which makes make72Safe simpler by one if and one subtraction and affects three places where it is called. While the bug says in its title it relates to JarFile.getManifest() it is actually caused in class Manifest. Therefore I added a test about exactly what the bug says in Bug6372077jarFileManifestAttr70 along with another test LineBreakLineWidth that covers the actual fix. All my other considerations and reasoning are already written down in either the preceding mails included below or as comments in the patch. I'd be glad to get any kind of feedback and am looking forward to closing another bug. Regards, Philipp On 21.11.2017 07:18, Philipp Kunz wrote: > Hi everyone, > > I haven't got any reply now for around three weeks and now i start to > wonder if I just missed it or if I should refine my approach to find a > sponsor or if it helped if I presented a ready patch or if noone > considers this important enough or anything else whatever. This is > only my second contribution hence I don't know the procedure well. > > One point maybe worth to point out again is that I don't want to > support manifest headers longer than 70 character, just up to 70, > which has always been the intention but has only worked up to 68. This > might have been written confusingly in my last email. > > As far as I understood, I should first get a sponsor. In any case, is > there any suggestion for how to proceed? > > Regards, > Philipp > > > > On 03.11.2017 00:04, Philipp Kunz wrote: >> Hi Sean and Max and all others, >> >> Thank you Sean for the hint about the right mailing list. And thanks >> also for his hint to Max to make smaller portions of changes. >> >> I would like to contribute a fix for JDK-6372077 which is about >> JarFile.getManifest() should handle manifest attribute name[s longer >> than] 70 bytes. >> >> It looks like the bug is caused by Manifest.make72Safe breaking lines >> at 70 bytes instead of 72 despite its comment and name >> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Manifest.java#l176).The >> resulting StringBuffer has then lines of 72 bytes maximum each >> including the following line break. Without the line break that >> leaves 70 bytes of characters per line. On the other hand, header >> names can be up to 70 bytes (only single-byte utf-8 characters) and >> cannot be broken across a line break and need to be followed by a >> colon and a space which must be on the same line too according to the >> specification. When breaking at 70 bytes excluding the line break, >> however, long header names don't fit in one line together with the >> colon space delimiter because there is not sufficient space. >> Manifests with names up to 70 bytes long can still be written without >> immediate exception but the resulting manifests are illegal in my >> opinion. When later reading such manifests >> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Attributes.java#l406), >> an error occurs as a consequence of the bad manifest. This is more or >> less the current situation and also what JDK-6372077 already knew. >> >> --> After all, in order to fix it, i'd like to propose to make >> manifest file lines wider by two bytes. >> >> The only proper alternative that came into my mind would be to change >> the manifest specification and reduce the maximum header name length >> there by two and also in the code. If that would break any existing >> code i guess that would affect code only that produced invalid >> manifests and would be acceptable. >> >> Supporting all existing and possibly invalid manifests would mean to >> add support for reading headers the delimiters of which are broken >> onto the next line which I consider too complex with respect to the >> value added and even more so considering that those invalid manifest >> can be assumed to have been detected as such by reading them and also >> because it would be a feature that would be used less and less over >> time if the code to write manifest is changed at the same time to >> produce only valid manifests in the discussed respect here. I don't >> think this should be actually done. >> >> Before I actually do the leg work, i'd like to ask, if there are >> concerns or objections or tips for such a change or if anyone can or >> cannot follow the reasoning and the conclusion to make manifests 2 >> bytes wider or if i missed an important point altogether. >> >> In case there will be a consent about how to solve this, would >> someone volunteer to sponsor? That may be less urgent at the moment >> than the question above about how to proceed. >> >> Philipp >> -------------- next part -------------- A non-text attachment was scrubbed... Name: 6372077.patch Type: text/x-patch Size: 18891 bytes Desc: not available URL: From martinrb at google.com Fri Dec 8 23:04:49 2017 From: martinrb at google.com (Martin Buchholz) Date: Fri, 8 Dec 2017 15:04:49 -0800 Subject: RFR: JDK-8192935 Fix EnumSet's SerializationProxy javadoc In-Reply-To: <30f47f0d-2c0c-3d46-1cdf-bcf0238da1a3@oracle.com> References: <1f0f416e-aa86-06d7-50e3-87d82bad6049@oracle.com> <6181b5cb-31ef-8d69-d6fe-0aa5150f700d@Oracle.com> <2e1ff97f-ce42-50f3-4f11-105af0159a6e@oracle.com> <30f47f0d-2c0c-3d46-1cdf-bcf0238da1a3@oracle.com> Message-ID: Pushed. In future I will use the phrase "Best of a Bad Lot Practice". On Mon, Dec 4, 2017 at 4:20 PM, Stuart Marks wrote: > If you don't like my alternative, fine; it has its own set of tradeoffs >>> that might be net positive or negative. If you want to proceed with >>> the >>> current approach, then I won't stand in the way. At the very least >>> there >>> should be some boilerplate added to EnumSet that makes it clear that >>> EnumSet itself never appears in the serial form. >>> >> I don't disagree, there are many things that could be improved. >> >> I only volunteered to bring EnumSet (as the poster child for the >> Serialization Proxy Pattern) into a no-worse state than other classes >> implementing the pattern. The doc of the writeReplace and readObject >> methods is pretty good implicit documentation that the pattern applies >> here. Serialization overall remains as deeply flawed as ever. >> >> I still plan to submit what I have now. >> > > Thanks for volunteering. It goes to show that no good deed goes > unpunished. :-) > > To close the loop on this, I think what you have is acceptable. I also > think that "no-worse state" is a better characterization than "Best > Practice," which seems to imply that no further improvement is possible or > necessary. And finally, Jon Gibbons has filed JDK-8193019 to cover future > javadoc enhancements to better support serialization. > > s'marks > From xueming.shen at oracle.com Fri Dec 8 23:09:31 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 08 Dec 2017 15:09:31 -0800 Subject: RFR: JDK-8184947:,ZipCoder performance improvements Message-ID: <5A2B1BAB.7010102@oracle.com> Hi, Please help review the changes for j.u.z.ZipCoder/JDK-8184947 (which also includes cleanup/improvement work in java.lang.StringCoding.java to speed up general String coding performance, especially for UTF8). issue: https://bugs.openjdk.java.net/browse/JDK-8184947 webrev: http://cr.openjdk.java.net/~sherman/8184947/webrev jmh benchmark: http://cr.openjdk.java.net/~sherman/8184947/ZipCodingBM.java http://cr.openjdk.java.net/~sherman/8184947/StringCodingBM.java Notes: (1) StringCoding.de/encode() for new String()/String.getBytes() with default charset. For historical reason the existing SC.decode(byte[], off, len)/encode(coder, val) implementation has code to handle any "possible" UnsupportedEncodingExcetion situation and turn to the slow "charset name" version of de/encode() for real work. Given the fact that the Charset.defaultCharset() now returns UTF8 as the fallback default charset if there is anything wrong to obtain a default charset (we did that in jdk7 or 8?), there is no need actually to handle the UEE. This also provides the opportunity to use fastpath for stateless UTF8/88591/ASCII de/encode(). The benchmark data for newString_xxx/ getBytes_xxx (which uses the default encoding, UTF8 in this case) suggests a big speed up fo ascii-only String. StringCodingBM size) Mode Cnt NEW Score Error OLD Score Error Units getBytes_ASCII 16 avgt 5 21.155 ?? 5.586 63.777 ?? 54.262 ns/op getBytes_ASCII 64 avgt 5 20.854 ?? 6.237 98.988 ?? 62.932 ns/op getBytes_ASCII 256 avgt 5 38.291 ?? 8.494 272.306 ?? 77.951 ns/op getBytes_Latin 16 avgt 5 80.968 ?? 15.814 76.769 ?? 38.512 ns/op getBytes_Latin 64 avgt 5 163.078 ?? 51.993 219.085 ?? 42.665 ns/op getBytes_Latin 256 avgt 5 759.548 ?? 99.386 824.594 ?? 763.735 ns/op getBytes_Unicode 16 avgt 5 94.311 ?? 22.189 124.185 ?? 32.751 ns/op getBytes_Unicode 64 avgt 5 289.603 ?? 152.056 321.541 ?? 103.703 ns/op getBytes_Unicode 256 avgt 5 1253.098 ?? 216.243 1201.667 ?? 512.532 ns/op newString_ASCII 16 avgt 5 33.273 ?? 13.780 50.402 ?? 17.574 ns/op newString_ASCII 64 avgt 5 30.420 ?? 6.207 84.989 ?? 43.355 ns/op newString_ASCII 256 avgt 5 54.391 ?? 10.451 208.096 ?? 102.716 ns/op newString_Latin 16 avgt 5 115.606 ?? 7.181 114.186 ?? 36.310 ns/op newString_Latin 64 avgt 5 393..710 ?? 73.478 414.286 ?? 176.837 ns/op newString_Latin 256 avgt 5 1618.967 ?? 289.044 1551.499 ?? 487.904 ns/op newString_Unicode 16 avgt 5 104.848 ?? 32.694 127.558 ?? 12.029 ns/op newString_Unicode 64 avgt 5 377.894 ?? 147.731 374.779 ?? 53.028 ns/op newString_Unicode 256 avgt 5 1557.977 ?? 318.652 1457.236 ?? 284.424 ns/op (2) updated to "fast path" UTF8/8859-1/ASCII in all de/coding operation, which are all implemented in static /stateless methods. (benchmark for MS932 [4] provide to make sure no regression for "other" charsets) (3) added "fast path" for "ascii-only' bytes for utf8 encoding/getBytes(). The benchmark [1] suggests a big speedup for ascii-only getBytes() with limited cost to non-ascii-only cases. (this helps big for (4), the ZipCoder situation, which mainly uses ascii only). (4) java.util.zip.ZipCoder This is where this patch actually started from. As the rfe suggested we are now using byte[] as the internal storage for the String class, the optimization we put in ZipCoder for UTF8 (which uses the byte[]/char[] interface of out UTF8 implementation to help avoid the relatively heavy ByteBuffer/CharBuffer coding interface) now appears to be not that "optimized". The to/from char[] copy/paste has become a waste. ZipCoder implementation can't use new String/String.getBytes() directly because of the the different malformed/unmappable character handing requirement. The proposed change here is to add a pair of special new String()/String.getBytes() in StrngCoding class to throw IAE instead of silent replacement, via (yet another) SharedSecrets interface. This brings us much faster de/encoding (30%-50% speed up) and much less memory usage (no more unnecessary byte[]/char[] allocation and in default mode, there is only ONE utf8 ZipCoder), on all "Jar/ZipEntry" related access operations. ZipCodeBenchMark [latest] * "New Score" is with the patch * getEntry() is mainly String.getBytes(), entries()/stream() is mainly new String(bytes)). Mode Cnt New Score Error Old Score Units jf_entries avgt 20 0.582 ?? 0.036 0.953 ?? 0.108 ms/op jf_getEntry avgt 20 1.506 ?? 0.158 2.052 ?? 0.171 ms/op jf_stream avgt 20 0.698 ?? 0.060 0.940 ?? 0.067 ms/op zf_entries avgt 20 0.691 ?? 0.057 0.917 ?? 0.080 ms/op zf_getEntry avgt 20 1.459 ?? 0.180 2.081 ?? 0.161 ms/op zf_stream avgt 20 0.626 ?? 0.074 0.909 ?? 0.075 ms/op Thanks, Sherman [1] http://cr.openjdk.java.net/~sherman/8184947/StringCoding.utf8 [2]http://cr.openjdk.java.net/~sherman/8184947/StringCoding.8859_1 [3] http://cr.openjdk.java.net/~sherman/8184947/StringCoding.ascii [4]http://cr.openjdk.java.net/~sherman/8184947/StringCoding.ms932 [5] http://cr.openjdk.java.net/~sherman/8184947/ZipCoding.bm From brian.burkhalter at oracle.com Fri Dec 8 23:12:37 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Dec 2017 15:12:37 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: On Dec 8, 2017, at 8:29 AM, Brian Burkhalter wrote: > On Dec 8, 2017, at 4:39 AM, Alan Bateman wrote: > >>> http://cr.openjdk.java.net/~bpb/4358774/webrev.02/. >>> >> I read through the javadoc again and I think it looks good. >> >> On the implementation then Sergey has a point, the requireNonNull isn't needed to check b when b.length is used on the next line. > > Already made this change locally. > >> One other nit is that the "overridden for efficiency" comment between @Override and the method declaration is a distraction. I don't think the comment is needed or just put it on the same line or above the @Overview so it doesn't get in the way. >> >> The NullXXX tests can be converted to TestNG unit tests if you have cycles. > > I?ll make the above two changes and re-post later today. All previous suggested changes have been made in http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ except for the possible change of name for the IS and OS nullStream() methods which awaits consensus. Thanks, Brian From stuart.marks at oracle.com Fri Dec 8 23:48:08 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 8 Dec 2017 15:48:08 -0800 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: <967605459.2165732.1512725727144.JavaMail.zimbra@u-pem.fr> References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> <967605459.2165732.1512725727144.JavaMail.zimbra@u-pem.fr> Message-ID: <04841738-95fc-c2fe-1c30-40dac1d5ca04@oracle.com> >>> Please review this changeset that introduces a new no-arg method >>> orElseThrow() to Optional, as a preferred alternative to the get() >>> method. >>> >> This looks good. The naming is consistent and I think better than the >> "getWhenPresent" proposed in the original thread. > > i agree with Alan. > > Having a name that starts with "get" as the great advantage as to be visible in the IDE completion box just after the method get() so it will help people to transition from get() to getWhenPresent() I'm not sure whether you're agreeing or disagreeing. :-) The current proposal is for orElseThrow(). Having a method that starts with "get" (such as "getWhenPresent" or "getOrThrow") is indeed helpful when doing auto-completion, but it sort-of starts to create a new family of get- methods that overlap with the existing orElse- methods in an odd way. I think the no-arg orElseThrow() fits better with the existing three orElse- methods. This leaves get() as the outlier, which is ok if we maybe eventually deprecate it. > BTW, i do not see the point to not deprecate get() at the same time. Much of the resistance to the earlier proposal was about deprecation of get(), so I wanted to set that aside in order to proceed with introduction of this alternative. s'marks From stuart.marks at oracle.com Sat Dec 9 00:00:06 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 8 Dec 2017 16:00:06 -0800 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: <1088674183.2171176.1512726335648.JavaMail.zimbra@u-pem.fr> References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> <1088674183.2171176.1512726335648.JavaMail.zimbra@u-pem.fr> Message-ID: <83fd6ac6-436a-a788-ceb8-5a1abffd7d08@oracle.com> Please, let's not derail this thread. :-) I think would be a good thing to think about, so I've filed a JIRA issue to track it: https://bugs.openjdk.java.net/browse/JDK-8193279 s'marks On 12/8/17 1:45 AM, Remi Forax wrote: > Let's gently derail this thread, the is another pain point with the current optional API, > it seems that are no simple way to transform an Optional to an OptionalInt and vice-versa. > > It's painful because we start to see interface that returns by example OptionalInt while the implementation you want to connect with return an Optional. > > The only workaround seems to be to use a Stream/IntStream: > Optional -> OptionalInt > optional.stream().mapToInt(x -> x).findFirst() > OptionalInt -> Optional > optionalInt.stream().boxed().findFirst(); > > I think, Optional should have the method mapTo*/flatMapTo* and Optional[Primitive] the method boxed(). > > regards, > R?mi > > ----- Mail original ----- >> De: "Stuart Marks" >> ?: "core-libs-dev" >> Envoy?: Vendredi 8 D?cembre 2017 01:33:41 >> Objet: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() > >> Hi all, >> >> Please review this changeset that introduces a new no-arg method orElseThrow() >> to Optional, as a preferred alternative to the get() method. >> >> Corresponding methods are also added to OptionalDouble, Int, and Long. >> >> The orElseThrow() method is functionally identical to get(). It has some >> affinity with the existing orElseThrow(exceptionSupplier) method, being >> equivalent to orElseThrow(NoSuchElementException::new). >> >> The get() method is functionally unchanged. It is NOT being deprecated, although >> some wording in the doc has been added to point to orElseThrow() as the >> "preferred alternative." This is part of a (slow) migration away from >> Optional.get(), which has an obvious, attractive name that is often misused. >> These issues have been discussed on this list previously: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-April/040484.html >> >> Please note that much of that discussion was about the then-proposed deprecation >> of Optional.get(), which is NOT part of this proposal. There are no plans to >> deprecate Optional.get() at this time. This proposal ONLY introduces a new >> method orElseThrow() that is identical in function to get(). >> >> Bug report: >> >> https://bugs.openjdk.java.net/browse/JDK-8140281 >> >> Webrev: >> >> http://cr.openjdk.java.net/~smarks/reviews/8140281/webrev.1/ >> >> Thanks, >> >> s'marks From huizhe.wang at oracle.com Sat Dec 9 00:29:57 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 08 Dec 2017 16:29:57 -0800 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: <5A2617F2.6030005@oracle.com> References: <5A20F2AB.3050608@oracle.com> <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> <5A21BD18.2050201@oracle.com> <34dacec0-7e2f-65db-1262-30c0c4a279c1@oracle.com> <5A2617F2.6030005@oracle.com> Message-ID: <5A2B2E85.9000105@oracle.com> Thanks Roger, Alan for the comments! I've updated the webrev accordingly. Here's the current webrev: http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html -Joe On 12/4/17, 7:52 PM, Joe Wang wrote: > > > On 12/3/17, 11:34 AM, Alan Bateman wrote: >> >> >> On 01/12/2017 20:35, Joe Wang wrote: >>> : >>> >>>> >>>> I think URLDecoder.decode methods should have @throws >>>> IllegalArgumentException. I realize it's implementation specific as >>>> to whether IAE is thrown with bad input but given that the RI does >>>> throw IAE then consumers of the API should be prepared for it. The >>>> @implNote can stay and probably should be copied into the legacy >>>> decode method too. >>> >>> I added @throws IAE. On a 2nd thought, would that give no >>> flexibility to alternative impls as the general (class) spec had >>> given? With this addition, it becomes a requirement. >> If the @throws description starts with "if the implementation ..." >> then it should be clear that it is optional. > > Thanks Alan. I made the change accordingly. > http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html > > -Joe > >> >> -Alan From Sergey.Bylokhov at oracle.com Sat Dec 9 00:34:11 2017 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Fri, 8 Dec 2017 16:34:11 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: Hi, Brian. On 08/12/2017 15:12, Brian Burkhalter wrote: > All previous suggested changes have been made in > > http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ One more issue that according to the spec the new method read(byte[], int, int) should throw an exception if the stream was closed, but as far as I understand it can return "0" if "len=0" even if the stream was closed before: 99 @Override 100 public int read(byte[] b, int off, int len) 101 Objects.checkFromIndexSize(off, len, b.length); 102 if (len == 0) { 103 return 0; 104 } 105 ensureOpen(); 106 return -1; 107 } -- Best regards, Sergey. From john.r.rose at oracle.com Sat Dec 9 00:45:41 2017 From: john.r.rose at oracle.com (John Rose) Date: Fri, 8 Dec 2017 16:45:41 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> On Dec 7, 2017, at 3:41 PM, Claes Redestad wrote: > > - the ListFactories test didn't explicitly verify that sublists retrieved from various List.of() impls aren't Serializable. Added tests to check no sub-list is Serializable, > and as a bonus this test now verifies that List.of(...).subList(...) behave like their List.of(..) counterparts in most other ways. List::subList is a view into a backing list. But the story is different for the value-based lists produced by List.of. If L is a value-based class, then it seems to me a meaningless (therefore useless) distinction to call L::subList a "view". A slice of a value-based list can only be another value-based list, because there is nothing else to slice, besides the component values of the original list. So I think we should move forward more confidently with subList here, and say that List.of(a,b,c).subList(1,2) is equivalent in all ways to List.of(b,c), and so on. Can anyone point out a reason why the value based lists of List.of() should serialize while the value based lists of List.of().subList() should not? Or is there some reason we should not allow subList to produce value based lists? ? John From brian.burkhalter at oracle.com Sat Dec 9 00:49:36 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Dec 2017 16:49:36 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: Hi Sergey, On Dec 8, 2017, at 4:34 PM, Sergey Bylokhov wrote: > One more issue that according to the spec the new method > read(byte[], int, int) should throw an exception if the stream was closed, but as far as I understand it can return "0" if "len=0" even if the stream was closed before: > 99 @Override > 100 public int read(byte[] b, int off, int len) > 101 Objects.checkFromIndexSize(off, len, b.length); > 102 if (len == 0) { > 103 return 0; > 104 } > 105 ensureOpen(); > 106 return -1; > 107 } I agree it looks strange but it is intentional as it matches the existing InputStream.read(byte[],int,in) [1]. (I will remove line 167 as part of this patch.) Note that the IOE for the stream being closed would not be thrown in the current code until line 173. Thanks, Brian [1] http://hg.openjdk.java.net/jdk/jdk/file/dd5157f363ab/src/java.base/share/classes/java/io/InputStream.java#l166 From brent.christian at oracle.com Sat Dec 9 00:50:31 2017 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 8 Dec 2017 16:50:31 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: Hi, Brian On 12/8/17 3:12 PM, Brian Burkhalter wrote: > > http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ > The changes look good to me. I just noticed a couple small things in the test: test/jdk/java/io/InputStream/NullInputStream.java 109 @Test(groups = "open") 110 public static void testreadNBytes() { 111 try { 112 assertEquals(0, openStream.readNBytes(new byte[1], 0, 1), 113 "readNBytes(byte[],int,int) != -1"); Line 113 should read "... != 0", yes? 129 @Test(groups = "open") 130 public static void testSkip() { 131 try { 132 assertEquals(0, openStream.skip(1), "transferTo() != 0"); On 132, it's "skip() != 0". -Brent From Sergey.Bylokhov at oracle.com Sat Dec 9 01:06:49 2017 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Fri, 8 Dec 2017 17:06:49 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: On 08/12/2017 16:49, Brian Burkhalter wrote: > I agree it looks strange but it is intentional as it matches the > existing InputStream.read(byte[],int,in) [1]. (I will remove line 167 as > part of this patch.) Note that the IOE for the stream being closed would > not be thrown in the current code until line 173. Yes it is match the behavior, but both have a different specifications: In the old methods there is a notion: "

If len is zero, then no bytes are read and 0 is returned;" but in the new method we have only one strong statement: "After the stream has been closed, these methods all throw {@code IOException}." -- Best regards, Sergey. From brian.burkhalter at oracle.com Sat Dec 9 01:03:06 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Dec 2017 17:03:06 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream References: Message-ID: <509A282A-E215-4CA4-B0EE-0184AA794B47@oracle.com> Hi Brent, On Dec 8, 2017, at 4:50 PM, Brent Christian wrote: > I just noticed a couple small things in the test: > > test/jdk/java/io/InputStream/NullInputStream.java > > [?] > > Line 113 should read "... != 0", yes? > > [?] > > On 132, it's "skip() != 0". You are correct on both accounts. I shuffled that test completely around today converting it to TestNG and obviously made some typos. Webrev updated in place http://cr.openjdk.java.net/~bpb/4358774/webrev.03/. Thanks, Brian From brian.burkhalter at oracle.com Sat Dec 9 01:09:12 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 8 Dec 2017 17:09:12 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: On Dec 8, 2017, at 5:06 PM, Sergey Bylokhov wrote: > On 08/12/2017 16:49, Brian Burkhalter wrote: >> I agree it looks strange but it is intentional as it matches the existing InputStream.read(byte[],int,in) [1]. (I will remove line 167 as part of this patch.) Note that the IOE for the stream being closed would not be thrown in the current code until line 173. > > Yes it is match the behavior, but both have a different specifications: > In the old methods there is a notion: "

If len is zero, then no bytes are read and 0 is returned;" but in the new method we have only one strong statement: "After the stream has been closed, these methods all throw {@code IOException}." Indeed you are correct. As we are trying to match the behavior I think the spec will need to be tweaked a little. The problem is that there is not a method-by-method specification in this case. This will need to be reexamined. Thanks, Brian From john.r.rose at oracle.com Sat Dec 9 01:09:24 2017 From: john.r.rose at oracle.com (John Rose) Date: Fri, 8 Dec 2017 17:09:24 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: On Dec 8, 2017, at 7:13 AM, Claes Redestad wrote: > > http://cr.openjdk.java.net/~redestad/8193128/open.03/ > + for (int i = elements.length - 1; i > 0; i--) { There's an OBOE in this line; should be i >= 0. Errors like this are a risk of hand-specializing code. It's probably worth it here, but it's still a risk. For immutable lists, it would be reasonable to recode the generic algorithms to use for-loops with integer indexes and get(int) access. That will remove the unused generality of iterators, and keep only the abstraction across get(). The benefits of customized inlining (to fold up the funny specialized get methods) can probably be obtained like this: class AbsImmList { @ForceInline int indexOf(Object x) { for (int i = 0, s = size(); i < s; i++) if (x.equals(this.get(i))) return i; } } final class List12 { int indexOf(Object x) { return super.indexOf(x); } // specialize size/get } final class ListN { int indexOf(Object x) { return super.indexOf(x); } // specialize size/get } The optimization of the funny 1/2 loop for List12 should unfold into the same code you'd write by hand; if not it's a bug to fix in C2. Likewise for the loop in ListN. ? John P.S. If you are going to hand-specialize, it might be easier on the optimizer if you would add in this line as needed: final E[] elements = this.elements; // cache field Usually C2 gets the same answer, but not always, since final fields are not fully trustable, and "might" (in some hypothetical system which C2 can't rule out) change over a call to Object::equals. OTOH, generic coding as above is preferable, and we can fix C2 to hoist the @Stable elements field if it doesn't already. P.P.S. Why does ListN have arity 1 and arity 2 constructors? From john.r.rose at oracle.com Sat Dec 9 01:23:43 2017 From: john.r.rose at oracle.com (John Rose) Date: Fri, 8 Dec 2017 17:23:43 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: +1 (there should) On Dec 8, 2017, at 9:44 AM, Martin Buchholz wrote: > > Should there be changes to general purpose collection testing classes like > test/jdk/java/util/concurrent/tck/CollectionTest.java > test/jdk/java/util/Collection/MOAT.java > that would have caught this bug? > (although those are mostly designed for mutable collections, but I think > some immutable collections (nCopies?) get tested somewhere. From john.r.rose at oracle.com Sat Dec 9 01:20:11 2017 From: john.r.rose at oracle.com (John Rose) Date: Fri, 8 Dec 2017 17:20:11 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> Message-ID: <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> On Dec 8, 2017, at 4:45 PM, John Rose wrote: > > Can anyone point out a reason why the value based > lists of List.of() should serialize while the value based > lists of List.of().subList() should not? Or is there some > reason we should not allow subList to produce value > based lists? This gets to the heart of a question about how thoroughly we are going to adopt the concept of value-based. I think we want to accept that a sub-collection derived from a value-based collection is itself value-based, whether or not the API point (designed *before* value based data structures) is supposed to deliver a ?view? or not. I.e., a view of a value-based collection is indistinguishable from a non-view, and there are performance costs to maintaining a pretended distinction, so let?s tune our spec. to avoid those costs. If I'm right about this, perhaps the most natural way to balance Claes's design is to add an offset field to ListN, and allow ListN to project arbitrary slices out of a backing array. Then we have only two classes (ListN, List12) even when folks are slicing all around their lists. That strikes me as (in terms of number of loaded classes) much more economical than the stuff with a new bespoke SubList class, with it's own new bespoke iterator class. And I think the extra int field (in ListN) will melt into the noise. ? John From claes.redestad at oracle.com Sat Dec 9 02:19:13 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 9 Dec 2017 03:19:13 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> Message-ID: <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> Hi John, On 2017-12-09 02:20, John Rose wrote: > On Dec 8, 2017, at 4:45 PM, John Rose > wrote: >> >> Can anyone point out a reason why the value based >> lists of List.of() should serialize while the value based >> lists of List.of().subList() should not? ?Or is there some >> reason we should not allow subList to produce value >> based lists? > > This gets to the heart of a question about how thoroughly we are going > to adopt the concept of value-based. ?I think we want to accept that a > sub-collection derived from a value-based collection is itself > value-based, whether or not the API point (designed *before* value > based data structures) is supposed to deliver a ?view? or not. ?I.e., > a view of a value-based collection is indistinguishable from a > non-view, and there are performance costs to maintaining a pretended > distinction, so let?s tune our spec. to avoid those costs. > > If I'm right about this, perhaps the most natural way to balance Claes's > design is to add an offset field to ListN, and allow ListN to project > arbitrary > slices out of a backing array. > > Then we have only two classes (ListN, List12) even when folks are > slicing all around their lists. ?That strikes me as (in terms of number > of loaded classes) much more economical than the stuff with a new > bespoke SubList class, with it's own new bespoke iterator class. > And I think the extra int field (in ListN) will melt into the noise. > > ? John I think one can interpret the @implSpec in AbstracList::subList to suggest that the implementation retrieved will subclass AbstractList, which implies it's *not* Serializable. As we move away from AbstractList then we can of course choose our own spec, but since it's established behavior I tried as best I could to retrofit behavior... As time is likely up for getting this into 10 then I guess we might as well take a step back and do this right for 11. I suspect we'll need a CSR for this RFE regardless. Keeping it all down to two classes as you suggest allows any mixed usage to stay in the realm of bimorphism which might bring a considerable speedup in some cases, and the reduction in number of implementation classes loaded is a boon. Having all methods in ListN account for the offset might cost us a few percent here and there, though. An int offset on ListN would actually be footprint neutral compared to the pre-existing implementation which pulls in the int modCount from AbstractList. Thanks! /Claes From claes.redestad at oracle.com Sat Dec 9 02:45:52 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 9 Dec 2017 03:45:52 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: <03bca637-87e2-b5eb-4181-3a0c23efa417@oracle.com> On 2017-12-09 02:09, John Rose wrote: > On Dec 8, 2017, at 7:13 AM, Claes Redestad wrote: >> http://cr.openjdk.java.net/~redestad/8193128/open.03/ >> > + for (int i = elements.length - 1; i > 0; i--) { > > There's an OBOE in this line; should be i >= 0. I thought I tested the lastIndexOf(..) == 0 cases, but obviously only for the List12 case... > > Errors like this are a risk of hand-specializing code. > It's probably worth it here, but it's still a risk. > > For immutable lists, it would be reasonable to recode > the generic algorithms to use for-loops with integer > indexes and get(int) access. That will remove > the unused generality of iterators, and keep only > the abstraction across get(). The benefits of customized > inlining (to fold up the funny specialized get methods) > can probably be obtained like this: > > class AbsImmList { > @ForceInline int indexOf(Object x) { > for (int i = 0, s = size(); i < s; i++) if (x.equals(this.get(i))) return i; > } > } > final class List12 { > int indexOf(Object x) { return super.indexOf(x); } // specialize size/get > } > final class ListN { > int indexOf(Object x) { return super.indexOf(x); } // specialize size/get > } > > The optimization of the funny 1/2 loop for List12 should > unfold into the same code you'd write by hand; if not it's > a bug to fix in C2. Likewise for the loop in ListN. I agree, but left this as is for now. If we go the route of re-specifying behavior of subList we'll want to make sure we get the NPE behavior of indexOf/lastIndexOf right, too. > > ? John > > P.S. If you are going to hand-specialize, it might be > easier on the optimizer if you would add in this line > as needed: > > final E[] elements = this.elements; // cache field > > Usually C2 gets the same answer, but not always, > since final fields are not fully trustable, and "might" > (in some hypothetical system which C2 can't rule out) > change over a call to Object::equals. OTOH, generic > coding as above is preferable, and we can fix C2 > to hoist the @Stable elements field if it doesn't already. Ok. > > P.P.S. Why does ListN have arity 1 and arity 2 constructors? Another left-over from my earlier experiments, removed: http://cr.openjdk.java.net/~redestad/8193128/open.04/ /Claes From john.r.rose at oracle.com Sat Dec 9 03:00:42 2017 From: john.r.rose at oracle.com (John Rose) Date: Fri, 8 Dec 2017 19:00:42 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> Message-ID: On Dec 8, 2017, at 6:19 PM, Claes Redestad wrote: > > I think one can interpret the @implSpec in AbstracList::subList to suggest > that the implementation retrieved will subclass AbstractList, which implies > it's *not* Serializable. As we move away from AbstractList then we can of > course choose our own spec, but since it's established behavior I tried as > best I could to retrofit behavior? Maybe you are right about that, but AbstractList is *not* part of the API of the List returned by List.of, so you are free to do whatever is right for the value-based list, ignoring all parts of AbstractList that are not immediately valuable to you. (Yes, someone could "notice" that the result of List.of seems to be castable to AbstractList, and *then* lodge a complaint that its sublist is serializable, but that's a forced scenario; if we deem it is a real bug to fix, at some date far in the future, we can fix it by removing AbstractList from the supers of the List.of result.) > As time is likely up for getting this into 10 then I guess we might as well > take a step back and do this right for 11. I suspect we'll need a CSR for > this RFE regardless. No, we don't need a CSR. It's unspecified behavior. And in a brand new API. This is a valid refinement from murky to clear, not a change in spec. (Because it's brand new, there's not even a de facto spec. to worry about.) > Keeping it all down to two classes as you suggest allows any mixed usage to > stay in the realm of bimorphism which might bring a considerable speedup > in some cases, and the reduction in number of implementation classes > loaded is a boon. Having all methods in ListN account for the offset might > cost us a few percent here and there, though. I don't think it will. It's a fetch of an addend (usually zero) from a cache line that is already hot. Hot fetches and extra addends are usually in the noise. > An int offset on ListN would actually be footprint neutral compared to the > pre-existing implementation which pulls in the int modCount from AbstractList. Yep. And it will be an improvement to the number of classes loaded, both in the status quo and in your current version (with bespoke sublists and their iterators). ? John P.S. The extra cut-n-paste of the helper classes was bothering me, enough to consider asking for new API points for sublist views. But if you do the ListN with an offset, those problems can be pushed off for another day. From xueming.shen at oracle.com Sat Dec 9 05:36:36 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 08 Dec 2017 21:36:36 -0800 Subject: RFR JDK-8187485: Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <30f9111b-933d-c5e6-39dc-f45261549068@oracle.com> References: <5A25D6BE.60203@oracle.com> <30f9111b-933d-c5e6-39dc-f45261549068@oracle.com> Message-ID: <5A2B7664.1040103@oracle.com> On 12/7/17, 8:31 AM, mandy chung wrote: > > > On 12/4/17 3:14 PM, Xueming Shen wrote: >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8185582 >> webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev >> > > ZStreamRef.java > > 79 static ZStreamRef get(Object owner, LongSupplier addr, LongConsumer end) { > > It may be better to have two factory methods that take the specific type (Deflater& Inflater) > instead of this single method taking Object owner. Just feel it is not worth creating two methods, and then two subclasses for two almost identical implementations. Arguably even the ZStreamRef should have two separate versions. So I choose to pay the price of two runtime "if". > > > ZipFile.java > > 701 // List of cached Inflater objects for decompression > 702 Deque inflaterCache; > > should this be a volatile field? I read it again, still feel the "volatile" is not necessary in this situation. Let me know if I'm doing it wrong. > > > TestCleaner.java > 27 * @summary Check the resources of Inflater, Deflater and ZipFile are always > 28 * cleaned/released when the instane is not unreachable > > typo: s/instane/instance and s/not unreachable/unreachable updated in webrev. Thanks, Sherman From scolebourne at joda.org Sat Dec 9 09:20:29 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Sat, 9 Dec 2017 09:20:29 +0000 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Message-ID: On 8 December 2017 at 00:33, Stuart Marks wrote: > Please review this changeset that introduces a new no-arg method > orElseThrow() to Optional, as a preferred alternative to the get() method. I am willing to accept this addition as it has an obvious parallel to the existing supplier method. My position on removing get() has not changed however, and as such I am displeased with the addition of the "preferred alternative" text. Stephen From andrej.golovnin at gmail.com Sat Dec 9 10:24:28 2017 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Sat, 9 Dec 2017 11:24:28 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> Message-ID: <6AE01F3A-6540-4CA4-8605-A861A78C1AD1@gmail.com> Hi Claes, > I think one can interpret the @implSpec in AbstracList::subList to suggest > that the implementation retrieved will subclass AbstractList, which implies > it's *not* Serializable. Not for me. java.util.ArrayList is a subclass of AbstractList and is serializable. And you can use j.u.ArrayList to implement the #subList-method in a subclass of AbstractList without violating the @implSpec in AbstractList#subList. And as John already mentioned AbstractList is not a part of the Public API of the lists returned by the List#of()-methods. So the only spec you should care about is the one defined in List#subList(). And this spec says nothing about whether the returned instance should be searializable or not. > As time is likely up for getting this into 10 then I guess we might as well > take a step back and do this right for 11. I suspect we'll need a CSR for > this RFE regardless. > > Keeping it all down to two classes as you suggest allows any mixed usage to > stay in the realm of bimorphism which might bring a considerable speedup > in some cases, and the reduction in number of implementation classes > loaded is a boon. Having all methods in ListN account for the offset might > cost us a few percent here and there, though. > > An int offset on ListN would actually be footprint neutral compared to the > pre-existing implementation which pulls in the int modCount from AbstractList. I really like the idea from John to use List12 and ListN for sublists as it would give us for free the fast implementations for the methods like indexOf, lastIndexOf, etc. even for sublists. But I would not use offset. Do you remember the problem with String#substring() when the #substring()-method has returned a view over very huge String object which then could not be garbage collected due to a reference from the substring? I would just add a new constructor to ListN: ListN(E[] elements, int fromIndex, int toIndex) to copy elements from the parent list. And this would allow us to keep the implementation of the ListN class as is. One more thing: Could you please add specialised implementations also for the following methods: List.forEach(Consumer) List.spliterator() For List12 when List12.size() == 1 please use Collections.singletonSpliterator() (this method should be moved to the Spliterators class and be public). For List12 when List12.size() == 2 please use Arrays.spliterator(). For ListN when List.isEmpty() == true please use Spliterators.emptySpliterator?() and otherwise use Arrays.spliterator(). > http://cr.openjdk.java.net/~redestad/8193128/open.04/ In case you still would like to use the SubList class: 115 int size; The size field should be final. Thanks! Best regards, Andrej Golovnin From andrej.golovnin at gmail.com Sat Dec 9 10:43:03 2017 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Sat, 9 Dec 2017 11:43:03 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <6AE01F3A-6540-4CA4-8605-A861A78C1AD1@gmail.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> <6AE01F3A-6540-4CA4-8605-A861A78C1AD1@gmail.com> Message-ID: <96CA211F-8EE2-4186-A433-81B728000705@gmail.com> Hi Claes, > One more thing: Could you please add specialised implementations also > for the following methods: > > List.forEach(Consumer) > > List.spliterator() > For List12 when List12.size() == 1 please use Collections.singletonSpliterator() > (this method should be moved to the Spliterators class and be public). > For List12 when List12.size() == 2 please use Arrays.spliterator(). > > For ListN when List.isEmpty() == true please use Spliterators.emptySpliterator?() > and otherwise use Arrays.spliterator(). > I?m sorry I forgot to mention, that Set12, SetN, Map12 and MapN should also have specialised implementations for the #forEach-methods and for the #spliterator()-methods in Set12, SetN. Thanks! Best regards, Andrej Golovnin From forax at univ-mlv.fr Sat Dec 9 12:32:47 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 9 Dec 2017 13:32:47 +0100 (CET) Subject: Add EnumMap.keyType() and EnumSet.elementType() In-Reply-To: References: Message-ID: <1828048998.2518653.1512822767584.JavaMail.zimbra@u-pem.fr> Hi Andrej, I do not think it's a good idea. It's like exposing reified type arguments, i can see these kind of change back firing in at least two ways. It's not future proof, part of valhalla requires to change generics to allow generics of value types which requires a form of reification, but we may end up with a light form of reification similar to Scala type manifest (type arguments are available at construction time but not after that point) exposing type arguments for EnumSet will be seen as a mismatch in that case. Also, we may want to introduce unmodifiable EnumSet in the future, currently an unmodifiable empty EnumSet do not need to store any type argument, the proposed API force us to have as many empty EnumSet as type arguments combination (it's why you have one static field by type specialization in C#), again it makes the proposed API not future proof. regards, R?mi ----- Mail original ----- > De: "Stephen Colebourne" > ?: "core-libs-dev" > Envoy?: Jeudi 7 D?cembre 2017 13:09:40 > Objet: Re: Add EnumMap.keyType() and EnumSet.elementType() > I'm surprised I've never run into this. This seems like a simple and > useful change. > Stephen > > On 7 December 2017 at 11:40, Andrej Golovnin wrote: >> Hi all, >> >> it would be nice if we would have access to the class of the enum type >> used to create an EnumMap or an EnumSet. >> >> This is usefull when you write a custom serialization library and >> would like to serialize/deserialize an empty EnumMap or an empty >> EnumSet. For the empty EnumSet there is a workaround to get the enum >> class: >> >> EnumSet empty = EnumSet.noneOf(MyEnum.class); >> EnumSet tmp = EnumSet.complementOf(empty); >> Class elementType = tmp.iterator().next().getClass(); >> >> But this only works when the enum class has at least one enum. For >> EnumMap there is no such workaround at all and we have to use the >> Reflection API. And you know the warnings since Java 9 :-) : >> >> WARNING: An illegal reflective access operation has occurred >> WARNING: Illegal reflective access by c.r.r.k.EnumMapSerializer to >> field java.util.EnumMap.keyType >> WARNING: Please consider reporting this to the maintainers of >> c.r.r.k.EnumMapSerializer >> WARNING: Use --illegal-access=warn to enable warnings of further >> illegal reflective access operations >> WARNING: All illegal access operations will be denied in a future release >> >> So to avoid this warning and to avoid that our application stops to >> work with a future release of Java I would like to propose to add the >> following two methods: >> >> EnumMap: >> /** >> * Returns the {@code Class} object of the key type for this enum map. >> * >> * @return the {@code Class} object of the key type for this enum map. >> * >> * @since 10 >> */ >> public Class keyType() >> >> >> EnumSet: >> /** >> * Returns the {@code Class} object of all the elements of this set. >> * >> * @return the {@code Class} object of all the elements of this set. >> * >> * @since 10 >> */ >> public Class elementType() >> >> The suggested change is attached as diff. >> >> Best reagrds, > > Andrej Golovnin From claes.redestad at oracle.com Sat Dec 9 16:04:04 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 9 Dec 2017 17:04:04 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <96CA211F-8EE2-4186-A433-81B728000705@gmail.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> <6AE01F3A-6540-4CA4-8605-A861A78C1AD1@gmail.com> <96CA211F-8EE2-4186-A433-81B728000705@gmail.com> Message-ID: <1e82f382-6a00-ef8c-4033-537c325f05a8@oracle.com> Hi Andrej, forEach seems like a no-brainer, but spliterator might require some extra care. For example, returning Spliterators.emptySpliterator() and Collections.singletonSpliterator when appropriate *sounds* like nice little optimizations, but Arrays.spliterator(T[]) with an empty or single-element array appears to be relatively cheap operations, whereas mixing implementation could make call-sites accepting List.of(foo).spliterator() become megamorphic. Thus I think these should be done independently as a follow-up, along with tests and microbenchmarks. Thanks! /Claes On 2017-12-09 11:43, Andrej Golovnin wrote: > Hi Claes, > >> One more thing: Could you please add specialised implementations also >> for the following methods: >> >> List.forEach(Consumer) >> >> List.spliterator() >> For List12 when List12.size() == 1 please use Collections.singletonSpliterator() >> (this method should be moved to the Spliterators class and be public). >> For List12 when List12.size() == 2 please use Arrays.spliterator(). >> >> For ListN when List.isEmpty() == true please use Spliterators.emptySpliterator?() >> and otherwise use Arrays.spliterator(). >> > I?m sorry I forgot to mention, that Set12, SetN, Map12 and MapN should also > have specialised implementations for the #forEach-methods and for > the #spliterator()-methods in Set12, SetN. > > Thanks! > > Best regards, > Andrej Golovnin > From andrej.golovnin at gmail.com Sat Dec 9 18:29:02 2017 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Sat, 9 Dec 2017 19:29:02 +0100 Subject: Add EnumMap.keyType() and EnumSet.elementType() In-Reply-To: <1828048998.2518653.1512822767584.JavaMail.zimbra@u-pem.fr> References: <1828048998.2518653.1512822767584.JavaMail.zimbra@u-pem.fr> Message-ID: <6F7DB289-574A-44E3-B5B8-E33323DA77D1@gmail.com> Hi R?mi, > It's like exposing reified type arguments, i can see these kind of change back firing in at least two ways. > It's not future proof, part of valhalla requires to change generics to allow generics of value types which requires a form of reification, but we may end up with a light form of reification similar to Scala type manifest (type arguments are available at construction time but not after that point) exposing type arguments for EnumSet will be seen as a mismatch in that case. Also, we may want to introduce unmodifiable EnumSet in the future, currently an unmodifiable empty EnumSet do not need to store any type argument, the proposed API force us to have as many empty EnumSet as type arguments combination (it's why you have one static field by type specialization in C#), again it makes the proposed API not future proof. Only the death is future proof. Everything else is debatable. (c) :-P Empty EnumSet is not a problem. As I already said there is a workaround (an ugly one, but it is there). The real problem is an empty EnumMap. When the valhalla project would provide a solution to my problem, then I?m fine with it. When not, then let us debate about an alternative solution. :-) Best regards, Andrej Golovnin From andrej.golovnin at gmail.com Sat Dec 9 18:50:52 2017 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Sat, 9 Dec 2017 19:50:52 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <1e82f382-6a00-ef8c-4033-537c325f05a8@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> <6AE01F3A-6540-4CA4-8605-A861A78C1AD1@gmail.com> <96CA211F-8EE2-4186-A433-81B728000705@gmail.com> <1e82f382-6a00-ef8c-4033-537c325f05a8@oracle.com> Message-ID: Hi Claes, > For example, returning Spliterators.emptySpliterator() and > Collections.singletonSpliterator when appropriate *sounds* like nice little > optimizations, but Arrays.spliterator(T[]) with an empty or single-element array > appears to be relatively cheap operations, whereas mixing implementation could > make call-sites accepting List.of(foo).spliterator() become megamorphic. > > Thus I think these should be done independently as a follow-up, along with > tests and microbenchmarks. I?m fine with it. The most important part is that you now aware of this problem and I?m sure that you would provide the best possible solution. :-) Best regards, Andrej Golovnin From peter.levart at gmail.com Sat Dec 9 20:01:31 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 9 Dec 2017 21:01:31 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> Message-ID: <5c6f8ce0-910c-8f3a-6cbc-6318a2f94ea9@gmail.com> Hi Claes, Claes Redestad je 09. 12. 2017 ob 03:19?napisal: > Hi John, > > On 2017-12-09 02:20, John Rose wrote: >> On Dec 8, 2017, at 4:45 PM, John Rose > > wrote: >>> >>> Can anyone point out a reason why the value based >>> lists of List.of() should serialize while the value based >>> lists of List.of().subList() should not? ?Or is there some >>> reason we should not allow subList to produce value >>> based lists? One thing that might be implied from the specification that talks about "...a view of the portion of this list between the specified indexes..." is that the view keeps a reference to the original list. This is observable if combined with Reference object(s). But I don't know if this is an important behavioral detail that any code depends on. Regards, Peter From peter.levart at gmail.com Sat Dec 9 20:14:28 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 9 Dec 2017 21:14:28 +0100 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <5c6f8ce0-910c-8f3a-6cbc-6318a2f94ea9@gmail.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> <5c6f8ce0-910c-8f3a-6cbc-6318a2f94ea9@gmail.com> Message-ID: Peter Levart je 09. 12. 2017 ob 21:01?napisal: > Hi Claes, > > Claes Redestad je 09. 12. 2017 ob 03:19?napisal: >> Hi John, >> >> On 2017-12-09 02:20, John Rose wrote: >>> On Dec 8, 2017, at 4:45 PM, John Rose >> > wrote: >>>> >>>> Can anyone point out a reason why the value based >>>> lists of List.of() should serialize while the value based >>>> lists of List.of().subList() should not? ?Or is there some >>>> reason we should not allow subList to produce value >>>> based lists? > > One thing that might be implied from the specification that talks > about "...a view of the portion of this list between the specified > indexes..." is that the view keeps a reference to the original list. > This is observable if combined with Reference object(s). But I don't > know if this is an important behavioral detail that any code depends on. > > Regards, Peter BTW, ImmutableCollections.AbstractImmutableList.SubList already violates that assumption and nobody is complaining. Peter From patrick at reini.net Sat Dec 9 21:28:27 2017 From: patrick at reini.net (Patrick Reinhart) Date: Sat, 9 Dec 2017 22:28:27 +0100 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: Hi Brian, > All previous suggested changes have been made in > > http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ > 99 @Override 100 public int read(byte[] b, int off, int len) throws IOException { 101 Objects.checkFromIndexSize(off, len, b.length); Should not be checked for a null byte buffer b here? -Patrick From martijnverburg at gmail.com Sat Dec 9 22:05:54 2017 From: martijnverburg at gmail.com (Martijn Verburg) Date: Sat, 9 Dec 2017 22:05:54 +0000 Subject: RFR(s): 8177681: Remove methods Runtime.getLocalized{Input, Output}Stream In-Reply-To: <20171206223534.921941454@eggemoggin.niobe.net> References: <418e5290-d0d5-c21c-9fde-258ef53c75fd@oracle.com> <20171206223534.921941454@eggemoggin.niobe.net> Message-ID: That must be oddly satisfying :-D Cheers, Martijn On 7 December 2017 at 03:35, wrote: > 2017/12/6 17:33:36 -0500, stuart.marks at oracle.com: > > Please review the removal of these methods, which were part of an > obsolete > > internationalization mechanism. They were deprecated in JDK 1.1 and > deprecated > > for removal in JDK 9. As far as I can see, there is no usage of these > methods > > anywhere. > > As the developer responsible for deprecating these methods > in JDK 1.1 -- way back in 1996 --- I heartily approve of this > change! > > - Mark > From peter.levart at gmail.com Sat Dec 9 22:33:04 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 9 Dec 2017 23:33:04 +0100 Subject: RFR 8191216: SimpleTimeZone.clone() has a data race on cache fields In-Reply-To: <86b95ce3-2acd-8fda-77cf-222f957ac687@oracle.com> References: <53644ce5-7c7c-18ff-4da9-7dfacd5b678d@linux.vnet.ibm.com> <576cd10d-c75e-c7e2-ecb9-e28d09e059cf@linux.vnet.ibm.com> <84ca56e0-67c8-3fbb-d558-4ebe797d513c@oracle.com> <8894f619-3510-acfa-3987-f013ca3638f3@gmail.com> <72a7efe3-ac34-1534-3a9b-146c647e1292@linux.vnet.ibm.com> <017c44ce-a3e2-a03d-d437-98e08655a6a1@gmail.com> <3bf0ce49-cb3d-f70b-717d-03680b001d3b@oracle.com> <9dbdf9bc-e93a-6ac4-d3ae-5168fd365403@oracle.com> <86b95ce3-2acd-8fda-77cf-222f957ac687@oracle.com> Message-ID: <03df6d4c-2392-642f-7e9f-048459bd7cf8@gmail.com> Hi Naoto, Thank you for reviewing. Naoto Sato je 06. 12. 2017 ob 20:41?napisal: > Hi Peter, Venkat, > > Thank you for the fix. It looks good to me. Improved performance is a > nice bonus! Would you be able to provide with a regression test? Sure, here it is: http://cr.openjdk.java.net/~plevart/jdk10-dev/8191216_SimpleTimeZone_clone_race/webrev.02/ The test reliably detects several races in 2 seconds of runtime which cause incorrect offset calculations in JDK 9: shared TZ: 7363986 correct, 0 incorrect calculations cloned TZ: 3960264 correct, 1180 incorrect calculations Exception in thread "main" java.lang.AssertionError: 1180 fatal data races detected ??? at SimpleTimeZoneCloneRaceTest.main(SimpleTimeZoneCloneRaceTest.java:86) With patch applied, there are no failures of course. Can this go into JDK 10 ? Regards, Peter > > Naoto > > On 12/6/17 6:10 AM, Peter Levart wrote: >> Hi, >> >> On 12/06/2017 02:30 PM, Alan Bateman wrote: >>> I think this class is normally maintained on i18n-dev but I think >>> introducing the Cache object looks good and making this much easier >>> to understand. >>> >>> -Alan >> >> Thanks Alan, I'm forwarding to i18n-dev to see if maintainers of that >> part of JDK have any comments... >> >> This is an official request for reviewing a patch for fixing a data >> race between cloning a SimpleTimeZone object and lazily initializing >> its 3 cache fields which may produce a clone with inconsistent cache >> state. Here's Jira issue with details: >> >> https://bugs.openjdk.java.net/browse/JDK-8191216 >> >> Venkat has proposed to simply call invalidateCache() on the clone >> before returning it from SimpleTimeZone.clone() method: >> >> ???? public Object clone() >> ???? { >> ???????? SimpleTimeZone clone = (SimpleTimeZone) super.clone(); >> ???????? clone.invalidateCache(); >> ???????? return clone; >> ???? } >> >> This fixes the issue and for the case of TimeZone.getDefault() which >> is called from ZoneId.systemDefault() even elides synchronization in >> clone.invalidateCache() and allocation of the clone, so JITed >> ZoneId.systemDefault() is unaffected by the patch. Initially I was >> satisfied with the fix, but then I tested something. Suppose someone >> sets default zone to SimpleTimeZone: >> >> ???????? TimeZone.setDefault( >> ???????????? new SimpleTimeZone(3600000, >> ??????????????????????????????? "Europe/Paris", >> ??????????????????????????????? Calendar.MARCH, -1, Calendar.SUNDAY, >> ??????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, >> ??????????????????????????????? Calendar.OCTOBER, -1, Calendar.SUNDAY, >> ??????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, >> ??????????????????????????????? 3600000) >> ???????? ); >> >> And then calls some methods that initialize the cache of the internal >> shared TimeZone.defaultTimeZone instance: >> >> ???? new Date().toString(); >> >> The code which after that tries to obtain default time zone and >> calculate the offset from UTC at some current point in time, for >> example: >> >> ???? TimeZone.getDefault().getOffset(now) >> >> can't use the cached state because it has been invalidated in the >> returned clone. Such code has to re-compute the offset every time it >> gets new clone. I measured this with a JMH benchmark and got the >> following result: >> >> Default: >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? >> ns/op >> >> Venkat's patch - invalidateCache(): >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 179,476 ? 1,942? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,538 ? 0,073? >> ns/op >> >> We see that ZoneId.systemDefault() is unaffected, but >> TimeZone.getDefault().getOffset(now) becomes 3x slower. >> >> This is not good, so I tried an alternative fix for the issue - >> simply making the SimpleTimeZone.clone() synchronized. Access to >> cache fields is already synchronized, so this should fix the race. >> Here's the JMH result: >> >> Default: >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? >> ns/op >> >> Synchronized clone(): >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 58,103 ? 0,936? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 4,154 ? 0,034? >> ns/op >> >> We see that caching works again, but synchronization has some >> overhead which is not big for single-threaded access, but might get >> bigger when multiple threads try to get default zone concurrently. >> >> So I created a 3rd variant of the fix which I'm presenting here and >> requesting a review for: >> >> http://cr.openjdk.java.net/~plevart/jdk10-dev/8191216_SimpleTimeZone_clone_race/webrev.01/ >> >> >> The JMH benchmark shows this: >> >> Default: >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040? >> ns/op >> >> Cache object: >> >> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 42,860 ? 0,274? >> ns/op >> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,545 ? 0,057? >> ns/op >> >> Not only does the fix not affect ZoneId.systemDefault() which is not >> surprising, but it also speeds-up cache lookup in single-threaded >> benchmark and certainly eliminates possible contention among threads >> looking up the shared instance. >> >> I have run the test/jdk/java/util/TimeZone and >> test/jdk/java/util/Calendar jtreg tests and there were 7 failures >> caused by compilation errors (package sun.util.locale.provider is not >> visible), while 59 other tests pass. >> >> So, what do you think? >> >> Regards, Peter >> >> >> Venkateswara R Chintala je 21. 11. 2017 ob 10:14?napisal: >>> Thanks Peter for sponsoring this patch. Is there anything else that >>> needs to be done from my end for this patch to be integrated? Please >>> let me know. >>> >>> -Venkat >>> >>> >>> On 14/11/17 8:46 PM, Peter Levart wrote: >>>> Hi Venkat, >>>> >>>> I created the following issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8191216 >>>> >>>> I can also sponsor this patch and push it for JDK 10. >>>> >>>> The patch that you are proposing looks good to me. Does anybody >>>> have anything else to say? >>>> >>>> Regards, Peter >>>> >>>> >>>> On 11/13/2017 11:28 AM, Venkateswara R Chintala wrote: >>>>> Thanks David, Peter for your review and comments. As I am new to >>>>> the community, can you please help me to open a bug and integrate >>>>> the changes into code base? >>>>> >>>>> -Venkat >>>>> >>>>> On 12/11/17 2:19 AM, Peter Levart wrote: >>>>>> Hi David, Venkat, >>>>>> >>>>>> On 11/11/17 21:15, Peter Levart wrote: >>>>>>> For example, take the following method: >>>>>>> >>>>>>> String defaultTZID() { >>>>>>> ??? return TimeZone.getDefault().getID(); >>>>>>> } >>>>>>> >>>>>>> When JIT compiles it and inlines invocations to other methods >>>>>>> within it, it can prove that cloned TimeZone instance never >>>>>>> escapes the call to defaultTZID() and can therefore skip >>>>>>> allocating the instance on heap. >>>>>>> >>>>>>> But this is fragile. If code in invoked methods changes, they >>>>>>> may not get inlined or EA may not be able to prove that the >>>>>>> cloned instance can't escape and allocation may be introduced. >>>>>>> ZoneId.systemDefault() is a hot method and it would be nice if >>>>>>> we manage to keep it allocation free. >>>>>> >>>>>> Well, I tried the following variant of SimpleTimeZone.clone() patch: >>>>>> >>>>>> ??? public Object clone() >>>>>> ??? { >>>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>>> ??????? // like tz.invalidateCache() but without holding a lock >>>>>> on clone >>>>>> ??????? tz.cacheYear = tz.startYear - 1; >>>>>> ??????? tz.cacheStart = tz.cacheEnd = 0; >>>>>> ??????? return tz; >>>>>> ??? } >>>>>> >>>>>> >>>>>> ...and the JMH benchmark with gc profiling shows that >>>>>> ZoneId.systemDefault() still manages to get JIT-compiled without >>>>>> introducing allocation. >>>>>> >>>>>> Even the following (original Venkat's) patch: >>>>>> >>>>>> ??? public Object clone() >>>>>> ??? { >>>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>>> ??????? tz.invalidateCache(); >>>>>> ??????? return tz; >>>>>> ??? } >>>>>> >>>>>> ...does the same and the locking in invalidateCache() is elided >>>>>> too. Allocation and lock-elision go hand-in-hand. When object >>>>>> doesn't escape, allocation on heap may be eliminated and locks on >>>>>> that instance elided. >>>>>> >>>>>> So this is better than synchronizing on the original instance >>>>>> during .clone() execution as it has potential to avoid locking >>>>>> overhead. >>>>>> >>>>>> So Venkat, go ahead. My fear was unjustified. >>>>>> >>>>>> Regards, Peter >>>>>> >>>>> >>>> >>>> >>> From patrick at reini.net Sun Dec 10 08:34:18 2017 From: patrick at reini.net (Patrick Reinhart) Date: Sun, 10 Dec 2017 09:34:18 +0100 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: Sorry, missed that thread: http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050458.html -Patrick Am 09.12.2017 um 22:28 schrieb Patrick Reinhart: > Hi Brian, > >> All previous suggested changes have been made in >> >> http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ >> > 99 @Override > 100 public int read(byte[] b, int off, int len) throws IOException { > 101 Objects.checkFromIndexSize(off, len, b.length); > > Should not be checked for a null byte buffer b here? > > -Patrick From forax at univ-mlv.fr Sun Dec 10 11:47:52 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Sun, 10 Dec 2017 12:47:52 +0100 (CET) Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: <04841738-95fc-c2fe-1c30-40dac1d5ca04@oracle.com> References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> <967605459.2165732.1512725727144.JavaMail.zimbra@u-pem.fr> <04841738-95fc-c2fe-1c30-40dac1d5ca04@oracle.com> Message-ID: <416797559.2683071.1512906472955.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stuart Marks" > ?: "Remi Forax" , "Alan Bateman" > Cc: "core-libs-dev" > Envoy?: Samedi 9 D?cembre 2017 00:48:08 > Objet: Re: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() >>>> Please review this changeset that introduces a new no-arg method >>>> orElseThrow() to Optional, as a preferred alternative to the get() >>>> method. >>>> >>> This looks good. The naming is consistent and I think better than the >>> "getWhenPresent" proposed in the original thread. >> >> i agree with Alan. >> >> Having a name that starts with "get" as the great advantage as to be visible in >> the IDE completion box just after the method get() so it will help people to >> transition from get() to getWhenPresent() > > I'm not sure whether you're agreeing or disagreeing. :-) oops, read the Alan's reply too fast. > > The current proposal is for orElseThrow(). > > Having a method that starts with "get" (such as "getWhenPresent" or > "getOrThrow") is indeed helpful when doing auto-completion, but it sort-of > starts to create a new family of get- methods that overlap with the existing > orElse- methods in an odd way. I think the no-arg orElseThrow() fits better with > the existing three orElse- methods. This leaves get() as the outlier, which is > ok if we maybe eventually deprecate it. ok, make sense. > >> BTW, i do not see the point to not deprecate get() at the same time. > > Much of the resistance to the earlier proposal was about deprecation of get(), > so I wanted to set that aside in order to proceed with introduction of this > alternative. ok > > s'marks R?mi From forax at univ-mlv.fr Sun Dec 10 11:48:18 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Sun, 10 Dec 2017 12:48:18 +0100 (CET) Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: <83fd6ac6-436a-a788-ceb8-5a1abffd7d08@oracle.com> References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> <1088674183.2171176.1512726335648.JavaMail.zimbra@u-pem.fr> <83fd6ac6-436a-a788-ceb8-5a1abffd7d08@oracle.com> Message-ID: <1137779798.2683114.1512906498979.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stuart Marks" > ?: "Remi Forax" > Cc: "core-libs-dev" > Envoy?: Samedi 9 D?cembre 2017 01:00:06 > Objet: Re: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() > Please, let's not derail this thread. :-) > > I think would be a good thing to think about, so I've filed a JIRA issue to > track it: > > https://bugs.openjdk.java.net/browse/JDK-8193279 > > s'marks Thanks, R?mi > > > On 12/8/17 1:45 AM, Remi Forax wrote: >> Let's gently derail this thread, the is another pain point with the current >> optional API, >> it seems that are no simple way to transform an Optional to an >> OptionalInt and vice-versa. >> >> It's painful because we start to see interface that returns by example >> OptionalInt while the implementation you want to connect with return an >> Optional. >> >> The only workaround seems to be to use a Stream/IntStream: >> Optional -> OptionalInt >> optional.stream().mapToInt(x -> x).findFirst() >> OptionalInt -> Optional >> optionalInt.stream().boxed().findFirst(); >> >> I think, Optional should have the method mapTo*/flatMapTo* and >> Optional[Primitive] the method boxed(). >> >> regards, >> R?mi >> >> ----- Mail original ----- >>> De: "Stuart Marks" >>> ?: "core-libs-dev" >>> Envoy?: Vendredi 8 D?cembre 2017 01:33:41 >>> Objet: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred >>> alternative to get() >> >>> Hi all, >>> >>> Please review this changeset that introduces a new no-arg method orElseThrow() >>> to Optional, as a preferred alternative to the get() method. >>> >>> Corresponding methods are also added to OptionalDouble, Int, and Long. >>> >>> The orElseThrow() method is functionally identical to get(). It has some >>> affinity with the existing orElseThrow(exceptionSupplier) method, being >>> equivalent to orElseThrow(NoSuchElementException::new). >>> >>> The get() method is functionally unchanged. It is NOT being deprecated, although >>> some wording in the doc has been added to point to orElseThrow() as the >>> "preferred alternative." This is part of a (slow) migration away from >>> Optional.get(), which has an obvious, attractive name that is often misused. >>> These issues have been discussed on this list previously: >>> >>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-April/040484.html >>> >>> Please note that much of that discussion was about the then-proposed deprecation >>> of Optional.get(), which is NOT part of this proposal. There are no plans to >>> deprecate Optional.get() at this time. This proposal ONLY introduces a new >>> method orElseThrow() that is identical in function to get(). >>> >>> Bug report: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8140281 >>> >>> Webrev: >>> >>> http://cr.openjdk.java.net/~smarks/reviews/8140281/webrev.1/ >>> >>> Thanks, >>> > >> s'marks From claes.redestad at oracle.com Sun Dec 10 23:12:49 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 11 Dec 2017 00:12:49 +0100 Subject: RFR: JDK-8184947:,ZipCoder performance improvements In-Reply-To: <5A2B1BAB.7010102@oracle.com> References: <5A2B1BAB.7010102@oracle.com> Message-ID: <0ced2d86-f2bc-54d2-bbfa-a8f295d69127@oracle.com> Hi Sherman, On 2017-12-09 00:09, Xueming Shen wrote: > Hi, > > Please help review the changes for j.u.z.ZipCoder/JDK-8184947 (which > also includes > cleanup/improvement work in java.lang.StringCoding.java to speed up > general String > coding performance, especially for UTF8). > > issue: https://bugs.openjdk.java.net/browse/JDK-8184947 > webrev: http://cr.openjdk.java.net/~sherman/8184947/webrev I've not fully reviewed this yet, but something struck me halfway through: As the ASCII fast-path is what's really important here, we could write that part without ever having to go via a StringCoding.Result. On four of your ZipCodingBM micros this improves performance a bit further (~10%): diff -r 848591d85052 src/java.base/share/classes/java/lang/StringCoding.java --- a/src/java.base/share/classes/java/lang/StringCoding.java??? Sun Dec 10 18:48:21 2017 +0100 +++ b/src/java.base/share/classes/java/lang/StringCoding.java??? Sun Dec 10 18:55:38 2017 +0100 @@ -937,7 +937,13 @@ ????? * Throws iae, instead of replacing, if malformed or unmmappble. ????? */ ???? static String newStringUTF8NoRepl(byte[] bytes, int off, int len) { -??????? Result ret = decodeUTF8(bytes, off, len, false); +??????? if (COMPACT_STRINGS && !hasNegatives(bytes, off, len)) { +??????????? return new String(Arrays.copyOfRange(bytes, off, off + len), LATIN1); +??????? } +??????? Result ret = decodeUTF8_0(bytes, off, len, false); ???????? return new String(ret.value, ret.coder); ???? } Benchmark??????????????? Mode? Cnt??? Score?? Error? Units ZipCodingBM.jf_entries?? avgt?? 25?? 43.682 ? 0.656? us/op ZipCodingBM.jf_stream??? avgt?? 25?? 42.075 ? 0.444? us/op ZipCodingBM.zf_entries?? avgt?? 25?? 43.323 ? 0.572? us/op ZipCodingBM.zf_stream??? avgt?? 25?? 40.237 ? 0.604? us/op After: Benchmark??????????????? Mode? Cnt??? Score?? Error? Units ZipCodingBM.jf_entries?? avgt?? 25?? 37.551 ? 1.198? us/op ZipCodingBM.jf_stream??? avgt?? 25?? 38.065 ? 0.628? us/op ZipCodingBM.zf_entries?? avgt?? 25?? 37.595 ? 0.686? us/op ZipCodingBM.zf_stream??? avgt?? 25?? 35.734 ? 0.442? us/op (I don't know which jar you using as test.jar, but results seems consistent across a few different ones) The gain is achieved by not going via the ThreadLocal resultCache, which checks out when inspecting the perfasm output. I'm a bit skeptical of ThreadLocal caching optimizations for such small objects (StringCoding.Result), and wonder if there's something else we can do to help the optimizer out here, possibly eliminating the allocation entirely. Thanks! /Claes From xueming.shen at oracle.com Mon Dec 11 04:08:54 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Sun, 10 Dec 2017 20:08:54 -0800 Subject: RFR: JDK-8184947:,ZipCoder performance improvements In-Reply-To: <0ced2d86-f2bc-54d2-bbfa-a8f295d69127@oracle.com> References: <5A2B1BAB.7010102@oracle.com> <0ced2d86-f2bc-54d2-bbfa-a8f295d69127@oracle.com> Message-ID: <5A2E04D6.7000708@oracle.com> On 12/10/17, 3:12 PM, Claes Redestad wrote: > Hi Sherman, > > On 2017-12-09 00:09, Xueming Shen wrote: >> Hi, >> >> Please help review the changes for j.u.z.ZipCoder/JDK-8184947 (which >> also includes >> cleanup/improvement work in java.lang.StringCoding.java to speed up >> general String >> coding performance, especially for UTF8). >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8184947 >> webrev: http://cr.openjdk.java.net/~sherman/8184947/webrev > > I've not fully reviewed this yet, but something struck me halfway > through: As the ASCII > fast-path is what's really important here, we could write that part > without ever having > to go via a StringCoding.Result. > > On four of your ZipCodingBM micros this improves performance a bit > further (~10%): > > diff -r 848591d85052 > src/java.base/share/classes/java/lang/StringCoding.java > --- a/src/java.base/share/classes/java/lang/StringCoding.java Sun > Dec 10 18:48:21 2017 +0100 > +++ b/src/java.base/share/classes/java/lang/StringCoding.java Sun > Dec 10 18:55:38 2017 +0100 > @@ -937,7 +937,13 @@ > * Throws iae, instead of replacing, if malformed or unmmappble. > */ > static String newStringUTF8NoRepl(byte[] bytes, int off, int len) { > - Result ret = decodeUTF8(bytes, off, len, false); > + if (COMPACT_STRINGS && !hasNegatives(bytes, off, len)) { > + return new String(Arrays.copyOfRange(bytes, off, off + > len), LATIN1); > + } > + Result ret = decodeUTF8_0(bytes, off, len, false); > return new String(ret.value, ret.coder); > } Hi Claes, You're definite right on this one. We dont need the Result for the ASCII case. webrev has been updated accordingly. http://cr.openjdk.java.net/~sherman/8184947/webrev Yes, understood the threadlocal might not be the best choice here. But just feel something need to be done for the temporary Result object after observed its usage with the jfr in #8184947. It is taking as many spaces as the overall String objects do. Sure, it's in young-gen, should be wiped with quickly. But my take is it might be worth the tradeoff of having each/every new String/cs) get a little slower instead of having the "global" vm has to do some extra clean up for this extra Result object, for now. thanks, sherman From nishit.jain at oracle.com Mon Dec 11 09:04:32 2017 From: nishit.jain at oracle.com (Nishit Jain) Date: Mon, 11 Dec 2017 14:34:32 +0530 Subject: [10]RFR 8190278: ClassCastException is thrown by java.util.Scanner when a NumberFormatProvider is used. Message-ID: <04d2ca68-7cb8-e255-6fb1-51cb157e17f5@oracle.com> Hi, Please review the fix for JDK-8190278 Bug: https://bugs.openjdk.java.net/browse/JDK-8190278 Webrev: http://cr.openjdk.java.net/~nishjain/8190278/webrev.03/ Fix: Modified the code to check whether the object returned by NumberFormat#getNumberInstance(Locale) is a DecimalFormat object, if not, ? ?? then use the DecimalFormat constructor way to create its object, where SPI provider implementation is ignored. Regards, Nishit Jain From srinivas.dama at oracle.com Mon Dec 11 15:26:18 2017 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 11 Dec 2017 07:26:18 -0800 (PST) Subject: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) Message-ID: <3558275d-6cd3-47bc-8203-da263835f747@default> Hi, Please review http://cr.openjdk.java.net/~sdama/8011697/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8011697 Fix is to make sure ScriptEngineManager always returns a particular engine on all platforms consistently. Regards, Srinivas From jai.forums2013 at gmail.com Mon Dec 11 15:46:06 2017 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Mon, 11 Dec 2017 21:16:06 +0530 Subject: Odd result with canonical paths with intermixed usage of java.io.File and java.nio.file.Files APIs In-Reply-To: <47fb187d-ef34-d707-f55a-c06872326795@oracle.com> References: <0cef83a0-174f-d7bc-9a19-fb2d895777ae@gmail.com> <47fb187d-ef34-d707-f55a-c06872326795@oracle.com> Message-ID: <40117b22-7f80-876f-f1ff-3071946d2f47@gmail.com> Moved this thread from discussions mailing list[1] to here. Comments inline. On 11/12/17 7:55 PM, Alan Bateman wrote: > On 11/12/2017 14:06, Jaikiran Pai wrote: >> : >> >> So a few related questions that I have are: >> >> ??? 1. Is this inconsistency an expected behaviour or is this a bug? >> ??? 2. If this is an expected behaviour, then would it be a better >> idea (as an application developer) to use Path.toRealPath[2] instead >> of using the File.getCanonicalPath()? Are these 2 APIs semantically >> equivalent? The File.getCanonicalPath() talks about the canonical >> path being "unique" paths but the Path.toRealPath has no such mentions. >> > The correctness issues with the canonicalization cache is a long > standing issue [1]. You can workaround it by running with > -Dsun.io.useCanonCaches=false as I think you have found. The goal is > to eventually disable and remove it. The first steps to get there > happened in JDK 9 when FilePermission was changed to not canonicalize > by default (FilePermission was the original motivation for this cache). > > If you have follow-up questions then please bring the thread to > core-libs-dev. > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-7066948 Thanks Alan. Given that the cache itself is being planned to be eventually removed, that answers the main part of my question and I can workaround this issue in a couple of ways (disabling the cache using that system property is one way) in the application, till that time. The only remaining part that I'm curious about is this: > ... would it be a better idea (as an application developer) to use Path.toRealPath[2] instead of using the File.getCanonicalPath()? Are these 2 APIs semantically equivalent? The File.getCanonicalPath() talks about the canonical path being "unique" paths but the Path.toRealPath has no such mentions. [1] http://mail.openjdk.java.net/pipermail/discuss/2017-December/004655.html -Jaikiran From Alan.Bateman at oracle.com Mon Dec 11 16:13:16 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Dec 2017 16:13:16 +0000 Subject: Odd result with canonical paths with intermixed usage of java.io.File and java.nio.file.Files APIs In-Reply-To: <40117b22-7f80-876f-f1ff-3071946d2f47@gmail.com> References: <0cef83a0-174f-d7bc-9a19-fb2d895777ae@gmail.com> <47fb187d-ef34-d707-f55a-c06872326795@oracle.com> <40117b22-7f80-876f-f1ff-3071946d2f47@gmail.com> Message-ID: On 11/12/2017 15:46, Jaikiran Pai wrote: > Thanks Alan. Given that the cache itself is being planned to be > eventually removed, that answers the main part of my question and I > can workaround this issue in a couple of ways (disabling the cache > using that system property is one way) in the application, till that > time. > > The only remaining part that I'm curious about is this: > > > ... would it be a better idea (as an application developer) to use > Path.toRealPath[2] instead of using the File.getCanonicalPath()? Are > these 2 APIs semantically equivalent? The File.getCanonicalPath() > talks about the canonical path being "unique" paths but the > Path.toRealPath has no such mentions. The main semantic difference is that toRealPath method can only return the real path of existing file (the file must exist). I don't know if that helps with your scenario or not. -Alan. From Alan.Bateman at oracle.com Mon Dec 11 16:34:47 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Dec 2017 16:34:47 +0000 Subject: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) In-Reply-To: <3558275d-6cd3-47bc-8203-da263835f747@default> References: <3558275d-6cd3-47bc-8203-da263835f747@default> Message-ID: <86f58aa7-2805-7c4b-4130-da1b520e47da@oracle.com> On 11/12/2017 15:26, Srinivas Dama wrote: > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8011697/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8011697 > > Fix is to make sure ScriptEngineManager always returns a particular engine on all platforms consistently. > I assume using a TreeSet would work too. A different approach is replace engineSpis with nameToFactory and extensionToFactory maps so that the getEngineByXXX methods don't need to a sequential search. -Alan From jai.forums2013 at gmail.com Mon Dec 11 16:43:45 2017 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Mon, 11 Dec 2017 22:13:45 +0530 Subject: Odd result with canonical paths with intermixed usage of java.io.File and java.nio.file.Files APIs In-Reply-To: References: <0cef83a0-174f-d7bc-9a19-fb2d895777ae@gmail.com> <47fb187d-ef34-d707-f55a-c06872326795@oracle.com> <40117b22-7f80-876f-f1ff-3071946d2f47@gmail.com> Message-ID: <619f1e41-b4f0-8fbe-fa39-33bfc5fa4995@gmail.com> On 11/12/17 9:43 PM, Alan Bateman wrote: > On 11/12/2017 15:46, Jaikiran Pai wrote: >> Thanks Alan. Given that the cache itself is being planned to be >> eventually removed, that answers the main part of my question and I >> can workaround this issue in a couple of ways (disabling the cache >> using that system property is one way) in the application, till that >> time. >> >> The only remaining part that I'm curious about is this: >> >> > ... would it be a better idea (as an application developer) to use >> Path.toRealPath[2] instead of using the File.getCanonicalPath()? Are >> these 2 APIs semantically equivalent? The File.getCanonicalPath() >> talks about the canonical path being "unique" paths but the >> Path.toRealPath has no such mentions. > The main semantic difference is that toRealPath method can only return > the real path of existing file (the file must exist). I don't know if > that helps with your scenario or not. It won't help in the specific case where I was thinking of using this, but it's a good detail to know. Thank you. -Jaikiran From Alan.Bateman at oracle.com Mon Dec 11 16:52:38 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Dec 2017 16:52:38 +0000 Subject: FW: RFR(M): 8189102: All tools should support -?, -h and --help In-Reply-To: <456c694ec25a42658af1b8600aaa136d@sap.com> References: <8aa25976d496465fac00a4ae3628e533@sap.com> <456c694ec25a42658af1b8600aaa136d@sap.com> Message-ID: <6d7d895f-9f23-fd4b-39ca-ef9bb0331fb9@oracle.com> On 07/12/2017 11:20, Lindenmaier, Goetz wrote: > Hi, > > ... missed some lists in my first post ... > > I prepared a fifth webrev for this change. Please review. > > It incorporates the changes requested by the CSR reviewers > (not to remove docuemtation of '-help' where is was documented > before) and the changes proposed by Kumar: > http://cr.openjdk.java.net/~goetz/wr17/8189102-helpMessage/webrev.05/ > > Looks like it still drops -help from the javadoc usage message, I can't tell if you meant to do that. -Alan. From naoto.sato at oracle.com Mon Dec 11 18:41:03 2017 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 11 Dec 2017 10:41:03 -0800 Subject: RFR 8191216: SimpleTimeZone.clone() has a data race on cache fields In-Reply-To: <03df6d4c-2392-642f-7e9f-048459bd7cf8@gmail.com> References: <53644ce5-7c7c-18ff-4da9-7dfacd5b678d@linux.vnet.ibm.com> <576cd10d-c75e-c7e2-ecb9-e28d09e059cf@linux.vnet.ibm.com> <84ca56e0-67c8-3fbb-d558-4ebe797d513c@oracle.com> <8894f619-3510-acfa-3987-f013ca3638f3@gmail.com> <72a7efe3-ac34-1534-3a9b-146c647e1292@linux.vnet.ibm.com> <017c44ce-a3e2-a03d-d437-98e08655a6a1@gmail.com> <3bf0ce49-cb3d-f70b-717d-03680b001d3b@oracle.com> <9dbdf9bc-e93a-6ac4-d3ae-5168fd365403@oracle.com> <86b95ce3-2acd-8fda-77cf-222f957ac687@oracle.com> <03df6d4c-2392-642f-7e9f-048459bd7cf8@gmail.com> Message-ID: Hi Peter, Thanks for the tests. Looks good to me. One nit: it should throw an Exception instead of AssertionError when the test fails. No further review is needed. > Can this go into JDK 10 ? You can push it before the JDK 10 fork. Naoto On 12/9/17 2:33 PM, Peter Levart wrote: > Hi Naoto, > > Thank you for reviewing. > > Naoto Sato je 06. 12. 2017 ob 20:41?napisal: >> Hi Peter, Venkat, >> >> Thank you for the fix. It looks good to me. Improved performance is a >> nice bonus! Would you be able to provide with a regression test? > > Sure, here it is: > > http://cr.openjdk.java.net/~plevart/jdk10-dev/8191216_SimpleTimeZone_clone_race/webrev.02/ > > > The test reliably detects several races in 2 seconds of runtime which > cause incorrect offset calculations in JDK 9: > > shared TZ: 7363986 correct, 0 incorrect calculations > cloned TZ: 3960264 correct, 1180 incorrect calculations > Exception in thread "main" java.lang.AssertionError: 1180 fatal data > races detected > ??? at > SimpleTimeZoneCloneRaceTest.main(SimpleTimeZoneCloneRaceTest.java:86) > > With patch applied, there are no failures of course. > > Can this go into JDK 10 ? > > Regards, Peter > >> >> Naoto >> >> On 12/6/17 6:10 AM, Peter Levart wrote: >>> Hi, >>> >>> On 12/06/2017 02:30 PM, Alan Bateman wrote: >>>> I think this class is normally maintained on i18n-dev but I think >>>> introducing the Cache object looks good and making this much easier >>>> to understand. >>>> >>>> -Alan >>> >>> Thanks Alan, I'm forwarding to i18n-dev to see if maintainers of that >>> part of JDK have any comments... >>> >>> This is an official request for reviewing a patch for fixing a data >>> race between cloning a SimpleTimeZone object and lazily initializing >>> its 3 cache fields which may produce a clone with inconsistent cache >>> state. Here's Jira issue with details: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8191216 >>> >>> Venkat has proposed to simply call invalidateCache() on the clone >>> before returning it from SimpleTimeZone.clone() method: >>> >>> ???? public Object clone() >>> ???? { >>> ???????? SimpleTimeZone clone = (SimpleTimeZone) super.clone(); >>> ???????? clone.invalidateCache(); >>> ???????? return clone; >>> ???? } >>> >>> This fixes the issue and for the case of TimeZone.getDefault() which >>> is called from ZoneId.systemDefault() even elides synchronization in >>> clone.invalidateCache() and allocation of the clone, so JITed >>> ZoneId.systemDefault() is unaffected by the patch. Initially I was >>> satisfied with the fix, but then I tested something. Suppose someone >>> sets default zone to SimpleTimeZone: >>> >>> ???????? TimeZone.setDefault( >>> ???????????? new SimpleTimeZone(3600000, >>> ??????????????????????????????? "Europe/Paris", >>> ??????????????????????????????? Calendar.MARCH, -1, Calendar.SUNDAY, >>> ??????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, >>> ??????????????????????????????? Calendar.OCTOBER, -1, Calendar.SUNDAY, >>> ??????????????????????????????? 3600000, SimpleTimeZone.UTC_TIME, >>> ??????????????????????????????? 3600000) >>> ???????? ); >>> >>> And then calls some methods that initialize the cache of the internal >>> shared TimeZone.defaultTimeZone instance: >>> >>> ???? new Date().toString(); >>> >>> The code which after that tries to obtain default time zone and >>> calculate the offset from UTC at some current point in time, for >>> example: >>> >>> ???? TimeZone.getDefault().getOffset(now) >>> >>> can't use the cached state because it has been invalidated in the >>> returned clone. Such code has to re-compute the offset every time it >>> gets new clone. I measured this with a JMH benchmark and got the >>> following result: >>> >>> Default: >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040 >>> ns/op >>> >>> Venkat's patch - invalidateCache(): >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 179,476 ? 1,942 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,538 ? 0,073 >>> ns/op >>> >>> We see that ZoneId.systemDefault() is unaffected, but >>> TimeZone.getDefault().getOffset(now) becomes 3x slower. >>> >>> This is not good, so I tried an alternative fix for the issue - >>> simply making the SimpleTimeZone.clone() synchronized. Access to >>> cache fields is already synchronized, so this should fix the race. >>> Here's the JMH result: >>> >>> Default: >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040 >>> ns/op >>> >>> Synchronized clone(): >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 58,103 ? 0,936 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 4,154 ? 0,034 >>> ns/op >>> >>> We see that caching works again, but synchronization has some >>> overhead which is not big for single-threaded access, but might get >>> bigger when multiple threads try to get default zone concurrently. >>> >>> So I created a 3rd variant of the fix which I'm presenting here and >>> requesting a review for: >>> >>> http://cr.openjdk.java.net/~plevart/jdk10-dev/8191216_SimpleTimeZone_clone_race/webrev.01/ >>> >>> >>> The JMH benchmark shows this: >>> >>> Default: >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 57,168 ? 0,501 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,558 ? 0,040 >>> ns/op >>> >>> Cache object: >>> >>> Benchmark????????????????????????????????? Mode? Cnt Score Error? Units >>> ZoneIdBench.TimeZone_getDefault_getOffset? avgt?? 10 42,860 ? 0,274 >>> ns/op >>> ZoneIdBench.ZoneId_systemDefault?????????? avgt?? 10 3,545 ? 0,057 >>> ns/op >>> >>> Not only does the fix not affect ZoneId.systemDefault() which is not >>> surprising, but it also speeds-up cache lookup in single-threaded >>> benchmark and certainly eliminates possible contention among threads >>> looking up the shared instance. >>> >>> I have run the test/jdk/java/util/TimeZone and >>> test/jdk/java/util/Calendar jtreg tests and there were 7 failures >>> caused by compilation errors (package sun.util.locale.provider is not >>> visible), while 59 other tests pass. >>> >>> So, what do you think? >>> >>> Regards, Peter >>> >>> >>> Venkateswara R Chintala je 21. 11. 2017 ob 10:14?napisal: >>>> Thanks Peter for sponsoring this patch. Is there anything else that >>>> needs to be done from my end for this patch to be integrated? Please >>>> let me know. >>>> >>>> -Venkat >>>> >>>> >>>> On 14/11/17 8:46 PM, Peter Levart wrote: >>>>> Hi Venkat, >>>>> >>>>> I created the following issue: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8191216 >>>>> >>>>> I can also sponsor this patch and push it for JDK 10. >>>>> >>>>> The patch that you are proposing looks good to me. Does anybody >>>>> have anything else to say? >>>>> >>>>> Regards, Peter >>>>> >>>>> >>>>> On 11/13/2017 11:28 AM, Venkateswara R Chintala wrote: >>>>>> Thanks David, Peter for your review and comments. As I am new to >>>>>> the community, can you please help me to open a bug and integrate >>>>>> the changes into code base? >>>>>> >>>>>> -Venkat >>>>>> >>>>>> On 12/11/17 2:19 AM, Peter Levart wrote: >>>>>>> Hi David, Venkat, >>>>>>> >>>>>>> On 11/11/17 21:15, Peter Levart wrote: >>>>>>>> For example, take the following method: >>>>>>>> >>>>>>>> String defaultTZID() { >>>>>>>> ??? return TimeZone.getDefault().getID(); >>>>>>>> } >>>>>>>> >>>>>>>> When JIT compiles it and inlines invocations to other methods >>>>>>>> within it, it can prove that cloned TimeZone instance never >>>>>>>> escapes the call to defaultTZID() and can therefore skip >>>>>>>> allocating the instance on heap. >>>>>>>> >>>>>>>> But this is fragile. If code in invoked methods changes, they >>>>>>>> may not get inlined or EA may not be able to prove that the >>>>>>>> cloned instance can't escape and allocation may be introduced. >>>>>>>> ZoneId.systemDefault() is a hot method and it would be nice if >>>>>>>> we manage to keep it allocation free. >>>>>>> >>>>>>> Well, I tried the following variant of SimpleTimeZone.clone() patch: >>>>>>> >>>>>>> ??? public Object clone() >>>>>>> ??? { >>>>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>>>> ??????? // like tz.invalidateCache() but without holding a lock >>>>>>> on clone >>>>>>> ??????? tz.cacheYear = tz.startYear - 1; >>>>>>> ??????? tz.cacheStart = tz.cacheEnd = 0; >>>>>>> ??????? return tz; >>>>>>> ??? } >>>>>>> >>>>>>> >>>>>>> ...and the JMH benchmark with gc profiling shows that >>>>>>> ZoneId.systemDefault() still manages to get JIT-compiled without >>>>>>> introducing allocation. >>>>>>> >>>>>>> Even the following (original Venkat's) patch: >>>>>>> >>>>>>> ??? public Object clone() >>>>>>> ??? { >>>>>>> ??????? SimpleTimeZone tz = (SimpleTimeZone) super.clone(); >>>>>>> ??????? tz.invalidateCache(); >>>>>>> ??????? return tz; >>>>>>> ??? } >>>>>>> >>>>>>> ...does the same and the locking in invalidateCache() is elided >>>>>>> too. Allocation and lock-elision go hand-in-hand. When object >>>>>>> doesn't escape, allocation on heap may be eliminated and locks on >>>>>>> that instance elided. >>>>>>> >>>>>>> So this is better than synchronizing on the original instance >>>>>>> during .clone() execution as it has potential to avoid locking >>>>>>> overhead. >>>>>>> >>>>>>> So Venkat, go ahead. My fear was unjustified. >>>>>>> >>>>>>> Regards, Peter >>>>>>> >>>>>> >>>>> >>>>> >>>> > From paul.sandoz at oracle.com Mon Dec 11 18:58:40 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 11 Dec 2017 10:58:40 -0800 Subject: [10] RFR 8171826 Comparator.reverseOrder(c) mishandles singleton comparators Message-ID: <5CE0E6E7-A714-41B5-960F-6DBF31D7BAA2@oracle.com> Hi, Please review this simple fix. http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8171826-comparator-equality/webrev/ The method Comparator reverseOrder(Comparator cmp) is updated to be a little smarter returning constant comparators for natural and reverse-natural. Paul. From Roger.Riggs at Oracle.com Mon Dec 11 19:18:33 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 11 Dec 2017 14:18:33 -0500 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: <5A2B2E85.9000105@oracle.com> References: <5A20F2AB.3050608@oracle.com> <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> <5A21BD18.2050201@oracle.com> <34dacec0-7e2f-65db-1262-30c0c4a279c1@oracle.com> <5A2617F2.6030005@oracle.com> <5A2B2E85.9000105@oracle.com> Message-ID: Hi Joe, The new tests using testng look good, a few comments: ?- BAOS/EncodingTest.java ?? 70:? The message gets printed if the condition is not true; so "results do not might" might read better ??? (also, there is Assert.assertEquals(str1, str2, msg) - that compares objects with .equals()) ?? A message in nice, though in many cases the testng generated exception/message is sufficient ?? especially for equals. ?- PrintStream/EncodingTest.java ?? 92: out should not be null; its probably a test bug;? the test should be removed - PrintWriter/EncodingTest.java ??? 93: out should not be null; its probably a test bug;? the test should be removed Thanks, Roger On 12/8/2017 7:29 PM, Joe Wang wrote: > Thanks Roger, Alan for the comments! > > I've updated the webrev accordingly. Here's the current webrev: > http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html > > -Joe > > On 12/4/17, 7:52 PM, Joe Wang wrote: >> >> >> On 12/3/17, 11:34 AM, Alan Bateman wrote: >>> >>> >>> On 01/12/2017 20:35, Joe Wang wrote: >>>> : >>>> >>>>> >>>>> I think URLDecoder.decode methods should have @throws >>>>> IllegalArgumentException. I realize it's implementation specific >>>>> as to whether IAE is thrown with bad input but given that the RI >>>>> does throw IAE then consumers of the API should be prepared for >>>>> it. The @implNote can stay and probably should be copied into the >>>>> legacy decode method too. >>>> >>>> I added @throws IAE. On a 2nd thought, would that give no >>>> flexibility to alternative impls as the general (class) spec had >>>> given? With this addition, it becomes a requirement. >>> If the @throws description starts with "if the implementation ..." >>> then it should be clear that it is optional. >> >> Thanks Alan. I made the change accordingly. >> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html >> >> -Joe >> >>> >>> -Alan From Roger.Riggs at Oracle.com Mon Dec 11 19:21:25 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 11 Dec 2017 14:21:25 -0500 Subject: [10] RFR 8171826 Comparator.reverseOrder(c) mishandles singleton comparators In-Reply-To: <5CE0E6E7-A714-41B5-960F-6DBF31D7BAA2@oracle.com> References: <5CE0E6E7-A714-41B5-960F-6DBF31D7BAA2@oracle.com> Message-ID: +1 On 12/11/2017 1:58 PM, Paul Sandoz wrote: > Hi, > > Please review this simple fix. > > http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8171826-comparator-equality/webrev/ > > The method Comparator reverseOrder(Comparator cmp) is updated to be a little smarter returning constant comparators for natural and reverse-natural. > > Paul. From huizhe.wang at oracle.com Mon Dec 11 20:04:36 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 11 Dec 2017 12:04:36 -0800 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: References: <5A20F2AB.3050608@oracle.com> <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> <5A21BD18.2050201@oracle.com> <34dacec0-7e2f-65db-1262-30c0c4a279c1@oracle.com> <5A2617F2.6030005@oracle.com> <5A2B2E85.9000105@oracle.com> Message-ID: <5A2EE4D4.8070408@oracle.com> On 12/11/17, 11:18 AM, Roger Riggs wrote: > Hi Joe, > > The new tests using testng look good, a few comments: > > - BAOS/EncodingTest.java > 70: The message gets printed if the condition is not true; so > "results do not might" might read better > (also, there is Assert.assertEquals(str1, str2, msg) - that > compares objects with .equals()) > A message in nice, though in many cases the testng generated > exception/message is sufficient > especially for equals. Assert.assertEquals it is! Removed other debug printout. > > - PrintStream/EncodingTest.java > 92: out should not be null; its probably a test bug; the test > should be removed Done. > > - PrintWriter/EncodingTest.java > 93: out should not be null; its probably a test bug; the test > should be removed Done. Updated webrev: http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html Thanks, Joe > > Thanks, Roger > > > On 12/8/2017 7:29 PM, Joe Wang wrote: >> Thanks Roger, Alan for the comments! >> >> I've updated the webrev accordingly. Here's the current webrev: >> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html >> >> -Joe >> >> On 12/4/17, 7:52 PM, Joe Wang wrote: >>> >>> >>> On 12/3/17, 11:34 AM, Alan Bateman wrote: >>>> >>>> >>>> On 01/12/2017 20:35, Joe Wang wrote: >>>>> : >>>>> >>>>>> >>>>>> I think URLDecoder.decode methods should have @throws >>>>>> IllegalArgumentException. I realize it's implementation specific >>>>>> as to whether IAE is thrown with bad input but given that the RI >>>>>> does throw IAE then consumers of the API should be prepared for >>>>>> it. The @implNote can stay and probably should be copied into the >>>>>> legacy decode method too. >>>>> >>>>> I added @throws IAE. On a 2nd thought, would that give no >>>>> flexibility to alternative impls as the general (class) spec had >>>>> given? With this addition, it becomes a requirement. >>>> If the @throws description starts with "if the implementation ..." >>>> then it should be clear that it is optional. >>> >>> Thanks Alan. I made the change accordingly. >>> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html >>> >>> -Joe >>> >>>> >>>> -Alan > From brian.burkhalter at oracle.com Mon Dec 11 20:47:13 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 11 Dec 2017 12:47:13 -0800 Subject: RFR of 8180451: ByteArrayInputStream should override readAllBytes, readNBytes, and transferTo Message-ID: <09F39E3A-76B0-4D36-98F3-78AD246B91A7@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8180451 http://cr.openjdk.java.net/~bpb/8180451/webrev.00 1. Add overrides of readAllBytes(), readNBytes(), and transferTo(). 2. Simplify parameter checks in read(byte[],int,int). 3. Change foo to {@code foo}. Thanks, Brian From brian.burkhalter at oracle.com Mon Dec 11 20:52:07 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 11 Dec 2017 12:52:07 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> Message-ID: <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> On Dec 8, 2017, at 3:12 PM, Brian Burkhalter wrote: > All previous suggested changes have been made in > > http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ > > except for the possible change of name for the IS and OS nullStream() methods which awaits consensus. As long as we?re still at it, given java.util.stream.Stream then InputStream.nullInput() OutputStream.nullOutput() Reader.nullReader() Writer.nullWriter() might not be a bad alternative. Thanks, Brian From naoto.sato at oracle.com Mon Dec 11 21:01:50 2017 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 11 Dec 2017 13:01:50 -0800 Subject: [10]RFR 8190278: ClassCastException is thrown by java.util.Scanner when a NumberFormatProvider is used. In-Reply-To: <04d2ca68-7cb8-e255-6fb1-51cb157e17f5@oracle.com> References: <04d2ca68-7cb8-e255-6fb1-51cb157e17f5@oracle.com> Message-ID: <8c95dd64-7b51-e67c-ed40-f85279717b17@oracle.com> Looks good to me. Naoto On 12/11/17 1:04 AM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8190278 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8190278 > Webrev: http://cr.openjdk.java.net/~nishjain/8190278/webrev.03/ > > Fix: Modified the code to check whether the object returned by > NumberFormat#getNumberInstance(Locale) is a DecimalFormat object, if not, > ? ?? then use the DecimalFormat constructor way to create its object, > where SPI provider implementation is ignored. > > Regards, > Nishit Jain > From huizhe.wang at oracle.com Mon Dec 11 21:11:58 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 11 Dec 2017 13:11:58 -0800 Subject: RFR(JDK10/JAXP) 8190823: Broken link in org/w3c/dom/ls/ Message-ID: <5A2EF49E.5040207@oracle.com> Hi, Please review a doc-only fix for 4 dom classes. The material change is s/broken link/correct link/g, that is replacing all occurrence of the non-existent link "http://www.w3.org/TR/DOM-Level-3-Core/core.html" with https://www.w3.org/TR/2004/REC-DOM-Level-3-Core-20040407/core.html. Other changes are format related. In order to avoid the extra space before the text of the hyperlink (e.g. " error handler"), in most cases, I kept the element in one line although it's unfortunately long. jbs: https://bugs.openjdk.java.net/browse/JDK-8190823 webrev: http://cr.openjdk.java.net/~joehw/jdk10/8190823/webrev/ Thanks, Joe From paul.sandoz at oracle.com Mon Dec 11 22:02:06 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 11 Dec 2017 14:02:06 -0800 Subject: [10] RFR 8187254 Rearrange private MethodType constructors Message-ID: <31DD352E-6A01-41D4-80FA-82BE838FE597@oracle.com> Hi, Please review this small fix to collapse the two private constructors of MethodType into one. This consolidates all the copying/validation logic into the factory method makeImpl. Thanks, Paul. diff -r 77866b9147be src/java.base/share/classes/java/lang/invoke/MethodType.java --- a/src/java.base/share/classes/java/lang/invoke/MethodType.java Mon Dec 11 10:50:17 2017 -0800 +++ b/src/java.base/share/classes/java/lang/invoke/MethodType.java Mon Dec 11 14:00:09 2017 -0800 @@ -105,23 +105,10 @@ private @Stable String methodDescriptor; // cache for toMethodDescriptorString /** - * Check the given parameters for validity and store them into the final fields. + * Constructor that performs no copying or validation. + * Should only be called from the factory method makeImpl */ - private MethodType(Class rtype, Class[] ptypes, boolean trusted) { - checkRtype(rtype); - checkPtypes(ptypes); - this.rtype = rtype; - // defensively copy the array passed in by the user - this.ptypes = trusted ? ptypes : Arrays.copyOf(ptypes, ptypes.length); - } - - /** - * Construct a temporary unchecked instance of MethodType for use only as a key to the intern table. - * Does not check the given parameters for validity, and must discarded (if untrusted) or checked - * (if trusted) after it has been used as a searching key. - * The parameters are reversed for this constructor, so that it is not accidentally used. - */ - private MethodType(Class[] ptypes, Class rtype) { + private MethodType(Class rtype, Class[] ptypes) { this.rtype = rtype; this.ptypes = ptypes; } @@ -308,18 +295,21 @@ if (ptypes.length == 0) { ptypes = NO_PTYPES; trusted = true; } - MethodType primordialMT = new MethodType(ptypes, rtype); + MethodType primordialMT = new MethodType(rtype, ptypes); MethodType mt = internTable.get(primordialMT); if (mt != null) return mt; // promote the object to the Real Thing, and reprobe + MethodType.checkRtype(rtype); if (trusted) { - MethodType.checkRtype(rtype); MethodType.checkPtypes(ptypes); mt = primordialMT; } else { - mt = new MethodType(rtype, ptypes, false); + // Make defensive copy then validate + ptypes = Arrays.copyOf(ptypes, ptypes.length); + MethodType.checkPtypes(ptypes); + mt = new MethodType(rtype, ptypes); } mt.form = MethodTypeForm.findForm(mt); return internTable.add(mt); From mandy.chung at oracle.com Mon Dec 11 21:53:17 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Dec 2017 13:53:17 -0800 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? In-Reply-To: References: Message-ID: I filed https://bugs.openjdk.java.net/browse/JDK-8193325.?? I can sponsor this patch for you. --- a/src/java.base/share/classes/java/lang/StackFrameInfo.java +++ b/src/java.base/share/classes/java/lang/StackFrameInfo.java @@ -93,7 +93,7 @@ ???????? if (isNativeMethod()) ???????????? return -1; -??????? return bci; +??????? return bci & 0xffff; ???? } Mandy On 12/7/17 8:19 PM, David Lloyd wrote: > I was doing some research related to AccessController, and I noted > this code [1] in StackFrameInfo#getByteCodeIndex(): > > @Override > public int getByteCodeIndex() { > // bci not available for native methods > if (isNativeMethod()) > return -1; > > return bci; > } > > Now bci is of type short, and given the spec of the method, should the > return not be: > > return bci & 0xffff; > > instead? Else it would return wrong values for methods with more than > 32767 bytecodes in them. > > [1] http://hg.openjdk.java.net/jdk/jdk/file/e3b6cb90d7ce/src/java.base/share/classes/java/lang/StackFrameInfo.java#l96 > From mandy.chung at oracle.com Mon Dec 11 22:24:13 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Dec 2017 14:24:13 -0800 Subject: [10] RFR 8187254 Rearrange private MethodType constructors In-Reply-To: <31DD352E-6A01-41D4-80FA-82BE838FE597@oracle.com> References: <31DD352E-6A01-41D4-80FA-82BE838FE597@oracle.com> Message-ID: It looks okay to me. Mandy On 12/11/17 2:02 PM, Paul Sandoz wrote: > Hi, > > Please review this small fix to collapse the two private constructors of MethodType into one. This consolidates all the copying/validation logic into the factory method makeImpl. > > Thanks, > Paul. > > diff -r 77866b9147be src/java.base/share/classes/java/lang/invoke/MethodType.java > --- a/src/java.base/share/classes/java/lang/invoke/MethodType.java Mon Dec 11 10:50:17 2017 -0800 > +++ b/src/java.base/share/classes/java/lang/invoke/MethodType.java Mon Dec 11 14:00:09 2017 -0800 > @@ -105,23 +105,10 @@ > private @Stable String methodDescriptor; // cache for toMethodDescriptorString > > /** > - * Check the given parameters for validity and store them into the final fields. > + * Constructor that performs no copying or validation. > + * Should only be called from the factory method makeImpl > */ > - private MethodType(Class rtype, Class[] ptypes, boolean trusted) { > - checkRtype(rtype); > - checkPtypes(ptypes); > - this.rtype = rtype; > - // defensively copy the array passed in by the user > - this.ptypes = trusted ? ptypes : Arrays.copyOf(ptypes, ptypes.length); > - } > - > - /** > - * Construct a temporary unchecked instance of MethodType for use only as a key to the intern table. > - * Does not check the given parameters for validity, and must discarded (if untrusted) or checked > - * (if trusted) after it has been used as a searching key. > - * The parameters are reversed for this constructor, so that it is not accidentally used. > - */ > - private MethodType(Class[] ptypes, Class rtype) { > + private MethodType(Class rtype, Class[] ptypes) { > this.rtype = rtype; > this.ptypes = ptypes; > } > @@ -308,18 +295,21 @@ > if (ptypes.length == 0) { > ptypes = NO_PTYPES; trusted = true; > } > - MethodType primordialMT = new MethodType(ptypes, rtype); > + MethodType primordialMT = new MethodType(rtype, ptypes); > MethodType mt = internTable.get(primordialMT); > if (mt != null) > return mt; > > // promote the object to the Real Thing, and reprobe > + MethodType.checkRtype(rtype); > if (trusted) { > - MethodType.checkRtype(rtype); > MethodType.checkPtypes(ptypes); > mt = primordialMT; > } else { > - mt = new MethodType(rtype, ptypes, false); > + // Make defensive copy then validate > + ptypes = Arrays.copyOf(ptypes, ptypes.length); > + MethodType.checkPtypes(ptypes); > + mt = new MethodType(rtype, ptypes); > } > mt.form = MethodTypeForm.findForm(mt); > return internTable.add(mt); From peter.levart at gmail.com Mon Dec 11 23:46:36 2017 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 12 Dec 2017 00:46:36 +0100 Subject: RFR 8191216: SimpleTimeZone.clone() has a data race on cache fields In-Reply-To: References: <576cd10d-c75e-c7e2-ecb9-e28d09e059cf@linux.vnet.ibm.com> <84ca56e0-67c8-3fbb-d558-4ebe797d513c@oracle.com> <8894f619-3510-acfa-3987-f013ca3638f3@gmail.com> <72a7efe3-ac34-1534-3a9b-146c647e1292@linux.vnet.ibm.com> <017c44ce-a3e2-a03d-d437-98e08655a6a1@gmail.com> <3bf0ce49-cb3d-f70b-717d-03680b001d3b@oracle.com> <9dbdf9bc-e93a-6ac4-d3ae-5168fd365403@oracle.com> <86b95ce3-2acd-8fda-77cf-222f957ac687@oracle.com> <03df6d4c-2392-642f-7e9f-048459bd7cf8@gmail.com> Message-ID: <00b82d4b-5c4d-a32a-4059-807dfcd82b49@gmail.com> Thanks Naoto, Alan, David and Venkat. The change is in. Regards, Peter Naoto Sato je 11. 12. 2017 ob 19:41?napisal: > Hi Peter, > > Thanks for the tests. Looks good to me. > > One nit: it should throw an Exception instead of AssertionError when > the test fails. No further review is needed. > > > Can this go into JDK 10 ? > > You can push it before the JDK 10 fork. > > Naoto > From stuart.marks at oracle.com Tue Dec 12 01:08:13 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 11 Dec 2017 17:08:13 -0800 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Message-ID: On 12/9/17 1:20 AM, Stephen Colebourne wrote: > On 8 December 2017 at 00:33, Stuart Marks wrote: >> Please review this changeset that introduces a new no-arg method >> orElseThrow() to Optional, as a preferred alternative to the get() method. > > I am willing to accept this addition as it has an obvious parallel to > the existing supplier method. > > My position on removing get() has not changed however, and as such I > am displeased with the addition of the "preferred alternative" text. Hi Stephen, Thanks for your comments on this issue. I know that you like get() (or at least you're OK with it). But even three years after the release of Java 8, I still see bad code written with get(), so I don't think this problem is going to go away. Also, I should clarify that it was never the intent to remove get(), just to deprecate it not-for-removal. However, I realize that even deprecation has a cost, which is why I'm not pursuing it as part of this proposal. s'marks From lance.andersen at oracle.com Tue Dec 12 01:49:40 2017 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 11 Dec 2017 20:49:40 -0500 Subject: RFR(JDK10/JAXP) 8190823: Broken link in org/w3c/dom/ls/ In-Reply-To: <5A2EF49E.5040207@oracle.com> References: <5A2EF49E.5040207@oracle.com> Message-ID: looks OK joe Best Lance > On Dec 11, 2017, at 4:11 PM, Joe Wang wrote: > > Hi, > > Please review a doc-only fix for 4 dom classes. The material change is s/broken link/correct link/g, that is replacing all occurrence of the non-existent link "http://www.w3.org/TR/DOM-Level-3-Core/core.html" with https://www.w3.org/TR/2004/REC-DOM-Level-3-Core-20040407/core.html. Other changes are format related. In order to avoid the extra space before the text of the hyperlink (e.g. " error handler"), in most cases, I kept the element in one line although it's unfortunately long. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8190823 > webrev: http://cr.openjdk.java.net/~joehw/jdk10/8190823/webrev/ > > Thanks, > Joe > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From john.r.rose at oracle.com Tue Dec 12 02:00:15 2017 From: john.r.rose at oracle.com (John Rose) Date: Mon, 11 Dec 2017 18:00:15 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: On Dec 7, 2017, at 5:03 PM, Martin Buchholz wrote: > > (I'm still trying to love this new API) > > The changes to the jsr166 tck are fine. > > I'm not convinced that the new implementation for ArrayList is progress. > The current implementation of toArray(T[]) does > > // Make a new array of a's runtime type, but my contents: > return (T[]) Arrays.copyOf(elementData, size, a.getClass()); > > which does not have to pay the cost of zeroing the array, so is potentially > faster. (depends on whether the VM provides cheap pre-zeroed memory?! what > does jmh say?) The VM does not do pre-zeroing. We've done experiments over the years with that and it has never beaten pre-garbaged memory, with zero filling at creation time. The reason is that we can vectorize the zero filling at any time *and* omit zeroing at all in some important cases like copyOf. OTOH, pre-zeroing assumes there is background bandwidth available for pre-processing cache lines that are not yet in use, which isn't always true; sometimes the mutator needs all the cache bandwidth. All of which is to say that the VM is likely *never* to do pre-zeroing, unless there is some special hardware feature that does so, in bulk, in main memory. Which AFAIK doesn't exist today. This particular set of performance problems is (IMO) sensitive to the number of passes over the input and output arrays, as well as over any intermediate arrays. Zeroing isn't a separate pass in the case of copyOf, but in any two phase array creation protocol, zeroing *is* likely to require a separate pass. What do I mean by "two phase"? I mean a scheme where in phase one an empty array is created, and in phase two the array is filled with content. If the two phases are not reliably juxtaposed (as they are in copyOf) the VM must conservatively fill the empty array with zeros. Different code shapes have (inherently) different inlining characteristics. In the case of toArray(IntFunction), three things must happen to drop the zeroing pass (as in Arrays.copyOf): First, the particular IntFunction must fully inline (that's phase one), and second, the (phase two) loop that populates the new array must be juxtaposed with the inlined code of the IntFunction call, without intermediate logic that might discourage the optimizer. Third, both code shapes must use intrinsics that are recognized by the optimizer. These three conditions are carefully engineered in the body of Arrays.copyOf, but appear "emergently" ("accidentally") via the toArray(IF) API. So the optimization isn't impossible, but it is subject to more risk, and/or will require more optimizer work to support that code shape. Bottom line: Arrays.copyOf is not as arbitrary as one may think; it is designed that way because it is easy to optimize. And it covers two interesting use cases in the present discussion, which are (a) copy my internal array wholesale, and (b) make me a fresh array from a zero-length exemplar. > If we're not getting type safety and not getting significantly better > performance, all we have is a more natural API. But toArray(IntFunction) > also seems not very natural to me, Whether or not it bears on the present API discussion, I think it would be fruitful to step back and ask ourselves the big picture question here, "What is the natural shape of an array factory?" I submit to you that such a factory is *not* an IntFunction, because that only creates half of an array (or 0.01% of one), the empty version that needs to be populated. A natural array factory API will provide two pieces of data: First a size (an accurate, not a provisional one like List.size), and second a series of values to put in the array. *That* structure is the natural, might I say categorical, argument to an array factory. Also a List factory. As you can probably tell, I'm taking the value-based view of things here. (Proof by generalization: If/when we eventually create frozen arrays, and flattened arrays, and volatile arrays, and hybrid instance-with-sized-tail arrays, it will not always be an option to create the empty array in a first phase, and then fill it from the outside. Frozen arrays are the obvious counterexample, but any immutable data structure will do, as will a private object tail, even if mutable.) The interface would look something like one of these: interface ArrayContent { int size(); T get(int i); } interface ArrayContent { int size(); Iterator iterator(); } interface ArrayContent { int count(); Stream stream(); } interface ArrayContent { Spliterator elements(); } //must be SIZED That last suggests that we don't need a new API at all, that perhaps the natural input to an array factory is just a Spliterator with the SIZED characteristic, and a range check on the estimateSize() result. Or maybe a Stream. There are obvious bootstrapping issues here, but I don't think they are complex, because the source object can create a tiny Stream or Spliterator or ArrayContent that provides a trivial view on that source object's internals. For ArrayList (or its sublists) it would be enough to return a three-field Spliterator thingy that peeks at a left-edge shrinking slice of an arbitrary T[] array. It's a little like what Claes and I were discussing for List.of (and that is a related problem). Because of its categoricity, the above ArrayContent thingy is also useful for creating things which aren't literally arrays but look enough like them (have the same contents). We could design an API that subsumes and generalizes both toArray and asList, by allowing the lambda to specify what sort of aggregate to produce: interface Collection { /** Produces a copy of the sequence of values in this collection. */ C copy(SequenceCopier copier); } // SequenceCopier == Function, C> // SequenceContent == ArrayContent or Stream or Spliterator The idea is that every major sequence-like type can provide its own "copier" factory function which will take a standard SequenceContent and package it up. And every collection or stream (or sequence) would also provide a copy operation which builds an internally appropriate SequenceContent cursor and calls the user-supplied factory method, as "return copier.apply(myContent);". class Arrays { // people who want x.toArray can also use x.copy(Arrays::make): static T[] make(SequenceContent content) { ? } // ArrayList.copy uses this one internally as "myContent": static SequenceContent slice(T[] a, int start, int end) { ? } } class List { // people who want x.asUnmodifiableList can also use x.copy(List::make): default SequenceCopier> make(SequenceContent content) { ? } // external access to a sized snapshot of a list: default SequenceContent sizedContent() { ? } } For my money, *that's* the natural API point we should be looking for. (Hence it's not necessarily relevant to 8060192.) It's a nice little three-way dance between a source collection, a lightweight snapshot of its contents, and a lambda that creates a destination value. There are enough small bikesheds here to keep us trying on new colors until spring, but I think that goes with the territory: There are three things that fit together here, and they all need nice names. To repeat: These points are in response to Martin's musing about a "natural" factory API for arrays, and don't necessarily apply to this RFE. I *do* think that the two-phase array creation protocol (first create empty, then fill later) is fundamentally weak, both in its lack of generality and its optimizability, but there may be reasons I'm unaware of why people would love to have more stuff in that vein. > and we'll have to live with the > toArray(new String[0]) wart forever anyways. (But is it really so bad?) > (Maybe toArray(Class) is actually more natural?) (I wish I had this API point: T[] Class.emptyArray() which would cache the darn empty array. It's T[] so it has to throw on primitives until we get Valhalla generics. I have written many a private-static-final to hold an empty array for this sort of thing. Would make toArray more tolerable IMO. In other dreams, empty varargs lists would use this array as well.) ? John From stuart.marks at oracle.com Tue Dec 12 02:02:24 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 11 Dec 2017 18:02:24 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> <3a77f8c0-0c8b-3735-a0a6-f39e2d0f5eb9@oracle.com> Message-ID: <909a2e6b-b8e3-ef2f-0da2-5b57c9d17fe1@oracle.com> > Frankly, I think the JDK is just sloppy here. Most of the existing > implementations didn't use @Override -- possibly because they were > introduced before annotations existed -- and later on, whoever implemented > the overrides of the Java 8 default methods decided to add @Override. > > Part of it is that Doug and I are not huge fans of @Override.? Is the gain in > safety really worth the added verbosity? > > If we do decide to mandate the use of @Override in all of openjdk, then the > errorprone project has a bug pattern check for you, which will provide more > safety yet. As I said, I don't think there's a rule that requires @Override. Thanks for verifying this. :-) I'm not proposing adding such a rule either. I agree, it adds little benefit, at least for the JDK, and it adds clutter. A quick grep reveals over 33,000 occurrences of @Override in the JDK sources. That seems like a lot, but if we were to add it to all locations where it could be applied, this might add hundreds if not thousands more occurrences. I'm not sure it's worth the effort to do either the additions or the removals that'd be necessary to make things consistent. Oh well. s'marks From john.r.rose at oracle.com Tue Dec 12 02:17:21 2017 From: john.r.rose at oracle.com (John Rose) Date: Mon, 11 Dec 2017 18:17:21 -0800 Subject: [10] RFR 8187254 Rearrange private MethodType constructors In-Reply-To: <31DD352E-6A01-41D4-80FA-82BE838FE597@oracle.com> References: <31DD352E-6A01-41D4-80FA-82BE838FE597@oracle.com> Message-ID: Good. On Dec 11, 2017, at 2:02 PM, Paul Sandoz wrote: > > Please review this small fix to collapse the two private constructors of MethodType into one. This consolidates all the copying/validation logic into the factory method makeImpl. From john.r.rose at oracle.com Tue Dec 12 02:07:06 2017 From: john.r.rose at oracle.com (John Rose) Date: Mon, 11 Dec 2017 18:07:06 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: <3F4561D8-34B5-4663-B13A-E2B35E6279F8@oracle.com> On Dec 11, 2017, at 6:00 PM, John Rose wrote: > > class Arrays { > // people who want x.toArray can also use x.copy(Arrays::make): > static T[] make(SequenceContent content) { ? } Correction; that one needs an array type to know what to make: static T[] make(SequenceContent content, Class arrayClass) { ? } > // ArrayList.copy uses this one internally as "myContent": > static SequenceContent slice(T[] a, int start, int end) { ? } > } > class List { > // people who want x.asUnmodifiableList can also use x.copy(List::make): > default SequenceCopier> make(SequenceContent content) { ? } > > // external access to a sized snapshot of a list: > default SequenceContent sizedContent() { ? } > } Also, sizedContent is there for completeness but doesn't carry as much weight as the other three. It might be derived from Collection::copy applied to the identity function, unless there are scoping rules on the operand to Collection::copy. From martinrb at google.com Tue Dec 12 02:31:33 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 11 Dec 2017 18:31:33 -0800 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? In-Reply-To: References: Message-ID: Java has an unsigned 16 bit type. Could bci be of type "char" ? On Mon, Dec 11, 2017 at 1:53 PM, mandy chung wrote: > I filed https://bugs.openjdk.java.net/browse/JDK-8193325. I can sponsor > this patch for you. > > --- a/src/java.base/share/classes/java/lang/StackFrameInfo.java > +++ b/src/java.base/share/classes/java/lang/StackFrameInfo.java > @@ -93,7 +93,7 @@ > if (isNativeMethod()) > return -1; > > - return bci; > + return bci & 0xffff; > } > > Mandy > > > On 12/7/17 8:19 PM, David Lloyd wrote: > >> I was doing some research related to AccessController, and I noted >> this code [1] in StackFrameInfo#getByteCodeIndex(): >> >> @Override >> public int getByteCodeIndex() { >> // bci not available for native methods >> if (isNativeMethod()) >> return -1; >> >> return bci; >> } >> >> Now bci is of type short, and given the spec of the method, should the >> return not be: >> >> return bci & 0xffff; >> >> instead? Else it would return wrong values for methods with more than >> 32767 bytecodes in them. >> >> [1] http://hg.openjdk.java.net/jdk/jdk/file/e3b6cb90d7ce/src/jav >> a.base/share/classes/java/lang/StackFrameInfo.java#l96 >> >> > From mandy.chung at oracle.com Tue Dec 12 03:03:51 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Dec 2017 19:03:51 -0800 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? In-Reply-To: References: Message-ID: <1883e507-312c-e1c6-d7f9-6c9a4bc2fafb@oracle.com> On 12/11/17 6:31 PM, Martin Buchholz wrote: > Java has an unsigned 16 bit type.? Could bci be of type "char" ? > Yes but I think keeping it short is fine. Mandy From huizhe.wang at oracle.com Tue Dec 12 04:02:34 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 11 Dec 2017 20:02:34 -0800 Subject: RFR(JDK10/JAXP) 8190823: Broken link in org/w3c/dom/ls/ In-Reply-To: References: <5A2EF49E.5040207@oracle.com> Message-ID: <5A2F54DA.4090509@oracle.com> Thanks Lance! On 12/11/17, 5:49 PM, Lance Andersen wrote: > looks OK joe > > Best > Lance >> On Dec 11, 2017, at 4:11 PM, Joe Wang > > wrote: >> >> Hi, >> >> Please review a doc-only fix for 4 dom classes. The material change >> is s/broken link/correct link/g, that is replacing all occurrence of >> the non-existent link >> "http://www.w3.org/TR/DOM-Level-3-Core/core.html" with >> https://www.w3.org/TR/2004/REC-DOM-Level-3-Core-20040407/core.html. >> Other changes are format related. In order to avoid the extra space >> before the text of the hyperlink (e.g. " error handler"), in most >> cases, I kept the element in one line although it's unfortunately >> long. >> >> jbs: https://bugs.openjdk.java.net/browse/JDK-8190823 >> webrev: http://cr.openjdk.java.net/~joehw/jdk10/8190823/webrev/ >> >> >> Thanks, >> Joe >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From sshamaia at in.ibm.com Tue Dec 12 10:22:19 2017 From: sshamaia at in.ibm.com (Srividya Shamaiah) Date: Tue, 12 Dec 2017 15:52:19 +0530 Subject: JDK-8190919 :StaX buffers do not really close underlying buffers Message-ID: Joe, Thanks you for your response. I agree with you that the specification says the underlying stream should not be closed. However, it is not clear what was the reason for decided not to close the underlying stream. Also, it is not evident at what point the underlying streams will get closed and whether it will result in a resource leak. We understand not to close the stream if it is shared by multiple threads. However, there should be a logic like reference count which can decide the point at which the stream can be closed. However, we are not finding any such mechanism in this area of code and hence worried about the resource leak. Please let me know how we will ensure that the underlying stream is not getting leaked. With Thanks and Regards, Srividya S From goetz.lindenmaier at sap.com Tue Dec 12 10:36:51 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 12 Dec 2017 10:36:51 +0000 Subject: FW: RFR(M): 8189102: All tools should support -?, -h and --help In-Reply-To: <6d7d895f-9f23-fd4b-39ca-ef9bb0331fb9@oracle.com> References: <8aa25976d496465fac00a4ae3628e533@sap.com> <456c694ec25a42658af1b8600aaa136d@sap.com> <6d7d895f-9f23-fd4b-39ca-ef9bb0331fb9@oracle.com> Message-ID: <7be3772e8060494cbecf3097bd0a8a85@sap.com> Hi Alan, Javadoc combines documentation and support of a flag in the way the flag handling is implemented. On the other side, it prints the help message anyways if a wrong flag is presented to it, so if you call it with -help you get the help message. Therefore, in my original change where I tried to get it more cleaned up, I removed -help support and documentation from Javadoc. I added it again and updated the table in the CSR: http://cr.openjdk.java.net/~goetz/wr17/8189102-helpMessage/webrev.06/ Best regards, Goetz. > -----Original Message----- > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Sent: Montag, 11. Dezember 2017 17:53 > To: Lindenmaier, Goetz ; core-libs- > dev at openjdk.java.net; 'compiler-dev at openjdk.java.net' dev at openjdk.java.net>; serviceability-dev (serviceability- > dev at openjdk.java.net) > Subject: Re: FW: RFR(M): 8189102: All tools should support -?, -h and --help > > > > On 07/12/2017 11:20, Lindenmaier, Goetz wrote: > > Hi, > > > > ... missed some lists in my first post ... > > > > I prepared a fifth webrev for this change. Please review. > > > > It incorporates the changes requested by the CSR reviewers > > (not to remove docuemtation of '-help' where is was documented > > before) and the changes proposed by Kumar: > > http://cr.openjdk.java.net/~goetz/wr17/8189102- > helpMessage/webrev.05/ > > > > > Looks like it still drops -help from the javadoc usage message, I can't > tell if you meant to do that. > > -Alan. From david.lloyd at redhat.com Tue Dec 12 13:44:22 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 12 Dec 2017 07:44:22 -0600 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? In-Reply-To: References: Message-ID: Thanks! On Mon, Dec 11, 2017 at 3:53 PM, mandy chung wrote: > I filed https://bugs.openjdk.java.net/browse/JDK-8193325. I can sponsor > this patch for you. > > --- a/src/java.base/share/classes/java/lang/StackFrameInfo.java > +++ b/src/java.base/share/classes/java/lang/StackFrameInfo.java > @@ -93,7 +93,7 @@ > if (isNativeMethod()) > return -1; > > - return bci; > + return bci & 0xffff; > } > > Mandy > > On 12/7/17 8:19 PM, David Lloyd wrote: > > I was doing some research related to AccessController, and I noted > this code [1] in StackFrameInfo#getByteCodeIndex(): > > @Override > public int getByteCodeIndex() { > // bci not available for native methods > if (isNativeMethod()) > return -1; > > return bci; > } > > Now bci is of type short, and given the spec of the method, should the > return not be: > > return bci & 0xffff; > > instead? Else it would return wrong values for methods with more than > 32767 bytecodes in them. > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/e3b6cb90d7ce/src/java.base/share/classes/java/lang/StackFrameInfo.java#l96 > > -- - DML From david.lloyd at redhat.com Tue Dec 12 13:45:38 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 12 Dec 2017 07:45:38 -0600 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? In-Reply-To: <1883e507-312c-e1c6-d7f9-6c9a4bc2fafb@oracle.com> References: <1883e507-312c-e1c6-d7f9-6c9a4bc2fafb@oracle.com> Message-ID: On Mon, Dec 11, 2017 at 9:03 PM, mandy chung wrote: > On 12/11/17 6:31 PM, Martin Buchholz wrote: >> Java has an unsigned 16 bit type. Could bci be of type "char" ? > > Yes but I think keeping it short is fine. Call me strange if you like, but I never liked using char as anything other than a UTF-16 character. Anyway one expects that the '0xffff' forms should be optimized fairly easily. -- - DML From vitalyd at gmail.com Tue Dec 12 14:11:06 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 12 Dec 2017 09:11:06 -0500 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? In-Reply-To: References: <1883e507-312c-e1c6-d7f9-6c9a4bc2fafb@oracle.com> Message-ID: On Tue, Dec 12, 2017 at 8:45 AM, David Lloyd wrote: > On Mon, Dec 11, 2017 at 9:03 PM, mandy chung > wrote: > > On 12/11/17 6:31 PM, Martin Buchholz wrote: > >> Java has an unsigned 16 bit type. Could bci be of type "char" ? > > > > Yes but I think keeping it short is fine. > > Call me strange if you like, but I never liked using char as anything > other than a UTF-16 character. Anyway one expects that the '0xffff' > forms should be optimized fairly easily. > It's optimized - just changes the width of the load. There's Short::toUnsignedInt (added in Java 8) if you want code reuse :). > > > -- > - DML > From david.lloyd at redhat.com Tue Dec 12 14:18:38 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 12 Dec 2017 08:18:38 -0600 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: On Mon, Dec 11, 2017 at 8:00 PM, John Rose wrote: > I submit to you that such a factory is *not* an IntFunction, because > that only creates half of an array (or 0.01% of one), the empty > version that needs to be populated. A natural array factory API > [...] > The interface would look something like one of these: > > interface ArrayContent { int size(); T get(int i); } > interface ArrayContent { int size(); Iterator iterator(); } > interface ArrayContent { int count(); Stream stream(); } > interface ArrayContent { Spliterator elements(); } //must be SIZED Wasn't it suggested somewhat recently that arrays themselves might be retrofitted with this kind of interface? It seems like it would make sense for arrays to be able act as their own factories, bringing the idea full circle. -- - DML From joaovarandas at inpaas.com Tue Dec 12 14:20:29 2017 From: joaovarandas at inpaas.com (=?UTF-8?Q?Jo=C3=A3o_Paulo_Varandas?=) Date: Tue, 12 Dec 2017 12:20:29 -0200 Subject: Any news from JDK-8151981 in Java8 ? Message-ID: Hi guys! We are also experiencing some odd issues here with setCallSiteTargetNormal with Java8. https://bugs.openjdk.java.net/browse/JDK-8151981 Our cenario is: - Web Application using Tomcat; - 8 HTTP Threads; - A single ScriptEngine for the whole application; When a request hits the server, that request may be processed by a JavaScript (using Nashorn), this script is evaluated during runtime, it may run queries(Jdbc) or execute business logic and then return results(Maps and/or Collections) that are serialized to JSON before being written to the response. The problem is... sometimes 7 or more threads are getting stuck in the setCallSiteTargetNormal method and I have no clue why is that happening. Could you guys help me out troubleshooting this? - Or, if there's a bug are there workarounds? By the way, those are not freshly created ScriptEngines(which differ from the normal issues related in the internet). -- "Esta mensagem, incluindo seus anexos, pode conter informacoes confidenciais e privilegiadas. Se voce a recebeu por engano, solicitamos que a apague e avise o remetente imediatamente. Opinioes ou informacoes aqui contidas nao refletem necessariamente a posicao oficial da Plusoft." "Antes de imprimir, pense em sua responsabilidade e compromisso com o MEIO AMBIENTE" From Roger.Riggs at Oracle.com Tue Dec 12 16:42:08 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 12 Dec 2017 11:42:08 -0500 Subject: [10]RFR 8190278: ClassCastException is thrown by java.util.Scanner when a NumberFormatProvider is used. In-Reply-To: <8c95dd64-7b51-e67c-ed40-f85279717b17@oracle.com> References: <04d2ca68-7cb8-e255-6fb1-51cb157e17f5@oracle.com> <8c95dd64-7b51-e67c-ed40-f85279717b17@oracle.com> Message-ID: <48ecc2cd-55fa-04e2-6ae7-e3ffdcf481b7@Oracle.com> Hi Nishit, Looks fine to me Regards, Roger On 12/11/2017 4:01 PM, Naoto Sato wrote: > Looks good to me. > > Naoto > > On 12/11/17 1:04 AM, Nishit Jain wrote: >> Hi, >> >> Please review the fix for JDK-8190278 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8190278 >> Webrev: http://cr.openjdk.java.net/~nishjain/8190278/webrev.03/ >> >> Fix: Modified the code to check whether the object returned by >> NumberFormat#getNumberInstance(Locale) is a DecimalFormat object, if >> not, >> ?? ?? then use the DecimalFormat constructor way to create its >> object, where SPI provider implementation is ignored. >> >> Regards, >> Nishit Jain >> From Roger.Riggs at Oracle.com Tue Dec 12 16:47:55 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 12 Dec 2017 11:47:55 -0500 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: <5A2EE4D4.8070408@oracle.com> References: <5A20F2AB.3050608@oracle.com> <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> <5A21BD18.2050201@oracle.com> <34dacec0-7e2f-65db-1262-30c0c4a279c1@oracle.com> <5A2617F2.6030005@oracle.com> <5A2B2E85.9000105@oracle.com> <5A2EE4D4.8070408@oracle.com> Message-ID: <10a7ad07-05c3-26da-7c7a-8226dcf5a7bb@Oracle.com> Hi Joe, Looks good;? +1 Regards, Roger On 12/11/2017 3:04 PM, Joe Wang wrote: > > On 12/11/17, 11:18 AM, Roger Riggs wrote: >> Hi Joe, >> >> The new tests using testng look good, a few comments: >> >> ?- BAOS/EncodingTest.java >> ?? 70:? The message gets printed if the condition is not true; so >> "results do not might" might read better >> ??? (also, there is Assert.assertEquals(str1, str2, msg) - that >> compares objects with .equals()) >> ?? A message in nice, though in many cases the testng generated >> exception/message is sufficient >> ?? especially for equals. > > Assert.assertEquals it is! Removed other debug printout. > >> >> ?- PrintStream/EncodingTest.java >> ?? 92: out should not be null; its probably a test bug;? the test >> should be removed > > Done. >> >> - PrintWriter/EncodingTest.java >> ??? 93: out should not be null; its probably a test bug;? the test >> should be removed > > Done. > > Updated webrev: > http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html > > Thanks, > Joe > >> >> Thanks, Roger >> >> >> On 12/8/2017 7:29 PM, Joe Wang wrote: >>> Thanks Roger, Alan for the comments! >>> >>> I've updated the webrev accordingly. Here's the current webrev: >>> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html >>> >>> -Joe >>> >>> On 12/4/17, 7:52 PM, Joe Wang wrote: >>>> >>>> >>>> On 12/3/17, 11:34 AM, Alan Bateman wrote: >>>>> >>>>> >>>>> On 01/12/2017 20:35, Joe Wang wrote: >>>>>> : >>>>>> >>>>>>> >>>>>>> I think URLDecoder.decode methods should have @throws >>>>>>> IllegalArgumentException. I realize it's implementation specific >>>>>>> as to whether IAE is thrown with bad input but given that the RI >>>>>>> does throw IAE then consumers of the API should be prepared for >>>>>>> it. The @implNote can stay and probably should be copied into >>>>>>> the legacy decode method too. >>>>>> >>>>>> I added @throws IAE. On a 2nd thought, would that give no >>>>>> flexibility to alternative impls as the general (class) spec had >>>>>> given? With this addition, it becomes a requirement. >>>>> If the @throws description starts with "if the implementation ..." >>>>> then it should be clear that it is optional. >>>> >>>> Thanks Alan. I made the change accordingly. >>>> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html >>>> >>>> -Joe >>>> >>>>> >>>>> -Alan >> From claes.redestad at oracle.com Tue Dec 12 17:58:31 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 12 Dec 2017 18:58:31 +0100 Subject: RFR: JDK-8184947:,ZipCoder performance improvements In-Reply-To: <5A2E04D6.7000708@oracle.com> References: <5A2B1BAB.7010102@oracle.com> <0ced2d86-f2bc-54d2-bbfa-a8f295d69127@oracle.com> <5A2E04D6.7000708@oracle.com> Message-ID: <510b8374-c26c-7c9b-cfc2-e9717df8bd3a@oracle.com> Hi Sherman, On 2017-12-11 05:08, Xueming Shen wrote: > http://cr.openjdk.java.net/~sherman/8184947/webrev thanks for incorporating my suggestion! It looks pretty good to me.? While many parts is just code that has been moved, this is still a pretty big change, so I hope we can get at least another pair of eyes on it. StringCoding.java: private static void throwMalformed(int nb) { throw new IllegalArgumentException("malformed input length : " + nb); } nb is the number of bytes of the *first* offending chunk of bytes(?); is this information generally useful? I think the error message can be improved as input length is ambiguous in this context, but I wouldn't mind if the nb parameter was dropped along with a simplification of the error message. Indentation errors at lines 321, 324, 728, 746, 913 TestStringCoding.java: Are the added System.out.println's intentional? Indentation. > > Yes, understood the threadlocal might not be the best choice here. But > just feel > something need to be done for the temporary Result object after > observed its > usage with the jfr in #8184947. It is taking as many spaces as the > overall String > objects do. Sure, it's in young-gen, should be wiped with quickly. But > my take is > it might be worth the tradeoff of having each/every new String/cs) get > a little slower > instead of having the "global" vm has to do some extra clean up for > this extra > Result object, for now. Right, as this can be cause for significant GC pressure then I see how a bit of ThreadLocal overhead is fine until we come up with something better. Thanks! /Claes > > thanks, > sherman > > From daniel.fuchs at oracle.com Tue Dec 12 18:13:04 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 12 Dec 2017 18:13:04 +0000 Subject: Review Request: JDK-8193192: jdeps --generate-module-info does not look at module path In-Reply-To: References: Message-ID: <6eaa7162-121a-832f-490a-69f0199559ce@oracle.com> Hi Mandy, I have seen nothing obviously wrong with this patch. Using tokens instead of a bunch of boolean variables is a nice simplification IMO. best regards, -- daniel On 07/12/2017 17:17, mandy chung wrote: > jdeps currently finds modules from the module path for analysis based on > the value specified via --add-modules option.? When the input classes > depend on a module on the given module path, it will report missing > dependences until the module is specified in --add-modules option.? This > fixes this issue and jdeps should look at the modules on the module path > when analyzing classes (except jdeps -m option). > > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8193192/webrev.00 > > thanks > Mandy From xueming.shen at oracle.com Tue Dec 12 18:55:34 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 12 Dec 2017 10:55:34 -0800 Subject: RFR: JDK-8184947:,ZipCoder performance improvements In-Reply-To: <510b8374-c26c-7c9b-cfc2-e9717df8bd3a@oracle.com> References: <5A2B1BAB.7010102@oracle.com> <0ced2d86-f2bc-54d2-bbfa-a8f295d69127@oracle.com> <5A2E04D6.7000708@oracle.com> <510b8374-c26c-7c9b-cfc2-e9717df8bd3a@oracle.com> Message-ID: <5A302626.3020908@oracle.com> On 12/12/2017 09:58 AM, Claes Redestad wrote: > > StringCoding.java: > > private static void throwMalformed(int nb) { > throw new IllegalArgumentException("malformed input length : " + nb); > } > > nb is the number of bytes of the *first* offending chunk of bytes(?); is this information > generally useful? I think the error message can be improved as input length is > ambiguous in this context, but I wouldn't mind if the nb parameter was dropped along > with a simplification of the error message. > It's the "copy/paste" of what MalformedInputException does for the toString(). It would/might be a help if with a "offset" info. I'm adding the offset info here with the hope it might help debugging. > Indentation errors at lines 321, 324, 728, 746, 913 > updated. > TestStringCoding.java: > > Are the added System.out.println's intentional? Indentation. that's leftover of the debugging. removed. webrev has been updated according. I believe Martin is also reviewing. Thanks! Sherman From martinrb at google.com Tue Dec 12 19:08:15 2017 From: martinrb at google.com (Martin Buchholz) Date: Tue, 12 Dec 2017 11:08:15 -0800 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? In-Reply-To: References: Message-ID: Out of curiosity, I looked more at StackFrameInfo, and saw: short bci is final and is only ever assigned to -1 in java code. What's up with that? Ohh, it seems to be magically frobbed in hotspot: void java_lang_StackFrameInfo::set_bci(oop element, int value) { element->int_field_put(_bci_offset, value); } BUT: bci should not be final if hotspot magic is frobbing it. Can we add a comment explaining the magic? Also set_bci seems to disagree about the type of bci - it's trying to set an int field, but bci is a short! Also, I expected "int value" above to be "jint value" - it's a java type! I would prefer bci to be a java int, not short, so that it can hold all possible bytecode index values in a natural way, together with -1 for "invalid". Right now, -1 is ambiguous - it might mean bci == 65535. Also, I see private final byte RETAIN_CLASS_REF = 0x01; Shouldn't that be static? From huizhe.wang at oracle.com Tue Dec 12 19:12:08 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 12 Dec 2017 11:12:08 -0800 Subject: RFR (JDK10) 8183743: Umbrella: add overloads that take a Charset parameter In-Reply-To: <10a7ad07-05c3-26da-7c7a-8226dcf5a7bb@Oracle.com> References: <5A20F2AB.3050608@oracle.com> <99589407-1dcd-4da3-dcc5-84b10594aa5d@oracle.com> <5A21BD18.2050201@oracle.com> <34dacec0-7e2f-65db-1262-30c0c4a279c1@oracle.com> <5A2617F2.6030005@oracle.com> <5A2B2E85.9000105@oracle.com> <5A2EE4D4.8070408@oracle.com> <10a7ad07-05c3-26da-7c7a-8226dcf5a7bb@Oracle.com> Message-ID: <5A302A08.7010101@oracle.com> Thanks Roger, Alan. I've pushed the changeset after re-ran the core tests. -Joe On 12/12/17, 8:47 AM, Roger Riggs wrote: > Hi Joe, > > Looks good; +1 > > Regards, Roger > > On 12/11/2017 3:04 PM, Joe Wang wrote: >> >> On 12/11/17, 11:18 AM, Roger Riggs wrote: >>> Hi Joe, >>> >>> The new tests using testng look good, a few comments: >>> >>> - BAOS/EncodingTest.java >>> 70: The message gets printed if the condition is not true; so >>> "results do not might" might read better >>> (also, there is Assert.assertEquals(str1, str2, msg) - that >>> compares objects with .equals()) >>> A message in nice, though in many cases the testng generated >>> exception/message is sufficient >>> especially for equals. >> >> Assert.assertEquals it is! Removed other debug printout. >> >>> >>> - PrintStream/EncodingTest.java >>> 92: out should not be null; its probably a test bug; the test >>> should be removed >> >> Done. >>> >>> - PrintWriter/EncodingTest.java >>> 93: out should not be null; its probably a test bug; the test >>> should be removed >> >> Done. >> >> Updated webrev: >> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html >> >> Thanks, >> Joe >> >>> >>> Thanks, Roger >>> >>> >>> On 12/8/2017 7:29 PM, Joe Wang wrote: >>>> Thanks Roger, Alan for the comments! >>>> >>>> I've updated the webrev accordingly. Here's the current webrev: >>>> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html >>>> >>>> -Joe >>>> >>>> On 12/4/17, 7:52 PM, Joe Wang wrote: >>>>> >>>>> >>>>> On 12/3/17, 11:34 AM, Alan Bateman wrote: >>>>>> >>>>>> >>>>>> On 01/12/2017 20:35, Joe Wang wrote: >>>>>>> : >>>>>>> >>>>>>>> >>>>>>>> I think URLDecoder.decode methods should have @throws >>>>>>>> IllegalArgumentException. I realize it's implementation >>>>>>>> specific as to whether IAE is thrown with bad input but given >>>>>>>> that the RI does throw IAE then consumers of the API should be >>>>>>>> prepared for it. The @implNote can stay and probably should be >>>>>>>> copied into the legacy decode method too. >>>>>>> >>>>>>> I added @throws IAE. On a 2nd thought, would that give no >>>>>>> flexibility to alternative impls as the general (class) spec had >>>>>>> given? With this addition, it becomes a requirement. >>>>>> If the @throws description starts with "if the implementation >>>>>> ..." then it should be clear that it is optional. >>>>> >>>>> Thanks Alan. I made the change accordingly. >>>>> http://cr.openjdk.java.net/~joehw/jdk10/8183743/webrev/index.html >>>>> >>>>> -Joe >>>>> >>>>>> >>>>>> -Alan >>> > From mandy.chung at oracle.com Tue Dec 12 19:28:47 2017 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 12 Dec 2017 11:28:47 -0800 Subject: Possible bug in StackFrameInfo#getByteCodeIndex? In-Reply-To: References: Message-ID: <2a707401-81ae-1580-7646-bba4d7fe026c@oracle.com> Good catch!? I agree that it should not be final and the constant field should be static. I will take JDK-8193325 and follow up this issue. -1 is ambiguous but should be initialized to a valid index when filled by the VM.? We tried to keep it compact for footprint concern.? Anyway I will propose a patch. Mandy On 12/12/17 11:08 AM, Martin Buchholz wrote: > Out of curiosity, I looked more at StackFrameInfo, and saw: > > short bci is final and is only ever assigned to -1 in java code. What's up > with that? Ohh, it seems to be magically frobbed in hotspot: > > void java_lang_StackFrameInfo::set_bci(oop element, int value) { > element->int_field_put(_bci_offset, value); > } > > BUT: bci should not be final if hotspot magic is frobbing it. Can we add a > comment explaining the magic? Also set_bci seems to disagree about the > type of bci - it's trying to set an int field, but bci is a short! > > Also, I expected "int value" above to be "jint value" - it's a java type! > > I would prefer bci to be a java int, not short, so that it can hold all > possible bytecode index values in a natural way, together with -1 for > "invalid". Right now, -1 is ambiguous - it might mean bci == 65535. > > Also, I see > private final byte RETAIN_CLASS_REF = 0x01; > Shouldn't that be static? From paul.sandoz at oracle.com Tue Dec 12 19:42:41 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Dec 2017 11:42:41 -0800 Subject: RFR of 8180451: ByteArrayInputStream should override readAllBytes, readNBytes, and transferTo In-Reply-To: <09F39E3A-76B0-4D36-98F3-78AD246B91A7@oracle.com> References: <09F39E3A-76B0-4D36-98F3-78AD246B91A7@oracle.com> Message-ID: <37818C83-0741-42EE-AD95-6259695E2DF9@oracle.com> > On 11 Dec 2017, at 12:47, Brian Burkhalter wrote: > > https://bugs.openjdk.java.net/browse/JDK-8180451 > http://cr.openjdk.java.net/~bpb/8180451/webrev.00 > > 1. Add overrides of readAllBytes(), readNBytes(), and transferTo(). > 2. Simplify parameter checks in read(byte[],int,int). > 3. Change foo to {@code foo}. > Minor thing (no need for another review round): 208 public synchronized long transferTo(OutputStream out) throws IOException { 209 int pos0 = pos; 210 out.write(buf, pos, count - pos); 211 pos = count; 212 return count - pos0; 213 } int len = count - pos out.write(but, pos, len); pos = count; return len; Paul. > Thanks, > > Brian From brian.burkhalter at oracle.com Tue Dec 12 19:44:07 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 12 Dec 2017 11:44:07 -0800 Subject: RFR of 8180451: ByteArrayInputStream should override readAllBytes, readNBytes, and transferTo In-Reply-To: <37818C83-0741-42EE-AD95-6259695E2DF9@oracle.com> References: <09F39E3A-76B0-4D36-98F3-78AD246B91A7@oracle.com> <37818C83-0741-42EE-AD95-6259695E2DF9@oracle.com> Message-ID: <1573685D-0D11-4C42-BC2E-27AD047DB449@oracle.com> On Dec 12, 2017, at 11:42 AM, Paul Sandoz wrote: > int len = count - pos > out.write(but, pos, len); > pos = count; > return len; Indeed that is clearer. Thanks, Brian From brent.christian at oracle.com Tue Dec 12 20:23:51 2017 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 12 Dec 2017 12:23:51 -0800 Subject: RFR of 8180451: ByteArrayInputStream should override readAllBytes, readNBytes, and transferTo In-Reply-To: <09F39E3A-76B0-4D36-98F3-78AD246B91A7@oracle.com> References: <09F39E3A-76B0-4D36-98F3-78AD246B91A7@oracle.com> Message-ID: <13322535-e651-fa8d-dfce-5b7c2d694ff5@oracle.com> Hi, Brian The changes look fine to me. I would have found the test case a little easier to follow if "off"/"len" weren't named so similarly to "offset"/"length", but it's not a big deal. -Brent On 12/11/17 12:47 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8180451 > http://cr.openjdk.java.net/~bpb/8180451/webrev.00 > > 1. Add overrides of readAllBytes(), readNBytes(), and transferTo(). > 2. Simplify parameter checks in read(byte[],int,int). > 3. Change foo to {@code foo}. > > Thanks, > > Brian > From stuart.marks at oracle.com Tue Dec 12 22:56:51 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 12 Dec 2017 14:56:51 -0800 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <5D0A53C3-80B9-41AC-A6EE-8E540D64956F@oracle.com> <8517efb9-e575-b8ae-2872-a6ed146ac4e6@oracle.com> <9a16aa5a-b06e-5be9-c894-5015678aba46@oracle.com> <0eb5dca7-a8f6-4aba-1cfe-a104dd8d7c8e@oracle.com> Message-ID: On 12/7/17 5:03 PM, Martin Buchholz wrote: > (I'm still trying to love this new API) > > The changes to the jsr166 tck are fine. > > I'm not convinced that the new implementation for ArrayList is progress.? The > current implementation of toArray(T[]) does > > ? ? ? ? ? ? // Make a new array of a's runtime type, but my contents: > ? ? ? ? ? ? return (T[]) Arrays.copyOf(elementData, size, a.getClass()); > > which does not have to pay the cost of zeroing the array, so is potentially > faster. ?(depends on whether the VM provides cheap pre-zeroed memory?! what does > jmh say?) > > If we're not getting type safety and not getting significantly better > performance, all we have is a more natural API.? But toArray(IntFunction) also > seems not very natural to me, and we'll have to live with the toArray(new > String[0]) wart forever anyways. ?(But is it really so bad?) ?(Maybe > toArray(Class) is actually more natural?) A lot of ideas flying around here. (Thanks John! :-)) But for this message let me focus on performance. I think implicitly there's some concern about the new API and whether it might suffer inherent performance disadvantages compared to the existing APIs. It certainly doesn't seem to provide any inherent advantages. I spent some time doing benchmarking more rigorously, and I was able to get similar results to those from Alexey Shipil?v's article: https://shipilev.net/blog/2016/arrays-wisdom-ancients/ That is, I was able to discern the difference between the copy-to-Object[] case, the copy to presized array case, and copy to zero-sized array case. Also added a potential new API toArray(Class) for comparison. Here are the results. For brevity, I ran only the tests with an ArrayList of size 1000: Benchmark (size) (type) Mode Cnt Score Error Units ToArrayBench.clazz 1000 arraylist avgt 30 1423.924 ? 11.437 ns/op ToArrayBench.clazz0 1000 arraylist avgt 30 1173.442 ? 11.614 ns/op ToArrayBench.clazzDf 1000 arraylist avgt 30 1179.648 ? 17.662 ns/op ToArrayBench.lambda 1000 arraylist avgt 30 1421.910 ? 21.320 ns/op ToArrayBench.lambda0 1000 arraylist avgt 30 1168.863 ? 11.109 ns/op ToArrayBench.lambdaDf 1000 arraylist avgt 30 1168.372 ? 9.512 ns/op ToArrayBench.simple 1000 arraylist avgt 30 462.371 ? 8.693 ns/op ToArrayBench.sized 1000 arraylist avgt 30 1417.483 ? 11.451 ns/op ToArrayBench.zero 1000 arraylist avgt 30 1182.932 ? 27.773 ns/op Legend: clazz - ArrayList override of toArray(Class) T[] a = (T[])java.lang.reflect.Array.newInstance(Foo.class, size); System.arraycopy(elementData, 0, a, 0, size); clazz0 - ArrayList override of toArray(Class) T[] a = (T[])java.lang.reflect.Array.newInstance(clazz, 0); return (T[]) Arrays.copyOf(elementData, size, a.getClass()); classDf - Collection default method toArray(Class) toArray((T[])java.lang.reflect.Array.newInstance(clazz, 0)) lambda - ArrayList override of toArray(IntFunc) T[] a = generator.apply(size); System.arraycopy(elementData, 0, a, 0, size); lambda0 - ArrayList override of toArray(IntFunc) (T[]) Arrays.copyOf(elementData, size, generator.apply(0).getClass()) lambdaDf - Collection default method toArray(IntFunc) toArray(generator.apply(0)) simple - existing no-arg toArray() method sized - existing toArray(T[]) called with presized array coll.toArray(new Foo[coll.size()]) zero - existing toArray(T[]) called with zero-sized array coll.toArray(new Foo[0]) ---------- A couple observations from this. First, "lambda", the ArrayList override of toArray(IntFunc), is slower than "lambda0" or "lambdaDf", both of which create zero-length arrays and pass them elsewhere. So Martin, you're right, the override I put into ArrayList isn't buying anything. I'll take it out. Oddly enough, just inheriting the default might be sufficient. (And the same probably applies to Arrays$ArrayList as well.) Second, setting aside "simple" case (which is fast because doesn't have to do any array store checking) we see performance falling into two buckets: one that takes around 1400ns, and the other that takes around 1170ns. It turns out that the faster cases all end up calling the Arrays.copyOf() method. This is an intrinsic that can avoid the work of zeroing the array, because it allocates the array immediately before filling it with data from the source. I would have thought that allocating the array immediately prior to filling it with System.arraycopy would have done the same, but apparently not. We can see the effect of intrinsics by disabling the Arrays.copyOf intrinsic by applying the -XX:+UnlockDiagnosticVMOptions -XX:DisableIntrinsic=_copyOf options: Benchmark (size) (type) Mode Cnt Score Error Units ToArrayBench.clazz 1000 arraylist avgt 30 1413.922 ? 9.570 ns/op ToArrayBench.clazz0 1000 arraylist avgt 30 1427.857 ? 13.669 ns/op ToArrayBench.clazzDf 1000 arraylist avgt 30 1448.274 ? 15.734 ns/op ToArrayBench.lambda 1000 arraylist avgt 30 1423.556 ? 17.312 ns/op ToArrayBench.lambda0 1000 arraylist avgt 30 1422.177 ? 20.650 ns/op ToArrayBench.lambdaDf 1000 arraylist avgt 30 1464.320 ? 26.199 ns/op ToArrayBench.simple 1000 arraylist avgt 30 399.856 ? 4.893 ns/op ToArrayBench.sized 1000 arraylist avgt 30 1417.668 ? 8.979 ns/op ToArrayBench.zero 1000 arraylist avgt 30 1438.527 ? 11.521 ns/op Here, we can see that everything (except "simple") has moved into the 1400ns range. (Not sure why "simple" appears to have gotten faster.) My conclusion from this is that the choice of one API over another doesn't offer any inherent performance advantage. You can make any of the APIs go fast if you can arrange for it to call an intrinsic, even if this ends up allocating an "extra" zero-length array. So the issue is really about which API we want to expose. I'll reserve that discussion for another message. s'marks From brent.christian at oracle.com Tue Dec 12 23:38:57 2017 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 12 Dec 2017 15:38:57 -0800 Subject: RFR[10] 8190984 : tools/launcher/TestXcheckJNIWarnings.java WARNING was found in the output Message-ID: Hi, Please review this small change to the System.initProperties() native method. The tools/launcher/TestXcheckJNIWarnings.java has begun to fail due to this warning being issued: WARNING: JNI local refs: 41, exceeds capacity: 40 at java.lang.System.initProperties(java.base/Native Method) at java.lang.System.initPhase1(java.base/System.java:1943) This has only been seen on solaris-sparc with LANG=C. In fact, the warning can also be seen with a simple HelloWorld program, run with -Xcheck:jni. This started happening as of JDK-8043224 ("-Xcheck:jni improvements to exception checking and excessive local refs"). The following change should be made to prevent the warning: diff -r ed1bb7743b3e src/java.base/share/native/libjava/System.c --- a/src/java.base/share/native/libjava/System.c Tue Dec 12 19:05:02 2017 +0100 +++ b/src/java.base/share/native/libjava/System.c Tue Dec 12 15:21:03 2017 -0800 @@ -186,6 +186,9 @@ jobject ret = NULL; jstring jVMVal = NULL; + if ((*env)->EnsureLocalCapacity(env, 50) < 0) { + return NULL; + } sprops = GetJavaProperties(env); CHECK_NULL_RETURN(sprops, NULL); Of note: an additional change is needed, to EnsureLocalCapacity itself: 8193222 : EnsureLocalCapacity() should maintain capacity requests through multiple calls That fix has already been proposed and reviewed. These two fixes should be pushed together via hs, which David has kindly offered to do, pending code review approval of my change here. Thanks, -Brent From david.holmes at oracle.com Wed Dec 13 00:00:42 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 13 Dec 2017 10:00:42 +1000 Subject: RFR[10] 8190984 : tools/launcher/TestXcheckJNIWarnings.java WARNING was found in the output In-Reply-To: References: Message-ID: Reviewed! :) Thanks Brent! I guess I can push these both now. David On 13/12/2017 9:38 AM, Brent Christian wrote: > Hi, > > Please review this small change to the System.initProperties() native > method. > > The tools/launcher/TestXcheckJNIWarnings.java has begun to fail due to > this warning being issued: > > WARNING: JNI local refs: 41, exceeds capacity: 40 > ??????? at java.lang.System.initProperties(java.base/Native Method) > ??????? at java.lang.System.initPhase1(java.base/System.java:1943) > > This has only been seen on solaris-sparc with LANG=C.? In fact, the > warning can also be seen with a simple HelloWorld program, run with > -Xcheck:jni.? This started happening as of JDK-8043224 ("-Xcheck:jni > improvements to exception checking and excessive local refs"). > > The following change should be made to prevent the warning: > > diff -r ed1bb7743b3e src/java.base/share/native/libjava/System.c > --- a/src/java.base/share/native/libjava/System.c??? Tue Dec 12 19:05:02 > 2017 +0100 > +++ b/src/java.base/share/native/libjava/System.c??? Tue Dec 12 15:21:03 > 2017 -0800 > @@ -186,6 +186,9 @@ > ???? jobject ret = NULL; > ???? jstring jVMVal = NULL; > > +??? if ((*env)->EnsureLocalCapacity(env, 50) < 0) { > +??????? return NULL; > +??? } > ???? sprops = GetJavaProperties(env); > ???? CHECK_NULL_RETURN(sprops, NULL); > > > Of note: an additional change is needed, to EnsureLocalCapacity itself: > > 8193222 : EnsureLocalCapacity() should maintain capacity requests > through multiple calls > > That fix has already been proposed and reviewed. > These two fixes should be pushed together via hs, which David has kindly > offered to do, pending code review approval of my change here. > > Thanks, > -Brent > From david.holmes at oracle.com Wed Dec 13 00:18:09 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 13 Dec 2017 10:18:09 +1000 Subject: RFR[10] 8190984 : tools/launcher/TestXcheckJNIWarnings.java WARNING was found in the output In-Reply-To: <5a131395-742d-1e24-a71d-6f65a50ecf2a@oracle.com> References: <5a131395-742d-1e24-a71d-6f65a50ecf2a@oracle.com> Message-ID: <7ad3e3f0-3482-dc14-1bdf-7ff15478f2df@oracle.com> Hi Mandy, On 13/12/2017 10:12 AM, mandy chung wrote: > How do we pick the value 50?? The JNI local refs are deleted in some > cases and it's unclear how 50 is computed.? What if new properties are > added in the future? IIRC Brent instrumented the localref methods to see how many were being created, and the "50" gives us sufficient capacity. The actual number of local refs introduced seems to be platform and locale specific. Note this only affects the warnings that -Xcheck:jni produces (at least on hotspot). If more properties are added that take us beyond the new capacity then we'll see test failures again and can increase it further. Thanks, David > Mandy > > On 12/12/17 4:00 PM, David Holmes wrote: >> Reviewed! :) >> >> Thanks Brent! >> >> I guess I can push these both now. >> >> David >> >> On 13/12/2017 9:38 AM, Brent Christian wrote: >>> Hi, >>> >>> Please review this small change to the System.initProperties() native >>> method. >>> >>> The tools/launcher/TestXcheckJNIWarnings.java has begun to fail due >>> to this warning being issued: >>> >>> WARNING: JNI local refs: 41, exceeds capacity: 40 >>> ???????? at java.lang.System.initProperties(java.base/Native Method) >>> ???????? at java.lang.System.initPhase1(java.base/System.java:1943) >>> >>> This has only been seen on solaris-sparc with LANG=C.? In fact, the >>> warning can also be seen with a simple HelloWorld program, run with >>> -Xcheck:jni.? This started happening as of JDK-8043224 ("-Xcheck:jni >>> improvements to exception checking and excessive local refs"). >>> >>> The following change should be made to prevent the warning: >>> >>> diff -r ed1bb7743b3e src/java.base/share/native/libjava/System.c >>> --- a/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >>> 19:05:02 2017 +0100 >>> +++ b/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >>> 15:21:03 2017 -0800 >>> @@ -186,6 +186,9 @@ >>> ????? jobject ret = NULL; >>> ????? jstring jVMVal = NULL; >>> >>> +??? if ((*env)->EnsureLocalCapacity(env, 50) < 0) { >>> +??????? return NULL; >>> +??? } >>> ????? sprops = GetJavaProperties(env); >>> ????? CHECK_NULL_RETURN(sprops, NULL); >>> >>> >>> Of note: an additional change is needed, to EnsureLocalCapacity itself: >>> >>> 8193222 : EnsureLocalCapacity() should maintain capacity requests >>> through multiple calls >>> >>> That fix has already been proposed and reviewed. >>> These two fixes should be pushed together via hs, which David has >>> kindly offered to do, pending code review approval of my change here. >>> >>> Thanks, >>> -Brent >>> > From mandy.chung at oracle.com Wed Dec 13 00:12:42 2017 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 12 Dec 2017 16:12:42 -0800 Subject: RFR[10] 8190984 : tools/launcher/TestXcheckJNIWarnings.java WARNING was found in the output In-Reply-To: References: Message-ID: <5a131395-742d-1e24-a71d-6f65a50ecf2a@oracle.com> How do we pick the value 50?? The JNI local refs are deleted in some cases and it's unclear how 50 is computed.? What if new properties are added in the future? Mandy On 12/12/17 4:00 PM, David Holmes wrote: > Reviewed! :) > > Thanks Brent! > > I guess I can push these both now. > > David > > On 13/12/2017 9:38 AM, Brent Christian wrote: >> Hi, >> >> Please review this small change to the System.initProperties() native >> method. >> >> The tools/launcher/TestXcheckJNIWarnings.java has begun to fail due >> to this warning being issued: >> >> WARNING: JNI local refs: 41, exceeds capacity: 40 >> ???????? at java.lang.System.initProperties(java.base/Native Method) >> ???????? at java.lang.System.initPhase1(java.base/System.java:1943) >> >> This has only been seen on solaris-sparc with LANG=C.? In fact, the >> warning can also be seen with a simple HelloWorld program, run with >> -Xcheck:jni.? This started happening as of JDK-8043224 ("-Xcheck:jni >> improvements to exception checking and excessive local refs"). >> >> The following change should be made to prevent the warning: >> >> diff -r ed1bb7743b3e src/java.base/share/native/libjava/System.c >> --- a/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >> 19:05:02 2017 +0100 >> +++ b/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >> 15:21:03 2017 -0800 >> @@ -186,6 +186,9 @@ >> ????? jobject ret = NULL; >> ????? jstring jVMVal = NULL; >> >> +??? if ((*env)->EnsureLocalCapacity(env, 50) < 0) { >> +??????? return NULL; >> +??? } >> ????? sprops = GetJavaProperties(env); >> ????? CHECK_NULL_RETURN(sprops, NULL); >> >> >> Of note: an additional change is needed, to EnsureLocalCapacity itself: >> >> 8193222 : EnsureLocalCapacity() should maintain capacity requests >> through multiple calls >> >> That fix has already been proposed and reviewed. >> These two fixes should be pushed together via hs, which David has >> kindly offered to do, pending code review approval of my change here. >> >> Thanks, >> -Brent >> From mandy.chung at oracle.com Wed Dec 13 00:42:36 2017 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 12 Dec 2017 16:42:36 -0800 Subject: Review Request JDK-8193325: StackFrameInfo::getByteCodeIndex returns wrong value if bci > 32767 Message-ID: <4cf575a1-d420-fb44-cc38-03287366ec3d@oracle.com> http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8193325/webrev.00/ This fixes the bug in hotspot that sets the bci field as int but it's of short type and also StackFrameInfo to return a proper bci value.? Coleen suggests not to change set_bci to take jushort but instead do the cast as in the proposed fix.? The VM is in favor of C++ types and stop propagation of Java types.? bci is of int in VM implementation and so be consistent. Mandy [1] mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050581.html From mandy.chung at oracle.com Wed Dec 13 00:53:16 2017 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 12 Dec 2017 16:53:16 -0800 Subject: RFR[10] 8190984 : tools/launcher/TestXcheckJNIWarnings.java WARNING was found in the output In-Reply-To: <7ad3e3f0-3482-dc14-1bdf-7ff15478f2df@oracle.com> References: <5a131395-742d-1e24-a71d-6f65a50ecf2a@oracle.com> <7ad3e3f0-3482-dc14-1bdf-7ff15478f2df@oracle.com> Message-ID: <999aa06f-f4ce-ac09-b961-0571d8a53d8c@oracle.com> On 12/12/17 4:18 PM, David Holmes wrote: > Hi Mandy, > > On 13/12/2017 10:12 AM, mandy chung wrote: >> How do we pick the value 50?? The JNI local refs are deleted in some >> cases and it's unclear how 50 is computed.? What if new properties >> are added in the future? > > IIRC Brent instrumented the localref methods to see how many were > being created, and the "50" gives us sufficient capacity. The actual > number of local refs introduced seems to be platform and locale specific. > This is what I guessed too before I replied.? The part that's unclear to me is whether DeleteLocalRef helps and whether any places are missing DeleteLocalRefs as it's expected to.? It reads to me that the JNI calls to add/put a property intends to clear up after itself. It's okay to push this fix to JDK 10. I hope we can clean up this native properties implementation at some point. Mandy > Note this only affects the warnings that -Xcheck:jni produces (at > least on hotspot). If more properties are added that take us beyond > the new capacity then we'll see test failures again and can increase > it further. > > Thanks, > David > >> Mandy >> >> On 12/12/17 4:00 PM, David Holmes wrote: >>> Reviewed! :) >>> >>> Thanks Brent! >>> >>> I guess I can push these both now. >>> >>> David >>> >>> On 13/12/2017 9:38 AM, Brent Christian wrote: >>>> Hi, >>>> >>>> Please review this small change to the System.initProperties() >>>> native method. >>>> >>>> The tools/launcher/TestXcheckJNIWarnings.java has begun to fail due >>>> to this warning being issued: >>>> >>>> WARNING: JNI local refs: 41, exceeds capacity: 40 >>>> ???????? at java.lang.System.initProperties(java.base/Native Method) >>>> ???????? at java.lang.System.initPhase1(java.base/System.java:1943) >>>> >>>> This has only been seen on solaris-sparc with LANG=C.? In fact, the >>>> warning can also be seen with a simple HelloWorld program, run with >>>> -Xcheck:jni.? This started happening as of JDK-8043224 >>>> ("-Xcheck:jni improvements to exception checking and excessive >>>> local refs"). >>>> >>>> The following change should be made to prevent the warning: >>>> >>>> diff -r ed1bb7743b3e src/java.base/share/native/libjava/System.c >>>> --- a/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >>>> 19:05:02 2017 +0100 >>>> +++ b/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >>>> 15:21:03 2017 -0800 >>>> @@ -186,6 +186,9 @@ >>>> ????? jobject ret = NULL; >>>> ????? jstring jVMVal = NULL; >>>> >>>> +??? if ((*env)->EnsureLocalCapacity(env, 50) < 0) { >>>> +??????? return NULL; >>>> +??? } >>>> ????? sprops = GetJavaProperties(env); >>>> ????? CHECK_NULL_RETURN(sprops, NULL); >>>> >>>> >>>> Of note: an additional change is needed, to EnsureLocalCapacity >>>> itself: >>>> >>>> 8193222 : EnsureLocalCapacity() should maintain capacity requests >>>> through multiple calls >>>> >>>> That fix has already been proposed and reviewed. >>>> These two fixes should be pushed together via hs, which David has >>>> kindly offered to do, pending code review approval of my change here. >>>> >>>> Thanks, >>>> -Brent >>>> >> From david.holmes at oracle.com Wed Dec 13 00:48:40 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 13 Dec 2017 10:48:40 +1000 Subject: RFR[10] 8190984 : tools/launcher/TestXcheckJNIWarnings.java WARNING was found in the output In-Reply-To: References: <5a131395-742d-1e24-a71d-6f65a50ecf2a@oracle.com> <7ad3e3f0-3482-dc14-1bdf-7ff15478f2df@oracle.com> Message-ID: <3503aec7-17ba-8002-eea1-5cbb5cba8d44@oracle.com> On 13/12/2017 10:41 AM, Brent Christian wrote: > What David said. :) > > One other wrinkle: I ProblemList-ed the test a few days ago[1] in I forgot about that :( > jdk/jdk.? AFAICT, that change is not yet in jdk/hs, so the changes > David's pushing can't be used to take the test back off of the > ProblemList.? I figure I'll have to file a new issue, to be fixed in > jdk/jdk (unless someone has a better idea). I will see if we can re-sync to hs before I push the changes. Otherwise I could switch everything to jdk/jdk. Otherwise a new issue will need to be filed. Thanks, David > Thanks, > -Brent > > 1. http://hg.openjdk.java.net/jdk/jdk/rev/b9a19d1e61f2 > > On 12/12/17 4:18 PM, David Holmes wrote: >> >> IIRC Brent instrumented the localref methods to see how many were >> being created, and the "50" gives us sufficient capacity. The actual >> number of local refs introduced seems to be platform and locale specific. >> >> Note this only affects the warnings that -Xcheck:jni produces (at >> least on hotspot). If more properties are added that take us beyond >> the new capacity then we'll see test failures again and can increase >> it further. >> >> Thanks, >> David >> >>> Mandy >>> >>> On 12/12/17 4:00 PM, David Holmes wrote: >>>> Reviewed! :) >>>> >>>> Thanks Brent! >>>> >>>> I guess I can push these both now. >>>> >>>> David >>>> >>>> On 13/12/2017 9:38 AM, Brent Christian wrote: >>>>> Hi, >>>>> >>>>> Please review this small change to the System.initProperties() >>>>> native method. >>>>> >>>>> The tools/launcher/TestXcheckJNIWarnings.java has begun to fail due >>>>> to this warning being issued: >>>>> >>>>> WARNING: JNI local refs: 41, exceeds capacity: 40 >>>>> ???????? at java.lang.System.initProperties(java.base/Native Method) >>>>> ???????? at java.lang.System.initPhase1(java.base/System.java:1943) >>>>> >>>>> This has only been seen on solaris-sparc with LANG=C.? In fact, the >>>>> warning can also be seen with a simple HelloWorld program, run with >>>>> -Xcheck:jni.? This started happening as of JDK-8043224 >>>>> ("-Xcheck:jni improvements to exception checking and excessive >>>>> local refs"). >>>>> >>>>> The following change should be made to prevent the warning: >>>>> >>>>> diff -r ed1bb7743b3e src/java.base/share/native/libjava/System.c >>>>> --- a/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >>>>> 19:05:02 2017 +0100 >>>>> +++ b/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >>>>> 15:21:03 2017 -0800 >>>>> @@ -186,6 +186,9 @@ >>>>> ????? jobject ret = NULL; >>>>> ????? jstring jVMVal = NULL; >>>>> >>>>> +??? if ((*env)->EnsureLocalCapacity(env, 50) < 0) { >>>>> +??????? return NULL; >>>>> +??? } >>>>> ????? sprops = GetJavaProperties(env); >>>>> ????? CHECK_NULL_RETURN(sprops, NULL); >>>>> >>>>> >>>>> Of note: an additional change is needed, to EnsureLocalCapacity >>>>> itself: >>>>> >>>>> 8193222 : EnsureLocalCapacity() should maintain capacity requests >>>>> through multiple calls >>>>> >>>>> That fix has already been proposed and reviewed. >>>>> These two fixes should be pushed together via hs, which David has >>>>> kindly offered to do, pending code review approval of my change here. >>>>> >>>>> Thanks, >>>>> -Brent >>>>> >>> From brent.christian at oracle.com Wed Dec 13 00:41:01 2017 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 12 Dec 2017 16:41:01 -0800 Subject: RFR[10] 8190984 : tools/launcher/TestXcheckJNIWarnings.java WARNING was found in the output In-Reply-To: <7ad3e3f0-3482-dc14-1bdf-7ff15478f2df@oracle.com> References: <5a131395-742d-1e24-a71d-6f65a50ecf2a@oracle.com> <7ad3e3f0-3482-dc14-1bdf-7ff15478f2df@oracle.com> Message-ID: What David said. :) One other wrinkle: I ProblemList-ed the test a few days ago[1] in jdk/jdk. AFAICT, that change is not yet in jdk/hs, so the changes David's pushing can't be used to take the test back off of the ProblemList. I figure I'll have to file a new issue, to be fixed in jdk/jdk (unless someone has a better idea). Thanks, -Brent 1. http://hg.openjdk.java.net/jdk/jdk/rev/b9a19d1e61f2 On 12/12/17 4:18 PM, David Holmes wrote: > > IIRC Brent instrumented the localref methods to see how many were being > created, and the "50" gives us sufficient capacity. The actual number of > local refs introduced seems to be platform and locale specific. > > Note this only affects the warnings that -Xcheck:jni produces (at least > on hotspot). If more properties are added that take us beyond the new > capacity then we'll see test failures again and can increase it further. > > Thanks, > David > >> Mandy >> >> On 12/12/17 4:00 PM, David Holmes wrote: >>> Reviewed! :) >>> >>> Thanks Brent! >>> >>> I guess I can push these both now. >>> >>> David >>> >>> On 13/12/2017 9:38 AM, Brent Christian wrote: >>>> Hi, >>>> >>>> Please review this small change to the System.initProperties() >>>> native method. >>>> >>>> The tools/launcher/TestXcheckJNIWarnings.java has begun to fail due >>>> to this warning being issued: >>>> >>>> WARNING: JNI local refs: 41, exceeds capacity: 40 >>>> ???????? at java.lang.System.initProperties(java.base/Native Method) >>>> ???????? at java.lang.System.initPhase1(java.base/System.java:1943) >>>> >>>> This has only been seen on solaris-sparc with LANG=C.? In fact, the >>>> warning can also be seen with a simple HelloWorld program, run with >>>> -Xcheck:jni.? This started happening as of JDK-8043224 ("-Xcheck:jni >>>> improvements to exception checking and excessive local refs"). >>>> >>>> The following change should be made to prevent the warning: >>>> >>>> diff -r ed1bb7743b3e src/java.base/share/native/libjava/System.c >>>> --- a/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >>>> 19:05:02 2017 +0100 >>>> +++ b/src/java.base/share/native/libjava/System.c??? Tue Dec 12 >>>> 15:21:03 2017 -0800 >>>> @@ -186,6 +186,9 @@ >>>> ????? jobject ret = NULL; >>>> ????? jstring jVMVal = NULL; >>>> >>>> +??? if ((*env)->EnsureLocalCapacity(env, 50) < 0) { >>>> +??????? return NULL; >>>> +??? } >>>> ????? sprops = GetJavaProperties(env); >>>> ????? CHECK_NULL_RETURN(sprops, NULL); >>>> >>>> >>>> Of note: an additional change is needed, to EnsureLocalCapacity itself: >>>> >>>> 8193222 : EnsureLocalCapacity() should maintain capacity requests >>>> through multiple calls >>>> >>>> That fix has already been proposed and reviewed. >>>> These two fixes should be pushed together via hs, which David has >>>> kindly offered to do, pending code review approval of my change here. >>>> >>>> Thanks, >>>> -Brent >>>> >> From david.holmes at oracle.com Wed Dec 13 01:24:55 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 13 Dec 2017 11:24:55 +1000 Subject: Review Request JDK-8193325: StackFrameInfo::getByteCodeIndex returns wrong value if bci > 32767 In-Reply-To: <4cf575a1-d420-fb44-cc38-03287366ec3d@oracle.com> References: <4cf575a1-d420-fb44-cc38-03287366ec3d@oracle.com> Message-ID: <8a326b47-176b-c41a-7fe1-d9d53f37bd5c@oracle.com> Hi Mandy, On 13/12/2017 10:42 AM, mandy chung wrote: > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8193325/webrev.00/ > > This fixes the bug in hotspot that sets the bci field as int but it's of > short type and also StackFrameInfo to return a proper bci value.? Coleen > suggests not to change set_bci to take jushort but instead do the cast > as in the proposed fix.? The VM is in favor of C++ types and stop > propagation of Java types.? bci is of int in VM implementation and so be > consistent. I must admit I'm unclear why the VM needs the BCI to be an int when logically it is a jushort, but perhaps, as per getByteCodeIndex() it needs to be able to store "-1" to indicate no-bci? The fix itself seems fine, though if we're going to chop off the VM value at 16-bits perhaps we should assert that it isn't a value that requires > 16 bits? Thanks, David > > Mandy > [1] mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050581.html From john.r.rose at oracle.com Wed Dec 13 01:29:41 2017 From: john.r.rose at oracle.com (John Rose) Date: Tue, 12 Dec 2017 17:29:41 -0800 Subject: Review Request JDK-8193325: StackFrameInfo::getByteCodeIndex returns wrong value if bci > 32767 In-Reply-To: <8a326b47-176b-c41a-7fe1-d9d53f37bd5c@oracle.com> References: <4cf575a1-d420-fb44-cc38-03287366ec3d@oracle.com> <8a326b47-176b-c41a-7fe1-d9d53f37bd5c@oracle.com> Message-ID: <4E31CF3C-D7BF-4070-85F5-66D281478F6E@oracle.com> On Dec 12, 2017, at 5:24 PM, David Holmes wrote: > > as per getByteCodeIndex() it needs to be able to store "-1" to indicate no-bci? Yes, that's the essential requirement, to encode a "none" value. FWIW, one natural way to do that here would be to make the int field @Stable and have the accessor return one less than the stored field value. Then, by the rules of @Stable, the field starts out at the logical value of -1, and can transition to a valid BCI if patched to a non-zero value in the range 1..(2^16). Making it "final" and initialized to -1 is not too terrible either, although that creates some tech. debt we need to discharge when upgrading finals to be trustable. ? John From vladimir.kozlov at oracle.com Wed Dec 13 08:16:39 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 13 Dec 2017 00:16:39 -0800 Subject: [10] (S) RFR 8191788: add jdk.internal.vm.compiler to --limit-modules if -Djvmci.Compiler=graal is in the command line In-Reply-To: <85ed009a-5c31-80fb-b8ae-1f889843e6e7@oracle.com> References: <25153d88-5ce9-34dd-e8d9-38dffc646f2f@oracle.com> <87f1dbbc-5b8b-be18-9a37-1568b0490521@oracle.com> <857e84fd-c86a-5c9b-1fdc-2b151874f000@oracle.com> <16aa4140-7843-b468-e24c-493bd89c2d65@oracle.com> <85ed009a-5c31-80fb-b8ae-1f889843e6e7@oracle.com> Message-ID: <7dc8336c-7024-e02a-accf-d8fc2a92d5bf@oracle.com> After several rounds of testing (which found different types of failures with Graal as JIT compiler) I decided to limit these changes for cases when --limit-modules is used: http://cr.openjdk.java.net/~kvn/8191788/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8191788 Thanks, Vladimir On 11/29/17 3:19 PM, Vladimir Kozlov wrote: > Yes, it was my limited understanding what --limit-modules is. We should > not add modules "under cover" when --limit-modules is used. User should > known all required modules *explicitly* to create correct image with > jlink based on result of runs with --limit-modules flag. > > Since it is limited set of tests which use --limit-modules (11 in > hotspot, 23 in jdk and few in closed) I will exclude these tests from > running with Graal as JIT by adding @require to them: > > ?@requires !vm.graal.enabled > > We may revisit this later in a future when Graal become default JIT > compiler (it should be supported on platforms at that time). > > Thanks, > Vladimir > > On 11/29/17 2:44 PM, mandy chung wrote: >> Vladimir and I discussed this offline. java --limit-modules should >> have the same set of observable modules as running on an image that is >> created with jlink.? When running with -XX:+UseJVMCICompiler on a >> runtime image with just java.base, it will fail in the same way as >> running with java --limit-modules java.base -XX:+UseJVMCICompiler. >> These tests are written to run on a runtime image with java.base only >> should be updated to get it run with -XX:+UseJVMCICompiler.? I will >> take a look and see how we can refactor the tests to support such >> testing configuration.?? In the meantime, Vladimir suggests to exclude >> these tests when running with -XX:+UseJVMCICompiler (specifying >> @requires). >> >> Mandy From martinrb at google.com Wed Dec 13 10:28:17 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 13 Dec 2017 02:28:17 -0800 Subject: RFR: JDK-8184947:,ZipCoder performance improvements In-Reply-To: <5A302626.3020908@oracle.com> References: <5A2B1BAB.7010102@oracle.com> <0ced2d86-f2bc-54d2-bbfa-a8f295d69127@oracle.com> <5A2E04D6.7000708@oracle.com> <510b8374-c26c-7c9b-cfc2-e9717df8bd3a@oracle.com> <5A302626.3020908@oracle.com> Message-ID: Sorry, I haven't had the time I would like to review. It would be good to make jdk10. I keep wishing what we do for performance here wouldn't get so messy. I keep thinking we should add some methods to the public Charset classes, e.g. canDecode(byte[], int, int) with one general purpose implementation and high-performance implementations for UTF-8, ASCII, Latin1 ASCII checking via hasNegatives has some hotspot help and that should be available as a high performance public API somewhere. One possibility is my canDecode suggestion. + if (b >= 0) + putChar(dst, dp++, (char)b); + else + putChar(dst, dp++, repl); why not coalesce into putChar(dst, dp++, (b >= 0) ? (char)b : repl) From martinrb at google.com Wed Dec 13 10:58:25 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 13 Dec 2017 02:58:25 -0800 Subject: RFR: JDK-8184947:,ZipCoder performance improvements In-Reply-To: References: <5A2B1BAB.7010102@oracle.com> <0ced2d86-f2bc-54d2-bbfa-a8f295d69127@oracle.com> <5A2E04D6.7000708@oracle.com> <510b8374-c26c-7c9b-cfc2-e9717df8bd3a@oracle.com> <5A302626.3020908@oracle.com> Message-ID: Fix typo: singlton Remove stray space: + ba = Arrays.copyOfRange(ba, off, off + len); From xueming.shen at oracle.com Wed Dec 13 15:08:16 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 13 Dec 2017 07:08:16 -0800 Subject: RFR: JDK-8184947:,ZipCoder performance improvements In-Reply-To: References: <5A2B1BAB.7010102@oracle.com> <0ced2d86-f2bc-54d2-bbfa-a8f295d69127@oracle.com> <5A2E04D6.7000708@oracle.com> <510b8374-c26c-7c9b-cfc2-e9717df8bd3a@oracle.com> <5A302626.3020908@oracle.com> Message-ID: <5A314260.3040304@oracle.com> Thanks Martin! webrev has been updated accordingly. http://cr.openjdk.java.net/~sherman/8184947/webrev I will try to see if I can do something for your canDecode(...) idea. I'm still have a relative big change in repo (mainly for the Strting(ByteBuffer ...) which might move all those utf8/asci/iso8859 back to where they were in sun.nio.cs. I'm running tests locally and on our mach5 system now. If everything comes back positively, with in next 50 minutes :-) I will try to push what I have right now into the jdk10. Thanks, Sherman On 12/13/17, 2:28 AM, Martin Buchholz wrote: > Sorry, I haven't had the time I would like to review. > It would be good to make jdk10. > I keep wishing what we do for performance here wouldn't get so messy. > I keep thinking we should add some methods to the public Charset > classes, e.g. canDecode(byte[], int, int) with one general purpose > implementation and high-performance implementations for UTF-8, ASCII, > Latin1 > ASCII checking via hasNegatives has some hotspot help and that should > be available as a high performance public API somewhere. One > possibility is my canDecode suggestion. > > + if (b>= 0) > + putChar(dst, dp++, (char)b); > + else > + putChar(dst, dp++, repl); > why not coalesce into putChar(dst, dp++, (b >= 0) ? (char)b : repl) > > From mandy.chung at oracle.com Wed Dec 13 16:47:43 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 13 Dec 2017 08:47:43 -0800 Subject: [10] (S) RFR 8191788: add jdk.internal.vm.compiler to --limit-modules if -Djvmci.Compiler=graal is in the command line In-Reply-To: <7dc8336c-7024-e02a-accf-d8fc2a92d5bf@oracle.com> References: <25153d88-5ce9-34dd-e8d9-38dffc646f2f@oracle.com> <87f1dbbc-5b8b-be18-9a37-1568b0490521@oracle.com> <857e84fd-c86a-5c9b-1fdc-2b151874f000@oracle.com> <16aa4140-7843-b468-e24c-493bd89c2d65@oracle.com> <85ed009a-5c31-80fb-b8ae-1f889843e6e7@oracle.com> <7dc8336c-7024-e02a-accf-d8fc2a92d5bf@oracle.com> Message-ID: On 12/13/17 12:16 AM, Vladimir Kozlov wrote: > After several rounds of testing (which found different types of > failures with Graal as JIT compiler) I decided to limit these changes > for cases when --limit-modules is used: > > http://cr.openjdk.java.net/~kvn/8191788/webrev.01/ Looks okay. Since the number of tests with --limit-modules is small, filtering them out when running with Graal as JIT compiler is a reasonable approach for JDK 10.?? At some point, we may revisit this and possibly a test library to deal with similar use case - tests with limited modules when running with additional VM flags requiring observable modules.? I will create an issue for it. Mandy From vladimir.kozlov at oracle.com Wed Dec 13 16:55:12 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 13 Dec 2017 08:55:12 -0800 Subject: [10] (S) RFR 8191788: add jdk.internal.vm.compiler to --limit-modules if -Djvmci.Compiler=graal is in the command line In-Reply-To: References: <25153d88-5ce9-34dd-e8d9-38dffc646f2f@oracle.com> <87f1dbbc-5b8b-be18-9a37-1568b0490521@oracle.com> <857e84fd-c86a-5c9b-1fdc-2b151874f000@oracle.com> <16aa4140-7843-b468-e24c-493bd89c2d65@oracle.com> <85ed009a-5c31-80fb-b8ae-1f889843e6e7@oracle.com> <7dc8336c-7024-e02a-accf-d8fc2a92d5bf@oracle.com> Message-ID: Thank you, Mandy Vladimir On 12/13/17 8:47 AM, mandy chung wrote: > > > On 12/13/17 12:16 AM, Vladimir Kozlov wrote: >> After several rounds of testing (which found different types of >> failures with Graal as JIT compiler) I decided to limit these changes >> for cases when --limit-modules is used: >> >> http://cr.openjdk.java.net/~kvn/8191788/webrev.01/ > > Looks okay. > > Since the number of tests with --limit-modules is small, filtering them > out when running with Graal as JIT compiler is a reasonable approach for > JDK 10.?? At some point, we may revisit this and possibly a test library > to deal with similar use case - tests with limited modules when running > with additional VM flags requiring observable modules.? I will create an > issue for it. > > Mandy From Alan.Bateman at oracle.com Wed Dec 13 16:58:30 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 13 Dec 2017 16:58:30 +0000 Subject: [10] (S) RFR 8191788: add jdk.internal.vm.compiler to --limit-modules if -Djvmci.Compiler=graal is in the command line In-Reply-To: References: <25153d88-5ce9-34dd-e8d9-38dffc646f2f@oracle.com> <87f1dbbc-5b8b-be18-9a37-1568b0490521@oracle.com> <857e84fd-c86a-5c9b-1fdc-2b151874f000@oracle.com> <16aa4140-7843-b468-e24c-493bd89c2d65@oracle.com> <85ed009a-5c31-80fb-b8ae-1f889843e6e7@oracle.com> <7dc8336c-7024-e02a-accf-d8fc2a92d5bf@oracle.com> Message-ID: <64006fa7-a538-6c9a-9237-04ca31afde5a@oracle.com> On 13/12/2017 16:47, mandy chung wrote: > > > On 12/13/17 12:16 AM, Vladimir Kozlov wrote: >> After several rounds of testing (which found different types of >> failures with Graal as JIT compiler) I decided to limit these changes >> for cases when --limit-modules is used: >> >> http://cr.openjdk.java.net/~kvn/8191788/webrev.01/ > > Looks okay. > > Since the number of tests with --limit-modules is small, filtering > them out when running with Graal as JIT compiler is a reasonable > approach for JDK 10.?? At some point, we may revisit this and possibly > a test library to deal with similar use case - tests with limited > modules when running with additional VM flags requiring observable > modules.? I will create an issue for it. I looked at the webrev too, looks good and much better than modifying the VM to augment the list specified to --limit-modules. -Alan From vladimir.kozlov at oracle.com Wed Dec 13 16:59:41 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 13 Dec 2017 08:59:41 -0800 Subject: [10] (S) RFR 8191788: add jdk.internal.vm.compiler to --limit-modules if -Djvmci.Compiler=graal is in the command line In-Reply-To: <64006fa7-a538-6c9a-9237-04ca31afde5a@oracle.com> References: <25153d88-5ce9-34dd-e8d9-38dffc646f2f@oracle.com> <87f1dbbc-5b8b-be18-9a37-1568b0490521@oracle.com> <857e84fd-c86a-5c9b-1fdc-2b151874f000@oracle.com> <16aa4140-7843-b468-e24c-493bd89c2d65@oracle.com> <85ed009a-5c31-80fb-b8ae-1f889843e6e7@oracle.com> <7dc8336c-7024-e02a-accf-d8fc2a92d5bf@oracle.com> <64006fa7-a538-6c9a-9237-04ca31afde5a@oracle.com> Message-ID: <15c22fd0-2dd6-d4af-e7ea-3448d3e34fe0@oracle.com> Thank you, Alan Vladimir On 12/13/17 8:58 AM, Alan Bateman wrote: > > > On 13/12/2017 16:47, mandy chung wrote: >> >> >> On 12/13/17 12:16 AM, Vladimir Kozlov wrote: >>> After several rounds of testing (which found different types of >>> failures with Graal as JIT compiler) I decided to limit these changes >>> for cases when --limit-modules is used: >>> >>> http://cr.openjdk.java.net/~kvn/8191788/webrev.01/ >> >> Looks okay. >> >> Since the number of tests with --limit-modules is small, filtering >> them out when running with Graal as JIT compiler is a reasonable >> approach for JDK 10.?? At some point, we may revisit this and possibly >> a test library to deal with similar use case - tests with limited >> modules when running with additional VM flags requiring observable >> modules.? I will create an issue for it. > I looked at the webrev too, looks good and much better than modifying > the VM to augment the list specified to --limit-modules. > > -Alan From brent.christian at oracle.com Wed Dec 13 18:03:03 2017 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 13 Dec 2017 10:03:03 -0800 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList Message-ID: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> Hi, Way back five days ago[1], TestXcheckJNIWarnings.java was added to the ProblemList. Well, with a couple of recent fixes[2][3], this test can be taken back off of the ProblemList. --- a/test/jdk/ProblemList.txt Fri Dec 08 13:04:43 2017 -0800 +++ b/test/jdk/ProblemList.txt Wed Dec 13 09:28:01 2017 -0800 @@ -257,7 +257,6 @@ tools/pack200/CommandLineTests.java 8059906 generic-all tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all -tools/launcher/TestXcheckJNIWarnings.java 8190984 solaris-all tools/jimage/JImageExtractTest.java 8170120 generic-all Thanks, -Brent 1. http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050496.html 2. https://bugs.openjdk.java.net/browse/JDK-8193222 3. https://bugs.openjdk.java.net/browse/JDK-8190984 From mandy.chung at oracle.com Wed Dec 13 18:07:00 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 13 Dec 2017 10:07:00 -0800 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> Message-ID: +1 Mandy On 12/13/17 10:03 AM, Brent Christian wrote: > Hi, > > Way back five days ago[1], TestXcheckJNIWarnings.java was added to the > ProblemList.? Well, with a couple of recent fixes[2][3], this test can > be taken back off of the ProblemList. > > --- a/test/jdk/ProblemList.txt??? Fri Dec 08 13:04:43 2017 -0800 > +++ b/test/jdk/ProblemList.txt??? Wed Dec 13 09:28:01 2017 -0800 > @@ -257,7 +257,6 @@ > ?tools/pack200/CommandLineTests.java 8059906 generic-all > > ?tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all > -tools/launcher/TestXcheckJNIWarnings.java 8190984 solaris-all > > ?tools/jimage/JImageExtractTest.java 8170120 generic-all > > Thanks, > -Brent > > 1. > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050496.html > 2. https://bugs.openjdk.java.net/browse/JDK-8193222 > 3. https://bugs.openjdk.java.net/browse/JDK-8190984 > From paul.sandoz at oracle.com Wed Dec 13 18:06:00 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 13 Dec 2017 10:06:00 -0800 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> Message-ID: <4B169833-7C5E-4144-B0CA-08A5B7FC3C8D@oracle.com> +1 Paul. > On 13 Dec 2017, at 10:03, Brent Christian wrote: > > Hi, > > Way back five days ago[1], TestXcheckJNIWarnings.java was added to the ProblemList. Well, with a couple of recent fixes[2][3], this test can be taken back off of the ProblemList. > > --- a/test/jdk/ProblemList.txt Fri Dec 08 13:04:43 2017 -0800 > +++ b/test/jdk/ProblemList.txt Wed Dec 13 09:28:01 2017 -0800 > @@ -257,7 +257,6 @@ > tools/pack200/CommandLineTests.java 8059906 generic-all > > tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all > -tools/launcher/TestXcheckJNIWarnings.java 8190984 solaris-all > > tools/jimage/JImageExtractTest.java 8170120 generic-all > > Thanks, > -Brent > > 1. http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050496.html > 2. https://bugs.openjdk.java.net/browse/JDK-8193222 > 3. https://bugs.openjdk.java.net/browse/JDK-8190984 > From joe.darcy at oracle.com Wed Dec 13 18:54:31 2017 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 13 Dec 2017 10:54:31 -0800 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> Message-ID: <6f108eb0-5528-ef1c-60c3-e260ecad0f5f@oracle.com> On 12/13/2017 10:03 AM, Brent Christian wrote: > Hi, > > Way back five days ago[1], TestXcheckJNIWarnings.java was added to the > ProblemList. Well, with a couple of recent fixes[2][3], this test can > be taken back off of the ProblemList. Good to see nimble usage of the problem list;,helps keep our test results clean all the time :-) -Joe From claes.redestad at oracle.com Wed Dec 13 19:36:03 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 13 Dec 2017 20:36:03 +0100 Subject: [10!] RFR: 8193471: Startup regression due to JDK-8185582 Message-ID: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> Hi, another startup regression due to very early use of lambdas and method refs that came in with JDK-8185582: Update Zip implementation to use Cleaner, not finalizers Webrev: http://cr.openjdk.java.net/~redestad/8193471/open.00/ /Claes From brent.christian at oracle.com Wed Dec 13 19:45:26 2017 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 13 Dec 2017 11:45:26 -0800 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> Message-ID: <79797d3c-1c75-04e2-f4de-69c2d3978bcd@oracle.com> Thank you, Paul and Mandy. -B From brent.christian at oracle.com Wed Dec 13 19:46:03 2017 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 13 Dec 2017 11:46:03 -0800 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <6f108eb0-5528-ef1c-60c3-e260ecad0f5f@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> <6f108eb0-5528-ef1c-60c3-e260ecad0f5f@oracle.com> Message-ID: <9ad870bf-a536-4151-a398-cd0e99850019@oracle.com> "Nimble" is one word for it... :D -B On 12/13/17 10:54 AM, joe darcy wrote: > > Good to see nimble usage of the problem list;,helps keep our test > results clean all the time :-) > > -Joe From paul.sandoz at oracle.com Wed Dec 13 20:10:57 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 13 Dec 2017 12:10:57 -0800 Subject: [10!] RFR: 8193471: Startup regression due to JDK-8185582 In-Reply-To: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> References: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> Message-ID: +1 Paul. > On 13 Dec 2017, at 11:36, Claes Redestad wrote: > > Hi, > > another startup regression due to very early use of lambdas and method refs that came in with JDK-8185582: Update Zip implementation to use Cleaner, not finalizers > > Webrev: http://cr.openjdk.java.net/~redestad/8193471/open.00/ > > /Claes From Roger.Riggs at Oracle.com Wed Dec 13 20:16:39 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 13 Dec 2017 15:16:39 -0500 Subject: [10!] RFR: 8193471: Startup regression due to JDK-8185582 In-Reply-To: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> References: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> Message-ID: Looks good Thanks, Roger On 12/13/2017 2:36 PM, Claes Redestad wrote: > Hi, > > another startup regression due to very early use of lambdas and method > refs that came in with JDK-8185582: Update Zip implementation to use > Cleaner, not finalizers > > Webrev: http://cr.openjdk.java.net/~redestad/8193471/open.00/ > > /Claes From martinrb at google.com Wed Dec 13 20:17:30 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 13 Dec 2017 12:17:30 -0800 Subject: [10!] RFR: 8193471: Startup regression due to JDK-8185582 In-Reply-To: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> References: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> Message-ID: fyi Google does some automated desugaring for lambdas using a program named ... desugar. From claes.redestad at oracle.com Wed Dec 13 20:40:51 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 13 Dec 2017 21:40:51 +0100 Subject: [10!] RFR: 8193471: Startup regression due to JDK-8185582 In-Reply-To: References: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> Message-ID: <81e9871e-f8db-8ec1-9806-49e006b9ecce@oracle.com> Lambdas and their overheads isn't much of an issue in the grand scheme of things, but forcing the initialization early when not much of the code java.lang.invoke depends on has had a chance to at least warm up somewhat does cause awkward hiccups in startup tests. Thus it makes sense to defer the first initialization somewhat, for now. I hope we'll continue making these overheads smaller so that we in time can use lambdas (or any new language feature) anywhere without regressing even the most synthetic startup tests. /Claes On 2017-12-13 21:17, Martin Buchholz wrote: > fyi Google does some automated desugaring for lambdas using a program > named ... desugar. > From xueming.shen at oracle.com Wed Dec 13 21:44:57 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 13 Dec 2017 13:44:57 -0800 Subject: [10!] RFR: 8193471: Startup regression due to JDK-8185582 In-Reply-To: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> References: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> Message-ID: <5A319F59.8060202@oracle.com> On 12/13/2017 11:36 AM, Claes Redestad wrote: > Hi, > > another startup regression due to very early use of lambdas and method refs that came in with JDK-8185582: Update Zip implementation to use Cleaner, not finalizers > > Webrev: http://cr.openjdk.java.net/~redestad/8193471/open.00/ > > /Claes Maybe it's time to do a "de-lambda" for the In/Deflater/ZStream. ZStream was a simple abstract to wrap the long address in the "old" day. It did not need to know In/Deflater and it was perfectly fine to be shared by Deflater and Inflater. During the work of migrating from finalize() to Cleaner it appears to be a good idea to leverage the ZStream itself to be a cleaner, so we don't have to create a separate "cleaner" object. So it needs to know which "end(long)" to call, and it appears a "LongConsumer" is perfect and cheap approach. The LongSupplier for address is simply a "looks better" (if my memory serves correctly. It might also have something to do with the proposal of adding something into the Cleaner api...) With the "fallback-to-finalize" code added later, it turns out the ZStream also needs to know the "class" of the owner ... Given the fact that we now have to create two objects for each Inflater to workaround the startup "regression", I think it might be the time to stop the sharing and to have separate ZStream for Inflater and Deflater, and put the dedicated "ZStream" back into the Inflater/Deflater as a static private class. We no longer need those lambda. The only cost is to have some bytes for the "duplicated" classes in the storage? http://cr.openjdk.java.net/~sherman/inf_desugar/webrev opinion? sherman From david.holmes at oracle.com Wed Dec 13 22:46:30 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Dec 2017 08:46:30 +1000 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> Message-ID: <374924f3-4003-734b-9f32-87bcf332deb1@oracle.com> Brent, I already took care of this as part of the fix for 8190984. David On 14/12/2017 4:03 AM, Brent Christian wrote: > Hi, > > Way back five days ago[1], TestXcheckJNIWarnings.java was added to the > ProblemList.? Well, with a couple of recent fixes[2][3], this test can > be taken back off of the ProblemList. > > --- a/test/jdk/ProblemList.txt??? Fri Dec 08 13:04:43 2017 -0800 > +++ b/test/jdk/ProblemList.txt??? Wed Dec 13 09:28:01 2017 -0800 > @@ -257,7 +257,6 @@ > ?tools/pack200/CommandLineTests.java 8059906 generic-all > > ?tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all > -tools/launcher/TestXcheckJNIWarnings.java?????????????????????? 8190984 > solaris-all > > ?tools/jimage/JImageExtractTest.java 8170120 generic-all > > Thanks, > -Brent > > 1. > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050496.html > > 2. https://bugs.openjdk.java.net/browse/JDK-8193222 > 3. https://bugs.openjdk.java.net/browse/JDK-8190984 > From david.holmes at oracle.com Wed Dec 13 23:17:13 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Dec 2017 09:17:13 +1000 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <374924f3-4003-734b-9f32-87bcf332deb1@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> <374924f3-4003-734b-9f32-87bcf332deb1@oracle.com> Message-ID: On 14/12/2017 8:46 AM, David Holmes wrote: > Brent, > > I already took care of this as part of the fix for 8190984. Now the test will run again in jdk/jdk CI but the fixes are still only in jdk/hs. David > David > > On 14/12/2017 4:03 AM, Brent Christian wrote: >> Hi, >> >> Way back five days ago[1], TestXcheckJNIWarnings.java was added to the >> ProblemList.? Well, with a couple of recent fixes[2][3], this test can >> be taken back off of the ProblemList. >> >> --- a/test/jdk/ProblemList.txt??? Fri Dec 08 13:04:43 2017 -0800 >> +++ b/test/jdk/ProblemList.txt??? Wed Dec 13 09:28:01 2017 -0800 >> @@ -257,7 +257,6 @@ >> ??tools/pack200/CommandLineTests.java 8059906 generic-all >> >> ??tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all >> -tools/launcher/TestXcheckJNIWarnings.java >> 8190984 solaris-all >> >> ??tools/jimage/JImageExtractTest.java 8170120 generic-all >> >> Thanks, >> -Brent >> >> 1. >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050496.html >> >> 2. https://bugs.openjdk.java.net/browse/JDK-8193222 >> 3. https://bugs.openjdk.java.net/browse/JDK-8190984 >> From brent.christian at oracle.com Wed Dec 13 23:24:10 2017 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 13 Dec 2017 15:24:10 -0800 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> <374924f3-4003-734b-9f32-87bcf332deb1@oracle.com> Message-ID: <88c0cbb0-2c70-4f76-9bbc-cbbe8fced301@oracle.com> On 12/13/17 3:17 PM, David Holmes wrote: >> >> I already took care of this as part of the fix for 8190984. OK, thanks. Well, crud - my apologies to the person doing the merge later. :\ (I saw that the push went to jdk/hs, but didn't look closely. I guess you must have updated from jdk/jdk first...) > Now the test will run again in jdk/jdk CI but the fixes are still only > in jdk/hs. Yeah :{ Hopefully not for too long... -Brent From david.holmes at oracle.com Wed Dec 13 23:28:00 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Dec 2017 09:28:00 +1000 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <88c0cbb0-2c70-4f76-9bbc-cbbe8fced301@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> <374924f3-4003-734b-9f32-87bcf332deb1@oracle.com> <88c0cbb0-2c70-4f76-9bbc-cbbe8fced301@oracle.com> Message-ID: On 14/12/2017 9:24 AM, Brent Christian wrote: > On 12/13/17 3:17 PM, David Holmes wrote: >>> >>> I already took care of this as part of the fix for 8190984. > > OK, thanks. > Well, crud - my apologies to the person doing the merge later.? :\ > > (I saw that the push went to jdk/hs, but didn't look closely.? I guess > you must have updated from jdk/jdk first...) Yes. Doing the pull down from jdk/jdk doesn't cause any issues. >> Now the test will run again in jdk/jdk CI but the fixes are still only >> in jdk/hs. > > Yeah :{ > Hopefully not for too long... At least a week I think as the next push up happens after the next hotspot PIT. David > -Brent From forax at univ-mlv.fr Wed Dec 13 23:44:52 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 14 Dec 2017 00:44:52 +0100 (CET) Subject: [10!] RFR: 8193471: Startup regression due to JDK-8185582 In-Reply-To: References: <75a50996-2a19-dc24-aeaf-d2c0fe550d40@oracle.com> Message-ID: <201133569.1529834.1513208692015.JavaMail.zimbra@u-pem.fr> It's a trade off, desugaring lambdas has several drawbacks, the main one being the desugared class being not unloaded even if the lambda instance is no more accessible. Also, it takes more space on disk, and captured values are not considered as really final by the JIT. You gain a faster startup (usually, it depends how it's desugared exactly) and you do not need a VM that supports invokedynamic*. R?mi * but given that string concatenation in 9, records and pattern matching in the future will also use invokedynamic, at some point your VM will need to support invokedynamic. ----- Mail original ----- > De: "Martin Buchholz" > ?: "Claes Redestad" > Cc: "core-libs-dev" > Envoy?: Mercredi 13 D?cembre 2017 21:17:30 > Objet: Re: [10!] RFR: 8193471: Startup regression due to JDK-8185582 > fyi Google does some automated desugaring for lambdas using a program named > ... desugar. From xueming.shen at oracle.com Thu Dec 14 00:01:41 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 13 Dec 2017 16:01:41 -0800 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 Message-ID: <5A31BF65.8090808@oracle.com> Hi Please help review the change for JDK-8193479 issue: https://bugs.openjdk.java.net/browse/JDK-8193479 webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev The internal interface ArrayEn/Decoder for ISO8859_1 has been removed because it is no longer used by StringCoding class, but it appears it is being used by hotspot Test6896617.java to verify the SSE instructions on x86. The proposed change here is to simply restore the internal interface ArrayEn/Decoder for ISO8859_1 to avoid blocking hotspot testing for now. thanks, Sherman From martinrb at google.com Thu Dec 14 00:52:35 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 13 Dec 2017 16:52:35 -0800 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: <5A31BF65.8090808@oracle.com> References: <5A31BF65.8090808@oracle.com> Message-ID: It would add more confusion to a something that's already difficult to understand. It's OK as a temporary measure, but I don't think we want product code just to enable tests in hotspot. Can the hotspot folks modify their test, perhaps making this code part of the test? On Wed, Dec 13, 2017 at 4:01 PM, Xueming Shen wrote: > Hi > > Please help review the change for JDK-8193479 > > issue: https://bugs.openjdk.java.net/browse/JDK-8193479 > webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev > > The internal interface ArrayEn/Decoder for ISO8859_1 has been removed > because it is no longer used by StringCoding class, but it appears it is > being used by hotspot Test6896617.java to verify the SSE instructions > on x86. The proposed change here is to simply restore the internal > interface > ArrayEn/Decoder for ISO8859_1 to avoid blocking hotspot testing for now. > > thanks, > Sherman > > > From david.holmes at oracle.com Thu Dec 14 01:05:22 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Dec 2017 11:05:22 +1000 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: <5A31BF65.8090808@oracle.com> References: <5A31BF65.8090808@oracle.com> Message-ID: Hi Sherman, On 14/12/2017 10:01 AM, Xueming Shen wrote: > Hi > > Please help review the change for JDK-8193479 > > issue: https://bugs.openjdk.java.net/browse/JDK-8193479 > webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev > > The internal interface ArrayEn/Decoder for ISO8859_1 has been removed > because it is no longer used by StringCoding class, but it appears it is > being used by hotspot Test6896617.java to verify the SSE instructions > on x86. The proposed change here is to simply restore the internal > interface > ArrayEn/Decoder for ISO8859_1 to avoid blocking hotspot testing for now. It would be better to just add the test to the ProblemList.txt IMHO. Thanks, David > thanks, > Sherman > > From xueming.shen at oracle.com Thu Dec 14 02:05:22 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 13 Dec 2017 18:05:22 -0800 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: References: <5A31BF65.8090808@oracle.com> Message-ID: <5A31DC62.60802@oracle.com> David, Martin, webrev has been updated to fix the test directly. http://cr.openjdk.java.net/~sherman/8193479/webrev Assume I would need hotspot guys' help to review and push into hs repo? thanks, Sherman On 12/13/17, 4:52 PM, Martin Buchholz wrote: > It would add more confusion to a something that's already difficult to > understand. It's OK as a temporary measure, but I don't think we want > product code just to enable tests in hotspot. Can the hotspot folks > modify their test, perhaps making this code part of the test? > > On Wed, Dec 13, 2017 at 4:01 PM, Xueming Shen > wrote: > > Hi > > Please help review the change for JDK-8193479 > > issue: https://bugs.openjdk.java.net/browse/JDK-8193479 > > webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev > > > The internal interface ArrayEn/Decoder for ISO8859_1 has been removed > because it is no longer used by StringCoding class, but it appears > it is > being used by hotspot Test6896617.java to verify the SSE instructions > on x86. The proposed change here is to simply restore the internal > interface > ArrayEn/Decoder for ISO8859_1 to avoid blocking hotspot testing > for now. > > thanks, > Sherman > > > From david.holmes at oracle.com Thu Dec 14 02:52:17 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Dec 2017 12:52:17 +1000 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: <5A31DC62.60802@oracle.com> References: <5A31BF65.8090808@oracle.com> <5A31DC62.60802@oracle.com> Message-ID: <0f744ab5-f419-de05-171e-1110ebc886a4@oracle.com> Hi Sherman, On 14/12/2017 12:05 PM, Xueming Shen wrote: > David, Martin, > > webrev has been updated to fix the test directly. > > http://cr.openjdk.java.net/~sherman/8193479/webrev That seems to me to be invalidating the whole point of the test, which was a regression test for 6896617 to check that the optimizations put in place actually get applied. ?? But I'll leave that up to the compiler guys to decide. > Assume I would need hotspot guys' help to review and push into hs repo? You'll need to fix in both jdk/hs and jdk/jdk as the breakage is now in both. Fix one and import to the other. But I still suggest adding the test to ProblemList.txt now so that this isn't broken in JDK 10 fork, then fix the actual test at a more leisurely pace in jdk/hs. But again I'll defer to compiler folk. David > thanks, > Sherman > > > On 12/13/17, 4:52 PM, Martin Buchholz wrote: >> It would add more confusion to a something that's already difficult to >> understand.? It's OK as a temporary measure, but I don't think we want >> product code just to enable tests in hotspot.? Can the hotspot folks >> modify their test, perhaps making this code part of the test? >> >> On Wed, Dec 13, 2017 at 4:01 PM, Xueming Shen > > wrote: >> >> Hi >> >> Please help review the change for JDK-8193479 >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8193479 >> >> webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev >> >> >> The internal interface ArrayEn/Decoder for ISO8859_1 has been removed >> because it is no longer used by StringCoding class, but it appears >> it is >> being used by hotspot Test6896617.java to verify the SSE instructions >> on x86. The proposed change here is to simply restore the internal >> interface >> ArrayEn/Decoder for ISO8859_1 to avoid blocking hotspot testing >> for now. >> >> thanks, >> Sherman >> >> >> > From david.holmes at oracle.com Thu Dec 14 02:53:41 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Dec 2017 12:53:41 +1000 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> <374924f3-4003-734b-9f32-87bcf332deb1@oracle.com> <88c0cbb0-2c70-4f76-9bbc-cbbe8fced301@oracle.com> Message-ID: <06b6db06-13b0-0a43-6bb9-dc263cf7953c@oracle.com> Brent, I'm going to manually import the missing changes from jdk/hs to jdk/jdk. I've conferred with Jesper and this shouldn't cause any problems. Though there may be a rather large merge changeset due to the just pushed changes to jdk/jdk. David On 14/12/2017 9:28 AM, David Holmes wrote: > On 14/12/2017 9:24 AM, Brent Christian wrote: >> On 12/13/17 3:17 PM, David Holmes wrote: >>>> >>>> I already took care of this as part of the fix for 8190984. >> >> OK, thanks. >> Well, crud - my apologies to the person doing the merge later.? :\ >> >> (I saw that the push went to jdk/hs, but didn't look closely.? I guess >> you must have updated from jdk/jdk first...) > > Yes. Doing the pull down from jdk/jdk doesn't cause any issues. > >>> Now the test will run again in jdk/jdk CI but the fixes are still >>> only in jdk/hs. >> >> Yeah :{ >> Hopefully not for too long... > > At least a week I think as the next push up happens after the next > hotspot PIT. > > David > >> -Brent From david.holmes at oracle.com Thu Dec 14 04:59:10 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Dec 2017 14:59:10 +1000 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <06b6db06-13b0-0a43-6bb9-dc263cf7953c@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> <374924f3-4003-734b-9f32-87bcf332deb1@oracle.com> <88c0cbb0-2c70-4f76-9bbc-cbbe8fced301@oracle.com> <06b6db06-13b0-0a43-6bb9-dc263cf7953c@oracle.com> Message-ID: <5d8e970c-5687-9004-88ac-2a537890fae9@oracle.com> On 14/12/2017 12:53 PM, David Holmes wrote: > Brent, > > I'm going to manually import the missing changes from jdk/hs to jdk/jdk. > I've conferred with Jesper and this shouldn't cause any problems. Though > there may be a rather large merge changeset due to the just pushed > changes to jdk/jdk. Unfortunately this was not possible as the changeset for 8190984 conflicts with this changeset (8193460). So the test will fail again in jdk/jdk until the changes make their way up from jdk/hs. If that is a concern then you will need to backout this changeset so that the test is problem-listed again. David > David > > On 14/12/2017 9:28 AM, David Holmes wrote: >> On 14/12/2017 9:24 AM, Brent Christian wrote: >>> On 12/13/17 3:17 PM, David Holmes wrote: >>>>> >>>>> I already took care of this as part of the fix for 8190984. >>> >>> OK, thanks. >>> Well, crud - my apologies to the person doing the merge later.? :\ >>> >>> (I saw that the push went to jdk/hs, but didn't look closely.? I >>> guess you must have updated from jdk/jdk first...) >> >> Yes. Doing the pull down from jdk/jdk doesn't cause any issues. >> >>>> Now the test will run again in jdk/jdk CI but the fixes are still >>>> only in jdk/hs. >>> >>> Yeah :{ >>> Hopefully not for too long... >> >> At least a week I think as the next push up happens after the next >> hotspot PIT. >> >> David >> >>> -Brent From Alan.Bateman at oracle.com Thu Dec 14 08:30:30 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Dec 2017 08:30:30 +0000 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: <5A31BF65.8090808@oracle.com> References: <5A31BF65.8090808@oracle.com> Message-ID: On 14/12/2017 00:01, Xueming Shen wrote: > Hi > > Please help review the change for JDK-8193479 > > issue: https://bugs.openjdk.java.net/browse/JDK-8193479 > webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev > > The internal interface ArrayEn/Decoder for ISO8859_1 has been removed > because it is no longer used by StringCoding class, but it appears it is > being used by hotspot Test6896617.java to verify the SSE instructions > on x86. The proposed change here is to simply restore the internal > interface > ArrayEn/Decoder for ISO8859_1 to avoid blocking hotspot testing for now. I can't tell if this update makes use of the SSE instructions or not. Would it be better to just exclude the test by adding to the ProblemList.txt and create a hotspot/compiler bug to get it updated to work with the new implementation? -Alan From claes.redestad at oracle.com Thu Dec 14 11:07:57 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 14 Dec 2017 12:07:57 +0100 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 Message-ID: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> Hi, my previous fix failed due to use of non-static inner classes which kept the cleanable objects around. Also, Sherman suggested a more thorough fix to Inflater/Deflater after I had already pushed. Webrev: http://cr.openjdk.java.net/~redestad/8193507/open.00/ Verified all java/util/zip tests pass this time. /Claes From Alan.Bateman at oracle.com Thu Dec 14 12:00:53 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Dec 2017 12:00:53 +0000 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> Message-ID: On 14/12/2017 11:07, Claes Redestad wrote: > Hi, > > my previous fix failed due to use of non-static inner classes which > kept the cleanable objects around. Also, Sherman suggested a more > thorough fix to Inflater/Deflater after I had already pushed. > > Webrev: http://cr.openjdk.java.net/~redestad/8193507/open.00/ > > Verified all java/util/zip tests pass this time. The changes means some code duplication but it's not too bad. Can the ZStreamRef(long) constructor go away so that we don't need to think about creating a ZStreamRef without a cleanable. Also can the fields in InflaterCleanupAction be final. -Alan From claes.redestad at oracle.com Thu Dec 14 12:25:14 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 14 Dec 2017 13:25:14 +0100 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> Message-ID: <1e4ca4d2-6e68-ea62-3843-b37dc2701d21@oracle.com> On 2017-12-14 13:00, Alan Bateman wrote: > On 14/12/2017 11:07, Claes Redestad wrote: >> Hi, >> >> my previous fix failed due to use of non-static inner classes which >> kept the cleanable objects around. Also, Sherman suggested a more >> thorough fix to Inflater/Deflater after I had already pushed. >> >> Webrev: http://cr.openjdk.java.net/~redestad/8193507/open.00/ >> >> Verified all java/util/zip tests pass this time. > The changes means some code duplication but it's not too bad. Can the > ZStreamRef(long) constructor go away so that we don't need to think > about creating a ZStreamRef without a cleanable. We need the cleanable = null case for the FinalizableZStreamRef case, otherwise we'd go into spin. The cleanable is implicitly checked for null in the ZStreamRef.get method, which is the only one used outside of these classes. > Also can the fields in InflaterCleanupAction be final. Sure: http://cr.openjdk.java.net/~redestad/8193507/open.01/ /Claes From Alan.Bateman at oracle.com Thu Dec 14 12:29:29 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Dec 2017 12:29:29 +0000 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <1e4ca4d2-6e68-ea62-3843-b37dc2701d21@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <1e4ca4d2-6e68-ea62-3843-b37dc2701d21@oracle.com> Message-ID: On 14/12/2017 12:25, Claes Redestad wrote: > > We need the cleanable = null case for the FinalizableZStreamRef case, > otherwise we'd go into spin. The cleanable is implicitly checked for > null in the ZStreamRef.get method, which is the only one > used outside of these classes. Okay but can the clean method at least check if cleanable is null to avoid needing to wonder if it's possible. Also can the constructors be grouped as it's very confusing to not have them grouped together (dropping the space in "ZStreamRef (" would help too. Otherwise I think the approach is okay. -Alan From claes.redestad at oracle.com Thu Dec 14 12:54:30 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 14 Dec 2017 13:54:30 +0100 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <1e4ca4d2-6e68-ea62-3843-b37dc2701d21@oracle.com> Message-ID: <6684929a-231a-27c4-491d-6efb1445f4ef@oracle.com> On 2017-12-14 13:29, Alan Bateman wrote: > On 14/12/2017 12:25, Claes Redestad wrote: >> >> We need the cleanable = null case for the FinalizableZStreamRef case, >> otherwise we'd go into spin. The cleanable is implicitly checked for >> null in the ZStreamRef.get method, which is the only one >> used outside of these classes. > Okay but can the clean method at least check if cleanable is null to > avoid needing to wonder if it's possible. It isn't possible: cleanable can be null only when we're a FinalizableZStreamRef, which overrides clean and ignores cleanable. This awkwardness stems from the fact that we *don't* want ZStreamRef to have a finalizer, so we need to inherit in the "unnatural" direction. We could possibly model this more cleanly with an interface and two disjoint implementations, but I don't want to attempt such a refactoring now. > Also can the constructors be grouped as it's very confusing to not > have them grouped together (dropping the space in "ZStreamRef (" would > help too. Otherwise I think the approach is okay. I've dropped the ZStreamRef(long) constructors and fixed a few whitespace issues in the latest webrev. /Claes From Alan.Bateman at oracle.com Thu Dec 14 14:16:53 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Dec 2017 14:16:53 +0000 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <6684929a-231a-27c4-491d-6efb1445f4ef@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <1e4ca4d2-6e68-ea62-3843-b37dc2701d21@oracle.com> <6684929a-231a-27c4-491d-6efb1445f4ef@oracle.com> Message-ID: <99d9f615-7f83-fd8b-03a9-2fda664d9a9b@oracle.com> On 14/12/2017 12:54, Claes Redestad wrote: > : > >> Also can the constructors be grouped as it's very confusing to not >> have them grouped together (dropping the space in "ZStreamRef (" >> would help too. Otherwise I think the approach is okay. > > I've dropped the ZStreamRef(long) constructors and fixed a few > whitespace issues in the latest webrev. webrev.01 looks good to me. -Alan From claes.redestad at oracle.com Thu Dec 14 14:20:22 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 14 Dec 2017 15:20:22 +0100 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <99d9f615-7f83-fd8b-03a9-2fda664d9a9b@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <1e4ca4d2-6e68-ea62-3843-b37dc2701d21@oracle.com> <6684929a-231a-27c4-491d-6efb1445f4ef@oracle.com> <99d9f615-7f83-fd8b-03a9-2fda664d9a9b@oracle.com> Message-ID: <67e6f353-3ff5-73e4-eef2-d70809619867@oracle.com> On 2017-12-14 15:16, Alan Bateman wrote: > webrev.01 looks good to me. Thanks, Alan! /Claes From Roger.Riggs at Oracle.com Thu Dec 14 15:03:09 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 14 Dec 2017 10:03:09 -0500 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <67e6f353-3ff5-73e4-eef2-d70809619867@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <1e4ca4d2-6e68-ea62-3843-b37dc2701d21@oracle.com> <6684929a-231a-27c4-491d-6efb1445f4ef@oracle.com> <99d9f615-7f83-fd8b-03a9-2fda664d9a9b@oracle.com> <67e6f353-3ff5-73e4-eef2-d70809619867@oracle.com> Message-ID: Hi Claes, Looks good; though I would rename the nested classes to have different names to avoid reader confusion.? Like InflaterZStreamRef and DeflaterZStreamRef. No further review if you take up the renames. Thanks, Roger On 12/14/2017 9:20 AM, Claes Redestad wrote: > On 2017-12-14 15:16, Alan Bateman wrote: >> webrev.01 looks good to me. > > Thanks, Alan! > > /Claes From claes.redestad at oracle.com Thu Dec 14 15:09:22 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 14 Dec 2017 16:09:22 +0100 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <1e4ca4d2-6e68-ea62-3843-b37dc2701d21@oracle.com> <6684929a-231a-27c4-491d-6efb1445f4ef@oracle.com> <99d9f615-7f83-fd8b-03a9-2fda664d9a9b@oracle.com> <67e6f353-3ff5-73e4-eef2-d70809619867@oracle.com> Message-ID: <8d9e81da-35a7-d912-0165-31f42dfdce8f@oracle.com> On 2017-12-14 16:03, Roger Riggs wrote: > Hi Claes, > > Looks good; though I would rename the nested classes to have different > names > to avoid reader confusion.? Like InflaterZStreamRef and > DeflaterZStreamRef. Sure: http://cr.openjdk.java.net/~redestad/8193507/open.02/ > > No further review if you take up the renames. I'll go ahead and push once tests complete. :-) Thanks! /Claes From hamada.kenai at gmail.com Thu Dec 14 17:03:32 2017 From: hamada.kenai at gmail.com (Hamada Abdelaziz) Date: Thu, 14 Dec 2017 09:03:32 -0800 Subject: StringBuilder default constructor in the class library Message-ID: While analyzing GC pressure of a run time, I noticed the top object being a char[], upon a closer inspection, it turns out to be a result of ObjectStreamClass use of StringBuilder, which is constructed with the default size of 16 bytes.? In such case 16 bytes is not a sufficient size, which leads to unnecessary garbage and incurs a performance hit. In a high volume situation this leads to high GC pressure, and in turn, non uniform performance. It's worthwhile revisiting all uses of StringBuilder default constructor in the class library and adjusting it appropriately. Thoughts? From xueming.shen at oracle.com Thu Dec 14 17:51:06 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 14 Dec 2017 09:51:06 -0800 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: References: <5A31BF65.8090808@oracle.com> Message-ID: <5A32BA0A.6060305@oracle.com> It appears the consensus is to put this one into the ProblemList and let the hotspot folks to fix the test case. issue: https://bugs.openjdk.java.net/browse/JDK-8193479 webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev The corresponding hotspot/compiler issue filed: JDK-8193549. I would assume now it needs to get into 11 and backport to 10 repo. Thanks, Sherman From vladimir.kozlov at oracle.com Thu Dec 14 17:58:53 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 14 Dec 2017 09:58:53 -0800 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: <0f744ab5-f419-de05-171e-1110ebc886a4@oracle.com> References: <5A31BF65.8090808@oracle.com> <5A31DC62.60802@oracle.com> <0f744ab5-f419-de05-171e-1110ebc886a4@oracle.com> Message-ID: <055440af-39eb-38e6-bd9f-41102868dbe8@oracle.com> On 12/13/17 6:52 PM, David Holmes wrote: > Hi Sherman, > > On 14/12/2017 12:05 PM, Xueming Shen wrote: >> David, Martin, >> >> webrev has been updated to fix the test directly. >> >> http://cr.openjdk.java.net/~sherman/8193479/webrev > > That seems to me to be invalidating the whole point of the test, which > was a regression test for 6896617 to check that the optimizations put in > place actually get applied. ?? > > But I'll leave that up to the compiler guys to decide. > >> Assume I would need hotspot guys' help to review and push into hs repo? > > You'll need to fix in both jdk/hs and jdk/jdk as the breakage is now in > both. Fix one and import to the other. > > But I still suggest adding the test to ProblemList.txt now so that this > isn't broken in JDK 10 fork, then fix the actual test at a more > leisurely pace in jdk/hs. I agree with David. Lets exclude it for now and fix it later properly without rush. Thanks Vladimir > > But again I'll defer to compiler folk. > > David > >> thanks, >> Sherman >> >> >> On 12/13/17, 4:52 PM, Martin Buchholz wrote: >>> It would add more confusion to a something that's already difficult >>> to understand.? It's OK as a temporary measure, but I don't think we >>> want product code just to enable tests in hotspot.? Can the hotspot >>> folks modify their test, perhaps making this code part of the test? >>> >>> On Wed, Dec 13, 2017 at 4:01 PM, Xueming Shen >>> > wrote: >>> >>> ??? Hi >>> >>> ??? Please help review the change for JDK-8193479 >>> >>> ??? issue: https://bugs.openjdk.java.net/browse/JDK-8193479 >>> ??? >>> ??? webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev >>> ??? >>> >>> ??? The internal interface ArrayEn/Decoder for ISO8859_1 has been >>> removed >>> ??? because it is no longer used by StringCoding class, but it appears >>> ??? it is >>> ??? being used by hotspot Test6896617.java to verify the SSE >>> instructions >>> ??? on x86. The proposed change here is to simply restore the internal >>> ??? interface >>> ??? ArrayEn/Decoder for ISO8859_1 to avoid blocking hotspot testing >>> ??? for now. >>> >>> ??? thanks, >>> ??? Sherman >>> >>> >>> >> From mandy.chung at oracle.com Thu Dec 14 19:29:28 2017 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 14 Dec 2017 11:29:28 -0800 Subject: Review Request JDK-8193325: StackFrameInfo::getByteCodeIndex returns wrong value if bci > 32767 In-Reply-To: <4E31CF3C-D7BF-4070-85F5-66D281478F6E@oracle.com> References: <4cf575a1-d420-fb44-cc38-03287366ec3d@oracle.com> <8a326b47-176b-c41a-7fe1-d9d53f37bd5c@oracle.com> <4E31CF3C-D7BF-4070-85F5-66D281478F6E@oracle.com> Message-ID: <0a060ad6-5444-cbe2-7124-a3a4809d14a4@oracle.com> On 12/12/17 5:29 PM, John Rose wrote: > On Dec 12, 2017, at 5:24 PM, David Holmes > wrote: >> >> as per getByteCodeIndex() it needs to be able to store "-1" to >> indicate no-bci? > > Yes, that's the essential requirement, to encode a "none" value. > > FWIW, one natural way to do that here would be to make the int field > @Stable and have the accessor return one less than the stored field value. > > Then, by the rules of @Stable, the field starts out at the logical value > of -1, and can transition to a valid BCI if patched to a non-zero value > in the range 1..(2^16). > Thanks for suggesting @Stable that would help this case.? I can't understand why the field has to be patched with non-zero value.? Is this something special requirement for @Stable? > Making it "final" and initialized to -1 is not too terrible either, > although > that creates some tech. debt we need to discharge when upgrading > finals to be trustable. Mandy [1] http://cr.openjdk.java.net/~bchristi/8185925/StackFrameInfo.jol.rmDeclClass.txt From brent.christian at oracle.com Thu Dec 14 19:32:48 2017 From: brent.christian at oracle.com (Brent Christian) Date: Thu, 14 Dec 2017 11:32:48 -0800 Subject: RFR[10] 8193460 : Take tools/launcher/TestXcheckJNIWarnings.java back off the ProblemList In-Reply-To: <5d8e970c-5687-9004-88ac-2a537890fae9@oracle.com> References: <78b1a06a-f637-44a7-b1c1-ae1b38c2b4fb@oracle.com> <374924f3-4003-734b-9f32-87bcf332deb1@oracle.com> <88c0cbb0-2c70-4f76-9bbc-cbbe8fced301@oracle.com> <06b6db06-13b0-0a43-6bb9-dc263cf7953c@oracle.com> <5d8e970c-5687-9004-88ac-2a537890fae9@oracle.com> Message-ID: <323a2c92-860e-bb19-935c-e47564a6c05c@oracle.com> On 12/13/17 8:59 PM, David Holmes wrote: >> >> I'm going to manually import the missing changes from jdk/hs to >> jdk/jdk. > > Unfortunately this was not possible as the changeset for 8190984 > conflicts with this changeset (8193460). So the test will fail again in > jdk/jdk until the changes make their way up from jdk/hs. If that is a > concern then you will need to backout this changeset so that the test is > problem-listed again. Thanks for giving it a try. My view is that we made it through several weeks of having this intermittent failure on one platform. IMO we can survive with it for a little while longer until the merge. I've described the situation in the bug report. Thanks, -Brent From Roger.Riggs at Oracle.com Thu Dec 14 20:02:20 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 14 Dec 2017 15:02:20 -0500 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: <5A32BA0A.6060305@oracle.com> References: <5A31BF65.8090808@oracle.com> <5A32BA0A.6060305@oracle.com> Message-ID: <9a54d901-3850-c05b-1339-5bf5311d1924@Oracle.com> +1 On 12/14/2017 12:51 PM, Xueming Shen wrote: > > It appears the consensus is to put this one into the ProblemList and > let the > hotspot folks to fix the test case. > > issue: https://bugs.openjdk.java.net/browse/JDK-8193479 > webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev > > The corresponding hotspot/compiler issue filed: JDK-8193549. > > I would assume now it needs to get into 11 and backport to 10 repo. > > Thanks, > Sherman From Roger.Riggs at Oracle.com Thu Dec 14 20:08:13 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 14 Dec 2017 15:08:13 -0500 Subject: StringBuilder default constructor in the class library In-Reply-To: References: Message-ID: Hi Hamada , Looking at initial sizes (of StringBuilder) would be a useful contribution. Take a look at http://openjdk.java.net/contribute/ for how to contribute. Regards, Roger On 12/14/2017 12:03 PM, Hamada Abdelaziz wrote: > > While analyzing GC pressure of a run time, I noticed the top object > being a char[], upon a closer inspection, it turns out to be a result > of ObjectStreamClass use of StringBuilder, which is constructed with > the default size of 16 bytes.? In such case 16 bytes is not a > sufficient size, which leads to unnecessary garbage and incurs a > performance hit. > > In a high volume situation this leads to high GC pressure, and in > turn, non uniform performance. > > It's worthwhile revisiting all uses of StringBuilder default > constructor in the class library and adjusting it appropriately. > > Thoughts? > > From xueming.shen at oracle.com Thu Dec 14 20:46:52 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 14 Dec 2017 12:46:52 -0800 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: <055440af-39eb-38e6-bd9f-41102868dbe8@oracle.com> References: <5A31BF65.8090808@oracle.com> <5A31DC62.60802@oracle.com> <0f744ab5-f419-de05-171e-1110ebc886a4@oracle.com> <055440af-39eb-38e6-bd9f-41102868dbe8@oracle.com> Message-ID: <5A32E33C.7070904@oracle.com> On 12/14/17, 9:58 AM, Vladimir Kozlov wrote: > On 12/13/17 6:52 PM, David Holmes wrote: >> Hi Sherman, >> >> On 14/12/2017 12:05 PM, Xueming Shen wrote: >>> David, Martin, >>> >>> webrev has been updated to fix the test directly. >>> >>> http://cr.openjdk.java.net/~sherman/8193479/webrev >> >> That seems to me to be invalidating the whole point of the test, >> which was a regression test for 6896617 to check that the >> optimizations put in place actually get applied. ?? >> >> But I'll leave that up to the compiler guys to decide. >> >>> Assume I would need hotspot guys' help to review and push into hs repo? >> >> You'll need to fix in both jdk/hs and jdk/jdk as the breakage is now >> in both. Fix one and import to the other. >> >> But I still suggest adding the test to ProblemList.txt now so that >> this isn't broken in JDK 10 fork, then fix the actual test at a more >> leisurely pace in jdk/hs. > > I agree with David. Lets exclude it for now and fix it later properly > without rush. > > Thanks > Vladimir > Thanks Vladimir, I have a webrev out there. But I will probably have to wait for the new jdk10 repo open next week. Or maybe the hs repo is still open for something like this? and you guys can help get it in directly there? Sherman -------------------------------------------- It appears the consensus is to put this one into the ProblemList and let the hotspot folks to fix the test case. issue: https://bugs.openjdk.java.net/browse/JDK-8193479 webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev The corresponding hotspot/compiler issue filed: JDK-8193549. Thanks, Sherman ---------------------------------------------- From david.buck at oracle.com Fri Dec 15 05:42:42 2017 From: david.buck at oracle.com (David Buck) Date: Fri, 15 Dec 2017 14:42:42 +0900 Subject: [8u] RFR(S) 8059036: Implement Diagnostic Commands for heap and finalizerinfo Message-ID: <0f3f7d99-fccb-0305-4939-6a77d110b908@oracle.com> Hi! May I please get a review of the following very simple backport of this serviceability improvement to JDK 8? The two hotspot jtreg test cases needed to be modified slightly because of the lack of dcmd-specific test support in JDK 8's HS code base. The only non-test difference from the JDK 9 changeset is modifying an include directive (objArrayOop.inline.hpp -> objArrayOop.hpp) to compensate for JDK-8072911. bug report: https://bugs.openjdk.java.net/browse/JDK-8059036 JDK 9 changesets: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/56a527afc34a http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/9b17d0a2720f JDK 9 review thread: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033169.html http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033835.html JDK 8 backport webrevs for review: http://cr.openjdk.java.net/~dbuck/8059036.0_jdk8/ Cheers, -Buck [0] https://bugs.openjdk.java.net/browse/JDK-8072911 From tobias.hartmann at oracle.com Fri Dec 15 09:48:45 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 15 Dec 2017 10:48:45 +0100 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: <5A32E33C.7070904@oracle.com> References: <5A31BF65.8090808@oracle.com> <5A31DC62.60802@oracle.com> <0f744ab5-f419-de05-171e-1110ebc886a4@oracle.com> <055440af-39eb-38e6-bd9f-41102868dbe8@oracle.com> <5A32E33C.7070904@oracle.com> Message-ID: Hi Sherman, On 14.12.2017 21:46, Xueming Shen wrote: > I have a webrev out there. But I will probably have to wait for the > new jdk10 repo open next week. > > Or maybe the hs repo is still open for something like this? and > you guys can help get it in directly there? I would say it's best to wait for the jdk10 repo to open. > It appears the consensus is to put this one into the ProblemList and let the > hotspot folks to fix the test case. > > issue: https://bugs.openjdk.java.net/browse/JDK-8193479 > webrev: http://cr.openjdk.java.net/~sherman/8193479/webrev > > The corresponding hotspot/compiler issue filed: JDK-8193549. The correct way is to create a subtask to quarantine the test. I've created one for you and closed (JDK-8193549) as duplicate: https://bugs.openjdk.java.net/browse/JDK-8193608 Please un-assign 8193479 if you don't plan to fix the test. Thanks, Tobias From tobias.hartmann at oracle.com Fri Dec 15 15:08:49 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 15 Dec 2017 16:08:49 +0100 Subject: RFR JDK-8193479: test/hotspot/jtreg/compiler/codegen/Test6896617.java fails after JDK-8184947 In-Reply-To: References: <5A31BF65.8090808@oracle.com> <5A31DC62.60802@oracle.com> <0f744ab5-f419-de05-171e-1110ebc886a4@oracle.com> <055440af-39eb-38e6-bd9f-41102868dbe8@oracle.com> <5A32E33C.7070904@oracle.com> Message-ID: <33386751-cea5-6b01-f1a6-060d1c298d4c@oracle.com> Hi Sherman, On 15.12.2017 10:48, Tobias Hartmann wrote: > I would say it's best to wait for the jdk10 repo to open. I've checked with Jesper and we can/should quarantine this test right away. I'll take care of it [1]. Best regards, Tobias [1] http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2017-December/027933.html From paul.sandoz at oracle.com Fri Dec 15 20:29:16 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Dec 2017 12:29:16 -0800 Subject: [11] RFR 8193085 Vectorize the nio Buffer equals and compareTo implementations Message-ID: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> Hi, Please review this patch to vectorize the buffer equals and compareTo implementations, using the same approach that was used for arrays: http://cr.openjdk.java.net/~psandoz/jdk/JDK-8193085-buffer-equals-compareTo-vectorize/webrev/ This patch expands on using the double address mode of unsafe to uniformly access a memory region covered by a buffer be it on or off heap. Only buffers with the same endianness can support optimized equality and comparison. ? A follow on issue will explore a mismatch method (thus enabling support for float/double comparison that is compatible with arrays), and equals/compareTo methods that accept absolute positions (to avoid the dup.pos.limit.slice dance). Thanks, Paul. From david.lloyd at redhat.com Fri Dec 15 21:01:43 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 15 Dec 2017 15:01:43 -0600 Subject: [11] RFR 8193085 Vectorize the nio Buffer equals and compareTo implementations In-Reply-To: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> References: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> Message-ID: I'm not a reviewer, but I was curious about this change; unfortunately the diff seems to be dominated by case and formatting changes making the actual functional aspect change hard to divine. Within the JBoss unit we have an informal policy that formatting changes should be presented separately so that it's easier to trace back problems in the future, as well as being much easier to review the change in the first place. Would I be stepping out of bounds to suggest that this change should be similarly divided? On Fri, Dec 15, 2017 at 2:29 PM, Paul Sandoz wrote: > Hi, > > Please review this patch to vectorize the buffer equals and compareTo implementations, using the same approach that was used for arrays: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8193085-buffer-equals-compareTo-vectorize/webrev/ > > This patch expands on using the double address mode of unsafe to uniformly access a memory region covered by a buffer be it on or off heap. Only buffers with the same endianness can support optimized equality and comparison. > > ? > > A follow on issue will explore a mismatch method (thus enabling support for float/double comparison that is compatible with arrays), and equals/compareTo methods that accept absolute positions (to avoid the dup.pos.limit.slice dance). > > Thanks, > Paul. -- - DML From paul.sandoz at oracle.com Fri Dec 15 22:21:45 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Dec 2017 14:21:45 -0800 Subject: [11] RFR 8193085 Vectorize the nio Buffer equals and compareTo implementations In-Reply-To: References: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> Message-ID: > On 15 Dec 2017, at 13:01, David Lloyd wrote: > > I'm not a reviewer, but I was curious about this change; unfortunately > the diff seems to be dominated by case and formatting changes making > the actual functional aspect change hard to divine. > If not already i recommend viewing via udiffs, i find that makes it easier. > Within the JBoss unit we have an informal policy that formatting > changes should be presented separately so that it's easier to trace > back problems in the future, as well as being much easier to review > the change in the first place. Would I be stepping out of bounds to > suggest that this change should be similarly divided? > In hindsight :-) at this point i would prefer not to split it unless reviewers are having a really hard time. I was furiously hacking on this and got fed up with the names making it harder for me to reason about the code so i changed ?em mid-flight when doing this work. Thanks, Paul. From peter.levart at gmail.com Fri Dec 15 23:43:26 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 16 Dec 2017 00:43:26 +0100 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> Message-ID: <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> Hi Claes, I see this is already pushed. I don't have any additional comments, but just want to know what was wrong with old code. You say that you used non-static inner classes. I don't see that in old code. All the lambdas you replaced with nested static classes should have already captured just local variables. I must be missing something and I would really like to know what. Regards, Peter Claes Redestad je 14. 12. 2017 ob 12:07?napisal: > Hi, > > my previous fix failed due to use of non-static inner classes which > kept the cleanable objects around. Also, Sherman suggested a more > thorough fix to Inflater/Deflater after I had already pushed. > > Webrev: http://cr.openjdk.java.net/~redestad/8193507/open.00/ > > Verified all java/util/zip tests pass this time. > > /Claes > > From xueming.shen at oracle.com Sat Dec 16 00:46:00 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 15 Dec 2017 16:46:00 -0800 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> Message-ID: <5A346CC8.9010008@oracle.com> On 12/15/17, 3:43 PM, Peter Levart wrote: > Hi Claes, > > I see this is already pushed. I don't have any additional comments, > but just want to know what was wrong with old code. You say that you > used non-static inner classes. I don't see that in old code. All the > lambdas you replaced with nested static classes should have already > captured just local variables. I must be missing something and I would > really like to know what. Perter, Since that code triggered all zip/cleaner regression tests failed. I rollback-ed that fix overnight to make our overnight tests happy. So what in the webrev is against my original code. Here is my webrev that undo-ed the non-static inner classes. http://cr.openjdk.java.net/~sherman/8193490/webrev/ Sherman > > Regards, Peter > > Claes Redestad je 14. 12. 2017 ob 12:07 napisal: >> Hi, >> >> my previous fix failed due to use of non-static inner classes which >> kept the cleanable objects around. Also, Sherman suggested a more >> thorough fix to Inflater/Deflater after I had already pushed. >> >> Webrev: http://cr.openjdk.java.net/~redestad/8193507/open.00/ >> >> Verified all java/util/zip tests pass this time. >> >> /Claes >> >> > From peter.levart at gmail.com Sat Dec 16 10:22:12 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 16 Dec 2017 11:22:12 +0100 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <5A346CC8.9010008@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> <5A346CC8.9010008@oracle.com> Message-ID: <8d8b1f1a-e258-7b4a-fa00-4f47702717ca@gmail.com> Hi Sherman, Xueming Shen je 16. 12. 2017 ob 01:46?napisal: > On 12/15/17, 3:43 PM, Peter Levart wrote: >> Hi Claes, >> >> I see this is already pushed. I don't have any additional comments, >> but just want to know what was wrong with old code. You say that you >> used non-static inner classes. I don't see that in old code. All the >> lambdas you replaced with nested static classes should have already >> captured just local variables. I must be missing something and I >> would really like to know what. > > Perter, > > Since that code triggered all zip/cleaner regression tests failed. I > rollback-ed that > fix overnight to make our overnight tests happy. So what in the webrev > is against my > original code. > > > Here is my webrev that undo-ed the non-static inner classes. > > http://cr.openjdk.java.net/~sherman/8193490/webrev/ > > Sherman I see. So the 1st change was an attempt to fix a startup performance regression (JDK-8193471) by replacing lambdas with anonymous inner classes, which unfortunately capture 'this' if they are constructed in instance context. I'm sorry I was busy and haven't noticed this RFR or I would probably spot the mistake. The 2nd change is a re-attempt to do this with static classes (JDK-8193507). This fixes the startup performance regression problem, but it also re-introduces another one, which was carefully avoided by using the lambdas in the initial version. The problem is that in latest version of code, the initialization order is: - init(nowrap) - Cleaner registration while in lambda version it was the other way around: - Cleaner registration - init(nowrap) If the registration of Cleanable fails, the latest version of code leaks a native resource. Now that you have specialized ZStreamRef classes, you could post-pone the native resource allocation in a simple way, directly in the XxxZStreamRef constructor: Index: src/java.base/share/classes/java/util/zip/Inflater.java IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- src/java.base/share/classes/java/util/zip/Inflater.java (revision 48342:003d6365ec6a529616d0e5b119144b4408c3a1c8) +++ src/java.base/share/classes/java/util/zip/Inflater.java (revision 48342+:003d6365ec6a+) @@ -118,7 +118,7 @@ ????? * @param nowrap if true then support GZIP compatible compression ????? */ ???? public Inflater(boolean nowrap) { -??????? this.zsRef = InflaterZStreamRef.get(this, init(nowrap)); +??????? this.zsRef = InflaterZStreamRef.get(this, nowrap); ???? } ???? /** @@ -442,9 +442,9 @@ ???????? private long address; ???????? private final Cleanable cleanable; -??????? private InflaterZStreamRef(Inflater owner, long addr) { +??????? private InflaterZStreamRef(Inflater owner, boolean nowrap) { ???????????? this.cleanable = (owner != null) ? CleanerFactory.cleaner().register(owner, this) : null; -??????????? this.address = addr; +??????????? this.address = init(nowrap); ???????? } ???????? long address() { @@ -470,23 +470,23 @@ ????????? * This mechanism will be removed when the {@code finalize} method is ????????? * removed from {@code Inflater}. ????????? */ -??????? static InflaterZStreamRef get(Inflater owner, long addr) { +??????? static InflaterZStreamRef get(Inflater owner, boolean nowrap) { ???????????? Class clz = owner.getClass(); ???????????? while (clz != Inflater.class) { ???????????????? try { ???????????????????? clz.getDeclaredMethod("end"); -??????????????????? return new FinalizableZStreamRef(owner, addr); +??????????????????? return new FinalizableZStreamRef(owner, nowrap); ???????????????? } catch (NoSuchMethodException nsme) {} ???????????????? clz = clz.getSuperclass(); ???????????? } -??????????? return new InflaterZStreamRef(owner, addr); +??????????? return new InflaterZStreamRef(owner, nowrap); ???????? } ???????? private static class FinalizableZStreamRef extends InflaterZStreamRef { ???????????? final Inflater owner; -??????????? FinalizableZStreamRef(Inflater owner, long addr) { -??????????????? super(null, addr); +??????????? FinalizableZStreamRef(Inflater owner, boolean nowrap) { +??????????????? super(null, nowrap); ???????????????? this.owner = owner; ???????????? } This could be a refinement of the last patch. What do you think? Regards, Peter > >> >> Regards, Peter >> >> Claes Redestad je 14. 12. 2017 ob 12:07 napisal: >>> Hi, >>> >>> my previous fix failed due to use of non-static inner classes which >>> kept the cleanable objects around. Also, Sherman suggested a more >>> thorough fix to Inflater/Deflater after I had already pushed. >>> >>> Webrev: http://cr.openjdk.java.net/~redestad/8193507/open.00/ >>> >>> Verified all java/util/zip tests pass this time. >>> >>> /Claes >>> >>> >> > From naufal11 at gmail.com Sat Dec 16 10:27:45 2017 From: naufal11 at gmail.com (Mohamed Naufal) Date: Sat, 16 Dec 2017 21:27:45 +1100 Subject: [PATCH] Support for UTC Zones with TimeZone.getTimeZone() Message-ID: Hi, I noticed that with the following data: LocalDateTime ldt = LocalDateTime.parse("2017-01-01T00:00:00"); ZonedDateTime dt1 = ZonedDateTime.of(ldt, ZoneId.of("GMT+10")); ZonedDateTime dt2 = ZonedDateTime.of(ldt, ZoneId.of("UTC+10")); dt1.equals(dt2) returns true as expected, but with: GregorianCalendar gc1 = GregorianCalendar.from(dt1); GregorianCalendar gc2 = GregorianCalendar.from(dt2); gc1.equals(gc2) returns false. Looking at the code, I see when a GregorianCalendar is being constructed, TimeZone.getTimeZone() gets called, but it doesn't recognise UTC time-zones with offsets, and falls back to GMT(+0), whereas ZoneId treats GMT and UTC based zones equivalently. PFA a patch to fix this. Thank you, Naufal -------------- next part -------------- diff -r 65464a307408 src/java.base/share/classes/java/util/TimeZone.java --- a/src/java.base/share/classes/java/util/TimeZone.java Thu Aug 03 18:56:59 2017 +0000 +++ b/src/java.base/share/classes/java/util/TimeZone.java Sat Dec 16 10:10:14 2017 +0000 @@ -782,7 +782,8 @@ private static volatile TimeZone defaultTimeZone; static final String GMT_ID = "GMT"; - private static final int GMT_ID_LENGTH = 3; + static final String UTC_ID = "UTC"; + private static final int GMT_UTC_ID_LENGTH = 3; // a static TimeZone we can reference if no AppContext is in place private static volatile TimeZone mainAppContextDefault; @@ -799,9 +800,9 @@ int length; // Error if the length of id isn't long enough or id doesn't - // start with "GMT". - if ((length = id.length()) < (GMT_ID_LENGTH + 2) || - id.indexOf(GMT_ID) != 0) { + // start with "GMT" or "UTC". + if ((length = id.length()) < (GMT_UTC_ID_LENGTH + 2) || + id.indexOf(GMT_ID) != 0 && id.indexOf(UTC_ID) != 0) { return null; } @@ -815,7 +816,7 @@ return zi; } - int index = GMT_ID_LENGTH; + int index = GMT_UTC_ID_LENGTH; boolean negative = false; char c = id.charAt(index++); if (c == '-') { From xueming.shen at oracle.com Sat Dec 16 19:29:24 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Sat, 16 Dec 2017 11:29:24 -0800 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <8d8b1f1a-e258-7b4a-fa00-4f47702717ca@gmail.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> <5A346CC8.9010008@oracle.com> <8d8b1f1a-e258-7b4a-fa00-4f47702717ca@gmail.com> Message-ID: <5A357414.4060308@oracle.com> HI Peter, We are back to the "the order of resource relocation and cleaner registration" discussion again :-) The approach you are suggesting and was implemented in the previous version for de/inflater via the lambda basically is the same to have a callback mechanism () to delay the resource allocation until the Cleaner successfully registers the target object. I understood the benefit of dong that and agree it might be desired in certain circumstance, with the expectation that the Cleaner.register() might fail. But I think the Cleaner API is designed and implemented (at least for now) way that the "register()" is not going to fail in "normal" situation and you are not supposed (requested) to check if the returned Cleanable is null (never?) or try to catch an unexpected unchecked exception. Basically if a fundamental mechanism like Cleaner.register() is broken/failed. You probably don't have lot of choices to recover from such a failure but simply propagate the failure message/exception to upper level. It might be preferred not have the underlying resource allocated in this situation but I doubt it makes lots of difference and is worth the extra effort. But I think we can leave this to the future discussion when adding those newly proposed methods into the Cleaner interface. That said, in this case, it appears it does not require any "extra effort" to simply move the init(nowrap) invocation further down into the zstream constructor. I'm fine to go with it. Thanks, Sherman On 12/16/17, 2:22 AM, Peter Levart wrote: > Hi Sherman, > > Xueming Shen je 16. 12. 2017 ob 01:46 napisal: >> On 12/15/17, 3:43 PM, Peter Levart wrote: >>> Hi Claes, >>> >>> I see this is already pushed. I don't have any additional comments, >>> but just want to know what was wrong with old code. You say that you >>> used non-static inner classes. I don't see that in old code. All the >>> lambdas you replaced with nested static classes should have already >>> captured just local variables. I must be missing something and I >>> would really like to know what. >> >> Perter, >> >> Since that code triggered all zip/cleaner regression tests failed. I >> rollback-ed that >> fix overnight to make our overnight tests happy. So what in the >> webrev is against my >> original code. >> >> >> Here is my webrev that undo-ed the non-static inner classes. >> >> http://cr.openjdk.java.net/~sherman/8193490/webrev/ >> >> Sherman > > I see. So the 1st change was an attempt to fix a startup performance > regression (JDK-8193471) by replacing lambdas with anonymous inner > classes, which unfortunately capture 'this' if they are constructed in > instance context. I'm sorry I was busy and haven't noticed this RFR or > I would probably spot the mistake. The 2nd change is a re-attempt to > do this with static classes (JDK-8193507). This fixes the startup > performance regression problem, but it also re-introduces another one, > which was carefully avoided by using the lambdas in the initial > version. The problem is that in latest version of code, the > initialization order is: > - init(nowrap) > - Cleaner registration > > while in lambda version it was the other way around: > - Cleaner registration > - init(nowrap) > > If the registration of Cleanable fails, the latest version of code > leaks a native resource. > > Now that you have specialized ZStreamRef classes, you could post-pone > the native resource allocation in a simple way, directly in the > XxxZStreamRef constructor: > > Index: src/java.base/share/classes/java/util/zip/Inflater.java > IDEA additional info: > Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP > <+>UTF-8 > =================================================================== > --- src/java.base/share/classes/java/util/zip/Inflater.java > (revision 48342:003d6365ec6a529616d0e5b119144b4408c3a1c8) > +++ src/java.base/share/classes/java/util/zip/Inflater.java > (revision 48342+:003d6365ec6a+) > @@ -118,7 +118,7 @@ > * @param nowrap if true then support GZIP compatible compression > */ > public Inflater(boolean nowrap) { > - this.zsRef = InflaterZStreamRef.get(this, init(nowrap)); > + this.zsRef = InflaterZStreamRef.get(this, nowrap); > } > > /** > @@ -442,9 +442,9 @@ > private long address; > private final Cleanable cleanable; > > - private InflaterZStreamRef(Inflater owner, long addr) { > + private InflaterZStreamRef(Inflater owner, boolean nowrap) { > this.cleanable = (owner != null) ? > CleanerFactory.cleaner().register(owner, this) : null; > - this.address = addr; > + this.address = init(nowrap); > } > > long address() { > @@ -470,23 +470,23 @@ > * This mechanism will be removed when the {@code finalize} > method is > * removed from {@code Inflater}. > */ > - static InflaterZStreamRef get(Inflater owner, long addr) { > + static InflaterZStreamRef get(Inflater owner, boolean nowrap) { > Class clz = owner.getClass(); > while (clz != Inflater.class) { > try { > clz.getDeclaredMethod("end"); > - return new FinalizableZStreamRef(owner, addr); > + return new FinalizableZStreamRef(owner, nowrap); > } catch (NoSuchMethodException nsme) {} > clz = clz.getSuperclass(); > } > - return new InflaterZStreamRef(owner, addr); > + return new InflaterZStreamRef(owner, nowrap); > } > > private static class FinalizableZStreamRef extends > InflaterZStreamRef { > final Inflater owner; > > - FinalizableZStreamRef(Inflater owner, long addr) { > - super(null, addr); > + FinalizableZStreamRef(Inflater owner, boolean nowrap) { > + super(null, nowrap); > this.owner = owner; > } > > > This could be a refinement of the last patch. What do you think? > > Regards, Peter > >> >>> >>> Regards, Peter >>> >>> Claes Redestad je 14. 12. 2017 ob 12:07 napisal: >>>> Hi, >>>> >>>> my previous fix failed due to use of non-static inner classes which >>>> kept the cleanable objects around. Also, Sherman suggested a more >>>> thorough fix to Inflater/Deflater after I had already pushed. >>>> >>>> Webrev: http://cr.openjdk.java.net/~redestad/8193507/open.00/ >>>> >>>> Verified all java/util/zip tests pass this time. >>>> >>>> /Claes >>>> >>>> >>> >> > From peter.levart at gmail.com Mon Dec 18 08:22:44 2017 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Dec 2017 09:22:44 +0100 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <5A357414.4060308@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> <5A346CC8.9010008@oracle.com> <8d8b1f1a-e258-7b4a-fa00-4f47702717ca@gmail.com> <5A357414.4060308@oracle.com> Message-ID: <1d6885e6-59dc-84fa-eb9b-0af2c4440113@gmail.com> Hi Sherman, Claes, On 12/16/2017 08:29 PM, Xueming Shen wrote: > HI Peter, > > We are back to the "the order of resource relocation and cleaner > registration" discussion again :-) Yes, the story repeats itself :-) > > The approach you are suggesting and was implemented in the previous > version for de/inflater > via the lambda? basically is the same to have a callback mechanism () > to delay the resource > allocation until the Cleaner successfully registers the target object. > I understood the benefit of > dong that and agree it might be desired in certain circumstance, with > the expectation that the > Cleaner.register() might fail. But I think the Cleaner API is designed > and implemented (at least > for now) way that the "register()" is not going to fail in "normal" > situation and you are not > supposed (requested) to check if the returned Cleanable is null (never?). No, Cleaner.register never returns null. This is by the spec. > or try to catch an > unexpected unchecked exception. No, you should not do that. > Basically if a fundamental mechanism like Cleaner.register() > is broken/failed. You probably don't have lot of choices to recover > from such a failure but simply > propagate the failure message/exception to upper level. Cleaner.register() is not broken but we can still expect a failure in the form of OutOfMemoryError, because registration involves allocation. Finalizable objects usually did not suffer from this as their registration happened very early in their construction. We should try to follow this order with Cleaner API too. Java is pretty robust in recovering from most OutOfMemoryError(s) regardless of the programmer effort, simply because it has GC. Handling locks and native resources are a couple of exceptions (and there might be others) where the programmer must be carefully to maintain overall robustness. JVM has recently been given special feature to enable robustness in critical sections of synchronization primitives with @ReservedStackAccess annotation. If locks are that important, why should native resources be much less so? > It might be preferred not have the > underlying resource allocated in this situation but I doubt it makes > lots of difference and is worth > the extra effort. But I think we can leave this to the future > discussion when adding those newly > proposed methods into the Cleaner interface. > > That said, in this case, it appears it does not require any "extra > effort" to simply move the > init(nowrap) invocation further down into the zstream constructor. I'm > fine to go with it. I created an enhancement request: https://bugs.openjdk.java.net/browse/JDK-8193685 Here's also the webrev that goes with it: http://cr.openjdk.java.net/~plevart/jdk-dev/8193685_ZipInDeFlater_cleanupImprovement/webrev.01/ Since jdk10 stabilization repo has already been forked, this will probably be destined to JDK 11. Unless you think it should go to JDK 10. In that case I shall ask for permission to push to stabilization repo. The fix is not critical, but is also a low risk. Regards, Peter > > Thanks, > Sherman > > > On 12/16/17, 2:22 AM, Peter Levart wrote: >> Hi Sherman, >> >> Xueming Shen je 16. 12. 2017 ob 01:46?napisal: >>> On 12/15/17, 3:43 PM, Peter Levart wrote: >>>> Hi Claes, >>>> >>>> I see this is already pushed. I don't have any additional comments, >>>> but just want to know what was wrong with old code. You say that >>>> you used non-static inner classes. I don't see that in old code. >>>> All the lambdas you replaced with nested static classes should have >>>> already captured just local variables. I must be missing something >>>> and I would really like to know what. >>> >>> Perter, >>> >>> Since that code triggered all zip/cleaner regression tests failed. I >>> rollback-ed that >>> fix overnight to make our overnight tests happy. So what in the >>> webrev is against my >>> original code. >>> >>> >>> Here is my webrev that undo-ed the non-static inner classes. >>> >>> http://cr.openjdk.java.net/~sherman/8193490/webrev/ >>> >>> Sherman >> >> I see. So the 1st change was an attempt to fix a startup performance >> regression (JDK-8193471) by replacing lambdas with anonymous inner >> classes, which unfortunately capture 'this' if they are constructed >> in instance context. I'm sorry I was busy and haven't noticed this >> RFR or I would probably spot the mistake. The 2nd change is a >> re-attempt to do this with static classes (JDK-8193507). This fixes >> the startup performance regression problem, but it also re-introduces >> another one, which was carefully avoided by using the lambdas in the >> initial version. The problem is that in latest version of code, the >> initialization order is: >> - init(nowrap) >> - Cleaner registration >> >> while in lambda version it was the other way around: >> - Cleaner registration >> - init(nowrap) >> >> If the registration of Cleanable fails, the latest version of code >> leaks a native resource. >> >> Now that you have specialized ZStreamRef classes, you could post-pone >> the native resource allocation in a simple way, directly in the >> XxxZStreamRef constructor: >> >> Index: src/java.base/share/classes/java/util/zip/Inflater.java >> IDEA additional info: >> Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP >> <+>UTF-8 >> =================================================================== >> --- src/java.base/share/classes/java/util/zip/Inflater.java (revision >> 48342:003d6365ec6a529616d0e5b119144b4408c3a1c8) >> +++ src/java.base/share/classes/java/util/zip/Inflater.java (revision >> 48342+:003d6365ec6a+) >> @@ -118,7 +118,7 @@ >> ????? * @param nowrap if true then support GZIP compatible compression >> ????? */ >> ???? public Inflater(boolean nowrap) { >> -??????? this.zsRef = InflaterZStreamRef.get(this, init(nowrap)); >> +??????? this.zsRef = InflaterZStreamRef.get(this, nowrap); >> ???? } >> >> ???? /** >> @@ -442,9 +442,9 @@ >> ???????? private long address; >> ???????? private final Cleanable cleanable; >> >> -??????? private InflaterZStreamRef(Inflater owner, long addr) { >> +??????? private InflaterZStreamRef(Inflater owner, boolean nowrap) { >> ???????????? this.cleanable = (owner != null) ? >> CleanerFactory.cleaner().register(owner, this) : null; >> -??????????? this.address = addr; >> +??????????? this.address = init(nowrap); >> ???????? } >> >> ???????? long address() { >> @@ -470,23 +470,23 @@ >> ????????? * This mechanism will be removed when the {@code finalize} >> method is >> ????????? * removed from {@code Inflater}. >> ????????? */ >> -??????? static InflaterZStreamRef get(Inflater owner, long addr) { >> +??????? static InflaterZStreamRef get(Inflater owner, boolean nowrap) { >> ???????????? Class clz = owner.getClass(); >> ???????????? while (clz != Inflater.class) { >> ???????????????? try { >> ???????????????????? clz.getDeclaredMethod("end"); >> -??????????????????? return new FinalizableZStreamRef(owner, addr); >> +??????????????????? return new FinalizableZStreamRef(owner, nowrap); >> ???????????????? } catch (NoSuchMethodException nsme) {} >> ???????????????? clz = clz.getSuperclass(); >> ???????????? } >> -??????????? return new InflaterZStreamRef(owner, addr); >> +??????????? return new InflaterZStreamRef(owner, nowrap); >> ???????? } >> >> ???????? private static class FinalizableZStreamRef extends >> InflaterZStreamRef { >> ???????????? final Inflater owner; >> >> -??????????? FinalizableZStreamRef(Inflater owner, long addr) { >> -??????????????? super(null, addr); >> +??????????? FinalizableZStreamRef(Inflater owner, boolean nowrap) { >> +??????????????? super(null, nowrap); >> ???????????????? this.owner = owner; >> ???????????? } >> >> >> This could be a refinement of the last patch. What do you think? >> >> Regards, Peter >> >>> >>>> >>>> Regards, Peter >>>> >>>> Claes Redestad je 14. 12. 2017 ob 12:07 napisal: >>>>> Hi, >>>>> >>>>> my previous fix failed due to use of non-static inner classes >>>>> which kept the cleanable objects around. Also, Sherman suggested a >>>>> more thorough fix to Inflater/Deflater after I had already pushed. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~redestad/8193507/open.00/ >>>>> >>>>> Verified all java/util/zip tests pass this time. >>>>> >>>>> /Claes >>>>> >>>>> >>>> >>> >> > From amaembo at gmail.com Mon Dec 18 10:31:47 2017 From: amaembo at gmail.com (Tagir Valeev) Date: Mon, 18 Dec 2017 17:31:47 +0700 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Message-ID: Hello! I think that both methods could co-exist with slightly different semantics (even though they implementation is identical): The get() method should be called when it's evident from the narrow context that Optional is present at the point and using functional alternatives (like ifPresent()) is inconvenient at this place, e.g.: for(X x : list) { Y y = calculateYPossiblyThrowingCheckedException(x); Optional optional = getOptional(x, y); if(optional.isPresent()) { return optional.get(); // get() will always succeed } } return null; // or whatever else default value Such loop is difficult to convert to Stream/Optional chain and not very easy to get rid of isPresent/get. One possibility is to change to T t = getOptional(x, y).orElse(null); if(t != null) return t; But it's really questionable (after all using optionals we want to get rid of nulls). So I think having get() here is fine. Think of get() as an assertion-like method: if get() throws, then it's a bug in the code (most likely right in this method). The orElseThrow() method should be called when we are not sure that Optional is present (e.g. it's received from the method call), but we are fine with default NoSuchElementException. If orElseThrow() throws, then it's an acceptable situation (e.g. we are inside the library method and it documents that NoSuchElementException could be thrown if some precondition is not met). In IntelliJ IDEA we warn by default if get() is called and we cannot statically determine that Optional is always present at this point (e.g. isPresent() was checked). In future Java we may suggest to replace with orElseThrow() in such cases. However I don't like issuing a warning if get() is used like in the code above. And if get() will be deprecated, then we would need to issue warning on every get() usage. Thus I'm against the deprecation. > I know that you like get() (or at least you're OK with it). But even three > years after the release of Java 8, I still see bad code written with get(), > so I don't think this problem is going to go away. I believe that for any kind of API you can find bad code which abuses it regardless on how many years passed since the API was released. That's not always the fault of API creators. With best regards, Tagir Valeev. From ramanand.patil at oracle.com Mon Dec 18 11:41:19 2017 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Mon, 18 Dec 2017 03:41:19 -0800 (PST) Subject: RFR: JDK8u Backport of 8153955: increase java.util.logging.FileHandler MAX_LOCKS limit Message-ID: <1b4b3c1e-a35d-453c-aa66-aee973405d15@default> Hi all, Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/browse/JDK-8153955 Backport Bug: https://bugs.openjdk.java.net/browse/JDK-8161266 Webrev: http://cr.openjdk.java.net/~rpatil/8161266/webrev.00/ Summary(also added to backport bug description): The fix from JDK9 cannot be backported as is into the jdk8u and earlier update releases, since it contains JDK API spec changes. In JDK9 the fix is added by altering the FileHandler spec, which introduces a new configurable property "java.util.logging.FileHandler.maxLocks" to java.util.logging.FileHandler, defined in .../conf/logging.properties. The solution proposed for update releases is: To introduce an internal JDK implementation specific property- "jdk.internal.FileHandlerLogging.maxLocks" which will be used to control/override FileHandler's MAX_LOCKS value. The default value of the maxLocks (100) will be retained if this new System property is not set. The new property is read only once during FileHandler class initialization. Regards, Ramanand. From Alan.Bateman at oracle.com Mon Dec 18 14:07:36 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Dec 2017 14:07:36 +0000 Subject: [11] RFR 8193085 Vectorize the nio Buffer equals and compareTo implementations In-Reply-To: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> References: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> Message-ID: <210e5718-38ba-d0b4-8001-68c815258ea0@oracle.com> On 15/12/2017 20:29, Paul Sandoz wrote: > Hi, > > Please review this patch to vectorize the buffer equals and compareTo implementations, using the same approach that was used for arrays: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8193085-buffer-equals-compareTo-vectorize/webrev/ > > This patch expands on using the double address mode of unsafe to uniformly access a memory region covered by a buffer be it on or off heap. Only buffers with the same endianness can support optimized equality and comparison. I've looked through the changes and it looks quite good. It might be useful to add a comment in StringCharBuffer to explain why equals/compareTo are overridden. Also it might be safer if the mismatch method that takes CharBuffers checks if one of charRegionOrders is null to guarantee that it will never do a vectorizedMismatch test with a wrapped char sequence (it would catch a serious bug if someone were to misuse to call it with two StringCharBuffers). Not for this patch but XXXBuffer.compareTo doesn't specify how it compares when the remaining elements in one buffer is a proper prefix of the remaining elements in the other buffer. It's hard to see the changes to ArraySupport. I assume it's just the package name and changing the methods to public. I can't tell why webrev doesn't handle the move in this case. Another one is Arrays where they are some re-formatting in the patch that webrev doesn't show (I can't tell if this is intended or not). It would be good if the really long lines in BufferMismatch could be trimmed down (maybe import static ArraySupport) as it will very annoying to do side-by-side diffs when reviewing future changes. There are several places where the styles differs to the style in this area but it's probably not worth spending time on. The test looks okay although it overlaps with the existing tests. -Alan From xueming.shen at oracle.com Mon Dec 18 14:35:30 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 18 Dec 2017 06:35:30 -0800 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <1d6885e6-59dc-84fa-eb9b-0af2c4440113@gmail.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> <5A346CC8.9010008@oracle.com> <8d8b1f1a-e258-7b4a-fa00-4f47702717ca@gmail.com> <5A357414.4060308@oracle.com> <1d6885e6-59dc-84fa-eb9b-0af2c4440113@gmail.com> Message-ID: <5A37D232.5080209@oracle.com> On 12/18/17, 12:22 AM, Peter Levart wrote: >> That said, in this case, it appears it does not require any "extra >> effort" to simply move the >> init(nowrap) invocation further down into the zstream constructor. >> I'm fine to go with it. > > > I created an enhancement request: > > https://bugs.openjdk.java.net/browse/JDK-8193685 > > Here's also the webrev that goes with it: > > http://cr.openjdk.java.net/~plevart/jdk-dev/8193685_ZipInDeFlater_cleanupImprovement/webrev.01/ > > > Since jdk10 stabilization repo has already been forked, this will > probably be destined to JDK 11. Unless you think it should go to JDK > 10. In that case I shall ask for permission to push to stabilization > repo. The fix is not critical, but is also a low risk. > Looks good. Let's do it for jdk11. -Sherman From Roger.Riggs at Oracle.com Mon Dec 18 15:46:03 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 18 Dec 2017 10:46:03 -0500 Subject: JDK-6372077: JarFile.getManifest() should handle manifest attribute names up to 70 bytes In-Reply-To: References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> <02abcb68-2294-c733-5048-b1f81ee99435@oracle.com> <88b89995-cfe0-d439-d63d-699c628e2315@paratix.ch> Message-ID: <07e62d8d-ae3a-9ffa-d426-81ab548ad252@Oracle.com> Hi Phillip, Sorry for the silence... I/we haven't had time to full understand the ramifications of the change you propose. It seems there is a difficult/unresolvable conflict in the specifications between the line length requirements and the header specs. Regards, Roger On 11/21/2017 1:18 AM, Philipp Kunz wrote: > Hi everyone, > > I haven't got any reply now for around three weeks and now i start to > wonder if I just missed it or if I should refine my approach to find a > sponsor or if it helped if I presented a ready patch or if noone > considers this important enough or anything else whatever. This is > only my second contribution hence I don't know the procedure well. > > One point maybe worth to point out again is that I don't want to > support manifest headers longer than 70 character, just up to 70, > which has always been the intention but has only worked up to 68. This > might have been written confusingly in my last email. > > As far as I understood, I should first get a sponsor. In any case, is > there any suggestion for how to proceed? > > Regards, > Philipp > > > > On 03.11.2017 00:04, Philipp Kunz wrote: >> Hi Sean and Max and all others, >> >> Thank you Sean for the hint about the right mailing list. And thanks >> also for his hint to Max to make smaller portions of changes. >> >> I would like to contribute a fix for JDK-6372077 which is about >> JarFile.getManifest() should handle manifest attribute name[s longer >> than] 70 bytes. >> >> It looks like the bug is caused by Manifest.make72Safe breaking lines >> at 70 bytes instead of 72 despite its comment and name >> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Manifest.java#l176).The >> resulting StringBuffer has then lines of 72 bytes maximum each >> including the following line break. Without the line break that >> leaves 70 bytes of characters per line. On the other hand, header >> names can be up to 70 bytes (only single-byte utf-8 characters) and >> cannot be broken across a line break and need to be followed by a >> colon and a space which must be on the same line too according to the >> specification. When breaking at 70 bytes excluding the line break, >> however, long header names don't fit in one line together with the >> colon space delimiter because there is not sufficient space. >> Manifests with names up to 70 bytes long can still be written without >> immediate exception but the resulting manifests are illegal in my >> opinion. When later reading such manifests >> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Attributes.java#l406), >> an error occurs as a consequence of the bad manifest. This is more or >> less the current situation and also what JDK-6372077 already knew. >> >> ?--> After all, in order to fix it, i'd like to propose to make >> manifest file lines wider by two bytes. >> >> The only proper alternative that came into my mind would be to change >> the manifest specification and reduce the maximum header name length >> there by two and also in the code. If that would break any existing >> code i guess that would affect code only that produced invalid >> manifests and would be acceptable. >> >> Supporting all existing and possibly invalid manifests would mean to >> add support for reading headers the delimiters of which are broken >> onto the next line which I consider too complex with respect to the >> value added and even more so considering that those invalid manifest >> can be assumed to have been detected as such by reading them and also >> because it would be a feature that would be used less and less over >> time if the code to write manifest is changed at the same time to >> produce only valid manifests in the discussed respect here. I don't >> think this should be actually done. >> >> Before I actually do the leg work, i'd like to ask, if there are >> concerns or objections or tips for such a change or if anyone can or >> cannot follow the reasoning and the conclusion to make manifests 2 >> bytes wider or if i missed an important point altogether. >> >> In case there will be a consent about how to solve this, would >> someone volunteer to sponsor? That may be less urgent at the moment >> than the question above about how to proceed. >> >> Philipp >> >> >> On 12.10.2017 22:32, Sean Mullan wrote: >>> Hi Phillip, >>> >>> All of these bugs are in the core-libs area, so I am copying the >>> core-libs-dev list since that is where they should be discussed and >>> reviewed. I have -bcc-ed security-dev (where this was originally >>> posted). >>> >>> --Sean >>> >>> On 10/2/17 1:24 PM, Philipp Kunz wrote: >>>> Hi, >>>> >>>> While fixing JDK-6695402 I came across other related bugs to >>>> manifests such as JDK-6372077, JDK-6202130, JDK-8176843, >>>> JDK-4842483, and JDK-6443578 which all relate to manifest reading >>>> and writing. Somewhere bug 7071055 is mentioned but I cannot find >>>> anywhere. Another group of bugs, JDK-6910466, JDK-4935610, and >>>> JDK-4271239 concern the mandatory manifest main attributes >>>> Manifest-Version or Signature-Version and at first glance are >>>> duplicates. If you know of more known bugs, not necessarily present >>>> in jira, I'd be glad to get notified. >>>> >>>> There are also some comments about utf handling and line breaking >>>> in the code of Manifest: >>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l299 >>>> >>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l327 >>>> >>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l370 >>>> >>>> >>>> Furthermore, Attributes.map should declare appropriate type >>>> parameters: >>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l61 >>>> >>>> The specification would also require that `header names must not >>>> start with _, - or "From"` >>>> (http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Section-Specification) >>>> but I would opt not to add this as a hard restriction because I >>>> guess it can be assumed that such names are in use now after having >>>> been working for years. A warning to a logger might be conceivable >>>> such as in >>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l424 >>>> >>>> Attribute values are never checked at all and invalid characters >>>> such as line breaks are never detected except that when reading the >>>> manifest again the values are cut off. >>>> The tab character (U+0008) does not work in manifest values. >>>> I suspect that there are also issues regarding the iteration order >>>> but I haven't got a prove yet unlike for the other points above: >>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Manifest.java#l54 >>>> >>>> There is duplicated or very similar code in Attributes and >>>> Manifest: Attributes.write-Manifest.write-Attributes.writeMain and >>>> Attributes.read-Manifest.read. >>>> Resolving JDK-6202130 would have the additional benefit to be able >>>> to view manifests with any utf-8 capable editor even if multi-byte >>>> characters break across lines. >>>> >>>> Fixing these issues individually looks like more complicated work >>>> than fixing them all at once, I guess, also because of a very low >>>> current test coverage. So I'd suggest to add some thorough tests >>>> along with fixing these issues. But before I start I would like to >>>> get some guidance, assistance or at least confirmation on how to >>>> proceed. I'm new to open jdk and have submitted only one patch so far. >>>> >>>> Is it ok to add tests for things that have worked before? >>>> Is it ok to refactor duplicated code just for the added value to >>>> reduce effort for testing? >>>> I assume compatibility to and from existing manifests is the >>>> highest priority, correct? This would also be the hard part in >>>> adding as complete test coverage as possible. What would be >>>> acceptable criteria to meet? >>>> Why does Attributes not extend LinkedHashMap and why does Manifest >>>> not extend HashMap? Any objection? >>>> While I would not want to write code that looks slow or change more >>>> than necessary one can never know before having performance >>>> actually measured. Is there some way this is dealt with or should I >>>> wait for such feedback until after patch submission? >>>> >>>> Would someone sponsor? >>>> >>>> Regards, >>>> Philipp >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> >>>> >>>> Paratix GmbH >>>> St Peterhofstatt 11 >>>> 8001 Z?rich >>>> >>>> +41 (0)76 397 79 35 >>>> philipp.kunz at paratix.ch >> >> -- >> >> >> Gruss Philipp >> >> >> >> ------------------------------------------------------------------------ >> >> >> >> Paratix GmbH >> St Peterhofstatt 11 >> 8001 Z?rich >> >> +41 (0)76 397 79 35 >> philipp.kunz at paratix.ch > From goetz.lindenmaier at sap.com Mon Dec 18 17:35:08 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 18 Dec 2017 17:35:08 +0000 Subject: FW: RFR(M): 8189102: All tools should support -?, -h and --help In-Reply-To: <7be3772e8060494cbecf3097bd0a8a85@sap.com> References: <8aa25976d496465fac00a4ae3628e533@sap.com> <456c694ec25a42658af1b8600aaa136d@sap.com> <6d7d895f-9f23-fd4b-39ca-ef9bb0331fb9@oracle.com> <7be3772e8060494cbecf3097bd0a8a85@sap.com> Message-ID: <2124c07d77df43369bf3080c54f314b1@sap.com> Hi, I had to update my webrev after the javah launcher had been removed: http://cr.openjdk.java.net/~goetz/wr17/8189102-helpMessage/webrev.07/ I would appreciate a final decision to push this. Best regards, Goetz. -----Original Message----- From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Tuesday, December 12, 2017 11:37 AM To: Alan Bateman ; core-libs-dev at openjdk.java.net; 'compiler-dev at openjdk.java.net' ; serviceability-dev (serviceability-dev at openjdk.java.net) Subject: RE: FW: RFR(M): 8189102: All tools should support -?, -h and --help Hi Alan, Javadoc combines documentation and support of a flag in the way the flag handling is implemented. On the other side, it prints the help message anyways if a wrong flag is presented to it, so if you call it with -help you get the help message. Therefore, in my original change where I tried to get it more cleaned up, I removed -help support and documentation from Javadoc. I added it again and updated the table in the CSR: http://cr.openjdk.java.net/~goetz/wr17/8189102-helpMessage/webrev.06/ Best regards, Goetz. > -----Original Message----- > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Sent: Montag, 11. Dezember 2017 17:53 > To: Lindenmaier, Goetz ; core-libs- > dev at openjdk.java.net; 'compiler-dev at openjdk.java.net' dev at openjdk.java.net>; serviceability-dev (serviceability- > dev at openjdk.java.net) > Subject: Re: FW: RFR(M): 8189102: All tools should support -?, -h and --help > > > > On 07/12/2017 11:20, Lindenmaier, Goetz wrote: > > Hi, > > > > ... missed some lists in my first post ... > > > > I prepared a fifth webrev for this change. Please review. > > > > It incorporates the changes requested by the CSR reviewers > > (not to remove docuemtation of '-help' where is was documented > > before) and the changes proposed by Kumar: > > http://cr.openjdk.java.net/~goetz/wr17/8189102- > helpMessage/webrev.05/ > > > > > Looks like it still drops -help from the javadoc usage message, I can't > tell if you meant to do that. > > -Alan. From mandy.chung at oracle.com Mon Dec 18 19:42:59 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 18 Dec 2017 11:42:59 -0800 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <1d6885e6-59dc-84fa-eb9b-0af2c4440113@gmail.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> <5A346CC8.9010008@oracle.com> <8d8b1f1a-e258-7b4a-fa00-4f47702717ca@gmail.com> <5A357414.4060308@oracle.com> <1d6885e6-59dc-84fa-eb9b-0af2c4440113@gmail.com> Message-ID: <8c273076-1171-8a85-0354-d97dd1ce7daa@oracle.com> On 12/18/17 12:22 AM, Peter Levart wrote: > I created an enhancement request: > > https://bugs.openjdk.java.net/browse/JDK-8193685 > > Here's also the webrev that goes with it: > > http://cr.openjdk.java.net/~plevart/jdk-dev/8193685_ZipInDeFlater_cleanupImprovement/webrev.01/ > > > > Since jdk10 stabilization repo has already been forked, this will > probably be destined to JDK 11. Unless you think it should go to JDK > 10. In that case I shall ask for permission to push to stabilization > repo. The fix is not critical, but is also a low risk. From mandy.chung at oracle.com Mon Dec 18 19:44:22 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 18 Dec 2017 11:44:22 -0800 Subject: [10] RFR: 8193507: [REDO] Startup regression due to JDK-8185582 In-Reply-To: <8c273076-1171-8a85-0354-d97dd1ce7daa@oracle.com> References: <03e94b47-5258-8cc2-0531-1d4ecd0349fc@oracle.com> <84f15d8e-be26-7065-e516-ac0ca47123d4@gmail.com> <5A346CC8.9010008@oracle.com> <8d8b1f1a-e258-7b4a-fa00-4f47702717ca@gmail.com> <5A357414.4060308@oracle.com> <1d6885e6-59dc-84fa-eb9b-0af2c4440113@gmail.com> <8c273076-1171-8a85-0354-d97dd1ce7daa@oracle.com> Message-ID: <63382262-2386-e299-34de-3d533da6453c@oracle.com> On 12/18/17 11:42 AM, mandy chung wrote: > > > On 12/18/17 12:22 AM, Peter Levart wrote: >> I created an enhancement request: >> >> https://bugs.openjdk.java.net/browse/JDK-8193685 >> >> Here's also the webrev that goes with it: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/8193685_ZipInDeFlater_cleanupImprovement/webrev.01/ >> >> >> >> Since jdk10 stabilization repo has already been forked, this will >> probably be destined to JDK 11. Unless you think it should go to JDK >> 10. In that case I shall ask for permission to push to stabilization >> repo. The fix is not critical, but is also a low risk. This patch looks good to me.? I agree that this is not critical to go to JDK 10. Mandy From Alan.Bateman at oracle.com Mon Dec 18 20:00:28 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Dec 2017 20:00:28 +0000 Subject: 8193758: Update copyright headers of files in src tree that are missing "Classpath" exception Message-ID: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> There are a small number of files in jdk/jdk10 that have the GPL header rather than the GPL and "Classpath" exception. This came up last week on jdk-dev. I need a Reviewer to adjust these files. The webrev with the changes is here: ?? http://cr.openjdk.java.net/~alanb/8193758/webrev/ There are a few other tools used in the build that may need to be adjusted too. If needed, there will be a follow-up issue for those. -Alan From mandy.chung at oracle.com Mon Dec 18 20:10:08 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 18 Dec 2017 12:10:08 -0800 Subject: 8193758: Update copyright headers of files in src tree that are missing "Classpath" exception In-Reply-To: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> References: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> Message-ID: <15973f19-85e1-d8f6-1ffb-4a8e06e00a5b@oracle.com> On 12/18/17 12:00 PM, Alan Bateman wrote: > > There are a small number of files in jdk/jdk10 that have the GPL > header rather than the GPL and "Classpath" exception. This came up > last week on jdk-dev. I need a Reviewer to adjust these files. The > webrev with the changes is here: > ?? http://cr.openjdk.java.net/~alanb/8193758/webrev/ > > There are a few other tools used in the build that may need to be > adjusted too. If needed, there will be a follow-up issue for those. +1 Mandy From mark.reinhold at oracle.com Mon Dec 18 20:10:15 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 18 Dec 2017 12:10:15 -0800 Subject: 8193758: Update copyright headers of files in src tree that are missing "Classpath" exception In-Reply-To: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> References: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> Message-ID: <20171218121015.367137455@eggemoggin.niobe.net> 2017/12/18 12:00:28 -0800, alan.bateman at oracle.com: > There are a small number of files in jdk/jdk10 that have the GPL header > rather than the GPL and "Classpath" exception. This came up last week on > jdk-dev. I need a Reviewer to adjust these files. The webrev with the > changes is here: > http://cr.openjdk.java.net/~alanb/8193758/webrev/ Looks good. - Mark From jonathan.gibbons at oracle.com Mon Dec 18 20:19:29 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 18 Dec 2017 12:19:29 -0800 Subject: 8193758: Update copyright headers of files in src tree that are missing "Classpath" exception In-Reply-To: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> References: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> Message-ID: OK for JavacTaskPool. -- Jon On 12/18/17 12:00 PM, Alan Bateman wrote: > > There are a small number of files in jdk/jdk10 that have the GPL > header rather than the GPL and "Classpath" exception. This came up > last week on jdk-dev. I need a Reviewer to adjust these files. The > webrev with the changes is here: > ?? http://cr.openjdk.java.net/~alanb/8193758/webrev/ > > There are a few other tools used in the build that may need to be > adjusted too. If needed, there will be a follow-up issue for those. > > -Alan From paul.sandoz at oracle.com Mon Dec 18 20:55:51 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 18 Dec 2017 12:55:51 -0800 Subject: [11] RFR 8193085 Vectorize the nio Buffer equals and compareTo implementations In-Reply-To: <210e5718-38ba-d0b4-8001-68c815258ea0@oracle.com> References: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> <210e5718-38ba-d0b4-8001-68c815258ea0@oracle.com> Message-ID: <1F5763B0-388D-4863-941C-5E23963B56DD@oracle.com> > On 18 Dec 2017, at 06:07, Alan Bateman wrote: > > On 15/12/2017 20:29, Paul Sandoz wrote: >> Hi, >> >> Please review this patch to vectorize the buffer equals and compareTo implementations, using the same approach that was used for arrays: >> >> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8193085-buffer-equals-compareTo-vectorize/webrev/ >> >> This patch expands on using the double address mode of unsafe to uniformly access a memory region covered by a buffer be it on or off heap. Only buffers with the same endianness can support optimized equality and comparison. > I've looked through the changes and it looks quite good. > > It might be useful to add a comment in StringCharBuffer to explain why equals/compareTo are overridden. See below. > Also it might be safer if the mismatch method that takes CharBuffers checks if one of charRegionOrders is null to guarantee that it will never do a vectorizedMismatch test with a wrapped char sequence (it would catch a serious bug if someone were to misuse to call it with two StringCharBuffers). There is already an assert, perhaps i can simplify this: 1) StringCharBuffer does not require special overrides. 2) Update the mismatch method: static int mismatch(CharBuffer a, int aOff, CharBuffer b, int bOff, int length) { int i = 0; // Ensure only heap or off-heap buffer instances use the // vectorized mismatch. If either buffer is a StringCharBuffer // (order is null) then the slow path is taken if (length > 3 && a.charRegionOrder() == b.charRegionOrder() && a.charRegionOrder() != null && b.charRegionOrder() != null) { I updated the webrev in place (i also updated the test to test big vs. little endian). > Not for this patch but XXXBuffer.compareTo doesn't specify how it compares when the remaining elements in one buffer is a proper prefix of the remaining elements in the other buffer. > Right. There is a follow on bug to add new API points and we can do a cleanup as part of that. > It's hard to see the changes to ArraySupport. I assume it's just the package name and changing the methods to public. I can't tell why webrev doesn't handle the move in this case. I dunno why that was not tracked. It?s just changes to the package name and method accessibility. > Another one is Arrays where they are some re-formatting in the patch that webrev doesn't show (I can't tell if this is intended or not). > That was a refactoring glitch, fixed: http://cr.openjdk.java.net/~psandoz/jdk/JDK-8193085-buffer-equals-compareTo-vectorize/webrev/src/java.base/share/classes/java/util/Arrays.java.sdiff.html > It would be good if the really long lines in BufferMismatch could be trimmed down (maybe import static ArraySupport) as it will very annoying to do side-by-side diffs when reviewing future changes. Done. > There are several places where the styles differs to the style in this area but it's probably not worth spending time on. > Agreed, the only sane way to do this is to have auto-formatting on commit (i have it set up in my IDE but it?s obviously not consistent will all source code). And it requires a benevolent dictator with good taste to state the format, since i suspect we will never reach consensus on such matters :-) > The test looks okay although it overlaps with the existing tests. > Yes, although the existing Buffer equals/compareTo tests are somewhat limited (especially regarding length). These new tests will also help if/when a public mismatch method is added. Thanks, Paul. From iris.clark at oracle.com Mon Dec 18 21:18:03 2017 From: iris.clark at oracle.com (Iris Clark) Date: Mon, 18 Dec 2017 13:18:03 -0800 (PST) Subject: 8193758: Update copyright headers of files in src tree that are missing "Classpath" exception In-Reply-To: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> References: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> Message-ID: <9ea1e8dd-5c73-4c45-b408-9b3fe6c6cc4f@default> Hi, Alan. > http://cr.openjdk.java.net/~alanb/8193758/webrev/ This looks good. Thanks for taking care of this. Iris From philip.race at oracle.com Mon Dec 18 21:25:32 2017 From: philip.race at oracle.com (Phil Race) Date: Mon, 18 Dec 2017 13:25:32 -0800 Subject: 8193758: Update copyright headers of files in src tree that are missing "Classpath" exception In-Reply-To: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> References: <3364ff91-8c2e-296a-f887-322636193271@oracle.com> Message-ID: Looks good. I had to do a double-take why the AWT files showed all lines updated but of course you fixed the indentation too .. -phil. On 12/18/2017 12:00 PM, Alan Bateman wrote: > > There are a small number of files in jdk/jdk10 that have the GPL > header rather than the GPL and "Classpath" exception. This came up > last week on jdk-dev. I need a Reviewer to adjust these files. The > webrev with the changes is here: > http://cr.openjdk.java.net/~alanb/8193758/webrev/ > > There are a few other tools used in the build that may need to be > adjusted too. If needed, there will be a follow-up issue for those. > > -Alan From david.buck at oracle.com Mon Dec 18 22:14:48 2017 From: david.buck at oracle.com (David Buck) Date: Tue, 19 Dec 2017 07:14:48 +0900 Subject: [8u] RFR(S) 8059036: Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <0f3f7d99-fccb-0305-4939-6a77d110b908@oracle.com> References: <0f3f7d99-fccb-0305-4939-6a77d110b908@oracle.com> Message-ID: <26d1f9ee-6abf-bfed-b25d-2947e78f9577@oracle.com> Hi! Any love out there for my review request? Cheers, -Buck On 2017/12/15 14:42, David Buck wrote: > Hi! > > May I please get a review of the following very simple backport of this > serviceability improvement to JDK 8? > > The two hotspot jtreg test cases needed to be modified slightly because > of the lack of dcmd-specific test support in JDK 8's HS code base. The > only non-test difference from the JDK 9 changeset is modifying an > include directive (objArrayOop.inline.hpp -> objArrayOop.hpp) to > compensate for JDK-8072911. > > bug report: > https://bugs.openjdk.java.net/browse/JDK-8059036 > > JDK 9 changesets: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/56a527afc34a > http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/9b17d0a2720f > > JDK 9 review thread: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033169.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033835.html > > JDK 8 backport webrevs for review: > http://cr.openjdk.java.net/~dbuck/8059036.0_jdk8/ > > Cheers, > -Buck > > [0] https://bugs.openjdk.java.net/browse/JDK-8072911 From mandy.chung at oracle.com Mon Dec 18 22:25:18 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 18 Dec 2017 14:25:18 -0800 Subject: [8u] RFR(S) 8059036: Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <0f3f7d99-fccb-0305-4939-6a77d110b908@oracle.com> References: <0f3f7d99-fccb-0305-4939-6a77d110b908@oracle.com> Message-ID: On 12/14/17 9:42 PM, David Buck wrote: > Hi! > > May I please get a review of the following very simple backport of > this serviceability improvement to JDK 8? > > : > > JDK 8 backport webrevs for review: > http://cr.openjdk.java.net/~dbuck/8059036.0_jdk8/ This patch looks okay to me. Mandy From mandy.chung at oracle.com Mon Dec 18 23:14:27 2017 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 18 Dec 2017 15:14:27 -0800 Subject: RFR: JDK-8193780: (ref) Remove an undocumented "jdk.lang.ref.disableClearBeforeEnqueue" system property Message-ID: <8b5cdbdc-c2d7-a3f6-d891-591adcf0397c@oracle.com> Webrev: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8193780/webrev.00/ CSR: ? https://bugs.openjdk.java.net/browse/JDK-8193781 This patch removes the "jdk.lang.ref.disableClearBeforeEnqueue" system property support which was added to provide an interim solution to restore to JDK 8 behavior of `Reference::enqueue` not to clear the referent.? I propose to remove this in 11. Mandy From philipp.kunz at paratix.ch Tue Dec 19 04:17:21 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Tue, 19 Dec 2017 05:17:21 +0100 Subject: JDK-6372077: JarFile.getManifest() should handle manifest attribute names up to 70 bytes In-Reply-To: <07e62d8d-ae3a-9ffa-d426-81ab548ad252@Oracle.com> References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> <02abcb68-2294-c733-5048-b1f81ee99435@oracle.com> <88b89995-cfe0-d439-d63d-699c628e2315@paratix.ch> <07e62d8d-ae3a-9ffa-d426-81ab548ad252@Oracle.com> Message-ID: Hi Roger, My suggested and also preferred approach is to read the manifest specification [1] in a way such that the line breaks are not included when counting the maximum line length. The specification does not state explicitly whether or not to include line breaks within the maximum line length limit but the following sentence from the specifications gives a hint: Because header names cannot be continued, the maximum length of a header name is 70 bytes (there must be a colon and a SPACE after the name). Above statement can be true only if line breaks are not counted for the maximum line length limit. Assuming so would in my opinion allow to understand the complete manifest specification without a conflict and effectively result in wider manifest files (maximum each line), wider by two bytes of a line break. In the meantime since the mail you replied to, I created a patch [3] mentioned in [2] which might be useful provided the manifest specification discussion is resolved. Regards, Philipp [1] https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Notes_on_Manifest_and_Signature_Files / https://docs.oracle.com/javase/9/docs/specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050500.html [3] http://files.paratix.ch/jdk/6372077/webrev.01/ On 18.12.2017 16:46, Roger Riggs wrote: > Hi Phillip, > > Sorry for the silence... > > I/we haven't had time to full understand the ramifications of the > change you propose. > It seems there is a difficult/unresolvable conflict in the > specifications between the line length > requirements and the header specs. > > Regards, Roger > > On 11/21/2017 1:18 AM, Philipp Kunz wrote: >> Hi everyone, >> >> I haven't got any reply now for around three weeks and now i start to >> wonder if I just missed it or if I should refine my approach to find >> a sponsor or if it helped if I presented a ready patch or if noone >> considers this important enough or anything else whatever. This is >> only my second contribution hence I don't know the procedure well. >> >> One point maybe worth to point out again is that I don't want to >> support manifest headers longer than 70 character, just up to 70, >> which has always been the intention but has only worked up to 68. >> This might have been written confusingly in my last email. >> >> As far as I understood, I should first get a sponsor. In any case, is >> there any suggestion for how to proceed? >> >> Regards, >> Philipp >> >> >> >> On 03.11.2017 00:04, Philipp Kunz wrote: >>> Hi Sean and Max and all others, >>> >>> Thank you Sean for the hint about the right mailing list. And thanks >>> also for his hint to Max to make smaller portions of changes. >>> >>> I would like to contribute a fix for JDK-6372077 which is about >>> JarFile.getManifest() should handle manifest attribute name[s longer >>> than] 70 bytes. >>> >>> It looks like the bug is caused by Manifest.make72Safe breaking >>> lines at 70 bytes instead of 72 despite its comment and name >>> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Manifest.java#l176).The >>> resulting StringBuffer has then lines of 72 bytes maximum each >>> including the following line break. Without the line break that >>> leaves 70 bytes of characters per line. On the other hand, header >>> names can be up to 70 bytes (only single-byte utf-8 characters) and >>> cannot be broken across a line break and need to be followed by a >>> colon and a space which must be on the same line too according to >>> the specification. When breaking at 70 bytes excluding the line >>> break, however, long header names don't fit in one line together >>> with the colon space delimiter because there is not sufficient space. >>> Manifests with names up to 70 bytes long can still be written >>> without immediate exception but the resulting manifests are illegal >>> in my opinion. When later reading such manifests >>> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Attributes.java#l406), >>> an error occurs as a consequence of the bad manifest. This is more >>> or less the current situation and also what JDK-6372077 already knew. >>> >>> --> After all, in order to fix it, i'd like to propose to make >>> manifest file lines wider by two bytes. >>> >>> The only proper alternative that came into my mind would be to >>> change the manifest specification and reduce the maximum header name >>> length there by two and also in the code. If that would break any >>> existing code i guess that would affect code only that produced >>> invalid manifests and would be acceptable. >>> >>> Supporting all existing and possibly invalid manifests would mean to >>> add support for reading headers the delimiters of which are broken >>> onto the next line which I consider too complex with respect to the >>> value added and even more so considering that those invalid manifest >>> can be assumed to have been detected as such by reading them and >>> also because it would be a feature that would be used less and less >>> over time if the code to write manifest is changed at the same time >>> to produce only valid manifests in the discussed respect here. I >>> don't think this should be actually done. >>> >>> Before I actually do the leg work, i'd like to ask, if there are >>> concerns or objections or tips for such a change or if anyone can or >>> cannot follow the reasoning and the conclusion to make manifests 2 >>> bytes wider or if i missed an important point altogether. >>> >>> In case there will be a consent about how to solve this, would >>> someone volunteer to sponsor? That may be less urgent at the moment >>> than the question above about how to proceed. >>> >>> Philipp >>> >>> >>> On 12.10.2017 22:32, Sean Mullan wrote: >>>> Hi Phillip, >>>> >>>> All of these bugs are in the core-libs area, so I am copying the >>>> core-libs-dev list since that is where they should be discussed and >>>> reviewed. I have -bcc-ed security-dev (where this was originally >>>> posted). >>>> >>>> --Sean >>>> >>>> On 10/2/17 1:24 PM, Philipp Kunz wrote: >>>>> Hi, >>>>> >>>>> While fixing JDK-6695402 I came across other related bugs to >>>>> manifests such as JDK-6372077, JDK-6202130, JDK-8176843, >>>>> JDK-4842483, and JDK-6443578 which all relate to manifest reading >>>>> and writing. Somewhere bug 7071055 is mentioned but I cannot find >>>>> anywhere. Another group of bugs, JDK-6910466, JDK-4935610, and >>>>> JDK-4271239 concern the mandatory manifest main attributes >>>>> Manifest-Version or Signature-Version and at first glance are >>>>> duplicates. If you know of more known bugs, not necessarily >>>>> present in jira, I'd be glad to get notified. >>>>> >>>>> There are also some comments about utf handling and line breaking >>>>> in the code of Manifest: >>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l299 >>>>> >>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l327 >>>>> >>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l370 >>>>> >>>>> >>>>> Furthermore, Attributes.map should declare appropriate type >>>>> parameters: >>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l61 >>>>> >>>>> The specification would also require that `header names must not >>>>> start with _, - or "From"` >>>>> (http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Section-Specification) >>>>> but I would opt not to add this as a hard restriction because I >>>>> guess it can be assumed that such names are in use now after >>>>> having been working for years. A warning to a logger might be >>>>> conceivable such as in >>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l424 >>>>> >>>>> Attribute values are never checked at all and invalid characters >>>>> such as line breaks are never detected except that when reading >>>>> the manifest again the values are cut off. >>>>> The tab character (U+0008) does not work in manifest values. >>>>> I suspect that there are also issues regarding the iteration order >>>>> but I haven't got a prove yet unlike for the other points above: >>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Manifest.java#l54 >>>>> >>>>> There is duplicated or very similar code in Attributes and >>>>> Manifest: Attributes.write-Manifest.write-Attributes.writeMain and >>>>> Attributes.read-Manifest.read. >>>>> Resolving JDK-6202130 would have the additional benefit to be able >>>>> to view manifests with any utf-8 capable editor even if multi-byte >>>>> characters break across lines. >>>>> >>>>> Fixing these issues individually looks like more complicated work >>>>> than fixing them all at once, I guess, also because of a very low >>>>> current test coverage. So I'd suggest to add some thorough tests >>>>> along with fixing these issues. But before I start I would like to >>>>> get some guidance, assistance or at least confirmation on how to >>>>> proceed. I'm new to open jdk and have submitted only one patch so >>>>> far. >>>>> >>>>> Is it ok to add tests for things that have worked before? >>>>> Is it ok to refactor duplicated code just for the added value to >>>>> reduce effort for testing? >>>>> I assume compatibility to and from existing manifests is the >>>>> highest priority, correct? This would also be the hard part in >>>>> adding as complete test coverage as possible. What would be >>>>> acceptable criteria to meet? >>>>> Why does Attributes not extend LinkedHashMap and why does Manifest >>>>> not extend HashMap? Any objection? >>>>> While I would not want to write code that looks slow or change >>>>> more than necessary one can never know before having performance >>>>> actually measured. Is there some way this is dealt with or should >>>>> I wait for such feedback until after patch submission? >>>>> >>>>> Would someone sponsor? >>>>> >>>>> Regards, >>>>> Philipp >>>>> >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> >>>>> Paratix GmbH >>>>> St Peterhofstatt 11 >>>>> 8001 Z?rich >>>>> >>>>> +41 (0)76 397 79 35 >>>>> philipp.kunz at paratix.ch >>> >>> -- >>> >>> >>> Gruss Philipp >>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> >>> >>> >>> Paratix GmbH >>> St Peterhofstatt 11 >>> 8001 Z?rich >>> >>> +41 (0)76 397 79 35 >>> philipp.kunz at paratix.ch >> > From rachna.goel at oracle.com Tue Dec 19 06:35:43 2017 From: rachna.goel at oracle.com (Rachna Goel) Date: Tue, 19 Dec 2017 12:05:43 +0530 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols Message-ID: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> Hi, Kindly review API Doc fix for java.text.DateFormatSymbols. JBS Issue : https://bugs.openjdk.java.net/browse/JDK-8146656 Webrev: http://cr.openjdk.java.net/~rgoel/8146656/webrev/ CSR: https://bugs.openjdk.java.net/browse/JDK-8191414 -- Thanks, Rachna From joe.darcy at oracle.com Tue Dec 19 08:37:31 2017 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 19 Dec 2017 00:37:31 -0800 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> Message-ID: <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> Hello Rachna, On 12/18/2017 10:35 PM, Rachna Goel wrote: > Hi, > > Kindly review API Doc fix for java.text.DateFormatSymbols. > > JBS Issue : https://bugs.openjdk.java.net/browse/JDK-8146656 > > Webrev: http://cr.openjdk.java.net/~rgoel/8146656/webrev/ > > CSR: https://bugs.openjdk.java.net/browse/JDK-8191414 > An addendum to my CSR review. The newly added text should be normative, not just informative. That is, the text should officially be part of the specification of the class. If you do not want the 13 elements behavior to be required of a subclass, change the @implNote into an @implSpec. If you want 13 elements to be required of subclasses too, replace @implNote with a paragraph begin. (For more guidance on @impNote vs @implspec, etc. see http://openjdk.java.net/jeps/8068562) The CSR should be updated with the amended API change. Thanks, -Joe From joe.darcy at oracle.com Tue Dec 19 09:25:21 2017 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 19 Dec 2017 01:25:21 -0800 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> Message-ID: Hi Rachna, On 12/19/2017 1:13 AM, Rachna Goel wrote: > > Hello Joe, > > Thanks for the review. > > Reason I added @implNote is that it's the case for the default > implementation. Not added as a part of spec, as some implementation > can just return 12 element array for same methods through the > "java.text.spi.DateFormatSymbolsProvider" SPI. > > That is precisely the sort of situation the @implSpec tag is intended for. It allows the specification to say DateFormatSymbols must behave this way while allowing subclasses to behave differently. Perhaps some general text can be added as normal specification, something like "An array with either 12 or 13 elements will be returned depending on whether or {@link Calendar.UNDECIMBER} is supported." paired with @implSpec This method returns 13 elements since @link Calendar.UNDECIMBER} is supported. HTH, -Joe From rachna.goel at oracle.com Tue Dec 19 09:13:55 2017 From: rachna.goel at oracle.com (Rachna Goel) Date: Tue, 19 Dec 2017 14:43:55 +0530 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> Message-ID: <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> Hello Joe, Thanks for the review. Reason I added @implNote is that it's the case for the default implementation. Not added as a part of spec, as some implementation can just return 12 element array for same methods through the "java.text.spi.DateFormatSymbolsProvider" SPI. Thanks, Rachna On 19/12/17 2:07 PM, joe darcy wrote: > Hello Rachna, > > > On 12/18/2017 10:35 PM, Rachna Goel wrote: >> Hi, >> >> Kindly review API Doc fix for java.text.DateFormatSymbols. >> >> JBS Issue : https://bugs.openjdk.java.net/browse/JDK-8146656 >> >> Webrev: http://cr.openjdk.java.net/~rgoel/8146656/webrev/ >> >> CSR: https://bugs.openjdk.java.net/browse/JDK-8191414 >> > > An addendum to my CSR review. The newly added text should be > normative, not just informative. That is, the text should officially > be part of the specification of the class. > > If you do not want the 13 elements behavior to be required of a > subclass, change the @implNote into an @implSpec. If you want 13 > elements to be required of subclasses too, replace @implNote with a > paragraph begin. (For more guidance on @impNote vs @implspec, etc. see > http://openjdk.java.net/jeps/8068562) > > The CSR should be updated with the amended API change. > > Thanks, > > -Joe -- Thanks, Rachna From Alan.Bateman at oracle.com Tue Dec 19 09:48:19 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 19 Dec 2017 09:48:19 +0000 Subject: RFR: JDK-8193780: (ref) Remove an undocumented "jdk.lang.ref.disableClearBeforeEnqueue" system property In-Reply-To: <8b5cdbdc-c2d7-a3f6-d891-591adcf0397c@oracle.com> References: <8b5cdbdc-c2d7-a3f6-d891-591adcf0397c@oracle.com> Message-ID: <9c82123a-8861-6500-8e0c-b7e63126f06b@oracle.com> On 18/12/2017 23:14, mandy chung wrote: > Webrev: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8193780/webrev.00/ > > CSR: > ? https://bugs.openjdk.java.net/browse/JDK-8193781 > > This patch removes the "jdk.lang.ref.disableClearBeforeEnqueue" system > property support which was added to provide an interim solution to > restore to JDK 8 behavior of `Reference::enqueue` not to clear the > referent.? I propose to remove this in 11. Looks fine. -Alan From rachna.goel at oracle.com Tue Dec 19 10:08:30 2017 From: rachna.goel at oracle.com (Rachna Goel) Date: Tue, 19 Dec 2017 15:38:30 +0530 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> Message-ID: Hi Joe, Thanks for the comments. I have updated the CSR to have @implSpec in place of @implNote. https://bugs.openjdk.java.net/browse/JDK-8191414 Regarding "An array with either 12 or 13 elements will be returned depending on whether or {@link Calendar.UNDECIMBER} is supported." , I would like to go with existing statement as this method always returns 13 elements where the 13th element may be empty string or may contain Calendar.UNDECIMBER, depending upon whether its supported by the Calendar instance. kindly suggest whether this looks fine! Thanks, Rachna On 19/12/17 2:55 PM, joe darcy wrote: > Hi Rachna, > > On 12/19/2017 1:13 AM, Rachna Goel wrote: >> >> Hello Joe, >> >> Thanks for the review. >> >> Reason I added @implNote is that it's the case for the default >> implementation. Not added as a part of spec, as some implementation >> can just return 12 element array for same methods through the >> "java.text.spi.DateFormatSymbolsProvider" SPI. >> >> > > That is precisely the sort of situation the @implSpec tag is intended > for. It allows the specification to say DateFormatSymbols must behave > this way while allowing subclasses to behave differently. > > Perhaps some general text can be added as normal specification, > something like > > "An array with either 12 or 13 elements will be returned depending on > whether or {@link Calendar.UNDECIMBER} is supported." > > paired with > > @implSpec This method returns 13 elements since @link > Calendar.UNDECIMBER} is supported. > > HTH, > > -Joe > -- Thanks, Rachna From Alan.Bateman at oracle.com Tue Dec 19 11:06:01 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 19 Dec 2017 11:06:01 +0000 Subject: Thread.single_step field, anything using it? Message-ID: I've been going through the fields in java.lang.Thread and I'm wondering if this field can be removed: ??? /* Whether or not to single_step this thread. */ ??? private boolean???? single_step; This field was used in the original Classic VM (pre-OpenJDK history). It doesn't appear to be used in the HotSpot VM. Does anyone know of any reason to keep it? Are there other VMs using it by any chance? -Alan From Alan.Bateman at oracle.com Tue Dec 19 19:35:30 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 19 Dec 2017 19:35:30 +0000 Subject: [11] RFR 8193085 Vectorize the nio Buffer equals and compareTo implementations In-Reply-To: <1F5763B0-388D-4863-941C-5E23963B56DD@oracle.com> References: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> <210e5718-38ba-d0b4-8001-68c815258ea0@oracle.com> <1F5763B0-388D-4863-941C-5E23963B56DD@oracle.com> Message-ID: <29c94181-a627-3dd8-792e-3b1d51383cc4@oracle.com> On 18/12/2017 20:55, Paul Sandoz wrote: > : > There is already an assert, perhaps i can simplify this: > > 1) StringCharBuffer does not require special overrides. > > 2) Update the mismatch method: > > static int mismatch(CharBuffer a, int aOff, CharBuffer b, int bOff, int length) { > int i = 0; > // Ensure only heap or off-heap buffer instances use the > // vectorized mismatch. If either buffer is a StringCharBuffer > // (order is null) then the slow path is taken > if (length > 3 && a.charRegionOrder() == b.charRegionOrder() > && a.charRegionOrder() != null && b.charRegionOrder() != null) { > > I updated the webrev in place (i also updated the test to test big vs. little endian). When I looked at it yesterday the CharBuffer version of mismatch wasn't checking both bases (or maybe I just mis-read it). Anyway, it looks good now and I see the other bug updated to list clarifying the compareTo specs too. -Alan From paul.sandoz at oracle.com Tue Dec 19 19:39:01 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 19 Dec 2017 11:39:01 -0800 Subject: [11] RFR 8193085 Vectorize the nio Buffer equals and compareTo implementations In-Reply-To: <29c94181-a627-3dd8-792e-3b1d51383cc4@oracle.com> References: <6BA74ADA-CB4D-4D28-A0BF-F0948FABDA7F@oracle.com> <210e5718-38ba-d0b4-8001-68c815258ea0@oracle.com> <1F5763B0-388D-4863-941C-5E23963B56DD@oracle.com> <29c94181-a627-3dd8-792e-3b1d51383cc4@oracle.com> Message-ID: <2161F2D2-5B97-4439-8AD0-5CC8989ECCD9@oracle.com> > On 19 Dec 2017, at 11:35, Alan Bateman wrote: > > On 18/12/2017 20:55, Paul Sandoz wrote: >> : >> There is already an assert, perhaps i can simplify this: >> >> 1) StringCharBuffer does not require special overrides. >> >> 2) Update the mismatch method: >> >> static int mismatch(CharBuffer a, int aOff, CharBuffer b, int bOff, int length) { >> int i = 0; >> // Ensure only heap or off-heap buffer instances use the >> // vectorized mismatch. If either buffer is a StringCharBuffer >> // (order is null) then the slow path is taken >> if (length > 3 && a.charRegionOrder() == b.charRegionOrder() >> && a.charRegionOrder() != null && b.charRegionOrder() != null) { >> >> I updated the webrev in place (i also updated the test to test big vs. little endian). > When I looked at it yesterday the CharBuffer version of mismatch wasn't checking both bases (or maybe I just mis-read it). > You did not misread it. Previously it strictly did not need to check both (only argument a) because the StringCharBuffer equals/compareTo did not call the mismatch method. > Anyway, it looks good now and I see the other bug updated to list clarifying the compareTo specs too. > Thanks, Paul. From brian.burkhalter at oracle.com Tue Dec 19 19:57:26 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 19 Dec 2017 11:57:26 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved Message-ID: https://bugs.openjdk.java.net/browse/JDK-8193832 http://cr.openjdk.java.net/~bpb/8193832/webrev.00/ The implementation of InputStream.readAllBytes() can be modified to be in general faster and to require less memory. The current implementation starts with an internal buffer of size 8192 bytes and doubles the size of the buffer each time more capacity is required resulting in a geometric progression in the buffer size. At the end if the buffer size is not exactly equal to the total number of bytes read, then another allocation equal to the total number read is performed. The amount of memory N required can be calculated for initial buffer size B and total bytes read L as M for L == B*(2^n) N = M + L for L != B*(2^n) where M = B*(2^(n + 1) - 1) and n = ceil(log2(L/B)). An alternative implementation is not to increase the size of the internal buffer each time more capacity is needed, but to instead maintain a list of buffers of up to B bytes each and gather those into an output buffer at the end. If there is only a single buffer in the list then gathering is not needed and it can be returned directly. The amount of memory N required in this case is B + L for L <= B N = B + 2*L for L > B A comparison of memory usage for a number of lengths L with a buffer size B of 8192 is: L N_old N_new 8191 16383 16383 8192 8192 16384 8193 32769 24578 16383 40959 40958 16384 24576 40960 16385 73729 40962 32767 90111 73726 32768 57344 73728 32769 155649 73730 65535 188415 139262 65536 122880 139264 65537 319489 139266 131071 385023 270334 131072 253952 270336 131073 647169 270338 262143 778239 532478 262144 516096 532480 262145 1302529 532482 In general if the size of the last intermediate buffer in the old version does not equal the total number of bytes read, then the old version will require more memory thaw the new. The performance for the same set of lengths was measured using JMH: Benchmark (base) (offset) Mode Cnt Score Error Units BenchReadAllBytesParam.readAllBytes 8192 -1 thrpt 5 578064.253 ? 18955.667 ops/s BenchReadAllBytesParam.readAllBytesArrayList 8192 -1 thrpt 5 530963.634 ? 4799.923 ops/s BenchReadAllBytesParam.readAllBytes 8192 0 thrpt 5 1414243.593 ? 68520.245 ops/s BenchReadAllBytesParam.readAllBytesArrayList 8192 0 thrpt 5 532019.149 ? 7051.738 ops/s BenchReadAllBytesParam.readAllBytes 8192 1 thrpt 5 293586.365 ? 8196.939 ops/s BenchReadAllBytesParam.readAllBytesArrayList 8192 1 thrpt 5 361682.265 ? 27081.111 ops/s BenchReadAllBytesParam.readAllBytes 16384 -1 thrpt 5 216809.517 ? 4853.852 ops/s BenchReadAllBytesParam.readAllBytesArrayList 16384 -1 thrpt 5 212346.244 ? 2980.916 ops/s BenchReadAllBytesParam.readAllBytes 16384 0 thrpt 5 379757.470 ? 14524.249 ops/s BenchReadAllBytesParam.readAllBytesArrayList 16384 0 thrpt 5 218802.256 ? 4361.080 ops/s BenchReadAllBytesParam.readAllBytes 16384 1 thrpt 5 123570.712 ? 1665.661 ops/s BenchReadAllBytesParam.readAllBytesArrayList 16384 1 thrpt 5 208801.577 ? 8005.196 ops/s BenchReadAllBytesParam.readAllBytes 32768 -1 thrpt 5 93083.146 ? 1024.309 ops/s BenchReadAllBytesParam.readAllBytesArrayList 32768 -1 thrpt 5 104398.304 ? 1310.509 ops/s BenchReadAllBytesParam.readAllBytes 32768 0 thrpt 5 151104.878 ? 3461.359 ops/s BenchReadAllBytesParam.readAllBytesArrayList 32768 0 thrpt 5 104849.088 ? 3841.224 ops/s BenchReadAllBytesParam.readAllBytes 32768 1 thrpt 5 58039.827 ? 398.908 ops/s BenchReadAllBytesParam.readAllBytesArrayList 32768 1 thrpt 5 104489.685 ? 2118.496 ops/s BenchReadAllBytesParam.readAllBytes 65536 -1 thrpt 5 43144.695 ? 440.349 ops/s BenchReadAllBytesParam.readAllBytesArrayList 65536 -1 thrpt 5 55583.338 ? 2115.162 ops/s BenchReadAllBytesParam.readAllBytes 65536 0 thrpt 5 67216.536 ? 2055.057 ops/s BenchReadAllBytesParam.readAllBytesArrayList 65536 0 thrpt 5 55680.238 ? 1235.571 ops/s BenchReadAllBytesParam.readAllBytes 65536 1 thrpt 5 27908.000 ? 568.473 ops/s BenchReadAllBytesParam.readAllBytesArrayList 65536 1 thrpt 5 55938.779 ? 813.991 ops/s BenchReadAllBytesParam.readAllBytes 131072 -1 thrpt 5 20299.014 ? 568.616 ops/s BenchReadAllBytesParam.readAllBytesArrayList 131072 -1 thrpt 5 28115.036 ? 842.392 ops/s BenchReadAllBytesParam.readAllBytes 131072 0 thrpt 5 31617.475 ? 467.378 ops/s BenchReadAllBytesParam.readAllBytesArrayList 131072 0 thrpt 5 28274.289 ? 259.699 ops/s BenchReadAllBytesParam.readAllBytes 131072 1 thrpt 5 13640.406 ? 303.125 ops/s BenchReadAllBytesParam.readAllBytesArrayList 131072 1 thrpt 5 28221.298 ? 515.030 ops/s BenchReadAllBytesParam.readAllBytes 262144 -1 thrpt 5 9995.183 ? 249.368 ops/s BenchReadAllBytesParam.readAllBytesArrayList 262144 -1 thrpt 5 14043.194 ? 138.026 ops/s BenchReadAllBytesParam.readAllBytes 262144 0 thrpt 5 15309.238 ? 353.752 ops/s BenchReadAllBytesParam.readAllBytesArrayList 262144 0 thrpt 5 14048.569 ? 348.699 ops/s BenchReadAllBytesParam.readAllBytes 262144 1 thrpt 5 5940.442 ? 206.855 ops/s BenchReadAllBytesParam.readAllBytesArrayList 262144 1 thrpt 5 14055.357 ? 412.470 ops/s In each case the number of bytes read is the sum of the base and offset parameters. Similar behavior with respect to data length is observed for performance as for memory usage: if the data length is not equal to the size of the last intermediate buffer used, then the performance of the old version is in general worse than that of the new. Thanks, Brian From paul.sandoz at oracle.com Tue Dec 19 20:52:03 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 19 Dec 2017 12:52:03 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: Message-ID: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> Hi, For the case of reading 2^N bytes i believe you can avoid doing a last copy by checking if ?n < 0" within the ?nread > 0? block when ?nread == DEAFULT_BUFFER_SIZE?. That might close the perf gap for smaller cases. You can also move "nread = 0? to the same block e.g.: var copy = (n < 0 && nread == DEAFULT_BUFFER_SIZE) ? buf : Arrays.copyOf(buf, nread); list.add(copy) nread = 0; 262 byte[] output = new byte[total]; 263 int offset = 0; 264 int numCached = list.size(); 265 for (int i = 0; i < numCached; i++) { 266 byte[] b = list.get(i); 267 System.arraycopy(b, 0, output, offset, b.length); 268 offset += b.length; 269 } You can simplify to: var result = new byte[total]; int offset = 0; for (buf : list) { System.arraycopy(buf, 0, result, offset, buf.length); offset += buf.length; } s/list/bufs and then you can use var for the declarations at the start of the method. Paul. > On 19 Dec 2017, at 11:57, Brian Burkhalter wrote: > > https://bugs.openjdk.java.net/browse/JDK-8193832 > http://cr.openjdk.java.net/~bpb/8193832/webrev.00/ > > The implementation of InputStream.readAllBytes() can be modified to be in general faster and to require less memory. The current implementation starts with an internal buffer of size 8192 bytes and doubles the size of the buffer each time more capacity is required resulting in a geometric progression in the buffer size. At the end if the buffer size is not exactly equal to the total number of bytes read, then another allocation equal to the total number read is performed. The amount of memory N required can be calculated for initial buffer size B and total bytes read L as > > M for L == B*(2^n) > N = > M + L for L != B*(2^n) > > where M = B*(2^(n + 1) - 1) and n = ceil(log2(L/B)). > > An alternative implementation is not to increase the size of the internal buffer each time more capacity is needed, but to instead maintain a list of buffers of up to B bytes each and gather those into an output buffer at the end. If there is only a single buffer in the list then gathering is not needed and it can be returned directly. The amount of memory N required in this case is > > B + L for L <= B > N = > B + 2*L for L > B > > A comparison of memory usage for a number of lengths L with a buffer size B of 8192 is: > > L N_old N_new > 8191 16383 16383 > 8192 8192 16384 > 8193 32769 24578 > 16383 40959 40958 > 16384 24576 40960 > 16385 73729 40962 > 32767 90111 73726 > 32768 57344 73728 > 32769 155649 73730 > 65535 188415 139262 > 65536 122880 139264 > 65537 319489 139266 > 131071 385023 270334 > 131072 253952 270336 > 131073 647169 270338 > 262143 778239 532478 > 262144 516096 532480 > 262145 1302529 532482 > > In general if the size of the last intermediate buffer in the old version does not equal the total number of bytes read, then the old version will require more memory thaw the new. The performance for the same set of lengths was measured using JMH: > > Benchmark (base) (offset) Mode Cnt Score Error Units > BenchReadAllBytesParam.readAllBytes 8192 -1 thrpt 5 578064.253 ? 18955.667 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 8192 -1 thrpt 5 530963.634 ? 4799.923 ops/s > BenchReadAllBytesParam.readAllBytes 8192 0 thrpt 5 1414243.593 ? 68520.245 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 8192 0 thrpt 5 532019.149 ? 7051.738 ops/s > BenchReadAllBytesParam.readAllBytes 8192 1 thrpt 5 293586.365 ? 8196.939 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 8192 1 thrpt 5 361682.265 ? 27081.111 ops/s > BenchReadAllBytesParam.readAllBytes 16384 -1 thrpt 5 216809.517 ? 4853.852 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 16384 -1 thrpt 5 212346.244 ? 2980.916 ops/s > BenchReadAllBytesParam.readAllBytes 16384 0 thrpt 5 379757.470 ? 14524.249 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 16384 0 thrpt 5 218802.256 ? 4361.080 ops/s > BenchReadAllBytesParam.readAllBytes 16384 1 thrpt 5 123570.712 ? 1665.661 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 16384 1 thrpt 5 208801.577 ? 8005.196 ops/s > BenchReadAllBytesParam.readAllBytes 32768 -1 thrpt 5 93083.146 ? 1024.309 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 32768 -1 thrpt 5 104398.304 ? 1310.509 ops/s > BenchReadAllBytesParam.readAllBytes 32768 0 thrpt 5 151104.878 ? 3461.359 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 32768 0 thrpt 5 104849.088 ? 3841.224 ops/s > BenchReadAllBytesParam.readAllBytes 32768 1 thrpt 5 58039.827 ? 398.908 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 32768 1 thrpt 5 104489.685 ? 2118.496 ops/s > BenchReadAllBytesParam.readAllBytes 65536 -1 thrpt 5 43144.695 ? 440.349 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 65536 -1 thrpt 5 55583.338 ? 2115.162 ops/s > BenchReadAllBytesParam.readAllBytes 65536 0 thrpt 5 67216.536 ? 2055.057 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 65536 0 thrpt 5 55680.238 ? 1235.571 ops/s > BenchReadAllBytesParam.readAllBytes 65536 1 thrpt 5 27908.000 ? 568.473 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 65536 1 thrpt 5 55938.779 ? 813.991 ops/s > BenchReadAllBytesParam.readAllBytes 131072 -1 thrpt 5 20299.014 ? 568.616 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 131072 -1 thrpt 5 28115.036 ? 842.392 ops/s > BenchReadAllBytesParam.readAllBytes 131072 0 thrpt 5 31617.475 ? 467.378 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 131072 0 thrpt 5 28274.289 ? 259.699 ops/s > BenchReadAllBytesParam.readAllBytes 131072 1 thrpt 5 13640.406 ? 303.125 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 131072 1 thrpt 5 28221.298 ? 515.030 ops/s > BenchReadAllBytesParam.readAllBytes 262144 -1 thrpt 5 9995.183 ? 249.368 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 262144 -1 thrpt 5 14043.194 ? 138.026 ops/s > BenchReadAllBytesParam.readAllBytes 262144 0 thrpt 5 15309.238 ? 353.752 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 262144 0 thrpt 5 14048.569 ? 348.699 ops/s > BenchReadAllBytesParam.readAllBytes 262144 1 thrpt 5 5940.442 ? 206.855 ops/s > BenchReadAllBytesParam.readAllBytesArrayList 262144 1 thrpt 5 14055.357 ? 412.470 ops/s > > In each case the number of bytes read is the sum of the base and offset parameters. Similar behavior with respect to data length is observed for performance as for memory usage: if the data length is not equal to the size of the last intermediate buffer used, then the performance of the old version is in general worse than that of the new. > > Thanks, > > Brian From forax at univ-mlv.fr Tue Dec 19 21:35:18 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 19 Dec 2017 22:35:18 +0100 (CET) Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> Message-ID: <1127305313.1015237.1513719318926.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Paul Sandoz" > ?: "Brian Burkhalter" > Cc: "core-libs-dev" > Envoy?: Mardi 19 D?cembre 2017 21:52:03 > Objet: Re: RFR 8193832: Performance of InputStream.readAllBytes() could be improved > Hi, > > For the case of reading 2^N bytes i believe you can avoid doing a last copy by > checking if ?n < 0" within the ?nread > 0? block when ?nread == > DEAFULT_BUFFER_SIZE?. That might close the perf gap for smaller cases. You can > also move "nread = 0? to the same block e.g.: > > var copy = (n < 0 && nread == DEAFULT_BUFFER_SIZE) ? buf : Arrays.copyOf(buf, > nread); > list.add(copy) > nread = 0; > > > 262 byte[] output = new byte[total]; > 263 int offset = 0; > 264 int numCached = list.size(); > 265 for (int i = 0; i < numCached; i++) { > 266 byte[] b = list.get(i); > 267 System.arraycopy(b, 0, output, offset, b.length); > 268 offset += b.length; > 269 } > > You can simplify to: > > var result = new byte[total]; > int offset = 0; > for (buf : list) { > System.arraycopy(buf, 0, result, offset, buf.length); > offset += buf.length; > } > > s/list/bufs and then you can use var for the declarations at the start of the > method. > > Paul. About using var, IMO var declaration makes usually the code more readable apart if you mix var declaration and classical declaration and if you call a method that has several overloads. Is there a usage guide somewhere ? R?mi From brian.burkhalter at oracle.com Tue Dec 19 21:43:28 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 19 Dec 2017 13:43:28 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> Message-ID: <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> On Dec 19, 2017, at 12:52 PM, Paul Sandoz wrote: > For the case of reading 2^N bytes i believe you can avoid doing a last copy by checking if ?n < 0" within the ?nread > 0? block when ?nread == DEAFULT_BUFFER_SIZE?. That might close the perf gap for smaller cases. You can also move "nread = 0? to the same block e.g.: > > var copy = (n < 0 && nread == DEAFULT_BUFFER_SIZE) ? buf : Arrays.copyOf(buf, nread); > list.add(copy) > nread = 0; Definitely improves performance and memory footprint for this case. > 262 byte[] output = new byte[total]; > 263 int offset = 0; > 264 int numCached = list.size(); > 265 for (int i = 0; i < numCached; i++) { > 266 byte[] b = list.get(i); > 267 System.arraycopy(b, 0, output, offset, b.length); > 268 offset += b.length; > 269 } > > You can simplify to: > > var result = new byte[total]; > int offset = 0; > for (buf : list) { > System.arraycopy(buf, 0, result, offset, buf.length); > offset += buf.length; > } Oh, yes, of course. > s/list/bufs and then you can use var for the declarations at the start of the method. Done. Updated: http://cr.openjdk.java.net/~bpb/8193832/webrev.01/ Good suggestions! Not sure however about line 237 as with var it has to be ?var n = 0;? instead of simply ?int n;? with no initialization. Also I?ve not tested the effect of the initial List capacity at line 233: the current value of 128 is arbitrary - might be better to go with the default? Thanks, Brian From paul.sandoz at oracle.com Tue Dec 19 22:28:48 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 19 Dec 2017 14:28:48 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> Message-ID: <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> > On 19 Dec 2017, at 13:43, Brian Burkhalter wrote: > > On Dec 19, 2017, at 12:52 PM, Paul Sandoz > wrote: > >> For the case of reading 2^N bytes i believe you can avoid doing a last copy by checking if ?n < 0" within the ?nread > 0? block when ?nread == DEAFULT_BUFFER_SIZE?. That might close the perf gap for smaller cases. You can also move "nread = 0? to the same block e.g.: >> >> var copy = (n < 0 && nread == DEAFULT_BUFFER_SIZE) ? buf : Arrays.copyOf(buf, nread); >> list.add(copy) >> nread = 0; > > Definitely improves performance and memory footprint for this case. > >> 262 byte[] output = new byte[total]; >> 263 int offset = 0; >> 264 int numCached = list.size(); >> 265 for (int i = 0; i < numCached; i++) { >> 266 byte[] b = list.get(i); >> 267 System.arraycopy(b, 0, output, offset, b.length); >> 268 offset += b.length; >> 269 } >> >> You can simplify to: >> >> var result = new byte[total]; >> int offset = 0; >> for (buf : list) { >> System.arraycopy(buf, 0, result, offset, buf.length); >> offset += buf.length; >> } > > Oh, yes, of course. > >> s/list/bufs and then you can use var for the declarations at the start of the method. > > Done. Updated: http://cr.openjdk.java.net/~bpb/8193832/webrev.01/ > You can also simplify the ?for(;;) + break" into a do while loop: do { int nread = 0; ... } while (n > 0); > Good suggestions! Not sure however about line 237 as with var it has to be ?var n = 0;? instead of simply ?int n;? with no initialization. I was only suggesting it?s use for the byte[] and ArrayList. IMHO it?s a little subjective but there is less upside for int, although one can argue consistent application and explicit initialization is a good thing here. > Also I?ve not tested the effect of the initial List capacity at line 233: the current value of 128 is arbitrary - might be better to go with the default? > My inclination would be to use 8 instead of 128, that allows 2^16 in size by default, rather than 2^20. Paul. From brian.burkhalter at oracle.com Tue Dec 19 22:36:32 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 19 Dec 2017 14:36:32 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> Message-ID: <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> On Dec 19, 2017, at 2:28 PM, Paul Sandoz wrote: >> Done. Updated: http://cr.openjdk.java.net/~bpb/8193832/webrev.01/ >> > > You can also simplify the ?for(;;) + break" into a do while loop: > > do { > int nread = 0; > ... > } while (n > 0); Good suggestion but I think that this needs to be "while (n >= 0)." >> Good suggestions! Not sure however about line 237 as with var it has to be ?var n = 0;? instead of simply ?int n;? with no initialization. > > I was only suggesting it?s use for the byte[] and ArrayList. IMHO it?s a little subjective but there is less upside for int, although one can argue consistent application and explicit initialization is a good thing here. I had left the ints as ints in an intermediate copy before Remi?s message. I also prefer to leave them as ints. Brian From brian.burkhalter at oracle.com Tue Dec 19 23:22:51 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 19 Dec 2017 15:22:51 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> Message-ID: <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> On Dec 19, 2017, at 2:36 PM, Brian Burkhalter wrote: >> You can also simplify the ?for(;;) + break" into a do while loop: >> >> do { >> int nread = 0; >> ... >> } while (n > 0); > > Good suggestion but I think that this needs to be "while (n >= 0)." Updated version here: http://cr.openjdk.java.net/~bpb/8193832/webrev.02/ Thanks, Brian From sean.coffey at oracle.com Wed Dec 20 08:17:45 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 20 Dec 2017 08:17:45 +0000 Subject: RFR: JDK8u Backport of 8153955: increase java.util.logging.FileHandler MAX_LOCKS limit In-Reply-To: <1b4b3c1e-a35d-453c-aa66-aee973405d15@default> References: <1b4b3c1e-a35d-453c-aa66-aee973405d15@default> Message-ID: <21e3851b-e57f-1fbe-67cd-a3a901c896c3@oracle.com> Looks fine to me. regards, Sean. On 18/12/2017 11:41, Ramanand Patil wrote: > Hi all, > Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/browse/JDK-8153955 > Backport Bug: https://bugs.openjdk.java.net/browse/JDK-8161266 > Webrev: http://cr.openjdk.java.net/~rpatil/8161266/webrev.00/ > > Summary(also added to backport bug description): > The fix from JDK9 cannot be backported as is into the jdk8u and earlier update releases, since it contains JDK API spec changes. > In JDK9 the fix is added by altering the FileHandler spec, which introduces a new configurable property "java.util.logging.FileHandler.maxLocks" to java.util.logging.FileHandler, defined in .../conf/logging.properties. > The solution proposed for update releases is: > To introduce an internal JDK implementation specific property- "jdk.internal.FileHandlerLogging.maxLocks" which will be used to control/override FileHandler's MAX_LOCKS value. The default value of the maxLocks (100) will be retained if this new System property is not set. The new property is read only once during FileHandler class initialization. > > > Regards, > Ramanand. From matthias.baesken at sap.com Wed Dec 20 08:50:54 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 20 Dec 2017 08:50:54 +0000 Subject: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad Message-ID: <6838c67f5a7f44c0a48a79e8ac47e889@sap.com> Hello , Mark reported this issue on AIX with OpenJDK8 : > >I'm getting an unsatisfied link error when using logstash on AIX but I suspect the issue is with AIX or the JRE, the exception is: > > java.lang.UnsatisfiedLinkError: sun.management.OperatingSystemImpl.getProcessCpuLoad()D > getProcessCpuLoad at sun/management/OperatingSystemImpl.java:-2 > at org/logstash/instrument/monitors/ProcessMonitor.java:40 > detect at org/logstash/instrument/monitors/ProcessMonitor.java:79 > generate at org/logstash/instrument/reports/ProcessReport.java:15 > > this is the line in logstash: > > this.cpuProcessPercent = scaleLoadToPercent(unixOsBean.getProcessCpuLoad()); > > https://github.com/elastic/logstash/blob/master/logstash-core/src/main/java/org/logstash/instrument/monitors/ProcessMonitor.java > > > Could anybody help steer me in the right direction? > This fix addresses the missing getSystemCpuLoad and getProcessCpuLoad on AIX . JDK8 is only affected , JDK9 and higher is not affected . Could I get a review for this change ? Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8193807/ Bug : https://bugs.openjdk.java.net/browse/JDK-8193807 Thanks, Matthias From daniel.fuchs at oracle.com Wed Dec 20 09:13:03 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 20 Dec 2017 09:13:03 +0000 Subject: RFR: JDK8u Backport of 8153955: increase java.util.logging.FileHandler MAX_LOCKS limit In-Reply-To: <1b4b3c1e-a35d-453c-aa66-aee973405d15@default> References: <1b4b3c1e-a35d-453c-aa66-aee973405d15@default> Message-ID: Hi Ramanand, On 18/12/2017 11:41, Ramanand Patil wrote: > Hi all, > Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/browse/JDK-8153955 > Backport Bug: https://bugs.openjdk.java.net/browse/JDK-8161266 > Webrev: http://cr.openjdk.java.net/~rpatil/8161266/webrev.00/ Looks good to me as well. best regards, -- daniel > > Summary(also added to backport bug description): > The fix from JDK9 cannot be backported as is into the jdk8u and earlier update releases, since it contains JDK API spec changes. > In JDK9 the fix is added by altering the FileHandler spec, which introduces a new configurable property "java.util.logging.FileHandler.maxLocks" to java.util.logging.FileHandler, defined in .../conf/logging.properties. > The solution proposed for update releases is: > To introduce an internal JDK implementation specific property- "jdk.internal.FileHandlerLogging.maxLocks" which will be used to control/override FileHandler's MAX_LOCKS value. The default value of the maxLocks (100) will be retained if this new System property is not set. The new property is read only once during FileHandler class initialization. > > > Regards, > Ramanand. > From volker.simonis at gmail.com Wed Dec 20 09:32:45 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 20 Dec 2017 10:32:45 +0100 Subject: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad In-Reply-To: <6838c67f5a7f44c0a48a79e8ac47e889@sap.com> References: <6838c67f5a7f44c0a48a79e8ac47e889@sap.com> Message-ID: Hi Matthias, the change looks good! I can sponsor it once we get the approval. Also forwarded to buil-dev for the minimal build change. Thank you and best regards, Volker On Wed, Dec 20, 2017 at 9:50 AM, Baesken, Matthias wrote: > Hello , Mark reported this issue on AIX with OpenJDK8 : > >> >>I'm getting an unsatisfied link error when using logstash on AIX but I suspect the issue is with AIX or the JRE, the exception is: >> >> java.lang.UnsatisfiedLinkError: sun.management.OperatingSystemImpl.getProcessCpuLoad()D >> getProcessCpuLoad at sun/management/OperatingSystemImpl.java:-2 >> at org/logstash/instrument/monitors/ProcessMonitor.java:40 >> detect at org/logstash/instrument/monitors/ProcessMonitor.java:79 >> generate at org/logstash/instrument/reports/ProcessReport.java:15 >> >> this is the line in logstash: >> >> this.cpuProcessPercent = scaleLoadToPercent(unixOsBean.getProcessCpuLoad()); >> >> https://github.com/elastic/logstash/blob/master/logstash-core/src/main/java/org/logstash/instrument/monitors/ProcessMonitor.java >> >> >> Could anybody help steer me in the right direction? >> > > This fix addresses the missing getSystemCpuLoad and getProcessCpuLoad on AIX . > JDK8 is only affected , JDK9 and higher is not affected . > > Could I get a review for this change ? > > > Change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8193807/ > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8193807 > > > Thanks, Matthias From david.holmes at oracle.com Wed Dec 20 10:22:36 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Dec 2017 20:22:36 +1000 Subject: [11] RFR (XS) 8193838: Update jtreg requiredVersion to 4.2 b11 for JDK 11 and 12 support Message-ID: <7b8e93f8-e465-ec13-215b-f33c16d29d02@oracle.com> Before we can update to JDK version 11, jtreg needs to be updated to recognize that release value - which is happening in jtreg 4.2 b11. So we then need to ensure that jtreg 4.2 b11 is used by updating the requiredVersion in each of the TEST.ROOT files, and in the jib configuiration. bug: https://bugs.openjdk.java.net/browse/JDK-8193838 webrev: http://cr.openjdk.java.net/~dholmes/8193838/webrev/ Testing - TBD once b11 is promoted - local build using jib - hs/jdk tier1 and tier2 Thanks, David From erik.joelsson at oracle.com Wed Dec 20 10:25:36 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 20 Dec 2017 11:25:36 +0100 Subject: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad In-Reply-To: References: <6838c67f5a7f44c0a48a79e8ac47e889@sap.com> Message-ID: Looks ok to me. /Erik On 2017-12-20 10:32, Volker Simonis wrote: > Hi Matthias, > > the change looks good! > I can sponsor it once we get the approval. > > Also forwarded to buil-dev for the minimal build change. > > Thank you and best regards, > Volker > > > On Wed, Dec 20, 2017 at 9:50 AM, Baesken, Matthias > wrote: >> Hello , Mark reported this issue on AIX with OpenJDK8 : >> >>> I'm getting an unsatisfied link error when using logstash on AIX but I suspect the issue is with AIX or the JRE, the exception is: >>> >>> java.lang.UnsatisfiedLinkError: sun.management.OperatingSystemImpl.getProcessCpuLoad()D >>> getProcessCpuLoad at sun/management/OperatingSystemImpl.java:-2 >>> at org/logstash/instrument/monitors/ProcessMonitor.java:40 >>> detect at org/logstash/instrument/monitors/ProcessMonitor.java:79 >>> generate at org/logstash/instrument/reports/ProcessReport.java:15 >>> >>> this is the line in logstash: >>> >>> this.cpuProcessPercent = scaleLoadToPercent(unixOsBean.getProcessCpuLoad()); >>> >>> https://github.com/elastic/logstash/blob/master/logstash-core/src/main/java/org/logstash/instrument/monitors/ProcessMonitor.java >>> >>> >>> Could anybody help steer me in the right direction? >>> >> This fix addresses the missing getSystemCpuLoad and getProcessCpuLoad on AIX . >> JDK8 is only affected , JDK9 and higher is not affected . >> >> Could I get a review for this change ? >> >> >> Change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8193807/ >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8193807 >> >> >> Thanks, Matthias From Alan.Bateman at oracle.com Wed Dec 20 10:32:36 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Dec 2017 10:32:36 +0000 Subject: [11] RFR (XS) 8193838: Update jtreg requiredVersion to 4.2 b11 for JDK 11 and 12 support In-Reply-To: <7b8e93f8-e465-ec13-215b-f33c16d29d02@oracle.com> References: <7b8e93f8-e465-ec13-215b-f33c16d29d02@oracle.com> Message-ID: <7a8316ea-5979-d30b-54fc-b96bffaef2cc@oracle.com> On 20/12/2017 10:22, David Holmes wrote: > Before we can update to JDK version 11, jtreg needs to be updated to > recognize that release value - which is happening in jtreg 4.2 b11. So > we then need to ensure that jtreg 4.2 b11 is used by updating the > requiredVersion in each of the TEST.ROOT files, and in the jib > configuiration. > > bug: https://bugs.openjdk.java.net/browse/JDK-8193838 > webrev: http://cr.openjdk.java.net/~dholmes/8193838/webrev/ Looks fine. From serguei.spitsyn at oracle.com Wed Dec 20 10:41:47 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 20 Dec 2017 02:41:47 -0800 Subject: [11] RFR (XS) 8193838: Update jtreg requiredVersion to 4.2 b11 for JDK 11 and 12 support In-Reply-To: <7b8e93f8-e465-ec13-215b-f33c16d29d02@oracle.com> References: <7b8e93f8-e465-ec13-215b-f33c16d29d02@oracle.com> Message-ID: <933a8584-e101-facd-a7ee-21583b546707@oracle.com> David, It looks good. Thanks, Serguei On 12/20/17 02:22, David Holmes wrote: > Before we can update to JDK version 11, jtreg needs to be updated to > recognize that release value - which is happening in jtreg 4.2 b11. So > we then need to ensure that jtreg 4.2 b11 is used by updating the > requiredVersion in each of the TEST.ROOT files, and in the jib > configuiration. > > bug: https://bugs.openjdk.java.net/browse/JDK-8193838 > webrev: http://cr.openjdk.java.net/~dholmes/8193838/webrev/ > > Testing - TBD once b11 is promoted > ?- local build using jib > ?- hs/jdk tier1 and tier2 > > Thanks, > David From peter.levart at gmail.com Wed Dec 20 11:45:03 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 20 Dec 2017 12:45:03 +0100 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> Message-ID: Hi Brian, On 12/20/2017 12:22 AM, Brian Burkhalter wrote: > On Dec 19, 2017, at 2:36 PM, Brian Burkhalter wrote: > >>> You can also simplify the ?for(;;) + break" into a do while loop: >>> >>> do { >>> int nread = 0; >>> ... >>> } while (n > 0); >> Good suggestion but I think that this needs to be "while (n >= 0)." > Updated version here: > > http://cr.openjdk.java.net/~bpb/8193832/webrev.02/ > > Thanks, > > Brian Looks good. There is one case that could be further optimized. When result.length <= DEFAULT_BUFFER_SIZE, the allocation of ArrayList could be avoided. Imagine a use case where lots of small files are read into byte[] arrays. For exmaple: ??? public byte[] readAllBytes() throws IOException { ??????? List bufs = null; ??????? byte[] result = null; ??????? byte[] buf = new byte[DEFAULT_BUFFER_SIZE]; ??????? int total = 0; ??????? int n; ??????? do { ??????????? int nread = 0; ??????????? // read to EOF which may read more or less than buffer size ??????????? while ((n = read(buf, nread, buf.length - nread)) > 0) { ??????????????? nread += n; ??????????? } ??????????? if (nread > 0) { ??????????????? if (MAX_BUFFER_SIZE - total < nread) { ??????????????????? throw new OutOfMemoryError("Required array size too large"); ??????????????? } ??????????????? total += nread; ??????????????? byte[] copy = (n < 0 && nread == DEFAULT_BUFFER_SIZE) ? ??????????????????? buf : Arrays.copyOf(buf, nread); ??????????????? if (result == null) { ??????????????????? result = copy; ??????????????? } else { ??????????????????? bufs = new ArrayList<>(8); ??????????????????? bufs.add(result); ??????????????????? bufs.add(copy); ??????????????? } ??????????? } ??????? } while (n >= 0); // if the last call to read returned -1, then break ??????? if (bufs == null) { ??????????? return result == null ? new byte[0] : result; ??????? } ??????? result = new byte[total]; ??????? int offset = 0; ??????? for (byte[] b : bufs) { ??????????? System.arraycopy(b, 0, result, offset, b.length); ??????????? offset += b.length; ??????? } ??????? return result; ??? } There is a possibility that JIT already avoids allocating ArrayList utilizing EA if all involved ArrayList methods inline, so this potential optimization should be tested 1st to see if it actually helps improve the "small file" case. Regards, Peter From peter.levart at gmail.com Wed Dec 20 11:59:41 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 20 Dec 2017 12:59:41 +0100 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> Message-ID: <3e4e52ba-60f3-bad1-daff-63a129d524e1@gmail.com> Hi Brian, There's also a variation of copy-ing fragment possible, that replaces copying with allocation: ??????????????? byte[] copy; ??????????????? if (nread == DEFAULT_BUFFER_SIZE) { ??????????????????? copy = buf; ??????????????????? if (n >= 0) { ??????????????????????? buf = new byte[DEFAULT_BUFFER_SIZE]; ??????????????????? } ??????????????? } else { ??????????????????? copy = Arrays.copyOf(buf, nread); ??????????????? } For big FileInputStream(s), the buf will be fully read (nread == DEFAULT_BUFFER_SIZE) most of the times. So this might be an improvement if allocation (involving pre-zeroing) is faster than Arrays.copyOf() which avoids pre-zeroing, but involves copying. Regards, Peter On 12/20/2017 12:45 PM, Peter Levart wrote: > Hi Brian, > > On 12/20/2017 12:22 AM, Brian Burkhalter wrote: >> On Dec 19, 2017, at 2:36 PM, Brian Burkhalter >> wrote: >> >>>> You can also simplify the ?for(;;) + break" into a do while loop: >>>> >>>> do { >>>> ? int nread = 0; >>>> ? ... >>>> } while (n > 0); >>> Good suggestion but I think that this needs to be "while (n >= 0)." >> Updated version here: >> >> http://cr.openjdk.java.net/~bpb/8193832/webrev.02/ >> >> Thanks, >> >> Brian > > Looks good. There is one case that could be further optimized. When > result.length <= DEFAULT_BUFFER_SIZE, the allocation of ArrayList > could be avoided. Imagine a use case where lots of small files are > read into byte[] arrays. For exmaple: > > ??? public byte[] readAllBytes() throws IOException { > ??????? List bufs = null; > ??????? byte[] result = null; > ??????? byte[] buf = new byte[DEFAULT_BUFFER_SIZE]; > ??????? int total = 0; > ??????? int n; > ??????? do { > ??????????? int nread = 0; > > ??????????? // read to EOF which may read more or less than buffer size > ??????????? while ((n = read(buf, nread, buf.length - nread)) > 0) { > ??????????????? nread += n; > ??????????? } > > ??????????? if (nread > 0) { > ??????????????? if (MAX_BUFFER_SIZE - total < nread) { > ??????????????????? throw new OutOfMemoryError("Required array size > too large"); > ??????????????? } > ??????????????? total += nread; > ??????????????? byte[] copy = (n < 0 && nread == DEFAULT_BUFFER_SIZE) ? > ??????????????????? buf : Arrays.copyOf(buf, nread); > ??????????????? if (result == null) { > ??????????????????? result = copy; > ??????????????? } else { > ??????????????????? bufs = new ArrayList<>(8); > ??????????????????? bufs.add(result); > ??????????????????? bufs.add(copy); > ??????????????? } > ??????????? } > ??????? } while (n >= 0); // if the last call to read returned -1, > then break > > ??????? if (bufs == null) { > ??????????? return result == null ? new byte[0] : result; > ??????? } > > ??????? result = new byte[total]; > ??????? int offset = 0; > ??????? for (byte[] b : bufs) { > ??????????? System.arraycopy(b, 0, result, offset, b.length); > ??????????? offset += b.length; > ??????? } > > ??????? return result; > ??? } > > > There is a possibility that JIT already avoids allocating ArrayList > utilizing EA if all involved ArrayList methods inline, so this > potential optimization should be tested 1st to see if it actually > helps improve the "small file" case. > > Regards, Peter > From peter.levart at gmail.com Wed Dec 20 12:40:32 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 20 Dec 2017 13:40:32 +0100 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <3e4e52ba-60f3-bad1-daff-63a129d524e1@gmail.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <3e4e52ba-60f3-bad1-daff-63a129d524e1@gmail.com> Message-ID: <1c72d6ee-79ac-dfbb-502d-0c8041966ce7@gmail.com> Hi Brian, I found another improvement. If you are reading from files, there's no difference. But if you read from say socket(s), there may be short reads (read() returning 0). Your current code bails out from inner loop when this happens and consequently does not fill the buf up to the top when it could fill it if it retried the inner loop. This is a variant where inner loop guarantees that either buf is filled to the top or stream is at EOF: ??? public byte[] readAllBytes() throws IOException { ??????? List bufs = null; ??????? byte[] result = null; ??????? byte[] buf = new byte[DEFAULT_BUFFER_SIZE]; ??????? int total = 0; ??????? int remaining; // # of bytes remaining to fill buf ??????? do { ??????????? remaining = buf.length; ??????????? int n; ??????????? // read to fill the buf or until EOF. Loop ends when either: ??????????? // - remaining == 0 and there's possibly more to be read from stream; or ??????????? // - remaining > 0 and the stream is at EOF ??????????? while (remaining > 0 && ?????????????????? (n = read(buf, buf.length - remaining, remaining)) >= 0) { ??????????????? remaining -= n; ??????????? } ??????????? int nread = buf.length - remaining; ??????????? if (nread > 0) { ??????????????? if (MAX_BUFFER_SIZE - total < nread) { ??????????????????? throw new OutOfMemoryError("Required array size too large"); ??????????????? } ??????????????? total += nread; ??????????????? byte[] copy; ??????????????? if (remaining == 0) { // buf is filled to the top ??????????????????? copy = buf; ??????????????????? buf = new byte[DEFAULT_BUFFER_SIZE]; ??????????????? } else { ??????????????????? copy = Arrays.copyOf(buf, nread); ??????????????? } ??????????????? if (result == null) { ??????????????????? result = copy; ??????????????? } else { ??????????????????? bufs = new ArrayList<>(8); ??????????????????? bufs.add(result); ??????????????????? bufs.add(copy); ??????????????? } ??????????? } ??????? } while (remaining == 0); // there may be more bytes in stream ??????? if (bufs == null) { ??????????? return result == null ? new byte[0] : result; ??????? } ??????? result = new byte[total]; ??????? int offset = 0; ??????? for (byte[] b : bufs) { ??????????? System.arraycopy(b, 0, result, offset, b.length); ??????????? offset += b.length; ??????? } ??????? return result; ??? } Regards, Peter On 12/20/2017 12:59 PM, Peter Levart wrote: > Hi Brian, > > There's also a variation of copy-ing fragment possible, that replaces > copying with allocation: > > ??????????????? byte[] copy; > ??????????????? if (nread == DEFAULT_BUFFER_SIZE) { > ??????????????????? copy = buf; > ??????????????????? if (n >= 0) { > ??????????????????????? buf = new byte[DEFAULT_BUFFER_SIZE]; > ??????????????????? } > ??????????????? } else { > ??????????????????? copy = Arrays.copyOf(buf, nread); > ??????????????? } > > For big FileInputStream(s), the buf will be fully read (nread == > DEFAULT_BUFFER_SIZE) most of the times. So this might be an > improvement if allocation (involving pre-zeroing) is faster than > Arrays.copyOf() which avoids pre-zeroing, but involves copying. > > Regards, Peter > > On 12/20/2017 12:45 PM, Peter Levart wrote: >> Hi Brian, >> >> On 12/20/2017 12:22 AM, Brian Burkhalter wrote: >>> On Dec 19, 2017, at 2:36 PM, Brian Burkhalter >>> wrote: >>> >>>>> You can also simplify the ?for(;;) + break" into a do while loop: >>>>> >>>>> do { >>>>> ? int nread = 0; >>>>> ? ... >>>>> } while (n > 0); >>>> Good suggestion but I think that this needs to be "while (n >= 0)." >>> Updated version here: >>> >>> http://cr.openjdk.java.net/~bpb/8193832/webrev.02/ >>> >>> Thanks, >>> >>> Brian >> >> Looks good. There is one case that could be further optimized. When >> result.length <= DEFAULT_BUFFER_SIZE, the allocation of ArrayList >> could be avoided. Imagine a use case where lots of small files are >> read into byte[] arrays. For exmaple: >> >> ??? public byte[] readAllBytes() throws IOException { >> ??????? List bufs = null; >> ??????? byte[] result = null; >> ??????? byte[] buf = new byte[DEFAULT_BUFFER_SIZE]; >> ??????? int total = 0; >> ??????? int n; >> ??????? do { >> ??????????? int nread = 0; >> >> ??????????? // read to EOF which may read more or less than buffer size >> ??????????? while ((n = read(buf, nread, buf.length - nread)) > 0) { >> ??????????????? nread += n; >> ??????????? } >> >> ??????????? if (nread > 0) { >> ??????????????? if (MAX_BUFFER_SIZE - total < nread) { >> ??????????????????? throw new OutOfMemoryError("Required array size >> too large"); >> ??????????????? } >> ??????????????? total += nread; >> ??????????????? byte[] copy = (n < 0 && nread == DEFAULT_BUFFER_SIZE) ? >> ??????????????????? buf : Arrays.copyOf(buf, nread); >> ??????????????? if (result == null) { >> ??????????????????? result = copy; >> ??????????????? } else { >> ??????????????????? bufs = new ArrayList<>(8); >> ??????????????????? bufs.add(result); >> ??????????????????? bufs.add(copy); >> ??????????????? } >> ??????????? } >> ??????? } while (n >= 0); // if the last call to read returned -1, >> then break >> >> ??????? if (bufs == null) { >> ??????????? return result == null ? new byte[0] : result; >> ??????? } >> >> ??????? result = new byte[total]; >> ??????? int offset = 0; >> ??????? for (byte[] b : bufs) { >> ??????????? System.arraycopy(b, 0, result, offset, b.length); >> ??????????? offset += b.length; >> ??????? } >> >> ??????? return result; >> ??? } >> >> >> There is a possibility that JIT already avoids allocating ArrayList >> utilizing EA if all involved ArrayList methods inline, so this >> potential optimization should be tested 1st to see if it actually >> helps improve the "small file" case. >> >> Regards, Peter >> > From david.holmes at oracle.com Wed Dec 20 12:40:37 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Dec 2017 22:40:37 +1000 Subject: [11] RFR (XS) 8193838: Update jtreg requiredVersion to 4.2 b11 for JDK 11 and 12 support In-Reply-To: <7a8316ea-5979-d30b-54fc-b96bffaef2cc@oracle.com> References: <7b8e93f8-e465-ec13-215b-f33c16d29d02@oracle.com> <7a8316ea-5979-d30b-54fc-b96bffaef2cc@oracle.com> Message-ID: Thanks Alan. David On 20/12/2017 8:32 PM, Alan Bateman wrote: > > > On 20/12/2017 10:22, David Holmes wrote: >> Before we can update to JDK version 11, jtreg needs to be updated to >> recognize that release value - which is happening in jtreg 4.2 b11. So >> we then need to ensure that jtreg 4.2 b11 is used by updating the >> requiredVersion in each of the TEST.ROOT files, and in the jib >> configuiration. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8193838 >> webrev: http://cr.openjdk.java.net/~dholmes/8193838/webrev/ > Looks fine. From david.holmes at oracle.com Wed Dec 20 12:40:56 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Dec 2017 22:40:56 +1000 Subject: [11] RFR (XS) 8193838: Update jtreg requiredVersion to 4.2 b11 for JDK 11 and 12 support In-Reply-To: <933a8584-e101-facd-a7ee-21583b546707@oracle.com> References: <7b8e93f8-e465-ec13-215b-f33c16d29d02@oracle.com> <933a8584-e101-facd-a7ee-21583b546707@oracle.com> Message-ID: <6c438b05-1e97-f4da-b364-560af7939aaf@oracle.com> Thanks Serguei! David On 20/12/2017 8:41 PM, serguei.spitsyn at oracle.com wrote: > David, > > It looks good. > > Thanks, > Serguei > > > On 12/20/17 02:22, David Holmes wrote: >> Before we can update to JDK version 11, jtreg needs to be updated to >> recognize that release value - which is happening in jtreg 4.2 b11. So >> we then need to ensure that jtreg 4.2 b11 is used by updating the >> requiredVersion in each of the TEST.ROOT files, and in the jib >> configuiration. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8193838 >> webrev: http://cr.openjdk.java.net/~dholmes/8193838/webrev/ >> >> Testing - TBD once b11 is promoted >> ?- local build using jib >> ?- hs/jdk tier1 and tier2 >> >> Thanks, >> David > From ramanand.patil at oracle.com Wed Dec 20 13:59:38 2017 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Wed, 20 Dec 2017 05:59:38 -0800 (PST) Subject: RFR: JDK8u Backport of 8153955: increase java.util.logging.FileHandler MAX_LOCKS limit In-Reply-To: References: <1b4b3c1e-a35d-453c-aa66-aee973405d15@default> Message-ID: Thank you Daniel and Sean for your reviews. Regards, Ramanand. > -----Original Message----- > From: Daniel Fuchs > Sent: Wednesday, December 20, 2017 2:43 PM > To: Ramanand Patil ; core-libs-dev dev at openjdk.java.net> > Subject: Re: RFR: JDK8u Backport of 8153955: increase > java.util.logging.FileHandler MAX_LOCKS limit > > Hi Ramanand, > > On 18/12/2017 11:41, Ramanand Patil wrote: > > Hi all, > > Please review the fix for JDK8u Backport of > https://bugs.openjdk.java.net/browse/JDK-8153955 > > Backport Bug: https://bugs.openjdk.java.net/browse/JDK-8161266 > > Webrev: http://cr.openjdk.java.net/~rpatil/8161266/webrev.00/ > > Looks good to me as well. > > best regards, > > -- daniel > > > > > Summary(also added to backport bug description): > > The fix from JDK9 cannot be backported as is into the jdk8u and earlier > update releases, since it contains JDK API spec changes. > > In JDK9 the fix is added by altering the FileHandler spec, which introduces a > new configurable property "java.util.logging.FileHandler.maxLocks" to > java.util.logging.FileHandler, defined in .../conf/logging.properties. > > The solution proposed for update releases is: > > To introduce an internal JDK implementation specific property- > "jdk.internal.FileHandlerLogging.maxLocks" which will be used to > control/override FileHandler's MAX_LOCKS value. The default value of the > maxLocks (100) will be retained if this new System property is not set. The > new property is read only once during FileHandler class initialization. > > > > > > Regards, > > Ramanand. > > > From Alan.Bateman at oracle.com Wed Dec 20 14:09:34 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Dec 2017 14:09:34 +0000 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <1c72d6ee-79ac-dfbb-502d-0c8041966ce7@gmail.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <3e4e52ba-60f3-bad1-daff-63a129d524e1@gmail.com> <1c72d6ee-79ac-dfbb-502d-0c8041966ce7@gmail.com> Message-ID: <861e8718-3139-5f08-d7f9-0540d2e61bbe@oracle.com> On 20/12/2017 12:40, Peter Levart wrote: > Hi Brian, > > I found another improvement. If you are reading from files, there's no > difference. But if you read from say socket(s), there may be short > reads (read() returning 0). InputStreams are blocking so if someone creates an InputStream over a socket configured non-blocking then they have to emulate blocking behavior. So assuming a non-zero byte array, then read should return a positive value or -1. -Alan. From chris.hegarty at oracle.com Wed Dec 20 15:16:59 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Dec 2017 15:16:59 +0000 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> Message-ID: <106671ad-1a28-329e-a847-ce4e54eb8db8@oracle.com> On 19/12/17 23:22, Brian Burkhalter wrote: > .. > Updated version here: > > http://cr.openjdk.java.net/~bpb/8193832/webrev.02/ Looks good to me. This is a nice improvement on the original implementation that I added in 9. Thanks. -Chris. From peter.levart at gmail.com Wed Dec 20 16:04:18 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 20 Dec 2017 17:04:18 +0100 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <861e8718-3139-5f08-d7f9-0540d2e61bbe@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <3e4e52ba-60f3-bad1-daff-63a129d524e1@gmail.com> <1c72d6ee-79ac-dfbb-502d-0c8041966ce7@gmail.com> <861e8718-3139-5f08-d7f9-0540d2e61bbe@oracle.com> Message-ID: <3d5bd583-3635-5e71-1d1e-fd783a05c769@gmail.com> On 12/20/2017 03:09 PM, Alan Bateman wrote: > > > On 20/12/2017 12:40, Peter Levart wrote: >> Hi Brian, >> >> I found another improvement. If you are reading from files, there's >> no difference. But if you read from say socket(s), there may be short >> reads (read() returning 0). > InputStreams are blocking so if someone creates an InputStream over a > socket configured non-blocking then they have to emulate blocking > behavior. So assuming a non-zero byte array, then read should return a > positive value or -1. > > -Alan. You are right Alan. I don't know why I assumed that 0 is a valid return value (for non-empty array). So my last suggestion is unnecessary. Each buf will be filled to the top before inner loop exits or the stream will be at EOF. Regards, Peter From brian.burkhalter at oracle.com Wed Dec 20 16:14:50 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 20 Dec 2017 08:14:50 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <3d5bd583-3635-5e71-1d1e-fd783a05c769@gmail.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <3e4e52ba-60f3-bad1-daff-63a129d524e1@gmail.com> <1c72d6ee-79ac-dfbb-502d-0c8041966ce7@gmail.com> <861e8718-3139-5f08-d7f9-0540d2e61bbe@oracle.com> <3d5bd583-3635-5e71-1d1e-fd783a05c769@gmail.com> Message-ID: Hi Peter, On Dec 20, 2017, at 8:04 AM, Peter Levart wrote: >>> found another improvement. If you are reading from files, there's no difference. But if you read from say socket(s), there may be short reads (read() returning 0). >> InputStreams are blocking so if someone creates an InputStream over a socket configured non-blocking then they have to emulate blocking behavior. So assuming a non-zero byte array, then read should return a positive value or -1. >> >> -Alan. > > You are right Alan. I don't know why I assumed that 0 is a valid return value (for non-empty array). So my last suggestion is unnecessary. Each buf will be filled to the top before inner loop exits or the stream will be at EOF. Thanks for the suggestions. I?ll take a look at the first two later today. Brian From joe.darcy at oracle.com Wed Dec 20 17:14:25 2017 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 20 Dec 2017 09:14:25 -0800 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> Message-ID: <46a67401-92e4-09a2-dfdf-2c10a977a71e@oracle.com> Hi Rachna, I think the revised version with the @implSpec tag switch is acceptable, but also think providing more text to describe this situation would be helpful to programmers unaware of a 13 month possibility. Cheers, -Joe On 12/19/2017 2:08 AM, Rachna Goel wrote: > > Hi Joe, > > Thanks for the comments. > > I have updated the CSR to have @implSpec in place of @implNote. > > https://bugs.openjdk.java.net/browse/JDK-8191414 > > Regarding "An array with either 12 or 13 elements will be returned > depending on whether or {@link Calendar.UNDECIMBER} is supported." , > I would like to go with existing statement as this method always > returns 13 elements where the 13th element may be empty string or may > contain Calendar.UNDECIMBER, depending upon whether its supported by > the Calendar instance. > > kindly suggest whether this looks fine! > > Thanks, > Rachna > > > On 19/12/17 2:55 PM, joe darcy wrote: >> Hi Rachna, >> >> On 12/19/2017 1:13 AM, Rachna Goel wrote: >>> >>> Hello Joe, >>> >>> Thanks for the review. >>> >>> Reason I added @implNote is that it's the case for the default >>> implementation. Not added as a part of spec, as some implementation >>> can just return 12 element array for same methods through the >>> "java.text.spi.DateFormatSymbolsProvider" SPI. >>> >>> >> >> That is precisely the sort of situation the @implSpec tag is intended >> for. It allows the specification to say DateFormatSymbols must behave >> this way while allowing subclasses to behave differently. >> >> Perhaps some general text can be added as normal specification, >> something like >> >> "An array with either 12 or 13 elements will be returned depending on >> whether or {@link Calendar.UNDECIMBER} is supported." >> >> paired with >> >> @implSpec This method returns 13 elements since @link >> Calendar.UNDECIMBER} is supported. >> >> HTH, >> >> -Joe >> > > -- > Thanks, > Rachna From naoto.sato at oracle.com Wed Dec 20 18:19:04 2017 From: naoto.sato at oracle.com (Naoto Sato) Date: Wed, 20 Dec 2017 10:19:04 -0800 Subject: [PATCH] Support for UTC Zones with TimeZone.getTimeZone() In-Reply-To: References: Message-ID: <8287adfe-079c-be54-789a-2b609d5721e9@oracle.com> Hi Naufal, Thank you for the patch. It does seem to be a bug. However, TimeZone class defines CustomID as "GMT+(offset)", so I'd rather fix it in GregorianCalendar.from() to recognize those "UTC+offset" zones. Naoto On 12/16/17 2:27 AM, Mohamed Naufal wrote: > Hi, > > I noticed that with the following data: > > LocalDateTime ldt = LocalDateTime.parse("2017-01-01T00:00:00"); > ZonedDateTime dt1 = ZonedDateTime.of(ldt, ZoneId.of("GMT+10")); > ZonedDateTime dt2 = ZonedDateTime.of(ldt, ZoneId.of("UTC+10")); > > dt1.equals(dt2) returns true as expected, but with: > > GregorianCalendar gc1 = GregorianCalendar.from(dt1); > GregorianCalendar gc2 = GregorianCalendar.from(dt2); > > gc1.equals(gc2) returns false. > > Looking at the code, I see when a GregorianCalendar is being constructed, > TimeZone.getTimeZone() gets called, but it doesn't recognise UTC time-zones > with offsets, and falls back to GMT(+0), whereas ZoneId treats GMT and UTC > based zones equivalently. > > PFA a patch to fix this. > > Thank you, > Naufal > From chris.hegarty at oracle.com Wed Dec 20 18:42:30 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Dec 2017 18:42:30 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass Message-ID: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> In JDK 9 sun.reflect.Reflection.getCallerClass, and its jdk.internal counterpart, where both deprecated-for-removal ( to give prior warning of the unsupported private API's intended removal ). The supported replacement, since Java SE 9, is java.lang.StackWalker. http://cr.openjdk.java.net/~chegar/8179424/webrev/ -Chris. From lance.andersen at oracle.com Wed Dec 20 18:53:18 2017 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 20 Dec 2017 13:53:18 -0500 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> Message-ID: <6F235C65-1966-4AE8-A5E7-3B5B461B964F@oracle.com> looks fine Chris Best Lance > On Dec 20, 2017, at 1:42 PM, Chris Hegarty wrote: > > http://cr.openjdk.java.net/~chegar/8179424/webrev/ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From john.r.rose at oracle.com Wed Dec 20 19:04:11 2017 From: john.r.rose at oracle.com (John Rose) Date: Wed, 20 Dec 2017 11:04:11 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> Message-ID: <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> On Dec 20, 2017, at 3:45 AM, Peter Levart wrote: > > allocation of ArrayList could be avoided. Imagine a use case where lots of small files are read into byte[] arrays +1 I was going to write a similar suggestion; thanks for sending it out. These sorts of things often need to be designed to perform well at all scales, not just large scales. The new ArrayList(8) is dead weight if the data fits in the buffer. So it's bad for scale < buffer length. The earlier new ArrayList(128) is a noticeable overhead if the data fits in two buffers. So it's not so good for scale = M * (buffer length) where M is about 2. For a large scale result (M > 10), the footprint difference between ArrayList(N) for various N is a second-order fraction. ? John P.S. Road not taken: Instead of new ArrayList(8) you could use the default ArrayList constructor, and allocate it unconditionally. It has a very low overhead if the ArrayList remains empty. Using that constructor could motivate you to simplify the code to use the ArrayList unconditionally, since that would be just a single heap node if it is not used to gather multiple results. But I like the "null" formulation despite its complexity. So I'd probably keep it the way Peter wrote it. From Roger.Riggs at Oracle.com Wed Dec 20 19:06:39 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 20 Dec 2017 14:06:39 -0500 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <6F235C65-1966-4AE8-A5E7-3B5B461B964F@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <6F235C65-1966-4AE8-A5E7-3B5B461B964F@oracle.com> Message-ID: +1 On 12/20/2017 1:53 PM, Lance Andersen wrote: > looks fine Chris > > Best > Lance >> On Dec 20, 2017, at 1:42 PM, Chris Hegarty wrote: >> >> http://cr.openjdk.java.net/~chegar/8179424/webrev/ > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From mandy.chung at oracle.com Wed Dec 20 19:20:03 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 20 Dec 2017 11:20:03 -0800 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> Message-ID: <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> On 12/20/17 10:42 AM, Chris Hegarty wrote: > In JDK 9 sun.reflect.Reflection.getCallerClass, and its > jdk.internal counterpart, where both deprecated-for-removal > ( to give prior warning of the unsupported private API's > intended removal ). The supported replacement, since > Java SE 9, is java.lang.StackWalker. > > http://cr.openjdk.java.net/~chegar/8179424/webrev/ > +1.?? Happy to see this removed. thanks Mandy From mandy.chung at oracle.com Wed Dec 20 19:21:18 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 20 Dec 2017 11:21:18 -0800 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> Message-ID: <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> The native side and hotspot side should also be cleaned up. Mandy On 12/20/17 11:20 AM, mandy chung wrote: > > > On 12/20/17 10:42 AM, Chris Hegarty wrote: >> In JDK 9 sun.reflect.Reflection.getCallerClass, and its >> jdk.internal counterpart, where both deprecated-for-removal >> ( to give prior warning of the unsupported private API's >> intended removal ). The supported replacement, since >> Java SE 9, is java.lang.StackWalker. >> >> http://cr.openjdk.java.net/~chegar/8179424/webrev/ >> > > +1.?? Happy to see this removed. > > thanks > Mandy From daniel.fuchs at oracle.com Wed Dec 20 19:44:23 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 20 Dec 2017 19:44:23 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> Message-ID: <1e883dc5-6dbf-fc22-608f-cdb088793dab@oracle.com> Looks good Chris! -- daniel On 20/12/2017 18:42, Chris Hegarty wrote: > In JDK 9 sun.reflect.Reflection.getCallerClass, and its > jdk.internal counterpart, where both deprecated-for-removal > ( to give prior warning of the unsupported private API's > intended removal ). The supported replacement, since > Java SE 9, is java.lang.StackWalker. > > http://cr.openjdk.java.net/~chegar/8179424/webrev/ > > -Chris. From paul.sandoz at oracle.com Wed Dec 20 19:52:44 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Dec 2017 11:52:44 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> Message-ID: > On 20 Dec 2017, at 11:04, John Rose wrote: > > On Dec 20, 2017, at 3:45 AM, Peter Levart > wrote: >> >> allocation of ArrayList could be avoided. Imagine a use case where lots of small files are read into byte[] arrays > > +1 I was going to write a similar suggestion; thanks for sending it out. > I was a little lassiaz-faire given that 8K bytes were anyway being allocated upfront. Peter?s changes look good. Brian, i would double check the tests to make sure the various code paths are tested. Paul. > These sorts of things often need to be designed to perform well at all > scales, not just large scales. > > The new ArrayList(8) is dead weight if the data fits in the buffer. > So it's bad for scale < buffer length. > > The earlier new ArrayList(128) is a noticeable overhead if the data > fits in two buffers. So it's not so good for scale = M * (buffer length) > where M is about 2. > > For a large scale result (M > 10), the footprint difference between > ArrayList(N) for various N is a second-order fraction. > > ? John > > P.S. Road not taken: Instead of new ArrayList(8) you could use > the default ArrayList constructor, and allocate it unconditionally. > It has a very low overhead if the ArrayList remains empty. Using > that constructor could motivate you to simplify the code to use > the ArrayList unconditionally, since that would be just a single > heap node if it is not used to gather multiple results. But I like > the "null" formulation despite its complexity. So I'd probably > keep it the way Peter wrote it. > From paul.sandoz at oracle.com Wed Dec 20 21:28:16 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Dec 2017 13:28:16 -0800 Subject: [10] RFR 8193856 takeWhile produces incorrect result with elements produced by flatMap Message-ID: <7EF95226-766D-4F71-BCEC-3C81DC1E3A1A@oracle.com> Hi, Please review this fix for a bug in the stream takeWhile operation: http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ The flatMap operation is currently aggressive and does not detect if a downstream operation may or has cancelled processing, and will push all of it?s elements downstream. Short-circuiting operations should be guarded against such behaviour but unfortunately takeWhile was not guarded. ? Separately i plan to figure out a way to ensure flatMap operations become less aggressive if there are short-circuiting downstream operations. This may increase efficiency and also allow support for flat map results that are infinite. Paul. From brian.burkhalter at oracle.com Wed Dec 20 21:54:44 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 20 Dec 2017 13:54:44 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> Message-ID: <91AFB134-F288-44A7-8943-84C359C741E3@oracle.com> Hi Peter, On Dec 20, 2017, at 3:45 AM, Peter Levart wrote: > if (result == null) { > result = copy; > } else { > bufs = new ArrayList<>(8); // bufs.add(result); > bufs.add(copy); > } I am probably missing something here, but if the do-while loop iterates three or more times with nread > 0 each time won?t data be lost? Should this not instead be: if (result == null) { result = copy; } else { if (bufs == null) { bufs = new ArrayList<>(8); bufs.add(result); } bufs.add(copy); } Thanks, Brian From brian.burkhalter at oracle.com Wed Dec 20 22:30:41 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 20 Dec 2017 14:30:41 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> Message-ID: <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> On Dec 20, 2017, at 11:52 AM, Paul Sandoz wrote: > I was a little lassiaz-faire given that 8K bytes were anyway being allocated upfront. Peter?s changes look good. > > Brian, i would double check the tests to make sure the various code paths are tested. http://cr.openjdk.java.net/~bpb/8193832/webrev.03/ The patch is updated to: * use Peter?s approach to avoid allocating an ArrayList when length <= DEFAULT_BUFFER_SIZE; * use the default ArrayList constructor instead of that with a specific initial capacity; * update the test to ensure that lengths which require three buffers are covered. Thanks, Brian From huizhe.wang at oracle.com Wed Dec 20 22:35:37 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 20 Dec 2017 14:35:37 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8193568 : @LastModified tag in license header Message-ID: <5A3AE5B9.2050006@oracle.com> Hi, Refer to the previous change for JDK-8191938[1], the @LastModified tag needs to be moved to the class description section. This change affects 341 files and may be hard to look through. But it's done with a script that simply moves the tag from the top header to the class description section, the majority therefore are two-line changes ( e.g "2 lines changed"). If the original description section didn't have any tags, an additional comment-line is added to separate the tag with the description ( look for "3 lines changed"). If a class didn't have a class description at all, a new one is added (therefore "4 lines changed"). Please review: JBS: https://bugs.openjdk.java.net/browse/JDK-8193568 webrev: http://cr.openjdk.java.net/~joehw/jdk10/8193568/webrev/ [1] https://bugs.openjdk.java.net/browse/JDK-8191938 Thanks, Joe From jbluettduncan at gmail.com Wed Dec 20 22:47:49 2017 From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan) Date: Wed, 20 Dec 2017 22:47:49 +0000 Subject: [10] RFR 8193856 takeWhile produces incorrect result with elements produced by flatMap In-Reply-To: <7EF95226-766D-4F71-BCEC-3C81DC1E3A1A@oracle.com> References: <7EF95226-766D-4F71-BCEC-3C81DC1E3A1A@oracle.com> Message-ID: Hi Paul, It seems that some clever Googler managed to find a workaround for aggressive `flatMap` operations in the form of a so-called `MoreStreams.flatten` operation, implemented in a side project called google/mug. I'm sharing the javadoc and GitHub project homepage with you and the rest of the mailing list in the hope that they prove to be useful. Cheers, Jonathan On 20 December 2017 at 21:28, Paul Sandoz wrote: > Hi, > > Please review this fix for a bug in the stream takeWhile operation: > > http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856- > takeWhile-incorrect-results/webrev/ psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/> > > The flatMap operation is currently aggressive and does not detect if a > downstream operation may or has cancelled processing, and will push all of > it?s elements downstream. Short-circuiting operations should be guarded > against such behaviour but unfortunately takeWhile was not guarded. > > ? > > Separately i plan to figure out a way to ensure flatMap operations become > less aggressive if there are short-circuiting downstream operations. This > may increase efficiency and also allow support for flat map results that > are infinite. > > Paul. > From paul.sandoz at oracle.com Wed Dec 20 23:03:51 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Dec 2017 15:03:51 -0800 Subject: [10] RFR 8193856 takeWhile produces incorrect result with elements produced by flatMap In-Reply-To: References: <7EF95226-766D-4F71-BCEC-3C81DC1E3A1A@oracle.com> Message-ID: <5123CF93-E7B2-46C6-B51B-B8E714FACB2C@oracle.com> Hi Jonathan, > On 20 Dec 2017, at 14:47, Jonathan Bluett-Duncan wrote: > > Hi Paul, > > It seems that some clever Googler managed to find a workaround for aggressive `flatMap` operations in the form of a so-called `MoreStreams.flatten` operation, implemented in a side project called google/mug. Thanks. MoreStreams.flatten ?lowers? a stream of streams to a spliterator of spliterators which in turn is input as a new stream source, enabling a cancellation check + tryAdvance for any following short-circuit or terminal ops. I have a solution in the works for flatMap based on sorted operation (which has to buffer all elements, sort ?em, then push ?em all downstream). Paul. > I'm sharing the javadoc and GitHub project homepage with you and the rest of the mailing list in the hope that they prove to be useful. > > Cheers, > Jonathan > > On 20 December 2017 at 21:28, Paul Sandoz > wrote: > Hi, > > Please review this fix for a bug in the stream takeWhile operation: > > http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ > > > The flatMap operation is currently aggressive and does not detect if a downstream operation may or has cancelled processing, and will push all of it?s elements downstream. Short-circuiting operations should be guarded against such behaviour but unfortunately takeWhile was not guarded. > > ? > > Separately i plan to figure out a way to ensure flatMap operations become less aggressive if there are short-circuiting downstream operations. This may increase efficiency and also allow support for flat map results that are infinite. > > Paul. > From chris.hegarty at oracle.com Wed Dec 20 23:45:58 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Dec 2017 23:45:58 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> Message-ID: <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> > On 20 Dec 2017, at 19:21, mandy chung wrote: > > The native side and hotspot side should also be cleaned up. Good call. I think the following is probably as far as we want to go. Maybe a follow-on issue could be filed if deeper VM clean up is needed? http://cr.openjdk.java.net/~chegar/8179424/webrev.01/ -Chris. P.S. jdk and hotspot tests are still running... From peter.levart at gmail.com Wed Dec 20 23:47:02 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Dec 2017 00:47:02 +0100 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <91AFB134-F288-44A7-8943-84C359C741E3@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <91AFB134-F288-44A7-8943-84C359C741E3@oracle.com> Message-ID: Hi Brian, Brian Burkhalter je 20. 12. 2017 ob 22:54?napisal: > Hi Peter, > > On Dec 20, 2017, at 3:45 AM, Peter Levart > wrote: > >> ??????????????? if (result == null) { >> result = copy; >> ??????????????? } else { >> bufs = new ArrayList<>(8);?// > bufs.add(result); >> bufs.add(copy); >> ??????????????? } > > I am probably missing something here, but if the do-while loop > iterates three or more times with nread > 0 each time won?t data be > lost? Should this not instead be: > > ? ? ? ? ? ? ? ? if (result == null) { > ??????????????????? result = copy; > ??????????????? } else { > ? ? ? ? ? ? ? ? ? ? if (bufs == null) { > ? ? ? ? ? ? ? ? ? ? ? ? bufs = new ArrayList<>(8); > ? ? ? ? ? ? ? ? ? ? ? ? bufs.add(result); > ? ? ? ? ? ? ? ? ? ? } > ? ? ? ? ? ? ? ? ? ? bufs.add(copy); > ??????????????? } > > Thanks, > > Brian Yes, of course. Good catch. Next time I should try running the code before proposing it... webrev.03 looks good. Regards, Peter From mandy.chung at oracle.com Wed Dec 20 23:52:08 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 20 Dec 2017 15:52:08 -0800 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> Message-ID: On 12/20/17 3:45 PM, Chris Hegarty wrote: > >> On 20 Dec 2017, at 19:21, mandy chung > > wrote: >> >> The native side and hotspot side should also be cleaned up. > > Good call. I think the following is probably as far as we want to > go. Maybe a follow-on issue could be filed if deeper VM clean > up is needed? > > http://cr.openjdk.java.net/~chegar/8179424/webrev.01/ > > Looks good.? This hotspot change is adequate for this fix. Mandy From stuart.marks at oracle.com Thu Dec 21 00:34:59 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 20 Dec 2017 16:34:59 -0800 Subject: [10] RFR 8193856 takeWhile produces incorrect result with elements produced by flatMap In-Reply-To: <7EF95226-766D-4F71-BCEC-3C81DC1E3A1A@oracle.com> References: <7EF95226-766D-4F71-BCEC-3C81DC1E3A1A@oracle.com> Message-ID: <8cb6e9da-4e5d-0919-ef43-9cc17f0aa9a8@oracle.com> On 12/20/17 1:28 PM, Paul Sandoz wrote: > Please review this fix for a bug in the stream takeWhile operation: > > http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ > > The flatMap operation is currently aggressive and does not detect if a downstream operation may or has cancelled processing, and will push all of it?s elements downstream. Short-circuiting operations should be guarded against such behaviour but unfortunately takeWhile was not guarded. Hi Paul, Change looks fine. Good to get this one into JDK 10. Thanks, s'marks From david.holmes at oracle.com Thu Dec 21 00:42:29 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Dec 2017 10:42:29 +1000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> Message-ID: <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> Hi Chris, Adding in hotspot-runtime-dev now that you have included the VM side of the cleanup. What repo are you planning on pushing this to? On 21/12/2017 9:45 AM, Chris Hegarty wrote: > >> On 20 Dec 2017, at 19:21, mandy chung wrote: >> >> The native side and hotspot side should also be cleaned up. Thanks Mandy, I was about call that out too :) > Good call. I think the following is probably as far as we want to > go. Maybe a follow-on issue could be filed if deeper VM clean > up is needed? > > http://cr.openjdk.java.net/~chegar/8179424/webrev.01/ src/hotspot/share/include/jvm.h Can you fix an existing typo please: ! * error if it is not marked propertly. propertly -> properly Also you seem to have missed this test reference: ./langtools/tools/jdeps/jdkinternals/src/p/Main.java: Class caller = Reflection.getCallerClass(2); Otherwise all changes seem fine. Thanks, David > -Chris. > > P.S. jdk and hotspot tests are still running... > From stuart.marks at oracle.com Thu Dec 21 00:53:56 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 20 Dec 2017 16:53:56 -0800 Subject: RFR(s): 8140281 add no-arg Optional.orElseThrow() as preferred alternative to get() In-Reply-To: References: <8ac502dd-9648-2801-1574-debadafec3aa@oracle.com> Message-ID: <7ca31ca7-7793-7fdb-5a6e-530860695065@oracle.com> On 12/18/17 2:31 AM, Tagir Valeev wrote: > I think that both methods could co-exist with slightly different > semantics (even though they implementation is identical): > ... > Think of get() as > an assertion-like method: if get() throws, then it's a bug in the code > (most likely right in this method). > ... > If orElseThrow() throws, then > it's an acceptable situation (e.g. we are inside the library method > and it documents that NoSuchElementException could be thrown if some > precondition is not met). Right, so the semantic difference is between an assertion check and a precondition check. I agree that there is a difference, but get() doesn't really mean "assertion check" to me. The orElseThrow() method could be used for both, without much loss. I think it's because we experts are used to get(), and it's all we had for both cases in JDK 8 and 9, that there is discomfort with the prospect of reducing the status of get(). > However I don't like issuing a warning if > get() is used like in the code above. And if get() will be deprecated, > then we would need to issue warning on every get() usage. Thus I'm > against the deprecation. Yes, better warnings control would clearly be necessary before we would proceed with deprecation. > I believe that for any kind of API you can find bad code which abuses > it regardless on how many years passed since the API was released. > That's not always the fault of API creators. It's certainly possible to write bad code in any language, using any API, certainly right after it's been introduced. As time goes on, people learn more, articles are written and conference talks presented, etc., and good usage patterns tend to emerge. The situation with get() seems qualitatively different. I consistently and repeatedly see it misused, and it doesn't seem to be getting any better. When this occurs, it indicates that there's something wrong with the API. s'marks From naufal11 at gmail.com Thu Dec 21 01:44:51 2017 From: naufal11 at gmail.com (Mohamed Naufal) Date: Thu, 21 Dec 2017 12:44:51 +1100 Subject: [PATCH] Support for UTC Zones with TimeZone.getTimeZone() In-Reply-To: <8287adfe-079c-be54-789a-2b609d5721e9@oracle.com> References: <8287adfe-079c-be54-789a-2b609d5721e9@oracle.com> Message-ID: Hi Naoto, Thank you for the feedback, I'll send an updated patch shortly. Best regards, Naufal On 21 December 2017 at 05:19, Naoto Sato wrote: > Hi Naufal, > > Thank you for the patch. It does seem to be a bug. However, TimeZone class > defines CustomID as "GMT+(offset)", so I'd rather fix it in > GregorianCalendar.from() to recognize those "UTC+offset" zones. > > Naoto > > > On 12/16/17 2:27 AM, Mohamed Naufal wrote: > >> Hi, >> >> I noticed that with the following data: >> >> LocalDateTime ldt = LocalDateTime.parse("2017-01-01T00:00:00"); >> ZonedDateTime dt1 = ZonedDateTime.of(ldt, ZoneId.of("GMT+10")); >> ZonedDateTime dt2 = ZonedDateTime.of(ldt, ZoneId.of("UTC+10")); >> >> dt1.equals(dt2) returns true as expected, but with: >> >> GregorianCalendar gc1 = GregorianCalendar.from(dt1); >> GregorianCalendar gc2 = GregorianCalendar.from(dt2); >> >> gc1.equals(gc2) returns false. >> >> Looking at the code, I see when a GregorianCalendar is being constructed, >> TimeZone.getTimeZone() gets called, but it doesn't recognise UTC >> time-zones >> with offsets, and falls back to GMT(+0), whereas ZoneId treats GMT and UTC >> based zones equivalently. >> >> PFA a patch to fix this. >> >> Thank you, >> Naufal >> >> From david.holmes at oracle.com Thu Dec 21 09:29:32 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Dec 2017 19:29:32 +1000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> Message-ID: On 21/12/2017 7:20 PM, Chris Hegarty wrote: > David, > >> On 21 Dec 2017, at 00:42, David Holmes wrote: >> >> Hi Chris, >> >> Adding in hotspot-runtime-dev now that you have included the VM side of the cleanup. What repo are you planning on pushing this to? > > Since there are now changes in the hotspot area, then it > probably makes sense to push this through jdk/hs. Okay. >> ... >> >> Can you fix an existing typo please: >> ! * error if it is not marked propertly. >> propertly -> properly > > Fixed. > >> Also you seem to have missed this test reference: >> ./langtools/tools/jdeps/jdkinternals/src/p/Main.java: Class caller = Reflection.getCallerClass(2); > > Oops. Adding langtools testing too. > > Updated webrev: > http://cr.openjdk.java.net/~chegar/8179424/webrev.02/ I don't quite follow the change to the langtools test. Is it just trying to reference something in jdk.unsupported? I don't know what the "patch" does. > What?s the correct set of tests and route into jdk/hs these > days? I?ve not pushed there in a while. I'll email you direct. Thanks, David > -Chris. > From chris.hegarty at oracle.com Thu Dec 21 09:20:11 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 21 Dec 2017 09:20:11 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> Message-ID: <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> David, > On 21 Dec 2017, at 00:42, David Holmes wrote: > > Hi Chris, > > Adding in hotspot-runtime-dev now that you have included the VM side of the cleanup. What repo are you planning on pushing this to? Since there are now changes in the hotspot area, then it probably makes sense to push this through jdk/hs. > ... > > Can you fix an existing typo please: > ! * error if it is not marked propertly. > propertly -> properly Fixed. > Also you seem to have missed this test reference: > ./langtools/tools/jdeps/jdkinternals/src/p/Main.java: Class caller = Reflection.getCallerClass(2); Oops. Adding langtools testing too. Updated webrev: http://cr.openjdk.java.net/~chegar/8179424/webrev.02/ What?s the correct set of tests and route into jdk/hs these days? I?ve not pushed there in a while. -Chris. From Alan.Bateman at oracle.com Thu Dec 21 09:54:32 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Dec 2017 09:54:32 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> Message-ID: <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> On 21/12/2017 09:29, David Holmes wrote: > : >> >> Updated webrev: >> ?? http://cr.openjdk.java.net/~chegar/8179424/webrev.02/ > > I don't quite follow the change to the langtools test. Is it just > trying to reference something in jdk.unsupported? I don't know what > the "patch" does. I looked through webrev.02 and it looks okay. I assume there will be a follow-up bug created to re-examine JVM_GetCallerClass. The update to the jdeps test does look a bit odd, wouldn't it be better to change it to another internal API? -Alan From peter.levart at gmail.com Thu Dec 21 10:03:53 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Dec 2017 11:03:53 +0100 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> Message-ID: <95b2324a-3157-0257-adcb-c2bda44ba440@gmail.com> Hi Brian, On 12/20/2017 11:30 PM, Brian Burkhalter wrote: > On Dec 20, 2017, at 11:52 AM, Paul Sandoz wrote: > >> I was a little lassiaz-faire given that 8K bytes were anyway being allocated upfront. Peter?s changes look good. >> >> Brian, i would double check the tests to make sure the various code paths are tested. > > http://cr.openjdk.java.net/~bpb/8193832/webrev.03/ > > The patch is updated to: > > * use Peter?s approach to avoid allocating an ArrayList when length <= DEFAULT_BUFFER_SIZE; > * use the default ArrayList constructor instead of that with a specific initial capacity; > * update the test to ensure that lengths which require three buffers are covered. > > Thanks, > > Brian This is OK as is, but I see another possible improvement to the logic. You decide whether it is worth trying to implement it. Currently the logic reads stream into buffers of DEFAULT_BUFFER_SIZE and adds them to an ArrayList, except the last buffer which is 1st copied into a shorter buffer before being appended to the list. This copying is unnecessary. The copied buffer has the same content, but shorter length. But the information about the length of final buffer is contained elsewhere too (for example implicitly in 'total'). So you copuld change the final "gathering" loop to extract this information for the final buffer and there would be no redundant copying of final buffer necessary. What do you think? Regards, Peter From chris.hegarty at oracle.com Thu Dec 21 10:20:18 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 21 Dec 2017 10:20:18 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> Message-ID: <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> David, Alan, > On 21 Dec 2017, at 09:54, Alan Bateman wrote: > > On 21/12/2017 09:29, David Holmes wrote: >> : >>> >>> Updated webrev: >>> http://cr.openjdk.java.net/~chegar/8179424/webrev.02/ >> >> I don't quite follow the change to the langtools test. Is it just trying to reference something in jdk.unsupported? I don't know what the "patch" does. > I looked through webrev.02 and it looks okay. I assume there will be a follow-up bug created to re-examine JVM_GetCallerClass. > > The update to the jdeps test does look a bit odd, wouldn't it be better to change it to another internal API? The test is about identifying StackWalker as the replacement supported API for getCallerClass, which is continues to do. I could add yet another scenario to test for a different internal API that also has a replacement, and add the appropriate @modules to the test to expose its package. -Chris. From Alan.Bateman at oracle.com Thu Dec 21 11:05:46 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Dec 2017 11:05:46 +0000 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> Message-ID: On 20/12/2017 22:30, Brian Burkhalter wrote: > : > http://cr.openjdk.java.net/~bpb/8193832/webrev.03/ > > The patch is updated to: > > * use Peter?s approach to avoid allocating an ArrayList when length <= DEFAULT_BUFFER_SIZE; > * use the default ArrayList constructor instead of that with a specific initial capacity; > * update the test to ensure that lengths which require three buffers are covered. > This version looks okay although fragile to maintain due to the code paths. Have you checked that the updated test covers all cases? -Alan From david.holmes at oracle.com Thu Dec 21 12:33:36 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Dec 2017 22:33:36 +1000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> Message-ID: <9fa09d1b-af25-cdb7-47f3-d03b025472ae@oracle.com> On 21/12/2017 8:20 PM, Chris Hegarty wrote: > David, Alan, > >> On 21 Dec 2017, at 09:54, Alan Bateman wrote: >> >> On 21/12/2017 09:29, David Holmes wrote: >>> : >>>> >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~chegar/8179424/webrev.02/ >>> >>> I don't quite follow the change to the langtools test. Is it just trying to reference something in jdk.unsupported? I don't know what the "patch" does. >> I looked through webrev.02 and it looks okay. I assume there will be a follow-up bug created to re-examine JVM_GetCallerClass. >> >> The update to the jdeps test does look a bit odd, wouldn't it be better to change it to another internal API? > > The test is about identifying StackWalker as the replacement > supported API for getCallerClass, which is continues to do. Won't pretend I actually understand that, but as long as the test works reliably then okay. Thanks, David > I could add yet another scenario to test for a different internal > API that also has a replacement, and add the appropriate > @modules to the test to expose its package. > > -Chris. > From chris.hegarty at oracle.com Thu Dec 21 12:39:44 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 21 Dec 2017 12:39:44 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <9fa09d1b-af25-cdb7-47f3-d03b025472ae@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> <9fa09d1b-af25-cdb7-47f3-d03b025472ae@oracle.com> Message-ID: <0CCA70D9-E178-4A11-BE2A-9255FE24B7B5@oracle.com> > On 21 Dec 2017, at 12:33, David Holmes wrote: > ... >> The test is about identifying StackWalker as the replacement >> supported API for getCallerClass, which is continues to do. > > Won't pretend I actually understand that, but as long as the test works reliably then okay. Jdeps prints helpful messages when it encounters usage of certain common internal APIs, e.g. "I see your are using sun.reflect.Reflection.getCallerClass(int). This is an internal API which has been replaced by java.lang.StackWalker. You may wanna update your code to use that API instead?. For example, http://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/resources/jdkinternals.properties#l16 -Chris. From mandy.chung at oracle.com Thu Dec 21 16:09:52 2017 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 21 Dec 2017 08:09:52 -0800 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> Message-ID: <4e811ed4-54c6-b253-44cb-e2f6a9dd83fe@oracle.com> On 12/21/17 2:20 AM, Chris Hegarty wrote: > David, Alan, > >> On 21 Dec 2017, at 09:54, Alan Bateman wrote: >> >> On 21/12/2017 09:29, David Holmes wrote: >>> : >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~chegar/8179424/webrev.02/ >>> I don't quite follow the change to the langtools test. Is it just trying to reference something in jdk.unsupported? I don't know what the "patch" does. >> I looked through webrev.02 and it looks okay. I assume there will be a follow-up bug created to re-examine JVM_GetCallerClass. >> >> The update to the jdeps test does look a bit odd, wouldn't it be better to change it to another internal API? > The test is about identifying StackWalker as the replacement > supported API for getCallerClass, which is continues to do. > I could add yet another scenario to test for a different internal > API that also has a replacement, and add the appropriate > @modules to the test to expose its package. The test shows sun.reflect.Reflection as a removed API seems odd since the class is present but not getCallerClass(int). p.Main is used to check that reference to sun.reflect.Reflection is flagged as JDK internal use and not a removed class.? I suggest to change it to use another sun.reflect.Reflection API and create an issue to follow up sun.reflect.Reflection as flagged as a removed API. Mandy From brian.burkhalter at oracle.com Thu Dec 21 16:32:20 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 21 Dec 2017 08:32:20 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> Message-ID: <8EF70405-4052-4CAD-A347-BC2FBB7DA49D@oracle.com> On Dec 21, 2017, at 3:05 AM, Alan Bateman wrote: > On 20/12/2017 22:30, Brian Burkhalter wrote: >> : >> http://cr.openjdk.java.net/~bpb/8193832/webrev.03/ >> >> The patch is updated to: >> >> * use Peter?s approach to avoid allocating an ArrayList when length <= DEFAULT_BUFFER_SIZE; >> * use the default ArrayList constructor instead of that with a specific initial capacity; >> * update the test to ensure that lengths which require three buffers are covered. >> > This version looks okay although fragile to maintain due to the code paths. > > Have you checked that the updated test covers all cases? I think it covers all of them except the OOME. I?ll review it again. Brian From brian.burkhalter at oracle.com Thu Dec 21 16:38:52 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 21 Dec 2017 08:38:52 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <95b2324a-3157-0257-adcb-c2bda44ba440@gmail.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <95b2324a-3157-0257-adcb-c2bda44ba440@gmail.com> Message-ID: Hi Peter, On Dec 21, 2017, at 2:03 AM, Peter Levart wrote: > This is OK as is, but I see another possible improvement to the logic. You decide whether it is worth trying to implement it. Currently the logic reads stream into buffers of DEFAULT_BUFFER_SIZE and adds them to an ArrayList, except the last buffer which is 1st copied into a shorter buffer before being appended to the list. This copying is unnecessary. The copied buffer has the same content, but shorter length. But the information about the length of final buffer is contained elsewhere too (for example implicitly in 'total'). So you copuld change the final "gathering" loop to extract this information for the final buffer and there would be no redundant copying of final buffer necessary. > > What do you think? Actually I had thought of that as well. I think that it would require maintaining a list of the number of valid bytes in each of the buffers or equivalent. If having a second ArrayList for this purpose would not be too much overhead then it would probably be worthwhile. Or I suppose a single List containing an object containing both the bytes and the length would work. One could for example us ByteBuffer bb = ByteBuffer.wrap(buf).limit(nread) for each set of reads and store the ByteBuffer in the list although maybe this would be too much overhead? Thanks, Brian From Roger.Riggs at Oracle.com Thu Dec 21 17:02:06 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 21 Dec 2017 12:02:06 -0500 Subject: RFR (JDK10/JAXP Doc-only) 8193568 : @LastModified tag in license header In-Reply-To: <5A3AE5B9.2050006@oracle.com> References: <5A3AE5B9.2050006@oracle.com> Message-ID: Hi Joe, Looks fine Thanks, Roger On 12/20/2017 5:35 PM, Joe Wang wrote: > Hi, > > Refer to the previous change for JDK-8191938[1], the @LastModified tag > needs to be moved to the class description section. This change > affects 341 files and may be hard to look through. But it's done with > a script that simply moves the tag from the top header to the class > description section, the majority therefore are two-line changes ( e.g > "2 lines changed"). If the original description section didn't have > any tags, an additional comment-line is added to separate the tag with > the description ( look for "3 lines changed"). If a class didn't have > a class description at all, a new one is added (therefore "4 lines > changed"). > > Please review: > JBS: https://bugs.openjdk.java.net/browse/JDK-8193568 > webrev: http://cr.openjdk.java.net/~joehw/jdk10/8193568/webrev/ > > > [1] https://bugs.openjdk.java.net/browse/JDK-8191938 > > Thanks, > Joe From naoto.sato at oracle.com Thu Dec 21 17:10:37 2017 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 21 Dec 2017 09:10:37 -0800 Subject: [PATCH] Support for UTC Zones with TimeZone.getTimeZone() In-Reply-To: References: <8287adfe-079c-be54-789a-2b609d5721e9@oracle.com> Message-ID: Hi Naufal, I can sponsor your fix. Here is the issue created for this: https://bugs.openjdk.java.net/browse/JDK-8193938 If haven't already, would you take a look at the OpenJDK contribute page? [1] Also, I will need a regression test for the fix as well. Thanks! Naoto [1] http://openjdk.java.net/contribute/ On 12/20/17 5:44 PM, Mohamed Naufal wrote: > Hi Naoto, > > Thank you for the feedback, I'll send an updated patch shortly. > > Best regards, > Naufal > > On 21 December 2017 at 05:19, Naoto Sato > wrote: > > Hi Naufal, > > Thank you for the patch. It does seem to be a bug. However, TimeZone > class defines CustomID as "GMT+(offset)", so I'd rather fix it in > GregorianCalendar.from() to recognize those "UTC+offset" zones. > > Naoto > > > On 12/16/17 2:27 AM, Mohamed Naufal wrote: > > Hi, > > I noticed that with the following data: > > LocalDateTime ldt = LocalDateTime.parse("2017-01-01T00:00:00"); > ZonedDateTime dt1 = ZonedDateTime.of(ldt, ZoneId.of("GMT+10")); > ZonedDateTime dt2 = ZonedDateTime.of(ldt, ZoneId.of("UTC+10")); > > dt1.equals(dt2) returns true as expected, but with: > > GregorianCalendar gc1 = GregorianCalendar.from(dt1); > GregorianCalendar gc2 = GregorianCalendar.from(dt2); > > gc1.equals(gc2) returns false. > > Looking at the code, I see when a GregorianCalendar is being > constructed, > TimeZone.getTimeZone() gets called, but it doesn't recognise UTC > time-zones > with offsets, and falls back to GMT(+0), whereas ZoneId treats > GMT and UTC > based zones equivalently. > > PFA a patch to fix this. > > Thank you, > Naufal > > From chris.hegarty at oracle.com Thu Dec 21 17:18:19 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 21 Dec 2017 17:18:19 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <4e811ed4-54c6-b253-44cb-e2f6a9dd83fe@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> <4e811ed4-54c6-b253-44cb-e2f6a9dd83fe@oracle.com> Message-ID: > On 21 Dec 2017, at 16:09, mandy chung wrote: > >>> ... >> The test is about identifying StackWalker as the replacement >> supported API for getCallerClass, which is continues to do. >> I could add yet another scenario to test for a different internal >> API that also has a replacement, and add the appropriate >> @modules to the test to expose its package. >> > > The test shows sun.reflect.Reflection as a removed API seems odd since the class is present but not getCallerClass(int). There appears to be some confusion here. My webrev REMOVES sun.reflect.Reflection completely, since getCallerClass(int) was its last method. For compilation purposes, the test uses a patched jdk.unsupported module with sun.reflect.Reflection, but that class is not present at runtime. So the sun.reflect.Reflection internal API has been removed, no? > p.Main is used to check that reference to sun.reflect.Reflection is flagged as JDK internal use and not a removed class. I suggest to change it to use another sun.reflect.Reflection API There is no other API in sun.reflect.Reflection, unless you mean to use something in sun.reflect.ReflectionFactory ? > and create an issue to follow up sun.reflect.Reflection as flagged as a removed API. That?s what the test now does with my changes. Why separate it out into a separate issue? If we need a test for an internal API, I can add a scenario that uses sun.reflect.ReflectionFactory. -Chris. From aleksej.efimov at oracle.com Thu Dec 21 17:27:03 2017 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Thu, 21 Dec 2017 17:27:03 +0000 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception Message-ID: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> Hello, Please, help to review the fix for jaxp bug that fixes SAXException to correctly set exception cause with 'setCause' method: http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ I've tried to keep the fix miminal with respect to serial form of this API class, i.e. kept private 'exception' field. The new test case was added to IssueTracker56Test unit test. Testing showed no related JCK/JTREG failures. With Best Regards, Aleksei From huizhe.wang at oracle.com Thu Dec 21 17:30:32 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 21 Dec 2017 09:30:32 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8193568 : @LastModified tag in license header In-Reply-To: References: <5A3AE5B9.2050006@oracle.com> Message-ID: <5A3BEFB8.8000501@oracle.com> Thanks Roger! On 12/21/17, 9:02 AM, Roger Riggs wrote: > Hi Joe, > > Looks fine > > Thanks, Roger > > > On 12/20/2017 5:35 PM, Joe Wang wrote: >> Hi, >> >> Refer to the previous change for JDK-8191938[1], the @LastModified >> tag needs to be moved to the class description section. This change >> affects 341 files and may be hard to look through. But it's done with >> a script that simply moves the tag from the top header to the class >> description section, the majority therefore are two-line changes ( >> e.g "2 lines changed"). If the original description section didn't >> have any tags, an additional comment-line is added to separate the >> tag with the description ( look for "3 lines changed"). If a class >> didn't have a class description at all, a new one is added (therefore >> "4 lines changed"). >> >> Please review: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8193568 >> webrev: http://cr.openjdk.java.net/~joehw/jdk10/8193568/webrev/ >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8191938 >> >> Thanks, >> Joe > From peter.levart at gmail.com Thu Dec 21 18:23:50 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Dec 2017 19:23:50 +0100 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <95b2324a-3157-0257-adcb-c2bda44ba440@gmail.com> Message-ID: <53d1ab3a-fe00-6430-53d0-4cc450f8817d@gmail.com> On 12/21/2017 05:38 PM, Brian Burkhalter wrote: > Hi Peter, > > On Dec 21, 2017, at 2:03 AM, Peter Levart wrote: > >> This is OK as is, but I see another possible improvement to the logic. You decide whether it is worth trying to implement it. Currently the logic reads stream into buffers of DEFAULT_BUFFER_SIZE and adds them to an ArrayList, except the last buffer which is 1st copied into a shorter buffer before being appended to the list. This copying is unnecessary. The copied buffer has the same content, but shorter length. But the information about the length of final buffer is contained elsewhere too (for example implicitly in 'total'). So you copuld change the final "gathering" loop to extract this information for the final buffer and there would be no redundant copying of final buffer necessary. >> >> What do you think? > Actually I had thought of that as well. I think that it would require maintaining a list of the number of valid bytes in each of the buffers or equivalent. If having a second ArrayList for this purpose would not be too much overhead then it would probably be worthwhile. Or I suppose a single List containing an object containing both the bytes and the length would work. One could for example us I don't think this would be necessary. All buffers but the last one are fully filled. The inner reading loop guarantees that the buffer is either fully read or the stream is at EOF. So in final gathering loop you could maintain a 'remaining' value, initialized to 'total' and decremented by DEFAULT_BUFFER_SIZE at each iteration. The number of bytes to copy for each buffer would then be Math.min(DEFAULT_BUFFER_SIZE, remaining). That's one possibility. There are others. But no new structures are necessary. Regards, Peter > > ByteBuffer bb = ByteBuffer.wrap(buf).limit(nread) > > for each set of reads and store the ByteBuffer in the list although maybe this would be too much overhead? > > Thanks, > > Brian From Roger.Riggs at Oracle.com Thu Dec 21 18:24:51 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 21 Dec 2017 13:24:51 -0500 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> Message-ID: <9e19db3d-c383-1172-a067-407b86ab9a57@Oracle.com> Hi Aleksei, In the case of creating SAXException and then calling initCause() it looks like the value returned by getException() and getCause() may differ. The past behavior and the descriptions of those two methods are the same. Is the change intentional? If not, you may need to override initCause() to update the 'exception' field. Roger On 12/21/2017 12:27 PM, Aleks Efimov wrote: > Hello, > > Please, help to review the fix for jaxp bug that fixes SAXException to > correctly set exception cause with 'setCause' method: > http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ > I've tried to keep the fix miminal with respect to serial form of this > API class, i.e. kept private 'exception' field. > > The new test case was added to IssueTracker56Test unit test. Testing > showed no related JCK/JTREG failures. > > With Best Regards, > Aleksei > > From brian.burkhalter at oracle.com Thu Dec 21 18:26:59 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 21 Dec 2017 10:26:59 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <53d1ab3a-fe00-6430-53d0-4cc450f8817d@gmail.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <95b2324a-3157-0257-adcb-c2bda44ba440@gmail.com> <53d1ab3a-fe00-6430-53d0-4cc450f8817d@gmail.com> Message-ID: <2B9492FE-7E45-421B-90C5-F55BE4633972@oracle.com> On Dec 21, 2017, at 10:23 AM, Peter Levart wrote: >> Or I suppose a single List containing an object containing both the bytes and the length would work. One could for example us > > I don't think this would be necessary. All buffers but the last one are fully filled. The inner reading loop guarantees that the buffer is either fully read or the stream is at EOF. So in final gathering loop you could maintain a 'remaining' value, initialized to 'total' and decremented by DEFAULT_BUFFER_SIZE at each iteration. The number of bytes to copy for each buffer would then be Math.min(DEFAULT_BUFFER_SIZE, remaining). That's one possibility. There are others. But no new structures are necessary. What about the case where read() returns 0, e.g., when reading from a socket, but subsequent reads return positive values? // read to EOF which may read more or less than buffer size while ((n = read(buf, nread, buf.length - nread)) > 0) { nread += n; } Then it does not look as if buffer is full or am I mistaken? Thanks, Brian From paul.sandoz at oracle.com Thu Dec 21 18:28:31 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 21 Dec 2017 10:28:31 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <8EF70405-4052-4CAD-A347-BC2FBB7DA49D@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <8EF70405-4052-4CAD-A347-BC2FBB7DA49D@oracle.com> Message-ID: <698B6461-AFCE-494F-9745-42737FECE15E@oracle.com> Hi, This looks ok, i think it?s definitely reached it?s complexity budget, and arguably over spent. ? I do have one follow on investigation we discussed off list that is worth measuring. At the end use the Unsafe array allocation with no zeroing, since the resulting array will be fully written into. This might result in an observable improvement. Paul. > On 21 Dec 2017, at 08:32, Brian Burkhalter wrote: > > On Dec 21, 2017, at 3:05 AM, Alan Bateman wrote: > >> On 20/12/2017 22:30, Brian Burkhalter wrote: >>> : >>> http://cr.openjdk.java.net/~bpb/8193832/webrev.03/ >>> >>> The patch is updated to: >>> >>> * use Peter?s approach to avoid allocating an ArrayList when length <= DEFAULT_BUFFER_SIZE; >>> * use the default ArrayList constructor instead of that with a specific initial capacity; >>> * update the test to ensure that lengths which require three buffers are covered. >>> >> This version looks okay although fragile to maintain due to the code paths. >> >> Have you checked that the updated test covers all cases? > > I think it covers all of them except the OOME. I?ll review it again. > > Brian From brian.burkhalter at oracle.com Thu Dec 21 18:45:23 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 21 Dec 2017 10:45:23 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <2B9492FE-7E45-421B-90C5-F55BE4633972@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <95b2324a-3157-0257-adcb-c2bda44ba440@gmail.com> <53d1ab3a-fe00-6430-53d0-4cc450f8817d@gmail.com> <2B9492FE-7E45-421B-90C5-F55BE4633972@oracle.com> Message-ID: <9C91725D-E34C-4179-98E0-24AA1E9B38FB@oracle.com> Hi Peter, On Dec 21, 2017, at 10:26 AM, Brian Burkhalter wrote: > What about the case where read() returns 0, e.g., when reading from a socket, but subsequent reads return positive values? > > // read to EOF which may read more or less than buffer size > while ((n = read(buf, nread, buf.length - nread)) > 0) { > nread += n; > } > > Then it does not look as if buffer is full or am I mistaken? Never mind, I think I am mistaken. Thanks, Brian From jason_mehrens at hotmail.com Thu Dec 21 19:00:21 2017 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Thu, 21 Dec 2017 19:00:21 +0000 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> Message-ID: Aleksei, You have to override all of the constructors to always disable initCause. Actually a similar issue was covered in: http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html Pretty much everything was hashed out in those threads. Here is the final webrev for that: http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html That should be a good cookbook for changes and tests required for this issue. Note that it gets rid of the Exception field and maintains serial compatibility. Looking back on that change, I would have changed it so the InvalidClassException added the double cause as suppressed exceptions. Jason ________________________________________ From: core-libs-dev on behalf of Aleks Efimov Sent: Thursday, December 21, 2017 11:27 AM To: core-libs-dev Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception Hello, Please, help to review the fix for jaxp bug that fixes SAXException to correctly set exception cause with 'setCause' method: http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ I've tried to keep the fix miminal with respect to serial form of this API class, i.e. kept private 'exception' field. The new test case was added to IssueTracker56Test unit test. Testing showed no related JCK/JTREG failures. With Best Regards, Aleksei From stevenschlansker at gmail.com Thu Dec 21 19:11:50 2017 From: stevenschlansker at gmail.com (Steven Schlansker) Date: Thu, 21 Dec 2017 11:11:50 -0800 Subject: Adding SocketChannel toString to connection exception messages Message-ID: Hi core-libs-dev, While tracking down a connectivity issue, we identified that two of our hosts are unable to talk to each other due to a misconfiguration of the network. This manifested as: 2017-12-21T11:00:34.840Z DEBUG <> [default-pool-34] o.e.j.client.AbstractConnectionPool - Connection 1/32 creation failed java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716) at org.eclipse.jetty.io.SelectorManager.doFinishConnect(SelectorManager.java:353) at org.eclipse.jetty.io.ManagedSelector.processConnect(ManagedSelector.java:157) ... The message, 'Connection refused', does tell you something -- but it doesn't emit really any information about what actually happened (i.e. host / port of refused connection) that would allow you to track down exactly where it failed. While the application can generally figure this out (it probably knows where it tried to connect) the SocketChannelImpl could helpfully attach this information to the message, rather than relying on the application to correctly figure out the information at every connect point. What if ConnectException included the attempted hostname / IP / port SocketAddress? java.net.ConnectException: Connection to 'foo.mycorp.com[10.x.x.x]:12345' refused Much more useful! This could also be extended to various other socket exceptions. Would such a contribution be accepted? Thanks for considering! From brian.burkhalter at oracle.com Thu Dec 21 19:05:40 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 21 Dec 2017 11:05:40 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: <698B6461-AFCE-494F-9745-42737FECE15E@oracle.com> References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <8EF70405-4052-4CAD-A347-BC2FBB7DA49D@oracle.com> <698B6461-AFCE-494F-9745-42737FECE15E@oracle.com> Message-ID: Hi Paul, On Dec 21, 2017, at 10:28 AM, Paul Sandoz wrote: > This looks ok, i think it?s definitely reached it?s complexity budget, and arguably over spent. I concur that this horse is almost dead from the beatings but since I already hacked up Peter?s suggestion which eliminates intermediate copies I might as well hang it out there (see below). > ? > > I do have one follow on investigation we discussed off list that is worth measuring. At the end use the Unsafe array allocation with no zeroing, since the resulting array will be fully written into. This might result in an observable improvement. I?ve not forgotten about that but do not know whether we want to include it as part of this issue or a subsequent one. Thanks, Brian public byte[] readAllBytes() throws IOException { List bufs = null; byte[] result = null; int total = 0; int n; do { byte[] buf = new byte[DEFAULT_BUFFER_SIZE]; int nread = 0; // read to EOF which may read more or less than buffer size while ((n = read(buf, nread, buf.length - nread)) > 0) { nread += n; } if (nread > 0) { if (MAX_BUFFER_SIZE - total < nread) { throw new OutOfMemoryError("Required array size too large"); } total += nread; if (result == null) { result = buf; } else { if (bufs == null) { bufs = new ArrayList<>(); bufs.add(result); } bufs.add(buf); } } } while (n >= 0); // if the last call to read returned -1, then break if (bufs == null) { if (result == null) { return new byte[0]; } return result.length == total ? result : Arrays.copyOf(result, total); } result = new byte[total]; int offset = 0; int remaining = total; for (byte[] b : bufs) { int len = Math.min(b.length, remaining); System.arraycopy(b, 0, result, offset, len); offset += len; remaining -= len; } return result; } From huizhe.wang at oracle.com Thu Dec 21 19:23:26 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 21 Dec 2017 11:23:26 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8184431: References to @sun.com Message-ID: <5A3C0A2E.5030500@oracle.com> Hi, This change uses a process to strip off the @sun.com email addresses and links, and keep just the name of the authors. Please review: JBS: https://bugs.openjdk.java.net/browse/JDK-8184431 webrev: http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html Thanks, Joe From paul.sandoz at oracle.com Thu Dec 21 19:44:07 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 21 Dec 2017 11:44:07 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <8EF70405-4052-4CAD-A347-BC2FBB7DA49D@oracle.com> <698B6461-AFCE-494F-9745-42737FECE15E@oracle.com> Message-ID: > On 21 Dec 2017, at 11:05, Brian Burkhalter wrote: > > Hi Paul, > > On Dec 21, 2017, at 10:28 AM, Paul Sandoz > wrote: > >> This looks ok, i think it?s definitely reached it?s complexity budget, and arguably over spent. > > I concur that this horse is almost dead from the beatings but since I already hacked up Peter?s suggestion which eliminates intermediate copies I might as well hang it out there (see below). That looks ok to me, i think keeping the buf allocation at the top of the loop tends to simplify the reasoning. > >> ? >> >> I do have one follow on investigation we discussed off list that is worth measuring. At the end use the Unsafe array allocation with no zeroing, since the resulting array will be fully written into. This might result in an observable improvement. > > I?ve not forgotten about that but do not know whether we want to include it as part of this issue or a subsequent one. > I suggest a follow on investigation. Paul. > Thanks, > > Brian > > public byte[] readAllBytes() throws IOException { > List bufs = null; > byte[] result = null; > int total = 0; > int n; > do { > byte[] buf = new byte[DEFAULT_BUFFER_SIZE]; > int nread = 0; > > // read to EOF which may read more or less than buffer size > while ((n = read(buf, nread, buf.length - nread)) > 0) { > nread += n; > } > > if (nread > 0) { > if (MAX_BUFFER_SIZE - total < nread) { > throw new OutOfMemoryError("Required array size too large"); > } > total += nread; > if (result == null) { > result = buf; > } else { > if (bufs == null) { > bufs = new ArrayList<>(); > bufs.add(result); > } > bufs.add(buf); > } > } > } while (n >= 0); // if the last call to read returned -1, then break > > if (bufs == null) { > if (result == null) { > return new byte[0]; > } > return result.length == total ? > result : Arrays.copyOf(result, total); > } > > result = new byte[total]; > int offset = 0; > int remaining = total; > for (byte[] b : bufs) { > int len = Math.min(b.length, remaining); > System.arraycopy(b, 0, result, offset, len); > offset += len; > remaining -= len; > } > > return result; > } From brian.burkhalter at oracle.com Thu Dec 21 20:44:23 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 21 Dec 2017 12:44:23 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <8EF70405-4052-4CAD-A347-BC2FBB7DA49D@oracle.com> <698B6461-AFCE-494F-9745-42737FECE15E@oracle.com> Message-ID: On Dec 21, 2017, at 11:44 AM, Paul Sandoz wrote: >> I concur that this horse is almost dead from the beatings but since I already hacked up Peter?s suggestion which eliminates intermediate copies I might as well hang it out there (see below). > > That looks ok to me, i think keeping the buf allocation at the top of the loop tends to simplify the reasoning. Agreed. Here?s the hopefully final version: http://cr.openjdk.java.net/~bpb/8193832/webrev.04/ I think the test case all the cases, i.e., possible data lengths, aside from the OOME case. >>> I do have one follow on investigation we discussed off list that is worth measuring. At the end use the Unsafe array allocation with no zeroing, since the resulting array will be fully written into. This might result in an observable improvement. >> >> I?ve not forgotten about that but do not know whether we want to include it as part of this issue or a subsequent one. >> > > I suggest a follow on investigation. I?ll file an issue. Thanks, Brian From mandy.chung at oracle.com Thu Dec 21 21:40:57 2017 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 21 Dec 2017 13:40:57 -0800 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> <4e811ed4-54c6-b253-44cb-e2f6a9dd83fe@oracle.com> Message-ID: <34412585-eac6-0e62-a5b8-e8f8dd42645e@oracle.com> On 12/21/17 9:18 AM, Chris Hegarty wrote: >> On 21 Dec 2017, at 16:09, mandy chung wrote: >> >>>> ... >>> The test is about identifying StackWalker as the replacement >>> supported API for getCallerClass, which is continues to do. >>> I could add yet another scenario to test for a different internal >>> API that also has a replacement, and add the appropriate >>> @modules to the test to expose its package. >>> >> The test shows sun.reflect.Reflection as a removed API seems odd since the class is present but not getCallerClass(int). > There appears to be some confusion here. My webrev REMOVES > sun.reflect.Reflection completely, since getCallerClass(int) was its > last method. For compilation purposes, the test uses a patched > jdk.unsupported module with sun.reflect.Reflection, but that class > is not present at runtime. So the sun.reflect.Reflection internal API > has been removed, no? That clarifies it.? Thanks. > >> p.Main is used to check that reference to sun.reflect.Reflection is flagged as JDK internal use and not a removed class. I suggest to change it to use another sun.reflect.Reflection API > There is no other API in sun.reflect.Reflection, unless you mean > to use something in sun.reflect.ReflectionFactory ? > ReflectionFactory or other class in sun.reflect package would do it. >> and create an issue to follow up sun.reflect.Reflection as flagged as a removed API. > That?s what the test now does with my changes. Why separate it > out into a separate issue? If we need a test for an internal API, I > can add a scenario that uses sun.reflect.ReflectionFactory. Yes please. That's what this case intends to verify. Thanks Mandy From lance.andersen at oracle.com Thu Dec 21 22:35:43 2017 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 21 Dec 2017 17:35:43 -0500 Subject: RFR (JDK10/JAXP Doc-only) 8184431: References to @sun.com In-Reply-To: <5A3C0A2E.5030500@oracle.com> References: <5A3C0A2E.5030500@oracle.com> Message-ID: <026EC5F7-E3EB-4FF4-9A1F-F7F6616B81F0@oracle.com> Hi Joe, Overall, this is fine, a few things to consider if you want to address Happy Holidays Best Lance Do we really need to keep the name in comments such as these: final class TestSeq { --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 11:04:12.188400062 -0800 +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 11:04:11.423325345 -0800 @@ -682,7 +682,7 @@ // If the new name has a different prefix, the list may become unsorted. // Maybe it would be better to resort the list, but the simplest // fix seems to be to remove the old attribute and re-insert it. - // -- Norman.Walsh at Sun.COM, 2 Feb 2007 + // -- Norman Walsh, 2 Feb 2007 Do we need Sun Microsystems, Inc as we are not consistent --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 11:05:03.519413044 -0800 +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 11:05:02.758338717 -0800 @@ -40,7 +40,7 @@ * calling NamespaceSupport methods. * * @author Neeraj Bajaj, Sun Microsystems, inc. - * @author Santiago.PericasGeertsen at sun.com + * @author Santiago PericasGeertsen * */ public class LocationImpl implements Location{ String systemId; --- old/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 11:05:31.112107741 -0800 +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 11:05:30.352033512 -0800 @@ -32,7 +32,7 @@ import javax.xml.XMLConstants; /** * - * @author Neeraj Bajaj,K.Venugopal at sun.com Sun Microsystems. + * @author Neeraj Bajaj,K Venugopal Sun Microsystems. */ > On Dec 21, 2017, at 2:23 PM, Joe Wang wrote: > > http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From paul.sandoz at oracle.com Thu Dec 21 22:40:18 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 21 Dec 2017 14:40:18 -0800 Subject: [10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations Message-ID: <85218D91-BE9C-458C-BDF6-8E9FEAEC9647@oracle.com> Hi, Please review the following webrev that makes flatMap non-aggressive when pushing elements downstream if any downstream operation is short-circuiting. http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8075939-flatMap-aggressive-push/webrev/index.html This enables support for flat mapping to an infinite stream (assuming there is a downstream short-circuiting operation to terminate the stream computation). Thanks, Paul. From forax at univ-mlv.fr Thu Dec 21 23:46:57 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 22 Dec 2017 00:46:57 +0100 (CET) Subject: [10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations In-Reply-To: <85218D91-BE9C-458C-BDF6-8E9FEAEC9647@oracle.com> References: <85218D91-BE9C-458C-BDF6-8E9FEAEC9647@oracle.com> Message-ID: <1907603756.1806400.1513900017261.JavaMail.zimbra@u-pem.fr> Hi Paul, three things: - I think you should add a comment to explain why you have chosen to create a the field downstream* in the primitive implementations, I suppose it's to avoid to allocate a lambda proxy at each call. - the fields in the inner classes cancellationRequested and downstream* should be private. - if you use var, you should use a meaningful name, here, 's' can be replaced by 'spliterator', making the code more readable. cheers, R?mi ----- Mail original ----- > De: "Paul Sandoz" > ?: "core-libs-dev" > Envoy?: Jeudi 21 D?cembre 2017 23:40:18 > Objet: [10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations > Hi, > > Please review the following webrev that makes flatMap non-aggressive when > pushing elements downstream if any downstream operation is short-circuiting. > > http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8075939-flatMap-aggressive-push/webrev/index.html > > > This enables support for flat mapping to an infinite stream (assuming there is a > downstream short-circuiting operation to terminate the stream computation). > > Thanks, > Paul. From huizhe.wang at oracle.com Thu Dec 21 23:48:22 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 21 Dec 2017 15:48:22 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8184431: References to @sun.com In-Reply-To: <026EC5F7-E3EB-4FF4-9A1F-F7F6616B81F0@oracle.com> References: <5A3C0A2E.5030500@oracle.com> <026EC5F7-E3EB-4FF4-9A1F-F7F6616B81F0@oracle.com> Message-ID: <5A3C4846.3010401@oracle.com> Hi Lance, Thanks for the review! As you suggested, the names in comments, and "Sun Microsystems" in a few cases in the following classes are removed. Updated webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html Happy Holidays! Best, Joe +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 15:18:42.614522112 -0800 @@ -682,7 +682,6 @@ // If the new name has a different prefix, the list may become unsorted. // Maybe it would be better to resort the list, but the simplest // fix seems to be to remove the old attribute and re-insert it. - // -- Norman.Walsh at Sun.COM, 2 Feb 2007 newAttr = (Attr) attributes.removeItem(newAttr, false); attributes.addItem(newAttr); +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11NSDocumentScannerImpl.java 2017-12-21 15:18:47.069957104 -0800 @@ -741,7 +741,7 @@ // Take advantage of the fact that next string _should_ be "fElementQName.rawName", //In scanners most of the time is consumed on checks done for XML characters, we can // optimize on it and avoid the checks done for endElement, - //we will also avoid symbol table lookup - neeraj.bajaj at sun.com + //we will also avoid symbol table lookup. +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XMLDocumentFragmentScannerImpl.java 2017-12-21 15:18:49.697213609 -0800 @@ -1670,7 +1670,7 @@ // Take advantage of the fact that next string _should_ be "fElementQName.rawName", //In scanners most of the time is consumed on checks done for XML characters, we can // optimize on it and avoid the checks done for endElement, - //we will also avoid symbol table lookup - neeraj.bajaj at sun.com + //we will also avoid symbol table lookup. // this should work both for namespace processing true or false... @@ -2461,7 +2461,6 @@ * we dont need to set the value for every end element encouterd. * For Well formedness checks we can have the same QName object that was pushed. * the values will be set only if application need to know about the endElement - * -- neeraj.bajaj at sun.com */ +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/writers/XMLStreamWriterImpl.java 2017-12-21 15:37:09.426583713 -0800 @@ -64,8 +64,8 @@ * * @author Neeraj Bajaj * @author K.Venugopal - * @author Santiago.Pericas-Geertsen at sun.com - * @author Sunitha.Reddy at sun.com + * @author Santiago Pericas-Geertsen + * @author Sunitha Reddy */ public final class XMLStreamWriterImpl extends AbstractMap implements XMLStreamWriterBase { @@ -2041,7 +2041,6 @@ * we dont need to set the value for every end element we encouter. * For Well formedness checks we can have the same QName object that was pushed. * the values will be set only if application need to know about the endElement - * -- neeraj.bajaj at sun.com */ public ElementState peek() { return fElements[fDepth - 1]; --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 15:19:37.505881261 -0800 +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 15:19:36.670799729 -0800 @@ -39,8 +39,8 @@ * can be exposed to the application, we must intern all Strings before * calling NamespaceSupport methods. * - * @author Neeraj Bajaj, Sun Microsystems, inc. - * @author Santiago.PericasGeertsen at sun.com + * @author Neeraj Bajaj + * @author Santiago PericasGeertsen +++ new/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serialize/BaseMarkupSerializer.java 2017-12-21 15:19:47.796885990 -0800 @@ -118,9 +118,9 @@ * * * @authorAssaf Arkin - * @authorRahul Srivastava + * @author Rahul Srivastava * @author Elena Litani, IBM - * @author Sunitha Reddy, Sun Microsystems + * @author Sunitha Reddy * @see Serializer * @see org.w3c.dom.ls.LSSerializer +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 15:20:05.831646762 -0800 @@ -32,7 +32,7 @@ import javax.xml.XMLConstants; /** * - * @author Neeraj Bajaj,K.Venugopal at sun.com Sun Microsystems. + * @author Neeraj Bajaj,K Venugopal On 12/21/17, 2:35 PM, Lance Andersen wrote: > Hi Joe, > > Overall, this is fine, a few things to consider if you want to address > > Happy Holidays > > Best > Lance > > Do we really need to keep the name in comments such as these: > > final class TestSeq { > --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 11:04:12.188400062 -0800 > +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 11:04:11.423325345 -0800 > @@ -682,7 +682,7 @@ > // If the new name has a different prefix, the list may become unsorted. > // Maybe it would be better to resort the list, but the simplest > // fix seems to be to remove the old attribute and re-insert it. > - // --Norman.Walsh at Sun.COM , 2 Feb 2007 > + // -- Norman Walsh, 2 Feb 2007 > > > Do we need Sun Microsystems, Inc as we are not consistent > > --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 11:05:03.519413044 -0800 > +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 11:05:02.758338717 -0800 > @@ -40,7 +40,7 @@ > * calling NamespaceSupport methods. > * > * @author Neeraj Bajaj, Sun Microsystems, inc. > - * @authorSantiago.PericasGeertsen at sun.com > + * @author Santiago PericasGeertsen > * > */ > public class LocationImpl implements Location{ > String systemId; > --- old/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 11:05:31.112107741 -0800 > +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 11:05:30.352033512 -0800 > @@ -32,7 +32,7 @@ > import javax.xml.XMLConstants; > /** > * > - * @author Neeraj Bajaj,K.Venugopal at sun.com Sun Microsystems. > + * @author Neeraj Bajaj,K Venugopal Sun Microsystems. > */ > >> On Dec 21, 2017, at 2:23 PM, Joe Wang > > wrote: >> >> http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From lance.andersen at oracle.com Fri Dec 22 00:12:18 2017 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Thu, 21 Dec 2017 19:12:18 -0500 Subject: RFR (JDK10/JAXP Doc-only) 8184431: References to @sun.com In-Reply-To: <5A3C4846.3010401@oracle.com> References: <5A3C0A2E.5030500@oracle.com> <026EC5F7-E3EB-4FF4-9A1F-F7F6616B81F0@oracle.com> <5A3C4846.3010401@oracle.com> Message-ID: <80480703-378B-4005-B9AE-259CCB57C6F9@oracle.com> Hi joe Looks fine Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad > On Dec 21, 2017, at 6:48 PM, Joe Wang wrote: > > Hi Lance, > > Thanks for the review! As you suggested, the names in comments, and "Sun Microsystems" in a few cases in the following classes are removed. > > Updated webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html > > Happy Holidays! > > Best, > Joe > > +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 15:18:42.614522112 -0800 > @@ -682,7 +682,6 @@ > // If the new name has a different prefix, the list may become unsorted. > // Maybe it would be better to resort the list, but the simplest > // fix seems to be to remove the old attribute and re-insert it. > - // -- Norman.Walsh at Sun.COM, 2 Feb 2007 > newAttr = (Attr) attributes.removeItem(newAttr, false); > attributes.addItem(newAttr); > > +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11NSDocumentScannerImpl.java 2017-12-21 15:18:47.069957104 -0800 > @@ -741,7 +741,7 @@ > // Take advantage of the fact that next string _should_ be "fElementQName.rawName", > //In scanners most of the time is consumed on checks done for XML characters, we can > // optimize on it and avoid the checks done for endElement, > - //we will also avoid symbol table lookup - neeraj.bajaj at sun.com > + //we will also avoid symbol table lookup. > > +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XMLDocumentFragmentScannerImpl.java 2017-12-21 15:18:49.697213609 -0800 > @@ -1670,7 +1670,7 @@ > // Take advantage of the fact that next string _should_ be "fElementQName.rawName", > //In scanners most of the time is consumed on checks done for XML characters, we can > // optimize on it and avoid the checks done for endElement, > - //we will also avoid symbol table lookup - neeraj.bajaj at sun.com > + //we will also avoid symbol table lookup. > > // this should work both for namespace processing true or false... > > @@ -2461,7 +2461,6 @@ > * we dont need to set the value for every end element encouterd. > * For Well formedness checks we can have the same QName object that was pushed. > * the values will be set only if application need to know about the endElement > - * -- neeraj.bajaj at sun.com > */ > > +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/writers/XMLStreamWriterImpl.java 2017-12-21 15:37:09.426583713 -0800 > @@ -64,8 +64,8 @@ > * > * @author Neeraj Bajaj > * @author K.Venugopal > - * @author Santiago.Pericas-Geertsen at sun.com > - * @author Sunitha.Reddy at sun.com > + * @author Santiago Pericas-Geertsen > + * @author Sunitha Reddy > */ > public final class XMLStreamWriterImpl extends AbstractMap > implements XMLStreamWriterBase { > @@ -2041,7 +2041,6 @@ > * we dont need to set the value for every end element we encouter. > * For Well formedness checks we can have the same QName object that was pushed. > * the values will be set only if application need to know about the endElement > - * -- neeraj.bajaj at sun.com > */ > public ElementState peek() { > return fElements[fDepth - 1]; > > > --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 15:19:37.505881261 -0800 > +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 15:19:36.670799729 -0800 > @@ -39,8 +39,8 @@ > * can be exposed to the application, we must intern all Strings before > * calling NamespaceSupport methods. > * > - * @author Neeraj Bajaj, Sun Microsystems, inc. > - * @author Santiago.PericasGeertsen at sun.com > + * @author Neeraj Bajaj > + * @author Santiago PericasGeertsen > > +++ new/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serialize/BaseMarkupSerializer.java 2017-12-21 15:19:47.796885990 -0800 > @@ -118,9 +118,9 @@ > * > * > * @author Assaf Arkin > - * @author Rahul Srivastava > + * @author Rahul Srivastava > * @author Elena Litani, IBM > - * @author Sunitha Reddy, Sun Microsystems > + * @author Sunitha Reddy > * @see Serializer > * @see org.w3c.dom.ls.LSSerializer > > +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 15:20:05.831646762 -0800 > @@ -32,7 +32,7 @@ > import javax.xml.XMLConstants; > /** > * > - * @author Neeraj Bajaj,K.Venugopal at sun.com Sun Microsystems. > + * @author Neeraj Bajaj,K Venugopal > > > >> On 12/21/17, 2:35 PM, Lance Andersen wrote: >> Hi Joe, >> >> Overall, this is fine, a few things to consider if you want to address >> >> Happy Holidays >> >> Best >> Lance >> >> Do we really need to keep the name in comments such as these: >> >> final class TestSeq { >> --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 11:04:12.188400062 -0800 >> +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 11:04:11.423325345 -0800 >> @@ -682,7 +682,7 @@ >> // If the new name has a different prefix, the list may become unsorted. >> // Maybe it would be better to resort the list, but the simplest >> // fix seems to be to remove the old attribute and re-insert it. >> - // -- Norman.Walsh at Sun.COM, 2 Feb 2007 >> + // -- Norman Walsh, 2 Feb 2007 >> >> >> Do we need Sun Microsystems, Inc as we are not consistent >> >> --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 11:05:03.519413044 -0800 >> +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 11:05:02.758338717 -0800 >> @@ -40,7 +40,7 @@ >> * calling NamespaceSupport methods. >> * >> * @author Neeraj Bajaj, Sun Microsystems, inc. >> - * @author Santiago.PericasGeertsen at sun.com >> + * @author Santiago PericasGeertsen >> * >> */ >> public class LocationImpl implements Location{ >> String systemId; >> --- old/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 11:05:31.112107741 -0800 >> +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 11:05:30.352033512 -0800 >> @@ -32,7 +32,7 @@ >> import javax.xml.XMLConstants; >> /** >> * >> - * @author Neeraj Bajaj,K.Venugopal at sun.com Sun Microsystems. >> + * @author Neeraj Bajaj,K Venugopal Sun Microsystems. >> */ >> >>> On Dec 21, 2017, at 2:23 PM, Joe Wang wrote: >>> >>> http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> From stevenschlansker at gmail.com Fri Dec 22 00:29:18 2017 From: stevenschlansker at gmail.com (Steven Schlansker) Date: Thu, 21 Dec 2017 16:29:18 -0800 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: Message-ID: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> > On Dec 21, 2017, at 11:11 AM, Steven Schlansker wrote: > > What if ConnectException included the attempted hostname / IP / port SocketAddress? > java.net.ConnectException: Connection to 'foo.mycorp.com[10.x.x.x]:12345' refused > Much more useful! This could also be extended to various other socket exceptions. I started hacking around with this, first with PlainSocketImpl only. I apologize in advance for what is likely to be terrible JNI style, as I don't have much experience with it and I omitted any error handling for the POC - but I think it proves that this isn't too big of a project, as I would want to cover SocketChannelImpl and maybe UnixAsynchronousSocketChannelImpl and call it good enough for my uses. Behold! [me at myhost]% java pkg.SocketFail java.net.ConnectException: Connection refused (Connection refused) [me at myhost]% ~/code/jdk10/build/macosx-x86_64-normal-server-release/jdk/bin/java pkg.SocketFail java.net.ConnectException: Connection refused (Connection refused: localhost/127.0.0.1:12345) Am I correct that the best path towards acceptance for this kind of change is to target jdk10 and then beg for a backport to jdk9u? Thanks again for considering this improvement! WIP patch below inline. diff --git a/src/java.base/unix/native/libnet/PlainSocketImpl.c b/src/java.base/unix/native/libnet/PlainSocketImpl.c --- a/src/java.base/unix/native/libnet/PlainSocketImpl.c +++ b/src/java.base/unix/native/libnet/PlainSocketImpl.c @@ -217,6 +217,9 @@ (*env)->SetIntField(env, fdObj, IO_fd_fdID, fd); } +// private helper for socketConnect to describe errors +static void sockErrDescribe(JNIEnv *env, char* errbuf, char* msg, jobject ia, jint port); + /* * inetAddress is the address object passed to the socket connect * call. @@ -230,6 +233,7 @@ jobject iaObj, jint port, jint timeout) { + char errmsg[256]; jint localport = (*env)->GetIntField(env, this, psi_localportID); int len = 0; /* fdObj is the FileDescriptor field on this */ @@ -249,7 +253,8 @@ int connect_rv = -1; if (IS_NULL(fdObj)) { - JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", "Socket closed"); + sockErrDescribe(env, errmsg, "Socket closed", iaObj, port); + JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", errmsg); return; } else { fd = (*env)->GetIntField(env, fdObj, IO_fd_fdID); @@ -329,8 +334,8 @@ jlong prevNanoTime = JVM_NanoTime(env, 0); if (errno != EINPROGRESS) { - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", - "connect failed"); + sockErrDescribe(env, errmsg, "connect failed", iaObj, port); + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", errmsg); SET_BLOCKING(fd); return; } @@ -372,8 +377,8 @@ } /* while */ if (connect_rv == 0) { - JNU_ThrowByName(env, JNU_JAVANETPKG "SocketTimeoutException", - "connect timed out"); + sockErrDescribe(env, errmsg, "connect timed out", iaObj, port); + JNU_ThrowByName(env, JNU_JAVANETPKG "SocketTimeoutException", errmsg); /* * Timeout out but connection may still be established. @@ -419,36 +424,37 @@ * a more descriptive exception text is used. */ if (connect_rv == -1 && errno == EINVAL) { - JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", - "Invalid argument or cannot assign requested address"); + sockErrDescribe(errmsg, "Invalid argument or cannot assign requested address", iaObj, port) + JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", errmsg); return; } #endif #if defined(EPROTO) if (errno == EPROTO) { - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ProtocolException", - "Protocol error"); + sockErrDescribe(env, errmsg, "Protocol error", iaObj, port); + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ProtocolException", errmsg); return; } #endif if (errno == ECONNREFUSED) { - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", - "Connection refused"); + sockErrDescribe(env, errmsg, "Connection refused", iaObj, port); + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", errmsg); } else if (errno == ETIMEDOUT) { - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", - "Connection timed out"); + sockErrDescribe(env, errmsg, "Connection timed out", iaObj, port); + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", errmsg); } else if (errno == EHOSTUNREACH) { - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "NoRouteToHostException", - "Host unreachable"); + sockErrDescribe(env, errmsg, "Host unreachable", iaObj, port); + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "NoRouteToHostException", errmsg); } else if (errno == EADDRNOTAVAIL) { - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "NoRouteToHostException", - "Address not available"); + sockErrDescribe(env, errmsg, "Address not available", iaObj, port); + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "NoRouteToHostException", errmsg); } else if ((errno == EISCONN) || (errno == EBADF)) { - JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", - "Socket closed"); + sockErrDescribe(env, errmsg, "Socket closed", iaObj, port); + JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", errmsg); } else { + sockErrDescribe(env, errmsg, "connect failed", iaObj, port); JNU_ThrowByNameWithMessageAndLastError - (env, JNU_JAVANETPKG "SocketException", "connect failed"); + (env, JNU_JAVANETPKG "SocketException", errmsg); } return; } @@ -479,6 +485,15 @@ } } +static void sockErrDescribe(JNIEnv *env, char* errbuf, char* msg, jobject ia, jint port) { + jclass obj = (*env)->FindClass(env, "java/lang/Object"); + jmethodID toStr = (*env)->GetMethodID(env, obj, "toString", "()Ljava/lang/String;"); + jstring iaJstr = (*env)->CallObjectMethod(env, ia, toStr); + const char *iaStr = (*env)->GetStringUTFChars(env, iaJstr, 0); + jio_snprintf(errbuf, 128, "%s: %s:%d", msg, iaStr, port); + (*env)->ReleaseStringUTFChars(env, iaJstr, iaStr); +} + /* * Class: java_net_PlainSocketImpl * Method: socketBind From david.holmes at oracle.com Fri Dec 22 00:35:33 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 22 Dec 2017 10:35:33 +1000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> Message-ID: On 22/12/2017 10:29 AM, Steven Schlansker wrote: > >> On Dec 21, 2017, at 11:11 AM, Steven Schlansker wrote: >> >> What if ConnectException included the attempted hostname / IP / port SocketAddress? >> java.net.ConnectException: Connection to 'foo.mycorp.com[10.x.x.x]:12345' refused >> Much more useful! This could also be extended to various other socket exceptions. I believe there are concerns with too much information that can be considered "sensitive" (like host names and IP addresses) appearing in error messages due to them ending up in log files and bug reports. David ----- > I started hacking around with this, first with PlainSocketImpl only. > I apologize in advance for what is likely to be terrible JNI style, as I don't have much experience > with it and I omitted any error handling for the POC - but I think it proves that this isn't too big of a project, as > I would want to cover SocketChannelImpl and maybe UnixAsynchronousSocketChannelImpl and call it good enough for my uses. > > Behold! > > [me at myhost]% java pkg.SocketFail > java.net.ConnectException: Connection refused (Connection refused) > > [me at myhost]% ~/code/jdk10/build/macosx-x86_64-normal-server-release/jdk/bin/java pkg.SocketFail > java.net.ConnectException: Connection refused (Connection refused: localhost/127.0.0.1:12345) > > > Am I correct that the best path towards acceptance for this kind of change is to target jdk10 and then beg for a backport to jdk9u? > Thanks again for considering this improvement! WIP patch below inline. > > > diff --git a/src/java.base/unix/native/libnet/PlainSocketImpl.c b/src/java.base/unix/native/libnet/PlainSocketImpl.c > --- a/src/java.base/unix/native/libnet/PlainSocketImpl.c > +++ b/src/java.base/unix/native/libnet/PlainSocketImpl.c > @@ -217,6 +217,9 @@ > (*env)->SetIntField(env, fdObj, IO_fd_fdID, fd); > } > > +// private helper for socketConnect to describe errors > +static void sockErrDescribe(JNIEnv *env, char* errbuf, char* msg, jobject ia, jint port); > + > /* > * inetAddress is the address object passed to the socket connect > * call. > @@ -230,6 +233,7 @@ > jobject iaObj, jint port, > jint timeout) > { > + char errmsg[256]; > jint localport = (*env)->GetIntField(env, this, psi_localportID); > int len = 0; > /* fdObj is the FileDescriptor field on this */ > @@ -249,7 +253,8 @@ > int connect_rv = -1; > > if (IS_NULL(fdObj)) { > - JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", "Socket closed"); > + sockErrDescribe(env, errmsg, "Socket closed", iaObj, port); > + JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", errmsg); > return; > } else { > fd = (*env)->GetIntField(env, fdObj, IO_fd_fdID); > @@ -329,8 +334,8 @@ > jlong prevNanoTime = JVM_NanoTime(env, 0); > > if (errno != EINPROGRESS) { > - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", > - "connect failed"); > + sockErrDescribe(env, errmsg, "connect failed", iaObj, port); > + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", errmsg); > SET_BLOCKING(fd); > return; > } > @@ -372,8 +377,8 @@ > } /* while */ > > if (connect_rv == 0) { > - JNU_ThrowByName(env, JNU_JAVANETPKG "SocketTimeoutException", > - "connect timed out"); > + sockErrDescribe(env, errmsg, "connect timed out", iaObj, port); > + JNU_ThrowByName(env, JNU_JAVANETPKG "SocketTimeoutException", errmsg); > > /* > * Timeout out but connection may still be established. > @@ -419,36 +424,37 @@ > * a more descriptive exception text is used. > */ > if (connect_rv == -1 && errno == EINVAL) { > - JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", > - "Invalid argument or cannot assign requested address"); > + sockErrDescribe(errmsg, "Invalid argument or cannot assign requested address", iaObj, port) > + JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", errmsg); > return; > } > #endif > #if defined(EPROTO) > if (errno == EPROTO) { > - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ProtocolException", > - "Protocol error"); > + sockErrDescribe(env, errmsg, "Protocol error", iaObj, port); > + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ProtocolException", errmsg); > return; > } > #endif > if (errno == ECONNREFUSED) { > - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", > - "Connection refused"); > + sockErrDescribe(env, errmsg, "Connection refused", iaObj, port); > + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", errmsg); > } else if (errno == ETIMEDOUT) { > - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", > - "Connection timed out"); > + sockErrDescribe(env, errmsg, "Connection timed out", iaObj, port); > + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "ConnectException", errmsg); > } else if (errno == EHOSTUNREACH) { > - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "NoRouteToHostException", > - "Host unreachable"); > + sockErrDescribe(env, errmsg, "Host unreachable", iaObj, port); > + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "NoRouteToHostException", errmsg); > } else if (errno == EADDRNOTAVAIL) { > - NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "NoRouteToHostException", > - "Address not available"); > + sockErrDescribe(env, errmsg, "Address not available", iaObj, port); > + NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "NoRouteToHostException", errmsg); > } else if ((errno == EISCONN) || (errno == EBADF)) { > - JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", > - "Socket closed"); > + sockErrDescribe(env, errmsg, "Socket closed", iaObj, port); > + JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", errmsg); > } else { > + sockErrDescribe(env, errmsg, "connect failed", iaObj, port); > JNU_ThrowByNameWithMessageAndLastError > - (env, JNU_JAVANETPKG "SocketException", "connect failed"); > + (env, JNU_JAVANETPKG "SocketException", errmsg); > } > return; > } > @@ -479,6 +485,15 @@ > } > } > > +static void sockErrDescribe(JNIEnv *env, char* errbuf, char* msg, jobject ia, jint port) { > + jclass obj = (*env)->FindClass(env, "java/lang/Object"); > + jmethodID toStr = (*env)->GetMethodID(env, obj, "toString", "()Ljava/lang/String;"); > + jstring iaJstr = (*env)->CallObjectMethod(env, ia, toStr); > + const char *iaStr = (*env)->GetStringUTFChars(env, iaJstr, 0); > + jio_snprintf(errbuf, 128, "%s: %s:%d", msg, iaStr, port); > + (*env)->ReleaseStringUTFChars(env, iaJstr, iaStr); > +} > + > /* > * Class: java_net_PlainSocketImpl > * Method: socketBind > > > From paul.sandoz at oracle.com Fri Dec 22 00:38:37 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 21 Dec 2017 16:38:37 -0800 Subject: [10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations In-Reply-To: <1907603756.1806400.1513900017261.JavaMail.zimbra@u-pem.fr> References: <85218D91-BE9C-458C-BDF6-8E9FEAEC9647@oracle.com> <1907603756.1806400.1513900017261.JavaMail.zimbra@u-pem.fr> Message-ID: > On 21 Dec 2017, at 15:46, Remi Forax wrote: > > Hi Paul, > three things: > - I think you should add a comment to explain why you have chosen to create a the field downstream* in the primitive implementations, > I suppose it's to avoid to allocate a lambda proxy at each call. Yes, i included this comment: // cache the consumer to avoid creation on every accepted element > - the fields in the inner classes cancellationRequested and downstream* should be private. Does it make any difference for such inner classes? (I lean towards package private as the default.) > - if you use var, you should use a meaningful name, here, 's' can be replaced by 'spliterator', making the code more readable. > Don?t agree in this case, given the locality of use and given the method name on the RHS. Thanks, Paul. From stevenschlansker at gmail.com Fri Dec 22 00:59:37 2017 From: stevenschlansker at gmail.com (Steven Schlansker) Date: Thu, 21 Dec 2017 16:59:37 -0800 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> Message-ID: <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> > On Dec 21, 2017, at 4:35 PM, David Holmes wrote: > > On 22/12/2017 10:29 AM, Steven Schlansker wrote: >>> On Dec 21, 2017, at 11:11 AM, Steven Schlansker wrote: >>> >>> What if ConnectException included the attempted hostname / IP / port SocketAddress? >>> java.net.ConnectException: Connection to 'foo.mycorp.com[10.x.x.x]:12345' refused >>> Much more useful! This could also be extended to various other socket exceptions. > > I believe there are concerns with too much information that can be considered "sensitive" (like host names and IP addresses) appearing in error messages due to them ending up in log files and bug reports. Unfortunately that's exactly the information that is crucial to someone trying to diagnose issues... Could it be an opt-in policy? Perhaps by a system property? Currently the alternative I'm faced with is going through every piece of user code and library that *might* throw this exception and wrapping it to add this critical diagnostic information. For an application that uses java.net heavily, you can imagine how that is a tall task and possibly even not realistically achievable... (Is there a written policy regarding this somewhere, or is it up to the personal feelings of the contributors?) From stuart.marks at oracle.com Fri Dec 22 01:04:07 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 21 Dec 2017 17:04:07 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: Finally catching up with this thread.... Yes, there should be some additional test cases added. When I added the immutable collection classes in JDK 9, I did modify MOAT.java so that its test cases would apply to the new implementations. A few more cases could be added that apply not only to the immutable lists but also to lists in general. I think the bugs mentioned here are with indexOf() and lastIndexOf(), with the latter accidentally being a copy of the former. Later in the thread John pointed out an off-by-one error with lastIndexOf(), so edge cases should be added as well. These would apply to all lists, not just the immutable ones. There is also the issue mentioned in JDK-8191418 List.of().indexOf(null) doesn't throw NullPointerException Tests should be added for that and related methods. Since many collections permit null, and I suspect some that disallow null might permit indexOf(null) and related, it's not clear to me these belong in MOAT.java. But if List.of().indexOf(null) were to throw NPE, I'd expect List.of().subList(...).indexOf(null) also to throw NPE. s'marks On 12/8/17 9:44 AM, Martin Buchholz wrote: > Should there be changes to general purpose collection testing classes like > test/jdk/java/util/concurrent/tck/CollectionTest.java > test/jdk/java/util/Collection/MOAT.java > that would have caught this bug? > (although those are mostly designed for mutable collections, but I think some > immutable collections (nCopies?) get tested somewhere. > > On Fri, Dec 8, 2017 at 7:13 AM, Claes Redestad > wrote: > > Hi, > > On 2017-12-08 07:54, Andrej Golovnin wrote: > > Hi Claes, > > http://cr.openjdk.java.net/~redestad/8193128/open.02/ > > > I think you should provide specialised implementations of the > #indexOf() and #lastIndexOf() methods in the classes List12 and ListN. > Using an iterator in List12 is an overkill. But even in ListN using a > simple for loop would be much better. > > > it's somewhat ironic that I started looking at this *because* > indexOf was slow due use of iterators[1], but then never got > around to specialize them in this patch. > > In any case please take look at > the implementation of the #lastIndexOf() method in the > AbstractImmutableList class. It looks like a copy of > AbstractImmutableList#indexOf() and this is wrong. > > > Nice catch! Quite the embarrassing copy-paste that... > > - Specialized the index/lastIndexOf methods for List12, ListN > - Fixed implementation of lastIndexOf in AbstractImmutableList. > - As AbstractImmutableList.indexOf/lastIndexOf are now only used > by AbstractImmutableList.SubList, I moved them there with some > commentary since it's not clear JDK-8191418 should add null > checkson the input for SubList or not. > - Added sanity tests of indexOf/lastIndexOf that touches all > the new implementations: > > http://cr.openjdk.java.net/~redestad/8193128/open.03/ > > > Thanks! > > /Claes > > [1] https://bugs.openjdk.java.net/browse/JDK-8191442 > > > From huizhe.wang at oracle.com Fri Dec 22 01:09:12 2017 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 21 Dec 2017 17:09:12 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8184431: References to @sun.com In-Reply-To: <80480703-378B-4005-B9AE-259CCB57C6F9@oracle.com> References: <5A3C0A2E.5030500@oracle.com> <026EC5F7-E3EB-4FF4-9A1F-F7F6616B81F0@oracle.com> <5A3C4846.3010401@oracle.com> <80480703-378B-4005-B9AE-259CCB57C6F9@oracle.com> Message-ID: <5A3C5B38.6090302@oracle.com> Thanks Lance! On 12/21/17, 4:12 PM, Lance @ Oracle wrote: > Hi joe > > Looks fine > > Best > Lance > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > Sent from my iPad > > On Dec 21, 2017, at 6:48 PM, Joe Wang > wrote: > >> Hi Lance, >> >> Thanks for the review! As you suggested, the names in comments, and >> "Sun Microsystems" in a few cases in the following classes are removed. >> >> Updated webrevs: >> http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html >> >> Happy Holidays! >> >> Best, >> Joe >> >> +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 15:18:42.614522112 -0800 >> @@ -682,7 +682,6 @@ >> // If the new name has a different prefix, the list may become unsorted. >> // Maybe it would be better to resort the list, but the simplest >> // fix seems to be to remove the old attribute and re-insert it. >> - // --Norman.Walsh at Sun.COM, 2 Feb 2007 >> newAttr = (Attr) attributes.removeItem(newAttr, false); >> attributes.addItem(newAttr); >> >> +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11NSDocumentScannerImpl.java 2017-12-21 15:18:47.069957104 -0800 >> @@ -741,7 +741,7 @@ >> // Take advantage of the fact that next string _should_ be "fElementQName.rawName", >> //In scanners most of the time is consumed on checks done for XML characters, we can >> // optimize on it and avoid the checks done for endElement, >> - //we will also avoid symbol table lookup -neeraj.bajaj at sun.com >> + //we will also avoid symbol table lookup. >> +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XMLDocumentFragmentScannerImpl.java 2017-12-21 15:18:49.697213609 -0800 >> @@ -1670,7 +1670,7 @@ >> // Take advantage of the fact that next string _should_ be "fElementQName.rawName", >> //In scanners most of the time is consumed on checks done for XML characters, we can >> // optimize on it and avoid the checks done for endElement, >> - //we will also avoid symbol table lookup -neeraj.bajaj at sun.com >> + //we will also avoid symbol table lookup. >> >> // this should work both for namespace processing true or false... >> >> @@ -2461,7 +2461,6 @@ >> * we dont need to set the value for every end element encouterd. >> * For Well formedness checks we can have the same QName object that was pushed. >> * the values will be set only if application need to know about the endElement >> - * --neeraj.bajaj at sun.com >> */ >> >> +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/writers/XMLStreamWriterImpl.java 2017-12-21 15:37:09.426583713 -0800 >> @@ -64,8 +64,8 @@ >> * >> * @author Neeraj Bajaj >> * @author K.Venugopal >> - * @authorSantiago.Pericas-Geertsen at sun.com >> - * @authorSunitha.Reddy at sun.com >> + * @author Santiago Pericas-Geertsen >> + * @author Sunitha Reddy >> */ >> public final class XMLStreamWriterImpl extends AbstractMap >> implements XMLStreamWriterBase { >> @@ -2041,7 +2041,6 @@ >> * we dont need to set the value for every end element we encouter. >> * For Well formedness checks we can have the same QName object that was pushed. >> * the values will be set only if application need to know about the endElement >> - * --neeraj.bajaj at sun.com >> */ >> public ElementState peek() { >> return fElements[fDepth - 1]; >> >> --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 15:19:37.505881261 -0800 >> +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 15:19:36.670799729 -0800 >> @@ -39,8 +39,8 @@ >> * can be exposed to the application, we must intern all Strings before >> * calling NamespaceSupport methods. >> * >> - * @author Neeraj Bajaj, Sun Microsystems, inc. >> - * @authorSantiago.PericasGeertsen at sun.com >> + * @author Neeraj Bajaj >> + * @author Santiago PericasGeertsen >> >> +++ new/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serialize/BaseMarkupSerializer.java 2017-12-21 15:19:47.796885990 -0800 >> @@ -118,9 +118,9 @@ >> * >> * >> * @authorAssaf Arkin >> - * @authorRahul Srivastava >> + * @author Rahul Srivastava >> * @author Elena Litani, IBM >> - * @author Sunitha Reddy, Sun Microsystems >> + * @author Sunitha Reddy >> * @see Serializer >> * @see org.w3c.dom.ls.LSSerializer >> >> +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 15:20:05.831646762 -0800 >> @@ -32,7 +32,7 @@ >> import javax.xml.XMLConstants; >> /** >> * >> - * @author Neeraj Bajaj,K.Venugopal at sun.com Sun Microsystems. >> + * @author Neeraj Bajaj,K Venugopal >> >> >> On 12/21/17, 2:35 PM, Lance Andersen wrote: >>> Hi Joe, >>> >>> Overall, this is fine, a few things to consider if you want to address >>> >>> Happy Holidays >>> >>> Best >>> Lance >>> >>> Do we really need to keep the name in comments such as these: >>> >>> final class TestSeq { >>> --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 11:04:12.188400062 -0800 >>> +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/dom/ElementImpl.java 2017-12-21 11:04:11.423325345 -0800 >>> @@ -682,7 +682,7 @@ >>> // If the new name has a different prefix, the list may become unsorted. >>> // Maybe it would be better to resort the list, but the simplest >>> // fix seems to be to remove the old attribute and re-insert it. >>> - // --Norman.Walsh at Sun.COM , 2 Feb 2007 >>> + // -- Norman Walsh, 2 Feb 2007 >>> >>> >>> Do we need Sun Microsystems, Inc as we are not consistent >>> >>> --- old/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 11:05:03.519413044 -0800 >>> +++ new/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/NamespaceContextWrapper.java 2017-12-21 11:05:02.758338717 -0800 >>> @@ -40,7 +40,7 @@ >>> * calling NamespaceSupport methods. >>> * >>> * @author Neeraj Bajaj, Sun Microsystems, inc. >>> - * @authorSantiago.PericasGeertsen at sun.com >>> + * @author Santiago PericasGeertsen >>> * >>> */ >>> public class LocationImpl implements Location{ >>> String systemId; >>> --- old/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 11:05:31.112107741 -0800 >>> +++ new/src/java.xml/share/classes/com/sun/xml/internal/stream/events/NamespaceImpl.java 2017-12-21 11:05:30.352033512 -0800 >>> @@ -32,7 +32,7 @@ >>> import javax.xml.XMLConstants; >>> /** >>> * >>> - * @author Neeraj Bajaj,K.Venugopal at sun.com Sun Microsystems. >>> + * @author Neeraj Bajaj,K Venugopal Sun Microsystems. >>> */ >>> >>>> On Dec 21, 2017, at 2:23 PM, Joe Wang >>> > wrote: >>>> >>>> http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html >>>> >>> >>> >>> >>> >>> Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> From david.holmes at oracle.com Fri Dec 22 01:27:14 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 22 Dec 2017 11:27:14 +1000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> Message-ID: cc'ing net-dev as that might be the more appropriate list. On 22/12/2017 10:59 AM, Steven Schlansker wrote: > >> On Dec 21, 2017, at 4:35 PM, David Holmes wrote: >> >> On 22/12/2017 10:29 AM, Steven Schlansker wrote: >>>> On Dec 21, 2017, at 11:11 AM, Steven Schlansker wrote: >>>> >>>> What if ConnectException included the attempted hostname / IP / port SocketAddress? >>>> java.net.ConnectException: Connection to 'foo.mycorp.com[10.x.x.x]:12345' refused >>>> Much more useful! This could also be extended to various other socket exceptions. >> >> I believe there are concerns with too much information that can be considered "sensitive" (like host names and IP addresses) appearing in error messages due to them ending up in log files and bug reports. > > Unfortunately that's exactly the information that is crucial to someone trying to diagnose issues... > Could it be an opt-in policy? Perhaps by a system property? The expectation is that such information should be added by layers higher in the call chain, rather than the JDK libraries assuming it is okay to do. > Currently the alternative I'm faced with is going through every piece of user code and library that *might* > throw this exception and wrapping it to add this critical diagnostic information. For an application that uses > java.net heavily, you can imagine how that is a tall task and possibly even not realistically achievable... > > (Is there a written policy regarding this somewhere, or is it up to the personal feelings of the contributors?) This is covered by the secure coding guidelines, under 'Confidential information': http://www.oracle.com/technetwork/java/seccodeguide-139067.html#2 "Guideline 2-1 / CONFIDENTIAL-1: Purge sensitive information from exceptions" I know for a fact that we'd have to scrub this information from test failures when putting the info into a bug report. If net-dev folk can't expand further on this then I suggest filing a CSR request so that the CSR group can consider it. But I think this will be a no-go (I'm a member of the CSR group). Cheers, David From stuart.marks at oracle.com Fri Dec 22 01:31:00 2017 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 21 Dec 2017 17:31:00 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> Message-ID: John Rose wrote: > Can anyone point out a reason why the value based > lists of List.of() should serialize while the value based > lists of List.of().subList() should not? Or is there some > reason we should not allow subList to produce value > based lists? >> I think one can interpret the @implSpec in AbstracList::subList to suggest >> that the implementation retrieved will subclass AbstractList, which implies >> it's*not* Serializable. As we move away from AbstractList then we can of >> course choose our own spec, but since it's established behavior I tried as >> best I could to retrofit behavior? > Maybe you are right about that, but AbstractList is*not* part of the API > of the List returned by List.of, so you are free to do whatever is right > for the value-based list, ignoring all parts of AbstractList that are not > immediately valuable to you. > > (Yes, someone could "notice" that the result of List.of seems to be > castable to AbstractList, and*then* lodge a complaint that its sublist > is serializable, but that's a forced scenario; if we deem it is a real > bug to fix, at some date far in the future, we can fix it by removing > AbstractList from the supers of the List.of result.) Right, there is no requirement that subList() return an instance of AbstractList. We have a blanket value-based statement over all these implementations. Anybody who cares whether subList() returns the same or a different instance, for example using References on them, is performing an identity-sensitive operation, which violates the value-based admonition. (Should use of References be added to the list of identity-sensitive operations in ValueBased.html?) I do have a bit of an issue with something like List.of().subList(0,0) returning 'this', since in this case the sublist will be serializable. This is an outlier, because, most existing sublists of the conventional collections aren't serializable. And in JDK 9 (and probably JDK 10) sublists of immutable lists aren't serializable, since the immutables' sublist implementation is inherited from AbstractList. Now suppose that we proceed with something similar to Claes' sublist implementation from webrev open.04, [1] except that if the requested sublist matches the bounds of the list, it simply returns 'this'. We'd then have a case where List.of().subList(0, 0) is serializable List.of(1).subList(0, 0) is not serializable List.of(1).subList(0, 1) is serializable and in general Object[] a = ... List.of(a).subList(m, n) ??? serializable ??? I'd like to avoid situations like this, where a sublist (and in general, view collections) are sometimes serializable and sometimes not, depending upon the input data. Instead, I'd like a blanket assertion that view collections are not serializable. Now this should be considered distinct from whether view collections are value-based; I'm fine with considering view collections to be value-based. It's just that some value-based collections are serializable and others aren't. In particular, given a serializable, value-based list, a sublist should not be serializable but it can be value-based. A sublist of a sublist should also not be serializable and can be value-based, and in this case we can do short-circuiting such as 'return this' if we like. s'marks [1] http://cr.openjdk.java.net/~redestad/8193128/open.04/ From peter.levart at gmail.com Fri Dec 22 09:21:27 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Dec 2017 10:21:27 +0100 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <8EF70405-4052-4CAD-A347-BC2FBB7DA49D@oracle.com> <698B6461-AFCE-494F-9745-42737FECE15E@oracle.com> Message-ID: Hi Brian, On 12/21/2017 09:44 PM, Brian Burkhalter wrote: > On Dec 21, 2017, at 11:44 AM, Paul Sandoz wrote: > >>> I concur that this horse is almost dead from the beatings but since I already hacked up Peter?s suggestion which eliminates intermediate copies I might as well hang it out there (see below). >> That looks ok to me, i think keeping the buf allocation at the top of the loop tends to simplify the reasoning. > Agreed. Here?s the hopefully final version: > > http://cr.openjdk.java.net/~bpb/8193832/webrev.04/ > > I think the test case all the cases, i.e., possible data lengths, aside from the OOME case. I think this looks good. No more suggestions from my side for this patch. As Paul notes, it would be interesting to see if using Unsafe.allocateUninitializedArray() to allocate the final array (and intermediate buffers too) has a measurable impact. Regards, Peter > >>>> I do have one follow on investigation we discussed off list that is worth measuring. At the end use the Unsafe array allocation with no zeroing, since the resulting array will be fully written into. This might result in an observable improvement. >>> I?ve not forgotten about that but do not know whether we want to include it as part of this issue or a subsequent one. >>> >> I suggest a follow on investigation. > I?ll file an issue. > > Thanks, > > Brian From chris.hegarty at oracle.com Fri Dec 22 09:57:48 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 22 Dec 2017 09:57:48 +0000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> Message-ID: <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> On 22/12/17 01:27, David Holmes wrote: > cc'ing net-dev as that might be the more appropriate list. > > On 22/12/2017 10:59 AM, Steven Schlansker wrote: >> >>> On Dec 21, 2017, at 4:35 PM, David Holmes >>> wrote: >>> >>> On 22/12/2017 10:29 AM, Steven Schlansker wrote: >>>>> On Dec 21, 2017, at 11:11 AM, Steven Schlansker >>>>> wrote: >>>>> >>>>> What if ConnectException included the attempted hostname / IP / >>>>> port SocketAddress? >>>>> java.net.ConnectException: Connection to >>>>> 'foo.mycorp.com[10.x.x.x]:12345' refused >>>>> Much more useful! This could also be extended to various other >>>>> socket exceptions. >>> >>> I believe there are concerns with too much information that can be >>> considered "sensitive" (like host names and IP addresses) appearing >>> in error messages due to them ending up in log files and bug reports. >> >> Unfortunately that's exactly the information that is crucial to >> someone trying to diagnose issues... And to someone trying to discern private information about a network. >> Could it be an opt-in policy? Perhaps by a system property? Well, you don't know where a log file or exception will end up. Most likely on stackoverflow. > The expectation is that such information should be added by layers > higher in the call chain, rather than the JDK libraries assuming it is > okay to do. Agreed. It may be some work, but if the issue is significant enough to the user then it may be worth their effort, as well as affording the opportunity to opt-out at any particular point in the code. >> Currently the alternative I'm faced with is going through every piece >> of user code and library that *might* >> throw this exception and wrapping it to add this critical diagnostic >> information. For an application that uses >> java.net heavily, you can imagine how that is a tall task and possibly >> even not realistically achievable... >> >> (Is there a written policy regarding this somewhere, or is it up to >> the personal feelings of the contributors?) > > This is covered by the secure coding guidelines, under 'Confidential > information': > > http://www.oracle.com/technetwork/java/seccodeguide-139067.html#2 > > "Guideline 2-1 / CONFIDENTIAL-1: Purge sensitive information from > exceptions" Exactly. > I know for a fact that we'd have to scrub this information from test > failures when putting the info into a bug report. > > If net-dev folk can't expand further on this then I suggest filing a CSR > request so that the CSR group can consider it. But I think this will be > a no-go (I'm a member of the CSR group). I have to agree with David here. I don't think that such a request could be supported. -Chris. From chris.hegarty at oracle.com Fri Dec 22 10:16:14 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 22 Dec 2017 10:16:14 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <34412585-eac6-0e62-a5b8-e8f8dd42645e@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> <4e811ed4-54c6-b253-44cb-e2f6a9dd83fe@oracle.com> <34412585-eac6-0e62-a5b8-e8f8dd42645e@oracle.com> Message-ID: <0a57854d-b5d5-a30a-02cb-14617ef625ca@oracle.com> On 21/12/17 21:40, mandy chung wrote: > ... > > ReflectionFactory or other class in sun.reflect package would do it. >>> and create an issue to follow up sun.reflect.Reflection as flagged as a removed API. >> That?s what the test now does with my changes. Why separate it >> out into a separate issue? If we need a test for an internal API, I >> can add a scenario that uses sun.reflect.ReflectionFactory. > > Yes please. That's what this case intends to verify. Webrev with the test updated as suggested: http://cr.openjdk.java.net/~chegar/8179424/webrev.03/ -Chris. From forax at univ-mlv.fr Fri Dec 22 11:15:33 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 22 Dec 2017 12:15:33 +0100 (CET) Subject: [10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations In-Reply-To: References: <85218D91-BE9C-458C-BDF6-8E9FEAEC9647@oracle.com> <1907603756.1806400.1513900017261.JavaMail.zimbra@u-pem.fr> Message-ID: <391296167.1979044.1513941333703.JavaMail.zimbra@u-pem.fr> > De: "Paul Sandoz" > ?: "Remi Forax" > Cc: "core-libs-dev" > Envoy?: Vendredi 22 D?cembre 2017 01:38:37 > Objet: Re: [10] RFR 8075939: Stream.flatMap() causes breaking of > short-circuiting of terminal operations >> On 21 Dec 2017, at 15:46, Remi Forax < [ mailto:forax at univ-mlv.fr | >> forax at univ-mlv.fr ] > wrote: >> Hi Paul, >> three things: >> - I think you should add a comment to explain why you have chosen to create a >> the field downstream* in the primitive implementations, >> I suppose it's to avoid to allocate a lambda proxy at each call. > Yes, i included this comment: > // cache the consumer to avoid creation on every accepted element >> - the fields in the inner classes cancellationRequested and downstream* should >> be private. > Does it make any difference for such inner classes? Usually, a stronger encapsulation is better than a weaker one. Being an anonymous class only means that you can not access the type, not that the fields are not accessible. > (I lean towards package private as the default.) I tend to think that before, but with nestmate around the corner, private now really means accessible in the same java file, so i do think that private should be the new default in nested classes. >> - if you use var, you should use a meaningful name, here, 's' can be replaced by >> 'spliterator', making the code more readable. > Don?t agree in this case, given the locality of use and given the method name on > the RHS. It means that when you want to know what a variable is you have to read its initialization part, it's a bit of a stretch compare to what Brian said about var, we can use var because the variable name is meaningful enough, so the type is redundant. I suppose it's not a big deal if you read the code sequentially but i tend to read the meat part of the code and extend above and below, so i was focus on the code that uses downstreamAsInt when i ask myself what 's' was. > Thanks, > Paul. cheers, R?mi From sean.coffey at oracle.com Fri Dec 22 14:59:42 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 22 Dec 2017 14:59:42 +0000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> Message-ID: <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> As someone who works with alot of log files, I'd like to chime in and support Steven's end goal. Looking at a "Connection refused" error in the middle of a log file that possibly extends to millions of lines is near useless. In the era of cloud compute, diagnosing network issues is sure to become a more common task. While we may not be able to put the sensitive information in an exception message, I think we could put it behind a (new?) system property which might be able to log this information. Logs contain all sorts of sensitive data. Take javax.net.debug=ssl output for example. Regards, Sean. On 22/12/17 09:57, Chris Hegarty wrote: > On 22/12/17 01:27, David Holmes wrote: >> cc'ing net-dev as that might be the more appropriate list. >> >> On 22/12/2017 10:59 AM, Steven Schlansker wrote: >>> >>>> On Dec 21, 2017, at 4:35 PM, David Holmes >>>> wrote: >>>> >>>> On 22/12/2017 10:29 AM, Steven Schlansker wrote: >>>>>> On Dec 21, 2017, at 11:11 AM, Steven Schlansker >>>>>> wrote: >>>>>> >>>>>> What if ConnectException included the attempted hostname / IP / >>>>>> port SocketAddress? >>>>>> java.net.ConnectException: Connection to >>>>>> 'foo.mycorp.com[10.x.x.x]:12345' refused >>>>>> Much more useful! This could also be extended to various other >>>>>> socket exceptions. >>>> >>>> I believe there are concerns with too much information that can be >>>> considered "sensitive" (like host names and IP addresses) appearing >>>> in error messages due to them ending up in log files and bug reports. >>> >>> Unfortunately that's exactly the information that is crucial to >>> someone trying to diagnose issues... > > And to someone trying to discern private information about a network. > >>> Could it be an opt-in policy? Perhaps by a system property? > > Well, you don't know where a log file or exception will end up. > Most likely on stackoverflow. > >> The expectation is that such information should be added by layers >> higher in the call chain, rather than the JDK libraries assuming it >> is okay to do. > > Agreed. It may be some work, but if the issue is significant > enough to the user then it may be worth their effort, as well > as affording the opportunity to opt-out at any particular > point in the code. > >>> Currently the alternative I'm faced with is going through every >>> piece of user code and library that *might* >>> throw this exception and wrapping it to add this critical diagnostic >>> information. For an application that uses >>> java.net heavily, you can imagine how that is a tall task and >>> possibly even not realistically achievable... >>> >>> (Is there a written policy regarding this somewhere, or is it up to >>> the personal feelings of the contributors?) >> >> This is covered by the secure coding guidelines, under 'Confidential >> information': >> >> http://www.oracle.com/technetwork/java/seccodeguide-139067.html#2 >> >> "Guideline 2-1 / CONFIDENTIAL-1: Purge sensitive information from >> exceptions" > > Exactly. > >> I know for a fact that we'd have to scrub this information from test >> failures when putting the info into a bug report. >> >> If net-dev folk can't expand further on this then I suggest filing a >> CSR request so that the CSR group can consider it. But I think this >> will be a no-go (I'm a member of the CSR group). > > I have to agree with David here. I don't think that such a request > could be supported. > > -Chris. From chris.hegarty at oracle.com Fri Dec 22 15:17:31 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 22 Dec 2017 15:17:31 +0000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> Message-ID: <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> On 22/12/17 14:59, Se?n Coffey wrote: > As someone who works with alot of log files, I'd like to chime in and > support Steven's end goal. Looking at a "Connection refused" error in > the middle of a log file that possibly extends to millions of lines is > near useless. In the era of cloud compute, diagnosing network issues is > sure to become a more common task. > > While we may not be able to put the sensitive information in an > exception message, I think we could put it behind a (new?) system > property which might be able to log this information. Logs contain all > sorts of sensitive data. Take javax.net.debug=ssl output for example. I have some sympathy for (capital-L)ogging such detail messages ( given the reasonable restriction on access to log files ), but a system property that effectively amends exception detail messages, or prints to stdout/stderr is not a runner in my opinion. Maybe we should be looking at instrumentation with JFR events, or similar. My point being, if someone has time and energy enough to spend on this, then we can do better than javax.net.debug=ssl. Also, someone should check that divulging such sensitive information, even in log files, is acceptable from a security perspective. I'm personally still dubious. -Chris. From david.lloyd at redhat.com Fri Dec 22 15:22:41 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 22 Dec 2017 09:22:41 -0600 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> Message-ID: On Thu, Dec 21, 2017 at 6:35 PM, David Holmes wrote: > I believe there are concerns with too much information that can be > considered "sensitive" (like host names and IP addresses) appearing in error > messages due to them ending up in log files and bug reports. I tend to agree here. However - and this is a big "however" - while this is a good policy for the JDK, higher-level applications and libraries often do not have such concerns, since they can often more accurately determine whether a piece of information is actually sensitive; the JDK must simply assume that it is. It should be *possible* for libraries and applications to *add* this information. Right now this is a very, very difficult task: there is an entire hierarchy of exceptions, and there is no easy way to change the message of an exception. The only option is to wrap the exception with another, more descriptive exception, which is not great for users or for developers. It would be nice if there were some way to set some supplementary information on SocketException (or even IOException) after the exception were constructed, which would be included in getMessage() if set. This would allow frameworks and applications to provide some context without causing a security problem for the JDK. -- - DML From forax at univ-mlv.fr Fri Dec 22 15:44:07 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 22 Dec 2017 16:44:07 +0100 (CET) Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> Message-ID: <1002657522.2100060.1513957447421.JavaMail.zimbra@u-pem.fr> Hi David, you can "abuse" of the suppressed exception feature for that, i.e add a CommentException as suppressed exception to the SocketException with no stacktrace and a descriptive error message. R?mi ----- Mail original ----- > De: "David Lloyd" > ?: "David Holmes" > Cc: "core-libs-dev" > Envoy?: Vendredi 22 D?cembre 2017 16:22:41 > Objet: Re: Adding SocketChannel toString to connection exception messages > On Thu, Dec 21, 2017 at 6:35 PM, David Holmes wrote: >> I believe there are concerns with too much information that can be >> considered "sensitive" (like host names and IP addresses) appearing in error >> messages due to them ending up in log files and bug reports. > > I tend to agree here. However - and this is a big "however" - while > this is a good policy for the JDK, higher-level applications and > libraries often do not have such concerns, since they can often more > accurately determine whether a piece of information is actually > sensitive; the JDK must simply assume that it is. It should be > *possible* for libraries and applications to *add* this information. > Right now this is a very, very difficult task: there is an entire > hierarchy of exceptions, and there is no easy way to change the > message of an exception. The only option is to wrap the exception > with another, more descriptive exception, which is not great for users > or for developers. > > It would be nice if there were some way to set some supplementary > information on SocketException (or even IOException) after the > exception were constructed, which would be included in getMessage() if > set. This would allow frameworks and applications to provide some > context without causing a security problem for the JDK. > > > -- > - DML From volker.simonis at gmail.com Fri Dec 22 16:24:45 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 22 Dec 2017 17:24:45 +0100 Subject: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad In-Reply-To: References: <6838c67f5a7f44c0a48a79e8ac47e889@sap.com> Message-ID: Hi Martin, I've just pushed the fix to jdk8u/jdk8u-dev [1] so it should be in the next regular 8u update (probably 8u172 ?). You can of course test it any time by building the tip of http://hg.openjdk.java.net/jdk8u/jdk8u-dev yourself :) Merry Christmas and a happy new year, Volker [1] http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/rev/def07b5ce3be On Wed, Dec 20, 2017 at 11:18 AM, Evans, Martin wrote: > Many thanks guys, I look forward to testing. > > Regards, > > Martin > > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: 20 December 2017 09:33 > To: Baesken, Matthias > Cc: jdk8u-dev-request at openjdk.java.net; core-libs-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Simonis, Volker ; Evans, Martin ; build-dev > Subject: Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad > > Hi Matthias, > > the change looks good! > I can sponsor it once we get the approval. > > Also forwarded to buil-dev for the minimal build change. > > Thank you and best regards, > Volker > > > On Wed, Dec 20, 2017 at 9:50 AM, Baesken, Matthias wrote: >> Hello , Mark reported this issue on AIX with OpenJDK8 : >> >>> >>>I'm getting an unsatisfied link error when using logstash on AIX but I suspect the issue is with AIX or the JRE, the exception is: >>> >>> java.lang.UnsatisfiedLinkError: sun.management.OperatingSystemImpl.getProcessCpuLoad()D >>> getProcessCpuLoad at sun/management/OperatingSystemImpl.java:-2 >>> at org/logstash/instrument/monitors/ProcessMonitor.java:40 >>> detect at org/logstash/instrument/monitors/ProcessMonitor.java:79 >>> generate at >>> org/logstash/instrument/reports/ProcessReport.java:15 >>> >>> this is the line in logstash: >>> >>> this.cpuProcessPercent = >>> scaleLoadToPercent(unixOsBean.getProcessCpuLoad()); >>> >>> https://github.com/elastic/logstash/blob/master/logstash-core/src/mai >>> n/java/org/logstash/instrument/monitors/ProcessMonitor.java >>> >>> >>> Could anybody help steer me in the right direction? >>> >> >> This fix addresses the missing getSystemCpuLoad and getProcessCpuLoad on AIX . >> JDK8 is only affected , JDK9 and higher is not affected . >> >> Could I get a review for this change ? >> >> >> Change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8193807/ >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8193807 >> >> >> Thanks, Matthias > ------------ Kingfisher plc Registered Office: 3 Sheldon Square, Paddington, London W2 6PX Registered in England, Number 1664812 This e-mail is only intended for the person(s) to whom it is addressed and may contain confidential information. Unless stated to the contrary, any opinions or comments are personal to the writer and do not represent the official view of the company. If you have received this e-mail in error, please notify us immediately by reply e-mail and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person. Thank you for your co-operation. From ecki at zusammenkunft.net Fri Dec 22 16:42:45 2017 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Fri, 22 Dec 2017 16:42:45 +0000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com>, <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> Message-ID: Hello, I also dearly miss Socket addresses in connection exceptions, however it looks like it is not going to make it. However if we add a getter for the Peer address (not included in toString) then logging frameworks could detect instances of ConnectException and process them accordingly. Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: net-dev on behalf of Chris Hegarty Sent: Friday, December 22, 2017 4:17:31 PM To: Se?n Coffey; David Holmes; Steven Schlansker Cc: core-libs-dev; net-dev at openjdk.java.net Subject: Re: Adding SocketChannel toString to connection exception messages On 22/12/17 14:59, Se?n Coffey wrote: > As someone who works with alot of log files, I'd like to chime in and > support Steven's end goal. Looking at a "Connection refused" error in > the middle of a log file that possibly extends to millions of lines is > near useless. In the era of cloud compute, diagnosing network issues is > sure to become a more common task. > > While we may not be able to put the sensitive information in an > exception message, I think we could put it behind a (new?) system > property which might be able to log this information. Logs contain all > sorts of sensitive data. Take javax.net.debug=ssl output for example. I have some sympathy for (capital-L)ogging such detail messages ( given the reasonable restriction on access to log files ), but a system property that effectively amends exception detail messages, or prints to stdout/stderr is not a runner in my opinion. Maybe we should be looking at instrumentation with JFR events, or similar. My point being, if someone has time and energy enough to spend on this, then we can do better than javax.net.debug=ssl. Also, someone should check that divulging such sensitive information, even in log files, is acceptable from a security perspective. I'm personally still dubious. -Chris. From mandy.chung at oracle.com Fri Dec 22 16:42:01 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 22 Dec 2017 08:42:01 -0800 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <0a57854d-b5d5-a30a-02cb-14617ef625ca@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> <4e811ed4-54c6-b253-44cb-e2f6a9dd83fe@oracle.com> <34412585-eac6-0e62-a5b8-e8f8dd42645e@oracle.com> <0a57854d-b5d5-a30a-02cb-14617ef625ca@oracle.com> Message-ID: <42fab3cb-3dc0-f311-31aa-26952d0c835e@oracle.com> On 12/22/17 2:16 AM, Chris Hegarty wrote: > > Webrev with the test updated as suggested: > ? http://cr.openjdk.java.net/~chegar/8179424/webrev.03/ > Looks good.? Thanks for updating the test. Mandy From brian.burkhalter at oracle.com Fri Dec 22 16:44:22 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 22 Dec 2017 08:44:22 -0800 Subject: RFR 8193832: Performance of InputStream.readAllBytes() could be improved In-Reply-To: References: <7D881EBD-E7BE-4F95-A672-0DCB3C1D9D95@oracle.com> <7C5CF60A-8C9F-47F2-962F-732FAA121D61@oracle.com> <3B9D20F1-8307-42D7-9A65-A907C6386EBB@oracle.com> <6287F4E4-003A-4214-B6A0-E9DEA75D9205@oracle.com> <6DFDE32A-497A-4D48-B617-51D310A4E9E3@oracle.com> <2E6D22C3-162D-43CB-8E0E-A5B0CBB3B760@oracle.com> <31B64C3F-8B83-465A-8225-703F40B165A7@oracle.com> <8EF70405-4052-4CAD-A347-BC2FBB7DA49D@oracle.com> <698B6461-AFCE-494F-9745-42737FECE15E@oracle.com> Message-ID: Hi Peter, On Dec 22, 2017, at 1:21 AM, Peter Levart wrote: > I think this looks good. No more suggestions from my side for this patch. As Paul notes, it would be interesting to see if using Unsafe.allocateUninitializedArray() to allocate the final array (and intermediate buffers too) has a measurable impact. Thanks for the final look. I am going to run regression tests once again today and if there is no problem I?ll check this in. I?ll also file a follow-up issue about the uninitialized array case. Brian From jason_mehrens at hotmail.com Fri Dec 22 19:07:55 2017 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Fri, 22 Dec 2017 19:07:55 +0000 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> , Message-ID: What are your thoughts on pushing the static EMPTY_XXX declarations from ImmutableCollections down in to each respective inner class? Doing that should allow for on demand class loading instead of getting everything when any factory is used. >http://cr.openjdk.java.net/~redestad/8193128/open.04/ Jason From john.r.rose at oracle.com Fri Dec 22 19:57:26 2017 From: john.r.rose at oracle.com (John Rose) Date: Fri, 22 Dec 2017 11:57:26 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> Message-ID: <204AD1FF-F373-49FD-A83B-65235A1736C7@oracle.com> On Dec 21, 2017, at 5:31 PM, Stuart Marks wrote: > > I'd like a blanket assertion that view collections are not serializable. > > Now this should be considered distinct from whether view collections are value-based; I'm fine with considering view collections to be value-based. It's just that some value-based collections are serializable and others aren't. In particular, given a serializable, value-based list, a sublist should not be serializable but it can be value-based. A sublist of a sublist should also not be serializable and can be value-based, and in this case we can do short-circuiting such as 'return this' if we like. The two concerns can be separated, but they necessarily have a priority, and I think you have got the priority backward. Here's the way I see it: 1. Value-based aggregates are inherently lossless when serialized, and so should (in general) be made serializable. 2. Views are generally lossy when serialized, *if the backing store has a significant identity*, and should (in general) not be made serializable. If we are going to make blanket statements, I claim the first is more fundamental than the second. The second has priority only in history, since serialization was invented before the value-based distinction. I think we should apply the rules in logical order 1, then 2, not historical order 2, then 1. From john.r.rose at oracle.com Fri Dec 22 20:04:52 2017 From: john.r.rose at oracle.com (John Rose) Date: Fri, 22 Dec 2017 12:04:52 -0800 Subject: [10?] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <204AD1FF-F373-49FD-A83B-65235A1736C7@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <9FA6B1FB-F8B1-4FC4-BDCB-F81C43294E25@oracle.com> <0B2FA135-45E6-4400-82E1-E7AD9EF6FBC7@oracle.com> <3a8ac251-9502-4210-fe45-c08e891de327@oracle.com> <204AD1FF-F373-49FD-A83B-65235A1736C7@oracle.com> Message-ID: On Dec 22, 2017, at 11:57 AM, John Rose wrote: > > On Dec 21, 2017, at 5:31 PM, Stuart Marks wrote: >> >> I'd like a blanket assertion that view collections are not serializable. >> >> Now this should be considered distinct from whether view collections are value-based; I'm fine with considering view collections to be value-based. It's just that some value-based collections are serializable and others aren't. In particular, given a serializable, value-based list, a sublist should not be serializable but it can be value-based. A sublist of a sublist should also not be serializable and can be value-based, and in this case we can do short-circuiting such as 'return this' if we like. > > The two concerns can be separated, but they necessarily > have a priority, and I think you have got the priority backward. > Here's the way I see it: > > 1. Value-based aggregates are inherently lossless when > serialized, and so should (in general) be made serializable. > > 2. Views are generally lossy when serialized, *if the backing > store has a significant identity*, and should (in general) not > be made serializable. > > If we are going to make blanket statements, I claim the first > is more fundamental than the second. The second has > priority only in history, since serialization was invented before > the value-based distinction. > > I think we should apply the rules in logical order 1, then 2, > not historical order 2, then 1. > P.S. I am talking about serialization as a general thing with a future. If you are talking only about legacy serialization, then I can see there might be historical influences that would invert the flow of the above logic. Still, I'd like to tweak legacy serialization in better directions AFAP, even if its ultimately futile, so that when we come up with something better we will have relatively more good precedents to preserve. From brian.burkhalter at oracle.com Fri Dec 22 21:51:17 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 22 Dec 2017 13:51:17 -0800 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method Message-ID: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8194133 http://cr.openjdk.java.net/~bpb/8194133/webrev.00/ Add jdk.internal.io.IOSupport with copy() methods for InputStream-to-OutputStream copying and modify some classes to use these new methods. One thing that I noticed when looking at this is that in the fix for https://bugs.openjdk.java.net/browse/JDK-8193842, the Files.copy() method had a loop like while ((n = read(?)) > 0) whereas InputStream.transferTo() had while((n = read(?)) >= 0) which is to say that Files.copy() would terminate if there were an empty read() but transferTo() would not. The patch for 8193842 therefore possibly introduced a subtle behavioral change which no one noticed. Thanks, Brian From forax at univ-mlv.fr Sat Dec 23 23:27:19 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 24 Dec 2017 00:27:19 +0100 (CET) Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> Message-ID: <1633500751.2343981.1514071639271.JavaMail.zimbra@u-pem.fr> Hi Brian, it's not clear to me that all internal usages should use IOSupport.copy and not InputStream.transferTo, a specific InputStream implementation (like DataInputStream) may come with a more optimized version of transferTo. It remember me Collections.sort()/List.sort(), at first it was decided that List.sort() should call Collections.sort() but after thinking a little bit more, Collections.sort() was re-written to call List.sort() because it may use faster implementation (like ArrayList.sort()). cheers, R?mi ----- Mail original ----- > De: "Brian Burkhalter" > ?: "core-libs-dev" > Envoy?: Vendredi 22 D?cembre 2017 22:51:17 > Objet: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method > https://bugs.openjdk.java.net/browse/JDK-8194133 > http://cr.openjdk.java.net/~bpb/8194133/webrev.00/ > > Add jdk.internal.io.IOSupport with copy() methods for > InputStream-to-OutputStream copying and modify some classes to use these new > methods. > > One thing that I noticed when looking at this is that in the fix for > https://bugs.openjdk.java.net/browse/JDK-8193842, the Files.copy() method had a > loop like > > while ((n = read(?)) > 0) > > whereas InputStream.transferTo() had > > while((n = read(?)) >= 0) > > which is to say that Files.copy() would terminate if there were an empty read() > but transferTo() would not. The patch for 8193842 therefore possibly introduced > a subtle behavioral change which no one noticed. > > Thanks, > > Brian From wenqian.peiwq at alibaba-inc.com Mon Dec 25 09:40:14 2017 From: wenqian.peiwq at alibaba-inc.com (Wenqian Pei) Date: Mon, 25 Dec 2017 17:40:14 +0800 Subject: =?UTF-8?B?UkZSOiA4MTk0MTU0IHBhdGNoIGZvciBjcmFzaCBhdCBGaWxlLmdldENhbm9uaWNhbFBhdGgo?= =?UTF-8?B?KQ==?= Message-ID: Hi: Bug:?https://bugs.openjdk.java.net/browse/JDK-8194154 We?found?that?if?user?defines?-Duser.dir?like?"/home/a/b/c/",?jvm?will?crash?at?File.getCanonicalPath()?in?java?process?bootstrap?(before?invoking?user's?java?code).?The?native?implematation?of?canonicalize_md.c:collapsible(char?*names)?has?problem?in?processing?double?'/',?parameter 'names' need?normalized?before?JNI_CALL. This?patch?normalize?parameters?before?call?canonicalize0()?in?this?call?path Patch?and?test?are?in?mailbox?attachments. Can?I?please?have?a?review?for?this?patch? Thanks Wenqian?Pei From Martin.Evans at kingfisher.com Fri Dec 22 20:53:49 2017 From: Martin.Evans at kingfisher.com (Evans, Martin) Date: Fri, 22 Dec 2017 20:53:49 +0000 Subject: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad In-Reply-To: References: <6838c67f5a7f44c0a48a79e8ac47e889@sap.com> Message-ID: Many thanks Volker, I am on vacation now but I will put it the top of my todo list. Merry Christmas and a happy new year to you too ?? Regards, Martin -----Original Message----- From: Volker Simonis [mailto:volker.simonis at gmail.com] Sent: 22 December 2017 16:25 To: Evans, Martin Cc: Baesken, Matthias ; jdk8u-dev-request at openjdk.java.net; core-libs-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Simonis, Volker ; build-dev Subject: Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad Hi Martin, I've just pushed the fix to jdk8u/jdk8u-dev [1] so it should be in the next regular 8u update (probably 8u172 ?). You can of course test it any time by building the tip of http://hg.openjdk.java.net/jdk8u/jdk8u-dev yourself :) Merry Christmas and a happy new year, Volker [1] http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/rev/def07b5ce3be On Wed, Dec 20, 2017 at 11:18 AM, Evans, Martin wrote: > Many thanks guys, I look forward to testing. > > Regards, > > Martin > > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: 20 December 2017 09:33 > To: Baesken, Matthias > Cc: jdk8u-dev-request at openjdk.java.net; > core-libs-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; > Simonis, Volker ; Evans, Martin > ; build-dev > Subject: Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by > providing empty basic implementations of getSystemCpuLoad and > getProcessCpuLoad > > Hi Matthias, > > the change looks good! > I can sponsor it once we get the approval. > > Also forwarded to buil-dev for the minimal build change. > > Thank you and best regards, > Volker > > > On Wed, Dec 20, 2017 at 9:50 AM, Baesken, Matthias wrote: >> Hello , Mark reported this issue on AIX with OpenJDK8 : >> >>> >>>I'm getting an unsatisfied link error when using logstash on AIX but I suspect the issue is with AIX or the JRE, the exception is: >>> >>> java.lang.UnsatisfiedLinkError: sun.management.OperatingSystemImpl.getProcessCpuLoad()D >>> getProcessCpuLoad at sun/management/OperatingSystemImpl.java:-2 >>> at org/logstash/instrument/monitors/ProcessMonitor.java:40 >>> detect at org/logstash/instrument/monitors/ProcessMonitor.java:79 >>> generate at >>> org/logstash/instrument/reports/ProcessReport.java:15 >>> >>> this is the line in logstash: >>> >>> this.cpuProcessPercent = >>> scaleLoadToPercent(unixOsBean.getProcessCpuLoad()); >>> >>> https://github.com/elastic/logstash/blob/master/logstash-core/src/ma >>> i n/java/org/logstash/instrument/monitors/ProcessMonitor.java >>> >>> >>> Could anybody help steer me in the right direction? >>> >> >> This fix addresses the missing getSystemCpuLoad and getProcessCpuLoad on AIX . >> JDK8 is only affected , JDK9 and higher is not affected . >> >> Could I get a review for this change ? >> >> >> Change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8193807/ >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8193807 >> >> >> Thanks, Matthias > ------------ Kingfisher plc Registered Office: 3 Sheldon Square, Paddington, London W2 6PX Registered in England, Number 1664812 This e-mail is only intended for the person(s) to whom it is addressed and may contain confidential information. Unless stated to the contrary, any opinions or comments are personal to the writer and do not represent the official view of the company. If you have received this e-mail in error, please notify us immediately by reply e-mail and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person. Thank you for your co-operation. ------------ Kingfisher plc Registered Office: 3 Sheldon Square, Paddington, London W2 6PX Registered in England, Number 1664812 This e-mail is only intended for the person(s) to whom it is addressed and may contain confidential information. Unless stated to the contrary, any opinions or comments are personal to the writer and do not represent the official view of the company. If you have received this e-mail in error, please notify us immediately by reply e-mail and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person. Thank you for your co-operation. From michael.osipov at siemens.com Fri Dec 22 09:17:26 2017 From: michael.osipov at siemens.com (Osipov, Michael) Date: Fri, 22 Dec 2017 09:17:26 +0000 Subject: [RFR] 8160768: Add capability to custom resolve host/domain names within the default JDNI LDAP provider In-Reply-To: <20171206182434.GB4409@vimes> References: <20171205174824.GA3515@vimes> <20171206182434.GB4409@vimes> Message-ID: <68644224DA0DE64CA5A49838ED219A042A6CDD70@DEFTHW99EJ5MSX.ww902.siemens.net> Hi Rob, a few comments on the webrev 07: 1. You have changed the semantics of domainName from null to "". This has implications on the StartTlsResponseImpl in regard with cert validation. This has to be discussion with someone who is firm with the STARTTLS command. 2. I miss documentation that says LdapDnsProviderResult can be null in LdapDnsProviderService(). If it is not null, getEndpoints() and getDomainName() must not be null. This uncertainty can easily lead to NPEs. 3. LdapCtxFactory: + // getDnsUrls(url, env) may throw a NamingException, which there is + // no need to wrap. There is no getDnsUrls anymore 4. What is the purpose of using Void in LdapDnsProvider? Michael > -----Original Message----- > From: Rob McKenna [mailto:rob.mckenna at oracle.com] > Sent: Wednesday, December 06, 2017 7:25 PM > To: vyom tewari > Cc: core-libs-dev at openjdk.java.net; Osipov, Michael (GS IT PD LD PLM) > Subject: Re: [RFR] 8160768: Add capability to custom resolve host/domain > names within the default JDNI LDAP provider > > Thanks Vyom, these should be fixed in: > > http://cr.openjdk.java.net/~robm/8160768/webrev.07/ > > Comments inline: > > On 06/12/17 22:14, vyom tewari wrote: > > Hi Rob, > > > > Please find below comments. > > > > DefaultLdapDnsProvider.java > > > > ?line 35: ??? convert it to for-each code will be more readable. > > > > LdapDnsProviderService.java > > > > ?line 21: constant name does not follow Java naming convention, there > are > > other places as well please fix it. > > > > getInstance() is already synchronized why you need another > "synchronized" > > block ? > > Ah, neglected to remove the outer synchronized keyword, thanks. > > > > > Line 70: Please use result.getEndpoints().isEmpty() in place of > > result.getEndpoints().size() == 0 > > > > LdapDnsProvider.java > > > > constructor LdapDnsProvider() calls LdapDnsProvider(Void unused) which > does > > nothing, can you explain why LdapDnsProvider()? calls > LdapDnsProvider(Void > > unused) ? > > I've copied this pattern from the System.LoggerFinder api and I had > forgotten what it was for too: > > http://www.oracle.com/technetwork/java/seccodeguide-139067.html#7 > > Section 7.3. > > > > > LdapDnsProviderResult.java > > > > ? Private field? domainName & endpoints both can be final. > > > > LdapDnsProviderTest.java > > > > Fix the tag Order it is not correct. correct order is as follows. > > > > /** > > ?* @test > > ?* @bug 8160768 > > ?* @summary ctx provider tests for ldap > > ?* @modules java.naming/com.sun.jndi.ldap > > ?* @compile dnsprovider/TestDnsProvider.java > > ?* @run main/othervm LdapDnsProviderTest > > ?* @run main/othervm LdapDnsProviderTest nosm > > ?* @run main/othervm LdapDnsProviderTest smnodns > > ?* @run main/othervm LdapDnsProviderTest smdns > > ?*/ > > > > Line 82,83 you don't need System.out.println(e); e.printStackTrace(); > > I'm going to leave these in as I've had a few failing tests in the past > where I've needed to push diagnostic changes like this to determine the > root cause. > > -Rob > > > > > Line 70: you don't need this try block > > > > Line 96 : constant name does not follow Java naming convention > > > > Line 105: you can use try-with-resources this will avoid some > boilerplate. > > > > Thanks, > > Vyom > > > > On Tuesday 05 December 2017 11:18 PM, Rob McKenna wrote: > > >As this enhancement is limited to JDK10 this frees us up to explore a > > >different approach. > > > > > >http://cr.openjdk.java.net/~robm/8160768/webrev.06/ > > > > > >In this webrev we're using the Service Provider Interface to load an > > >implementation of the new LdapDnsProvider class. Implementations of > this > > >class are responsible for resolving DNS endpoints for a given ldap url > > >should the default implementation be insufficient in a particular > > >environment. > > > > > >Note: if a security manager is installed the ldapDnsProvider > > >RuntimePermission must be granted. > > > > > > -Rob > > > > > From david.holmes at oracle.com Wed Dec 27 06:14:44 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Dec 2017 16:14:44 +1000 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: References: Message-ID: Hi, Attachments get stripped (usually). You'll need to include it inline. Cheers, David On 25/12/2017 7:40 PM, Wenqian Pei wrote: > > Hi: > > Bug:?https://bugs.openjdk.java.net/browse/JDK-8194154 > > We?found?that?if?user?defines?-Duser.dir?like?"/home/a/b/c/",?jvm?will?crash?at?File.getCanonicalPath()?in?java?process?bootstrap?(before?invoking?user's?java?code).?The?native?implematation?of?canonicalize_md.c:collapsible(char?*names)?has?problem?in?processing?double?'/',?parameter 'names' need?normalized?before?JNI_CALL. > > This?patch?normalize?parameters?before?call?canonicalize0()?in?this?call?path > > Patch?and?test?are?in?mailbox?attachments. > > Can?I?please?have?a?review?for?this?patch? > > > Thanks > > > Wenqian?Pei > From wenqian.peiwq at alibaba-inc.com Wed Dec 27 06:44:14 2017 From: wenqian.peiwq at alibaba-inc.com (Wenqian Pei) Date: Wed, 27 Dec 2017 14:44:14 +0800 Subject: =?UTF-8?B?5Zue5aSN77yaUkZSOiA4MTk0MTU0IHBhdGNoIGZvciBjcmFzaCBhdCBGaWxlLmdldENhbm9u?= =?UTF-8?B?aWNhbFBhdGgoKQ==?= In-Reply-To: f42ef9c7-75d6-4e8a-f911-a416085c3fd1@oracle.com References: , f42ef9c7-75d6-4e8a-f911-a416085c3fd1@oracle.com Message-ID: Sorry, the patch and test is below: diff?-r?777356696811?src/java.base/unix/classes/java/io/UnixFileSystem.java ---?a/src/java.base/unix/classes/java/io/UnixFileSystem.java????Fri?Sep?08?18:24:17?2017?+0000 +++?b/src/java.base/unix/classes/java/io/UnixFileSystem.java????Mon?Dec?25?16:48:36?2017?+0800 @@?-100,10?+100,10?@@ ?????????if?(child.equals(""))?return?parent; ?????????if?(child.charAt(0)?==?'/')?{ ?????????????if?(parent.equals("/"))?return?child; -????????????return?parent?+?child; +????????????return?normalize(parent?+?child); ?????????} ?????????if?(parent.equals("/"))?return?parent?+?child; -????????return?parent?+?'/'?+?child; +????????return?normalize(parent?+?'/'?+?child); ?????} ? ?????public?String?getDefaultParent()?{ diff?-r?777356696811?test/java/io/File/GetCanonicalPath.java ---?a/test/java/io/File/GetCanonicalPath.java???Fri?Sep?08?18:24:17?2017?+0000 +++?b/test/java/io/File/GetCanonicalPath.java???Mon?Dec?25?16:48:36?2017?+0800 @@?-23,20?+23,35?@@ ? ?/*?@test ????@bug?4899022 +???@library?/lib/testlibrary +???@run?main/othervm?GetCanonicalPath? ????@summary?Look?for?erroneous?representation?of?drive?letter ??*/ ? ?import?java.io.*; +import?java.lang.reflect.Field; +import?java.util.Properties; ? ?public?class?GetCanonicalPath?{ ?????public?static?void?main(String[]?args)?throws?Exception?{ ?????????if?(File.separatorChar?==?'\\')?{ ?????????????testDriveLetter(); ?????????} +????????String?osName?=?System.getProperty("os.name"); +????????if?(osName.startsWith("Linux"))?{ +????????????testDuplicateSeparators(); +????????} ?????} ?????private?static?void?testDriveLetter()?throws?Exception?{ ?????????String?path?=?new?File("c:/").getCanonicalPath(); ?????????if?(path.length()?>?3) ?????????????throw?new?RuntimeException("Drive?letter?incorrectly?represented"); ?????} +????private?static?void?testDuplicateSeparators()?throws?Exception?{ +????????Field?f?=?System.class.getDeclaredField("props"); +????????f.setAccessible(true); +????????Properties?p?=?(Properties)?f.get(null); +????????p.setProperty("user.dir",?"/home/a/b/c/"); +????????System.out.println(new?File("./a").getCanonicalPath()); +???} ?}? ps: only test on Linux :) ------------------------------------------------------------------????David Holmes ?????2017?12?27?(???) 14:14???????(??) ; core-libs-dev ???????(??) ; ???(??) ????Re: RFR: 8194154 patch for crash at File.getCanonicalPath() Hi, Attachments?get?stripped?(usually).?You'll?need?to?include?it?inline. Cheers, David On?25/12/2017?7:40?PM,?Wenqian?Pei?wrote: >? >?Hi: >? >?Bug:?https://bugs.openjdk.java.net/browse/JDK-8194154 >? >?We?found?that?if?user?defines?-Duser.dir?like?"/home/a/b/c/",?jvm?will?crash?at?File.getCanonicalPath()?in?java?process?bootstrap?(before?invoking?user's?java?code).?The?native?implematation?of?canonicalize_md.c:collapsible(char?*names)?has?problem?in?processing?double?'/',?parameter?'names'?need?normalized?before?JNI_CALL. >? >?This?patch?normalize?parameters?before?call?canonicalize0()?in?this?call?path >? >?Patch?and?test?are?in?mailbox?attachments. >? >?Can?I?please?have?a?review?for?this?patch? >? >? >?Thanks >? >? >?Wenqian?Pei >? From hboehm at google.com Thu Dec 28 20:32:50 2017 From: hboehm at google.com (Hans Boehm) Date: Thu, 28 Dec 2017 12:32:50 -0800 Subject: FileOutputStream.getFD() vs finalization Message-ID: The design of the getFD() calls in some Java *Stream classes seems problematic, and doesn't seem cleanly fixable without a spec change. I first noticed this in JDK 8 code, but Roger Riggs recent JDK 10 changes also don't seem to fully address this specific problem, particularly since the code paths remain more-or-less similar when a FOS subclass has a close method. It probably remains easier to describe this in a JDK 8/9 setting, so I'll do so initially. Close() on the *Stream closes the FD. The finalize() method on the Stream (not the FD!) calls close(). Assuming JLS 12.6.2 finalization semantics, if my program consists of: { FileOutputStream f1 = new FileOutputStream("foo"); FileDescriptor fd = f1.getFD(); A: FileOutputStream f2 = new FileOutputStream(fd); } f2 may be associated with a closed fd, since f1 could have been finalized at any pointer after its construction, in particular at point A. This is slightly aggravated by the fact that close() is implemented in a way that doesn't guarantee reachability of the FileOutputStream while, or before, it runs. Close() could easily be inlined, and the *Stream fields promoted to registers. Thus even if you always call close() explicitly, I think you are still affected by this problem. Fundamentally the result of getFD() is only valid as long as the underlying *Stream is reachable, and that is not defined in a way that's understandable by most programmers. That also makes this API un-Java-like, in that the user has to worry about very subtle object lifetime rules. The only semi-plausible solution seems to involve judicious use of ReachabilityFence() on FileOutputStreams after the getFD() result is no longer needed. I personally don't think that's really a very plausible request of the programmer. I think Roger's JDK 10 changes may help in the case in which there is not an overridden close() method, since the cleanup action is registered on the underlying FileDescriptor. But if a subclass overrides close(), I think we're basically still in the same boat. For the reasons he already gave, that seems hard to avoid. And again I think the problem applies even if close() is explicitly called, since that doesn't really prevent finalization of the Stream object. Has this been discussed anywhere? Opinions about how to deal with it? (This arose while going through the core libraries, looking for premature finalization issues. So far, this one seems special, in that It really seems to require an API change.) Hans From stevenschlansker at gmail.com Fri Dec 29 00:33:10 2017 From: stevenschlansker at gmail.com (Steven Schlansker) Date: Thu, 28 Dec 2017 16:33:10 -0800 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> Message-ID: Thanks for the discussion! So, it sounds like amending the message by default is going to be a non-starter -- but at least adding the information otherwise might be possible. One possibility would be to add new fields to SocketException, for example an InetSocketAddress representing both the local and remote (if available). This would not be rendered by default, but could be retrieved by a cooperating application or library. Or maybe a second step could be a '-Djavax.net.debug=exceptions' that then appends this information by default, but that sounds more controversial. Then at least libraries and applications have the ability to get these diagnostics, even if they choose not to. Is this an acceptable approach? Would it be accepted as a patch? > On Dec 22, 2017, at 8:42 AM, Bernd Eckenfels wrote: > > Hello, > > I also dearly miss Socket addresses in connection exceptions, however it looks like it is not going to make it. However if we add a getter for the Peer address (not included in toString) then logging frameworks could detect instances of ConnectException and process them accordingly. > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ________________________________ > From: net-dev on behalf of Chris Hegarty > Sent: Friday, December 22, 2017 4:17:31 PM > To: Se?n Coffey; David Holmes; Steven Schlansker > Cc: core-libs-dev; net-dev at openjdk.java.net > Subject: Re: Adding SocketChannel toString to connection exception messages > > > On 22/12/17 14:59, Se?n Coffey wrote: >> As someone who works with alot of log files, I'd like to chime in and >> support Steven's end goal. Looking at a "Connection refused" error in >> the middle of a log file that possibly extends to millions of lines is >> near useless. In the era of cloud compute, diagnosing network issues is >> sure to become a more common task. >> >> While we may not be able to put the sensitive information in an >> exception message, I think we could put it behind a (new?) system >> property which might be able to log this information. Logs contain all >> sorts of sensitive data. Take javax.net.debug=ssl output for example. > > I have some sympathy for (capital-L)ogging such detail messages > ( given the reasonable restriction on access to log files ), but > a system property that effectively amends exception detail > messages, or prints to stdout/stderr is not a runner in my > opinion. > > Maybe we should be looking at instrumentation with JFR events, or > similar. My point being, if someone has time and energy enough > to spend on this, then we can do better than javax.net.debug=ssl. > Also, someone should check that divulging such sensitive information, > even in log files, is acceptable from a security perspective. I'm > personally still dubious. > > -Chris. From chris.hegarty at oracle.com Fri Dec 29 17:15:14 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 29 Dec 2017 17:15:14 +0000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> Message-ID: > On 29 Dec 2017, at 00:33, Steven Schlansker wrote: > > Thanks for the discussion! > > So, it sounds like amending the message by default is going to be a non-starter -- but at least adding the information otherwise might be possible. > > One possibility would be to add new fields to SocketException, for example an InetSocketAddress representing both the local and remote (if available). You would need to careful to not disclose resolved addresses to untrusted code. SocketException, since a subtype of IOException, can wind up in many places. Would you be proposing to add these new fields to the serial-form of SocketException? What behaviour do you envisage for deserializing instances from previous releases? This will have an impact of any potential API. > This would not be rendered by default, but could be retrieved by a cooperating application or library. Or maybe a second step could be a '-Djavax.net.debug=exceptions' that then appends this information by default, but that sounds more controversial. Logging seems like a better alternative than a system property, to me. > Then at least libraries and applications have the ability to get these diagnostics, even if they choose not to. > Is this an acceptable approach? Would it be accepted as a patch? I suspect that a webrev/patch would help drive the discussion, rather than a commitment to accepting it into the JDK. -Chris. >> On Dec 22, 2017, at 8:42 AM, Bernd Eckenfels wrote: >> >> Hello, >> >> I also dearly miss Socket addresses in connection exceptions, however it looks like it is not going to make it. However if we add a getter for the Peer address (not included in toString) then logging frameworks could detect instances of ConnectException and process them accordingly. >> >> Gruss >> Bernd >> -- >> http://bernd.eckenfels.net >> ________________________________ >> From: net-dev on behalf of Chris Hegarty >> Sent: Friday, December 22, 2017 4:17:31 PM >> To: Se?n Coffey; David Holmes; Steven Schlansker >> Cc: core-libs-dev; net-dev at openjdk.java.net >> Subject: Re: Adding SocketChannel toString to connection exception messages >> >> >>> On 22/12/17 14:59, Se?n Coffey wrote: >>> As someone who works with alot of log files, I'd like to chime in and >>> support Steven's end goal. Looking at a "Connection refused" error in >>> the middle of a log file that possibly extends to millions of lines is >>> near useless. In the era of cloud compute, diagnosing network issues is >>> sure to become a more common task. >>> >>> While we may not be able to put the sensitive information in an >>> exception message, I think we could put it behind a (new?) system >>> property which might be able to log this information. Logs contain all >>> sorts of sensitive data. Take javax.net.debug=ssl output for example. >> >> I have some sympathy for (capital-L)ogging such detail messages >> ( given the reasonable restriction on access to log files ), but >> a system property that effectively amends exception detail >> messages, or prints to stdout/stderr is not a runner in my >> opinion. >> >> Maybe we should be looking at instrumentation with JFR events, or >> similar. My point being, if someone has time and energy enough >> to spend on this, then we can do better than javax.net.debug=ssl. >> Also, someone should check that divulging such sensitive information, >> even in log files, is acceptable from a security perspective. I'm >> personally still dubious. >> >> -Chris. > From peter.levart at gmail.com Sun Dec 31 11:53:51 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 31 Dec 2017 12:53:51 +0100 Subject: FileOutputStream.getFD() vs finalization In-Reply-To: References: Message-ID: <3b973fb9-beac-b0cb-67ce-d61977dd6f6d@gmail.com> Hi, Maybe I'm missing something but... Hans Boehm je 28. 12. 2017 ob 21:32?napisal: > The design of the getFD() calls in some Java *Stream classes seems > problematic, and doesn't seem cleanly fixable without a spec change. I > first noticed this in JDK 8 code, but Roger Riggs recent JDK 10 changes > also don't seem to fully address this specific problem, particularly since > the code paths remain more-or-less similar when a FOS subclass has a close > method. It probably remains easier to describe this in a JDK 8/9 setting, > so I'll do so initially. > > Close() on the *Stream closes the FD. The finalize() method on the Stream > (not the FD!) calls close(). Assuming JLS 12.6.2 finalization semantics, if > my program consists of: > > { > FileOutputStream f1 = new FileOutputStream("foo"); > FileDescriptor fd = f1.getFD(); > A: > FileOutputStream f2 = new FileOutputStream(fd); > } > > f2 may be associated with a closed fd, since f1 could have been finalized > at any pointer after its construction, in particular at point A. I think there is no such problem here. Each stream (FileInputStream, FileOutputStream and RandomAccessFile) keeps a reference to the FileDescriptor (via 'fd' field) and FileDescriptor keeps a reference to all associated streams (established in stream constructor via FileDescriptor.attach(Closeable) where the Closeable instance is the stream instance). So as long as 'fd' above is reachable, so is 'f1'. > This is slightly aggravated by the fact that close() is implemented in a > way that doesn't guarantee reachability of the FileOutputStream while, or > before, it runs. Close() could easily be inlined, and the *Stream fields > promoted to registers. Thus even if you always call close() explicitly, I > think you are still affected by this problem. File[Input|Output]Stream.close() is an instance method that among other things, calls close0() native instance method which accesses field 'fd' in native code via JNI. While close0() native instance method is being executed, the stream instance is kept reachable (by the JNI spec). In fact, all stream native methods that do make use of 'fd' are instance methods that keep stream instance is reachable while being executed - making the native resource access. So I think FileInputStream, FileOutputStream and RandomAccessFile are examples of code where finalization was actually done right, because it is consistently combined with native instance methods. Unlike direct NIO buffers that keep native resource (long address) in a field and pass its value around to other classes (Unsafe) not making sure that the buffer instance which is tracked by Cleaner is kept reachable everywhere the underlying native memory is being accessed. Regards, Peter > Fundamentally the result of getFD() is only valid as long as the underlying > *Stream is reachable, and that is not defined in a way that's > understandable by most programmers. That also makes this API un-Java-like, > in that the user has to worry about very subtle object lifetime rules. > > The only semi-plausible solution seems to involve judicious use of > ReachabilityFence() on FileOutputStreams after the getFD() result is no > longer needed. I personally don't think that's really a very plausible > request of the programmer. > > I think Roger's JDK 10 changes may help in the case in which there is not > an overridden close() method, since the cleanup action is registered on the > underlying FileDescriptor. But if a subclass overrides close(), I think > we're basically still in the same boat. For the reasons he already gave, > that seems hard to avoid. And again I think the problem applies even if > close() is explicitly called, since that doesn't really prevent > finalization of the Stream object. > > Has this been discussed anywhere? Opinions about how to deal with it? > > (This arose while going through the core libraries, looking for premature > finalization issues. So far, this one seems special, in that It really > seems to require an API change.) > > Hans From peter.levart at gmail.com Sun Dec 31 15:24:20 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 31 Dec 2017 16:24:20 +0100 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> Message-ID: Hi, David Holmes je 22. 12. 2017 ob 01:35?napisal: > On 22/12/2017 10:29 AM, Steven Schlansker wrote: >> >>> On Dec 21, 2017, at 11:11 AM, Steven Schlansker >>> wrote: >>> >>> What if ConnectException included the attempted hostname / IP / port >>> SocketAddress? >>> java.net.ConnectException: Connection to >>> 'foo.mycorp.com[10.x.x.x]:12345' refused >>> Much more useful!? This could also be extended to various other >>> socket exceptions. > > I believe there are concerns with too much information that can be > considered "sensitive" (like host names and IP addresses) appearing in > error messages due to them ending up in log files and bug reports. > > David For debugging purposes it might sometimes be enough to get just a hint about the actual address / port but not reveal it entirely. The person doing debugging probably knows more about the environment than an average person so the hint might give him enough information to discern the actual address / port. Exposing just the last octet of an IP address and the last digit of the port might do. For example: java.net.ConnectException: Connection to X.X.X.205:XXX8 refused. So Steven, I'm curious whether such hint would help in your case? An attacker that knows something about the environment could find out the missing pieces without such hints anyway (simply by scanning IPs / ports), so such partial information is not that sensitive nowadays. Another idea: define a one way function that maps the IP:port pair into a value which is displayed in the exception message. For debugging purposes this might be enough since the one doing debugging might know the set of possible IP:port pairs in advance. He could then apply the function to each of them in turn and find out the matching pair. Regards, Peter