From cushon at google.com Fri Jun 1 00:39:00 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Thu, 31 May 2018 17:39:00 -0700 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> <5A90B204.4070005@oracle.com> <5A960359.4070507@oracle.com> Message-ID: Hi, Are there any concerns with the patch that I can address? I was hoping to get this in to JDK 11. I confirmed that it still builds and passes all tests at head. Thanks, Liam On Mon, May 14, 2018 at 10:38 AM Liam Miller-Cushon wrote: > Ping > > On Thu, Apr 5, 2018 at 10:57 AM Liam Miller-Cushon > wrote: > >> Hi, >> >> Is there any more feedback on the patch? >> >> On Wed, Feb 28, 2018 at 4:26 PM Liam Miller-Cushon >> wrote: >> >>> On Wed, Feb 28, 2018 at 4:13 PM, Martin Buchholz >>> wrote: >>> >>>> Follow their lead and rename your method to assertArrayEquals >>>> >>> >>> Done: http://cr.openjdk.java.net/~cushon/7183985/webrev.04/ >>> >> From mandy.chung at oracle.com Fri Jun 1 03:36:35 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 31 May 2018 20:36:35 -0700 Subject: RFR: 8203357 Container Metrics In-Reply-To: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <6b8f5228-03ea-9521-aa66-e74b5595722b@oracle.com> Hi Bob, On 5/30/18 12:45 PM, Bob Vandette wrote:> > RFE: Container Metrics > > https://bugs.openjdk.java.net/browse/JDK-8203357 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.00 Looks fine in general. It's good to have this internal API ready for JFR and other library code to use. I skimmed through the new tests. It'd be good to add some comments to describe what it does (for example, set up a docker image etc). launcher.properties 154 \ -XshowSettings:system (Linux Only) show host system or container\n\ 155 \ configuration and continue\n\ A newline can be placed after -XshowSettings:system consistent with other suboptions. test/lib/jdk/test/lib/containers/docker/DockerTestUtils.java There are several long lines in the new test files such as: MetricsCpuTester.java MetricsMemoryTester.java MetricsTester.java It'd help future side-by-side review if they are wrapped. I think most of them are the construction of an exception. I see a pattern of a name after @test and that is not strictly needed. TestCgroupMetrics.java 25 * @test TestCgroupMetrics TestDockerCpuMetrics.java 34 * @test TestSystemMetrics TestDockerMemoryMetrics.java 30 * @test TestSystemMetrics TestSystemMetrics.java 25 * @test TestSystemMetrics This needs a CSR for the new -XshowSettings:system option. Mandy From Alan.Bateman at oracle.com Fri Jun 1 07:53:59 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 1 Jun 2018 08:53:59 +0100 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> Message-ID: On 31/05/2018 01:10, Kevin Rushforth wrote: > I would like to propose the following Draft JEP [1] for discussion. > > JDK-8200758: Packaging Tool > > This is intended as a JDK-scope replacement for the existing > javapackager tool that ships with Oracle JDK 10 (and earlier Oracle > JDK releases), and was delivered as part of the JavaFX build. The > javapackager tool has been removed from Oracle JDK 11 along with the > removal of JavaFX. > > Comments on this JEP are welcome. It is currently not targeted for a > specific release, but we are aiming for JDK 12. > > -- Kevin > > [1] https://bugs.openjdk.java.net/browse/JDK-8200758 > I read through the draft. The Goals and Non-Goals looks very reasonable. The Summary includes "self-contained Java Server ... applications" but the JEP doesn't say too much about that part. Can we assume it can be started and stopped with /etc/init.d or equivalent scripts, logs with the server output etc? A general observation is that most of the issues around end-user/GUI applications are clearly documented in the draft, the headless application environment case less so. It makes me wonder if this JEP should be split so that it initially focuses on the former. I think it would be useful if the JEP explained what an "application" is. Is it a JAR file (with a Main-Class) that is deployed on the class path? Can it be a module? What about modules and libraries that the application uses. Users of javapackager might know these things but readers of the JEP may not. The JEP mentions that JavaFX will be removed in JDK 11. I assume this should be clarified so that it's clear it will no longer be shipped in Oracle's JDK. It's a none issue for those using OpenJDK builds as the the JavaFX modules were never bundled/imported by default. There are several references to "server JRE" that probably should be clarified as this notion does not exist in OpenJDK. It may be that the JEP just needs to clearer as to the modules that it links into the run-time image. There are several references to "application launcher" and "native launcher". At some point we need to work out some issues with jlink as it it will need to generate native launchers too. The JEP could list it as an issue as the development of this JEP will at least touch on some of this. The draft doesn't have a section on testing. If someone is building OpenJDK then will they will require additional tools in the build environment and can the tests be run without manual interaction? Also how about a developer using the tool, will the generated native packages be easy to un-install so they can easily test in a local environment? -Alan From bhamaram at in.ibm.com Fri Jun 1 08:03:00 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Fri, 1 Jun 2018 08:03:00 +0000 Subject: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 In-Reply-To: References: , <5331ad7ba3c63a066895d41a91bd9593@linux.vnet.ibm.com> Message-ID: Hi Thomas, Sorry, I was on vacation. I will submit webrev with jtreg testcase. Thanks, Bhaktavatsal -----"Thomas St?fe" wrote: ----- To: Bhaktavatsal R Maram From: "Thomas St?fe" Date: 05/23/2018 12:32AM Cc: Ichiroh Takiguchi , Volker Simonis , Java Core Libs Subject: Re: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 Hi Bhaktavatsal, the fix is fine. Reviewed. Thanks a lot for your work! Could you please (its okay in a future patch) add a regression test for IBM943 and IBM943C, in the form of a jtreg test? I would like to test conversions for both code pages (at least for the crucial tilde/backslash code points). Additionally, that when started on AIX with Ja_JP.IBM-943, we correctly default to IBM943C. As an example, here is an old test I wrote years ago. You won't be able to use it verbatim, so it is just a suggestion. Feel free to do it differently. http://cr.openjdk.java.net/~stuefe/other/IBM943MappingTest.java If you have not written jtreg tests before: http://openjdk.java.net/jtreg/ Also, under /test are many many examples. Best Regards, Thomas On Mon, May 21, 2018 at 9:47 AM, Bhaktavatsal R Maram wrote: > Hi Thomas, > > Is this fix ready to merge? > > Thanks, > Bhaktavatsal > > > > > -----"Thomas St?fe" wrote: ----- > To: Ichiroh Takiguchi , Bhaktavatsal R Maram > From: "Thomas St?fe" > Date: 05/11/2018 11:49AM > Cc: Volker Simonis , Java Core Libs > Subject: Re: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 > > Hi, > > I'll test and review next week. We also have some in-house tests which > I'd like to run. > > You IBM folks should really apply for authorship so that this > contribution process gets streamlined. After all, if something breaks > in this code, you want to be able to fix it, yes? So even if you do > not contribute much else, more patches may be forthcoming. > > Of course I hope these are not your last contributions :) > > Best, Thomas > > > > On Fri, May 11, 2018 at 7:57 AM, Ichiroh Takiguchi > wrote: >> Hi. >> >> I tested this fix on AIX. >> >> I got following results. >> $ LANG=Ja_JP ~/jdk/bin/java PrintDefaultCharset >> Ja_JP x-IBM943C IBM-943C IBM-943C >> $ LANG=Ja_JP.IBM-943 ~/jdk/bin/java PrintDefaultCharset >> Ja_JP.IBM-943 x-IBM943C IBM-943C IBM-943C >> $ LANG=Zh_TW ~/jdk/bin/java PrintDefaultCharset >> Zh_TW x-IBM950 IBM-950 IBM-950 >> $ LANG=Zh_TW.big5 ~/jdk/bin/java PrintDefaultCharset >> Zh_TW.big5 x-IBM950 IBM-950 IBM-950 >> >> Also I reviewed source code, it's fine >> >> Since this testing requires locale installation for Ja_JP and Zh_TW, >> so it's not easy to test it... >> (At least, I think bos.loc.pc.Ja_JP and bos.loc.iso.Zh_TW filesets are >> required) >> >> >> On 2018-05-02 18:32, Volker Simonis wrote: >>> >>> Hi Bhaktavatsal Reddy, >>> >>> your change looks good. I can sponsor it. >>> >>> Just waiting for a second review... >>> >>> Thank you and best regards, >>> Volker >>> >>> >>> On Mon, Apr 30, 2018 at 11:29 AM, Bhaktavatsal R Maram >>> wrote: >>>> >>>> Hi All, >>>> >>>> Please review the fix. >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202329 >>>> webrev: http://cr.openjdk.java.net/~aleonard/8202329/webrev.00/ >>>> >>>> Thanks, >>>> Bhaktavatsal Reddy >>>> >>>> -----"core-libs-dev" wrote: >>>> ----- >>>> To: Volker Simonis >>>> From: "Bhaktavatsal R Maram" >>>> Sent by: "core-libs-dev" >>>> Date: 04/26/2018 09:31PM >>>> Cc: Java Core Libs >>>> Subject: Re: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>> >>>> Hi Volker, >>>> >>>> Thank you. I will address your review comments and send webrev for >>>> review. >>>> >>>> - Bhaktavatsal Reddy >>>> >>>> >>>> >>>> -----Volker Simonis wrote: ----- >>>> To: Bhaktavatsal R Maram >>>> From: Volker Simonis >>>> Date: 04/26/2018 09:12PM >>>> Cc: Java Core Libs >>>> Subject: Re: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>> >>>> Hi Bhaktavatsal Reddy, >>>> >>>> I've opened the following issue for this problem: >>>> >>>> 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8202329 >>>> >>>> Looking at you fix, can you please replace the "#elif AIX" by "#ifdef >>>> AIX" and the original "#else" by "#ifdef __solaris__". The original >>>> else branch contains Solaris-only code anyway and it is an historical >>>> omission that there are still a lot of places in the code where "not >>>> Linux" implicitly means "Solaris", but that's often wrong. >>>> >>>> Regards, >>>> Volker >>>> >>>> >>>> On Thu, Apr 26, 2018 at 4:02 PM, Bhaktavatsal R Maram >>>> wrote: >>>>> >>>>> Oops! Looks like there is problem with attachment (might be because I >>>>> attached .class file as well). I'm pasting the fix and test program here in >>>>> mail. >>>>> >>>>> Test Program: >>>>> >>>>> import java.nio.charset.*; >>>>> class PrintDefaultCharset { >>>>> public static void main(String[] args) { >>>>> System.out.println("LANG = "+System.getenv("LANG")); >>>>> System.out.println("Default charset = >>>>> "+Charset.defaultCharset().name()); >>>>> System.out.println("file.encoding = >>>>> "+System.getProperty("file.encoding")); >>>>> System.out.println("sun.jnu.encoding = >>>>> "+System.getProperty("sun.jnu.encoding")); >>>>> } >>>>> } >>>>> >>>>> >>>>> Fix: >>>>> >>>>> diff --git a/src/java.base/unix/native/libjava/java_props_md.c >>>>> b/src/java.base/unix/native/libjava/java_props_md.c >>>>> --- a/src/java.base/unix/native/libjava/java_props_md.c >>>>> +++ b/src/java.base/unix/native/libjava/java_props_md.c >>>>> @@ -1,5 +1,5 @@ >>>>> /* >>>>> - * Copyright (c) 1998, 2016, Oracle and/or its affiliates. All rights >>>>> reserved. >>>>> + * Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights >>>>> reserved. >>>>> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>>> * >>>>> * This code is free software; you can redistribute it and/or modify it >>>>> @@ -297,6 +297,18 @@ >>>>> if (strcmp(p, "EUC-JP") == 0) { >>>>> *std_encoding = "EUC-JP-LINUX"; >>>>> } >>>>> +#elif defined _AIX >>>>> + if (strcmp(p, "big5") == 0) { >>>>> + /* On AIX Traditional Chinese Big5 codeset is mapped to >>>>> IBM-950 */ >>>>> + *std_encoding = "IBM-950"; >>>>> + } else if (strcmp(p, "IBM-943") == 0) { >>>>> + /* >>>>> + * On AIX, IBM-943 is mapped to IBM-943C in which symbol >>>>> 'yen' and >>>>> + * 'overline' are replaced with 'backslash' and 'tilde' >>>>> from ASCII >>>>> + * making first 96 code points same as ASCII. >>>>> + */ >>>>> + *std_encoding = "IBM-943C"; >>>>> + } >>>>> #else >>>>> if (strcmp(p,"eucJP") == 0) { >>>>> /* For Solaris use customized vendor defined character >>>>> >>>>> >>>>> Thanks, >>>>> Bhaktavatsal Reddy >>>>> >>>>> >>>>> -----"core-libs-dev" wrote: >>>>> ----- >>>>> To: "Java Core Libs" >>>>> From: "Bhaktavatsal R Maram" >>>>> Sent by: "core-libs-dev" >>>>> Date: 04/26/2018 07:26PM >>>>> Subject: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>>> >>>>> Hi All, >>>>> >>>>> This issue is continuation to bug 8201540 (Extend the set of supported >>>>> charsets in java.base on AIX) in which we have moved default charsets of >>>>> most of the locales supported by Operating System to java.base module thus >>>>> enabling OpenJDK on those locales for AIX platform. >>>>> >>>>> As part of that, charsets for locales Ja_JP (IBM-943) and Zh_TW (big5) >>>>> also have been moved. However, corresponding charsets mapped in Java is not >>>>> correct for them on AIX. Following are the details: >>>>> >>>>> 1. IBM-943 [1] for locale Ja_JP should be mapped to IBM-943C [2] >>>>> >>>>> Fundamental difference between IBM-943 and IBM-943C is that IBM-943C is >>>>> ASCII compatible which means code points 'yen' and 'overline' of IBM-943 is >>>>> replaced with 'backslash' and 'tilde' from ASCII character set. >>>>> >>>>> >>>>> 2. Big5 for locale Zh_TW should be mapped to IBM-950 [3] >>>>> >>>>> I've attached simple test program to print the default charset along >>>>> with fix for this issue. When run test program (PrintDefaultCharset) with >>>>> IBM JDK 8 (on AIX) for locales Ja_JP & Zh_TW, following is output. >>>>> >>>>> -bash-4.4$ LANG=Ja_JP ~/JDKs/IBM/80/ON/sdk/jre/bin/java >>>>> PrintDefaultCharset >>>>> LANG = Ja_JP >>>>> Default charset = x-IBM943C >>>>> file.encoding = IBM-943C >>>>> sun.jnu.encoding = IBM-943C >>>>> >>>>> -bash-4.4$ LANG=Zh_TW ~/JDKs/IBM/80/ON/sdk/jre/bin/java >>>>> PrintDefaultCharset >>>>> LANG = Zh_TW >>>>> Default charset = x-IBM950 >>>>> file.encoding = IBM-950 >>>>> sun.jnu.encoding = IBM-950 >>>>> >>>>> >>>>> Same test run with openJDK 11 gives following output >>>>> >>>>> -bash-4.4$ LANG=Ja_JP ~/jdk/bin/java PrintDefaultCharset >>>>> LANG = Ja_JP >>>>> Default charset = x-IBM943 >>>>> file.encoding = IBM-943 >>>>> sun.jnu.encoding = IBM-943 >>>>> >>>>> -bash-4.4$ LANG=Zh_TW ~/jdk/bin/java PrintDefaultCharset >>>>> LANG = Zh_TW >>>>> Default charset = Big5 >>>>> file.encoding = big5 >>>>> sun.jnu.encoding = big5 >>>>> >>>>> I will get webrev hosted in >>>>> http://cr.openjdk.java.net >>>>> for this change and send it for review once JIRA bug is created. >>>>> >>>>> [1] >>>>> http://demo.icu-project.org/icu-bin/convexp?conv=ibm-943_P130-1999&s=JAVA >>>>> [2] >>>>> http://demo.icu-project.org/icu-bin/convexp?conv=ibm-943_P15A-2003&s=ALL >>>>> [3] >>>>> https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.nlsgdrf/big5.htm >>>>> >>>>> >>>>> Thanks, >>>>> Bhaktavatsal Reddy >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >> > > From christoph.langer at sap.com Fri Jun 1 08:56:34 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 1 Jun 2018 08:56:34 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <0c73caec8810b9844a463343bdaec064@linux.vnet.ibm.com> References: <0c73caec8810b9844a463343bdaec064@linux.vnet.ibm.com> Message-ID: Hi Ichiroh, we do not use the XLC 13 compiler on AIX yet here at SAP and I believe nobody of my colleagues has played with it yet. So you are on a new playground here ?? However, I believe the idea in OpenJDK with the abolition of map files is that symbols should be invisible externally unless they are declared exported, e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the correct default and whatever JNIEXPORT expands to should contain the right attributes to get that symbol visible. Can you check if either my assumption is completely wrong, JNIEXPORT does not expand to the right thing, XLC 13 has a bug or maybe just sume specific required symbols are not declared correctly? Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Donnerstag, 31. Mai 2018 09:55 > To: Langer, Christoph > Cc: Baesken, Matthias ; 'build- > dev at openjdk.java.net' ; ppc-aix-port- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > Goetz > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hello. > 8202322 was integrated into jdk-11+15. > I'm using XLC 13.1.3 on AIX 7.1.4. > Build was failed because of "-qvisibility=hidden" on > make/lib/LibCommon.gmk. > According to "XL C/C++ for AIX 13.1.3" documentation [1], > "-qvisibility=hidden" cannot create shared libraries entry points. > For example, libverify.so was there, but entry points were not resolved > by "-lverify" option. > I think it should be "-qvisibility=default" (I tried, it worked) > or "-qvisibility=protected" (I had not tried) ? > I'm not familiar with -qvisibility option, but I'd like to find out > right way. > > [1] > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > > On 2018-05-16 16:08, Langer, Christoph wrote: > > Hi Matthias, > > > > yes, reviewed. > > > > Best regards > > Christoph > > > > From: Baesken, Matthias > > Sent: Mittwoch, 16. Mai 2018 09:06 > > To: Langer, Christoph ; > > 'build-dev at openjdk.java.net' ; > > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > > Cc: Lindenmaier, Goetz > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > xlc 12.1 > > > > Hi Christoph can I add you as second reviewer (other reviewer was > > Erik Joelsson) can push the change ? > > > > Best regards, Matthias > > > > > > > > From: Langer, Christoph > > Sent: Donnerstag, 26. April 2018 16:38 > > To: Baesken, Matthias > > >; > > 'build-dev at openjdk.java.net' > > >; > > ppc-aix-port-dev at openjdk.java.net dev at openjdk.java.net>; > > core-libs-dev at openjdk.java.net > > Cc: Simonis, Volker > > > > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > xlc 12.1 > > > > Hi Matthias, > > > > to me the change in principal looks good. > > > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > > extract major number before the first dot, then compare numerically) - > > but maybe it is too complicated and the current single version compare > > suits our needs ? > > > > Best regards > > Christoph > > > > From: Baesken, Matthias > > Sent: Donnerstag, 26. April 2018 16:14 > > To: 'build-dev at openjdk.java.net' > > >; > > ppc-aix-port-dev at openjdk.java.net dev at openjdk.java.net>; > > core-libs-dev at openjdk.java.net > > Cc: Langer, Christoph > > >; Simonis, > > Volker > > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > > 12.1 > > > > Hello , could you please review this small adjustment to the symbol > > visibility compilation settings on AIX ? > > Currently we use XLC 12.1 to compile JDK on AIX . > > > > However XLC 12.1 does not support the "-qvisibility=hidden" > > setting currently set on AIX. > > It was introduced with XLC 13.1 . Christoph found some info about it > > here : > > > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > visibility-part2/index.html > > > > Setting it only generates hundreds of warnings in the build log , > > warnings look like this : > > XlC12.1 > > > > bash-4.4$ xlC -qversion > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > > Version: 12.01.0000.0019 > > > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > > of valid options. > > > > Compare to XLC13.1 > > > > bash-3.00$ xlC -qversion > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > > Version: 13.01.0000.0008 > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > > > > > So it is better to avoid setting these flags when using xlc12.1 . > > Please review : > > > > Bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8202322 > > > > Change : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > > > > > > Best regards, Matthias From matthias.baesken at sap.com Fri Jun 1 12:12:38 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 1 Jun 2018 12:12:38 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <0c73caec8810b9844a463343bdaec064@linux.vnet.ibm.com> Message-ID: <81453d0cbe05477ea558917de263ada2@sap.com> Hi , my change 8202322 just handled the fact that the visibility - flags are not supported with xlc 12.1 , so setting them generated a TON of compile - time warnings . The introduction of the "-qvisibility=hidden" came with the mapfile removal changes : 8200358: Remove mapfiles for JDK executables http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 8200178: Remove mapfiles for JDK native libraries http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 I guess it might need further testing+adjustments to make the "visibility hiding" work nicely with XLC13 , but currently we build only with XLC12 . As a workaround you might want to remove the "-qvisibility=hidden" setting for XLC 13 as well , like I did for XLC12 with the change 8202322 . Best regards, Matthias > -----Original Message----- > From: Langer, Christoph > Sent: Freitag, 1. Juni 2018 10:57 > To: Ichiroh Takiguchi > Cc: Baesken, Matthias ; 'build- > dev at openjdk.java.net' ; ppc-aix-port- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > Goetz > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hi Ichiroh, > > we do not use the XLC 13 compiler on AIX yet here at SAP and I believe > nobody of my colleagues has played with it yet. So you are on a new > playground here ?? > > However, I believe the idea in OpenJDK with the abolition of map files is that > symbols should be invisible externally unless they are declared exported, > e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the correct > default and whatever JNIEXPORT expands to should contain the right > attributes to get that symbol visible. > > Can you check if either my assumption is completely wrong, JNIEXPORT does > not expand to the right thing, XLC 13 has a bug or maybe just sume specific > required symbols are not declared correctly? > > Best regards > Christoph > > > -----Original Message----- > > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > > Sent: Donnerstag, 31. Mai 2018 09:55 > > To: Langer, Christoph > > Cc: Baesken, Matthias ; 'build- > > dev at openjdk.java.net' ; ppc-aix-port- > > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > > Goetz > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > > > Hello. > > 8202322 was integrated into jdk-11+15. > > I'm using XLC 13.1.3 on AIX 7.1.4. > > Build was failed because of "-qvisibility=hidden" on > > make/lib/LibCommon.gmk. > > According to "XL C/C++ for AIX 13.1.3" documentation [1], > > "-qvisibility=hidden" cannot create shared libraries entry points. > > For example, libverify.so was there, but entry points were not resolved > > by "-lverify" option. > > I think it should be "-qvisibility=default" (I tried, it worked) > > or "-qvisibility=protected" (I had not tried) ? > > I'm not familiar with -qvisibility option, but I'd like to find out > > right way. > > > > [1] > > > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > > > > On 2018-05-16 16:08, Langer, Christoph wrote: > > > Hi Matthias, > > > > > > yes, reviewed. > > > > > > Best regards > > > Christoph > > > > > > From: Baesken, Matthias > > > Sent: Mittwoch, 16. Mai 2018 09:06 > > > To: Langer, Christoph ; > > > 'build-dev at openjdk.java.net' ; > > > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > > > Cc: Lindenmaier, Goetz > > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > > xlc 12.1 > > > > > > Hi Christoph can I add you as second reviewer (other reviewer was > > > Erik Joelsson) can push the change ? > > > > > > Best regards, Matthias > > > > > > > > > > > > From: Langer, Christoph > > > Sent: Donnerstag, 26. April 2018 16:38 > > > To: Baesken, Matthias > > > >; > > > 'build-dev at openjdk.java.net' > > > >; > > > ppc-aix-port-dev at openjdk.java.net > dev at openjdk.java.net>; > > > core-libs-dev at openjdk.java.net dev at openjdk.java.net> > > > Cc: Simonis, Volker > > > > > > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > > xlc 12.1 > > > > > > Hi Matthias, > > > > > > to me the change in principal looks good. > > > > > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > > > extract major number before the first dot, then compare numerically) - > > > but maybe it is too complicated and the current single version compare > > > suits our needs ? > > > > > > Best regards > > > Christoph > > > > > > From: Baesken, Matthias > > > Sent: Donnerstag, 26. April 2018 16:14 > > > To: 'build-dev at openjdk.java.net' > > > >; > > > ppc-aix-port-dev at openjdk.java.net > dev at openjdk.java.net>; > > > core-libs-dev at openjdk.java.net dev at openjdk.java.net> > > > Cc: Langer, Christoph > > > >; > Simonis, > > > Volker > > > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > > > 12.1 > > > > > > Hello , could you please review this small adjustment to the symbol > > > visibility compilation settings on AIX ? > > > Currently we use XLC 12.1 to compile JDK on AIX . > > > > > > However XLC 12.1 does not support the "-qvisibility=hidden" > > > setting currently set on AIX. > > > It was introduced with XLC 13.1 . Christoph found some info about it > > > here : > > > > > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > > visibility-part2/index.html > > > > > > Setting it only generates hundreds of warnings in the build log , > > > warnings look like this : > > > XlC12.1 > > > > > > bash-4.4$ xlC -qversion > > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > > > Version: 12.01.0000.0019 > > > > > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > > > of valid options. > > > > > > Compare to XLC13.1 > > > > > > bash-3.00$ xlC -qversion > > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > > > Version: 13.01.0000.0008 > > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > > > > > > > > So it is better to avoid setting these flags when using xlc12.1 . > > > Please review : > > > > > > Bug : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8202322 > > > > > > Change : > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > > > > > > > > > Best regards, Matthias From bob.vandette at oracle.com Fri Jun 1 15:52:54 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 1 Jun 2018 11:52:54 -0400 Subject: RFR: 8203357 Container Metrics In-Reply-To: <6b8f5228-03ea-9521-aa66-e74b5595722b@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <6b8f5228-03ea-9521-aa66-e74b5595722b@oracle.com> Message-ID: <57ADBBD0-381B-4890-9943-BF3222F19839@oracle.com> > On May 31, 2018, at 11:36 PM, mandy chung wrote: > > Hi Bob, > > On 5/30/18 12:45 PM, Bob Vandette wrote:> >> RFE: Container Metrics >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> WEBREV: >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00 Thanks for the review, here an updated webrev: http://cr.openjdk.java.net/~bobv/8203357/webrev.01/ > > Looks fine in general. It's good to have this internal API ready > for JFR and other library code to use. > > I skimmed through the new tests. It'd be good to add some comments > to describe what it does (for example, set up a docker image etc). DockerTestUtils.java does contain some comments describing what the utility functions do. I?ll add a brief comment in TestDockerCpuMetrics.java and TestDockerMemoryMetrics.java explaining the test process. > > launcher.properties > 154 \ -XshowSettings:system (Linux Only) show host system or container\n\ > 155 \ configuration and continue\n\ > > A newline can be placed after -XshowSettings:system consistent with > other suboptions. Done. I also added a newline after the vm sub-option. > > test/lib/jdk/test/lib/containers/docker/DockerTestUtils.java > > There are several long lines in the new test files such as: > MetricsCpuTester.java > MetricsMemoryTester.java > MetricsTester.java > > It'd help future side-by-side review if they are wrapped. I think > most of them are the construction of an exception. Fixed. > > I see a pattern of a name after @test and that is not strictly needed. > > TestCgroupMetrics.java > 25 * @test TestCgroupMetrics > > TestDockerCpuMetrics.java > 34 * @test TestSystemMetrics > > TestDockerMemoryMetrics.java > 30 * @test TestSystemMetrics > > TestSystemMetrics.java > 25 * @test TestSystemMetrics Remove the names after @test. > > This needs a CSR for the new -XshowSettings:system option. I filed a CSR for this a few days ago. https://bugs.openjdk.java.net/browse/JDK-8204107 Bob. > > Mandy From mandy.chung at oracle.com Fri Jun 1 20:00:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Jun 2018 13:00:22 -0700 Subject: RFR: 8203357 Container Metrics In-Reply-To: <57ADBBD0-381B-4890-9943-BF3222F19839@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <6b8f5228-03ea-9521-aa66-e74b5595722b@oracle.com> <57ADBBD0-381B-4890-9943-BF3222F19839@oracle.com> Message-ID: <548859ab-db3a-d028-0c92-d6598c722052@oracle.com> On 6/1/18 8:52 AM, Bob Vandette wrote: > I filed a CSR for this a few days ago. > > https://bugs.openjdk.java.net/browse/JDK-8204107 Typo: s/-XshowSetting/-XshowSettings In the specification section, you can include the new lines adding to java --extra-help output (that's the spec) and drop the diff. It may worth to state that the output is implementation detail. It may help to clarify the behavior when running on non-linux platform, i.e. the current launcher implementation accepts any suboption value. So java -XshowSettings:system on non-linux platform displays vm, properties, locale settings. (BTW, I file JDK-8204246 about invalid suboption). Mandy From stuart.marks at oracle.com Fri Jun 1 21:16:35 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 1 Jun 2018 14:16:35 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) Message-ID: Hi all, Please review this changeset to remove the Thread.destroy() and Thread.stop(Throwable) methods. Both of these methods have been deprecated for a long time, and they were deprecated for removal in Java SE 9. Thread.destroy() was never implemented, and has always thrown NoSuchMethodError back to the caller. Thread.stop(Throwable) was made non-functional in JDK 8, throwing UnsupportedOperationException back to the caller. Note that the no-arg Thread.stop() method remains deprecated, not for removal, and is unaffected by this changeset. Note also that ThreadGroup.destroy() doesn't actually destroy any threads -- just thread groups -- and that it is also unaffected by this changeset. Bug: https://bugs.openjdk.java.net/browse/JDK-8204243 Webrev: http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.0/ Thanks, s'marks aka @DrDeprecator From lance.andersen at oracle.com Fri Jun 1 21:37:45 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 1 Jun 2018 17:37:45 -0400 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: Message-ID: <2B7D7000-FAE5-4276-A36D-AEDFCD987A6B@oracle.com> Hi Stuart, The changes look fine. Is there a CSR or a release note that also needs to be reviewed? Best Lance > On Jun 1, 2018, at 5:16 PM, Stuart Marks wrote: > > Hi all, > > Please review this changeset to remove the Thread.destroy() and Thread.stop(Throwable) methods. Both of these methods have been deprecated for a long time, and they were deprecated for removal in Java SE 9. > > Thread.destroy() was never implemented, and has always thrown NoSuchMethodError back to the caller. Thread.stop(Throwable) was made non-functional in JDK 8, throwing UnsupportedOperationException back to the caller. > > Note that the no-arg Thread.stop() method remains deprecated, not for removal, and is unaffected by this changeset. > > Note also that ThreadGroup.destroy() doesn't actually destroy any threads -- just thread groups -- and that it is also unaffected by this changeset. > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8204243 > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.0/ > > Thanks, > > s'marks > aka @DrDeprecator Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From ecki at zusammenkunft.net Fri Jun 1 22:24:38 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Sat, 2 Jun 2018 00:24:38 +0200 Subject: jlink / jmods version compatibiltiy Message-ID: <5b11c7a6.1c69fb81.e9246.a5a0@mx.google.com> Hello, I am not sure what the Policy for backward/Forward compatibility for JMOD files is, but when I use JDK-9.0.4 jlink on 11ea JMODs I get a IllegalArgumentException and ?error reading? by JDK-10.0.1 with no further Details. If the JMOD files require newer jlink, should they have a Version identifier to allow rejecting this. Or is it expected to normally work? C:\ >"c:\Program Files\Java\jdk-9.0.4\bin\jlink" --verbose --module-path "jdk-11-oraopenjdk\jmods" --output o1 --add-modules=java.base,jdk.crypto.mscapi,jdk.jartool,jdk.jcmd,jdk.jshell,jdk.naming.dns --strip-debug --compress 2 --no-header-files ? Error: java.lang.IllegalArgumentException C:\>"c:\Program Files\Java\jdk-10.0.1\bin\jlink" --verbose --module-path "jdk-11-oraopenjdk\jmods" --output o2 --add-modules=java.base,jdk.crypto.mscapi,jdk.jartool,jdk.jcmd,jdk.jshell,jdk.naming.dns --strip-debug --compress 2 --no-header-files Error: Error reading module: jdk-11-oraopenjdk\jmods\java.base.jmod C:\>"c:\Program Files\Java\jdk-11\bin\jlink" --verbose --module-path "jdk-11-oraopenjdk\jmods" --output o3 --add-modules=java.base,jdk.crypto.mscapi,jdk.jartool,jdk.jcmd,jdk.jshell,jdk.naming.dns --strip-debug --compress 2 --no-header-files ? Providers: ? Gruss Bernd -- http://bernd.eckenfels.net From mandy.chung at oracle.com Fri Jun 1 22:58:09 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Jun 2018 15:58:09 -0700 Subject: jlink / jmods version compatibiltiy In-Reply-To: <5b11c7a6.1c69fb81.e9246.a5a0@mx.google.com> References: <5b11c7a6.1c69fb81.e9246.a5a0@mx.google.com> Message-ID: <42883c05-89a3-2caa-dd14-3dc9dee7fcfd@oracle.com> On 6/1/18 3:24 PM, Bernd Eckenfels wrote: > Hello, > > I am not sure what the Policy for backward/Forward compatibility for > JMOD files is, but when I use JDK-9.0.4 jlink on 11ea JMODs I get a > IllegalArgumentException and ?error reading? by JDK-10.0.1 with no > further Details. jlink only supports linking modules of the same runtime version as jlink because jlink has a few plugins such as system-modules and generate-jli-classes that are deeply tied with java.base. It can be enhanced in the future in particular for plugins to support version <= jlink version. See JDK-8185130 that was integrated in JDK 10. jlink linking with older version of JMODS outputs a clear error message: $ jdk11/bin/jlink --module-path jdk10/jmods --output test-image --add-modules java.base Error: jlink version 11.0 does not match target java.base version 10.0 > If the JMOD files require newer jlink, should they have a Version > identifier to allow rejecting this. Or is it expected to normally > work? : > Error reading module: jdk-11-oraopenjdk\jmods\java.base.jmod JDK 10 jlink does not support class file version 55 when linking with JDK 11 modules. I agree the error message should be improved. I created https://bugs.openjdk.java.net/browse/JDK-8204256. Mandy From david.holmes at oracle.com Fri Jun 1 23:40:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 2 Jun 2018 09:40:56 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: Message-ID: Hi Stuart! Nooooooo! Just kidding! Yaaaaaay!!!! Actual removal looks fine but what about the corresponding JDI code: ./jdk.jdi/share/classes/com/sun/jdi/ThreadReference.java it still has a stop(Throwable) function (it doesn't have the no-arg one!). What happens to it? The JDI functions were never deprecated in line with the Thread functions (suspend/resume/stop). The ThreadReference javadoc has: * @see java.lang.Thread#stop(Throwable) so will need fixing. Thanks, David On 2/06/2018 7:16 AM, Stuart Marks wrote: > Hi all, > > Please review this changeset to remove the Thread.destroy() and > Thread.stop(Throwable) methods. Both of these methods have been > deprecated for a long time, and they were deprecated for removal in Java > SE 9. > > Thread.destroy() was never implemented, and has always thrown > NoSuchMethodError back to the caller. Thread.stop(Throwable) was made > non-functional in JDK 8, throwing UnsupportedOperationException back to > the caller. > > Note that the no-arg Thread.stop() method remains deprecated, not for > removal, and is unaffected by this changeset. > > Note also that ThreadGroup.destroy() doesn't actually destroy any > threads -- just thread groups -- and that it is also unaffected by this > changeset. > > Bug: > > ????https://bugs.openjdk.java.net/browse/JDK-8204243 > > Webrev: > > ????http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.0/ > > Thanks, > > s'marks > aka @DrDeprecator From stuart.marks at oracle.com Fri Jun 1 23:53:05 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 1 Jun 2018 16:53:05 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <2B7D7000-FAE5-4276-A36D-AEDFCD987A6B@oracle.com> References: <2B7D7000-FAE5-4276-A36D-AEDFCD987A6B@oracle.com> Message-ID: <1991283d-cb28-2b2e-af12-1ba08a244e8c@oracle.com> On 6/1/18 2:37 PM, Lance Andersen wrote: > The changes look fine. ? ?Is there a CSR or a release note that also needs to be > reviewed? Thanks. No CSR yet. I usually do those after the code review. Good point about a release note, though; I'll add the label and subtask for one. s'marks From david.holmes at oracle.com Sat Jun 2 00:15:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 2 Jun 2018 10:15:50 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <1991283d-cb28-2b2e-af12-1ba08a244e8c@oracle.com> References: <2B7D7000-FAE5-4276-A36D-AEDFCD987A6B@oracle.com> <1991283d-cb28-2b2e-af12-1ba08a244e8c@oracle.com> Message-ID: On 2/06/2018 9:53 AM, Stuart Marks wrote: > On 6/1/18 2:37 PM, Lance Andersen wrote: >> The changes look fine. ? ?Is there a CSR or a release note that also >> needs to be reviewed? > > Thanks. No CSR yet. I usually do those after the code review. Good point > about a release note, though; I'll add the label and subtask for one. I would expect the CSR that marked them as deprecated for removal, also serves for the actual removal. Certainly for VM flags we don't do a separate CSR for each phase (deprecation, obsoletion, expiration). David > s'marks > From stuart.marks at oracle.com Sat Jun 2 00:43:33 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 1 Jun 2018 17:43:33 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: Message-ID: On 6/1/18 4:40 PM, David Holmes wrote: > Hi Stuart! > > Nooooooo! > > Just kidding! > > Yaaaaaay!!!! :-) > Actual removal looks fine but what about the corresponding JDI code: > > ./jdk.jdi/share/classes/com/sun/jdi/ThreadReference.java > > it still has a stop(Throwable) function (it doesn't have the no-arg one!). What > happens to it? The JDI functions were never deprecated in line with the Thread > functions (suspend/resume/stop). > > The ThreadReference javadoc has: > > ?? * @see java.lang.Thread#stop(Throwable) > > so will need fixing. Good catch! I had grepped around the source tree for stuff like this but I had missed this one. It looks to me like this interface resides on the debugger side of JDWP. I don't know exactly where it's implemented in the target VM, but I imagine it goes through some VM-internal or JDWP implementation interface, not through the target VM's public java.lang.Thread.stop(thr) interface. Is that right? (I'm not that familiar with this area of the system.) I'm presuming that it doesn't go through j.l.Thread.stop(Throwable) since that method has essentially been non-functional since JDK 8. I see a couple tests for this in test/hotspot/jtreg/vmTestbase/nsk/jdi/ThreadReference/stop and I can see some evidence in our internal build/test systems that they're still being run, so it looks like these interfaces are still active. As you observe, they haven't been deprecated. There might even be valid uses cases for stopping a thread this way in a debugging situation that aren't valid for applications. I guess that means the thing to do is to update the documentation to remove the reference to the soon-to-be-deleted Thread.stop(Throwable) but otherwise leave this stuff unchanged. s'marks From stuart.marks at oracle.com Sat Jun 2 01:07:17 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 1 Jun 2018 18:07:17 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <2B7D7000-FAE5-4276-A36D-AEDFCD987A6B@oracle.com> <1991283d-cb28-2b2e-af12-1ba08a244e8c@oracle.com> Message-ID: <5e109773-12b9-6e32-6b73-aef48ec8d24f@oracle.com> On 6/1/18 5:15 PM, David Holmes wrote: > I would expect the CSR that marked them as deprecated for removal, also serves > for the actual removal. Certainly for VM flags we don't do a separate CSR for > each phase (deprecation, obsoletion, expiration). Hm. Well, this hasn't been tested for Java SE APIs yet, as most of the deprecations-for-removal occurred in Java SE 9, before the CSR was active. Instead, those deprecations went through the (Oracle internal) CCC process. Now that we're fully on the CSR system, I'd expect that deprecations (whether or not for removal) and removals of Java SE APIs would have separate CSR requests. The reason is that adding or changing a deprecation annotation is a spec change, and removing the API is a distinct spec change. They also occur in different releases. I can easily see that a different procedure would be followed for VM flags, though, since they aren't part of Java SE. s'marks From david.holmes at oracle.com Sat Jun 2 05:36:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 2 Jun 2018 15:36:42 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: Message-ID: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> On 2/06/2018 10:43 AM, Stuart Marks wrote: > On 6/1/18 4:40 PM, David Holmes wrote: >> Hi Stuart! >> >> Nooooooo! >> >> Just kidding! >> >> Yaaaaaay!!!! > > :-) > >> Actual removal looks fine but what about the corresponding JDI code: >> >> ./jdk.jdi/share/classes/com/sun/jdi/ThreadReference.java >> >> it still has a stop(Throwable) function (it doesn't have the no-arg >> one!). What happens to it? The JDI functions were never deprecated in >> line with the Thread functions (suspend/resume/stop). >> >> The ThreadReference javadoc has: >> >> ??? * @see java.lang.Thread#stop(Throwable) >> >> so will need fixing. > > Good catch! I had grepped around the source tree for stuff like this but > I had missed this one. > > It looks to me like this interface resides on the debugger side of JDWP. > I don't know exactly where it's implemented in the target VM, but I > imagine it goes through some VM-internal or JDWP implementation > interface, not through the target VM's public java.lang.Thread.stop(thr) > interface. Is that right? (I'm not that familiar with this area of the > system.) It's a rather complex setup but the backend that does the work is (usually?) JVM TI, which won't go back out through the Java API. > > I'm presuming that it doesn't go through j.l.Thread.stop(Throwable) > since that method has essentially been non-functional since JDK 8. > > I see a couple tests for this in > > ??? test/hotspot/jtreg/vmTestbase/nsk/jdi/ThreadReference/stop > > and I can see some evidence in our internal build/test systems that > they're still being run, so it looks like these interfaces are still > active. As you observe, they haven't been deprecated. There might even > be valid uses cases for stopping a thread this way in a debugging > situation that aren't valid for applications. I'm yet to find one :) I pointed out this inconsistency many many years ago but it remains. > I guess that means the thing to do is to update the documentation to > remove the reference to the soon-to-be-deleted Thread.stop(Throwable) > but otherwise leave this stuff unchanged. Yep that's what I thought too. Thanks, David > s'marks From david.holmes at oracle.com Sat Jun 2 05:45:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 2 Jun 2018 15:45:18 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <5e109773-12b9-6e32-6b73-aef48ec8d24f@oracle.com> References: <2B7D7000-FAE5-4276-A36D-AEDFCD987A6B@oracle.com> <1991283d-cb28-2b2e-af12-1ba08a244e8c@oracle.com> <5e109773-12b9-6e32-6b73-aef48ec8d24f@oracle.com> Message-ID: <1f5ed1d9-2a4b-c693-a46f-90186aeaec4b@oracle.com> On 2/06/2018 11:07 AM, Stuart Marks wrote: > On 6/1/18 5:15 PM, David Holmes wrote: >> I would expect the CSR that marked them as deprecated for removal, >> also serves for the actual removal. Certainly for VM flags we don't do >> a separate CSR for each phase (deprecation, obsoletion, expiration). > > Hm. Well, this hasn't been tested for Java SE APIs yet, as most of the > deprecations-for-removal occurred in Java SE 9, before the CSR was > active. Instead, those deprecations went through the (Oracle internal) > CCC process. > > Now that we're fully on the CSR system, I'd expect that deprecations > (whether or not for removal) and removals of Java SE APIs would have > separate CSR requests. The reason is that adding or changing a > deprecation annotation is a spec change, and removing the API is a > distinct spec change. They also occur in different releases. Good points. Though given the annotation is on the method being removed it's really only one spec change. Cheers, David > I can easily see that a different procedure would be followed for VM > flags, though, since they aren't part of Java SE. > > s'marks From Alan.Bateman at oracle.com Sat Jun 2 06:20:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 2 Jun 2018 07:20:14 +0100 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <1f5ed1d9-2a4b-c693-a46f-90186aeaec4b@oracle.com> References: <2B7D7000-FAE5-4276-A36D-AEDFCD987A6B@oracle.com> <1991283d-cb28-2b2e-af12-1ba08a244e8c@oracle.com> <5e109773-12b9-6e32-6b73-aef48ec8d24f@oracle.com> <1f5ed1d9-2a4b-c693-a46f-90186aeaec4b@oracle.com> Message-ID: <23829a3f-fb12-b1d8-8871-9818b8618c00@oracle.com> On 02/06/2018 06:45, David Holmes wrote: > On 2/06/2018 11:07 AM, Stuart Marks wrote: >> On 6/1/18 5:15 PM, David Holmes wrote: >>> I would expect the CSR that marked them as deprecated for removal, >>> also serves for the actual removal. Certainly for VM flags we don't >>> do a separate CSR for each phase (deprecation, obsoletion, expiration). >> >> Hm. Well, this hasn't been tested for Java SE APIs yet, as most of >> the deprecations-for-removal occurred in Java SE 9, before the CSR >> was active. Instead, those deprecations went through the (Oracle >> internal) CCC process. >> >> Now that we're fully on the CSR system, I'd expect that deprecations >> (whether or not for removal) and removals of Java SE APIs would have >> separate CSR requests. The reason is that adding or changing a >> deprecation annotation is a spec change, and removing the API is a >> distinct spec change. They also occur in different releases. > > Good points. Though given the annotation is on the method being > removed it's really only one spec change. Adding @Deprecated(forRemoval=true) in one release and then removing the element in a later release has to be tracked as two spec changes, and thus two CSRs. There may be more steps of course. In the case of Thread.stop(Throwable) under discussion here, and several SecurityManager methods, the methods were "degraded" (to always throw an exception or only check AllPermission in the case of the SM methods) before removal. So that's another intermediate spec change that has to be tracked via the CSR. -Alan From Alan.Bateman at oracle.com Sat Jun 2 06:38:34 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 2 Jun 2018 07:38:34 +0100 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> Message-ID: On 02/06/2018 06:36, David Holmes wrote: > On 2/06/2018 10:43 AM, Stuart Marks wrote: >> : >> >> It looks to me like this interface resides on the debugger side of >> JDWP. I don't know exactly where it's implemented in the target VM, >> but I imagine it goes through some VM-internal or JDWP implementation >> interface, not through the target VM's public >> java.lang.Thread.stop(thr) interface. Is that right? (I'm not that >> familiar with this area of the system.) > > It's a rather complex setup but the backend that does the work is > (usually?) JVM TI, which won't go back out through the Java API. Right, there's a StopThread function in JVM TI and a corresponding Stop Command in the JDWP protocol. The JVM TI spec has a statement to say that it works "similar to Thread.stop" and the JDWP spec has "as if done by Thread.stop" so it would be good to update both of these. -Alan From david.holmes at oracle.com Sun Jun 3 12:11:11 2018 From: david.holmes at oracle.com (David Holmes) Date: Sun, 3 Jun 2018 22:11:11 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> Message-ID: <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> On 2/06/2018 4:38 PM, Alan Bateman wrote: > On 02/06/2018 06:36, David Holmes wrote: >> On 2/06/2018 10:43 AM, Stuart Marks wrote: >>> : >>> >>> It looks to me like this interface resides on the debugger side of >>> JDWP. I don't know exactly where it's implemented in the target VM, >>> but I imagine it goes through some VM-internal or JDWP implementation >>> interface, not through the target VM's public >>> java.lang.Thread.stop(thr) interface. Is that right? (I'm not that >>> familiar with this area of the system.) >> >> It's a rather complex setup but the backend that does the work is >> (usually?) JVM TI, which won't go back out through the Java API. > Right, there's a StopThread function in JVM TI and a corresponding Stop > Command in the JDWP protocol. The JVM TI spec has a statement to say > that it works "similar to Thread.stop" and the JDWP spec has "as if done > by Thread.stop" so it would be good to update both of these. Any suggestions as to how to do that in a practical and effective way? "As if done by the highly-dangerous, long-deprecated and finally removed Thread.stop(Throwable)" ? ;-) In all seriousness I hate to do anything that might suggest these are valid API's even for tools. Maybe we should deprecate them as well (separate RFE of course). David > -Alan From Alan.Bateman at oracle.com Sun Jun 3 15:55:43 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 3 Jun 2018 16:55:43 +0100 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> Message-ID: <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> On 03/06/2018 13:11, David Holmes wrote: > > Any suggestions as to how to do that in a practical and effective way? > > "As if done by the highly-dangerous, long-deprecated and finally > removed Thread.stop(Throwable)" > > ? ;-) > > In all seriousness I hate to do anything that might suggest these are > valid API's even for tools. Maybe we should deprecate them as well > (separate RFE of course). These date back to JPDA in JDK 1.2 (it was JVMDI at the time, pre-dates JVM TI) and would need research to see if there are debuggers or maybe fault injection tools are using it. For Stuart's current effort then it will need a bit of text, maybe borrowing old spec where the operation was specified to "stop" a thread with an asynchronous exception. Warnings about the dangers are probably appropriate but these are of course interfaces that developers are unlikely to ever use directly. -Alan From vicente.romero at oracle.com Sun Jun 3 16:55:29 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Sun, 3 Jun 2018 12:55:29 -0400 Subject: RFR: implementation for JEP 334: JVM Constants API In-Reply-To: <2f058178-862b-ab26-39e0-4594f387c625@oracle.com> References: <799c736f-ad0b-6373-f7e4-6acafe2a8486@oracle.com> <21d1eab3-51e2-b647-5c4f-e686c2e573e4@oracle.com> <2f058178-862b-ab26-39e0-4594f387c625@oracle.com> Message-ID: Hi all, There is a new iteration of the implementation at: [1] http://cr.openjdk.java.net/~vromero/constant.api/webrev.10 [2] http://cr.openjdk.java.net/~vromero/constant.api/javadoc.10 main changes in this iteration: we have added additional tests to improve code coverage. Thanks, Vicente On 05/31/2018 05:09 PM, Vicente Romero wrote: > Hi all, > > I have uploaded another iteration of the implementation of the > constants API + the javadoc. > > Thanks for the comments so far, > Vicente > > > [1] http://cr.openjdk.java.net/~vromero/constant.api/webrev.09 > [2] http://cr.openjdk.java.net/~vromero/constant.api/javadoc.09 > > On 05/24/2018 09:09 PM, Vicente Romero wrote: >> Thanks for the comments so far, I have uploaded another iteration of >> the implementation + javadoc >> >> Vicente >> >> [1] http://cr.openjdk.java.net/~vromero/constant.api/webrev.08 >> [2] http://cr.openjdk.java.net/~vromero/constant.api/javadoc.08 >> >> >> On 05/23/2018 02:41 PM, Vicente Romero wrote: >>> Hi all, >>> >>> Please review the proposed implementation for JEP 334 [1]. The >>> webrev can be found at [2] and the javadoc at [3]. >>> >>> Thanks, >>> Vicente >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8203252 >>> [2] >>> http://cr.openjdk.java.net/~vromero/constant.api/webrev.07/constants.api.patch >>> [3] >>> http://cr.openjdk.java.net/~vromero/constant.api/javadoc.07/overview-summary.html >>> >>> PS. We are offering a MacBook Wheel to the authors of the first 5 >>> comments :) >> > From lance.andersen at oracle.com Mon Jun 4 11:22:03 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 4 Jun 2018 07:22:03 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html Message-ID: Hi, Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html As part of this fix, I took the liberty to move from package.html to package-info.java The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From deepak.kejriwal at oracle.com Mon Jun 4 11:46:23 2018 From: deepak.kejriwal at oracle.com (Deepak Kejriwal) Date: Mon, 4 Jun 2018 04:46:23 -0700 (PDT) Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries Message-ID: Hi all, Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/browse/JDK-8186171 Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.00/ Summary(also added to backport bug description): The back port for test files is not clean back port as all tests files are extending JSR166TestCase.java which was added to JDK 9 as part of HYPERLINK "https://bugs.openjdk.java.net/browse/JDK-8146467"JDK-8146467: Integrate JSR 166 jck tests into JDK repo. This is not present in JDK8 version. Therefore, I have extracted test are relevant to the fix done JDK-8186171 and created a new test file Bug8186171Test.java that contains below two test methods: . testBug8186171NonDeterministic : This method is copy of "testBug8186171" present in "MapTest.java" of original changeset http://hg.openjdk.java.net/jdk10/master/rev/3f5f9bc0bdc2. As it is based on randomization and runs 1000 times, the method name is suffixed with "NonDeterministic". . testBug8186171Deterministic : This is a new method that runs single time as it produces the exact scenario mentioned in JDK-8186171. Therefore, it is not needed to run multiple times to produce the scenario mentioned in bug. Hence the method name is suffixed with "Deterministic" Regards, Deepak From martijnverburg at gmail.com Mon Jun 4 13:09:06 2018 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 4 Jun 2018 14:09:06 +0100 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> Message-ID: Hi Kevin, Looks good, I assume as part of the work several existing javapackager bugs will be swept up along the way? We use javapackager and are very happy with what it gives us considering it is "free as in beer", but there are some significant workarounds required for code signing, especially on Mac OS X. Sign us (jClarity) up as early testers :-). Cheers, Martijn On 1 June 2018 at 08:53, Alan Bateman wrote: > On 31/05/2018 01:10, Kevin Rushforth wrote: > >> I would like to propose the following Draft JEP [1] for discussion. >> >> JDK-8200758: Packaging Tool >> >> This is intended as a JDK-scope replacement for the existing javapackager >> tool that ships with Oracle JDK 10 (and earlier Oracle JDK releases), and >> was delivered as part of the JavaFX build. The javapackager tool has been >> removed from Oracle JDK 11 along with the removal of JavaFX. >> >> Comments on this JEP are welcome. It is currently not targeted for a >> specific release, but we are aiming for JDK 12. >> >> -- Kevin >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >> >> I read through the draft. > > The Goals and Non-Goals looks very reasonable. > > The Summary includes "self-contained Java Server ... applications" but the > JEP doesn't say too much about that part. Can we assume it can be started > and stopped with /etc/init.d or equivalent scripts, logs with the server > output etc? A general observation is that most of the issues around > end-user/GUI applications are clearly documented in the draft, the headless > application environment case less so. It makes me wonder if this JEP should > be split so that it initially focuses on the former. > > I think it would be useful if the JEP explained what an "application" is. > Is it a JAR file (with a Main-Class) that is deployed on the class path? > Can it be a module? What about modules and libraries that the application > uses. Users of javapackager might know these things but readers of the JEP > may not. > > The JEP mentions that JavaFX will be removed in JDK 11. I assume this > should be clarified so that it's clear it will no longer be shipped in > Oracle's JDK. It's a none issue for those using OpenJDK builds as the the > JavaFX modules were never bundled/imported by default. > > There are several references to "server JRE" that probably should be > clarified as this notion does not exist in OpenJDK. It may be that the JEP > just needs to clearer as to the modules that it links into the run-time > image. > > There are several references to "application launcher" and "native > launcher". At some point we need to work out some issues with jlink as it > it will need to generate native launchers too. The JEP could list it as an > issue as the development of this JEP will at least touch on some of this. > > The draft doesn't have a section on testing. If someone is building > OpenJDK then will they will require additional tools in the build > environment and can the tests be run without manual interaction? Also how > about a developer using the tool, will the generated native packages be > easy to un-install so they can easily test in a local environment? > > -Alan > From Roger.Riggs at Oracle.com Mon Jun 4 13:32:55 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Jun 2018 09:32:55 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only Message-ID: Please review a change to make the values of java.home, user.home, user.dir, and user.name effectively read-only for internal use.? The values are cached during initialization and the cached values are used. Webrev: ? http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ Issue: ? https://bugs.openjdk.java.net/browse/JDK-8066709 CSR: ? https://bugs.openjdk.java.net/browse/JDK-8204235 Thanks, Roger From sundararajan.athijegannathan at oracle.com Mon Jun 4 14:24:44 2018 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 04 Jun 2018 19:54:44 +0530 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: <5B154BAC.7090509@oracle.com> Looks good -Sundar On 04/06/18, 7:02 PM, Roger Riggs wrote: > Please review a change to make the values of java.home, user.home, > user.dir, and user.name > effectively read-only for internal use. The values are cached during > initialization and the > cached values are used. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8204235 > > Thanks, Roger > > From li.jiang at oracle.com Mon Jun 4 14:41:59 2018 From: li.jiang at oracle.com (li.jiang at oracle.com) Date: Mon, 4 Jun 2018 22:41:59 +0800 Subject: [11] RFR: 8202026 8193552 8204269 : ISO 4217 Amendment #165 # 166 #167 Update In-Reply-To: <2099b4c0-b97a-4c1a-f291-fec8f7beb294@oracle.com> References: <5d50ddec-884c-e78c-4c2c-b9e11d8c3236@oracle.com> <62217aa4-16c5-2ec2-15f7-177d0906c53b@oracle.com> <2099b4c0-b97a-4c1a-f291-fec8f7beb294@oracle.com> Message-ID: <9181f81a-00f4-475c-553f-fa155439fa15@oracle.com> Hi Naoto, Pls review the updated code: http://cr.openjdk.java.net/~ljiang/8202026/webrev.02/ - As suggested, clean up the old transition dates. - Update copyright year. - ISO 4217 Amendment #167 was published today, which discards the #166, so withdraw the change for #166 in webrev.02. Bug for #167: https://bugs.openjdk.java.net/browse/JDK-8204269 Test passed on mach5. Thanks, Leo On 26/04/2018 1:00 AM, naoto.sato at oracle.com wrote: > Hi Leo, > > Although JDK11 is slated in 09/2018, enabling amendment 166 now is > technically a bug, as it will be effective from June 4. Please use the > transition mechanism in make/data/currency/CurrencyData.properties and > test/jdk/java/util/Currency/tablea1.txt. > > OTOH, there are old (past) transition entries. I would clean up those > entries, such as: > > ?326 # LATVIA > ?327 LV=LVL;2013-12-31-22-00-00;EUR > > in CurrencyData.properties. This applies to tabela1.txt as well. > > Naoto > > On 4/24/18 8:52 PM, Leo Jiang wrote: >> Forgot to mention, the tests in Currency fold are passed on Mach5. >> >> -Leo >> >> On 04/25/2018 09:33 AM, Leo Jiang wrote: >>> Hi, >>> >>> Please review the changes to address the ISO 4217 Amendment 165 166 >>> update. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8193552? 165 >>> https://bugs.openjdk.java.net/browse/JDK-8202026? 166 >>> >>> CR: >>> http://cr.openjdk.java.net/~ljiang/8202026/webrev.00/ >>> >>> >>> Detail: >>> #165 >>> From: >>> MAURITANIA??? Ouguiya??? MRO??? 478??? 2 >>> To: >>> MAURITANIA??? Ouguiya??? MRU??? 929??? 2 >>> >>> #166 >>> From: >>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var??? VEF??? 937??? 2 >>> To: >>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var Soberano??? VES >>> 928??? 2 >>> >>> >>> Thanks, >>> Leo From Alan.Bateman at oracle.com Mon Jun 4 15:00:36 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 4 Jun 2018 16:00:36 +0100 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <5B0FBC37.2090103@oracle.com> References: <5B0FBC37.2090103@oracle.com> Message-ID: <93d8a88f-7c63-6019-c222-e9687c4a7394@oracle.com> On 31/05/2018 10:11, Jan Lahoda wrote: > Hi, > > I'd like to upgrade the JOpt Simple library we are using to version > 5.0.4. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203891 > Complete webrev: > http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/complete/ > > Delta webrev only showing (all) JDK changes in JOpt Simple and related > changes in tests needed for the upgrade, etc.: > http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/joptsimple.delta/ > > Probably the biggest issue with this upgrade is that for two > subsequent parameters: > "--libs=", "/tmp" > "/tmp" used to be interpreted as the parameter of "libs", but now the > "libs" parameter is empty (as there's nothing behind the '='). See the > changes to test/jdk/tools/jmod/JmodTest.java for an example. > Hopefully, this is a reasonable change. > > How does this look? Surprising to see that jmod needs to be updated too but I think it's okay, the update to the jmods tests too. One question - does the update to make/CompileJavaModules.gmk mean that we were missing the JOpt Simple properties file from the run-time image? -Alan. From jan.lahoda at oracle.com Mon Jun 4 15:23:29 2018 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Mon, 4 Jun 2018 17:23:29 +0200 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <93d8a88f-7c63-6019-c222-e9687c4a7394@oracle.com> References: <5B0FBC37.2090103@oracle.com> <93d8a88f-7c63-6019-c222-e9687c4a7394@oracle.com> Message-ID: <5B155971.7040504@oracle.com> On 4.6.2018 17:00, Alan Bateman wrote: > On 31/05/2018 10:11, Jan Lahoda wrote: >> Hi, >> >> I'd like to upgrade the JOpt Simple library we are using to version >> 5.0.4. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203891 >> Complete webrev: >> http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/complete/ >> >> Delta webrev only showing (all) JDK changes in JOpt Simple and related >> changes in tests needed for the upgrade, etc.: >> http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/joptsimple.delta/ >> >> Probably the biggest issue with this upgrade is that for two >> subsequent parameters: >> "--libs=", "/tmp" >> "/tmp" used to be interpreted as the parameter of "libs", but now the >> "libs" parameter is empty (as there's nothing behind the '='). See the >> changes to test/jdk/tools/jmod/JmodTest.java for an example. >> Hopefully, this is a reasonable change. >> >> How does this look? > Surprising to see that jmod needs to be updated too but I think it's > okay, the update to the jmods tests too. I believe the API has been changed to use List instead of Collection on a few places here: https://github.com/jopt-simple/jopt-simple/commit/b8f859ac95a37dcb9abb084fe226da341963950d So the changes to jmod reflect that. > > One question - does the update to make/CompileJavaModules.gmk mean that > we were missing the JOpt Simple properties file from the run-time image? I believe the properties files have not been part of 4.6, the messages have been hardcoded in the classfiles. So the need to copy them is new. Thanks, Jan > > -Alan. From weijun.wang at oracle.com Mon Jun 4 15:41:09 2018 From: weijun.wang at oracle.com (Wang Weijun) Date: Mon, 4 Jun 2018 23:41:09 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: <8D9FFD19-4501-46A8-9E7E-D3A622CF81EC@oracle.com> Not a native English speaker, so my feeling might be incorrect. Will someone interpret this as that System.getProperty() will return a cached value? I would say ?Although getProperty() always returns the last value set by setProperty() (I assume this is the current behavior), it is not uncommon that consumers of a system property may read it once and cache the value it for later use, which means setting the property may not have the desired effect?. I also don?t think it?s worth listing the 4 property names in the spec. Quite some other system properties are also cached. Listing them there could further suggests that calling getProperty() on them will return the cached value. Thanks Max > ? 2018?6?4????9:32?Roger Riggs ??? > > Please review a change to make the values of java.home, user.home, user.dir, and user.name > effectively read-only for internal use. The values are cached during initialization and the > cached values are used. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8204235 > > Thanks, Roger > > From lance.andersen at oracle.com Mon Jun 4 16:09:31 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 4 Jun 2018 12:09:31 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: <647D84B5-F364-46F4-A80B-8854A3F889B1@oracle.com> Hi Roger, Overall, it looks very good. A couple of quick thoughts, nothing that needs any action unless you feel the desire :-) Line 671 in System.java, it mentions ?standard? system properties. (as does the existing Implementation note) but it does not really specify what a standard system property is (though it can be assumed) in getProperties overview. Not a big deal, just thought I would point it out in case you had a alternative thought. Do you think we should had a mention in getProperty() about this cached properties (or do we think the reference to getProperties() is enough) Best Lance > On Jun 4, 2018, at 9:32 AM, Roger Riggs wrote: > > Please review a change to make the values of java.home, user.home, user.dir, and user.name > effectively read-only for internal use. The values are cached during initialization and the > cached values are used. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8204235 > > Thanks, Roger > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Alan.Bateman at oracle.com Mon Jun 4 16:21:23 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 4 Jun 2018 17:21:23 +0100 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> On 04/06/2018 14:32, Roger Riggs wrote: > Please review a change to make the values of java.home, user.home, > user.dir, and user.name > effectively read-only for internal use.? The values are cached during > initialization and the > cached values are used. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > ? https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > ? https://bugs.openjdk.java.net/browse/JDK-8204235 Can the @apiNote be split up so that the list of 4 properties moves to an @implNote? Also "changing any standard property" may need clarification as it can mean both changing a system property in a running VM and also changing it by starting the VM with a value for one of these properties. I think I agree with Max on finding another name for the internal class as it is a class to get the original value of a few critical properties. Also I think we need to look at other ways to do the initialization, maybe in conjunction with VM.saveAndRemoveProperties so that we are saving the initial values of properties in one place. Are the changes to SocksSocketImpl correct? I may have missed something but the original called System.getProperty("user.dir") in a privileged block so I'm wondering if getUserNameChecked is needed. The static USER_DIR can be dropped from WindowsFileSystemProvider as there is only one instance of the provider. The other usages look okay to me. -Alan From srinivas.dama at oracle.com Mon Jun 4 17:37:46 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 4 Jun 2018 10:37:46 -0700 (PDT) Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) Message-ID: Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but these warnings are disabled earlier.I have enabled these warnings and fixed in sources. Regards, Srinivas From mandy.chung at oracle.com Mon Jun 4 17:40:10 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 4 Jun 2018 10:40:10 -0700 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <5B0FBC37.2090103@oracle.com> References: <5B0FBC37.2090103@oracle.com> Message-ID: <799bd973-3d11-b4cc-2f16-f4dc6383b4b2@oracle.com> Hi Jan, On 5/31/18 2:11 AM, Jan Lahoda wrote: > Hi, > > I'd like to upgrade the JOpt Simple library we are using to version 5.0.4. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203891 > Complete webrev: > http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/complete/ > > Delta webrev only showing (all) JDK changes in JOpt Simple and related > changes in tests needed for the upgrade, etc.: > http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/joptsimple.delta/ > > Probably the biggest issue with this upgrade is that for two subsequent > parameters: > "--libs=", "/tmp" > "/tmp" used to be interpreted as the parameter of "libs", but now the > "libs" parameter is empty (as there's nothing behind the '='). This is a reasonable fix and no whitespace is expected following `=`. > See the changes to test/jdk/tools/jmod/JmodTest.java for an example. Hopefully, > this is a reasonable change. 114 "--libs=" + libDir.toString(), Alternatively, you can take out `=` and run it as jmod --libs /tmp "--libs", libDir.toString(), Mandy From naoto.sato at oracle.com Mon Jun 4 18:27:41 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Mon, 4 Jun 2018 11:27:41 -0700 Subject: [11] RFR: 8202026 8193552 8204269 : ISO 4217 Amendment #165 # 166 #167 Update In-Reply-To: <9181f81a-00f4-475c-553f-fa155439fa15@oracle.com> References: <5d50ddec-884c-e78c-4c2c-b9e11d8c3236@oracle.com> <62217aa4-16c5-2ec2-15f7-177d0906c53b@oracle.com> <2099b4c0-b97a-4c1a-f291-fec8f7beb294@oracle.com> <9181f81a-00f4-475c-553f-fa155439fa15@oracle.com> Message-ID: <875fa321-9b73-17f4-c63e-b542e32b6424@oracle.com> Hi Leo, Change looks good. One leftover from the previous: >> in CurrencyData.properties. This applies to tabela1.txt as well. Can you please clean those LV/LT entries in tablea1.txt as well? Naoto On 6/4/18 7:41 AM, li.jiang at oracle.com wrote: > Hi Naoto, > > Pls review the updated code: > > http://cr.openjdk.java.net/~ljiang/8202026/webrev.02/ > > - As suggested, clean up the old transition dates. > - Update copyright year. > - ISO 4217 Amendment #167 was published today, which discards the #166, > so withdraw the change for #166 in webrev.02. > > Bug for #167: > https://bugs.openjdk.java.net/browse/JDK-8204269 > > Test passed on mach5. > > Thanks, > Leo > > On 26/04/2018 1:00 AM, naoto.sato at oracle.com wrote: >> Hi Leo, >> >> Although JDK11 is slated in 09/2018, enabling amendment 166 now is >> technically a bug, as it will be effective from June 4. Please use the >> transition mechanism in make/data/currency/CurrencyData.properties and >> test/jdk/java/util/Currency/tablea1.txt. >> >> OTOH, there are old (past) transition entries. I would clean up those >> entries, such as: >> >> ??326 # LATVIA >> ??327 LV=LVL;2013-12-31-22-00-00;EUR >> >> in CurrencyData.properties. This applies to tabela1.txt as well. >> >> Naoto >> >> On 4/24/18 8:52 PM, Leo Jiang wrote: >>> Forgot to mention, the tests in Currency fold are passed on Mach5. >>> >>> -Leo >>> >>> On 04/25/2018 09:33 AM, Leo Jiang wrote: >>>> Hi, >>>> >>>> Please review the changes to address the ISO 4217 Amendment 165 166 >>>> update. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8193552? 165 >>>> https://bugs.openjdk.java.net/browse/JDK-8202026? 166 >>>> >>>> CR: >>>> http://cr.openjdk.java.net/~ljiang/8202026/webrev.00/ >>>> >>>> >>>> Detail: >>>> #165 >>>> From: >>>> MAURITANIA??? Ouguiya??? MRO??? 478??? 2 >>>> To: >>>> MAURITANIA??? Ouguiya??? MRU??? 929??? 2 >>>> >>>> #166 >>>> From: >>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var??? VEF??? 937??? 2 >>>> To: >>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var Soberano??? VES 928??? 2 >>>> >>>> >>>> Thanks, >>>> Leo From paul.sandoz at oracle.com Mon Jun 4 18:51:22 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 4 Jun 2018 11:51:22 -0700 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: References: Message-ID: <12736123-651D-4AD9-9E0E-709698BB9253@oracle.com> > On Jun 4, 2018, at 4:22 AM, Lance Andersen wrote: > > Hi, > > Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html > > As part of this fix, I took the liberty to move from package.html to package-info.java > > The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ > > I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java > The attachment got dropped. Upload it to cr? Paul. > Best > Lance > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > From lance.andersen at oracle.com Mon Jun 4 19:00:03 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 4 Jun 2018 15:00:03 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: <12736123-651D-4AD9-9E0E-709698BB9253@oracle.com> References: <12736123-651D-4AD9-9E0E-709698BB9253@oracle.com> Message-ID: <765ADD67-D7EE-4061-8050-CA0CD0C0DA25@oracle.com> > On Jun 4, 2018, at 2:51 PM, Paul Sandoz wrote: > > > >> On Jun 4, 2018, at 4:22 AM, Lance Andersen wrote: >> >> Hi, >> >> Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html >> >> As part of this fix, I took the liberty to move from package.html to package-info.java >> >> The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ >> >> I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java >> > > The attachment got dropped. Upload it to cr? Sorry about that, forgot that attachments are stripped. It is attached now to https://bugs.openjdk.java.net/browse/JDK-8201608 Best Lance > > Paul. > >> Best >> Lance >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From paul.sandoz at oracle.com Mon Jun 4 19:07:37 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 4 Jun 2018 12:07:37 -0700 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: <765ADD67-D7EE-4061-8050-CA0CD0C0DA25@oracle.com> References: <12736123-651D-4AD9-9E0E-709698BB9253@oracle.com> <765ADD67-D7EE-4061-8050-CA0CD0C0DA25@oracle.com> Message-ID: <9A6FD84B-7FD9-42E7-8E97-BDC13A05E35A@oracle.com> > On Jun 4, 2018, at 12:00 PM, Lance Andersen wrote: > > >> On Jun 4, 2018, at 2:51 PM, Paul Sandoz > wrote: >> >> >> >>> On Jun 4, 2018, at 4:22 AM, Lance Andersen > wrote: >>> >>> Hi, >>> >>> Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html >>> >>> As part of this fix, I took the liberty to move from package.html to package-info.java >>> >>> The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ > >>> >>> I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java >>> >> >> The attachment got dropped. Upload it to cr? > > Sorry about that, forgot that attachments are stripped. It is attached now to https://bugs.openjdk.java.net/browse/JDK-8201608 > Thanks, +1 (from mostly reviewing the diff and not the package-info.java itself). Paul. From Roger.Riggs at Oracle.com Mon Jun 4 19:54:42 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Jun 2018 15:54:42 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <8D9FFD19-4501-46A8-9E7E-D3A622CF81EC@oracle.com> References: <8D9FFD19-4501-46A8-9E7E-D3A622CF81EC@oracle.com> Message-ID: <4dcfbb49-a0dc-e69f-d44b-2faa2189c5de@Oracle.com> Hi Max, On 6/4/2018 11:41 AM, Wang Weijun wrote: > Not a native English speaker, so my feeling might be incorrect. > > Will someone interpret this as that System.getProperty() will return a cached value? I don't think so, it should be clear that the values are cached at initialization or first use. > > I would say ?Although getProperty() always returns the last value set by setProperty() (I assume this is the current behavior), it is not uncommon that consumers of a system property may read it once and cache the value it for later use, which means setting the property may not have the desired effect?. The purpose of the change is to clarify the behavior of the java.base module internal access to the properties. How applications handle properties is beyond the intended scope. > > I also don?t think it?s worth listing the 4 property names in the spec. Quite some other system properties are also cached. Listing them there could further suggests that calling getProperty() on them will return the cached value. It seemed worthwhile to highlight the specific properties affected and the release note should be specific. I moved them to an implNote as Alan suggested. Thanks, Roger > > Thanks > Max > >> ? 2018?6?4????9:32?Roger Riggs ??? >> >> Please review a change to make the values of java.home, user.home, user.dir, and user.name >> effectively read-only for internal use. The values are cached during initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8204235 >> >> Thanks, Roger >> >> From Roger.Riggs at Oracle.com Mon Jun 4 19:55:12 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Jun 2018 15:55:12 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <647D84B5-F364-46F4-A80B-8854A3F889B1@oracle.com> References: <647D84B5-F364-46F4-A80B-8854A3F889B1@oracle.com> Message-ID: Hi Lance, On 6/4/2018 12:09 PM, Lance Andersen wrote: > Hi Roger, > > Overall, it looks very good. > > A couple of quick thoughts, nothing that needs any action unless you > feel the desire :-) > > Line 671 in System.java, it mentions ?standard? system properties. > ?(as does the existing Implementation note) but it does not really > specify what a standard system property is (though it can be assumed) > in getProperties overview. ?Not a big deal, just thought I would point > it out in case you had a alternative thought. Yes, it is a bit vague and the documentation of properties is spread over many packages and classes. 'Standard' generally refers to any property defined in the scope of Java SE or the documented JDK properties. (For example, those subject to the CSR process). > > Do you think we should had a mention in getProperty() about this > cached properties (or do we think the reference to getProperties() is > enough) Hard to say,? there will be a release note, but the properties methods are well known and it is likely no one reads the javadoc anymore.? getProperty delegates to getProperties for most of its behavior. Adding a repeat of the API note to System.setProperties and System.setProperty might be more to the point, warning against setting standard system properties via the API. I added the apinote to each of the 6 methods for setting, clearing, or getting properties. If that seems excessive, I'm open to removing it from those that provide the least value. Thanks, Roger > > Best > Lance >> On Jun 4, 2018, at 9:32 AM, Roger Riggs > > wrote: >> >> Please review a change to make the values of java.home, user.home, >> user.dir, and user.name >> effectively read-only for internal use.? The values are cached during >> initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8204235 >> >> Thanks, Roger >> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From Roger.Riggs at Oracle.com Mon Jun 4 19:59:04 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 4 Jun 2018 15:59:04 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> Message-ID: <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> Hi Alan, Updated webrev in place: http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ On 6/4/2018 12:21 PM, Alan Bateman wrote: > On 04/06/2018 14:32, Roger Riggs wrote: >> Please review a change to make the values of java.home, user.home, >> user.dir, and user.name >> effectively read-only for internal use.? The values are cached during >> initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> Issue: >> ? https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> ? https://bugs.openjdk.java.net/browse/JDK-8204235 > Can the @apiNote be split up so that the list of 4 properties moves to > an @implNote? Also "changing any standard property" may need > clarification as it can mean both changing a system property in a > running VM and also changing it by starting the VM with a value for > one of these properties. ok, will clarify that setting the properties using setProperty or the getProperties().set() methods after initialization may have unpredictable effects. > > I think I agree with Max on finding another name for the internal > class as it is a class to get the original value of a few critical > properties. Also I think we need to look at other ways to do the > initialization, maybe in conjunction with VM.saveAndRemoveProperties > so that we are saving the initial values of properties in one place. There is a future, more comprehensive change that will address a more consistent model for system properties. For now the task is to reduce the impact of inadvisable programmatic setting of standard system properties for a few of the most (likely to be) abused properties. > > Are the changes to SocksSocketImpl correct? I may have missed > something but the original called System.getProperty("user.dir") in a > privileged block so I'm wondering if getUserNameChecked is needed. The existing code in SocksSocketImpl is inconsistent with respect to access to user.name; some flows use doPriv to access the property and others did not.? If someone familiar with the Socks networking function can recommend the proper access, it can be revised.? The intent was to have the same security checks as before. > > The static USER_DIR can be dropped from WindowsFileSystemProvider as > there is only one instance of the provider. ok > > The other usages look okay to me. Thanks, Roger > > -Alan > From lance.andersen at oracle.com Mon Jun 4 20:39:17 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 4 Jun 2018 16:39:17 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <647D84B5-F364-46F4-A80B-8854A3F889B1@oracle.com> Message-ID: <12B561C2-8EE6-4DAE-A352-D53ED485F53C@oracle.com> Hi Roger, In System.java ????? changing any standard system property may have unpredictable effects. ???????? Perhaps capitalize ?changing' and maybe consider ?results? vs ?effects? HTH Best Lance > On Jun 4, 2018, at 3:55 PM, Roger Riggs wrote: > > Hi Lance, > > On 6/4/2018 12:09 PM, Lance Andersen wrote: >> Hi Roger, >> >> Overall, it looks very good. >> >> A couple of quick thoughts, nothing that needs any action unless you feel the desire :-) >> >> Line 671 in System.java, it mentions ?standard? system properties. (as does the existing Implementation note) but it does not really specify what a standard system property is (though it can be assumed) in getProperties overview. Not a big deal, just thought I would point it out in case you had a alternative thought. > Yes, it is a bit vague and the documentation of properties is spread over many packages and classes. > 'Standard' generally refers to any property defined in the scope of Java SE or the documented JDK properties. > (For example, those subject to the CSR process). >> >> Do you think we should had a mention in getProperty() about this cached properties (or do we think the reference to getProperties() is enough) > Hard to say, there will be a release note, but the properties methods are well known and it is > likely no one reads the javadoc anymore. getProperty delegates to getProperties for most of its behavior. > > Adding a repeat of the API note to System.setProperties and System.setProperty might be more to the point, > warning against setting standard system properties via the API. > I added the apinote to each of the 6 methods for setting, clearing, or getting properties. If that seems excessive, > I'm open to removing it from those that provide the least value. > > Thanks, Roger > >> >> Best >> Lance >>> On Jun 4, 2018, at 9:32 AM, Roger Riggs > wrote: >>> >>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>> effectively read-only for internal use. The values are cached during initialization and the >>> cached values are used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>> >>> Thanks, Roger >>> >>> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From martinrb at google.com Mon Jun 4 21:47:31 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Jun 2018 14:47:31 -0700 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: TimeUnit#toDuration is gone. Here's a non-strawman attempt at TimeUnit#convert(Duration). I was astonished how hard it was to get this really correct instead of merely mostly correct. For negative numbers, the long/int representation of Duration is sort-of a "floored" division instead of java's default "truncating" division. /** * Converts the given time duration to this unit. * *

For any TimeUnit {@code unit}, * {@code unit.convert(Duration.ofNanos(n))} * is equivalent to * {@code unit.convert(n, NANOSECONDS)}, and * {@code unit.convert(Duration.of(n, unit.toChronoUnit()))} * is equivalent to {@code n} (in the absence of overflow). * * @param duration the time duration * @return the converted duration in this unit, * or {@code Long.MIN_VALUE} if conversion would negatively overflow, * or {@code Long.MAX_VALUE} if it would positively overflow. * @throws NullPointerException if {@code duration} is null * @see Duration#of(long,TemporalUnit) * @since 11 */ public long convert(Duration duration) { final long secs = duration.getSeconds(); final int nano = duration.getNano(); final long s, candidate; if ((s = scale) == SECOND_SCALE) return (secs < 0 && nano > 0) ? secs + 1 : secs; if (s > SECOND_SCALE) { long div = secs / secRatio; if (secs < 0 && nano > 0 && div * secRatio == secs) div++; return div; } return (secs >= 0) ? ((secs > maxSecs || (candidate = secs * secRatio + nano / s) < 0) ? Long.MAX_VALUE : candidate) : ((secs < - maxSecs - 1 || (candidate = secs * secRatio + (nano + s - 1) / s) > 0) ? Long.MIN_VALUE : candidate); } But is it really correct? Here are some tests as well: /** * convert(Duration) roundtrips with Duration.ofXXXX and Duration.of(long, ChronoUnit) */ public void testConvertDuration_roundtripDurationOf() { long n = ThreadLocalRandom.current().nextLong(); assertEquals(n, NANOSECONDS.convert(Duration.ofNanos(n))); assertEquals(n, NANOSECONDS.convert(Duration.of(n, ChronoUnit.NANOS))); assertEquals(n, MILLISECONDS.convert(Duration.ofMillis(n))); assertEquals(n, MILLISECONDS.convert(Duration.of(n, ChronoUnit.MILLIS))); assertEquals(n, SECONDS.convert(Duration.ofSeconds(n))); assertEquals(n, SECONDS.convert(Duration.of(n, ChronoUnit.SECONDS))); n /= 60; assertEquals(n, MINUTES.convert(Duration.ofMinutes(n))); assertEquals(n, MINUTES.convert(Duration.of(n, ChronoUnit.MINUTES))); n /= 60; assertEquals(n, HOURS.convert(Duration.ofHours(n))); assertEquals(n, HOURS.convert(Duration.of(n, ChronoUnit.HOURS))); n /= 24; assertEquals(n, DAYS.convert(Duration.ofDays(n))); assertEquals(n, DAYS.convert(Duration.of(n, ChronoUnit.DAYS))); } /** * convert(Duration.ofNanos(n)) agrees with convert(n, NANOSECONDS) */ public void testConvertDuration_roundtripDurationOfNanos() { // Test values near unit transitions and near overflow. LongStream.concat( Arrays.stream(TimeUnit.values()).mapToLong(u -> u.toNanos(1)), LongStream.of(Long.MAX_VALUE, Long.MIN_VALUE)) .flatMap(n -> LongStream.of(n, n + 1, n - 1)) .flatMap(n -> LongStream.of(n, n + 1_000_000_000, n - 1_000_000_000)) .flatMap(n -> LongStream.of(n, -n)) // .peek(System.err::println) .forEach(n -> Arrays.stream(TimeUnit.values()).forEach( u -> assertEquals(u.convert(n, NANOSECONDS), u.convert(Duration.ofNanos(n))))); } /** * convert(Duration) doesn't misbehave near Long.MAX_VALUE and Long.MIN_VALUE. */ public void testConvertDuration_nearOverflow() { ChronoUnit NANOS = ChronoUnit.NANOS; Duration maxDuration = Duration.ofSeconds(Long.MAX_VALUE, 999_999_999); Duration minDuration = Duration.ofSeconds(Long.MIN_VALUE, 0); for (TimeUnit u : TimeUnit.values()) { ChronoUnit cu = u.toChronoUnit(); long r; if (u.toNanos(1) > SECONDS.toNanos(1)) { r = u.toNanos(1) / SECONDS.toNanos(1); assertThrows(ArithmeticException.class, () -> Duration.of(Long.MAX_VALUE, cu), () -> Duration.of(Long.MIN_VALUE, cu)); } else { r = 1; Duration max = Duration.of(Long.MAX_VALUE, cu); Duration min = Duration.of(Long.MIN_VALUE, cu); assertEquals(Long.MAX_VALUE, u.convert(max)); assertEquals(Long.MAX_VALUE - 1, u.convert(max.minus(1, NANOS))); assertEquals(Long.MAX_VALUE - 1, u.convert(max.minus(1, cu))); assertEquals(Long.MIN_VALUE, u.convert(min)); assertEquals(Long.MIN_VALUE + 1, u.convert(min.plus(1, NANOS))); assertEquals(Long.MIN_VALUE + 1, u.convert(min.plus(1, cu))); assertEquals(Long.MAX_VALUE, u.convert(max.plus(1, NANOS))); if (u != SECONDS) { assertEquals(Long.MAX_VALUE, u.convert(max.plus(1, cu))); assertEquals(Long.MIN_VALUE, u.convert(min.minus(1, NANOS))); assertEquals(Long.MIN_VALUE, u.convert(min.minus(1, cu))); } } assertEquals(Long.MAX_VALUE / r, u.convert(maxDuration)); assertEquals(Long.MIN_VALUE / r, u.convert(minDuration)); } } On Thu, May 31, 2018 at 12:32 AM, Stephen Colebourne wrote: > I'm not convinced TimeUnit::toDuration(long amount) has enough value. > We don't have a similar method on ChronoUnit > > Duration.of(amount, timeUnit.toChronoUnit()) seems sufficient. Maybe > document this in the convert(Duration) method? > > Stephen > > > On 31 May 2018 at 01:19, Martin Buchholz wrote: > > v.0.2 has both conversion methods in TimeUnit. The unexpected weirdness > is > > that convert(Duration) saturates while toDuration throws > > ArithmeticException, but both seem author-culture-consistent. Perhaps > > TimeUnit#toDuration doesn't provide enough value in view of the existing > > Duration.of and TimeUnit#toChronoUnit. And most of the time you'd > expect to > > convert from Duration to long, just before calling a TimeUnit based > method. > > > > /** > > * Converts the given time duration to this unit. > > * > > * @param duration the time duration > > * @return the converted duration in this unit, > > * or {@code Long.MIN_VALUE} if conversion would negatively overflow, > > * or {@code Long.MAX_VALUE} if it would positively overflow. > > * @throws NullPointerException if {@code duration} is null > > */ > > public long convert(Duration duration) { > > long s = convert(duration.getSeconds(), SECONDS); > > if (s == Long.MIN_VALUE) return s; > > long n = convert(duration.getNano(), NANOSECONDS); > > assert n >= 0 && n < 1_000_000_000; > > return (s + n < s) ? Long.MAX_VALUE : s + n; > > } > > > > /** > > * Converts the given time duration in this unit to a Duration. > > * > > * @param duration the time duration > > * @return the time duration represented as a Duration > > * @throws ArithmeticException if the duration cannot be represented > > * as a Duration due to numeric overflow > > */ > > public Duration toDuration(long duration) { > > return Duration.of(duration, toChronoUnit()); > > } > > > From brent.christian at oracle.com Mon Jun 4 22:10:20 2018 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 4 Jun 2018 15:10:20 -0700 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: Hi, Roger A few things I noticed in src/java.base/share/classes/java/lang/System.java: * Properties can also be set via setProperties(Properties p), though that method is not mentioned in the doc changes (other than to setProperties() itself). 750 * setting a standard property after initialization 's' -> 'S' in "setting" 788 * setting a standard property after initialization using 's' -> 'S' in "setting" 789 * {@link #getProperties()} or {@link #setProperty(String, String)} The doc change for getProperties() also mentions "clearProperty()", but that one is not mentioned here. The rest looks fine to me. Thanks, -Brent On 6/4/18 6:32 AM, Roger Riggs wrote: > Please review a change to make the values of java.home, user.home, > user.dir, and user.name > effectively read-only for internal use.? The values are cached during > initialization and the > cached values are used. > > Webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > ? https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > ? https://bugs.openjdk.java.net/browse/JDK-8204235 > > Thanks, Roger > > From weijun.wang at oracle.com Mon Jun 4 23:24:38 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 5 Jun 2018 07:24:38 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <4dcfbb49-a0dc-e69f-d44b-2faa2189c5de@Oracle.com> References: <8D9FFD19-4501-46A8-9E7E-D3A622CF81EC@oracle.com> <4dcfbb49-a0dc-e69f-d44b-2faa2189c5de@Oracle.com> Message-ID: Hi Roger Thanks for the explanation. Another question: Before this change, a SecurityException might be thrown when getProperty() is called, does you new code simulate this behavior? Or in these cases this is unnecessary? Also, looks like all change is inside java.base, do you have a suggestion we use it in other JDK modules? Thanks Max > On Jun 5, 2018, at 3:54 AM, Roger Riggs wrote: > > Hi Max, > > On 6/4/2018 11:41 AM, Wang Weijun wrote: >> Not a native English speaker, so my feeling might be incorrect. >> >> Will someone interpret this as that System.getProperty() will return a cached value? >> > I don't think so, it should be clear that the values are cached at initialization or first use. >> >> I would say ?Although getProperty() always returns the last value set by setProperty() (I assume this is the current behavior), it is not uncommon that consumers of a system property may read it once and cache the value it for later use, which means setting the property may not have the desired effect?. >> > The purpose of the change is to clarify the behavior of the java.base module internal access to the properties. > How applications handle properties is beyond the intended scope. >> >> I also don?t think it?s worth listing the 4 property names in the spec. Quite some other system properties are also cached. Listing them there could further suggests that calling getProperty() on them will return the cached value. >> > It seemed worthwhile to highlight the specific properties affected and the release note should be specific. > I moved them to an implNote as Alan suggested. > > Thanks, Roger > >> >> Thanks >> Max >> >> >>> ? 2018?6?4????9:32?Roger Riggs >>> ??? >>> >>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>> effectively read-only for internal use. The values are cached during initialization and the >>> cached values are used. >>> >>> Webrev: >>> >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> >>> Issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> >>> CSR: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>> >>> >>> Thanks, Roger >>> >>> >>> > From martinrb at google.com Tue Jun 5 00:05:24 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Jun 2018 17:05:24 -0700 Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: References: Message-ID: Hej Deepak, This looks alright, but you really need to add that trailing newline on the test file (a Martin pet peeve). I wonder if instead we invest a little more work, but backport the entire tck directory. All tests should pass except for those that test features not yet backported. On Mon, Jun 4, 2018 at 4:46 AM, Deepak Kejriwal wrote: > Hi all, > > > > Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/ > browse/JDK-8186171 > > Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.00/ > > > > Summary(also added to backport bug description): > > > > The back port for test files is not clean back port as all tests files are > extending JSR166TestCase.java which was added to JDK 9 as part of HYPERLINK > "https://bugs.openjdk.java.net/browse/JDK-8146467"JDK-8146467: Integrate > JSR 166 jck tests into JDK repo. This is not present in JDK8 version. > > Therefore, I have extracted test are relevant to the fix done JDK-8186171 > and created a new test file Bug8186171Test.java that contains below two > test methods: > > > > . testBug8186171NonDeterministic : This method is copy of > "testBug8186171" present in "MapTest.java" of original changeset > http://hg.openjdk.java.net/jdk10/master/rev/3f5f9bc0bdc2. As it is based > on randomization and runs 1000 times, the method name is suffixed with > "NonDeterministic". > > . testBug8186171Deterministic : This is a new method that runs > single time as it produces the exact scenario mentioned in JDK-8186171. > Therefore, it is not needed to run multiple times to produce the scenario > mentioned in bug. Hence the method name is suffixed with "Deterministic" > > > > Regards, > > Deepak > From stuart.marks at oracle.com Tue Jun 5 00:09:13 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Jun 2018 17:09:13 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> Message-ID: <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> On 6/3/18 8:55 AM, Alan Bateman wrote: > On 03/06/2018 13:11, David Holmes wrote: >> >> Any suggestions as to how to do that in a practical and effective way? >> >> "As if done by the highly-dangerous, long-deprecated and finally removed >> Thread.stop(Throwable)" >> >> ? ;-) >> >> In all seriousness I hate to do anything that might suggest these are valid >> API's even for tools. Maybe we should deprecate them as well (separate RFE of >> course). > These date back to JPDA in JDK 1.2 (it was JVMDI at the time, pre-dates JVM TI) > and would need research to see if there are debuggers or maybe fault injection > tools are using it. > > For Stuart's current effort then it will need a bit of text, maybe borrowing old > spec where the operation was specified to "stop" a thread with an asynchronous > exception. Warnings about the dangers are probably appropriate but these are of > course interfaces that developers are unlikely to ever use directly. The spec for com.sun.jdk.ThreadReference.stop(Throwable) seems sufficiently descriptive to stand alone if the @see Thread.stop(Throwable) cross reference is simply removed. Are there any other changes that need to be done as part of this changeset? s'marks From david.holmes at oracle.com Tue Jun 5 00:56:57 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 5 Jun 2018 10:56:57 +1000 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <647D84B5-F364-46F4-A80B-8854A3F889B1@oracle.com> Message-ID: Hi Roger, On 5/06/2018 5:55 AM, Roger Riggs wrote: > Hi Lance, > > On 6/4/2018 12:09 PM, Lance Andersen wrote: >> Hi Roger, >> >> Overall, it looks very good. >> >> A couple of quick thoughts, nothing that needs any action unless you >> feel the desire :-) >> >> Line 671 in System.java, it mentions ?standard? system properties. >> ?(as does the existing Implementation note) but it does not really >> specify what a standard system property is (though it can be assumed) >> in getProperties overview. ?Not a big deal, just thought I would point >> it out in case you had a alternative thought. > Yes, it is a bit vague and the documentation of properties is spread > over many packages and classes. > 'Standard' generally refers to any property defined in the scope of Java > SE or the documented JDK properties. > (For example, those subject to the CSR process). The "standard properties" are those documented in the table in System.getProperties() following "This set of system properties always includes values for the following keys:" and which is followed shortly after by: "In addition to the standard system properties, the system properties may include the following keys:" These are the properties that must be provided by all implementations of the Java platform. David ----- >> >> Do you think we should had a mention in getProperty() about this >> cached properties (or do we think the reference to getProperties() is >> enough) > Hard to say,? there will be a release note, but the properties methods > are well known and it is > likely no one reads the javadoc anymore.? getProperty delegates to > getProperties for most of its behavior. > > Adding a repeat of the API note to System.setProperties and > System.setProperty might be more to the point, > warning against setting standard system properties via the API. > I added the apinote to each of the 6 methods for setting, clearing, or > getting properties. If that seems excessive, > I'm open to removing it from those that provide the least value. > > Thanks, Roger > >> >> Best >> Lance >>> On Jun 4, 2018, at 9:32 AM, Roger Riggs >> > wrote: >>> >>> Please review a change to make the values of java.home, user.home, >>> user.dir, and user.name >>> effectively read-only for internal use.? The values are cached during >>> initialization and the >>> cached values are used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>> >>> Thanks, Roger >>> >>> >> >> >> >> >> Lance >> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle?Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > From stuart.marks at oracle.com Tue Jun 5 01:52:55 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Jun 2018 18:52:55 -0700 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com> On 6/4/18 6:32 AM, Roger Riggs wrote: > Please review a change to make the values of java.home, user.home, user.dir, and > user.name > effectively read-only for internal use.? The values are cached during > initialization and the > cached values are used. > > Webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > ? https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > ? https://bugs.openjdk.java.net/browse/JDK-8204235 Hi Roger, Overall I think the intent of the change is a good one, as is the reimplementation of internal uses of system properties to use an internal API instead. I think it would be good to have an explicit initialization of the SystemProperty class (or whatever it ends up being called) instead of implicitly initializing via a static initializer. If the class is initialized too early, for some reason, things would go wrong. Should there be an error, a warning, or assertion checking to make sure that none of the cached values are null? They should all be non-null, right? s'marks From martinrb at google.com Tue Jun 5 03:52:26 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Jun 2018 20:52:26 -0700 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: Looks like if you switch representation for negative Duration, much of the hair goes away. This version also optimizes for NANOSECONDS, which is very likely to be the target in practice: public long convert(Duration duration) { long secs = duration.getSeconds(); int nano = duration.getNano(); if (secs < 0 && nano > 0) { // use representation compatible with integer division secs++; nano -= SECOND_SCALE; } final long s; if ((s = scale) >= SECOND_SCALE) return (s == SECOND_SCALE) ? secs : secs / secRatio; long val = secs * secRatio + ((s == NANO_SCALE) ? nano : nano / s); return ((secs | nano) >= 0) ? ((secs > maxSecs || val < 0) ? Long.MAX_VALUE : val) : ((secs < -maxSecs || val > 0) ? Long.MIN_VALUE : val); } On Mon, Jun 4, 2018 at 2:47 PM, Martin Buchholz wrote: > TimeUnit#toDuration is gone. > > Here's a non-strawman attempt at TimeUnit#convert(Duration). > I was astonished how hard it was to get this really correct instead of > merely mostly correct. > For negative numbers, the long/int representation of Duration is sort-of a > "floored" division instead of java's default "truncating" division. > > public long convert(Duration duration) { > final long secs = duration.getSeconds(); > final int nano = duration.getNano(); > final long s, candidate; > if ((s = scale) == SECOND_SCALE) > return (secs < 0 && nano > 0) ? secs + 1 : secs; > if (s > SECOND_SCALE) { > long div = secs / secRatio; > if (secs < 0 && nano > 0 && div * secRatio == secs) > div++; > return div; > } > return (secs >= 0) > ? ((secs > maxSecs > || (candidate = secs * secRatio + nano / s) < 0) > ? Long.MAX_VALUE : candidate) > : ((secs < - maxSecs - 1 > || (candidate = secs * secRatio + (nano + s - 1) / s) > 0) > ? Long.MIN_VALUE : candidate); > } > > From ivan.gerasimov at oracle.com Tue Jun 5 04:26:48 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 4 Jun 2018 21:26:48 -0700 Subject: RFR 8203768 : Avoid reallocation in java.base/unix/classes/java/lang/ProcessImpl.java Message-ID: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> Hello! When closing a Process, its stdout and stderr are drained: The remaining bytes are copied into an array, and then the out/err stream is replaced with ByteArrayInputStream(). In a case when a fewer number of bytes were read then the Stream.available() suggested, the array is reallocated with Arrays.copyOf(). It is possible to avoid reallocation, and use three-args ByteArrayInputStream() constructor to specify a portion of the array used. Would you please help review this trivial fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8203768 WEBREV: http://cr.openjdk.java.net/~igerasim/8203768/00/webrev/ All existing tests pass on all supported platforms. Thanks in advance! -- With kind regards, Ivan Gerasimov From martinrb at google.com Tue Jun 5 04:55:21 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Jun 2018 21:55:21 -0700 Subject: RFR 8203768 : Avoid reallocation in java.base/unix/classes/java/lang/ProcessImpl.java In-Reply-To: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> References: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> Message-ID: Hej Ivan, IIRC I wrote the first iteration of this code. I would expect that almost always either the buffer will be empty or the call to available() is accurate and the read will actually read everything in one chunk, so the byte array will be perfectly allocated with no need for re-allocation in practice. Do you have evidence it would be otherwise in practice? But suppose available() incorrectly claims that a million bytes are available, but you only read 10. Then there's a tradeoff - the cost of reallocation versus the risk that the mostly empty backing array will be retained for a long time if the Process object is not garbage collected. I'm leaning towards the status quo. On Mon, Jun 4, 2018 at 9:26 PM, Ivan Gerasimov wrote: > Hello! > > When closing a Process, its stdout and stderr are drained: The remaining > bytes are copied into an array, and then the out/err stream is replaced > with ByteArrayInputStream(). > > In a case when a fewer number of bytes were read then the > Stream.available() suggested, the array is reallocated with Arrays.copyOf(). > > It is possible to avoid reallocation, and use three-args > ByteArrayInputStream() constructor to specify a portion of the array used. > > Would you please help review this trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8203768 > WEBREV: http://cr.openjdk.java.net/~igerasim/8203768/00/webrev/ > > All existing tests pass on all supported platforms. > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > > From nishit.jain at oracle.com Tue Jun 5 05:13:08 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Tue, 5 Jun 2018 10:43:08 +0530 Subject: RFR JDK-8203872: Upgrading JDK with latest available LSR data from IANA Message-ID: <6aa57b43-c15e-cfee-cb32-922e4eec8489@oracle.com> Hi, Please review the fix for JDK-8203872. Bug: https://bugs.openjdk.java.net/browse/JDK-8203872 Webrev: http://cr.openjdk.java.net/~nishjain/8203872/webrev.01/ Fix: Updated the LSR data to the version 2017-08-15 Regards, Nishit Jain From ivan.gerasimov at oracle.com Tue Jun 5 05:31:00 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 4 Jun 2018 22:31:00 -0700 Subject: RFR 8203768 : Avoid reallocation in java.base/unix/classes/java/lang/ProcessImpl.java In-Reply-To: References: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> Message-ID: Hej Martin! On 6/4/18 9:55 PM, Martin Buchholz wrote: > Hej Ivan, > > IIRC I wrote the first iteration of this code. > > I would expect that almost always either the buffer will be empty or > the call to available() is accurate and the read will actually read > everything in one chunk, so the byte array will be perfectly allocated > with no need for re-allocation in practice. Do you have evidence it > would be otherwise in practice? > No evidence. It is just an observation from reading code. > But suppose available() incorrectly claims that a million bytes are > available, but you only read 10. Then there's a tradeoff - the cost > of reallocation versus the risk that the mostly empty backing array > will be retained for a long time if the Process object is not garbage > collected. > On the other hand, if available() claims a million bytes, and then only 999999990 bytes were read, there will be mostly unnecessary allocation and copying. > I'm leaning towards the status quo. > Okay, let's leave it as is :) It was meant mostly for cleaning up the code, so if there is a doubt, then it's better keep what we have. With kind regards, Ivan > > On Mon, Jun 4, 2018 at 9:26 PM, Ivan Gerasimov > > wrote: > > Hello! > > When closing a Process, its stdout and stderr are drained: The > remaining bytes are copied into an array, and then the out/err > stream is replaced with ByteArrayInputStream(). > > In a case when a fewer number of bytes were read then the > Stream.available() suggested, the array is reallocated with > Arrays.copyOf(). > > It is possible to avoid reallocation, and use three-args > ByteArrayInputStream() constructor to specify a portion of the > array used. > > Would you please help review this trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8203768 > > WEBREV: http://cr.openjdk.java.net/~igerasim/8203768/00/webrev/ > > > All existing tests pass on all supported platforms. > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > > -- With kind regards, Ivan Gerasimov From xueming.shen at oracle.com Tue Jun 5 05:38:45 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 04 Jun 2018 22:38:45 -0700 Subject: RFR JDK-8200530: '\r' is not supported as "newline" in java.util.jar.Manifest. Message-ID: <5B1621E5.80301@oracle.com> Hi, Please help review the changeset for JDK-8200530. "newline" is specified as |CR LF | LF | CR|(/not followed by/|LF|) in Jar spec [1] from the very beginning but our implementation actually never supports "\r"/CR (not followed by LF) case. The proposed change here is to add CR as an individual supported "newline"/line separator. issue: https://bugs.openjdk.java.net/browse/JDK-8200530 webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev Thanks, Sherman [1] https://docs.oracle.com/javase/10/docs/specs/jar/jar.html From Alan.Bateman at oracle.com Tue Jun 5 06:11:05 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jun 2018 07:11:05 +0100 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: On 05/06/2018 01:09, Stuart Marks wrote: > : > > The spec for com.sun.jdk.ThreadReference.stop(Throwable) seems > sufficiently descriptive to stand alone if the @see > Thread.stop(Throwable) cross reference is simply removed. > > Are there any other changes that need to be done as part of this > changeset? The source file that is used to generate the JDWP protocol code and the spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for this change-set, assuming it is followed up quickly with another change-set to update those specs. -Alan From Alan.Bateman at oracle.com Tue Jun 5 06:19:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jun 2018 07:19:22 +0100 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> Message-ID: <8a1ad331-411b-3e0b-58d1-36f1caad3dac@oracle.com> On 04/06/2018 20:59, Roger Riggs wrote: > : > >> >> Are the changes to SocksSocketImpl correct? I may have missed >> something but the original called System.getProperty("user.dir") in a >> privileged block so I'm wondering if getUserNameChecked is needed. > The existing code in SocksSocketImpl is inconsistent with respect to > access to user.name; some flows > use doPriv to access the property and others did not.? If someone > familiar with the Socks networking function > can recommend the proper access, it can be revised.? The intent was to > have the same security checks > as before. The original code at L181 is using GetPropertyAction.privilegedGetProperty so it looks like it reads the value of the property in a privileged block. The replacement code is doing an explicit permission check. If I read the original code correctly then it should only be doing a permission check for the proxy case. So I think it needs to be checked, another set of eyes would be useful. -Alan From ivan.gerasimov at oracle.com Tue Jun 5 06:23:00 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 4 Jun 2018 23:23:00 -0700 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows Message-ID: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> Hello! When a file size is modified via RandomAccessFile.setLength(int) on Windows, it is currently done in three steps: SetFilePointer(), SetEndOfFile(), SetFilePointerEx(). First two can be combined in one call of SetFileInformationByHandle(), which will make the code shorter and more aligned with Unix variant (ftruncate64 is used on Unix systems, which behaves close to SetFileInformationByHandle(_,FileEndOfFileInfo,_)). Would you please help review this trivial fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/00/webrev/ All the existing tests pass Ok. -- With kind regards, Ivan Gerasimov From li.jiang at oracle.com Tue Jun 5 06:30:16 2018 From: li.jiang at oracle.com (li.jiang at oracle.com) Date: Tue, 5 Jun 2018 14:30:16 +0800 Subject: [11] RFR: 8202026 8193552 8204269 : ISO 4217 Amendment #165 # 166 #167 Update In-Reply-To: <875fa321-9b73-17f4-c63e-b542e32b6424@oracle.com> References: <5d50ddec-884c-e78c-4c2c-b9e11d8c3236@oracle.com> <62217aa4-16c5-2ec2-15f7-177d0906c53b@oracle.com> <2099b4c0-b97a-4c1a-f291-fec8f7beb294@oracle.com> <9181f81a-00f4-475c-553f-fa155439fa15@oracle.com> <875fa321-9b73-17f4-c63e-b542e32b6424@oracle.com> Message-ID: <156571a8-d7ab-2c51-39ea-3242e33399f0@oracle.com> Hi Naoto, I removed the obsoleted currency LTL and LVL from tablea1.txt, added them into otherCodes in ValidateISO4217.java. new webrev as below: http://cr.openjdk.java.net/~ljiang/8202026/webrev.03/ Thanks, Leo On 05/06/2018 2:27 AM, naoto.sato at oracle.com wrote: > Hi Leo, > > Change looks good. One leftover from the previous: > > >> in CurrencyData.properties. This applies to tabela1.txt as well. > > Can you please clean those LV/LT entries in tablea1.txt as well? > > Naoto > > On 6/4/18 7:41 AM, li.jiang at oracle.com wrote: >> Hi Naoto, >> >> Pls review the updated code: >> >> http://cr.openjdk.java.net/~ljiang/8202026/webrev.02/ >> >> - As suggested, clean up the old transition dates. >> - Update copyright year. >> - ISO 4217 Amendment #167 was published today, which discards the >> #166, so withdraw the change for #166 in webrev.02. >> >> Bug for #167: >> https://bugs.openjdk.java.net/browse/JDK-8204269 >> >> Test passed on mach5. >> >> Thanks, >> Leo >> >> On 26/04/2018 1:00 AM, naoto.sato at oracle.com wrote: >>> Hi Leo, >>> >>> Although JDK11 is slated in 09/2018, enabling amendment 166 now is >>> technically a bug, as it will be effective from June 4. Please use >>> the transition mechanism in >>> make/data/currency/CurrencyData.properties and >>> test/jdk/java/util/Currency/tablea1.txt. >>> >>> OTOH, there are old (past) transition entries. I would clean up those >>> entries, such as: >>> >>> ??326 # LATVIA >>> ??327 LV=LVL;2013-12-31-22-00-00;EUR >>> >>> in CurrencyData.properties. This applies to tabela1.txt as well. >>> >>> Naoto >>> >>> On 4/24/18 8:52 PM, Leo Jiang wrote: >>>> Forgot to mention, the tests in Currency fold are passed on Mach5. >>>> >>>> -Leo >>>> >>>> On 04/25/2018 09:33 AM, Leo Jiang wrote: >>>>> Hi, >>>>> >>>>> Please review the changes to address the ISO 4217 Amendment 165 166 >>>>> update. >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8193552? 165 >>>>> https://bugs.openjdk.java.net/browse/JDK-8202026? 166 >>>>> >>>>> CR: >>>>> http://cr.openjdk.java.net/~ljiang/8202026/webrev.00/ >>>>> >>>>> >>>>> Detail: >>>>> #165 >>>>> From: >>>>> MAURITANIA??? Ouguiya??? MRO??? 478??? 2 >>>>> To: >>>>> MAURITANIA??? Ouguiya??? MRU??? 929??? 2 >>>>> >>>>> #166 >>>>> From: >>>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var??? VEF??? 937??? 2 >>>>> To: >>>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var Soberano??? VES 928??? 2 >>>>> >>>>> >>>>> Thanks, >>>>> Leo From takiguc at linux.vnet.ibm.com Tue Jun 5 06:59:01 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Tue, 05 Jun 2018 15:59:01 +0900 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <81453d0cbe05477ea558917de263ada2@sap.com> References: <0c73caec8810b9844a463343bdaec064@linux.vnet.ibm.com> <81453d0cbe05477ea558917de263ada2@sap.com> Message-ID: Hello Matthias and Christoph. Thank you for your explanations. I did not have enough knowledge about "visibility". I created following patches. ================================ diff -r 02934b0d661b src/java.base/share/native/libjimage/NativeImageBuffer.cpp --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed May 30 14:46:28 2018 +0200 +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Tue Jun 05 12:10:41 2018 +0900 @@ -39,7 +39,9 @@ #include "imageFile.hpp" #include "inttypes.hpp" #include "jimage.hpp" +#if !defined(_AIX) #include "osSupport.hpp" +#endif #include "jdk_internal_jimage_NativeImageBuffer.h" diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 2018 +0200 +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 2018 +0900 @@ -29,7 +29,8 @@ #ifndef __has_attribute #define __has_attribute(x) 0 #endif -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) #ifdef ARM #define JNIEXPORT __attribute__((externally_visible,visibility("default"))) #define JNIIMPORT __attribute__((externally_visible,visibility("default"))) ================================ If "osSupport.hpp" was included, XLC++ compiler complained. I could not understand, which part was invalid... I'm not sure, "osSupport.hpp" is really required on NativeImageBuffer.cpp or not... I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. [1] 0xD01 means 13.1 by hexadecimal. I checked symbol table by "dump -Tv -X64". [2] It seemed it was fine, some symbols were hidden. Does someone review my code? [1] https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.ibm.xlc1313.aix.doc/compiler_ref/xlmacros.html [2] https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility/index.html On 2018-06-01 21:12, Baesken, Matthias wrote: > Hi , my change 8202322 just handled the fact that the > visibility - flags are not supported with xlc 12.1 , so setting > them generated a TON of compile - time warnings . > > The introduction of the "-qvisibility=hidden" came with the > mapfile removal changes : > > 8200358: Remove mapfiles for JDK executables > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > > 8200178: Remove mapfiles for JDK native libraries > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > > I guess it might need further testing+adjustments to make the > "visibility hiding" work nicely with XLC13 , but currently we > build only with XLC12 . > > As a workaround you might want to remove the "-qvisibility=hidden" > setting for XLC 13 as well , like I did for XLC12 with the change > 8202322 . > > > Best regards, Matthias > > > > >> -----Original Message----- >> From: Langer, Christoph >> Sent: Freitag, 1. Juni 2018 10:57 >> To: Ichiroh Takiguchi >> Cc: Baesken, Matthias ; 'build- >> dev at openjdk.java.net' ; ppc-aix-port- >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, >> Goetz >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support >> on xlc 12.1 >> >> Hi Ichiroh, >> >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe >> nobody of my colleagues has played with it yet. So you are on a new >> playground here ?? >> >> However, I believe the idea in OpenJDK with the abolition of map files >> is that >> symbols should be invisible externally unless they are declared >> exported, >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the >> correct >> default and whatever JNIEXPORT expands to should contain the right >> attributes to get that symbol visible. >> >> Can you check if either my assumption is completely wrong, JNIEXPORT >> does >> not expand to the right thing, XLC 13 has a bug or maybe just sume >> specific >> required symbols are not declared correctly? >> >> Best regards >> Christoph >> >> > -----Original Message----- >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] >> > Sent: Donnerstag, 31. Mai 2018 09:55 >> > To: Langer, Christoph >> > Cc: Baesken, Matthias ; 'build- >> > dev at openjdk.java.net' ; ppc-aix-port- >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, >> > Goetz >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> > >> > Hello. >> > 8202322 was integrated into jdk-11+15. >> > I'm using XLC 13.1.3 on AIX 7.1.4. >> > Build was failed because of "-qvisibility=hidden" on >> > make/lib/LibCommon.gmk. >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], >> > "-qvisibility=hidden" cannot create shared libraries entry points. >> > For example, libverify.so was there, but entry points were not resolved >> > by "-lverify" option. >> > I think it should be "-qvisibility=default" (I tried, it worked) >> > or "-qvisibility=protected" (I had not tried) ? >> > I'm not familiar with -qvisibility option, but I'd like to find out >> > right way. >> > >> > [1] >> > >> https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html >> > >> > On 2018-05-16 16:08, Langer, Christoph wrote: >> > > Hi Matthias, >> > > >> > > yes, reviewed. >> > > >> > > Best regards >> > > Christoph >> > > >> > > From: Baesken, Matthias >> > > Sent: Mittwoch, 16. Mai 2018 09:06 >> > > To: Langer, Christoph ; >> > > 'build-dev at openjdk.java.net' ; >> > > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> > > Cc: Lindenmaier, Goetz >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > > xlc 12.1 >> > > >> > > Hi Christoph can I add you as second reviewer (other reviewer was >> > > Erik Joelsson) can push the change ? >> > > >> > > Best regards, Matthias >> > > >> > > >> > > >> > > From: Langer, Christoph >> > > Sent: Donnerstag, 26. April 2018 16:38 >> > > To: Baesken, Matthias >> > > >; >> > > 'build-dev at openjdk.java.net' >> > > >; >> > > ppc-aix-port-dev at openjdk.java.net> > dev at openjdk.java.net>; >> > > core-libs-dev at openjdk.java.net> dev at openjdk.java.net> >> > > Cc: Simonis, Volker >> > > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > > xlc 12.1 >> > > >> > > Hi Matthias, >> > > >> > > to me the change in principal looks good. >> > > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. >> > > extract major number before the first dot, then compare numerically) - >> > > but maybe it is too complicated and the current single version compare >> > > suits our needs ? >> > > >> > > Best regards >> > > Christoph >> > > >> > > From: Baesken, Matthias >> > > Sent: Donnerstag, 26. April 2018 16:14 >> > > To: 'build-dev at openjdk.java.net' >> > > >; >> > > ppc-aix-port-dev at openjdk.java.net> > dev at openjdk.java.net>; >> > > core-libs-dev at openjdk.java.net> dev at openjdk.java.net> >> > > Cc: Langer, Christoph >> > > >; >> Simonis, >> > > Volker > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> > > 12.1 >> > > >> > > Hello , could you please review this small adjustment to the symbol >> > > visibility compilation settings on AIX ? >> > > Currently we use XLC 12.1 to compile JDK on AIX . >> > > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" >> > > setting currently set on AIX. >> > > It was introduced with XLC 13.1 . Christoph found some info about it >> > > here : >> > > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> > visibility-part2/index.html >> > > >> > > Setting it only generates hundreds of warnings in the build log , >> > > warnings look like this : >> > > XlC12.1 >> > > >> > > bash-4.4$ xlC -qversion >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) >> > > Version: 12.01.0000.0019 >> > > >> > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list >> > > of valid options. >> > > >> > > Compare to XLC13.1 >> > > >> > > bash-3.00$ xlC -qversion >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) >> > > Version: 13.01.0000.0008 >> > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc >> > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> > > >> > > >> > > So it is better to avoid setting these flags when using xlc12.1 . >> > > Please review : >> > > >> > > Bug : >> > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 >> > > >> > > Change : >> > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ >> > > >> > > >> > > Best regards, Matthias From jan.lahoda at oracle.com Tue Jun 5 07:29:01 2018 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Tue, 5 Jun 2018 09:29:01 +0200 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <799bd973-3d11-b4cc-2f16-f4dc6383b4b2@oracle.com> References: <5B0FBC37.2090103@oracle.com> <799bd973-3d11-b4cc-2f16-f4dc6383b4b2@oracle.com> Message-ID: <5B163BBD.9060001@oracle.com> On 4.6.2018 19:40, mandy chung wrote: > Hi Jan, > > On 5/31/18 2:11 AM, Jan Lahoda wrote: >> Hi, >> >> I'd like to upgrade the JOpt Simple library we are using to version >> 5.0.4. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203891 >> Complete webrev: >> http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/complete/ >> >> Delta webrev only showing (all) JDK changes in JOpt Simple and related >> changes in tests needed for the upgrade, etc.: >> http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/joptsimple.delta/ > >> Probably the biggest issue with this upgrade is that for two >> subsequent parameters: >> "--libs=", "/tmp" >> "/tmp" used to be interpreted as the parameter of "libs", but now the >> "libs" parameter is empty (as there's nothing behind the '='). > > This is a reasonable fix and no whitespace is expected following `=`. > >> See the changes to test/jdk/tools/jmod/JmodTest.java for an example. >> Hopefully, this is a reasonable change. > > 114 "--libs=" + libDir.toString(), > > Alternatively, you can take out `=` and run it as jmod --libs /tmp > "--libs", libDir.toString(), Yes, that would work as well, but there are already invocations like this in the test. So I opted for using "--libs=" + ... and similar, so that both variants are covered by the test. Thanks, Jan > > Mandy From Alan.Bateman at oracle.com Tue Jun 5 09:06:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jun 2018 10:06:29 +0100 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> Message-ID: On 04/06/2018 20:59, Roger Riggs wrote: > Hi Alan, > > Updated webrev in place: > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ The splitting of the note into @apiNote and @implNote mostly looks good. I think it should be "cached by the Java virtual machine" rather than "cached internally". It would be good to fix inconsistent line length while you are there. I still think we need to find a better name for SystemProperty and it would be good to re-visit how this is initialized in initPhase1. I think this is Stuart's point too. It would be nice to call it with "props" so that it captures the initial value of the interesting properties. A second best would be to a SystemProperties.capture() or something explicit. There's something not quite right with the change to SocksSocketImpl that I mentioned in the other mail. -Alan From james.laskey at oracle.com Tue Jun 5 12:17:18 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Tue, 5 Jun 2018 09:17:18 -0300 Subject: RFR JDK-8200530: '\r' is not supported as "newline" in java.util.jar.Manifest. In-Reply-To: <5B1621E5.80301@oracle.com> References: <5B1621E5.80301@oracle.com> Message-ID: <514183E1-18ED-499C-B42F-14853BED8966@oracle.com> Attributes.java:380 nit - assign c at decl - only test len if decremented >>>>>> byte c; if ((c = lbuf[--len]) != '\n' && c != '\r') { throw new IOException("line too long"); } if (len > 0 && lbuf[len-1] == '\r') { --len; } if (len == 0) { break; } ====== byte c = lbuf[--len]; if (c != '\n' && c != '\r') { throw new IOException("line too long"); } if (len > 0 && lbuf[len-1] == '\r' && --len == 0) { break; } <<<<<< Manifest.java:208 nit - same as above >>>>>> byte c; if ((c = lbuf[--len]) != '\n' && c != '\r') { throw new IOException("manifest line too long"); } if (len > 0 && lbuf[len-1] == '\r') { --len; } if (len == 0 && skipEmptyLines) { continue; } ====== byte c = lbuf[--len]; if (c != '\n' && c != '\r') { throw new IOException("manifest line too long"); } if (len > 0 && lbuf[len-1] == '\r' && --len == 0 && skipEmptyLines) { continue; } <<<<<< Manifest.java:396 nit - Shouldn?t c already be loaded with tbuf[tpos-1] (or tbuf[tpos-2]) if ?\r\n?) >>>>>> if ((c = tbuf[tpos-1]) == '\n' || c == '\r') { break; } ====== if (c == '\n' || c == '\r') { break; } <<<<<< +1 Cheers, ? Jim > On Jun 5, 2018, at 2:38 AM, Xueming Shen wrote: > > Hi, > > Please help review the changeset for JDK-8200530. > > "newline" is specified as |CR LF | LF | CR|(/not followed by/|LF|) in Jar spec [1] from > the very beginning but our implementation actually never supports "\r"/CR (not > followed by LF) case. The proposed change here is to add CR as an individual > supported "newline"/line separator. > > issue: https://bugs.openjdk.java.net/browse/JDK-8200530 > webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev > > Thanks, > Sherman > > > [1] https://docs.oracle.com/javase/10/docs/specs/jar/jar.html From adam.farley at uk.ibm.com Tue Jun 5 13:46:08 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Tue, 5 Jun 2018 14:46:08 +0100 Subject: RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers Message-ID: Hi All, Native memory allocation for DBBs is tracked in java.nio.Bits, but that only includes what the user thinks they are allocating. When the VM adds extra memory to the allocation amount this extra bit is not represented in the Bits total. A cursory glance shows, minimum, that we round the requested memory quantity up to the heap word size in the Unsafe.allocateMemory code, and something to do with nmt_header_size in os:malloc() (os.cpp) too. On its own, and in small quantities, align_up(sz, HeapWordSize) isn't that big of an issue. But when you allocate a lot of DBBs, and coupled with the nmt_header_size business, it makes the Bits values wrong. The more DBB allocations, the more inaccurate those numbers will be. To get the "+X", it seems to me that the best option would be to introduce an native method in Bits that fetches "X" directly from Hotspot, using the same code that Hotspot uses (so we'd have to abstract-out the Hotspot logic that adds X to the memory quantity). This way, anyone modifying the Hotspot logic won't risk rendering the Bits logic wrong again. That's only one way to fix the accuracy problem here though. Suggestions welcome. Best Regards Adam Farley Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From tprintezis at twitter.com Tue Jun 5 14:09:03 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 5 Jun 2018 07:09:03 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> Message-ID: Hey Alan, Any thoughts on this? (with apologies for the ping) Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 30, 2018 at 5:16:44 PM, Peter Levart (peter.levart at gmail.com) wrote: I thought there would be some hint from Alan about which of the two paths we should take for more refinement. The Tony's: http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ Or the Alan's with my mods: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ Regards, Peter On 05/30/18 22:46, Tony Printezis wrote: Hi all, Any more thoughts on this? (with apologies for the ping) Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 18, 2018 at 3:58:57 PM, Tony Printezis (tprintezis at twitter.com) wrote: Hi again, Stylistically, I strongly prefer this version over the previous one (the one with the doubly-linked list of JdkThreadLocal entries) you had posted. This one is a lot cleaner. A few suggestions: * I?m not crazy about the fact that the users have to override calculateInitialValue() instead of initialValue() as it will be a bit error-prone. If they accidentally override initialValue() then the per-thread registering is not going to happen. Maybe I?m overthinking this. One way to get round this is for each JdkThreadLocal instance to delegate calls to get(), set(), and remove() to a private ThreadLocal instance, which can in turn delegate initialValue() to the outer instance. (Hope this makes sense?) I did a quick prototype of this and it seems to work, but I didn?t heavily test it. * I would prefer if the method users override actually took the thread-local value as a parameter, i.e., protected void threadTerminated(T value). This is very easy to add: protected void threadTerminated(T value) { } private void threadTerminated() { threadTerminated(get()); } and I think it will be easier to use, as most uses will probably need to do get() anyway. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 18, 2018 at 4:23:19 AM, Peter Levart (peter.levart at gmail.com) wrote: It's good to have alternative implementations on the table. Here's another variant that is space neutral for threads that don't use JdkThreadLocal, but also scales to many JdkThreadLocal instances: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ Now we only need an arbiter to decide which one! :-) Regards, Peter On 05/17/18 22:39, Peter Levart wrote: Hi Tony, If we anticipate only small number of JdkThreadLocal(s) (there will be only one for the start) then this is a plausible solution. Then perhaps this limitation should be written somewhere in the JdkThreadLocal javadoc so that one day somebody will not be tempted to use JdkThreadLocal(s) en masse. Let's say there will be a few more JdkThreadLocal(s) in the future. Are we willing to pay for a few lookups into a ThreadLocalMap at each and every thread's exit even though such thread did not register a mapping for any JdkThreadLocal? Is an additional reference field in each and every ThreadLocalMap (one per Thread that uses thread locals) a bigger price to pay? I don't know. Will let others comment on this. Otherwise the code looks good. Just a couple of observations: Since private static method JdkThreadLocal.add is only called from JdkThreadLocal constructor with just constructed instance (this), there's no possibility for it to be called twice or more times with the same instance. The check for duplicates could go away then, right? You keep an array of Entry objects which are just wrappers for JdkThreadLocal objects. Are you already planning for Entry to become a WeakReference? Otherwise you could just keep JdkThreadLocal objects in the array directly. Regards, Peter On 05/17/18 20:25, Tony Printezis wrote: Hi all, I have a new version of the code for your consideration: http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ I basically combined our two approaches. The usage is as Alan had proposed it: Users have to use JdkThreadLocal (which is only available within java.base) and override threadTerminated(). However, I keep track of JdkThreadLocal instances globally (as I did before) and not per-thread. This way we don?t need to add any unnecessary complexity to ThreadLocalMap. Currently, I don?t allow entries to be purged if the JdkThreadLocal instance becomes otherwise unreachable. I can easily add that functionality if needed (I can use WeakReferences for that). However, for the uses we?re considering, is it really necessary? Thoughts? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 14, 2018 at 12:40:28 PM, Tony Printezis (tprintezis at twitter.com) wrote: Peter, In my proposal, you can register the exit hook in the ThreadLocal c?tor, so it?s almost as nice as Alan?s in that respect (and it doesn't require an extra field per ThreadLocal plus two extra fields per JdkEntry). :-) But, I do like the addition of the JdkEntry list to avoid unnecessarily iterating over all the map entries (which was my main concern with Alan?s original webrev). I?ll be totally happy with a version of this. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 12, 2018 at 6:44:08 AM, Peter Levart (peter.levart at gmail.com) wrote: Hi, On 05/11/18 16:13, Alan Bateman wrote: On 08/05/2018 16:07, Tony Printezis wrote: Hi all, Following the discussion on this a few weeks ago, here?s the first version of the change: http://cr.openjdk.java.net/~tonyp/8202788/webrev.0/ I think the consensus was that it?d be easier if the exit hooks were only available within java.base. Is it enough that I added the functionality to the jdk.internal.misc package? (And is jdk.internal.misc the best place for this?) I?ll also add a test for the new functionality. But let?s first come up with an approach that everyone is happy with. :-) Peter's approach in early April was clean (and we should come to the getIfPresent discussion) but it adds a field to Thread for the callback list. If I read your approach correctly, you are avoiding that by maintaining an array of hooks in ThreadLocalExitHooks. Another approach to try is a java.base-internal ThreadLocal that defines a method to be invoked when a thread terminates. Something like the following: http://cr.openjdk.java.net/~alanb/8202788/webrev/index.html -Alan >From the API perspective, Alan's suggestion is the most attractive one as it puts the method to where it belongs - into the ThreadLocal instance. But the implementation could be improved a bit. I don't like the necessity to iterate over all ThreadLocal(s) to filter out JdkThreadLocal(s). There might be a situation where there's plenty of ThreadLocal(s) registered per exiting thread which would produce a spike in CPU processing at thread exit. The way to avoid this would be to maintain a separate linked list of entries that contains just those with JdkThreadLocal(s). Like in this modification of Alan's patch, for example: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.01/ (Only ThreadLocal class is modified from Alan's patch) What do you think? Regards, Peter From Roger.Riggs at Oracle.com Tue Jun 5 14:18:23 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 10:18:23 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com> References: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com> Message-ID: Hi Stuart, On 6/4/2018 9:52 PM, Stuart Marks wrote: > > > On 6/4/18 6:32 AM, Roger Riggs wrote: >> Please review a change to make the values of java.home, user.home, >> user.dir, and user.name >> effectively read-only for internal use.? The values are cached during >> initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> Issue: >> ?? https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> ?? https://bugs.openjdk.java.net/browse/JDK-8204235 > > Hi Roger, > > Overall I think the intent of the change is a good one, as is the > reimplementation of internal uses of system properties to use an > internal API instead. > > I think it would be good to have an explicit initialization of the > SystemProperty class (or whatever it ends up being called) instead of > implicitly initializing via a static initializer. If the class is > initialized too early, for some reason, things would go wrong. > That would be a no-op;? I want the values of the properties to be final statics which means they have to be initialized by the static initializer. > Should there be an error, a warning, or assertion checking to make > sure that none of the cached values are null? They should all be > non-null, right? Yes, null will be a fatal InternalError. Thanks, Roger > > s'marks From Roger.Riggs at Oracle.com Tue Jun 5 14:57:03 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 10:57:03 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <8D9FFD19-4501-46A8-9E7E-D3A622CF81EC@oracle.com> <4dcfbb49-a0dc-e69f-d44b-2faa2189c5de@Oracle.com> Message-ID: Hi Max, On 6/4/2018 7:24 PM, Weijun Wang wrote: > Hi Roger > > Thanks for the explanation. > > Another question: Before this change, a SecurityException might be thrown when getProperty() is called, does you new code simulate this behavior? Or in these cases this is unnecessary? Many of the usages were already in doPriv or GetPropertyAction flows so no security check was being done. In a few cases, the existing doPriv is in a different file (e.g. calls to SunEntries.getDeviceFile(utl)). Those would be safer to leave as is, trading off the consistency with confidence about the property check (or add the property access check). The usage in SocksSocketImpl needs a second look, Alan pointed out a difference in behavior. The use of user.name in MimeTable and MailToURLConnection were missing a security check. > Also, looks like all change is inside java.base, do you have a suggestion we use it in other JDK modules? Not at this time. Roger > > Thanks > Max > >> On Jun 5, 2018, at 3:54 AM, Roger Riggs wrote: >> >> Hi Max, >> >> On 6/4/2018 11:41 AM, Wang Weijun wrote: >>> Not a native English speaker, so my feeling might be incorrect. >>> >>> Will someone interpret this as that System.getProperty() will return a cached value? >>> >> I don't think so, it should be clear that the values are cached at initialization or first use. >>> I would say ?Although getProperty() always returns the last value set by setProperty() (I assume this is the current behavior), it is not uncommon that consumers of a system property may read it once and cache the value it for later use, which means setting the property may not have the desired effect?. >>> >> The purpose of the change is to clarify the behavior of the java.base module internal access to the properties. >> How applications handle properties is beyond the intended scope. >>> I also don?t think it?s worth listing the 4 property names in the spec. Quite some other system properties are also cached. Listing them there could further suggests that calling getProperty() on them will return the cached value. >>> >> It seemed worthwhile to highlight the specific properties affected and the release note should be specific. >> I moved them to an implNote as Alan suggested. >> >> Thanks, Roger >> >>> Thanks >>> Max >>> >>> >>>> ? 2018?6?4????9:32?Roger Riggs >>>> ??? >>>> >>>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>>> effectively read-only for internal use. The values are cached during initialization and the >>>> cached values are used. >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>>> >>>> >>>> Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>>> >>>> >>>> CSR: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>>> >>>> >>>> Thanks, Roger >>>> >>>> >>>> From Roger.Riggs at Oracle.com Tue Jun 5 15:01:22 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 11:01:22 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <12B561C2-8EE6-4DAE-A352-D53ED485F53C@oracle.com> References: <647D84B5-F364-46F4-A80B-8854A3F889B1@oracle.com> <12B561C2-8EE6-4DAE-A352-D53ED485F53C@oracle.com> Message-ID: Hi Lance, The "unless otherwise specified" part is important so reworeded as: ???? * Changing any standard system property may have unpredictable results ???? * unless otherwise specified. Thanks, Roger On 6/4/2018 4:39 PM, Lance Andersen wrote: > Hi Roger, > > > > In System.java > > ????? > changing any standard system property may have unpredictable > effects. > > ???????? > > Perhaps capitalize ?changing' and maybe consider ?results? vs ?effects? > > > > HTH > > Best > Lance >> On Jun 4, 2018, at 3:55 PM, Roger Riggs > > wrote: >> >> Hi Lance, >> >> On 6/4/2018 12:09 PM, Lance Andersen wrote: >>> Hi Roger, >>> >>> Overall, it looks very good. >>> >>> A couple of quick thoughts, nothing that needs any action unless you >>> feel the desire :-) >>> >>> Line 671 in System.java, it mentions ?standard? system properties. >>> ?(as does the existing Implementation note) but it does not really >>> specify what a standard system property is (though it can be >>> assumed) in getProperties overview. ?Not a big deal, just thought I >>> would point it out in case you had a alternative thought. >> Yes, it is a bit vague and the documentation of properties is spread >> over many packages and classes. >> 'Standard' generally refers to any property defined in the scope of >> Java SE or the documented JDK properties. >> (For example, those subject to the CSR process). >>> >>> Do you think we should had a mention in getProperty() about this >>> cached properties (or do we think the reference to getProperties() >>> is enough) >> Hard to say,? there will be a release note, but the properties >> methods are well known and it is >> likely no one reads the javadoc anymore.? getProperty delegates to >> getProperties for most of its behavior. >> >> Adding a repeat of the API note to System.setProperties and >> System.setProperty might be more to the point, >> warning against setting standard system properties via the API. >> I added the apinote to each of the 6 methods for setting, clearing, >> or getting properties. If that seems excessive, >> I'm open to removing it from those that provide the least value. >> >> Thanks, Roger >> >>> >>> Best >>> Lance >>>> On Jun 4, 2018, at 9:32 AM, Roger Riggs >>> > wrote: >>>> >>>> Please review a change to make the values of java.home, user.home, >>>> user.dir, and user.name >>>> effectively read-only for internal use.? The values are cached >>>> during initialization and the >>>> cached values are used. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>>> >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>>> >>>> CSR: >>>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>>> >>>> Thanks, Roger >>>> >>>> >>> >>> >>> >>> >>> Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle?Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From chris.hegarty at oracle.com Tue Jun 5 15:05:42 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 5 Jun 2018 16:05:42 +0100 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <5B0FBC37.2090103@oracle.com> References: <5B0FBC37.2090103@oracle.com> Message-ID: <18b569fb-e7ab-f0db-e30e-51e7ef299aea@oracle.com> On 31/05/18 10:11, Jan Lahoda wrote: > Hi, > > I'd like to upgrade the JOpt Simple library we are using to version 5.0.4. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203891 > Complete webrev: > http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/complete/ > > Delta webrev only showing (all) JDK changes in JOpt Simple and related > changes in tests needed for the upgrade, etc.: > http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/joptsimple.delta/ > > Probably the biggest issue with this upgrade is that for two subsequent > parameters: > "--libs=", "/tmp" > "/tmp" used to be interpreted as the parameter of "libs", but now the > "libs" parameter is empty (as there's nothing behind the '='). See the > changes to test/jdk/tools/jmod/JmodTest.java for an example. Hopefully, > this is a reasonable change. > > How does this look? Thanks Jan, I think this looks good. -Chris. From Roger.Riggs at Oracle.com Tue Jun 5 15:10:09 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 11:10:09 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <8a1ad331-411b-3e0b-58d1-36f1caad3dac@oracle.com> References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> <8a1ad331-411b-3e0b-58d1-36f1caad3dac@oracle.com> Message-ID: <1cb3e4f9-da59-78d5-a8ff-049d1bed7f4c@Oracle.com> Hi Alan, On 6/5/2018 2:19 AM, Alan Bateman wrote: > On 04/06/2018 20:59, Roger Riggs wrote: >> : >> >>> >>> Are the changes to SocksSocketImpl correct? I may have missed >>> something but the original called System.getProperty("user.dir") in >>> a privileged block so I'm wondering if getUserNameChecked is needed. >> The existing code in SocksSocketImpl is inconsistent with respect to >> access to user.name; some flows >> use doPriv to access the property and others did not.? If someone >> familiar with the Socks networking function >> can recommend the proper access, it can be revised.? The intent was >> to have the same security checks >> as before. > The original code at L181 is using > GetPropertyAction.privilegedGetProperty so it looks like it reads the > value of the property in a privileged block. The replacement code is > doing an explicit permission check. If I read the original code > correctly then it should only be doing a permission check for the > proxy case. So I think it needs to be checked, another set of eyes > would be useful. Yes, that case did not need the propertyAccess check. (The local getUsername method could be simplified since applicationSetProxy can never be true (assuming no reflection setting it)). Thanks, Roger > > -Alan From Roger.Riggs at Oracle.com Tue Jun 5 15:14:20 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 11:14:20 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> Message-ID: <73a07938-6e9f-03e8-8919-f1ee3836ffff@Oracle.com> Hi Alan, On 6/5/2018 5:06 AM, Alan Bateman wrote: > On 04/06/2018 20:59, Roger Riggs wrote: >> Hi Alan, >> >> Updated webrev in place: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > The splitting of the note into @apiNote and @implNote mostly looks > good. I think it should be "cached by the Java virtual machine" rather > than "cached internally". It would be good to fix inconsistent line > length while you are there. I'm not keen on the abuse of 'Java Virtual Machine' to mean the entire java runtime. These values are part of the runtime state, not specifically the JVM. > > I still think we need to find a better name for SystemProperty and it > would be good to re-visit how this is initialized in initPhase1. I can suggest StaticProperties? (but I think it will change back in a future enhancement). > I think this is Stuart's point too. It would be nice to call it with > "props" so that it captures the initial value of the interesting > properties. A second best would be to a SystemProperties.capture() or > something explicit. The values should be static final Strings so they need to be initialized in the static initializer. The code can be made more complicated by adding another class but that does not seem warranted. System initialization code is know to be sensitive to ordering, etc. Thanks, Roger > > There's something not quite right with the change to SocksSocketImpl > that I mentioned in the other mail. > > -Alan From xueming.shen at oracle.com Tue Jun 5 15:46:51 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 05 Jun 2018 08:46:51 -0700 Subject: RFR JDK-8200530: '\r' is not supported as "newline" in java.util.jar.Manifest. In-Reply-To: <514183E1-18ED-499C-B42F-14853BED8966@oracle.com> References: <5B1621E5.80301@oracle.com> <514183E1-18ED-499C-B42F-14853BED8966@oracle.com> Message-ID: <5B16B06B.6050109@oracle.com> Thanks Jim! webrev has been updated as suggested. http://cr.openjdk.java.net/~sherman/8200530/webrev -sherman On 06/05/2018 05:17 AM, Jim Laskey wrote: > Attributes.java:380 nit > > - assign c at decl > - only test len if decremented > > byte c; > if ((c = lbuf[--len]) != '\n'&& c != '\r') { > throw new IOException("line too long"); > } > if (len> 0&& lbuf[len-1] == '\r') { > --len; > } > if (len == 0) { > break; > } > ====== > byte c = lbuf[--len]; > if (c != '\n'&& c != '\r') { > throw new IOException("line too long"); > } > if (len> 0&& lbuf[len-1] == '\r'&& --len == 0) { > break; > } > <<<<<< > > > Manifest.java:208 nit > > - same as above > > byte c; > if ((c = lbuf[--len]) != '\n'&& c != '\r') { > throw new IOException("manifest line too long"); > } > if (len> 0&& lbuf[len-1] == '\r') { > --len; > } > if (len == 0&& skipEmptyLines) { > continue; > } > ====== > byte c = lbuf[--len]; > if (c != '\n'&& c != '\r') { > throw new IOException("manifest line too long"); > } > if (len> 0&& lbuf[len-1] == '\r'&& --len == 0&& skipEmptyLines) { > continue; > } > <<<<<< > > Manifest.java:396 nit > > - Shouldn?t c already be loaded with tbuf[tpos-1] (or tbuf[tpos-2]) if ?\r\n?) > > if ((c = tbuf[tpos-1]) == '\n' || c == '\r') { > break; > } > ====== > if (c == '\n' || c == '\r') { > break; > } > <<<<<< > > > +1 > > Cheers, > > ? Jim > > >> On Jun 5, 2018, at 2:38 AM, Xueming Shen wrote: >> >> Hi, >> >> Please help review the changeset for JDK-8200530. >> >> "newline" is specified as |CR LF | LF | CR|(/not followed by/|LF|) in Jar spec [1] from >> the very beginning but our implementation actually never supports "\r"/CR (not >> followed by LF) case. The proposed change here is to add CR as an individual >> supported "newline"/line separator. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8200530 >> webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev >> >> Thanks, >> Sherman >> >> >> [1] https://docs.oracle.com/javase/10/docs/specs/jar/jar.html From naoto.sato at oracle.com Tue Jun 5 15:48:22 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 5 Jun 2018 08:48:22 -0700 Subject: [11] RFR: 8202026 8193552 8204269 : ISO 4217 Amendment #165 # 166 #167 Update In-Reply-To: <156571a8-d7ab-2c51-39ea-3242e33399f0@oracle.com> References: <5d50ddec-884c-e78c-4c2c-b9e11d8c3236@oracle.com> <62217aa4-16c5-2ec2-15f7-177d0906c53b@oracle.com> <2099b4c0-b97a-4c1a-f291-fec8f7beb294@oracle.com> <9181f81a-00f4-475c-553f-fa155439fa15@oracle.com> <875fa321-9b73-17f4-c63e-b542e32b6424@oracle.com> <156571a8-d7ab-2c51-39ea-3242e33399f0@oracle.com> Message-ID: <37c4fc7a-a6ac-0dc4-be12-a5f7f764e743@oracle.com> Looks good. Naoto On 6/4/18 11:30 PM, li.jiang at oracle.com wrote: > Hi Naoto, > > I removed the obsoleted currency LTL and LVL from tablea1.txt, added > them into otherCodes in ValidateISO4217.java. > > new webrev as below: > http://cr.openjdk.java.net/~ljiang/8202026/webrev.03/ > > Thanks, > Leo > > On 05/06/2018 2:27 AM, naoto.sato at oracle.com wrote: >> Hi Leo, >> >> Change looks good. One leftover from the previous: >> >> ?>> in CurrencyData.properties. This applies to tabela1.txt as well. >> >> Can you please clean those LV/LT entries in tablea1.txt as well? >> >> Naoto >> >> On 6/4/18 7:41 AM, li.jiang at oracle.com wrote: >>> Hi Naoto, >>> >>> Pls review the updated code: >>> >>> http://cr.openjdk.java.net/~ljiang/8202026/webrev.02/ >>> >>> - As suggested, clean up the old transition dates. >>> - Update copyright year. >>> - ISO 4217 Amendment #167 was published today, which discards the >>> #166, so withdraw the change for #166 in webrev.02. >>> >>> Bug for #167: >>> https://bugs.openjdk.java.net/browse/JDK-8204269 >>> >>> Test passed on mach5. >>> >>> Thanks, >>> Leo >>> >>> On 26/04/2018 1:00 AM, naoto.sato at oracle.com wrote: >>>> Hi Leo, >>>> >>>> Although JDK11 is slated in 09/2018, enabling amendment 166 now is >>>> technically a bug, as it will be effective from June 4. Please use >>>> the transition mechanism in >>>> make/data/currency/CurrencyData.properties and >>>> test/jdk/java/util/Currency/tablea1.txt. >>>> >>>> OTOH, there are old (past) transition entries. I would clean up >>>> those entries, such as: >>>> >>>> ??326 # LATVIA >>>> ??327 LV=LVL;2013-12-31-22-00-00;EUR >>>> >>>> in CurrencyData.properties. This applies to tabela1.txt as well. >>>> >>>> Naoto >>>> >>>> On 4/24/18 8:52 PM, Leo Jiang wrote: >>>>> Forgot to mention, the tests in Currency fold are passed on Mach5. >>>>> >>>>> -Leo >>>>> >>>>> On 04/25/2018 09:33 AM, Leo Jiang wrote: >>>>>> Hi, >>>>>> >>>>>> Please review the changes to address the ISO 4217 Amendment 165 >>>>>> 166 update. >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8193552? 165 >>>>>> https://bugs.openjdk.java.net/browse/JDK-8202026? 166 >>>>>> >>>>>> CR: >>>>>> http://cr.openjdk.java.net/~ljiang/8202026/webrev.00/ >>>>>> >>>>>> >>>>>> Detail: >>>>>> #165 >>>>>> From: >>>>>> MAURITANIA??? Ouguiya??? MRO??? 478??? 2 >>>>>> To: >>>>>> MAURITANIA??? Ouguiya??? MRU??? 929??? 2 >>>>>> >>>>>> #166 >>>>>> From: >>>>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var??? VEF??? 937??? 2 >>>>>> To: >>>>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var Soberano??? VES >>>>>> 928??? 2 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Leo From thomas.stuefe at gmail.com Tue Jun 5 16:10:02 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 5 Jun 2018 18:10:02 +0200 Subject: RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: Message-ID: On Tue, Jun 5, 2018 at 3:46 PM, Adam Farley8 wrote: > Hi All, > > Native memory allocation for DBBs is tracked in java.nio.Bits, but that > only includes what the user thinks they are allocating. > Which is exactly what I would expect as a user... > When the VM adds extra memory to the allocation amount this extra bit is > not represented in the Bits total. A cursory glance > shows, minimum, that we round the requested memory quantity up to the heap > word size in the Unsafe.allocateMemory code which I do not understand either - why do we do this? After all, normal allocations from inside hotspot do not get aligned up in size, and the java doc to Unsafe allocateMemory does not state anything about the size being aligned. In addition to questioning the align up of the user requested size, I would be in favor of adding a new NMT tag for these, maybe "mtUnsafe"? That would be an easy fix. >, and > something to do with nmt_header_size in os:malloc() (os.cpp) too. That is mighty unspecific and also wrong. The align-up mentioned above goes into the size reported by Bits; the nmt header size does not. > > On its own, and in small quantities, align_up(sz, HeapWordSize) isn't that > big of an issue. But when you allocate a lot of DBBs, > and coupled with the nmt_header_size business, it makes the Bits values > wrong. The more DBB allocations, the more inaccurate those > numbers will be. To be annoyingly precise, it will never be more wrong than 1:7 on 64bit machines :) - if all memory requested via Unsafe.allocateMemory would be of size 1 byte. > > To get the "+X", it seems to me that the best option would be to introduce > an native method in Bits that fetches "X" directly > from Hotspot, using the same code that Hotspot uses (so we'd have to > abstract-out the Hotspot logic that adds X to the memory > quantity). This way, anyone modifying the Hotspot logic won't risk > rendering the Bits logic wrong again. I don't follow that. > > That's only one way to fix the accuracy problem here though. Suggestions > welcome. You are throwing two effects together: - As mentioned above, I consider the align-up of the user requested size to be at least questionable. It shows up as user size in NMT which should not be. I also fail to see a compelling reason for it, but maybe someone else can enlighten me. - But anything else - NMT headers, overwriter guards, etc added by the VM I consider in the same class as any other overhead incurred e.g. by the CRT or the OS when calling malloc (e.g. malloc allocator bucket size). Basically, rss will go up by more than size requested by malloc. Something maybe worth noting, but IMHO not as part of the numbers returned by java.nio.Bits. Just my 2 cents. Best Regards, Thomas > > Best Regards > > Adam Farley > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From martinrb at google.com Tue Jun 5 16:40:14 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 5 Jun 2018 09:40:14 -0700 Subject: RFR 8203768 : Avoid reallocation in java.base/unix/classes/java/lang/ProcessImpl.java In-Reply-To: References: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> Message-ID: On Mon, Jun 4, 2018 at 10:31 PM, Ivan Gerasimov wrote: > But suppose available() incorrectly claims that a million bytes are > available, but you only read 10. Then there's a tradeoff - the cost of > reallocation versus the risk that the mostly empty backing array will be > retained for a long time if the Process object is not garbage collected. > > On the other hand, if available() claims a million bytes, and then only > 999999990 bytes were read, there will be mostly unnecessary allocation and > copying. > > Yup. > I'm leaning towards the status quo. > > Okay, let's leave it as is :) > It was meant mostly for cleaning up the code, so if there is a doubt, then > it's better keep what we have. > If your version were the status quo, it would not be worth changing. When in doubt, status quo wins. From naoto.sato at oracle.com Tue Jun 5 17:14:10 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 5 Jun 2018 10:14:10 -0700 Subject: RFR JDK-8203872: Upgrading JDK with latest available LSR data from IANA In-Reply-To: <6aa57b43-c15e-cfee-cb32-922e4eec8489@oracle.com> References: <6aa57b43-c15e-cfee-cb32-922e4eec8489@oracle.com> Message-ID: <623bc32c-5a6e-b4a4-985b-d832e17af203@oracle.com> Looks good. Naoto On 6/4/18 10:13 PM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8203872. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203872 > Webrev: http://cr.openjdk.java.net/~nishjain/8203872/webrev.01/ > > Fix: Updated the LSR data to the version 2017-08-15 > > Regards, > Nishit Jain From mandy.chung at oracle.com Tue Jun 5 17:16:51 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 5 Jun 2018 10:16:51 -0700 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <5B163BBD.9060001@oracle.com> References: <5B0FBC37.2090103@oracle.com> <799bd973-3d11-b4cc-2f16-f4dc6383b4b2@oracle.com> <5B163BBD.9060001@oracle.com> Message-ID: <308ce43a-d585-dc92-9bba-5873780ee555@oracle.com> On 6/5/18 12:29 AM, Jan Lahoda wrote: > > Yes, that would work as well, but there are already invocations like > this in the test. So I opted for using > "--libs=" + ... > and similar, so that both variants are covered by the test. That is fine to keep it as is. The change looks good. Mandy From roger.riggs at oracle.com Tue Jun 5 21:54:45 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 17:54:45 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: References: Message-ID: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> Hi Lance, Can the name change be done using? hg rename to preserve the continuity? Also, while you are there, how about converting to {@code...} etc. Thanks, Roger On 6/4/18 7:22 AM, Lance Andersen wrote: > Hi, > > Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html > > As part of this fix, I took the liberty to move from package.html to package-info.java > > The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ > > I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java > > Best > Lance > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > From martinrb at google.com Tue Jun 5 22:41:45 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 5 Jun 2018 15:41:45 -0700 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: JDK-8204375: Add TimeUnit#convert(Duration) From stuart.marks at oracle.com Wed Jun 6 00:05:33 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 5 Jun 2018 17:05:33 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: [adding serviceability-dev] Hi serviceability folks, I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) from the Java SE API. Alan and David have pointed out that there are some cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the JDI ThreadReference API. I've adjusted the relevant files. See please review this updated webrev: http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ For context, please see the full review thread on core-libs-dev: http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html On 6/4/18 11:11 PM, Alan Bateman wrote: > The source file that is used to generate the JDWP protocol code and the spec is > in make/data/jdwp/jdwp.spec. The JVM TI spec is at > src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for this > change-set, assuming it is followed up quickly with another change-set to update > those specs. OK. I took a look at those other files and they seem simple enough to update as part of this changeset. If there aren't any objections from anyone, might as well get this all done at once. Thanks, s'marks From zgu at redhat.com Wed Jun 6 00:58:18 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 5 Jun 2018 20:58:18 -0400 Subject: RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: Message-ID: <9c1af95d-2c0f-b96d-76a7-5b40228996de@redhat.com> On 06/05/2018 12:10 PM, Thomas St?fe wrote:> On Tue, Jun 5, 2018 at 3:46 PM, Adam Farley8 wrote: >> Hi All, >> >> Native memory allocation for DBBs is tracked in java.nio.Bits, but that >> only includes what the user thinks they are allocating. >> > > Which is exactly what I would expect as a user... > I agree with Thomas, there is no point for a user to aware of tracking overhead, and the overhead only incurs when native memory tracking is on. As a matter of fact, it can really confuse user that values can be varied, depending on whether native memory tracking is on. Thanks, -Zhengyu >> When the VM adds extra memory to the allocation amount this extra bit is >> not represented in the Bits total. A cursory glance >> shows, minimum, that we round the requested memory quantity up to the heap >> word size in the Unsafe.allocateMemory code > > which I do not understand either - why do we do this? After all, > normal allocations from inside hotspot do not get aligned up in size, > and the java doc to Unsafe allocateMemory does not state anything > about the size being aligned. > > In addition to questioning the align up of the user requested size, I > would be in favor of adding a new NMT tag for these, maybe "mtUnsafe"? > That would be an easy fix. > >> , and >> something to do with nmt_header_size in os:malloc() (os.cpp) too. > > That is mighty unspecific and also wrong. The align-up mentioned above > goes into the size reported by Bits; the nmt header size does not. > >> >> On its own, and in small quantities, align_up(sz, HeapWordSize) isn't that >> big of an issue. But when you allocate a lot of DBBs, >> and coupled with the nmt_header_size business, it makes the Bits values >> wrong. The more DBB allocations, the more inaccurate those >> numbers will be. > > To be annoyingly precise, it will never be more wrong than 1:7 on > 64bit machines :) - if all memory requested via Unsafe.allocateMemory > would be of size 1 byte. > >> >> To get the "+X", it seems to me that the best option would be to introduce >> an native method in Bits that fetches "X" directly >> from Hotspot, using the same code that Hotspot uses (so we'd have to >> abstract-out the Hotspot logic that adds X to the memory >> quantity). This way, anyone modifying the Hotspot logic won't risk >> rendering the Bits logic wrong again. > > I don't follow that. > >> >> That's only one way to fix the accuracy problem here though. Suggestions >> welcome. > > You are throwing two effects together: > > - As mentioned above, I consider the align-up of the user requested > size to be at least questionable. It shows up as user size in NMT > which should not be. I also fail to see a compelling reason for it, > but maybe someone else can enlighten me. > > - But anything else - NMT headers, overwriter guards, etc added by the > VM I consider in the same class as any other overhead incurred e.g. by > the CRT or the OS when calling malloc (e.g. malloc allocator bucket > size). Basically, rss will go up by more than size requested by > malloc. Something maybe worth noting, but IMHO not as part of the > numbers returned by java.nio.Bits. > > Just my 2 cents. > > Best Regards, Thomas > >> >> Best Regards >> >> Adam Farley >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From david.holmes at oracle.com Wed Jun 6 03:48:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Jun 2018 13:48:30 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <33e2e693-1b5e-5e4f-cfd0-a8d5140b0eba@oracle.com> Hi Stuart, This all looks fine to me. One minor nit in threadPrimitiveDeprecation.html is whether the new statements you reference a particular Java version? It reads a little oddly to go through all the explanation and then say "Thread.x has been removed". In the spirit of @since you might say "has been removed as of JDK 11". Or arguably you could remove discussion of the removed methods altogether. Thanks, David On 6/06/2018 10:05 AM, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have pointed > out that there are some cross-references to Thread.stop(Throwable) in > the JDWP and JVMTI specs, as well as in the JDI ThreadReference API. > I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ > > For context, please see the full review thread on core-libs-dev: > > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html > > > > On 6/4/18 11:11 PM, Alan Bateman wrote: >> The source file that is used to generate the JDWP protocol code and >> the spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for >> this change-set, assuming it is followed up quickly with another >> change-set to update those specs. > > OK. I took a look at those other files and they seem simple enough to > update as part of this changeset. If there aren't any objections from > anyone, might as well get this all done at once. > > Thanks, > > s'marks From turbanoff at gmail.com Tue Jun 5 20:17:35 2018 From: turbanoff at gmail.com (=?UTF-8?B?0JDQvdC00YDQtdC5INCi0YPRgNCx0LDQvdC+0LI=?=) Date: Tue, 5 Jun 2018 23:17:35 +0300 Subject: Implementations of PrimitiveIterator.OfInt#forEachRemaining(IntConsumer) doesn't check IntConsumer for null Message-ID: Hello. I observe that implementations of `forEachRemaining` method inside java.lang.CharSequence doesn't always compare IntConsumer parameter with null. Is there any reason why there is no nonNull check? Performance? Andrey Turbanov. From deepak.kejriwal at oracle.com Wed Jun 6 07:07:09 2018 From: deepak.kejriwal at oracle.com (Deepak Kejriwal) Date: Wed, 6 Jun 2018 00:07:09 -0700 (PDT) Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: References: Message-ID: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> Hi Martin, ? Backporting entire tck directory would be over kill as most of files under tck were checked in as part of ?https://bugs.openjdk.java.net/browse/JDK-8146467 and are not really related to JDK-8186171. May be down the line if needed we can invest time to address it. ? The scope Bug8186171Test.java was to test the fix for scenario mentioned in JDK-8186171. Regards, Deepak ? From: Martin Buchholz Sent: Tuesday, June 5, 2018 5:35 AM To: Deepak Kejriwal ; Doug Lea

Cc: core-libs-dev Subject: Re: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries ? Hej Deepak, ? This looks alright, but you really need to add that trailing newline on the test file (a Martin pet peeve). I wonder if instead we invest a little more work, but backport the entire tck directory. All tests should pass except for those that test features not yet backported. ? On Mon, Jun 4, 2018 at 4:46 AM, Deepak Kejriwal wrote: Hi all, Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/browse/JDK-8186171 Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.00/ Summary(also added to backport bug description): The back port for test files is not clean back port as all tests files are extending JSR166TestCase.java which was added to JDK 9 as part of HYPERLINK "https://bugs.openjdk.java.net/browse/JDK-8146467"JDK-8146467: Integrate JSR 166 jck tests into JDK repo. This is not present in JDK8 version. Therefore, I have extracted test are relevant to the fix done JDK-8186171 and created a new test file Bug8186171Test.java that contains below two test methods: .? ? ? ? ?testBug8186171NonDeterministic : This method is copy of "testBug8186171" present in "MapTest.java" of original changeset http://hg.openjdk.java.net/jdk10/master/rev/3f5f9bc0bdc2. As it is based on randomization and runs 1000 times, the method name is suffixed with "NonDeterministic". .? ? ? ? ?testBug8186171Deterministic : This is a new method that runs single time as it produces the exact scenario mentioned in JDK-8186171. Therefore, it is not needed to run multiple times to produce the scenario mentioned in bug. Hence the method name is suffixed with "Deterministic" Regards, Deepak ? From Alan.Bateman at oracle.com Wed Jun 6 10:40:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jun 2018 11:40:24 +0100 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> On 06/06/2018 01:05, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have > pointed out that there are some cross-references to > Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the > JDI ThreadReference API. I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ Have you considered removing the "What about Thread.stop(Throwable)" section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. -Alan From Alan.Bateman at oracle.com Wed Jun 6 13:37:56 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jun 2018 14:37:56 +0100 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> Message-ID: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> On 30/05/2018 22:16, Peter Levart wrote: > I thought there would be some hint from Alan about which of the two > paths we should take for more refinement. > > The Tony's: > > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ > > Or the Alan's with my mods: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ > Finally getting back to this one. I don't think the two approaches are all that different now. Tony's point on the number of hooks vs. number of locals was an important point but with Peter's update to use a thread local registry then I think we have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. So I think I prefer that approach. We need to better name for "JdkThreadLocal", something to indicate that it holds resources or it notified when a thread terminates. Also along the way, we touched on exposing getIfPresent and we should look at that again. If it is expose then it fits well with the second approach too. -Alan From Alan.Bateman at oracle.com Wed Jun 6 13:57:06 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jun 2018 14:57:06 +0100 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> Message-ID: <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> On 05/06/2018 07:23, Ivan Gerasimov wrote: > Hello! > > When a file size is modified via RandomAccessFile.setLength(int) on > Windows, it is currently done in three steps: > > SetFilePointer(), SetEndOfFile(), SetFilePointerEx(). > > First two can be combined in one call of SetFileInformationByHandle(), > which will make the code shorter and more aligned with Unix variant > (ftruncate64 is used on Unix systems, which behaves close to > SetFileInformationByHandle(_,FileEndOfFileInfo,_)). > > Would you please help review this trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 > WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/00/webrev/ > > All the existing tests pass Ok. > I think this should be okay but the Windows implementation has a long history of biting the fingers of anyone that dares touch it. Sometimes unexpected behavior changes only come to light long after the change. The recent mails here about Kafka and sparse files is a good example of that. So I think it's important to check the test coverage before pushing this change. Specifically I think we need to check that we have tests that 1. Exercise shrinking, extending, and not change the file length 2. Check getFilePointer when used with setLength. 3. Check how FileChannel behaves when created from a RandomAccessFile but the file length is changed after the FileChannel is obtained. I realize this is more work than what you might have wanted to put into this but we have to extremely careful here. -Alan From jason_mehrens at hotmail.com Wed Jun 6 14:34:04 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Wed, 6 Jun 2018 14:34:04 +0000 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com>, Message-ID: Roger, Looks like StaticProperty.initProperty is never called. I assume that was suppose to be called on lines 40-43? Jason ________________________________________ From: core-libs-dev on behalf of Roger Riggs Sent: Tuesday, June 5, 2018 9:18 AM To: Stuart Marks Cc: Core-Libs-Dev Subject: Re: RFR JDK-8066709 Make some JDK system properties read only Hi Stuart, On 6/4/2018 9:52 PM, Stuart Marks wrote: > > > On 6/4/18 6:32 AM, Roger Riggs wrote: >> Please review a change to make the values of java.home, user.home, >> user.dir, and user.name >> effectively read-only for internal use. The values are cached during >> initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8204235 > > Hi Roger, > > Overall I think the intent of the change is a good one, as is the > reimplementation of internal uses of system properties to use an > internal API instead. > > I think it would be good to have an explicit initialization of the > SystemProperty class (or whatever it ends up being called) instead of > implicitly initializing via a static initializer. If the class is > initialized too early, for some reason, things would go wrong. > That would be a no-op; I want the values of the properties to be final statics which means they have to be initialized by the static initializer. > Should there be an error, a warning, or assertion checking to make > sure that none of the cached values are null? They should all be > non-null, right? Yes, null will be a fatal InternalError. Thanks, Roger > > s'marks From tprintezis at twitter.com Wed Jun 6 14:37:56 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 6 Jun 2018 07:37:56 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> Message-ID: Alan, Thanks for your thoughts! Peter, Any chance of taking the two suggestions I made in an earlier e-mail into account? a) It?d be nice if users can override initialValue(), like when using the standard ThreadLocal class, instead of calculateInitialValue(). (I can?t come up with a clean solution on how to do that, though. I?ll think about it for a bit.) b) It?d be very helpful to pass T value to threadTerminated(), as I cannot imagine many use-cases where the value will not be needed. Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be very helpful and can avoid completely unnecessary allocations. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com) wrote: On 30/05/2018 22:16, Peter Levart wrote: > I thought there would be some hint from Alan about which of the two > paths we should take for more refinement. > > The Tony's: > > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ > > Or the Alan's with my mods: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ > Finally getting back to this one. I don't think the two approaches are all that different now. Tony's point on the number of hooks vs. number of locals was an important point but with Peter's update to use a thread local registry then I think we have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. So I think I prefer that approach. We need to better name for "JdkThreadLocal", something to indicate that it holds resources or it notified when a thread terminates. Also along the way, we touched on exposing getIfPresent and we should look at that again. If it is expose then it fits well with the second approach too. -Alan From lance.andersen at oracle.com Wed Jun 6 14:56:47 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 6 Jun 2018 10:56:47 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> References: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> Message-ID: <8BEFA400-F64C-4A49-8F2D-CE8BCF9582B6@oracle.com> Hi Roger > On Jun 5, 2018, at 5:54 PM, Roger Riggs wrote: > > Hi Lance, > > Can the name change be done using hg rename to preserve the continuity? I did do an hg rename, and just did it again in a different workspace: hg rename package.html package-info.java ljanders-mac:rowset ljanders$ hg status -mar M test/jdk/tools/jmod/hashes/HashesTest.java M test/jdk/tools/launcher/modules/addexports/AddExportsTest.java A src/java.sql.rowset/share/classes/javax/sql/rowset/package-info.java R src/java.sql.rowset/share/classes/javax/sql/rowset/package.html > > Also, while you are there, how about converting to {@code...} etc. I do plan to do this, but thought I would keep things minimal for this updateand do that in a follow-on due to the renaming so it is easier to follow in the webrev. Best Lance > > Thanks, Roger > > > On 6/4/18 7:22 AM, Lance Andersen wrote: >> Hi, >> >> Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html >> >> As part of this fix, I took the liberty to move from package.html to package-info.java >> >> The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ >> >> I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java >> >> Best >> Lance >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From peter.levart at gmail.com Wed Jun 6 15:12:42 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Jun 2018 17:12:42 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> Message-ID: <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> Hi Tony, Alan, On 06/06/2018 04:37 PM, Tony Printezis wrote: > Alan, > > Thanks for your thoughts! > > Peter, > > Any chance of taking the two suggestions I made in an earlier e-mail > into account? Right, I'll prepare new webrev with that shortly. > > a) It?d be nice if users can override initialValue(), like when using > the standard ThreadLocal class, instead of calculateInitialValue(). (I > can?t come up with a clean solution on how to do that, though. I?ll > think about it for a bit.) Your concern was that users might accidentally override initialValue() instead of calculateInitialValue(), if I understood you correctly. If that was the concern, they can't, because it is final. If the concern was that users would want to override initialValue() because they are used to do so, then perhaps a javadoc on final initialiValue() pointing to calculateInitialValue() might help them. Do you agree? This is currently internal API and consistent naming is not a big concern. > b) It?d be very helpful to pass T value to threadTerminated(), as I > cannot imagine many use-cases where the value will not be needed. Agree. Will include that in new webrev. > > Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? Hm. Exit Hooks are already something that is used in JVM (Threads started when VM is about to exit), so this might be confusing for someone. - DisposableThreadLocal - ThreadLocalWithCleanup And my favorite: - TerminatingThreadLocal > > Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be > very helpful and can avoid completely unnecessary allocations. I agree that this would be generally useful functionality. Might be good to ask for opinion on concurrency-interest mailing list first. Will do that. Regards, Peter > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com > ) wrote: > >> >> >> On 30/05/2018 22:16, Peter Levart wrote: >> > I thought there would be some hint from Alan about which of the two >> > paths we should take for more refinement. >> > >> > The Tony's: >> > >> > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ >> >> > >> > Or the Alan's with my mods: >> > >> > >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ >> >> >> > >> Finally getting back to this one. >> >> I don't think the two approaches are all that different now. Tony's >> point on the number of hooks vs. number of locals was an important point >> but with Peter's update to use a thread local registry then I think we >> have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. >> So I think I prefer that approach. We need to better name for >> "JdkThreadLocal", something to indicate that it holds resources or it >> notified when a thread terminates. >> >> Also along the way, we touched on exposing getIfPresent and we should >> look at that again. If it is expose then it fits well with the second >> approach too. >> >> -Alan From roger.riggs at oracle.com Wed Jun 6 15:26:51 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 6 Jun 2018 11:26:51 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: <8BEFA400-F64C-4A49-8F2D-CE8BCF9582B6@oracle.com> References: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> <8BEFA400-F64C-4A49-8F2D-CE8BCF9582B6@oracle.com> Message-ID: <3f52b00b-bc67-dcc9-b7da-abab78d08624@oracle.com> Hi Lance, That's fine, the conversion from .html to .java made the diff extensive; hiding the original link fix. The changes are fine by me. Roger On 6/6/18 10:56 AM, Lance Andersen wrote: > Hi Roger >> On Jun 5, 2018, at 5:54 PM, Roger Riggs > > wrote: >> >> Hi Lance, >> >> Can the name change be done using? hg rename to preserve the continuity? > > I did do an hg rename, and just did it again in a different workspace: > > hg rename package.html package-info.java > ljanders-mac:rowset ljanders$ hg status -mar > M test/jdk/tools/jmod/hashes/HashesTest.java > M test/jdk/tools/launcher/modules/addexports/AddExportsTest.java > A src/java.sql.rowset/share/classes/javax/sql/rowset/package-info.java > R src/java.sql.rowset/share/classes/javax/sql/rowset/package.html >> >> Also, while you are there, how about converting to {@code...} etc. > > I do plan to do this, but thought I would keep things minimal ?for > this updateand do that in a follow-on due to the renaming so it is > easier to follow in the webrev. > > > Best > Lance >> >> Thanks, Roger >> >> >> On 6/4/18 7:22 AM, Lance Andersen wrote: >>> Hi, >>> >>> Bug 8201608 highlights a few broken links in >>> javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html >>> >>> As part of this fix, I took the liberty to move from package.html to >>> package-info.java >>> >>> The webrev can be found at >>> http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ >>> >>> >> > >>> >>> I have also attached a diff of the changes as it is less obvious of >>> the webrev prior to the migration to package-info.java >>> >>> Best >>> Lance >>> >>> ? >>> ? >>> >>> ?Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From Roger.Riggs at Oracle.com Wed Jun 6 15:31:34 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Jun 2018 11:31:34 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com> Message-ID: <0d86047d-285a-a5ae-1fd3-8c3484d55fdc@Oracle.com> Thanks Jason,? 1/2 a fix is not enough. Thanks, Roger p.s. I don't think the comments and suggestions are all in yet. On 6/6/2018 10:34 AM, Jason Mehrens wrote: > Roger, > > Looks like StaticProperty.initProperty is never called. I assume that was suppose to be called on lines 40-43? > > Jason > ________________________________________ > From: core-libs-dev on behalf of Roger Riggs > Sent: Tuesday, June 5, 2018 9:18 AM > To: Stuart Marks > Cc: Core-Libs-Dev > Subject: Re: RFR JDK-8066709 Make some JDK system properties read only > > Hi Stuart, > > On 6/4/2018 9:52 PM, Stuart Marks wrote: >> >> On 6/4/18 6:32 AM, Roger Riggs wrote: >>> Please review a change to make the values of java.home, user.home, >>> user.dir, and user.name >>> effectively read-only for internal use. The values are cached during >>> initialization and the >>> cached values are used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >> Hi Roger, >> >> Overall I think the intent of the change is a good one, as is the >> reimplementation of internal uses of system properties to use an >> internal API instead. >> >> I think it would be good to have an explicit initialization of the >> SystemProperty class (or whatever it ends up being called) instead of >> implicitly initializing via a static initializer. If the class is >> initialized too early, for some reason, things would go wrong. >> > That would be a no-op; I want the values of the properties to be final > statics which means they have to > be initialized by the static initializer. >> Should there be an error, a warning, or assertion checking to make >> sure that none of the cached values are null? They should all be >> non-null, right? > Yes, null will be a fatal InternalError. > > Thanks, Roger > >> s'marks From tprintezis at twitter.com Wed Jun 6 15:41:20 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 6 Jun 2018 08:41:20 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> Message-ID: Peter, You?re totally right re: JdkThreadLocal::initialValue being final and cannot be overridden (I had completely missed the final keyword when I first looked at the webrev). But it?d be nice to have the same API in both versions of ThreadLocal. I assume you didn?t want to override get() since you only wanted to update the REGISTRY the first time get() is called by a thread. If getIfPresent() is exposed, would something like the following work? @Override public T get() { final T value = getIfPresent(); if (value != null) { return value; } REGISTRY.get().add(this); return super.get(); } One more question re: getIfPresent() (and maybe I?m overthinking this): It returns null to indicate that a value is not present. Isn?t null a valid ThreadLocal value? Would using an Optional here be more appropriate? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com) wrote: Hi Tony, Alan, On 06/06/2018 04:37 PM, Tony Printezis wrote: Alan, Thanks for your thoughts! Peter, Any chance of taking the two suggestions I made in an earlier e-mail into account? Right, I'll prepare new webrev with that shortly. a) It?d be nice if users can override initialValue(), like when using the standard ThreadLocal class, instead of calculateInitialValue(). (I can?t come up with a clean solution on how to do that, though. I?ll think about it for a bit.) Your concern was that users might accidentally override initialValue() instead of calculateInitialValue(), if I understood you correctly. If that was the concern, they can't, because it is final. If the concern was that users would want to override initialValue() because they are used to do so, then perhaps a javadoc on final initialiValue() pointing to calculateInitialValue() might help them. Do you agree? This is currently internal API and consistent naming is not a big concern. b) It?d be very helpful to pass T value to threadTerminated(), as I cannot imagine many use-cases where the value will not be needed. Agree. Will include that in new webrev. Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? Hm. Exit Hooks are already something that is used in JVM (Threads started when VM is about to exit), so this might be confusing for someone. - DisposableThreadLocal - ThreadLocalWithCleanup And my favorite: - TerminatingThreadLocal Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be very helpful and can avoid completely unnecessary allocations. I agree that this would be generally useful functionality. Might be good to ask for opinion on concurrency-interest mailing list first. Will do that. Regards, Peter Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com) wrote: On 30/05/2018 22:16, Peter Levart wrote: > I thought there would be some hint from Alan about which of the two > paths we should take for more refinement. > > The Tony's: > > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ > > Or the Alan's with my mods: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ > Finally getting back to this one. I don't think the two approaches are all that different now. Tony's point on the number of hooks vs. number of locals was an important point but with Peter's update to use a thread local registry then I think we have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. So I think I prefer that approach. We need to better name for "JdkThreadLocal", something to indicate that it holds resources or it notified when a thread terminates. Also along the way, we touched on exposing getIfPresent and we should look at that again. If it is expose then it fits well with the second approach too. -Alan From lance.andersen at oracle.com Wed Jun 6 15:46:11 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 6 Jun 2018 11:46:11 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: <3f52b00b-bc67-dcc9-b7da-abab78d08624@oracle.com> References: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> <8BEFA400-F64C-4A49-8F2D-CE8BCF9582B6@oracle.com> <3f52b00b-bc67-dcc9-b7da-abab78d08624@oracle.com> Message-ID: <2B8887F4-9529-4909-B5BB-F9FD815858D1@oracle.com> Hi Roger > On Jun 6, 2018, at 11:26 AM, Roger Riggs wrote: > > Hi Lance, > > That's fine, the conversion from .html to .java made the diff extensive; hiding the original link fix. Yes, I had attached them originally to the email with the diff against package.html and then forgot it would be stripped by the mail server so I attached it to the bug per Paul?s suggestion. > The changes are fine by me. Thank you Best Lance > > Roger > > > On 6/6/18 10:56 AM, Lance Andersen wrote: >> Hi Roger >>> On Jun 5, 2018, at 5:54 PM, Roger Riggs > wrote: >>> >>> Hi Lance, >>> >>> Can the name change be done using hg rename to preserve the continuity? >> >> I did do an hg rename, and just did it again in a different workspace: >> >> hg rename package.html package-info.java >> ljanders-mac:rowset ljanders$ hg status -mar >> M test/jdk/tools/jmod/hashes/HashesTest.java >> M test/jdk/tools/launcher/modules/addexports/AddExportsTest.java >> A src/java.sql.rowset/share/classes/javax/sql/rowset/package-info.java >> R src/java.sql.rowset/share/classes/javax/sql/rowset/package.html >>> >>> Also, while you are there, how about converting to {@code...} etc. >> >> I do plan to do this, but thought I would keep things minimal for this updateand do that in a follow-on due to the renaming so it is easier to follow in the webrev. >> >> >> Best >> Lance >>> >>> Thanks, Roger >>> >>> >>> On 6/4/18 7:22 AM, Lance Andersen wrote: >>>> Hi, >>>> >>>> Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html >>>> >>>> As part of this fix, I took the liberty to move from package.html to package-info.java >>>> >>>> The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ > >>>> >>>> I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java >>>> >>>> Best >>>> Lance >>>> >>>> > >>>> > > >>>> >Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>> Oracle Java Engineering >>>> 1 Network Drive >>>> Burlington, MA 01803 >>>> Lance.Andersen at oracle.com > >>>> >>>> >>> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From paul.sandoz at oracle.com Wed Jun 6 15:53:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 6 Jun 2018 08:53:17 -0700 Subject: Implementations of PrimitiveIterator.OfInt#forEachRemaining(IntConsumer) doesn't check IntConsumer for null In-Reply-To: References: Message-ID: Performance should not be a concern, null checking is highly optimized by the JIT and is anyway required when the consumer is accessed in the loop. It?s probably an oversight but FWIW these internal iterators are never exposed directly and are wrapped in spliterators and those wrappers will perform dominating null checks. Paul. > On Jun 5, 2018, at 1:17 PM, ?????? ???????? wrote: > > Hello. > > I observe that implementations of `forEachRemaining` method inside > java.lang.CharSequence doesn't always compare IntConsumer parameter with > null. > Is there any reason why there is no nonNull check? Performance? > > Andrey Turbanov. From iris.clark at oracle.com Wed Jun 6 16:28:15 2018 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 6 Jun 2018 09:28:15 -0700 (PDT) Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> Message-ID: <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> Hi, Stuart. I think you need to make changes to this file too: http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html Thanks, iris -----Original Message----- From: Alan Bateman Sent: Wednesday, June 6, 2018 3:40 AM To: Stuart Marks ; serviceability-dev Cc: core-libs-dev Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) On 06/06/2018 01:05, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have > pointed out that there are some cross-references to > Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the > JDI ThreadReference API. I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ Have you considered removing the "What about Thread.stop(Throwable)" section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. -Alan From peter.levart at gmail.com Wed Jun 6 16:58:33 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Jun 2018 18:58:33 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> Message-ID: <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> On 06/06/18 17:41, Tony Printezis wrote: > Peter, > > You?re totally right re: JdkThreadLocal::initialValue being final and > cannot be overridden (I had completely missed the final keyword when I > first looked at the webrev). But it?d be nice to have the same API in > both versions of ThreadLocal. I assume you didn?t want to override > get() since you only wanted to update the REGISTRY the first time > get() is called by a thread. If getIfPresent() is exposed, would > something like the following work? > > @Override > public T get() { > ? final T value = getIfPresent(); > ? if (value != null) { > ? ? ? return value; > ? } > ? REGISTRY.get().add(this); > ? return super.get(); > } This would work, but if mapped value was 'null', it would keep calling REGISTRY.get().add(this) for each get(). Logically correct, but with useless overhead. ?Let me try to do something that might satisfy you... (in next webrev). > > One more question re: getIfPresent() (and maybe I?m overthinking > this): It returns null to indicate that a value is not present. Isn?t > null a valid ThreadLocal value? Would using an Optional here be more > appropriate? The problem with Optional is that it does not provide an additional value. Optional.empty() is a replacement for null. There's no Optional.of(null). So what would getIfPresent() return when there was a mapping present, but that mapping was null? Regards, Peter > > Tony > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com > ) wrote: > >> Hi Tony, Alan, >> >> On 06/06/2018 04:37 PM, Tony Printezis wrote: >>> Alan, >>> >>> Thanks for your thoughts! >>> >>> Peter, >>> >>> Any chance of taking the two suggestions I made in an earlier e-mail >>> into account? >> >> Right, I'll prepare new webrev with that shortly. >> >>> >>> a) It?d be nice if users can override initialValue(), like when >>> using the standard ThreadLocal class, instead of >>> calculateInitialValue(). (I can?t come up with a clean solution on >>> how to do that, though. I?ll think about it for a bit.) >> >> Your concern was that users might accidentally override >> initialValue() instead of calculateInitialValue(), if I understood >> you correctly. If that was the concern, they can't, because it is >> final. If the concern was that users would want to override >> initialValue() because they are used to do so, then perhaps a javadoc >> on final initialiValue() pointing to calculateInitialValue() might >> help them. Do you agree? This is currently internal API and >> consistent naming is not a big concern. >> >>> b) It?d be very helpful to pass T value to threadTerminated(), as I >>> cannot imagine many use-cases where the value will not be needed. >> >> Agree. Will include that in new webrev. >> >>> >>> Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? >> >> Hm. Exit Hooks are already something that is used in JVM (Threads >> started when VM is about to exit), so this might be confusing for >> someone. >> >> - DisposableThreadLocal >> - ThreadLocalWithCleanup >> >> And my favorite: >> >> - TerminatingThreadLocal >> >> >>> >>> Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be >>> very helpful and can avoid completely unnecessary allocations. >> >> I agree that this would be generally useful functionality. Might be >> good to ask for opinion on concurrency-interest mailing list first. >> Will do that. >> >> Regards, Peter >> >>> >>> Tony >>> >>> >>> ????? >>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>> >>> >>> >>> On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com >>> ) wrote: >>> >>>> >>>> >>>> On 30/05/2018 22:16, Peter Levart wrote: >>>> > I thought there would be some hint from Alan about which of the two >>>> > paths we should take for more refinement. >>>> > >>>> > The Tony's: >>>> > >>>> > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ >>>> >>>> > >>>> > Or the Alan's with my mods: >>>> > >>>> > >>>> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ >>>> >>>> > >>>> Finally getting back to this one. >>>> >>>> I don't think the two approaches are all that different now. Tony's >>>> point on the number of hooks vs. number of locals was an important >>>> point >>>> but with Peter's update to use a thread local registry then I think we >>>> have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. >>>> So I think I prefer that approach. We need to better name for >>>> "JdkThreadLocal", something to indicate that it holds resources or it >>>> notified when a thread terminates. >>>> >>>> Also along the way, we touched on exposing getIfPresent and we should >>>> look at that again. If it is expose then it fits well with the second >>>> approach too. >>>> >>>> -Alan >> From martinrb at google.com Wed Jun 6 17:03:09 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Jun 2018 10:03:09 -0700 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: OK, here is a RFR for low-hanging changes (some of these changes have already been reviewed by some of you): 8204375: Add TimeUnit#convert(Duration) http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/convertDuration/index.html https://bugs.openjdk.java.net/browse/JDK-8204375 8204444: java.time cleanup http://cr.openjdk.java.net/~martin/webrevs/jdk/time-lint/ https://bugs.openjdk.java.net/browse/JDK-8204444 8204377: Rename Object#wait parameter name from "timeout" to "timeoutMillis" http://cr.openjdk.java.net/~martin/webrevs/jdk/Object-wait-timeout/ https://bugs.openjdk.java.net/browse/JDK-8204377 From mandy.chung at oracle.com Wed Jun 6 17:13:49 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 6 Jun 2018 10:13:49 -0700 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> Message-ID: On 5/30/18 5:10 PM, Kevin Rushforth wrote: > I would like to propose the following Draft JEP [1] for discussion. > > JDK-8200758: Packaging Tool > > This is intended as a JDK-scope replacement for the existing > javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK > releases), and was delivered as part of the JavaFX build. The > javapackager tool has been removed from Oracle JDK 11 along with the > removal of JavaFX. > > Comments on this JEP are welcome. It is currently not targeted for a > specific release, but we are aiming for JDK 12. > > -- Kevin > > [1] https://bugs.openjdk.java.net/browse/JDK-820075 Good to see this moving along. A couple of comments: I think the JEP may need to mention the layout of an application image and define what's supported and unsupported to make it clear for developers what should or should not depend on, for example, the location of the application launcher and user-editable configuration files etcs. javapackager strips the tool launchers from the run-time image and enables compression. Does jpackager do something similar? As the development moves along, it'd be useful to include some command-line examples and describe the content of the run-time image produced. As part of this JEP, we need to look at the difference of the native launcher created by jpackager and by jlink. jlink creates a native launcher for modular application and possibly share same mechanism in creating native launchers. Mandy From martinrb at google.com Wed Jun 6 18:01:39 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Jun 2018 11:01:39 -0700 Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> References: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> Message-ID: Alright, I leave it to you. Your current change is fine. On Wed, Jun 6, 2018 at 12:07 AM, Deepak Kejriwal wrote: > Hi Martin, > > > > Backporting entire tck directory would be over kill as most of files under > tck were checked in as part of https://bugs.openjdk.java. > net/browse/JDK-8146467 and are not really related to JDK-8186171. May be > down the line if needed we can invest time to address it. > > > > The scope Bug8186171Test.java was to test the fix for scenario mentioned > in JDK-8186171. > > Regards, > > Deepak > > > > *From:* Martin Buchholz > *Sent:* Tuesday, June 5, 2018 5:35 AM > *To:* Deepak Kejriwal ; Doug Lea < > dl at cs.oswego.edu> > *Cc:* core-libs-dev > *Subject:* Re: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue > may not work after Iterator.remove() called for previous entries > > > > Hej Deepak, > > > > This looks alright, but you really need to add that trailing newline on > the test file (a Martin pet peeve). > > I wonder if instead we invest a little more work, but backport the entire > tck directory. > > All tests should pass except for those that test features not yet > backported. > > > > On Mon, Jun 4, 2018 at 4:46 AM, Deepak Kejriwal < > deepak.kejriwal at oracle.com> wrote: > > Hi all, > > > > Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/ > browse/JDK-8186171 > > Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.00/ > > > > Summary(also added to backport bug description): > > > > The back port for test files is not clean back port as all tests files are > extending JSR166TestCase.java which was added to JDK 9 as part of HYPERLINK > "https://bugs.openjdk.java.net/browse/JDK-8146467"JDK-8146467: Integrate > JSR 166 jck tests into JDK repo. This is not present in JDK8 version. > > Therefore, I have extracted test are relevant to the fix done JDK-8186171 > and created a new test file Bug8186171Test.java that contains below two > test methods: > > > > . testBug8186171NonDeterministic : This method is copy of > "testBug8186171" present in "MapTest.java" of original changeset > http://hg.openjdk.java.net/jdk10/master/rev/3f5f9bc0bdc2. As it is based > on randomization and runs 1000 times, the method name is suffixed with > "NonDeterministic". > > . testBug8186171Deterministic : This is a new method that runs > single time as it produces the exact scenario mentioned in JDK-8186171. > Therefore, it is not needed to run multiple times to produce the scenario > mentioned in bug. Hence the method name is suffixed with "Deterministic" > > > > Regards, > > Deepak > > > From martinrb at google.com Wed Jun 6 18:03:17 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Jun 2018 11:03:17 -0700 Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: References: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> Message-ID: On Wed, Jun 6, 2018 at 11:01 AM, Martin Buchholz wrote: > Alright, I leave it to you. Your current change is fine. > Except, add that trailing newline to the test! From peter.levart at gmail.com Wed Jun 6 18:55:49 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Jun 2018 20:55:49 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> Message-ID: <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Ok, here's next webrev: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.03/ The changes from webrev.02 are: - renamed JdkThreadLocal -> TerminatingThreadLocal - instead of overriding initialValue(), ThreadLocal.setInitialValue() calls TerminatingThreadLocal.register() at the right moment - TerminatingThreadLocal.threadTerminated() now takes a (T value) parameter - TerminatingThreadLocal.REGISTRY uses IdentityHashSet instead of HashSet (if .equals()/.hashCode() are overriden in some TerminatingThreadLocal subclass) David Lloyd has commented on concurrency-interest about ThreadLocal.getIfPresent(). There is a concern that such new method would be inconsistent with what ThreadLocal.get() returns from existing ThreadLocal subclasses that override .get() and possibly transform the value returned from super.get() on the way. An alternative to "T getIfPresent()" is a "boolean isPresent()" method. Even if it remains package-private, with such method TerminatingThreadLocal could be implemented as: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ If TreadLocal.isPresent() was made public, the code could be further simplified. Regards, Peter On 06/06/18 18:58, Peter Levart wrote: > > > On 06/06/18 17:41, Tony Printezis wrote: >> Peter, >> >> You?re totally right re: JdkThreadLocal::initialValue being final and >> cannot be overridden (I had completely missed the final keyword when >> I first looked at the webrev). But it?d be nice to have the same API >> in both versions of ThreadLocal. I assume you didn?t want to override >> get() since you only wanted to update the REGISTRY the first time >> get() is called by a thread. If getIfPresent() is exposed, would >> something like the following work? >> >> @Override >> public T get() { >> ? final T value = getIfPresent(); >> ? if (value != null) { >> ? ? ? return value; >> ? } >> ? REGISTRY.get().add(this); >> ? return super.get(); >> } > > This would work, but if mapped value was 'null', it would keep calling > REGISTRY.get().add(this) for each get(). Logically correct, but with > useless overhead. > > ?Let me try to do something that might satisfy you... (in next webrev). > >> >> One more question re: getIfPresent() (and maybe I?m overthinking >> this): It returns null to indicate that a value is not present. Isn?t >> null a valid ThreadLocal value? Would using an Optional here be more >> appropriate? > > The problem with Optional is that it does not provide an additional > value. Optional.empty() is a replacement for null. There's no > Optional.of(null). So what would getIfPresent() return when there was > a mapping present, but that mapping was null? > > Regards, Peter > >> >> Tony >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> >> >> On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com >> ) wrote: >> >>> Hi Tony, Alan, >>> >>> On 06/06/2018 04:37 PM, Tony Printezis wrote: >>>> Alan, >>>> >>>> Thanks for your thoughts! >>>> >>>> Peter, >>>> >>>> Any chance of taking the two suggestions I made in an earlier >>>> e-mail into account? >>> >>> Right, I'll prepare new webrev with that shortly. >>> >>>> >>>> a) It?d be nice if users can override initialValue(), like when >>>> using the standard ThreadLocal class, instead of >>>> calculateInitialValue(). (I can?t come up with a clean solution on >>>> how to do that, though. I?ll think about it for a bit.) >>> >>> Your concern was that users might accidentally override >>> initialValue() instead of calculateInitialValue(), if I understood >>> you correctly. If that was the concern, they can't, because it is >>> final. If the concern was that users would want to override >>> initialValue() because they are used to do so, then perhaps a >>> javadoc on final initialiValue() pointing to calculateInitialValue() >>> might help them. Do you agree? This is currently internal API and >>> consistent naming is not a big concern. >>> >>>> b) It?d be very helpful to pass T value to threadTerminated(), as I >>>> cannot imagine many use-cases where the value will not be needed. >>> >>> Agree. Will include that in new webrev. >>> >>>> >>>> Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? >>> >>> Hm. Exit Hooks are already something that is used in JVM (Threads >>> started when VM is about to exit), so this might be confusing for >>> someone. >>> >>> - DisposableThreadLocal >>> - ThreadLocalWithCleanup >>> >>> And my favorite: >>> >>> - TerminatingThreadLocal >>> >>> >>>> >>>> Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be >>>> very helpful and can avoid completely unnecessary allocations. >>> >>> I agree that this would be generally useful functionality. Might be >>> good to ask for opinion on concurrency-interest mailing list first. >>> Will do that. >>> >>> Regards, Peter >>> >>>> >>>> Tony >>>> >>>> >>>> ????? >>>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>>> >>>> >>>> >>>> On June 6, 2018 at 9:38:05 AM, Alan Bateman >>>> (alan.bateman at oracle.com ) wrote: >>>> >>>>> >>>>> >>>>> On 30/05/2018 22:16, Peter Levart wrote: >>>>> > I thought there would be some hint from Alan about which of the two >>>>> > paths we should take for more refinement. >>>>> > >>>>> > The Tony's: >>>>> > >>>>> > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ >>>>> >>>>> > >>>>> > Or the Alan's with my mods: >>>>> > >>>>> > >>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ >>>>> >>>>> > >>>>> Finally getting back to this one. >>>>> >>>>> I don't think the two approaches are all that different now. Tony's >>>>> point on the number of hooks vs. number of locals was an important >>>>> point >>>>> but with Peter's update to use a thread local registry then I think we >>>>> have easy to maintain solution in the DBBCache_Cleanup/webrev.02 >>>>> patch. >>>>> So I think I prefer that approach. We need to better name for >>>>> "JdkThreadLocal", something to indicate that it holds resources or it >>>>> notified when a thread terminates. >>>>> >>>>> Also along the way, we touched on exposing getIfPresent and we should >>>>> look at that again. If it is expose then it fits well with the second >>>>> approach too. >>>>> >>>>> -Alan >>> > From xueming.shen at oracle.com Wed Jun 6 19:39:02 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 06 Jun 2018 12:39:02 -0700 Subject: RFR https://bugs.openjdk.java.net/browse/JDK-8204494 Message-ID: <5B183856.60108@oracle.com> Hi Please help review the change for JDK-8204494, a regression caused by the fix for JDK-8200530 [1]. It appears that fix fails to deal with the corner case that a pair of \r\n is at the boundary of the internal buf of Manifest.FastInputtStream, which is 8192 for the default. In that scenario, the previous change fails to consume the trailing \n, then triggers the Attributes parser fails to work on the "continuation line" defined by the jar spec. issue: https://bugs.openjdk.java.net/browse/JDK-8204494 webrev: http://cr.openjdk.java.net/~sherman/8204494/webrev/ Fix has been verified by the regression reported in INTJDK-7627771. Thanks, Sherman [1] issue: https://bugs.openjdk.java.net/browse/JDK-8200530 webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev From roger.riggs at oracle.com Wed Jun 6 20:02:43 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 6 Jun 2018 16:02:43 -0400 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: Hi Martin, All three look fine to me.? +3 Roger On 6/6/18 1:03 PM, Martin Buchholz wrote: > OK, here is a RFR for low-hanging changes (some of these changes have > already been reviewed by some of you): > > 8204375: Add TimeUnit#convert(Duration) > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/convertDuration/index.html > > https://bugs.openjdk.java.net/browse/JDK-8204375 > > 8204444: java.time cleanup > http://cr.openjdk.java.net/~martin/webrevs/jdk/time-lint/ > > https://bugs.openjdk.java.net/browse/JDK-8204444 > > 8204377: Rename Object#wait parameter name from "timeout" to > "timeoutMillis" > http://cr.openjdk.java.net/~martin/webrevs/jdk/Object-wait-timeout/ > > https://bugs.openjdk.java.net/browse/JDK-8204377 > From tprintezis at twitter.com Wed Jun 6 20:38:06 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 6 Jun 2018 13:38:06 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: Hi Peter, Thanks for the updated webrev! Please see inline. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 2:55:51 PM, Peter Levart (peter.levart at gmail.com) wrote: Ok, here's next webrev: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.03/ The changes from webrev.02 are: - renamed JdkThreadLocal -> TerminatingThreadLocal #ThumbsUp - instead of overriding initialValue(), ThreadLocal.setInitialValue() calls TerminatingThreadLocal.register() at the right moment Thanks. I much prefer not having to introduce calculateInitialValue(). One extra suggestion: Given you introduced a call to TerminatingThreadLocal from ThreadLocal, would it make sense to maybe do the same for set() and remove() (you?d have to add an appropriate check in unregister) and not override them in TerminatingTreadLocal? I totally get why you might not want to do that (an extra check when a ThreadLocal is not the Terminating one). I?m OK either way. - TerminatingThreadLocal.threadTerminated() now takes a (T value) parameter #ThumbsUp - TerminatingThreadLocal.REGISTRY uses IdentityHashSet instead of HashSet (if .equals()/.hashCode() are overriden in some TerminatingThreadLocal subclass) #ThumbsUp David Lloyd has commented on concurrency-interest about ThreadLocal.getIfPresent(). There is a concern that such new method would be inconsistent with what ThreadLocal.get() returns from existing ThreadLocal subclasses that override .get() and possibly transform the value returned from super.get() on the way. That?s a very good point. An alternative to "T getIfPresent()" is a "boolean isPresent()" method. Even if it remains package-private, with such method TerminatingThreadLocal could be implemented as: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ If TreadLocal.isPresent() was made public, the code could be further simplified. Something like: if (tl.isPresent()) { T val = t.get(); ?. } will do two look-ups if the value exists. However, that?s better than allocating unnecessarily. So, I?ll take isPresent() over not having a way to check whether a value exists. Thumbs up from me. Tony Regards, Peter On 06/06/18 18:58, Peter Levart wrote: On 06/06/18 17:41, Tony Printezis wrote: Peter, You?re totally right re: JdkThreadLocal::initialValue being final and cannot be overridden (I had completely missed the final keyword when I first looked at the webrev). But it?d be nice to have the same API in both versions of ThreadLocal. I assume you didn?t want to override get() since you only wanted to update the REGISTRY the first time get() is called by a thread. If getIfPresent() is exposed, would something like the following work? @Override public T get() { final T value = getIfPresent(); if (value != null) { return value; } REGISTRY.get().add(this); return super.get(); } This would work, but if mapped value was 'null', it would keep calling REGISTRY.get().add(this) for each get(). Logically correct, but with useless overhead. Let me try to do something that might satisfy you... (in next webrev). One more question re: getIfPresent() (and maybe I?m overthinking this): It returns null to indicate that a value is not present. Isn?t null a valid ThreadLocal value? Would using an Optional here be more appropriate? The problem with Optional is that it does not provide an additional value. Optional.empty() is a replacement for null. There's no Optional.of(null). So what would getIfPresent() return when there was a mapping present, but that mapping was null? Regards, Peter Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com) wrote: Hi Tony, Alan, On 06/06/2018 04:37 PM, Tony Printezis wrote: Alan, Thanks for your thoughts! Peter, Any chance of taking the two suggestions I made in an earlier e-mail into account? Right, I'll prepare new webrev with that shortly. a) It?d be nice if users can override initialValue(), like when using the standard ThreadLocal class, instead of calculateInitialValue(). (I can?t come up with a clean solution on how to do that, though. I?ll think about it for a bit.) Your concern was that users might accidentally override initialValue() instead of calculateInitialValue(), if I understood you correctly. If that was the concern, they can't, because it is final. If the concern was that users would want to override initialValue() because they are used to do so, then perhaps a javadoc on final initialiValue() pointing to calculateInitialValue() might help them. Do you agree? This is currently internal API and consistent naming is not a big concern. b) It?d be very helpful to pass T value to threadTerminated(), as I cannot imagine many use-cases where the value will not be needed. Agree. Will include that in new webrev. Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? Hm. Exit Hooks are already something that is used in JVM (Threads started when VM is about to exit), so this might be confusing for someone. - DisposableThreadLocal - ThreadLocalWithCleanup And my favorite: - TerminatingThreadLocal Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be very helpful and can avoid completely unnecessary allocations. I agree that this would be generally useful functionality. Might be good to ask for opinion on concurrency-interest mailing list first. Will do that. Regards, Peter Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com) wrote: On 30/05/2018 22:16, Peter Levart wrote: > I thought there would be some hint from Alan about which of the two > paths we should take for more refinement. > > The Tony's: > > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ > > Or the Alan's with my mods: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ > Finally getting back to this one. I don't think the two approaches are all that different now. Tony's point on the number of hooks vs. number of locals was an important point but with Peter's update to use a thread local registry then I think we have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. So I think I prefer that approach. We need to better name for "JdkThreadLocal", something to indicate that it holds resources or it notified when a thread terminates. Also along the way, we touched on exposing getIfPresent and we should look at that again. If it is expose then it fits well with the second approach too. -Alan From stuart.marks at oracle.com Wed Jun 6 21:02:26 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:02:26 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <34074732-06a4-cc54-d935-835edc69a558@oracle.com> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: > But the fix below is not clear to me: > > http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html > > Stop Thread > > - Send the specified asynchronous exception to the specified thread > - (similar to java.lang.Thread.stop). > + Send the specified asynchronous exception to the specified thread. > Normally, this function is used to kill the specified thread with an > - instance of the exception ThreadDeath. > + instance of the exception ThreadDeath, similar to > + java.lang.Thread.stop. > > A reference to the java.lang.Thread.stop has been removed in one place and > added in another. > I thought, we wanted to get rid of these references in the spec. > Do I miss anything? Could you, explain this a little bit? Sure. The current state (through JDK 10) is that there two APIs: ??? 1. Thread.stop(Throwable) ??? 2. Thread.stop() // no-arg They are both deprecated, but only (1) is deprecated for removal, and it's the one being removed by this changeset. Method (2) will remain in the platform for the forseeable future. Method (1) causes the target thread asynchronously to throw the argument, which can be an instance of any subtype of Throwable. Method (2) causes the target thread to throw ThreadDeath (a subtype of Error) asynchronously. The wording in the JVMTI doc isn't terribly explicit. The original wording actually means "similar to Thread.stop(Throwable)" that is method (1). My rewording places the mention of Thread.stop into the context of throwing ThreadDeath, implying method (2). Since that will be the only Thread.stop method remaining, there's no ambiguity. So yes, the wording change looks odd, but there is a subtle shift in the meaning, and I think the meaning is clear after (1) has been removed. But I can remove the "similar to Thread.stop" bit if you prefer. s'marks > > Thanks, > Serguei > > On 6/5/18 17:05, Stuart Marks wrote: >> [adding serviceability-dev] >> >> Hi serviceability folks, >> >> I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) >> from the Java SE API. Alan and David have pointed out that there are some >> cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as >> well as in the JDI ThreadReference API. I've adjusted the relevant files. >> >> See please review this updated webrev: >> >> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >> >> For context, please see the full review thread on core-libs-dev: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >> >> >> >> On 6/4/18 11:11 PM, Alan Bateman wrote: >>> The source file that is used to generate the JDWP protocol code and the spec >>> is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for this >>> change-set, assuming it is followed up quickly with another change-set to >>> update those specs. >> >> OK. I took a look at those other files and they seem simple enough to update >> as part of this changeset. If there aren't any objections from anyone, might >> as well get this all done at once. >> >> Thanks, >> >> s'marks > From roger.riggs at oracle.com Wed Jun 6 21:13:57 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 6 Jun 2018 17:13:57 -0400 Subject: RFR https://bugs.openjdk.java.net/browse/JDK-8204494 In-Reply-To: <5B183856.60108@oracle.com> References: <5B183856.60108@oracle.com> Message-ID: <8ae3eb45-3d64-e4c2-8de9-21e6df04445f@oracle.com> Hi Sherman, Looks fine Roger On 6/6/18 3:39 PM, Xueming Shen wrote: > Hi > > Please help review the change for JDK-8204494, a regression caused by > the fix for > JDK-8200530 [1]. It appears that fix fails to deal with the corner > case that a pair of > \r\n is at the boundary of the internal buf of > Manifest.FastInputtStream, which is > 8192 for the default. In that scenario, the previous change fails to > consume the trailing > \n, then triggers the Attributes parser fails to work on the > "continuation line" defined > by the jar spec. > > issue: https://bugs.openjdk.java.net/browse/JDK-8204494 > webrev: http://cr.openjdk.java.net/~sherman/8204494/webrev/ > > Fix has been verified by the regression reported in INTJDK-7627771. > > Thanks, > Sherman > > [1] issue: https://bugs.openjdk.java.net/browse/JDK-8200530 > ???? webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev > From stuart.marks at oracle.com Wed Jun 6 21:29:58 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:29:58 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> Message-ID: <8e780019-dc81-9f6b-5bbc-06d2ffe234d1@oracle.com> Yeah, maybe it's better simply to remove the mentions of the methods that are being removed. I'll do so. I note that there are many other updates that could be done (the examples use applets!) but I think that's a task for another time. s'marks On 6/6/18 9:28 AM, Iris Clark wrote: > Hi, Stuart. > > I think you need to make changes to this file too: > > http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html > > Thanks, > iris > > -----Original Message----- > From: Alan Bateman > Sent: Wednesday, June 6, 2018 3:40 AM > To: Stuart Marks ; serviceability-dev > Cc: core-libs-dev > Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) > > On 06/06/2018 01:05, Stuart Marks wrote: >> [adding serviceability-dev] >> >> Hi serviceability folks, >> >> I'm in the process of removing Thread.destroy() and >> Thread.stop(Throwable) from the Java SE API. Alan and David have >> pointed out that there are some cross-references to >> Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the >> JDI ThreadReference API. I've adjusted the relevant files. >> >> See please review this updated webrev: >> >> ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ > Have you considered removing the "What about Thread.stop(Throwable)" > section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. > > Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. > > -Alan > From stuart.marks at oracle.com Wed Jun 6 21:58:13 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:58:13 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: Serguei, great! Thanks. All, I've updated the webrev to with changes to threadPrimitiveDeprecation.html to remove the sections that mention the removed methods. http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ s'marks On 6/6/18 2:37 PM, serguei.spitsyn at oracle.com wrote: > Stuart, > > Thank you for explanation! > I'm Okay with the fix it is now. > > Thanks, > Serguei > > > On 6/6/18 14:02, Stuart Marks wrote: >> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: >>> But the fix below is not clear to me: >>> >>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html >>> >>> Stop Thread >>> >>> - Send the specified asynchronous exception to the specified thread >>> - (similar to java.lang.Thread.stop). >>> + Send the specified asynchronous exception to the specified thread. >>> Normally, this function is used to kill the specified thread with an >>> - instance of the exception ThreadDeath. >>> + instance of the exception ThreadDeath, similar to >>> + java.lang.Thread.stop. >>> >>> A reference to the java.lang.Thread.stop has been removed in one place and >>> added in another. >>> I thought, we wanted to get rid of these references in the spec. >>> Do I miss anything? Could you, explain this a little bit? >> Sure. The current state (through JDK 10) is that there two APIs: >> >> ??? 1. Thread.stop(Throwable) >> ??? 2. Thread.stop() // no-arg >> >> They are both deprecated, but only (1) is deprecated for removal, and it's >> the one being removed by this changeset. Method (2) will remain in the >> platform for the forseeable future. >> >> Method (1) causes the target thread asynchronously to throw the argument, >> which can be an instance of any subtype of Throwable. Method (2) causes the >> target thread to throw ThreadDeath (a subtype of Error) asynchronously. >> >> The wording in the JVMTI doc isn't terribly explicit. The original wording >> actually means "similar to Thread.stop(Throwable)" that is method (1). My >> rewording places the mention of Thread.stop into the context of throwing >> ThreadDeath, implying method (2). Since that will be the only Thread.stop >> method remaining, there's no ambiguity. So yes, the wording change looks odd, >> but there is a subtle shift in the meaning, and I think the meaning is clear >> after (1) has been removed. >> >> But I can remove the "similar to Thread.stop" bit if you prefer. >> >> s'marks >>> >>> Thanks, >>> Serguei >>> >>> On 6/5/18 17:05, Stuart Marks wrote: >>>> [adding serviceability-dev] >>>> >>>> Hi serviceability folks, >>>> >>>> I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) >>>> from the Java SE API. Alan and David have pointed out that there are some >>>> cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as >>>> well as in the JDI ThreadReference API. I've adjusted the relevant files. >>>> >>>> See please review this updated webrev: >>>> >>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >>>> >>>> For context, please see the full review thread on core-libs-dev: >>>> >>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >>>> >>>> >>>> >>>> On 6/4/18 11:11 PM, Alan Bateman wrote: >>>>> The source file that is used to generate the JDWP protocol code and the >>>>> spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>>>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for >>>>> this change-set, assuming it is followed up quickly with another >>>>> change-set to update those specs. >>>> >>>> OK. I took a look at those other files and they seem simple enough to >>>> update as part of this changeset. If there aren't any objections from >>>> anyone, might as well get this all done at once. >>>> >>>> Thanks, >>>> >>>> s'marks >>> >> > From iris.clark at oracle.com Wed Jun 6 22:18:02 2018 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 6 Jun 2018 15:18:02 -0700 (PDT) Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: <4a93f51f-01b9-489f-9e53-cb5bba47506c@default> Hi, Stuart. ? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ ? The simple update to remove references to the removed methods looks good. ? Thanks, iris ? From: Stuart Marks Sent: Wednesday, June 6, 2018 2:58 PM To: Serguei Spitsyn ; David Holmes ; Alan Bateman ; Iris Clark Cc: serviceability-dev ; core-libs-dev Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) ? Serguei, great! Thanks. All, I've updated the webrev to with changes to threadPrimitiveDeprecation.html to remove the sections that mention the removed methods. http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ s'marks ? From david.holmes at oracle.com Thu Jun 7 02:13:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 7 Jun 2018 12:13:32 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: On 7/06/2018 7:58 AM, Stuart Marks wrote: > Serguei, great! Thanks. > > All, I've updated the webrev to with changes to > threadPrimitiveDeprecation.html to remove the sections that mention the > removed methods. That reads much better! Thanks, David > http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ > > s'marks > > > On 6/6/18 2:37 PM, serguei.spitsyn at oracle.com wrote: >> Stuart, >> >> Thank you for explanation! >> I'm Okay with the fix it is now. >> >> Thanks, >> Serguei >> >> >> On 6/6/18 14:02, Stuart Marks wrote: >>> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: >>>> But the fix below is not clear to me: >>>> >>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html >>>> >>>> Stop Thread >>>> >>>> - Send the specified asynchronous exception to the specified thread >>>> - (similar to java.lang.Thread.stop). >>>> + Send the specified asynchronous exception to the specified thread. >>>> Normally, this function is used to kill the specified thread with an >>>> - instance of the exception ThreadDeath. >>>> + instance of the exception ThreadDeath, similar to >>>> + java.lang.Thread.stop. >>>> >>>> A reference to the java.lang.Thread.stop has been removed in one >>>> place and added in another. >>>> I thought, we wanted to get rid of these references in the spec. >>>> Do I miss anything? Could you, explain this a little bit? >>> Sure. The current state (through JDK 10) is that there two APIs: >>> >>> ??? 1. Thread.stop(Throwable) >>> ??? 2. Thread.stop() // no-arg >>> >>> They are both deprecated, but only (1) is deprecated for removal, and >>> it's the one being removed by this changeset. Method (2) will remain >>> in the platform for the forseeable future. >>> >>> Method (1) causes the target thread asynchronously to throw the >>> argument, which can be an instance of any subtype of Throwable. >>> Method (2) causes the target thread to throw ThreadDeath (a subtype >>> of Error) asynchronously. >>> >>> The wording in the JVMTI doc isn't terribly explicit. The original >>> wording actually means "similar to Thread.stop(Throwable)" that is >>> method (1). My rewording places the mention of Thread.stop into the >>> context of throwing ThreadDeath, implying method (2). Since that will >>> be the only Thread.stop method remaining, there's no ambiguity. So >>> yes, the wording change looks odd, but there is a subtle shift in the >>> meaning, and I think the meaning is clear after (1) has been removed. >>> >>> But I can remove the "similar to Thread.stop" bit if you prefer. >>> >>> s'marks >>>> >>>> Thanks, >>>> Serguei >>>> >>>> On 6/5/18 17:05, Stuart Marks wrote: >>>>> [adding serviceability-dev] >>>>> >>>>> Hi serviceability folks, >>>>> >>>>> I'm in the process of removing Thread.destroy() and >>>>> Thread.stop(Throwable) from the Java SE API. Alan and David have >>>>> pointed out that there are some cross-references to >>>>> Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in >>>>> the JDI ThreadReference API. I've adjusted the relevant files. >>>>> >>>>> See please review this updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >>>>> >>>>> For context, please see the full review thread on core-libs-dev: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >>>>> >>>>> >>>>> >>>>> On 6/4/18 11:11 PM, Alan Bateman wrote: >>>>>> The source file that is used to generate the JDWP protocol code >>>>>> and the spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>>>>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those >>>>>> for this change-set, assuming it is followed up quickly with >>>>>> another change-set to update those specs. >>>>> >>>>> OK. I took a look at those other files and they seem simple enough >>>>> to update as part of this changeset. If there aren't any objections >>>>> from anyone, might as well get this all done at once. >>>>> >>>>> Thanks, >>>>> >>>>> s'marks >>>> >>> >> > From ivan.gerasimov at oracle.com Thu Jun 7 08:12:54 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 7 Jun 2018 01:12:54 -0700 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> Message-ID: Hi Alan! On 6/6/18 6:57 AM, Alan Bateman wrote: > I think this should be okay but the Windows implementation has a long > history of biting the fingers of anyone that dares touch it. Sometimes > unexpected behavior changes only come to light long after the change. > The recent mails here about Kafka and sparse files is a good example > of that. So I think it's important to check the test coverage before > pushing this change. Specifically I think we need to check that we > have tests that > > 1. Exercise shrinking, extending, and not change the file length > > 2. Check getFilePointer when used with setLength. > > 3. Check how FileChannel behaves when created from a RandomAccessFile > but the file length is changed after the FileChannel is obtained. > > I realize this is more work than what you might have wanted to put > into this but we have to extremely careful here. > > -Alan > I agree it's a good idea to extend coverage in this area. I'll check existing tests and will add whatever is not yet there. With kind regards, Ivan From xu.y.yin at oracle.com Thu Jun 7 08:32:08 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Thu, 7 Jun 2018 16:32:08 +0800 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK Message-ID: Please review below new added test to check for package versioning information which customized in jar(s) manifest, many thanks bug: https://bugs.openjdk.java.net/browse/JDK-8201528 webrev: http://cr.openjdk.java.net/~xyin/8201528/webrev.00/ Regards, Chris From peter.levart at gmail.com Thu Jun 7 09:21:41 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 7 Jun 2018 11:21:41 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: <160715cd-9d39-0c3f-21f7-f029ef4139c6@gmail.com> Hi Tony, Thanks for taking a look. Just a couple of comments inline... On 06/06/18 22:38, Tony Printezis wrote: >> >> - instead of overriding initialValue(), ThreadLocal.setInitialValue() >> calls TerminatingThreadLocal.register() at the right moment > > > Thanks. I much prefer not having to introduce calculateInitialValue(). > > One extra suggestion: Given you introduced a call to > TerminatingThreadLocal from ThreadLocal, would it make sense to maybe > do the same for set() and remove() (you?d have to add an appropriate > check in unregister) and not override them in TerminatingTreadLocal? I > totally get why you might not want to do that (an extra check when a > ThreadLocal is not the Terminating one). I?m OK either way. > Yes, precisely. My 1st version did just that, but since there are usages of ThreadLocal out there that are very performance sensitive, I opted for an approach where there is zero performance regression for non-TerminatingThreadLocal(s) when calling set() or remove(). A call-back from setInitialValue is not so problematic as it usually occurs just once per thread, but I can imagine a scenario where calls to get() and set() occur very rapidly. >> >> An alternative to "T getIfPresent()" is a "boolean isPresent()" >> method. Even if it remains package-private, with such method >> TerminatingThreadLocal could be implemented as: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ >> >> If TreadLocal.isPresent() was made public, the code could be further >> simplified. > > > Something like: > > if (tl.isPresent()) { > > ? ?T val = t.get(); > > ? ??. > > } > > will do two look-ups if the value exists. However, that?s better than > allocating unnecessarily. So, I?ll take isPresent() over not having a > way to check whether a value exists. > Right, and in our scenario, it is just isPresent() that is called for every termination of every thread (REGISTRY.isPresent()). .get() is then called only when there's actually something to do. > > Thumbs up from me. > Let's just wait for a day or two to see whether the discussion on concurrency-interest gives us any additional ideas. Currently I'm thinking of proposing the addition of isPresent() API. As far as this RFR is concerned, I'm consequently promoting the latest webrev.04. So any comments on that one? Alan? Regards, Peter From deepak.kejriwal at oracle.com Thu Jun 7 10:08:35 2018 From: deepak.kejriwal at oracle.com (Deepak Kejriwal) Date: Thu, 7 Jun 2018 03:08:35 -0700 (PDT) Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: References: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> Message-ID: <163052d6-7682-40f9-9839-02b6324a2369@default> Hi Martin, Thanks for input. I have added the missing trailing newline to the test. Please find below updated webrev details:- Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.01/ Regards, Deepak From: Martin Buchholz Sent: Wednesday, June 6, 2018 11:33 PM To: Deepak Kejriwal Cc: Doug Lea
; core-libs-dev Subject: Re: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries On Wed, Jun 6, 2018 at 11:01 AM, Martin Buchholz wrote: Alright, I leave it to you.? Your current change is fine.?? Except, add that trailing newline to the test!? From matthias.baesken at sap.com Thu Jun 7 11:35:24 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 7 Jun 2018 11:35:24 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Message-ID: Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From christoph.langer at sap.com Thu Jun 7 11:43:54 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 7 Jun 2018 11:43:54 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <0c73caec8810b9844a463343bdaec064@linux.vnet.ibm.com> <81453d0cbe05477ea558917de263ada2@sap.com> Message-ID: <91ad0c1765a44b90bed87ae0fb61ae09@sap.com> Hi Ichiroh, your proposal seems to make sense. I have created a bug for this: https://bugs.openjdk.java.net/browse/JDK-8204541 Can you please generate a webrev (referencing this bug, -c option of webrev.ksh) and mail it over to me. Then I'll upload it and you can post an official RFR mail. Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Dienstag, 5. Juni 2018 08:59 > To: Baesken, Matthias > Cc: Langer, Christoph ; 'build- > dev at openjdk.java.net' ; ppc-aix-port- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > Goetz > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hello Matthias and Christoph. > Thank you for your explanations. > > I did not have enough knowledge about "visibility". > > I created following patches. > > ================================ > diff -r 02934b0d661b > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed > May > 30 14:46:28 2018 +0200 > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Tue Jun > 05 12:10:41 2018 +0900 > @@ -39,7 +39,9 @@ > #include "imageFile.hpp" > #include "inttypes.hpp" > #include "jimage.hpp" > +#if !defined(_AIX) > #include "osSupport.hpp" > +#endif > > #include "jdk_internal_jimage_NativeImageBuffer.h" > > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 > 2018 +0200 > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 > 2018 +0900 > @@ -29,7 +29,8 @@ > #ifndef __has_attribute > #define __has_attribute(x) 0 > #endif > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) > #ifdef ARM > #define JNIEXPORT > __attribute__((externally_visible,visibility("default"))) > #define JNIIMPORT > __attribute__((externally_visible,visibility("default"))) > ================================ > > If "osSupport.hpp" was included, XLC++ compiler complained. > I could not understand, which part was invalid... > I'm not sure, "osSupport.hpp" is really required on > NativeImageBuffer.cpp or not... > > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. [1] > 0xD01 means 13.1 by hexadecimal. > > I checked symbol table by "dump -Tv -X64". [2] > It seemed it was fine, some symbols were hidden. > > Does someone review my code? > > [1] > https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html > [2] > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > visibility/index.html > > On 2018-06-01 21:12, Baesken, Matthias wrote: > > Hi , my change 8202322 just handled the fact that the > > visibility - flags are not supported with xlc 12.1 , so setting > > them generated a TON of compile - time warnings . > > > > The introduction of the "-qvisibility=hidden" came with the > > mapfile removal changes : > > > > 8200358: Remove mapfiles for JDK executables > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > > > > 8200178: Remove mapfiles for JDK native libraries > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > > > > I guess it might need further testing+adjustments to make the > > "visibility hiding" work nicely with XLC13 , but currently we > > build only with XLC12 . > > > > As a workaround you might want to remove the "-qvisibility=hidden" > > setting for XLC 13 as well , like I did for XLC12 with the change > > 8202322 . > > > > > > Best regards, Matthias > > > > > > > > > >> -----Original Message----- > >> From: Langer, Christoph > >> Sent: Freitag, 1. Juni 2018 10:57 > >> To: Ichiroh Takiguchi > >> Cc: Baesken, Matthias ; 'build- > >> dev at openjdk.java.net' ; ppc-aix-port- > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> Goetz > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> on xlc 12.1 > >> > >> Hi Ichiroh, > >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe > >> nobody of my colleagues has played with it yet. So you are on a new > >> playground here ?? > >> > >> However, I believe the idea in OpenJDK with the abolition of map files > >> is that > >> symbols should be invisible externally unless they are declared > >> exported, > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the > >> correct > >> default and whatever JNIEXPORT expands to should contain the right > >> attributes to get that symbol visible. > >> > >> Can you check if either my assumption is completely wrong, JNIEXPORT > >> does > >> not expand to the right thing, XLC 13 has a bug or maybe just sume > >> specific > >> required symbols are not declared correctly? > >> > >> Best regards > >> Christoph > >> > >> > -----Original Message----- > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> > Sent: Donnerstag, 31. Mai 2018 09:55 > >> > To: Langer, Christoph > >> > Cc: Baesken, Matthias ; 'build- > >> > dev at openjdk.java.net' ; ppc-aix-port- > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> > Goetz > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> > > >> > Hello. > >> > 8202322 was integrated into jdk-11+15. > >> > I'm using XLC 13.1.3 on AIX 7.1.4. > >> > Build was failed because of "-qvisibility=hidden" on > >> > make/lib/LibCommon.gmk. > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], > >> > "-qvisibility=hidden" cannot create shared libraries entry points. > >> > For example, libverify.so was there, but entry points were not resolved > >> > by "-lverify" option. > >> > I think it should be "-qvisibility=default" (I tried, it worked) > >> > or "-qvisibility=protected" (I had not tried) ? > >> > I'm not familiar with -qvisibility option, but I'd like to find out > >> > right way. > >> > > >> > [1] > >> > > >> > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > >> > > >> > On 2018-05-16 16:08, Langer, Christoph wrote: > >> > > Hi Matthias, > >> > > > >> > > yes, reviewed. > >> > > > >> > > Best regards > >> > > Christoph > >> > > > >> > > From: Baesken, Matthias > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 > >> > > To: Langer, Christoph ; > >> > > 'build-dev at openjdk.java.net' ; > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> > > Cc: Lindenmaier, Goetz > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > > xlc 12.1 > >> > > > >> > > Hi Christoph can I add you as second reviewer (other reviewer was > >> > > Erik Joelsson) can push the change ? > >> > > > >> > > Best regards, Matthias > >> > > > >> > > > >> > > > >> > > From: Langer, Christoph > >> > > Sent: Donnerstag, 26. April 2018 16:38 > >> > > To: Baesken, Matthias > >> > > >; > >> > > 'build-dev at openjdk.java.net' > >> > > dev at openjdk.java.net>>; > >> > > ppc-aix-port-dev at openjdk.java.net >> > dev at openjdk.java.net>; > >> > > core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> > >> > > Cc: Simonis, Volker > >> > > > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > > xlc 12.1 > >> > > > >> > > Hi Matthias, > >> > > > >> > > to me the change in principal looks good. > >> > > > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > >> > > extract major number before the first dot, then compare numerically) > - > >> > > but maybe it is too complicated and the current single version > compare > >> > > suits our needs ? > >> > > > >> > > Best regards > >> > > Christoph > >> > > > >> > > From: Baesken, Matthias > >> > > Sent: Donnerstag, 26. April 2018 16:14 > >> > > To: 'build-dev at openjdk.java.net' > >> > > dev at openjdk.java.net>>; > >> > > ppc-aix-port-dev at openjdk.java.net >> > dev at openjdk.java.net>; > >> > > core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> > >> > > Cc: Langer, Christoph > >> > > >; > >> Simonis, > >> > > Volker > > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > >> > > 12.1 > >> > > > >> > > Hello , could you please review this small adjustment to the symbol > >> > > visibility compilation settings on AIX ? > >> > > Currently we use XLC 12.1 to compile JDK on AIX . > >> > > > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" > >> > > setting currently set on AIX. > >> > > It was introduced with XLC 13.1 . Christoph found some info about it > >> > > here : > >> > > > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > >> > visibility-part2/index.html > >> > > > >> > > Setting it only generates hundreds of warnings in the build log , > >> > > warnings look like this : > >> > > XlC12.1 > >> > > > >> > > bash-4.4$ xlC -qversion > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> > > Version: 12.01.0000.0019 > >> > > > >> > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > >> > > of valid options. > >> > > > >> > > Compare to XLC13.1 > >> > > > >> > > bash-3.00$ xlC -qversion > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> > > Version: 13.01.0000.0008 > >> > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > > > >> > > > >> > > So it is better to avoid setting these flags when using xlc12.1 . > >> > > Please review : > >> > > > >> > > Bug : > >> > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 > >> > > > >> > > Change : > >> > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> > > > >> > > > >> > > Best regards, Matthias From christoph.langer at sap.com Thu Jun 7 11:51:36 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 7 Jun 2018 11:51:36 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: References: Message-ID: <67fa98e2ed454a208af23a7a56e4627b@sap.com> Hi Matthias, in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? Also, mapFileName can't be used at this place, because it has already been free'ed there. But in general the additions make sense and will make it easier to find issues in the tzmappings file. Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 7. Juni 2018 13:35 To: core-libs-dev at openjdk.java.net Cc: Langer, Christoph ; Lindenmaier, Goetz Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From matthias.baesken at sap.com Thu Jun 7 12:03:01 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 7 Jun 2018 12:03:01 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <67fa98e2ed454a208af23a7a56e4627b@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> Message-ID: Hi Christoph you are correct I compiled it on the wrong platforms , stupid mistake ! Will send an updated web rev . * Also, mapFileName can't be used at this place, because it has already been free'ed there. Yes, makes sense. Will move the free-calls . Best regards, Matthias From: Langer, Christoph Sent: Donnerstag, 7. Juni 2018 13:52 To: Baesken, Matthias ; core-libs-dev at openjdk.java.net Cc: Lindenmaier, Goetz Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi Matthias, in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? Also, mapFileName can't be used at this place, because it has already been free'ed there. But in general the additions make sense and will make it easier to find issues in the tzmappings file. Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 7. Juni 2018 13:35 To: core-libs-dev at openjdk.java.net Cc: Langer, Christoph >; Lindenmaier, Goetz > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From sean.coffey at oracle.com Thu Jun 7 12:30:10 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 7 Jun 2018 13:30:10 +0100 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> Message-ID: <00ad2878-3f6c-4f5e-5529-baf76cfc275a@oracle.com> On 07/06/2018 13:03, Baesken, Matthias wrote: > Hi Christoph you are correct I compiled it on the wrong platforms , stupid mistake ! > Will send an updated web rev . > > > * Also, mapFileName can't be used at this place, because it has already been free'ed there. > > Yes, makes sense. Will move the free-calls . Please make sure you run some manual tests also to ensure the message looks correct. Not sure if you really need mapFileName to be printed - it's generally a static file within java.home regards, Sean. > > Best regards, Matthias > > From: Langer, Christoph > Sent: Donnerstag, 7. Juni 2018 13:52 > To: Baesken, Matthias ; core-libs-dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > Hi Matthias, > > in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? > > Also, mapFileName can't be used at this place, because it has already been free'ed there. > > But in general the additions make sense and will make it easier to find issues in the tzmappings file. > > Best regards > Christoph > > From: Baesken, Matthias > Sent: Donnerstag, 7. Juni 2018 13:35 > To: core-libs-dev at openjdk.java.net > Cc: Langer, Christoph >; Lindenmaier, Goetz > > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > Hi, could you please review this small change that improves the error messages in matchJavaTZ . > A reason of the failure is added to the message , and also the offset where the error happened . > > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8204539 > > > Webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ > > > Thanks, Matthias From takiguc at linux.vnet.ibm.com Thu Jun 7 12:53:06 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Thu, 07 Jun 2018 21:53:06 +0900 Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags Message-ID: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> Hello. Could you review it ? Bug: https://bugs.openjdk.java.net/browse/JDK-8204541 Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ Thanks, Ichiroh Takiguchi IBM Japan, Ltd. -------- Original Message -------- Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Date: 2018-06-07 20:43 From: "Langer, Christoph" To: Ichiroh Takiguchi Cc: "'build-dev at openjdk.java.net'" , "ppc-aix-port-dev at openjdk.java.net" , "core-libs-dev at openjdk.java.net" , "Lindenmaier, Goetz" , "Baesken, Matthias" Hi Ichiroh, your proposal seems to make sense. I have created a bug for this: https://bugs.openjdk.java.net/browse/JDK-8204541 Can you please generate a webrev (referencing this bug, -c option of webrev.ksh) and mail it over to me. Then I'll upload it and you can post an official RFR mail. Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Dienstag, 5. Juni 2018 08:59 > To: Baesken, Matthias > Cc: Langer, Christoph ; 'build- > dev at openjdk.java.net' ; ppc-aix-port- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > Goetz > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > xlc 12.1 > > Hello Matthias and Christoph. > Thank you for your explanations. > > I did not have enough knowledge about "visibility". > > I created following patches. > > ================================ > diff -r 02934b0d661b > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed > May > 30 14:46:28 2018 +0200 > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Tue > Jun > 05 12:10:41 2018 +0900 > @@ -39,7 +39,9 @@ > #include "imageFile.hpp" > #include "inttypes.hpp" > #include "jimage.hpp" > +#if !defined(_AIX) > #include "osSupport.hpp" > +#endif > > #include "jdk_internal_jimage_NativeImageBuffer.h" > > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 > 2018 +0200 > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 > 2018 +0900 > @@ -29,7 +29,8 @@ > #ifndef __has_attribute > #define __has_attribute(x) 0 > #endif > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) > #ifdef ARM > #define JNIEXPORT > __attribute__((externally_visible,visibility("default"))) > #define JNIIMPORT > __attribute__((externally_visible,visibility("default"))) > ================================ > > If "osSupport.hpp" was included, XLC++ compiler complained. > I could not understand, which part was invalid... > I'm not sure, "osSupport.hpp" is really required on > NativeImageBuffer.cpp or not... > > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. > [1] > 0xD01 means 13.1 by hexadecimal. > > I checked symbol table by "dump -Tv -X64". [2] > It seemed it was fine, some symbols were hidden. > > Does someone review my code? > > [1] > https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html > [2] > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > visibility/index.html > > On 2018-06-01 21:12, Baesken, Matthias wrote: > > Hi , my change 8202322 just handled the fact that the > > visibility - flags are not supported with xlc 12.1 , so setting > > them generated a TON of compile - time warnings . > > > > The introduction of the "-qvisibility=hidden" came with the > > mapfile removal changes : > > > > 8200358: Remove mapfiles for JDK executables > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > > > > 8200178: Remove mapfiles for JDK native libraries > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > > > > I guess it might need further testing+adjustments to make the > > "visibility hiding" work nicely with XLC13 , but currently we > > build only with XLC12 . > > > > As a workaround you might want to remove the "-qvisibility=hidden" > > setting for XLC 13 as well , like I did for XLC12 with the change > > 8202322 . > > > > > > Best regards, Matthias > > > > > > > > > >> -----Original Message----- > >> From: Langer, Christoph > >> Sent: Freitag, 1. Juni 2018 10:57 > >> To: Ichiroh Takiguchi > >> Cc: Baesken, Matthias ; 'build- > >> dev at openjdk.java.net' ; ppc-aix-port- > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> Goetz > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> on xlc 12.1 > >> > >> Hi Ichiroh, > >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe > >> nobody of my colleagues has played with it yet. So you are on a new > >> playground here ?? > >> > >> However, I believe the idea in OpenJDK with the abolition of map files > >> is that > >> symbols should be invisible externally unless they are declared > >> exported, > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the > >> correct > >> default and whatever JNIEXPORT expands to should contain the right > >> attributes to get that symbol visible. > >> > >> Can you check if either my assumption is completely wrong, JNIEXPORT > >> does > >> not expand to the right thing, XLC 13 has a bug or maybe just sume > >> specific > >> required symbols are not declared correctly? > >> > >> Best regards > >> Christoph > >> > >> > -----Original Message----- > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> > Sent: Donnerstag, 31. Mai 2018 09:55 > >> > To: Langer, Christoph > >> > Cc: Baesken, Matthias ; 'build- > >> > dev at openjdk.java.net' ; ppc-aix-port- > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> > Goetz > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> > > >> > Hello. > >> > 8202322 was integrated into jdk-11+15. > >> > I'm using XLC 13.1.3 on AIX 7.1.4. > >> > Build was failed because of "-qvisibility=hidden" on > >> > make/lib/LibCommon.gmk. > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], > >> > "-qvisibility=hidden" cannot create shared libraries entry points. > >> > For example, libverify.so was there, but entry points were not resolved > >> > by "-lverify" option. > >> > I think it should be "-qvisibility=default" (I tried, it worked) > >> > or "-qvisibility=protected" (I had not tried) ? > >> > I'm not familiar with -qvisibility option, but I'd like to find out > >> > right way. > >> > > >> > [1] > >> > > >> > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > >> > > >> > On 2018-05-16 16:08, Langer, Christoph wrote: > >> > > Hi Matthias, > >> > > > >> > > yes, reviewed. > >> > > > >> > > Best regards > >> > > Christoph > >> > > > >> > > From: Baesken, Matthias > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 > >> > > To: Langer, Christoph ; > >> > > 'build-dev at openjdk.java.net' ; > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> > > Cc: Lindenmaier, Goetz > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > > xlc 12.1 > >> > > > >> > > Hi Christoph can I add you as second reviewer (other reviewer was > >> > > Erik Joelsson) can push the change ? > >> > > > >> > > Best regards, Matthias > >> > > > >> > > > >> > > > >> > > From: Langer, Christoph > >> > > Sent: Donnerstag, 26. April 2018 16:38 > >> > > To: Baesken, Matthias > >> > > >; > >> > > 'build-dev at openjdk.java.net' > >> > > dev at openjdk.java.net>>; > >> > > ppc-aix-port-dev at openjdk.java.net >> > dev at openjdk.java.net>; > >> > > core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> > >> > > Cc: Simonis, Volker > >> > > > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > > xlc 12.1 > >> > > > >> > > Hi Matthias, > >> > > > >> > > to me the change in principal looks good. > >> > > > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > >> > > extract major number before the first dot, then compare numerically) > - > >> > > but maybe it is too complicated and the current single version > compare > >> > > suits our needs ? > >> > > > >> > > Best regards > >> > > Christoph > >> > > > >> > > From: Baesken, Matthias > >> > > Sent: Donnerstag, 26. April 2018 16:14 > >> > > To: 'build-dev at openjdk.java.net' > >> > > dev at openjdk.java.net>>; > >> > > ppc-aix-port-dev at openjdk.java.net >> > dev at openjdk.java.net>; > >> > > core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> > >> > > Cc: Langer, Christoph > >> > > >; > >> Simonis, > >> > > Volker > > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > >> > > 12.1 > >> > > > >> > > Hello , could you please review this small adjustment to the symbol > >> > > visibility compilation settings on AIX ? > >> > > Currently we use XLC 12.1 to compile JDK on AIX . > >> > > > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" > >> > > setting currently set on AIX. > >> > > It was introduced with XLC 13.1 . Christoph found some info about it > >> > > here : > >> > > > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > >> > visibility-part2/index.html > >> > > > >> > > Setting it only generates hundreds of warnings in the build log , > >> > > warnings look like this : > >> > > XlC12.1 > >> > > > >> > > bash-4.4$ xlC -qversion > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> > > Version: 12.01.0000.0019 > >> > > > >> > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > >> > > of valid options. > >> > > > >> > > Compare to XLC13.1 > >> > > > >> > > bash-3.00$ xlC -qversion > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> > > Version: 13.01.0000.0008 > >> > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > > > >> > > > >> > > So it is better to avoid setting these flags when using xlc12.1 . > >> > > Please review : > >> > > > >> > > Bug : > >> > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 > >> > > > >> > > Change : > >> > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> > > > >> > > > >> > > Best regards, Matthias From christoph.langer at sap.com Thu Jun 7 13:06:42 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 7 Jun 2018 13:06:42 +0000 Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags In-Reply-To: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> References: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> Message-ID: Hi Ichiroh, what's the exact error message if you #include "osSupport.hpp"? (I have no xlC 13 at hand to try myself...) Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Donnerstag, 7. Juni 2018 14:53 > To: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; core- > libs-dev at openjdk.java.net > Cc: Lindenmaier, Goetz ; Baesken, Matthias > ; Langer, Christoph > > Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags > > Hello. > > Could you review it ? > Bug: https://bugs.openjdk.java.net/browse/JDK-8204541 > Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ > > Thanks, > Ichiroh Takiguchi > IBM Japan, Ltd. > > -------- Original Message -------- > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > xlc 12.1 > Date: 2018-06-07 20:43 > From: "Langer, Christoph" > To: Ichiroh Takiguchi > Cc: "'build-dev at openjdk.java.net'" , > "ppc-aix-port-dev at openjdk.java.net" dev at openjdk.java.net>, > "core-libs-dev at openjdk.java.net" , > "Lindenmaier, Goetz" , "Baesken, Matthias" > > > Hi Ichiroh, > > your proposal seems to make sense. I have created a bug for this: > https://bugs.openjdk.java.net/browse/JDK-8204541 > > Can you please generate a webrev (referencing this bug, -c option of > webrev.ksh) and mail it over to me. Then I'll upload it and you can post > an official RFR mail. > > Best regards > Christoph > > > -----Original Message----- > > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > > Sent: Dienstag, 5. Juni 2018 08:59 > > To: Baesken, Matthias > > Cc: Langer, Christoph ; 'build- > > dev at openjdk.java.net' ; ppc-aix-port- > > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > > Goetz > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > xlc 12.1 > > > > Hello Matthias and Christoph. > > Thank you for your explanations. > > > > I did not have enough knowledge about "visibility". > > > > I created following patches. > > > > ================================ > > diff -r 02934b0d661b > > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed > > May > > 30 14:46:28 2018 +0200 > > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp > Tue > > Jun > > 05 12:10:41 2018 +0900 > > @@ -39,7 +39,9 @@ > > #include "imageFile.hpp" > > #include "inttypes.hpp" > > #include "jimage.hpp" > > +#if !defined(_AIX) > > #include "osSupport.hpp" > > +#endif > > > > #include "jdk_internal_jimage_NativeImageBuffer.h" > > > > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 > > 2018 +0200 > > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 > > 2018 +0900 > > @@ -29,7 +29,8 @@ > > #ifndef __has_attribute > > #define __has_attribute(x) 0 > > #endif > > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) > > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ > > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) > > #ifdef ARM > > #define JNIEXPORT > > __attribute__((externally_visible,visibility("default"))) > > #define JNIIMPORT > > __attribute__((externally_visible,visibility("default"))) > > ================================ > > > > If "osSupport.hpp" was included, XLC++ compiler complained. > > I could not understand, which part was invalid... > > I'm not sure, "osSupport.hpp" is really required on > > NativeImageBuffer.cpp or not... > > > > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. > > [1] > > 0xD01 means 13.1 by hexadecimal. > > > > I checked symbol table by "dump -Tv -X64". [2] > > It seemed it was fine, some symbols were hidden. > > > > Does someone review my code? > > > > [1] > > > https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i > > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html > > [2] > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > > visibility/index.html > > > > On 2018-06-01 21:12, Baesken, Matthias wrote: > > > Hi , my change 8202322 just handled the fact that the > > > visibility - flags are not supported with xlc 12.1 , so setting > > > them generated a TON of compile - time warnings . > > > > > > The introduction of the "-qvisibility=hidden" came with the > > > mapfile removal changes : > > > > > > 8200358: Remove mapfiles for JDK executables > > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > > > > > > 8200178: Remove mapfiles for JDK native libraries > > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > > > > > > I guess it might need further testing+adjustments to make the > > > "visibility hiding" work nicely with XLC13 , but currently we > > > build only with XLC12 . > > > > > > As a workaround you might want to remove the "-qvisibility=hidden" > > > setting for XLC 13 as well , like I did for XLC12 with the change > > > 8202322 . > > > > > > > > > Best regards, Matthias > > > > > > > > > > > > > > >> -----Original Message----- > > >> From: Langer, Christoph > > >> Sent: Freitag, 1. Juni 2018 10:57 > > >> To: Ichiroh Takiguchi > > >> Cc: Baesken, Matthias ; 'build- > > >> dev at openjdk.java.net' ; ppc-aix-port- > > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > > >> Goetz > > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > > >> on xlc 12.1 > > >> > > >> Hi Ichiroh, > > >> > > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe > > >> nobody of my colleagues has played with it yet. So you are on a new > > >> playground here ?? > > >> > > >> However, I believe the idea in OpenJDK with the abolition of map files > > >> is that > > >> symbols should be invisible externally unless they are declared > > >> exported, > > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the > > >> correct > > >> default and whatever JNIEXPORT expands to should contain the right > > >> attributes to get that symbol visible. > > >> > > >> Can you check if either my assumption is completely wrong, JNIEXPORT > > >> does > > >> not expand to the right thing, XLC 13 has a bug or maybe just sume > > >> specific > > >> required symbols are not declared correctly? > > >> > > >> Best regards > > >> Christoph > > >> > > >> > -----Original Message----- > > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > > >> > Sent: Donnerstag, 31. Mai 2018 09:55 > > >> > To: Langer, Christoph > > >> > Cc: Baesken, Matthias ; 'build- > > >> > dev at openjdk.java.net' ; ppc-aix-port- > > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > Lindenmaier, > > >> > Goetz > > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > xlc > > 12.1 > > >> > > > >> > Hello. > > >> > 8202322 was integrated into jdk-11+15. > > >> > I'm using XLC 13.1.3 on AIX 7.1.4. > > >> > Build was failed because of "-qvisibility=hidden" on > > >> > make/lib/LibCommon.gmk. > > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], > > >> > "-qvisibility=hidden" cannot create shared libraries entry points. > > >> > For example, libverify.so was there, but entry points were not > resolved > > >> > by "-lverify" option. > > >> > I think it should be "-qvisibility=default" (I tried, it worked) > > >> > or "-qvisibility=protected" (I had not tried) ? > > >> > I'm not familiar with -qvisibility option, but I'd like to find out > > >> > right way. > > >> > > > >> > [1] > > >> > > > >> > > > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > > >> > > > >> > On 2018-05-16 16:08, Langer, Christoph wrote: > > >> > > Hi Matthias, > > >> > > > > >> > > yes, reviewed. > > >> > > > > >> > > Best regards > > >> > > Christoph > > >> > > > > >> > > From: Baesken, Matthias > > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 > > >> > > To: Langer, Christoph ; > > >> > > 'build-dev at openjdk.java.net' ; > > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs- > dev at openjdk.java.net > > >> > > Cc: Lindenmaier, Goetz > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > >> > > xlc 12.1 > > >> > > > > >> > > Hi Christoph can I add you as second reviewer (other reviewer was > > >> > > Erik Joelsson) can push the change ? > > >> > > > > >> > > Best regards, Matthias > > >> > > > > >> > > > > >> > > > > >> > > From: Langer, Christoph > > >> > > Sent: Donnerstag, 26. April 2018 16:38 > > >> > > To: Baesken, Matthias > > >> > > > >; > > >> > > 'build-dev at openjdk.java.net' > > >> > > > dev at openjdk.java.net>>; > > >> > > ppc-aix-port-dev at openjdk.java.net > >> > dev at openjdk.java.net>; > > >> > > core-libs-dev at openjdk.java.net > >> dev at openjdk.java.net> > > >> > > Cc: Simonis, Volker > > >> > > > > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > >> > > xlc 12.1 > > >> > > > > >> > > Hi Matthias, > > >> > > > > >> > > to me the change in principal looks good. > > >> > > > > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > > >> > > extract major number before the first dot, then compare > numerically) > > - > > >> > > but maybe it is too complicated and the current single version > > compare > > >> > > suits our needs ? > > >> > > > > >> > > Best regards > > >> > > Christoph > > >> > > > > >> > > From: Baesken, Matthias > > >> > > Sent: Donnerstag, 26. April 2018 16:14 > > >> > > To: 'build-dev at openjdk.java.net' > > >> > > > dev at openjdk.java.net>>; > > >> > > ppc-aix-port-dev at openjdk.java.net > >> > dev at openjdk.java.net>; > > >> > > core-libs-dev at openjdk.java.net > >> dev at openjdk.java.net> > > >> > > Cc: Langer, Christoph > > >> > > >; > > >> Simonis, > > >> > > Volker > > > > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > > >> > > 12.1 > > >> > > > > >> > > Hello , could you please review this small adjustment to the symbol > > >> > > visibility compilation settings on AIX ? > > >> > > Currently we use XLC 12.1 to compile JDK on AIX . > > >> > > > > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" > > >> > > setting currently set on AIX. > > >> > > It was introduced with XLC 13.1 . Christoph found some info about it > > >> > > here : > > >> > > > > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > > >> > visibility-part2/index.html > > >> > > > > >> > > Setting it only generates hundreds of warnings in the build log , > > >> > > warnings look like this : > > >> > > XlC12.1 > > >> > > > > >> > > bash-4.4$ xlC -qversion > > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > > >> > > Version: 12.01.0000.0019 > > >> > > > > >> > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > > >> > > of valid options. > > >> > > > > >> > > Compare to XLC13.1 > > >> > > > > >> > > bash-3.00$ xlC -qversion > > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > > >> > > Version: 13.01.0000.0008 > > >> > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > > >> > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > >> > > > > >> > > > > >> > > So it is better to avoid setting these flags when using xlc12.1 . > > >> > > Please review : > > >> > > > > >> > > Bug : > > >> > > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 > > >> > > > > >> > > Change : > > >> > > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > > >> > > > > >> > > > > >> > > Best regards, Matthias From srinivas.dama at oracle.com Thu Jun 7 14:01:38 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 7 Jun 2018 07:01:38 -0700 (PDT) Subject: RFR: 8196990 :Resolve disabled warnings for libjli Message-ID: Hi, Please review http://cr.openjdk.java.net/~sdama/8196990/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196990 Regards, Srinivas From tprintezis at twitter.com Thu Jun 7 14:07:25 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 7 Jun 2018 07:07:25 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <160715cd-9d39-0c3f-21f7-f029ef4139c6@gmail.com> References: <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <160715cd-9d39-0c3f-21f7-f029ef4139c6@gmail.com> Message-ID: Peter, Inline. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 7, 2018 at 5:21:43 AM, Peter Levart (peter.levart at gmail.com) wrote: Hi Tony, Thanks for taking a look. Just a couple of comments inline... On 06/06/18 22:38, Tony Printezis wrote: - instead of overriding initialValue(), ThreadLocal.setInitialValue() calls TerminatingThreadLocal.register() at the right moment Thanks. I much prefer not having to introduce calculateInitialValue(). One extra suggestion: Given you introduced a call to TerminatingThreadLocal from ThreadLocal, would it make sense to maybe do the same for set() and remove() (you?d have to add an appropriate check in unregister) and not override them in TerminatingTreadLocal? I totally get why you might not want to do that (an extra check when a ThreadLocal is not the Terminating one). I?m OK either way. Yes, precisely. My 1st version did just that, but since there are usages of ThreadLocal out there that are very performance sensitive, I opted for an approach where there is zero performance regression for non-TerminatingThreadLocal(s) when calling set() or remove(). A call-back from setInitialValue is not so problematic as it usually occurs just once per thread, but I can imagine a scenario where calls to get() and set() occur very rapidly. Yeah, I agree that this is a good trade-off given how the code is currently structured. An alternative to "T getIfPresent()" is a "boolean isPresent()" method. Even if it remains package-private, with such method TerminatingThreadLocal could be implemented as: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ If TreadLocal.isPresent() was made public, the code could be further simplified. Something like: if (tl.isPresent()) { T val = t.get(); ?. } will do two look-ups if the value exists. However, that?s better than allocating unnecessarily. So, I?ll take isPresent() over not having a way to check whether a value exists. Right, and in our scenario, it is just isPresent() that is called for every termination of every thread (REGISTRY.isPresent()). .get() is then called only when there's actually something to do. Yes. I?m not worried about that at all. Thumbs up from me. Let's just wait for a day or two to see whether the discussion on concurrency-interest gives us any additional ideas. Currently I'm thinking of proposing the addition of isPresent() API. As far as this RFR is concerned, I'm consequently promoting the latest webrev.04. Again, #ThumbsUp from me. Thanks, Tony So any comments on that one? Alan? Regards, Peter From james.laskey at oracle.com Thu Jun 7 14:07:46 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 7 Jun 2018 11:07:46 -0300 Subject: RFR: 8196990 :Resolve disabled warnings for libjli In-Reply-To: References: Message-ID: <830AE99F-314D-4965-A6D3-1A36DCFA0479@oracle.com> traditionally there is a space after // simplify comment to something like (rest is redundant) // initialize to avoid -Werror=maybe-uninitialized issues from gcc 7.3 onwards. > On Jun 7, 2018, at 11:01 AM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196990/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196990 > > Regards, > Srinivas From srinivas.dama at oracle.com Thu Jun 7 15:14:55 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 7 Jun 2018 08:14:55 -0700 (PDT) Subject: RFR: 8196990 :Resolve disabled warnings for libjli Message-ID: Ok. Thank you Jim. Please find revised webrev at http://cr.openjdk.java.net/~sdama/8196990/webrev.01/. Regards, Srinivas ----- Original Message ----- From: james.laskey at oracle.com To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net Sent: Thursday, 7 June, 2018 7:37:48 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: RFR: 8196990 :Resolve disabled warnings for libjli traditionally there is a space after // simplify comment to something like (rest is redundant) // initialize to avoid -Werror=maybe-uninitialized issues from gcc 7.3 onwards. > On Jun 7, 2018, at 11:01 AM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196990/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196990 > > Regards, > Srinivas From james.laskey at oracle.com Thu Jun 7 15:16:16 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 7 Jun 2018 12:16:16 -0300 Subject: RFR: 8196990 :Resolve disabled warnings for libjli In-Reply-To: References: Message-ID: <218D2C33-C208-407C-987B-1E9D40420687@oracle.com> +1 > On Jun 7, 2018, at 12:14 PM, Srinivas Dama wrote: > > Ok. Thank you Jim. > Please find revised webrev at http://cr.openjdk.java.net/~sdama/8196990/webrev.01/. > > Regards, > Srinivas > > ----- Original Message ----- > From: james.laskey at oracle.com > To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net > Sent: Thursday, 7 June, 2018 7:37:48 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: Re: RFR: 8196990 :Resolve disabled warnings for libjli > > traditionally there is a space after // > > simplify comment to something like (rest is redundant) > > // initialize to avoid -Werror=maybe-uninitialized issues from gcc 7.3 onwards. > > > >> On Jun 7, 2018, at 11:01 AM, Srinivas Dama wrote: >> >> Hi, >> >> Please review http://cr.openjdk.java.net/~sdama/8196990/webrev.00/ >> for https://bugs.openjdk.java.net/browse/JDK-8196990 >> >> Regards, >> Srinivas > From takiguc at linux.vnet.ibm.com Thu Jun 7 16:29:09 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 08 Jun 2018 01:29:09 +0900 Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags In-Reply-To: References: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> Message-ID: <29f9a3494ea8ebb6f5db05e4fb06fa30@linux.vnet.ibm.com> Hello Christoph According to build log, if <#include "osSupport.hpp"> was there: "/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server-release/support/headers/java.base/jdk_internal_jimage_NativeImageBuffer.h", line 15.27: 1540-0040 (S) The text "Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap" is unexpected. "visibility" may be undeclared or ambiguous. make[3]: *** [/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server-release/support/native/java.base/libjimage/NativeImageBuffer.o] Error 1 make[3]: Leaving directory `/home/jdktest/sandbox/jdk/make' make[2]: *** [java.base-libs] Error 2 make[2]: *** Waiting for unfinished jobs.... On 2018-06-07 22:06, Langer, Christoph wrote: > Hi Ichiroh, > > what's the exact error message if you #include "osSupport.hpp"? (I > have no xlC 13 at hand to try myself...) > > Best regards > Christoph > >> -----Original Message----- >> From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] >> Sent: Donnerstag, 7. Juni 2018 14:53 >> To: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; >> core- >> libs-dev at openjdk.java.net >> Cc: Lindenmaier, Goetz ; Baesken, Matthias >> ; Langer, Christoph >> >> Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol >> visibility flags >> >> Hello. >> >> Could you review it ? >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204541 >> Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ >> >> Thanks, >> Ichiroh Takiguchi >> IBM Japan, Ltd. >> >> -------- Original Message -------- >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support >> on >> xlc 12.1 >> Date: 2018-06-07 20:43 >> From: "Langer, Christoph" >> To: Ichiroh Takiguchi >> Cc: "'build-dev at openjdk.java.net'" , >> "ppc-aix-port-dev at openjdk.java.net" > dev at openjdk.java.net>, >> "core-libs-dev at openjdk.java.net" , >> "Lindenmaier, Goetz" , "Baesken, Matthias" >> >> >> Hi Ichiroh, >> >> your proposal seems to make sense. I have created a bug for this: >> https://bugs.openjdk.java.net/browse/JDK-8204541 >> >> Can you please generate a webrev (referencing this bug, -c option of >> webrev.ksh) and mail it over to me. Then I'll upload it and you can >> post >> an official RFR mail. >> >> Best regards >> Christoph >> >> > -----Original Message----- >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] >> > Sent: Dienstag, 5. Juni 2018 08:59 >> > To: Baesken, Matthias >> > Cc: Langer, Christoph ; 'build- >> > dev at openjdk.java.net' ; ppc-aix-port- >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, >> > Goetz >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > xlc 12.1 >> > >> > Hello Matthias and Christoph. >> > Thank you for your explanations. >> > >> > I did not have enough knowledge about "visibility". >> > >> > I created following patches. >> > >> > ================================ >> > diff -r 02934b0d661b >> > src/java.base/share/native/libjimage/NativeImageBuffer.cpp >> > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed >> > May >> > 30 14:46:28 2018 +0200 >> > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp >> Tue >> > Jun >> > 05 12:10:41 2018 +0900 >> > @@ -39,7 +39,9 @@ >> > #include "imageFile.hpp" >> > #include "inttypes.hpp" >> > #include "jimage.hpp" >> > +#if !defined(_AIX) >> > #include "osSupport.hpp" >> > +#endif >> > >> > #include "jdk_internal_jimage_NativeImageBuffer.h" >> > >> > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h >> > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 >> > 2018 +0200 >> > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 >> > 2018 +0900 >> > @@ -29,7 +29,8 @@ >> > #ifndef __has_attribute >> > #define __has_attribute(x) 0 >> > #endif >> > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) >> > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ >> > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) >> > #ifdef ARM >> > #define JNIEXPORT >> > __attribute__((externally_visible,visibility("default"))) >> > #define JNIIMPORT >> > __attribute__((externally_visible,visibility("default"))) >> > ================================ >> > >> > If "osSupport.hpp" was included, XLC++ compiler complained. >> > I could not understand, which part was invalid... >> > I'm not sure, "osSupport.hpp" is really required on >> > NativeImageBuffer.cpp or not... >> > >> > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. >> > [1] >> > 0xD01 means 13.1 by hexadecimal. >> > >> > I checked symbol table by "dump -Tv -X64". [2] >> > It seemed it was fine, some symbols were hidden. >> > >> > Does someone review my code? >> > >> > [1] >> > >> https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i >> > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html >> > [2] >> > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> > visibility/index.html >> > >> > On 2018-06-01 21:12, Baesken, Matthias wrote: >> > > Hi , my change 8202322 just handled the fact that the >> > > visibility - flags are not supported with xlc 12.1 , so setting >> > > them generated a TON of compile - time warnings . >> > > >> > > The introduction of the "-qvisibility=hidden" came with the >> > > mapfile removal changes : >> > > >> > > 8200358: Remove mapfiles for JDK executables >> > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 >> > > >> > > 8200178: Remove mapfiles for JDK native libraries >> > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 >> > > >> > > I guess it might need further testing+adjustments to make the >> > > "visibility hiding" work nicely with XLC13 , but currently we >> > > build only with XLC12 . >> > > >> > > As a workaround you might want to remove the "-qvisibility=hidden" >> > > setting for XLC 13 as well , like I did for XLC12 with the change >> > > 8202322 . >> > > >> > > >> > > Best regards, Matthias >> > > >> > > >> > > >> > > >> > >> -----Original Message----- >> > >> From: Langer, Christoph >> > >> Sent: Freitag, 1. Juni 2018 10:57 >> > >> To: Ichiroh Takiguchi >> > >> Cc: Baesken, Matthias ; 'build- >> > >> dev at openjdk.java.net' ; ppc-aix-port- >> > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, >> > >> Goetz >> > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support >> > >> on xlc 12.1 >> > >> >> > >> Hi Ichiroh, >> > >> >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe >> > >> nobody of my colleagues has played with it yet. So you are on a new >> > >> playground here ?? >> > >> >> > >> However, I believe the idea in OpenJDK with the abolition of map files >> > >> is that >> > >> symbols should be invisible externally unless they are declared >> > >> exported, >> > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the >> > >> correct >> > >> default and whatever JNIEXPORT expands to should contain the right >> > >> attributes to get that symbol visible. >> > >> >> > >> Can you check if either my assumption is completely wrong, JNIEXPORT >> > >> does >> > >> not expand to the right thing, XLC 13 has a bug or maybe just sume >> > >> specific >> > >> required symbols are not declared correctly? >> > >> >> > >> Best regards >> > >> Christoph >> > >> >> > >> > -----Original Message----- >> > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] >> > >> > Sent: Donnerstag, 31. Mai 2018 09:55 >> > >> > To: Langer, Christoph >> > >> > Cc: Baesken, Matthias ; 'build- >> > >> > dev at openjdk.java.net' ; ppc-aix-port- >> > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; >> Lindenmaier, >> > >> > Goetz >> > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> xlc >> > 12.1 >> > >> > >> > >> > Hello. >> > >> > 8202322 was integrated into jdk-11+15. >> > >> > I'm using XLC 13.1.3 on AIX 7.1.4. >> > >> > Build was failed because of "-qvisibility=hidden" on >> > >> > make/lib/LibCommon.gmk. >> > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], >> > >> > "-qvisibility=hidden" cannot create shared libraries entry points. >> > >> > For example, libverify.so was there, but entry points were not >> resolved >> > >> > by "-lverify" option. >> > >> > I think it should be "-qvisibility=default" (I tried, it worked) >> > >> > or "-qvisibility=protected" (I had not tried) ? >> > >> > I'm not familiar with -qvisibility option, but I'd like to find out >> > >> > right way. >> > >> > >> > >> > [1] >> > >> > >> > >> >> > >> https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. >> > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html >> > >> > >> > >> > On 2018-05-16 16:08, Langer, Christoph wrote: >> > >> > > Hi Matthias, >> > >> > > >> > >> > > yes, reviewed. >> > >> > > >> > >> > > Best regards >> > >> > > Christoph >> > >> > > >> > >> > > From: Baesken, Matthias >> > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 >> > >> > > To: Langer, Christoph ; >> > >> > > 'build-dev at openjdk.java.net' ; >> > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs- >> dev at openjdk.java.net >> > >> > > Cc: Lindenmaier, Goetz >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > >> > > xlc 12.1 >> > >> > > >> > >> > > Hi Christoph can I add you as second reviewer (other reviewer was >> > >> > > Erik Joelsson) can push the change ? >> > >> > > >> > >> > > Best regards, Matthias >> > >> > > >> > >> > > >> > >> > > >> > >> > > From: Langer, Christoph >> > >> > > Sent: Donnerstag, 26. April 2018 16:38 >> > >> > > To: Baesken, Matthias >> > >> > > >> >; >> > >> > > 'build-dev at openjdk.java.net' >> > >> > > > > dev at openjdk.java.net>>; >> > >> > > ppc-aix-port-dev at openjdk.java.net> > >> > dev at openjdk.java.net>; >> > >> > > core-libs-dev at openjdk.java.net> > >> dev at openjdk.java.net> >> > >> > > Cc: Simonis, Volker >> > >> > > > >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > >> > > xlc 12.1 >> > >> > > >> > >> > > Hi Matthias, >> > >> > > >> > >> > > to me the change in principal looks good. >> > >> > > >> > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. >> > >> > > extract major number before the first dot, then compare >> numerically) >> > - >> > >> > > but maybe it is too complicated and the current single version >> > compare >> > >> > > suits our needs ? >> > >> > > >> > >> > > Best regards >> > >> > > Christoph >> > >> > > >> > >> > > From: Baesken, Matthias >> > >> > > Sent: Donnerstag, 26. April 2018 16:14 >> > >> > > To: 'build-dev at openjdk.java.net' >> > >> > > > > dev at openjdk.java.net>>; >> > >> > > ppc-aix-port-dev at openjdk.java.net> > >> > dev at openjdk.java.net>; >> > >> > > core-libs-dev at openjdk.java.net> > >> dev at openjdk.java.net> >> > >> > > Cc: Langer, Christoph >> > >> > > >; >> > >> Simonis, >> > >> > > Volker >> > >> > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> > >> > > 12.1 >> > >> > > >> > >> > > Hello , could you please review this small adjustment to the symbol >> > >> > > visibility compilation settings on AIX ? >> > >> > > Currently we use XLC 12.1 to compile JDK on AIX . >> > >> > > >> > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" >> > >> > > setting currently set on AIX. >> > >> > > It was introduced with XLC 13.1 . Christoph found some info about it >> > >> > > here : >> > >> > > >> > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> > >> > visibility-part2/index.html >> > >> > > >> > >> > > Setting it only generates hundreds of warnings in the build log , >> > >> > > warnings look like this : >> > >> > > XlC12.1 >> > >> > > >> > >> > > bash-4.4$ xlC -qversion >> > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) >> > >> > > Version: 12.01.0000.0019 >> > >> > > >> > >> > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list >> > >> > > of valid options. >> > >> > > >> > >> > > Compare to XLC13.1 >> > >> > > >> > >> > > bash-3.00$ xlC -qversion >> > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) >> > >> > > Version: 13.01.0000.0008 >> > >> > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc >> > >> > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> > >> > > >> > >> > > >> > >> > > So it is better to avoid setting these flags when using xlc12.1 . >> > >> > > Please review : >> > >> > > >> > >> > > Bug : >> > >> > > >> > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 >> > >> > > >> > >> > > Change : >> > >> > > >> > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ >> > >> > > >> > >> > > >> > >> > > Best regards, Matthias From bob.vandette at oracle.com Thu Jun 7 17:43:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 7 Jun 2018 13:43:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: Can I get one more reviewer for this RFE so I can integrate it? > http://cr.openjdk.java.net/~bobv/8203357/webrev.01 Mandy Chung has reviewed this change. I?ve run Mach5 hotspot and core lib tests. I?ve reviewed the tests which were written by Harsha Wardhana I filed a CSR for the command line change and it?s now approved and closed. Thanks, Bob. > On May 30, 2018, at 3:45 PM, Bob Vandette wrote: > > Please review the following RFE which adds an internal API, along with jtreg tests that provide > access to Docker container configuration data and metrics. In addition to the API which we hope to > take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional > option to -XshowSettings:system than dumps out the container or host cgroup confguration > information. See the sample output below: > > RFE: Container Metrics > > https://bugs.openjdk.java.net/browse/JDK-8203357 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > > This commit will also include a fix for the following bug. > > BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails > > https://bugs.openjdk.java.net/browse/JDK-8203691 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html > > SAMPLE USAGE and OUTPUT: > > docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash > ./java -XshowSettings:system > Operating System Metrics: > Provider: cgroupv1 > Effective CPU Count: 4 > CPU Period: 100000 > CPU Quota: -1 > CPU Shares: -1 > List of Processors, 4 total: > 4 5 6 7 > List of Effective Processors, 4 total: > 4 5 6 7 > List of Memory Nodes, 2 total: > 0 1 > List of Available Memory Nodes, 2 total: > 0 1 > CPUSet Memory Pressure Enabled: false > Memory Limit: 256.00M > Memory Soft Limit: Unlimited > Memory & Swap Limit: 512.00M > Kernel Memory Limit: Unlimited > TCP Memory Limit: Unlimited > Out Of Memory Killer Enabled: true > > TEST RESULTS: > > testing runtime container APIs > Directory "JTwork" not found: creating > Passed: runtime/containers/cgroup/PlainRead.java > Passed: runtime/containers/docker/DockerBasicTest.java > Passed: runtime/containers/docker/TestCPUAwareness.java > Passed: runtime/containers/docker/TestCPUSets.java > Passed: runtime/containers/docker/TestMemoryAwareness.java > Passed: runtime/containers/docker/TestMisc.java > Test results: passed: 6 > Results written to /export/users/bobv/jdk11/build/jtreg/JTwork > > testing jdk.internal.platform APIs > Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java > Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java > Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java > Passed: jdk/internal/platform/docker/TestSystemMetrics.java > Test results: passed: 4 > Results written to /export/users/bobv/jdk11/build/jtreg/JTwork > > testing -XshowSettings:system launcher option > Passed: tools/launcher/Settings.java > Test results: passed: 1 > > > Bob. > > From brent.christian at oracle.com Thu Jun 7 19:13:46 2018 From: brent.christian at oracle.com (Brent Christian) Date: Thu, 7 Jun 2018 12:13:46 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to $FEATURE Message-ID: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> Hi, Please review this doc-only change. From the bug report: 'With the integration of JEP 322 "Time-Based Release Versioning" into JDK 10, VERSION_FEATURE is used to set the value of the system properties "java.specification.version" [1] and "java.vm.specification.version" [2] (though the term "major" is still used in VM code, see JDK-8193719). We can update the System.getProperties() javadoc to be more specific about the value reported in these system properties.' Issue: https://bugs.openjdk.java.net/browse/JDK-8204565 Webrev: http://cr.openjdk.java.net/~bchristi/8204565/webrev/ Thanks, -Brent 1. http://hg.openjdk.java.net/jdk/jdk/rev/d2a837cf9ff1#l6.23 2. http://hg.openjdk.java.net/jdk/jdk/rev/d2a837cf9ff1#l15.17 From magnus.ihse.bursie at oracle.com Thu Jun 7 20:22:48 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 7 Jun 2018 22:22:48 +0200 Subject: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically Message-ID: This change needs some background information. I've been working on simplifying and streamlining the compilation of native libraries of the JDK. Previously, I introduced the SetupJdkLibrary function, and started to get a better control of compiler flags. This patch continues on both paths. The original intent of this change, which I naively thought was going to be much simpler than it turned out :-) was to separate the -I flags (location of header files) from other flags, and to generate these automatically, wherever possible. Since we have a standard way of locating native code, and most libraries just want to have an -I path to their own source base and the generated Java header, we should be able to provide this in SetupJdkLibrary. This turned out to be closely related to SetupJdkLibrary being able to discover the location of the SRC directories itself, using "convention over configuration" and assuming that the library "libfoo" for "java.module" would be located in java.module/*/native/libfoo. While this sounds simple in theory, when actually trying to implement this I of course ran into all the places where some special handling was indeed needed. So even if like 90% of all libraries were simple to get to build using automated discovery of source and header directories, the 10% that did not caused me much more headaches than I had anticipated. On the other hand, now that I've sorted out all those places, the few remaining odd solutions is clearly documented and not just something that "just happens" due to strange configurations. One file deserves mentioning specifically: Awt2dLibraries.gmk. The java.desktop libraries are unfortunately quite entangled with each other, and do not readily follow the patterns that are used elsewhere in the code base. So it might just look like the file has just gone from one state of messiness, to another, which would hardly be an improvement. :-( I would still argue that the new messiness is better: It is now much clearer in what ways the libraries diverge from our standard assumption, and what course of action needs to be taken to minimize these differences. (Which is something I believe should be done -- these issues are not just cosmetic but is the root of most of the issues we always see for these libraries, when upgrading compilers, etc.) During this change, I noticed that not all native libraries include the proper generated header file. This is a dangerous coding practice, since a change in the Java part of the interface might not get picked up properly in the native part. I've added the missing includes that I've detected, and due to these changes, I'm also including the component teams in what is really only a build change. As can be seen for jdk.crypto.mscapi; there had indeed been changes that needed correcting. Since this is (basically) a pure build change, my gold standard here has been the build compare script. In essence, the build output prior to my change and with this change are 100% identical. In truth, this is a bit too strong a claim. A few changes has occurred, but none of them should matter. Here's a breakdown of the compare.sh results: * Windows-x64: No differences at all. * Solaris: Two libraries are reported to differ: libsaproc.so and libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing this, I found that the object files used to link these two libraries has no disass differences. They have a slight binary difference and a difference in size, due to the include paths being different (and this is stored in the .o file, which makes it different). Somehow this apparently triggers the linker to generate a slightly different code in a few places, using a different register or so. (Weird...) * MacOS: Two libraries are reported to differ: libjava.dylib and libmlib_image.dylib, both of them just reported as a binary diff (no symbol, disass or fulldump differences). This is not really unsuspected, but I analyzed it anyway. I found that for libjava.dylib, a single .o file was different. This one was actually picked up from closed sources, and are not really relevant for OpenJDK. Anyway, the reason for the difference was the same as for the Solaris libs; the include paths had changes, which caused a binary diff. For libmlib_image.dylib, the link order had changed causing the noted binary difference. * Linux: On linux, the compare script noted differences for libextnet, libjava, libmlib_image, libprefs, libsaproc, libsplashscreen and libsunec. The differences for libextnet, libprefs, libsplashscreen and libsunec turned out to be caused by the added #include of the generated Java headers. This caused binary differences (reasonably), and for some odd reason also a symbol difference in java_awt_SplashScreen.o (clazz.10057 and mid.10058 were replaced by clazz.10015 and mid.10016). I can't claim to understand this, but I'm assuming it's some kind of generated code. libsaproc and libjava changes was caused by closed source changes, and is therefore not relevant to OpenJDK. For libmlib_image.dylib, the link order had changed causing the noted binary difference, as on MacOS. Bug: https://bugs.openjdk.java.net/browse/JDK-8204572 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.01 /Magnus From mandy.chung at oracle.com Thu Jun 7 20:24:49 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Jun 2018 13:24:49 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to $FEATURE In-Reply-To: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> References: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> Message-ID: <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> Hi Brent, On 6/7/18 12:13 PM, Brent Christian wrote: > Hi, > > Please review this doc-only change.? From the bug report: > > 'With the integration of JEP 322 "Time-Based Release Versioning" > into JDK 10, VERSION_FEATURE is used to set the value of the system > properties "java.specification.version" [1] and > "java.vm.specification.version" [2] (though the term "major" is still > used in VM code, see JDK-8193719). > > We can update the System.getProperties() javadoc to be more specific > about the value reported in these system properties.' > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8204565 > > Webrev: > http://cr.openjdk.java.net/~bchristi/8204565/webrev/ Is there an existing test validating this? If not, we should add a test for this change? Mandy From jonathan.gibbons at oracle.com Thu Jun 7 21:19:23 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 07 Jun 2018 14:19:23 -0700 Subject: Locale languageTag anomaly Message-ID: <5B19A15B.3000702@oracle.com> Regarding Locale.toLanguageTag, Locale.forLanguageTag Should I be surprised (i.e. is it a bug) that out of 736 installed locales in a standard build of JDK, exactly 1 locale fails the following round-trip test: locale.equals(Locale.forLanguageTag(locale.toLanguageTag())) The locale "no_NO_NY" comes back as "nn_NO". Attached is a test program. Here is the output from the test program: $ /opt/jdk/1.9.0/bin/java -cp play/locales/classes Locales test locale: no_NO_NY [no,NO,NY] nn-NO forLanguageTag: nn_NO Tested 736 locales Mismatch 1 locales -- Jon From erik.joelsson at oracle.com Thu Jun 7 21:20:52 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 7 Jun 2018 14:20:52 -0700 Subject: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically In-Reply-To: References: Message-ID: Hello Magnus, Very nice refactoring! JdkNativeCompilation.gmk line 126-127 looks a bit long. There is an extra space on 126. Also, why not addprefix for adding -I instead of clunky foreach? Not that I care greatly, but I usually prefer that construct. Otherwise looks good. /Erik On 2018-06-07 13:22, Magnus Ihse Bursie wrote: > This change needs some background information. > > I've been working on simplifying and streamlining the compilation of > native libraries of the JDK. Previously, I introduced the > SetupJdkLibrary function, and started to get a better control of > compiler flags. This patch continues on both paths. > > The original intent of this change, which I naively thought was going > to be much simpler than it turned out :-) was to separate the -I flags > (location of header files) from other flags, and to generate these > automatically, wherever possible. Since we have a standard way of > locating native code, and most libraries just want to have an -I path > to their own source base and the generated Java header, we should be > able to provide this in SetupJdkLibrary. > > This turned out to be closely related to SetupJdkLibrary being able to > discover the location of the SRC directories itself, using "convention > over configuration" and assuming that the library "libfoo" for > "java.module" would be located in java.module/*/native/libfoo. > > While this sounds simple in theory, when actually trying to implement > this I of course ran into all the places where some special handling > was indeed needed. So even if like 90% of all libraries were simple to > get to build using automated discovery of source and header > directories, the 10% that did not caused me much more headaches than I > had anticipated. On the other hand, now that I've sorted out all those > places, the few remaining odd solutions is clearly documented and not > just something that "just happens" due to strange configurations. > > One file deserves mentioning specifically: Awt2dLibraries.gmk. The > java.desktop libraries are unfortunately quite entangled with each > other, and do not readily follow the patterns that are used elsewhere > in the code base. So it might just look like the file has just gone > from one state of messiness, to another, which would hardly be an > improvement. :-( I would still argue that the new messiness is better: > It is now much clearer in what ways the libraries diverge from our > standard assumption, and what course of action needs to be taken to > minimize these differences. (Which is something I believe should be > done -- these issues are not just cosmetic but is the root of most of > the issues we always see for these libraries, when upgrading > compilers, etc.) > > During this change, I noticed that not all native libraries include > the proper generated header file. This is a dangerous coding practice, > since a change in the Java part of the interface might not get picked > up properly in the native part. I've added the missing includes that > I've detected, and due to these changes, I'm also including the > component teams in what is really only a build change. As can be seen > for jdk.crypto.mscapi; there had indeed been changes that needed > correcting. > > Since this is (basically) a pure build change, my gold standard here > has been the build compare script. In essence, the build output prior > to my change and with this change are 100% identical. In truth, this > is a bit too strong a claim. A few changes has occurred, but none of > them should matter. Here's a breakdown of the compare.sh results: > > * Windows-x64: > > No differences at all. > > * Solaris: > > Two libraries are reported to differ: libsaproc.so and > libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing > this, I found that the object files used to link these two libraries > has no disass differences. They have a slight binary difference and a > difference in size, due to the include paths being different (and this > is stored in the .o file, which makes it different). Somehow this > apparently triggers the linker to generate a slightly different code > in a few places, using a different register or so. (Weird...) > > * MacOS: > > Two libraries are reported to differ: libjava.dylib and > libmlib_image.dylib, both of them just reported as a binary diff (no > symbol, disass or fulldump differences). This is not really > unsuspected, but I analyzed it anyway. > > I found that for libjava.dylib, a single .o file was different. This > one was actually picked up from closed sources, and are not really > relevant for OpenJDK. Anyway, the reason for the difference was the > same as for the Solaris libs; the include paths had changes, which > caused a binary diff. > > For libmlib_image.dylib, the link order had changed causing the noted > binary difference. > > * Linux: > > On linux, the compare script noted differences for libextnet, libjava, > libmlib_image, libprefs, libsaproc, libsplashscreen and libsunec. > > The differences for libextnet, libprefs, libsplashscreen and libsunec > turned out to be caused by the added #include of the generated Java > headers. This caused binary differences (reasonably), and for some odd > reason also a symbol difference in java_awt_SplashScreen.o > (clazz.10057 and mid.10058 were replaced by clazz.10015 and > mid.10016). I can't claim to understand this, but I'm assuming it's > some kind of generated code. > > libsaproc and libjava changes was caused by closed source changes, and > is therefore not relevant to OpenJDK. > > For libmlib_image.dylib, the link order had changed causing the noted > binary difference, as on MacOS. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204572 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.01 > > /Magnus > From mikhailo.seledtsov at oracle.com Thu Jun 7 21:30:24 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Thu, 07 Jun 2018 14:30:24 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <5B19A3F0.1010804@oracle.com> Hi Bob, I looked at the tests. In general they look good. I am a bit concerned about the use of ERROR_MARGIN in one of the tests. We need to make sure that the tests are stable, and do not produce intermittent failures. Thank you, Misha On 6/7/18, 10:43 AM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory& Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> From naoto.sato at oracle.com Thu Jun 7 21:42:26 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 7 Jun 2018 14:42:26 -0700 Subject: Locale languageTag anomaly In-Reply-To: <5B19A15B.3000702@oracle.com> References: <5B19A15B.3000702@oracle.com> Message-ID: Hi Jon, JDK historically represents Norwegian Nynorsk language with no_NO_NY (JDK unique "NY" variant), because it predates the ISO 639-1:2002 standard which introduced "nn" as the language code for Nynorsk. This legacy JDK locale is converted to BCP47 compliant language tag using "nn" language code. Locale.toLanguageTag() has the following sentence for this: --- A locale with language "no", country "NO", and variant "NY", representing Norwegian Nynorsk (Norway), is converted to a language tag "nn-NO". --- So in short, that is the expected behavior. Naoto On 6/7/18 2:19 PM, Jonathan Gibbons wrote: > Regarding Locale.toLanguageTag, Locale.forLanguageTag > > Should I be surprised (i.e. is it a bug) that out of 736 installed > locales in a standard build of JDK, exactly 1 locale fails the following > round-trip test: > > ??? locale.equals(Locale.forLanguageTag(locale.toLanguageTag())) > > The locale "no_NO_NY" comes back as "nn_NO". > > Attached is a test program.? Here is the output from the test program: > > $ /opt/jdk/1.9.0/bin/java -cp play/locales/classes Locales > test locale: no_NO_NY [no,NO,NY] nn-NO > ???? forLanguageTag: nn_NO > Tested 736 locales > Mismatch 1 locales > > -- Jon From mandy.chung at oracle.com Thu Jun 7 22:38:16 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Jun 2018 15:38:16 -0700 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: References: Message-ID: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> Hi Chris, On 6/7/18 1:32 AM, Chris Yin wrote: > Please review below new added test to check for package versioning > information which customized in jar(s) manifest, many thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8201528 webrev: > http://cr.openjdk.java.net/~xyin/8201528/webrev.00/ Thanks for adding the new test cases that we are missing. Can you describe all these test cases? In the class javadoc is fine. 32 * @run main PackageFromManifest runJar single 33 * @run main/othervm PackageFromManifest runUrlLoader single 34 * @run main PackageFromManifest runJar 1 35 * @run main PackageFromManifest runJar 2 36 * @run main/othervm PackageFromManifest runUrlLoader 1 37 * @run main/othervm PackageFromManifest runUrlLoader 2 74 if (runType.equalsIgnoreCase("runTest")) { The test accepts "runTest" but such test case is not listed above. It'd be good to refactor the main method into separate cases and a switch/if statement on different runtType will help the readability. 123 runTest(cl.loadClass( 124 PACKAGE_NAME + "." + TEST_CLASS_NAME1) 125 .getPackage(), option); Can you use Class::forName here? 200 cmds = new String[] { "-cp", 201 TEST_JAR_FILE1 + File.pathSeparator + TEST_JAR_FILE2, 202 "PackageFromManifest", "runTest", option }; Thanks for adding the split package test. It'd be good to test 1) loading TEST_CLASS_NAME1 first and TEST_CLASS_NAME2 second 2) loading TEST_CLASS_NAME2 first and TEST_CLASS_NAME1 second Package should use the manifest attribute from the JAR file from which the first class of package foo is loaded. Looks like it creates the JAR files each time the non-runTest case is run. You could do the setup as the first @run and subsequent @run are executing individual test cases (or make it testng test if you prefer). Mandy From jonathan.gibbons at oracle.com Thu Jun 7 22:38:46 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 07 Jun 2018 15:38:46 -0700 Subject: Locale languageTag anomaly In-Reply-To: References: <5B19A15B.3000702@oracle.com> Message-ID: <5B19B3F6.6020501@oracle.com> Naoto, Thanks, and I guess I didn't think to check out the special case rules. Sorry about that. -- Jon On 06/07/2018 02:42 PM, Naoto Sato wrote: > Hi Jon, > > JDK historically represents Norwegian Nynorsk language with no_NO_NY > (JDK unique "NY" variant), because it predates the ISO 639-1:2002 > standard which introduced "nn" as the language code for Nynorsk. This > legacy JDK locale is converted to BCP47 compliant language tag using > "nn" language code. > > Locale.toLanguageTag() has the following sentence for this: > > --- > A locale with language "no", country "NO", and variant "NY", > representing Norwegian Nynorsk (Norway), is converted to a language > tag "nn-NO". > --- > > So in short, that is the expected behavior. > > Naoto > > On 6/7/18 2:19 PM, Jonathan Gibbons wrote: >> Regarding Locale.toLanguageTag, Locale.forLanguageTag >> >> Should I be surprised (i.e. is it a bug) that out of 736 installed >> locales in a standard build of JDK, exactly 1 locale fails the >> following round-trip test: >> >> locale.equals(Locale.forLanguageTag(locale.toLanguageTag())) >> >> The locale "no_NO_NY" comes back as "nn_NO". >> >> Attached is a test program. Here is the output from the test program: >> >> $ /opt/jdk/1.9.0/bin/java -cp play/locales/classes Locales >> test locale: no_NO_NY [no,NO,NY] nn-NO >> forLanguageTag: nn_NO >> Tested 736 locales >> Mismatch 1 locales >> >> -- Jon From mandy.chung at oracle.com Thu Jun 7 23:06:15 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Jun 2018 16:06:15 -0700 Subject: (XS) Review Request JDK-8204584: jdeps generates illegal dot file containing ranksep=0,600000 Message-ID: The dot files are generated and used as module graph in the docs build. The format of double should use no localization; otherwise an illegal ranksep attribute. Mandy diff --git a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java --- a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java +++ b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java @@ -44,6 +44,7 @@ import java.util.Deque; import java.util.HashSet; import java.util.List; +import java.util.Locale; import java.util.Map; import java.util.Objects; import java.util.Set; @@ -329,7 +330,7 @@ out.format("digraph \"%s\" {%n", name); out.format(" nodesep=.5;%n"); - out.format(" ranksep=%f;%n", attributes.rankSep()); + out.format((Locale)null, " ranksep=%f;%n", attributes.rankSep()); out.format(" pencolor=transparent;%n"); out.format(" node [shape=plaintext, fontcolor=\"%s\", fontname=\"%s\"," + " fontsize=%d, margin=\".2,.2\"];%n", From jonathan.gibbons at oracle.com Thu Jun 7 23:10:36 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 07 Jun 2018 16:10:36 -0700 Subject: (XS) Review Request JDK-8204584: jdeps generates illegal dot file containing ranksep=0,600000 In-Reply-To: References: Message-ID: <5B19BB6C.6080505@oracle.com> Looks OK to me. -- Jon On 06/07/2018 04:06 PM, mandy chung wrote: > The dot files are generated and used as module graph in the docs build. > The format of double should use no localization; otherwise an illegal > ranksep attribute. > > Mandy > > > diff --git > a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java > b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java > --- a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java > +++ b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java > @@ -44,6 +44,7 @@ > import java.util.Deque; > import java.util.HashSet; > import java.util.List; > +import java.util.Locale; > import java.util.Map; > import java.util.Objects; > import java.util.Set; > @@ -329,7 +330,7 @@ > > out.format("digraph \"%s\" {%n", name); > out.format(" nodesep=.5;%n"); > - out.format(" ranksep=%f;%n", attributes.rankSep()); > + out.format((Locale)null, " ranksep=%f;%n", > attributes.rankSep()); > out.format(" pencolor=transparent;%n"); > out.format(" node [shape=plaintext, > fontcolor=\"%s\", fontname=\"%s\"," > + " fontsize=%d, margin=\".2,.2\"];%n", From paul.sandoz at oracle.com Fri Jun 8 00:42:04 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 7 Jun 2018 17:42:04 -0700 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> Message-ID: Hi Andrew, Jonathan, Sandhya gave an overview to a few of us Oracle folks. I agree with what Sandhya says regarding the API, a small surface, and on pursuing an unsafe intrinsic. I like it and would encourage the writing of a draft JEP, especially to give this visibility. I expect this will be beneficial for experimentation with the Panama foreign API where we can use a Pointer to reference into a byte buffer and scribble on it. Further, i hope this work may also benefit the persistent collections effort (PCJ). It intersects with https://bugs.openjdk.java.net/browse/JDK-8153111 ((bf) Allocating ByteBuffer on heterogeneous memory), which is attempting to be more generic. We might also need to increase the velocity on https://bugs.openjdk.java.net/browse/JDK-8180628 (retrofit direct buffer support for size beyond gigabyte scales), and i would be very interested your views on this, how you might be currently working around such size limitations, and what buffer enhancements would work for you. Thanks, Paul. > On May 30, 2018, at 10:21 PM, Viswanathan, Sandhya wrote: > > Hi Andrew/Jonathan, > > Thanks a lot for sharing this work. Copying hotspot-compiler-dev to get their feedback as well. > > Couple of thoughts/observations below: > * Supporting ByteBuffer on persistent memory using existing FileChannel and MappedByteBuffer mechanism sounds like a very good idea. > > * Extending FileChannel.map to take additional parameter to indicate that the ByteBuffer is backed by persistent memory is a small API change. > > * Adding MappedByteBuffer.force(int from, int to) method on smaller range is very useful in addition to the force() on entire ByteBuffer. > > * The underlying force0_mapsync() could be implemented in terms of new unsafe APIs which in turn could be intrinsified. > The advantage of this is that the unsafe APIs could then be used for other future persistent memory APIs in the JRE. > Specifically the following two unsafe APIs would be useful: > a) public native void flush(long address, long size); > b) public native void storeFence(); > storeFence() exists today but doesn?t generate any instruction for x86. > Wondering if we could have additional boolean parameter to force the sfence generation. > * DEFAULT_CACHE_LINE_SIZE is 128 in src/hotspot/cpu/x86/globalDefinitions_x86.hpp whereas actual cache line on the hardware is 64 bytes. > This could be the cause for some of performance that you saw with compiler intrinsic vs pure C native. > > Best Regards, > Sandhya > > > > RFC: Experiment in accessing/managing persistent memory from Java > Andrew Dinn adinn at redhat.com > Mon May 21 09:47:46 UTC 2018 > > I have been helping one of my Red Hat colleagues, Jonathan Halliday, to > investigate provision of a Java equivalent to Intel's libpmem suite of C > libraries [1]. This approach avoids the significant cost of using the > Intel libraries from Java via JNI (or, worse, as a virtual driver for a > persistent memory device). > > Jonathan has modified the JVM/JDK to allow a MappedByteBuffer to be > mapped over persistent memory, providing equivalent function to libpmem > itself. > > On top of this he implemented a Java journaled log class providing > equivalent functionality to one of the Intel client libs, libpmemlog, > built over libpmem. > > The modified MappedByteBuffer can be configured to use either i) a > registered native method or ii) a JIT intrinsic to perform the critical > task of cache line writeback i.e. the persistence step (the intrinsic is > my contribution). > > Jonathan's tests compare use of JNI, registered native and intrinsic > with an equivalent C program to write a large swathe of records to a > journaled log file stored in persistent memory. Performance is worse > than C when relying on JNI and significantly better with JVM/JDK > support. Indeed, as one might reasonably expect, use of the JIT > intrinsic almost completely eliminates writeback costs. > > The journaled log code, jdk dev tree patch, build instructions, test > code plus C equivalent and test results are all available from > Jonathan's git repo [2]. > > For those who do not want to look at the actual code, the README file > [3] provides background to use of persistent memory, an overview of the > design, and summary details of the test process and results. > > [1] https://pmem.io/pmdk/ > [2] https://github.com/jhalliday/pmem > [3] http://github.com://jhalliday/pmem/README.md > > n.b. Jonathan has experimented with using this same prototype to replace > the journaled log used in the Red Hat Narayana transaction manager. It > provides a significant improvement on the current disk file based log, > both for throughput and latency (the code is not yet available as > getting it to work involved some horrible hacking of the build to > migrate up to jdk11). > > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From david.holmes at oracle.com Fri Jun 8 02:20:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 8 Jun 2018 12:20:39 +1000 Subject: (trivial) RFR: 8204589: ProblemList failing launcher tests Message-ID: <33d65120-5ae0-0dfb-6450-b3475d87a61e@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8204589 webrev: http://cr.openjdk.java.net/~dholmes/8204589/webrev/ After the push for: JDK-8201274 Launch Single-File Source-Code Programs there are a few test failures in our CI testing, so they are being ProblemListed to keep tiers 1 and 2 clean. Thanks, David From joe.darcy at oracle.com Fri Jun 8 02:22:22 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Thu, 07 Jun 2018 19:22:22 -0700 Subject: (trivial) RFR: 8204589: ProblemList failing launcher tests In-Reply-To: <33d65120-5ae0-0dfb-6450-b3475d87a61e@oracle.com> References: <33d65120-5ae0-0dfb-6450-b3475d87a61e@oracle.com> Message-ID: <5B19E85E.40805@oracle.com> +1 -Joe On 6/7/2018 7:20 PM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8204589 > webrev: http://cr.openjdk.java.net/~dholmes/8204589/webrev/ > > After the push for: > > JDK-8201274 Launch Single-File Source-Code Programs > > there are a few test failures in our CI testing, so they are being > ProblemListed to keep tiers 1 and 2 clean. > > Thanks, > David From david.holmes at oracle.com Fri Jun 8 02:30:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 8 Jun 2018 12:30:43 +1000 Subject: (trivial) RFR: 8204589: ProblemList failing launcher tests In-Reply-To: <5B19E85E.40805@oracle.com> References: <33d65120-5ae0-0dfb-6450-b3475d87a61e@oracle.com> <5B19E85E.40805@oracle.com> Message-ID: Thanks Joe! Pushed. Should be in CI 753. David On 8/06/2018 12:22 PM, Joseph D. Darcy wrote: > +1 > > -Joe > > On 6/7/2018 7:20 PM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204589 >> webrev: http://cr.openjdk.java.net/~dholmes/8204589/webrev/ >> >> After the push for: >> >> ?JDK-8201274 Launch Single-File Source-Code Programs >> >> there are a few test failures in our CI testing, so they are being >> ProblemListed to keep tiers 1 and 2 clean. >> >> Thanks, >> David > From xueming.shen at oracle.com Fri Jun 8 02:33:11 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 07 Jun 2018 19:33:11 -0700 Subject: RFR JDK-8204229: Formatter and String.format ignore the width with the percent modifier (%5%) Message-ID: <5B19EAE7.9030509@oracle.com> Hi, Please help review the change for JDK-8204229. It appears to be a overlook in the implementation. We do have a method print(String, Locale) that adjust the "padding spaces" issue: https://bugs.openjdk.java.net/browse/JDK-8204229 webrev: http://cr.openjdk.java.net/~sherman/8204229/webrev Thanks, Sherman From harsha.wardhana.b at oracle.com Fri Jun 8 04:30:39 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Fri, 8 Jun 2018 10:00:39 +0530 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <5B19A3F0.1010804@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> Message-ID: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> [Replying to all mailing-lists] Hi Misha, The ERROR_MARGIN in tests was introduced to make the tests stable. There are times where metric values (specifically CPU usage) can change drastically in between two reads. The metrics value got from the API and the cgroup file can be different and 0.1 ERROR_MARGIN should take care of that, though at times even that may not be enough. Hence the CPU usage related tests only print a warning if ERROR_MARGIN is exceeded. Thanks Harsha On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: > Hi Bob, > > ? I looked at the tests. In general they look good. I am a bit > concerned about the use of ERROR_MARGIN in one of the tests. We need > to make sure that the tests are stable, and do not produce > intermittent failures. > > > Thank you, > Misha > > On 6/7/18, 10:43 AM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> Mandy Chung has reviewed this change. >> >> I?ve run Mach5 hotspot and core lib tests. >> >> I?ve reviewed the tests which were written by Harsha Wardhana >> >> I filed a CSR for the command line change and it?s now approved and >> closed. >> >> Thanks, >> Bob. >> >> >>> On May 30, 2018, at 3:45 PM, Bob Vandette? >>> wrote: >>> >>> Please review the following RFE which adds an internal API, along >>> with jtreg tests that provide >>> access to Docker container configuration data and metrics.? In >>> addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX >>> Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host >>> cgroup confguration >>> information.? See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> ??? Provider: cgroupv1 >>> ??? Effective CPU Count: 4 >>> ??? CPU Period: 100000 >>> ??? CPU Quota: -1 >>> ??? CPU Shares: -1 >>> ??? List of Processors, 4 total: >>> ??? 4 5 6 7 >>> ??? List of Effective Processors, 4 total: >>> ??? 4 5 6 7 >>> ??? List of Memory Nodes, 2 total: >>> ??? 0 1 >>> ??? List of Available Memory Nodes, 2 total: >>> ??? 0 1 >>> ??? CPUSet Memory Pressure Enabled: false >>> ??? Memory Limit: 256.00M >>> ??? Memory Soft Limit: Unlimited >>> ??? Memory&? Swap Limit: 512.00M >>> ??? Kernel Memory Limit: Unlimited >>> ??? TCP Memory Limit: Unlimited >>> ??? Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> From srinivas.dama at oracle.com Fri Jun 8 06:17:10 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 7 Jun 2018 23:17:10 -0700 (PDT) Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) Message-ID: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: Modified earlier fix to handle warning messages specific to libjimage only. Regards, Srinivas ----- Original Message ----- From: srinivas.dama at oracle.com To: core-libs-dev at openjdk.java.net Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but these warnings are disabled earlier.I have enabled these warnings and fixed in sources. Regards, Srinivas From matthias.baesken at sap.com Fri Jun 8 06:51:18 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 8 Jun 2018 06:51:18 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <67fa98e2ed454a208af23a7a56e4627b@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> Message-ID: <091861e7d9c94e27af146eed3e57d3eb@sap.com> Hi, looks like I uploaded a wrong version of the patch. Updated webrev is here : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ I followed the advice of Sean and removed the mapFileName from the output because it is usually a static name . Updated webrev was going through our internal build/tests . Best regards, Matthias From: Langer, Christoph Sent: Donnerstag, 7. Juni 2018 13:52 To: Baesken, Matthias ; core-libs-dev at openjdk.java.net Cc: Lindenmaier, Goetz Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi Matthias, in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? Also, mapFileName can't be used at this place, because it has already been free'ed there. But in general the additions make sense and will make it easier to find issues in the tzmappings file. Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 7. Juni 2018 13:35 To: core-libs-dev at openjdk.java.net Cc: Langer, Christoph >; Lindenmaier, Goetz > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From xu.y.yin at oracle.com Fri Jun 8 07:07:29 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Fri, 8 Jun 2018 15:07:29 +0800 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> Message-ID: <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> Hi, Mandy Many thanks for your detailed review and comments, updates new webrev as below, and comment inline, thanks webrev: http://cr.openjdk.java.net/~xyin/8201528/webrev.01/ > On 8 Jun 2018, at 6:38 AM, mandy chung wrote: > > Hi Chris, > > On 6/7/18 1:32 AM, Chris Yin wrote: >> Please review below new added test to check for package versioning >> information which customized in jar(s) manifest, many thanks >> bug: https://bugs.openjdk.java.net/browse/JDK-8201528 webrev: >> http://cr.openjdk.java.net/~xyin/8201528/webrev.00/ > > Thanks for adding the new test cases that we are missing. > > Can you describe all these test cases? In the class javadoc is fine. Sure, just added class javadoc to explain input arguments and each @run usage > > 32 * @run main PackageFromManifest runJar single > 33 * @run main/othervm PackageFromManifest runUrlLoader single > 34 * @run main PackageFromManifest runJar 1 > 35 * @run main PackageFromManifest runJar 2 > 36 * @run main/othervm PackageFromManifest runUrlLoader 1 > 37 * @run main/othervm PackageFromManifest runUrlLoader 2 > > 74 if (runType.equalsIgnoreCase("runTest")) { > > The test accepts "runTest" but such test case is not listed above. Thanks for checking this, ?runTest? run type will be used in test logic runJar only (for example, called from runJar method, "java -cp testpack1.jar PackageFromManifest runTest single") > > It'd be good to refactor the main method into separate cases and > a switch/if statement on different runtType will help the readability. Thanks, refactored in new webrev as you suggested, it looks more clear now > > 123 runTest(cl.loadClass( > 124 PACKAGE_NAME + "." + TEST_CLASS_NAME1) > 125 .getPackage(), option); > > Can you use Class::forName here? Sure, fixed in new webrev > > 200 cmds = new String[] { "-cp", > 201 TEST_JAR_FILE1 + File.pathSeparator + TEST_JAR_FILE2, > 202 "PackageFromManifest", "runTest", option }; > > Thanks for adding the split package test. It'd be good to test > 1) loading TEST_CLASS_NAME1 first and TEST_CLASS_NAME2 second > 2) loading TEST_CLASS_NAME2 first and TEST_CLASS_NAME1 second > > Package should use the manifest attribute from the JAR file from > which the first class of package foo is loaded. Yep, to make it clear, in new webrev, the first loaded class name will be printed (combine load and print) before runTest, such as System.out.println("Load " + Class.forName(PACKAGE_NAME + ?.? + TEST_CLASS_PREFIX + option) + " first"); > > Looks like it creates the JAR files each time the non-runTest case > is run. You could do the setup as the first @run and subsequent > @run are executing individual test cases (or make it testng test > if you prefer). Used setup at the first @run in new webrev as you suggested, thanks Regards, Chris > > Mandy From christoph.langer at sap.com Fri Jun 8 07:14:16 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 8 Jun 2018 07:14:16 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <091861e7d9c94e27af146eed3e57d3eb@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> Message-ID: Hi Matthias, this looks good. Reviewed from my end. Best regards Christoph From: Baesken, Matthias Sent: Freitag, 8. Juni 2018 08:51 To: Langer, Christoph ; core-libs-dev at openjdk.java.net; 'sean.coffey at oracle.com' Cc: Lindenmaier, Goetz Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, looks like I uploaded a wrong version of the patch. Updated webrev is here : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ I followed the advice of Sean and removed the mapFileName from the output because it is usually a static name . Updated webrev was going through our internal build/tests . Best regards, Matthias From: Langer, Christoph Sent: Donnerstag, 7. Juni 2018 13:52 To: Baesken, Matthias >; core-libs-dev at openjdk.java.net Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi Matthias, in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? Also, mapFileName can't be used at this place, because it has already been free'ed there. But in general the additions make sense and will make it easier to find issues in the tzmappings file. Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 7. Juni 2018 13:35 To: core-libs-dev at openjdk.java.net Cc: Langer, Christoph >; Lindenmaier, Goetz > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From goetz.lindenmaier at sap.com Fri Jun 8 07:33:45 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 8 Jun 2018 07:33:45 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <091861e7d9c94e27af146eed3e57d3eb@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> Message-ID: <3922fa39aa4749c1919b54e467541f7c@sap.com> Hi Matthias, thanks for adding this information, looks good! Best regards, Goety. > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 8. Juni 2018 08:51 > To: Langer, Christoph ; core-libs- > dev at openjdk.java.net; 'sean.coffey at oracle.com' > > Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > Hi, looks like I uploaded a wrong version of the patch. > > Updated webrev is here : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ > > > > I followed the advice of Sean and removed the mapFileName from the > output because it is usually a static name . > > Updated webrev was going through our internal build/tests . > > > > > > Best regards, Matthias > > > > > > From: Langer, Christoph > Sent: Donnerstag, 7. Juni 2018 13:52 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > > > Hi Matthias, > > > > in line 527, where the actual output is done, I think you would need to > replace variable 'message' with 'outputMessage', otherwise I guess it won't > compile?? > > > > Also, mapFileName can't be used at this place, because it has already been > free'ed there. > > > > But in general the additions make sense and will make it easier to find issues > in the tzmappings file. > > > > Best regards > > Christoph > > > > From: Baesken, Matthias > Sent: Donnerstag, 7. Juni 2018 13:35 > To: core-libs-dev at openjdk.java.net dev at openjdk.java.net> > Cc: Langer, Christoph >; Lindenmaier, Goetz > > > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > > > Hi, could you please review this small change that improves the error > messages in matchJavaTZ . > > A reason of the failure is added to the message , and also the offset where > the error happened . > > > > > > Bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8204539 > > > > > > Webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ > > > > > > Thanks, Matthias From christoph.langer at sap.com Fri Jun 8 07:35:48 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 8 Jun 2018 07:35:48 +0000 Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags In-Reply-To: <29f9a3494ea8ebb6f5db05e4fb06fa30@linux.vnet.ibm.com> References: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> <29f9a3494ea8ebb6f5db05e4fb06fa30@linux.vnet.ibm.com> Message-ID: Hi Ichiroh, Ok, so as per the output, via the include of osSupport.hpp, something must happen which undeclares "visibility" or makes it ambiguous. Looking at osSupport.hpp, I can't see anything special. It would just include pthread.h and declare some c++ classes. You could try to get and analyze the preprocessed output of xlC by specifying the option -P or -E to the compile call. You will get the original xlC command line by calling 'cat support/native/java.base/libjimage/NativeImageBuffer.o.cmdline' in your build directory. I think we should really understand what's happening there and fix it correctly instead of just excluding osSupport.hpp. Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Donnerstag, 7. Juni 2018 18:29 > To: Langer, Christoph > Cc: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; core- > libs-dev at openjdk.java.net; Lindenmaier, Goetz > ; Baesken, Matthias > > Subject: RE: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags > > Hello Christoph > > According to build log, if <#include "osSupport.hpp"> was there: > "/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server- > release/support/headers/java.base/jdk_internal_jimage_NativeImageBuffe > r.h", > line 15.27: 1540-0040 (S) The text > "Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap" is > unexpected. "visibility" may be undeclared or ambiguous. > make[3]: *** > [/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server- > release/support/native/java.base/libjimage/NativeImageBuffer.o] > Error 1 > make[3]: Leaving directory `/home/jdktest/sandbox/jdk/make' > make[2]: *** [java.base-libs] Error 2 > make[2]: *** Waiting for unfinished jobs.... > > On 2018-06-07 22:06, Langer, Christoph wrote: > > Hi Ichiroh, > > > > what's the exact error message if you #include "osSupport.hpp"? (I > > have no xlC 13 at hand to try myself...) > > > > Best regards > > Christoph > > > >> -----Original Message----- > >> From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> Sent: Donnerstag, 7. Juni 2018 14:53 > >> To: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; > >> core- > >> libs-dev at openjdk.java.net > >> Cc: Lindenmaier, Goetz ; Baesken, > Matthias > >> ; Langer, Christoph > >> > >> Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol > >> visibility flags > >> > >> Hello. > >> > >> Could you review it ? > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204541 > >> Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ > >> > >> Thanks, > >> Ichiroh Takiguchi > >> IBM Japan, Ltd. > >> > >> -------- Original Message -------- > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> on > >> xlc 12.1 > >> Date: 2018-06-07 20:43 > >> From: "Langer, Christoph" > >> To: Ichiroh Takiguchi > >> Cc: "'build-dev at openjdk.java.net'" , > >> "ppc-aix-port-dev at openjdk.java.net" >> dev at openjdk.java.net>, > >> "core-libs-dev at openjdk.java.net" , > >> "Lindenmaier, Goetz" , "Baesken, > Matthias" > >> > >> > >> Hi Ichiroh, > >> > >> your proposal seems to make sense. I have created a bug for this: > >> https://bugs.openjdk.java.net/browse/JDK-8204541 > >> > >> Can you please generate a webrev (referencing this bug, -c option of > >> webrev.ksh) and mail it over to me. Then I'll upload it and you can > >> post > >> an official RFR mail. > >> > >> Best regards > >> Christoph > >> > >> > -----Original Message----- > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> > Sent: Dienstag, 5. Juni 2018 08:59 > >> > To: Baesken, Matthias > >> > Cc: Langer, Christoph ; 'build- > >> > dev at openjdk.java.net' ; ppc-aix-port- > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> > Goetz > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > xlc 12.1 > >> > > >> > Hello Matthias and Christoph. > >> > Thank you for your explanations. > >> > > >> > I did not have enough knowledge about "visibility". > >> > > >> > I created following patches. > >> > > >> > ================================ > >> > diff -r 02934b0d661b > >> > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > >> > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp > Wed > >> > May > >> > 30 14:46:28 2018 +0200 > >> > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp > >> Tue > >> > Jun > >> > 05 12:10:41 2018 +0900 > >> > @@ -39,7 +39,9 @@ > >> > #include "imageFile.hpp" > >> > #include "inttypes.hpp" > >> > #include "jimage.hpp" > >> > +#if !defined(_AIX) > >> > #include "osSupport.hpp" > >> > +#endif > >> > > >> > #include "jdk_internal_jimage_NativeImageBuffer.h" > >> > > >> > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > >> > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 > 14:46:28 > >> > 2018 +0200 > >> > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 > 12:10:41 > >> > 2018 +0900 > >> > @@ -29,7 +29,8 @@ > >> > #ifndef __has_attribute > >> > #define __has_attribute(x) 0 > >> > #endif > >> > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) > && > >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) > >> > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) > && > >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ > >> > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) > >> > #ifdef ARM > >> > #define JNIEXPORT > >> > __attribute__((externally_visible,visibility("default"))) > >> > #define JNIIMPORT > >> > __attribute__((externally_visible,visibility("default"))) > >> > ================================ > >> > > >> > If "osSupport.hpp" was included, XLC++ compiler complained. > >> > I could not understand, which part was invalid... > >> > I'm not sure, "osSupport.hpp" is really required on > >> > NativeImageBuffer.cpp or not... > >> > > >> > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. > >> > [1] > >> > 0xD01 means 13.1 by hexadecimal. > >> > > >> > I checked symbol table by "dump -Tv -X64". [2] > >> > It seemed it was fine, some symbols were hidden. > >> > > >> > Does someone review my code? > >> > > >> > [1] > >> > > >> > https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i > >> > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html > >> > [2] > >> > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > >> > visibility/index.html > >> > > >> > On 2018-06-01 21:12, Baesken, Matthias wrote: > >> > > Hi , my change 8202322 just handled the fact that the > >> > > visibility - flags are not supported with xlc 12.1 , so setting > >> > > them generated a TON of compile - time warnings . > >> > > > >> > > The introduction of the "-qvisibility=hidden" came with the > >> > > mapfile removal changes : > >> > > > >> > > 8200358: Remove mapfiles for JDK executables > >> > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > >> > > > >> > > 8200178: Remove mapfiles for JDK native libraries > >> > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > >> > > > >> > > I guess it might need further testing+adjustments to make the > >> > > "visibility hiding" work nicely with XLC13 , but currently we > >> > > build only with XLC12 . > >> > > > >> > > As a workaround you might want to remove the "- > qvisibility=hidden" > >> > > setting for XLC 13 as well , like I did for XLC12 with the change > >> > > 8202322 . > >> > > > >> > > > >> > > Best regards, Matthias > >> > > > >> > > > >> > > > >> > > > >> > >> -----Original Message----- > >> > >> From: Langer, Christoph > >> > >> Sent: Freitag, 1. Juni 2018 10:57 > >> > >> To: Ichiroh Takiguchi > >> > >> Cc: Baesken, Matthias ; 'build- > >> > >> dev at openjdk.java.net' ; ppc-aix- > port- > >> > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > Lindenmaier, > >> > >> Goetz > >> > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> > >> on xlc 12.1 > >> > >> > >> > >> Hi Ichiroh, > >> > >> > >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I > believe > >> > >> nobody of my colleagues has played with it yet. So you are on a new > >> > >> playground here ?? > >> > >> > >> > >> However, I believe the idea in OpenJDK with the abolition of map > files > >> > >> is that > >> > >> symbols should be invisible externally unless they are declared > >> > >> exported, > >> > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the > >> > >> correct > >> > >> default and whatever JNIEXPORT expands to should contain the right > >> > >> attributes to get that symbol visible. > >> > >> > >> > >> Can you check if either my assumption is completely wrong, > JNIEXPORT > >> > >> does > >> > >> not expand to the right thing, XLC 13 has a bug or maybe just sume > >> > >> specific > >> > >> required symbols are not declared correctly? > >> > >> > >> > >> Best regards > >> > >> Christoph > >> > >> > >> > >> > -----Original Message----- > >> > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> > >> > Sent: Donnerstag, 31. Mai 2018 09:55 > >> > >> > To: Langer, Christoph > >> > >> > Cc: Baesken, Matthias ; 'build- > >> > >> > dev at openjdk.java.net' ; ppc-aix- > port- > >> > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > >> Lindenmaier, > >> > >> > Goetz > >> > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > on > >> xlc > >> > 12.1 > >> > >> > > >> > >> > Hello. > >> > >> > 8202322 was integrated into jdk-11+15. > >> > >> > I'm using XLC 13.1.3 on AIX 7.1.4. > >> > >> > Build was failed because of "-qvisibility=hidden" on > >> > >> > make/lib/LibCommon.gmk. > >> > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], > >> > >> > "-qvisibility=hidden" cannot create shared libraries entry points. > >> > >> > For example, libverify.so was there, but entry points were not > >> resolved > >> > >> > by "-lverify" option. > >> > >> > I think it should be "-qvisibility=default" (I tried, it worked) > >> > >> > or "-qvisibility=protected" (I had not tried) ? > >> > >> > I'm not familiar with -qvisibility option, but I'd like to find out > >> > >> > right way. > >> > >> > > >> > >> > [1] > >> > >> > > >> > >> > >> > > >> > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > >> > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > >> > >> > > >> > >> > On 2018-05-16 16:08, Langer, Christoph wrote: > >> > >> > > Hi Matthias, > >> > >> > > > >> > >> > > yes, reviewed. > >> > >> > > > >> > >> > > Best regards > >> > >> > > Christoph > >> > >> > > > >> > >> > > From: Baesken, Matthias > >> > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 > >> > >> > > To: Langer, Christoph ; > >> > >> > > 'build-dev at openjdk.java.net' ; > >> > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs- > >> dev at openjdk.java.net > >> > >> > > Cc: Lindenmaier, Goetz > >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > on > >> > >> > > xlc 12.1 > >> > >> > > > >> > >> > > Hi Christoph can I add you as second reviewer (other reviewer > was > >> > >> > > Erik Joelsson) can push the change ? > >> > >> > > > >> > >> > > Best regards, Matthias > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > From: Langer, Christoph > >> > >> > > Sent: Donnerstag, 26. April 2018 16:38 > >> > >> > > To: Baesken, Matthias > >> > >> > > > >> >; > >> > >> > > 'build-dev at openjdk.java.net' > >> > >> > > >> > dev at openjdk.java.net>>; > >> > >> > > ppc-aix-port-dev at openjdk.java.net >> > >> > dev at openjdk.java.net>; > >> > >> > > core-libs-dev at openjdk.java.net >> > >> dev at openjdk.java.net> > >> > >> > > Cc: Simonis, Volker > >> > >> > > > > >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > on > >> > >> > > xlc 12.1 > >> > >> > > > >> > >> > > Hi Matthias, > >> > >> > > > >> > >> > > to me the change in principal looks good. > >> > >> > > > >> > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > >> > >> > > extract major number before the first dot, then compare > >> numerically) > >> > - > >> > >> > > but maybe it is too complicated and the current single version > >> > compare > >> > >> > > suits our needs ? > >> > >> > > > >> > >> > > Best regards > >> > >> > > Christoph > >> > >> > > > >> > >> > > From: Baesken, Matthias > >> > >> > > Sent: Donnerstag, 26. April 2018 16:14 > >> > >> > > To: 'build-dev at openjdk.java.net' > >> > >> > > >> > dev at openjdk.java.net>>; > >> > >> > > ppc-aix-port-dev at openjdk.java.net >> > >> > dev at openjdk.java.net>; > >> > >> > > core-libs-dev at openjdk.java.net >> > >> dev at openjdk.java.net> > >> > >> > > Cc: Langer, Christoph > >> > >> > > > >; > >> > >> Simonis, > >> > >> > > Volker > >> > > >> > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on > xlc > >> > >> > > 12.1 > >> > >> > > > >> > >> > > Hello , could you please review this small adjustment to the > symbol > >> > >> > > visibility compilation settings on AIX ? > >> > >> > > Currently we use XLC 12.1 to compile JDK on AIX . > >> > >> > > > >> > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" > >> > >> > > setting currently set on AIX. > >> > >> > > It was introduced with XLC 13.1 . Christoph found some info > about it > >> > >> > > here : > >> > >> > > > >> > >> > > https://www.ibm.com/developerworks/aix/library/au-aix- > symbol- > >> > >> > visibility-part2/index.html > >> > >> > > > >> > >> > > Setting it only generates hundreds of warnings in the build log , > >> > >> > > warnings look like this : > >> > >> > > XlC12.1 > >> > >> > > > >> > >> > > bash-4.4$ xlC -qversion > >> > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> > >> > > Version: 12.01.0000.0019 > >> > >> > > > >> > >> > > bash-4.4$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > >> > >> > > of valid options. > >> > >> > > > >> > >> > > Compare to XLC13.1 > >> > >> > > > >> > >> > > bash-3.00$ xlC -qversion > >> > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> > >> > > Version: 13.01.0000.0008 > >> > >> > > bash-3.00$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> > >> > > bash-3.00$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > >> > > > >> > >> > > > >> > >> > > So it is better to avoid setting these flags when using xlc12.1 . > >> > >> > > Please review : > >> > >> > > > >> > >> > > Bug : > >> > >> > > > >> > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 > >> > >> > > > >> > >> > > Change : > >> > >> > > > >> > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> > >> > > > >> > >> > > > >> > >> > > Best regards, Matthias From sean.coffey at oracle.com Fri Jun 8 07:36:30 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 8 Jun 2018 08:36:30 +0100 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> Message-ID: Not sure if this has been tested. Don't you need to escape the printing of the \0 character ? errorMessage = "illegal character \0 found"; regards, Sean. On 08/06/2018 08:14, Langer, Christoph wrote: > > Hi Matthias, > > this looks good. Reviewed from my end. > > Best regards > > Christoph > > *From:*Baesken, Matthias > *Sent:* Freitag, 8. Juni 2018 08:51 > *To:* Langer, Christoph ; > core-libs-dev at openjdk.java.net; 'sean.coffey at oracle.com' > > *Cc:* Lindenmaier, Goetz > *Subject:* RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > Hi,? looks like I uploaded a wrong version of the patch. > > Updated webrev is here? : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ > > > I followed the advice of Sean and? removed? the? mapFileName?? from > the output because it is usually a static name . > > Updated webrev was going through our internal build/tests . > > Best regards, Matthias > > *From:*Langer, Christoph > *Sent:* Donnerstag, 7. Juni 2018 13:52 > *To:* Baesken, Matthias >; core-libs-dev at openjdk.java.net > > *Cc:* Lindenmaier, Goetz > > *Subject:* RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > Hi Matthias, > > in line 527, where the actual output is done, I think you would need > to replace variable ?message? with ?outputMessage?, otherwise I guess > it won?t compile?? > > Also, mapFileName can?t be used at this place, because it has already > been free?ed there. > > But in general the additions make sense and will make it easier to > find issues in the tzmappings file. > > Best regards > > Christoph > > *From:*Baesken, Matthias > *Sent:* Donnerstag, 7. Juni 2018 13:35 > *To:* core-libs-dev at openjdk.java.net > > *Cc:* Langer, Christoph >; Lindenmaier, Goetz > > > *Subject:* [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > Hi, could you please? review this small? change that improves? the > error messages in matchJavaTZ . > > A reason of the failure is added to the message , and also? the offset > where the error happened . > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8204539 > > Webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ > > > Thanks, Matthias > From magnus.ihse.bursie at oracle.com Fri Jun 8 08:50:37 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 8 Jun 2018 10:50:37 +0200 Subject: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically In-Reply-To: References: Message-ID: <4ff74b09-ec9f-0a80-31d5-49039d5e1e8c@oracle.com> On 2018-06-07 23:20, Erik Joelsson wrote: > Hello Magnus, > > Very nice refactoring! Thanks! > > JdkNativeCompilation.gmk > line 126-127 looks a bit long. There is an extra space on 126. Also, > why not addprefix for adding -I instead of clunky foreach? Not that I > care greatly, but I usually prefer that construct. Yeah, I agree, addprefix is better. I just forgot about it; foreach is a nice allround tool. Updated webrev: http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.02/ (Only changes is in JdkNativeCompilation.gmk, as per above comments). /Magnus > > Otherwise looks good. > > /Erik > > > On 2018-06-07 13:22, Magnus Ihse Bursie wrote: >> This change needs some background information. >> >> I've been working on simplifying and streamlining the compilation of >> native libraries of the JDK. Previously, I introduced the >> SetupJdkLibrary function, and started to get a better control of >> compiler flags. This patch continues on both paths. >> >> The original intent of this change, which I naively thought was going >> to be much simpler than it turned out :-) was to separate the -I >> flags (location of header files) from other flags, and to generate >> these automatically, wherever possible. Since we have a standard way >> of locating native code, and most libraries just want to have an -I >> path to their own source base and the generated Java header, we >> should be able to provide this in SetupJdkLibrary. >> >> This turned out to be closely related to SetupJdkLibrary being able >> to discover the location of the SRC directories itself, using >> "convention over configuration" and assuming that the library >> "libfoo" for "java.module" would be located in >> java.module/*/native/libfoo. >> >> While this sounds simple in theory, when actually trying to implement >> this I of course ran into all the places where some special handling >> was indeed needed. So even if like 90% of all libraries were simple >> to get to build using automated discovery of source and header >> directories, the 10% that did not caused me much more headaches than >> I had anticipated. On the other hand, now that I've sorted out all >> those places, the few remaining odd solutions is clearly documented >> and not just something that "just happens" due to strange >> configurations. >> >> One file deserves mentioning specifically: Awt2dLibraries.gmk. The >> java.desktop libraries are unfortunately quite entangled with each >> other, and do not readily follow the patterns that are used elsewhere >> in the code base. So it might just look like the file has just gone >> from one state of messiness, to another, which would hardly be an >> improvement. :-( I would still argue that the new messiness is >> better: It is now much clearer in what ways the libraries diverge >> from our standard assumption, and what course of action needs to be >> taken to minimize these differences. (Which is something I believe >> should be done -- these issues are not just cosmetic but is the root >> of most of the issues we always see for these libraries, when >> upgrading compilers, etc.) >> >> During this change, I noticed that not all native libraries include >> the proper generated header file. This is a dangerous coding >> practice, since a change in the Java part of the interface might not >> get picked up properly in the native part. I've added the missing >> includes that I've detected, and due to these changes, I'm also >> including the component teams in what is really only a build change. >> As can be seen for jdk.crypto.mscapi; there had indeed been changes >> that needed correcting. >> >> Since this is (basically) a pure build change, my gold standard here >> has been the build compare script. In essence, the build output prior >> to my change and with this change are 100% identical. In truth, this >> is a bit too strong a claim. A few changes has occurred, but none of >> them should matter. Here's a breakdown of the compare.sh results: >> >> * Windows-x64: >> >> No differences at all. >> >> * Solaris: >> >> Two libraries are reported to differ: libsaproc.so and >> libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing >> this, I found that the object files used to link these two libraries >> has no disass differences. They have a slight binary difference and a >> difference in size, due to the include paths being different (and >> this is stored in the .o file, which makes it different). Somehow >> this apparently triggers the linker to generate a slightly different >> code in a few places, using a different register or so. (Weird...) >> >> * MacOS: >> >> Two libraries are reported to differ: libjava.dylib and >> libmlib_image.dylib, both of them just reported as a binary diff (no >> symbol, disass or fulldump differences). This is not really >> unsuspected, but I analyzed it anyway. >> >> I found that for libjava.dylib, a single .o file was different. This >> one was actually picked up from closed sources, and are not really >> relevant for OpenJDK. Anyway, the reason for the difference was the >> same as for the Solaris libs; the include paths had changes, which >> caused a binary diff. >> >> For libmlib_image.dylib, the link order had changed causing the noted >> binary difference. >> >> * Linux: >> >> On linux, the compare script noted differences for libextnet, >> libjava, libmlib_image, libprefs, libsaproc, libsplashscreen and >> libsunec. >> >> The differences for libextnet, libprefs, libsplashscreen and libsunec >> turned out to be caused by the added #include of the generated Java >> headers. This caused binary differences (reasonably), and for some >> odd reason also a symbol difference in java_awt_SplashScreen.o >> (clazz.10057 and mid.10058 were replaced by clazz.10015 and >> mid.10016). I can't claim to understand this, but I'm assuming it's >> some kind of generated code. >> >> libsaproc and libjava changes was caused by closed source changes, >> and is therefore not relevant to OpenJDK. >> >> For libmlib_image.dylib, the link order had changed causing the noted >> binary difference, as on MacOS. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204572 >> WebRev: >> http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.01 >> >> /Magnus >> > From matthias.baesken at sap.com Fri Jun 8 11:28:12 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 8 Jun 2018 11:28:12 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <3922fa39aa4749c1919b54e467541f7c@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> <3922fa39aa4749c1919b54e467541f7c@sap.com> Message-ID: <403aead86f1346a2bfb9df8a98be436d@sap.com> Hi Goetz/Christoph, thanks for the reviews . However of course Sean is absolutely correct about the null character message output. Updated the null character related message output : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.2/ Best regards, Matthias > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Freitag, 8. Juni 2018 09:34 > To: Baesken, Matthias ; Langer, Christoph > ; core-libs-dev at openjdk.java.net; > 'sean.coffey at oracle.com' > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > Hi Matthias, > > thanks for adding this information, looks good! > > Best regards, > Goety. > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Freitag, 8. Juni 2018 08:51 > > To: Langer, Christoph ; core-libs- > > dev at openjdk.java.net; 'sean.coffey at oracle.com' > > > > Cc: Lindenmaier, Goetz > > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > > [windows] > > > > Hi, looks like I uploaded a wrong version of the patch. > > > > Updated webrev is here : > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ > > > > > > > > I followed the advice of Sean and removed the mapFileName from the > > output because it is usually a static name . > > > > Updated webrev was going through our internal build/tests . > > > > > > > > > > > > Best regards, Matthias > > > > > > > > > > > > From: Langer, Christoph > > Sent: Donnerstag, 7. Juni 2018 13:52 > > To: Baesken, Matthias ; core-libs- > > dev at openjdk.java.net > > Cc: Lindenmaier, Goetz > > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > > [windows] > > > > > > > > Hi Matthias, > > > > > > > > in line 527, where the actual output is done, I think you would need to > > replace variable 'message' with 'outputMessage', otherwise I guess it > won't > > compile?? > > > > > > > > Also, mapFileName can't be used at this place, because it has already been > > free'ed there. > > > > > > > > But in general the additions make sense and will make it easier to find > issues > > in the tzmappings file. > > > > > > > > Best regards > > > > Christoph > > > > > > > > From: Baesken, Matthias > > Sent: Donnerstag, 7. Juni 2018 13:35 > > To: core-libs-dev at openjdk.java.net > dev at openjdk.java.net> > > Cc: Langer, Christoph > >; Lindenmaier, Goetz > > > > > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > > > > > > > Hi, could you please review this small change that improves the error > > messages in matchJavaTZ . > > > > A reason of the failure is added to the message , and also the offset where > > the error happened . > > > > > > > > > > > > Bug : > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8204539 > > > > > > > > > > > > Webrev : > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ > > > > > > > > > > > > Thanks, Matthias From sean.coffey at oracle.com Fri Jun 8 12:05:13 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 8 Jun 2018 13:05:13 +0100 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <403aead86f1346a2bfb9df8a98be436d@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> <3922fa39aa4749c1919b54e467541f7c@sap.com> <403aead86f1346a2bfb9df8a98be436d@sap.com> Message-ID: Looks good. Thanks for adding this improvement. Regards, Sean. On 08/06/18 12:28, Baesken, Matthias wrote: > Hi Goetz/Christoph, thanks for the reviews . > However of course Sean is absolutely correct about the null character message output. > Updated the null character related message output : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.2/ > > Best regards, Matthias > > >> -----Original Message----- >> From: Lindenmaier, Goetz >> Sent: Freitag, 8. Juni 2018 09:34 >> To: Baesken, Matthias ; Langer, Christoph >> ; core-libs-dev at openjdk.java.net; >> 'sean.coffey at oracle.com' >> Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ >> [windows] >> >> Hi Matthias, >> >> thanks for adding this information, looks good! >> >> Best regards, >> Goety. >> >>> -----Original Message----- >>> From: Baesken, Matthias >>> Sent: Freitag, 8. Juni 2018 08:51 >>> To: Langer, Christoph ; core-libs- >>> dev at openjdk.java.net; 'sean.coffey at oracle.com' >>> >>> Cc: Lindenmaier, Goetz >>> Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ >>> [windows] >>> >>> Hi, looks like I uploaded a wrong version of the patch. >>> >>> Updated webrev is here : >>> >>> >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ >>> >>> >>> >>> I followed the advice of Sean and removed the mapFileName from the >>> output because it is usually a static name . >>> >>> Updated webrev was going through our internal build/tests . >>> >>> >>> >>> >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> >>> From: Langer, Christoph >>> Sent: Donnerstag, 7. Juni 2018 13:52 >>> To: Baesken, Matthias ; core-libs- >>> dev at openjdk.java.net >>> Cc: Lindenmaier, Goetz >>> Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ >>> [windows] >>> >>> >>> >>> Hi Matthias, >>> >>> >>> >>> in line 527, where the actual output is done, I think you would need to >>> replace variable 'message' with 'outputMessage', otherwise I guess it >> won't >>> compile?? >>> >>> >>> >>> Also, mapFileName can't be used at this place, because it has already been >>> free'ed there. >>> >>> >>> >>> But in general the additions make sense and will make it easier to find >> issues >>> in the tzmappings file. >>> >>> >>> >>> Best regards >>> >>> Christoph >>> >>> >>> >>> From: Baesken, Matthias >>> Sent: Donnerstag, 7. Juni 2018 13:35 >>> To: core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> >>> Cc: Langer, Christoph >> >; Lindenmaier, Goetz >>> > >>> Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] >>> >>> >>> >>> Hi, could you please review this small change that improves the error >>> messages in matchJavaTZ . >>> >>> A reason of the failure is added to the message , and also the offset where >>> the error happened . >>> >>> >>> >>> >>> >>> Bug : >>> >>> >>> >>> https://bugs.openjdk.java.net/browse/JDK-8204539 >>> >>> >>> >>> >>> >>> Webrev : >>> >>> >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ >>> >>> >>> >>> >>> >>> Thanks, Matthias From james.laskey at oracle.com Fri Jun 8 12:10:50 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 8 Jun 2018 09:10:50 -0300 Subject: RFR JDK-8204229: Formatter and String.format ignore the width with the percent modifier (%5%) In-Reply-To: <5B19EAE7.9030509@oracle.com> References: <5B19EAE7.9030509@oracle.com> Message-ID: +1 > On Jun 7, 2018, at 11:33 PM, Xueming Shen wrote: > > Hi, > > Please help review the change for JDK-8204229. It appears to be a overlook > in the implementation. We do have a method print(String, Locale) that adjust > the "padding spaces" > > issue: https://bugs.openjdk.java.net/browse/JDK-8204229 > webrev: http://cr.openjdk.java.net/~sherman/8204229/webrev > > Thanks, > Sherman From srinivas.dama at oracle.com Fri Jun 8 13:02:58 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Fri, 8 Jun 2018 06:02:58 -0700 (PDT) Subject: RFR: 8196993: Resolve disabled warnings for libunpack Message-ID: Hi, Please review http://cr.openjdk.java.net/~sdama/8196993/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196993 Regards, Srinivas From roger.riggs at oracle.com Fri Jun 8 14:04:26 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 8 Jun 2018 10:04:26 -0400 Subject: [11] RFR: 8202088: Japanese new era implementation In-Reply-To: <2bf7e4b1-28de-5dcb-5027-bfdb64573d5d@oracle.com> References: <079f3232-1cc0-1860-4dc4-671f71827f34@oracle.com> <76102e2c-f2ad-f98c-8acd-9709551c2dd5@oracle.com> <2bf7e4b1-28de-5dcb-5027-bfdb64573d5d@oracle.com> Message-ID: Hi Naoto, test/jdk/java/util/Calendar/SupplementalJapaneseEraTest.java: 125 missing spaces around "+". Please rename the test to have functional name. (test/jdk/java/util/Calendar/Bug8202088.java) We're trying to get away for uninformative bug numbers for tests. No further review from me needed with those changes. Thanks, Roger On 5/25/18 4:13 PM, naoto.sato at oracle.com wrote: > Thanks for the review, Roger and Max. I modified the webrev according > to your suggestions. Also, from an internal comment, I added a test > case (Bug8202088.java) for the modification below. Here is the updated > webrev: > > http://cr.openjdk.java.net/~naoto/8202088/webrev.04/ > > Max, > > --- > BTW, why must 8202088 be pushed before Apr 2019? If someone try to > show the Japanese calendar date of a day in 2020 now, do you really > expect them to see "NewEra year 2"? (or something like it, I am not > sure). > --- > > I think so, given that they understand "NewEra" is a placeholder. > Actually it was requested by customers that we implement the era way > before the actual transition so that their applications should have > been tested accordingly with the new era ahead of time (let alone we > need to backport it to jdk8u/etc. lines) > > Naoto > > On 5/24/18 8:45 AM, Naoto Sato wrote: >> Found an issue on retrieving the localized era name for >> java.util.Calendar. The reason is that even we provide the l10n in >> our own resource bundles, the current CLDR does not provide the name >> (duh!). Added a fallback to COMPAT provider in such a case. >> >> Here is the updated webrev: >> >> http://cr.openjdk.java.net/~naoto/8202088/webrev.03/ >> >> Naoto >> >> On 5/18/18 3:26 PM, naoto.sato at oracle.com wrote: >>> Hi Roger, thank you for the comments. >>> >>> On 5/18/18 11:11 AM, Roger Riggs wrote: >>>> Hi Naoto, >>>> >>>> Is there a reference to the official description or anticipation of >>>> the new Era? >>> >>> AFAIK, the most recent information was for Japanese Govt. to >>> announce the new era name one month prior to the ascension. This is >>> indeed the reason I decided to introduce the placeholder. >>> >>>> >>>> JapaneseImperialCalendar: 134 NEWERA = 5;?? (The real name can also >>>> be defined later; but still might be more unique as ERA_MAY_1_2019.) >>> >>> I wanted to keep the name "NEWERA" for the convenience when they are >>> to be replaced with the real name. I changed the access modifier to >>> "private", though. >>> >>>> >>>> Syntax style: >>>> >>>> ??- TCKJapaneseChronology:692: align the? columns of decimal values. >>>> >>>> ??- TestJapaneseChronology:61-62: space before the '}' brackets >>>> ???? :89: extra space before '}'? //? inconsistent within the file >>>> but local consistency is good >>>> >>>> TestUmmAlQuraChronology: there might be test dates that would not >>>> require more changes later when the era name changes. >>> >>> Fixed as suggested. The updated webrev is here: >>> >>> http://cr.openjdk.java.net/~naoto/8202088/webrev.02/ >>> >>> Naoto >>> >>>> >>>> Regards, Roger >>>> >>>> >>>> On 5/17/18 4:31 PM, Naoto Sato wrote: >>>>> Hi, >>>>> >>>>> Please review the changes to the subject issue: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8202088 >>>>> >>>>> The proposed change is located at: >>>>> >>>>> http://cr.openjdk.java.net/~naoto/8202088/webrev.01/ >>>>> >>>>> This is the implementation part of the upcoming Japanese new era, >>>>> starting from May 1st, 2019. Current anticipation is that the new >>>>> name won't be known till one month prior to the beginning of the >>>>> era. So it's not possible to make changes to the JDK with the >>>>> proper name. Instead, here we are going to implement the new era >>>>> with a place holder name which will be immediately replaced with >>>>> the proper name once it's known. The CSR is currently under review: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8202336 >>>>> >>>>> Naoto >>>> From bob.vandette at oracle.com Fri Jun 8 15:03:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 8 Jun 2018 11:03:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> Message-ID: <4B4CBD97-1081-4DB6-871F-5BF292BF4DD0@oracle.com> I didn?t actually have any ERROR_MARGIN problems during testing. I had issues with the testCpuConsumption test in http://cr.openjdk.java.net/~bobv/8203357/webrev.01/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java.html I had to initialize the cpu usage values during setup rather than inside the test to ensure that sufficient cpu usage had occurred by the time the test was run. The original code executed and received the same values after attempting to exec a linux utility. My change uses the time taken to run several tests instead. This seems to have eliminated any intermittent failures. Bob. > On Jun 8, 2018, at 12:30 AM, Harsha Wardhana B wrote: > > [Replying to all mailing-lists] > Hi Misha, > > The ERROR_MARGIN in tests was introduced to make the tests stable. There are times where metric values (specifically CPU usage) can change drastically in between two reads. The metrics value got from the API and the cgroup file can be different and 0.1 ERROR_MARGIN should take care of that, though at times even that may not be enough. Hence the CPU usage related tests only print a warning if ERROR_MARGIN is exceeded. > > Thanks > Harsha > > On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: >> Hi Bob, >> >> I looked at the tests. In general they look good. I am a bit concerned about the use of ERROR_MARGIN in one of the tests. We need to make sure that the tests are stable, and do not produce intermittent failures. >> >> >> Thank you, >> Misha >> >> On 6/7/18, 10:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> >>> I?ve run Mach5 hotspot and core lib tests. >>> >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> >>> I filed a CSR for the command line change and it?s now approved and closed. >>> >>> Thanks, >>> Bob. >>> >>> >>>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>>> >>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>> information. See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> Provider: cgroupv1 >>>> Effective CPU Count: 4 >>>> CPU Period: 100000 >>>> CPU Quota: -1 >>>> CPU Shares: -1 >>>> List of Processors, 4 total: >>>> 4 5 6 7 >>>> List of Effective Processors, 4 total: >>>> 4 5 6 7 >>>> List of Memory Nodes, 2 total: >>>> 0 1 >>>> List of Available Memory Nodes, 2 total: >>>> 0 1 >>>> CPUSet Memory Pressure Enabled: false >>>> Memory Limit: 256.00M >>>> Memory Soft Limit: Unlimited >>>> Memory& Swap Limit: 512.00M >>>> Kernel Memory Limit: Unlimited >>>> TCP Memory Limit: Unlimited >>>> Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > From mikhailo.seledtsov at oracle.com Fri Jun 8 15:31:52 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Fri, 8 Jun 2018 08:31:52 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> Message-ID: <4f9ac7ba-5a81-c9e4-feaa-42023d9f518f@oracle.com> Hi Harsha, ? Thank you for the explanation, makes sense to me. Please be aware, if a specific test turns out to be unstable in CI testing, it should be problem listed until solution is found to make it more stable. If the test is highly intermittent (fails intermittently but rarely) then it should be tagged with intermittent keyword. Overall tests look good to me, Thank you, Misha On 06/07/2018 09:30 PM, Harsha Wardhana B wrote: > [Replying to all mailing-lists] > Hi Misha, > > The ERROR_MARGIN in tests was introduced to make the tests stable. > There are times where metric values (specifically CPU usage) can > change drastically in between two reads. The metrics value got from > the API and the cgroup file can be different and 0.1 ERROR_MARGIN > should take care of that, though at times even that may not be enough. > Hence the CPU usage related tests only print a warning if ERROR_MARGIN > is exceeded. > > Thanks > Harsha > > On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: >> Hi Bob, >> >> ? I looked at the tests. In general they look good. I am a bit >> concerned about the use of ERROR_MARGIN in one of the tests. We need >> to make sure that the tests are stable, and do not produce >> intermittent failures. >> >> >> Thank you, >> Misha >> >> On 6/7/18, 10:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> >>> I?ve run Mach5 hotspot and core lib tests. >>> >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> >>> I filed a CSR for the command line change and it?s now approved and >>> closed. >>> >>> Thanks, >>> Bob. >>> >>> >>>> On May 30, 2018, at 3:45 PM, Bob Vandette? >>>> wrote: >>>> >>>> Please review the following RFE which adds an internal API, along >>>> with jtreg tests that provide >>>> access to Docker container configuration data and metrics. In >>>> addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX >>>> Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or >>>> host cgroup confguration >>>> information.? See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> ??? Provider: cgroupv1 >>>> ??? Effective CPU Count: 4 >>>> ??? CPU Period: 100000 >>>> ??? CPU Quota: -1 >>>> ??? CPU Shares: -1 >>>> ??? List of Processors, 4 total: >>>> ??? 4 5 6 7 >>>> ??? List of Effective Processors, 4 total: >>>> ??? 4 5 6 7 >>>> ??? List of Memory Nodes, 2 total: >>>> ??? 0 1 >>>> ??? List of Available Memory Nodes, 2 total: >>>> ??? 0 1 >>>> ??? CPUSet Memory Pressure Enabled: false >>>> ??? Memory Limit: 256.00M >>>> ??? Memory Soft Limit: Unlimited >>>> ??? Memory&? Swap Limit: 512.00M >>>> ??? Kernel Memory Limit: Unlimited >>>> ??? TCP Memory Limit: Unlimited >>>> ??? Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > From erik.joelsson at oracle.com Fri Jun 8 15:37:10 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 8 Jun 2018 08:37:10 -0700 Subject: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically In-Reply-To: <4ff74b09-ec9f-0a80-31d5-49039d5e1e8c@oracle.com> References: <4ff74b09-ec9f-0a80-31d5-49039d5e1e8c@oracle.com> Message-ID: <04d6b9dc-e091-4d63-1816-15846a7b34d3@oracle.com> Looks good. /Erik On 2018-06-08 01:50, Magnus Ihse Bursie wrote: > On 2018-06-07 23:20, Erik Joelsson wrote: >> Hello Magnus, >> >> Very nice refactoring! > Thanks! > >> >> JdkNativeCompilation.gmk >> line 126-127 looks a bit long. There is an extra space on 126. Also, >> why not addprefix for adding -I instead of clunky foreach? Not that I >> care greatly, but I usually prefer that construct. > > Yeah, I agree, addprefix is better. I just forgot about it; foreach is > a nice allround tool. > > Updated webrev: > http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.02/ > (Only changes is in JdkNativeCompilation.gmk, as per above comments). > > /Magnus > >> >> Otherwise looks good. >> >> /Erik >> >> >> On 2018-06-07 13:22, Magnus Ihse Bursie wrote: >>> This change needs some background information. >>> >>> I've been working on simplifying and streamlining the compilation of >>> native libraries of the JDK. Previously, I introduced the >>> SetupJdkLibrary function, and started to get a better control of >>> compiler flags. This patch continues on both paths. >>> >>> The original intent of this change, which I naively thought was >>> going to be much simpler than it turned out :-) was to separate the >>> -I flags (location of header files) from other flags, and to >>> generate these automatically, wherever possible. Since we have a >>> standard way of locating native code, and most libraries just want >>> to have an -I path to their own source base and the generated Java >>> header, we should be able to provide this in SetupJdkLibrary. >>> >>> This turned out to be closely related to SetupJdkLibrary being able >>> to discover the location of the SRC directories itself, using >>> "convention over configuration" and assuming that the library >>> "libfoo" for "java.module" would be located in >>> java.module/*/native/libfoo. >>> >>> While this sounds simple in theory, when actually trying to >>> implement this I of course ran into all the places where some >>> special handling was indeed needed. So even if like 90% of all >>> libraries were simple to get to build using automated discovery of >>> source and header directories, the 10% that did not caused me much >>> more headaches than I had anticipated. On the other hand, now that >>> I've sorted out all those places, the few remaining odd solutions is >>> clearly documented and not just something that "just happens" due to >>> strange configurations. >>> >>> One file deserves mentioning specifically: Awt2dLibraries.gmk. The >>> java.desktop libraries are unfortunately quite entangled with each >>> other, and do not readily follow the patterns that are used >>> elsewhere in the code base. So it might just look like the file has >>> just gone from one state of messiness, to another, which would >>> hardly be an improvement. :-( I would still argue that the new >>> messiness is better: It is now much clearer in what ways the >>> libraries diverge from our standard assumption, and what course of >>> action needs to be taken to minimize these differences. (Which is >>> something I believe should be done -- these issues are not just >>> cosmetic but is the root of most of the issues we always see for >>> these libraries, when upgrading compilers, etc.) >>> >>> During this change, I noticed that not all native libraries include >>> the proper generated header file. This is a dangerous coding >>> practice, since a change in the Java part of the interface might not >>> get picked up properly in the native part. I've added the missing >>> includes that I've detected, and due to these changes, I'm also >>> including the component teams in what is really only a build change. >>> As can be seen for jdk.crypto.mscapi; there had indeed been changes >>> that needed correcting. >>> >>> Since this is (basically) a pure build change, my gold standard here >>> has been the build compare script. In essence, the build output >>> prior to my change and with this change are 100% identical. In >>> truth, this is a bit too strong a claim. A few changes has occurred, >>> but none of them should matter. Here's a breakdown of the compare.sh >>> results: >>> >>> * Windows-x64: >>> >>> No differences at all. >>> >>> * Solaris: >>> >>> Two libraries are reported to differ: libsaproc.so and >>> libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing >>> this, I found that the object files used to link these two libraries >>> has no disass differences. They have a slight binary difference and >>> a difference in size, due to the include paths being different (and >>> this is stored in the .o file, which makes it different). Somehow >>> this apparently triggers the linker to generate a slightly different >>> code in a few places, using a different register or so. (Weird...) >>> >>> * MacOS: >>> >>> Two libraries are reported to differ: libjava.dylib and >>> libmlib_image.dylib, both of them just reported as a binary diff (no >>> symbol, disass or fulldump differences). This is not really >>> unsuspected, but I analyzed it anyway. >>> >>> I found that for libjava.dylib, a single .o file was different. This >>> one was actually picked up from closed sources, and are not really >>> relevant for OpenJDK. Anyway, the reason for the difference was the >>> same as for the Solaris libs; the include paths had changes, which >>> caused a binary diff. >>> >>> For libmlib_image.dylib, the link order had changed causing the >>> noted binary difference. >>> >>> * Linux: >>> >>> On linux, the compare script noted differences for libextnet, >>> libjava, libmlib_image, libprefs, libsaproc, libsplashscreen and >>> libsunec. >>> >>> The differences for libextnet, libprefs, libsplashscreen and >>> libsunec turned out to be caused by the added #include of the >>> generated Java headers. This caused binary differences (reasonably), >>> and for some odd reason also a symbol difference in >>> java_awt_SplashScreen.o (clazz.10057 and mid.10058 were replaced by >>> clazz.10015 and mid.10016). I can't claim to understand this, but >>> I'm assuming it's some kind of generated code. >>> >>> libsaproc and libjava changes was caused by closed source changes, >>> and is therefore not relevant to OpenJDK. >>> >>> For libmlib_image.dylib, the link order had changed causing the >>> noted binary difference, as on MacOS. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204572 >>> WebRev: >>> http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.01 >>> >>> /Magnus >>> >> > From jonathan.halliday at redhat.com Fri Jun 8 10:59:58 2018 From: jonathan.halliday at redhat.com (Jonathan Halliday) Date: Fri, 8 Jun 2018 11:59:58 +0100 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> Message-ID: Hi Paul Looks like we're all on the same page regarding the basic approach of using a small API and making the critical bits intrinsic. We perhaps have some way to go on exactly what that API looks like in terms of the classes and methods, but iterating on it by discussion of a JEP seems like the best way forward. The important thing from my perspective is that so far nobody has come forward with a use case that is not covered by the proposed primitives. So it's a small API, but not too small. As far a tweaks go, we have considered making the low level primitive method / intrinsic just a flushCacheline(base_address), since the arithmetic and loop for writing flush(from,to) in terms of that low level op is something that can be optimized fine by the JIT already. Though that does mean exposing the cache line size to the Java layer, whereas currently it's only visible in the C code. My own background and focus is transactions systems, so I'm more about the speed and fault tolerance than the capacity, but I can see long vs. int being of interest to our Infinispan data grid team and likewise for e.g. Oracle Coherence or databases like Cassandra. OTOH it's not uncommon to prefer moderately sized files and shard over them, which sidesteps the issue. Utility code to assist with fine-grained memory management within the buffer/file may be more useful than support for really large buffers, since they tend to be used with some form of internal block/heap structure anyhow, rather than to hold very large objects. Providing that may be the role of a 3rd party pure Java library like PCJ though, rather than something we want in the JDK itself at this early stage. The researcher in me is kinda interested in how much of the memory allocation and GC code can be re-purposed here though... What's the intended timeline on long buffer indexing at present? My feeling is a pmem API JEP is probably targeting around JDK 13, but we're flexible on that. We may also want to look at related enhancements like unmapping buffers. I think those pieces are sufficient decoupled that they won't be dependencies for the pmem API though, unlike other factors such as the availability of test hardware. Regards Jonathan. On 08/06/18 01:42, Paul Sandoz wrote: > Hi Andrew, Jonathan, > > Sandhya gave an overview to a few of us Oracle folks. I agree with what Sandhya says regarding the API, a small surface, and on pursuing an unsafe intrinsic. I like it and would encourage the writing of a draft JEP, especially to give this visibility. > > I expect this will be beneficial for experimentation with the Panama foreign API where we can use a Pointer to reference into a byte buffer and scribble on it. Further, i hope this work may also benefit the persistent collections effort (PCJ). > > > It intersects with https://bugs.openjdk.java.net/browse/JDK-8153111 ((bf) Allocating ByteBuffer on heterogeneous memory), which is attempting to be more generic. > > We might also need to increase the velocity on https://bugs.openjdk.java.net/browse/JDK-8180628 (retrofit direct buffer support for size beyond gigabyte scales), and i would be very interested your views on this, how you might be currently working around such size limitations, and what buffer enhancements would work for you. > > Thanks, > Paul. > > >> On May 30, 2018, at 10:21 PM, Viswanathan, Sandhya wrote: >> >> Hi Andrew/Jonathan, >> >> Thanks a lot for sharing this work. Copying hotspot-compiler-dev to get their feedback as well. >> >> Couple of thoughts/observations below: >> * Supporting ByteBuffer on persistent memory using existing FileChannel and MappedByteBuffer mechanism sounds like a very good idea. >> >> * Extending FileChannel.map to take additional parameter to indicate that the ByteBuffer is backed by persistent memory is a small API change. >> >> * Adding MappedByteBuffer.force(int from, int to) method on smaller range is very useful in addition to the force() on entire ByteBuffer. >> >> * The underlying force0_mapsync() could be implemented in terms of new unsafe APIs which in turn could be intrinsified. >> The advantage of this is that the unsafe APIs could then be used for other future persistent memory APIs in the JRE. >> Specifically the following two unsafe APIs would be useful: >> a) public native void flush(long address, long size); >> b) public native void storeFence(); >> storeFence() exists today but doesn?t generate any instruction for x86. >> Wondering if we could have additional boolean parameter to force the sfence generation. >> * DEFAULT_CACHE_LINE_SIZE is 128 in src/hotspot/cpu/x86/globalDefinitions_x86.hpp whereas actual cache line on the hardware is 64 bytes. >> This could be the cause for some of performance that you saw with compiler intrinsic vs pure C native. >> >> Best Regards, >> Sandhya >> >> >> >> RFC: Experiment in accessing/managing persistent memory from Java >> Andrew Dinn adinn at redhat.com >> Mon May 21 09:47:46 UTC 2018 >> >> I have been helping one of my Red Hat colleagues, Jonathan Halliday, to >> investigate provision of a Java equivalent to Intel's libpmem suite of C >> libraries [1]. This approach avoids the significant cost of using the >> Intel libraries from Java via JNI (or, worse, as a virtual driver for a >> persistent memory device). >> >> Jonathan has modified the JVM/JDK to allow a MappedByteBuffer to be >> mapped over persistent memory, providing equivalent function to libpmem >> itself. >> >> On top of this he implemented a Java journaled log class providing >> equivalent functionality to one of the Intel client libs, libpmemlog, >> built over libpmem. >> >> The modified MappedByteBuffer can be configured to use either i) a >> registered native method or ii) a JIT intrinsic to perform the critical >> task of cache line writeback i.e. the persistence step (the intrinsic is >> my contribution). >> >> Jonathan's tests compare use of JNI, registered native and intrinsic >> with an equivalent C program to write a large swathe of records to a >> journaled log file stored in persistent memory. Performance is worse >> than C when relying on JNI and significantly better with JVM/JDK >> support. Indeed, as one might reasonably expect, use of the JIT >> intrinsic almost completely eliminates writeback costs. >> >> The journaled log code, jdk dev tree patch, build instructions, test >> code plus C equivalent and test results are all available from >> Jonathan's git repo [2]. >> >> For those who do not want to look at the actual code, the README file >> [3] provides background to use of persistent memory, an overview of the >> design, and summary details of the test process and results. >> >> [1] https://pmem.io/pmdk/ >> [2] https://github.com/jhalliday/pmem >> [3] http://github.com://jhalliday/pmem/README.md >> >> n.b. Jonathan has experimented with using this same prototype to replace >> the journaled log used in the Red Hat Narayana transaction manager. It >> provides a significant improvement on the current disk file based log, >> both for throughput and latency (the code is not yet available as >> getting it to work involved some horrible hacking of the build to >> migrate up to jdk11). >> >> >> regards, >> >> >> Andrew Dinn >> ----------- >> Senior Principal Software Engineer >> Red Hat UK Ltd >> Registered in England and Wales under Company Registration No. 03798903 >> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > -- Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From brent.christian at oracle.com Fri Jun 8 19:11:49 2018 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 8 Jun 2018 12:11:49 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to $FEATURE In-Reply-To: <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> References: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> Message-ID: <2786fbb3-4cff-427c-2f72-2d0dfbf37031@oracle.com> On 6/7/18 1:24 PM, mandy chung wrote: >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8204565 >> >> Webrev: >> http://cr.openjdk.java.net/~bchristi/8204565/webrev/ > > Is there an existing test validating this? Looks like there is (kind of), for libs and for hotspot. I've rev'ed the webrev in place with how the tests might be updated. (or I can leave the hotspot test to be updated along with 8193719 if that's preferred). Thanks, -Brent From mandy.chung at oracle.com Fri Jun 8 19:27:57 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 12:27:57 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to $FEATURE In-Reply-To: <2786fbb3-4cff-427c-2f72-2d0dfbf37031@oracle.com> References: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> <2786fbb3-4cff-427c-2f72-2d0dfbf37031@oracle.com> Message-ID: On 6/8/18 12:11 PM, Brent Christian wrote: > On 6/7/18 1:24 PM, mandy chung wrote: >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8204565 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~bchristi/8204565/webrev/ >> >> Is there an existing test validating this? > > Looks like there is (kind of), for libs and for hotspot.? I've rev'ed > the webrev in place with how the tests might be updated. (or I can leave > the hotspot test to be updated along with 8193719 if that's preferred). test/jdk/java/lang/System/Versions.java it can also verify java.vm.specification.version. The hotspot test looks to me that it should expect the test be run with OpenJDK build and the vendor verification should consider that. That's a separate issue unrelated to this change. I suggest to remove the comment at line 36. Otherwise, looks good. Mandy From mandy.chung at oracle.com Fri Jun 8 20:22:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 13:22:22 -0700 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> Message-ID: <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> On 6/8/18 12:07 AM, Chris Yin wrote: > Hi, Mandy > > Many thanks for your detailed review and comments, updates new webrev > as below, and comment inline, thanks > > webrev: http://cr.openjdk.java.net/~xyin/8201528/webrev.01/ Thanks for adding the test description, that is very helpful. This is looking pretty good. I have some suggestion on the test arguments that one can translate to the command-line easily. 69 * PackageFromManifest runJar single 73 * PackageFromManifest runUrlLoader single single means running with "testpack1.jar". What about putting the JAR file name as the option which makes it very clear what is being tested. Nit: "testpack1" could be confused with pack200 and maybe simply test1.jar. How about @run main PackageFromManifest runJar test1.jar 77 * PackageFromManifest runJar 1 81 * PackageFromManifest runJar 2 These tests load two JAR files and load foo.Foo. What do you think to express: PackageFromManifest runJar test1.jar test2.jar foo.Foo1 PackageFromManifest runJar test1.jar test2.jar foo.Foo2 85 * PackageFromManifest runUrlLoader 1 89 * PackageFromManifest runUrlLoader 2 PackageFromManifest runUrlLoader test1.jar test2.jar foo.Foo1 PackageFromManifest runUrlLoader test1.jar test2.jar foo.Foo2 Nit: I suggest to group the runType together. i.e. move line 34 to above line 37. "

" not strictly needed as we won't generate the javadoc. You can consider leaving it a blank line. Mandy From brent.christian at oracle.com Fri Jun 8 21:39:55 2018 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 8 Jun 2018 14:39:55 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to $FEATURE In-Reply-To: References: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> <2786fbb3-4cff-427c-2f72-2d0dfbf37031@oracle.com> Message-ID: <3977f83a-72d8-bf3c-663f-2f469505362d@oracle.com> On 6/8/18 12:27 PM, mandy chung wrote: >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~bchristi/8204565/webrev/ >>> > > test/jdk/java/lang/System/Versions.java > ? it can also verify java.vm.specification.version. > > The hotspot test looks to me that it should expect the test be run > with OpenJDK build and the vendor verification should consider that. > That's a separate issue unrelated to this change.? I suggest to > remove the comment at line 36. > > Otherwise, looks good. OK, thanks - done. (Also updated the @bug, copyright year, and removed unused 'VMversion'. -Brent From jonathan.gibbons at oracle.com Fri Jun 8 22:06:26 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 8 Jun 2018 15:06:26 -0700 Subject: RFR: JDK-8204588: Test failures after "Launch Single-File Source-Code Programs" Message-ID: <7c0d8d00-bbe4-558f-68f5-66114e59ce6b@oracle.com> Please review two test fixes related to the source launcher feature. In one test, the fix is to use File.separator to construct "golden output" for comparison. In the other test, the failure was caused by excessively long paths to the Java launcher in some test execution environments, causing the shebang line to overflow the underlying system limit of 128 characters.? The fix is to skip test cases when that occurs, as well as to augment the test with more tracing information to help diagnose failures on remote build and test systems. The tests are removed from the corresponding ProblemList files. JBS: https://bugs.openjdk.java.net/browse/JDK-8204588 Webrev: http://cr.openjdk.java.net/~jjg/8204588/webrev.00/index.html -- Jon From mandy.chung at oracle.com Fri Jun 8 22:20:01 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 15:20:01 -0700 Subject: RFR: JDK-8204588: Test failures after "Launch Single-File Source-Code Programs" In-Reply-To: <7c0d8d00-bbe4-558f-68f5-66114e59ce6b@oracle.com> References: <7c0d8d00-bbe4-558f-68f5-66114e59ce6b@oracle.com> Message-ID: <7efdde87-741b-3a22-5321-d68bba4e3b4f@oracle.com> On 6/8/18 3:06 PM, Jonathan Gibbons wrote: > Please review two test fixes related to the source launcher feature. > > In one test, the fix is to use File.separator to construct "golden > output" for comparison. > > In the other test, the failure was caused by excessively long paths to > the Java launcher in some test execution environments, causing the > shebang line to overflow the underlying system limit of 128 characters. > The fix is to skip test cases when that occurs, as well as to augment > the test with more tracing information to help diagnose failures on > remote build and test systems. > > The tests are removed from the corresponding ProblemList files. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8204588 > Webrev: http://cr.openjdk.java.net/~jjg/8204588/webrev.00/index.html The change looks good to me. Skipping the test when it exceeds the system limit which is reasonable. Mandy From mandy.chung at oracle.com Sat Jun 9 04:29:53 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 21:29:53 -0700 Subject: Review Request: 8204648: test/jdk/tools/launchers/SourceMode.java fails with long shebang line Message-ID: <8b28dd7a-1de3-7ee0-da73-40834f139e2c@oracle.com> JDK-8204588 [1] fixed the test failure caused by long paths to the Java launcher in some test execution environments, causing the shebang line to overflow the underlying system limit of 128 characters. The test needs a small tweak to the max javaCmd length to reduce from 100 to 90 since the arguments passed to java command are more than 28 characters. This is a quick fix for the test failure. A better fix would be to compute the length of the entire shebang line in each test case and determine if it should be skipped. That can be done as a follow up fix. diff --git a/test/jdk/tools/launcher/SourceMode.java b/test/jdk/tools/launcher/SourceMode.java --- a/test/jdk/tools/launcher/SourceMode.java +++ b/test/jdk/tools/launcher/SourceMode.java @@ -71,7 +71,7 @@ // limit of 120 characters for a shebang line. Path p = cwd.relativize(cmd); shortJavaCmd = (p.toString().length() < cmd.toString().length()) ? p : cmd; - skipShebangTest = shortJavaCmd.toString().length() > 100; + skipShebangTest = shortJavaCmd.toString().length() > 90; } log = System.err; thanks Mandy [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053700.html From mandy.chung at oracle.com Sat Jun 9 04:57:16 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 21:57:16 -0700 Subject: Review Request: 8204648: test/jdk/tools/launchers/SourceMode.java fails with long shebang line In-Reply-To: <8b28dd7a-1de3-7ee0-da73-40834f139e2c@oracle.com> References: <8b28dd7a-1de3-7ee0-da73-40834f139e2c@oracle.com> Message-ID: <428b1358-bc26-0b9a-81c6-b6070d10cb98@oracle.com> I run into some issue with shebang tests. Since Jon is on vacation, I revise the patch to skip the shebang test temporarily until he returns. Mandy diff --git a/test/jdk/tools/launcher/SourceMode.java b/test/jdk/tools/launcher/SourceMode.java --- a/test/jdk/tools/launcher/SourceMode.java +++ b/test/jdk/tools/launcher/SourceMode.java @@ -71,7 +71,8 @@ // limit of 120 characters for a shebang line. Path p = cwd.relativize(cmd); shortJavaCmd = (p.toString().length() < cmd.toString().length()) ? p : cmd; - skipShebangTest = shortJavaCmd.toString().length() > 100; + // skipShebangTest = shortJavaCmd.toString().length() > 90; + skipShebangTest = true; } log = System.err; On 6/8/18 9:29 PM, mandy chung wrote: > JDK-8204588 [1] fixed the test failure caused by long paths to the Java > launcher in some test execution environments, causing the shebang line > to overflow the underlying system limit of 128 characters. > > The test needs a small tweak to the max javaCmd length to reduce from > 100 to 90 since the arguments passed to java command are more than 28 > characters.? This is a quick fix for the test failure.? A better fix > would be to compute the length of the entire shebang line in each test > case and determine if it should be skipped.? That can be done as a > follow up fix. > > diff --git a/test/jdk/tools/launcher/SourceMode.java > b/test/jdk/tools/launcher/SourceMode.java > --- a/test/jdk/tools/launcher/SourceMode.java > +++ b/test/jdk/tools/launcher/SourceMode.java > @@ -71,7 +71,7 @@ > ???????????? // limit of 120 characters for a shebang line. > ???????????? Path p = cwd.relativize(cmd); > ???????????? shortJavaCmd = (p.toString().length() < > cmd.toString().length()) ? p : cmd; > -??????????? skipShebangTest = shortJavaCmd.toString().length() > 100; > +??????????? skipShebangTest = shortJavaCmd.toString().length() > 90; > ???????? } > > ???????? log = System.err; > > thanks > Mandy > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053700.html From fw at deneb.enyo.de Sat Jun 9 10:27:55 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 09 Jun 2018 12:27:55 +0200 Subject: The store for byte strings Message-ID: <87y3fo8das.fsf@mid.deneb.enyo.de> Lately I've been thinking about string representation. The world turned out not to be UCS-2 or UTF-16, after all, and we often have to deal with strings generally encoded as ASCII or UTF-8, but we aren't always encoded this way (and there might not even be a charset declaration, see the ELF spec). (a) byte[] with defensive copies. Internal storage is byte[], copy is made before returning it to the caller. Quite common across the JDK. (b) byte[] without defensive copies. Internal storage is byte[], and a reference is returned. In the past, this could be a security bug, and usually, it was adjusted to (a) when noticed. Without security requirements, this can be quite efficient, but there is ample potential for API misuse. (c) java.lang.String with ISO-8859-1 decoding/encoding. Sometimes done by reconfiguring the entire JVM to run with ISO-8859-1, usually so that it is possible to process malformed UTF-8. The advantage is that there is rich API support, including regular expressions, and good optimization. There is also language support for string literals. (d) java.lang.String with UTF-8 decoding/encoding and replacement. This seems to be very common, but is not completely accurate and can lead to subtle bugs (or completely non-processible data). Otherwise has the same advantages as (c). (e) Various variants of ByteBuffer. Have not seen this much in practice (outside binary file format parsers). In the past, it needed deep defensive copies on input for security (because there isn't an immutably backed ByteBuffer), and shallow copies for access. The ByteBuffer objects themselves are also quite heavy when they can't be optimized away. For that reason, probably most useful on interfaces, and not for storage. (f) Custom, immutable ByteString class. Quite common, but has cross-library interoperability issues, and a full complement of support (matching java.lang.String) is quite hard. (g) Something based on VarHandle. Haven't seen this yet. Probably not useful for storage. Anything that I have missed? Considering these choices, what is the expected direction on the JDK side for new code? Option (d) for things generally ASCII/UTF-8, and (b) for things of a more binary nature? What to do if the choice is difficult? From henry.jen at oracle.com Sat Jun 9 16:16:31 2018 From: henry.jen at oracle.com (Henry Jen) Date: Sat, 9 Jun 2018 09:16:31 -0700 Subject: RFR: 8199871: Deprecate pack200 and unpack200 tools In-Reply-To: <78256921-225B-4AEF-8313-42E135BDB886@oracle.com> References: <78256921-225B-4AEF-8313-42E135BDB886@oracle.com> Message-ID: <84F9D20B-201E-4E16-B5F1-7A885343E29D@oracle.com> I revised the webrev to have warning only executable name for unpack200 instead of full path on Windows, which I believe is what was intended all along, the test is also revised to take unpack200.exe on the Windows platform. The updated version is at http://cr.openjdk.java.net/~henryjen/jdk11/8199871/1/webrev/ Cheers, Henry > On Jun 8, 2018, at 6:56 PM, Henry Jen wrote: > > Hi, > > Please review this webrev[1] in which we mark Pack200 related API and tools deprecate, and print a warning about tools to be remove in a future JDK release. To avoid interrupt existing tools depends on the output, an option is provided to suppress the warning message. > > The option name is following the convention of javah, not the GNU style as we should. The rationale is that this is a temporary option and an existing convention would make it more consistent for users. Anyhow, if we have a strong feeling this should be changed, we can do that. > > For jar tool, the normalize option use Pack200 is also deprecated, the suppress option only available to the new GNU style option mode. As if user need to provide the option, the command line change is necessary, so without compatible mode option should not be an issue. > > Cheers, > Henry > > [1] http://cr.openjdk.java.net/~henryjen/jdk11/8199871/0/webrev/ From joe.darcy at oracle.com Sat Jun 9 16:39:21 2018 From: joe.darcy at oracle.com (joe darcy) Date: Sat, 9 Jun 2018 09:39:21 -0700 Subject: Review Request: 8204648: test/jdk/tools/launchers/SourceMode.java fails with long shebang line In-Reply-To: <428b1358-bc26-0b9a-81c6-b6070d10cb98@oracle.com> References: <8b28dd7a-1de3-7ee0-da73-40834f139e2c@oracle.com> <428b1358-bc26-0b9a-81c6-b6070d10cb98@oracle.com> Message-ID: Skipping the shebang tests is fine a workaround Mandy; thanks, -Joe On 6/8/2018 9:57 PM, mandy chung wrote: > I run into some issue with shebang tests.? Since Jon is on vacation, > I revise the patch to skip the shebang test temporarily until he returns. > > Mandy > > diff --git a/test/jdk/tools/launcher/SourceMode.java > b/test/jdk/tools/launcher/SourceMode.java > --- a/test/jdk/tools/launcher/SourceMode.java > +++ b/test/jdk/tools/launcher/SourceMode.java > @@ -71,7 +71,8 @@ > ???????????? // limit of 120 characters for a shebang line. > ???????????? Path p = cwd.relativize(cmd); > ???????????? shortJavaCmd = (p.toString().length() < > cmd.toString().length()) ? p : cmd; > -??????????? skipShebangTest = shortJavaCmd.toString().length() > 100; > +??????????? // skipShebangTest = shortJavaCmd.toString().length() > 90; > +??????????? skipShebangTest = true; > ???????? } > > ???????? log = System.err; > > > On 6/8/18 9:29 PM, mandy chung wrote: >> JDK-8204588 [1] fixed the test failure caused by long paths to the >> Java launcher in some test execution environments, causing the >> shebang line to overflow the underlying system limit of 128 characters. >> >> The test needs a small tweak to the max javaCmd length to reduce from >> 100 to 90 since the arguments passed to java command are more than 28 >> characters.? This is a quick fix for the test failure.? A better fix >> would be to compute the length of the entire shebang line in each >> test case and determine if it should be skipped.? That can be done as >> a follow up fix. >> >> diff --git a/test/jdk/tools/launcher/SourceMode.java >> b/test/jdk/tools/launcher/SourceMode.java >> --- a/test/jdk/tools/launcher/SourceMode.java >> +++ b/test/jdk/tools/launcher/SourceMode.java >> @@ -71,7 +71,7 @@ >> ????????????? // limit of 120 characters for a shebang line. >> ????????????? Path p = cwd.relativize(cmd); >> ????????????? shortJavaCmd = (p.toString().length() < >> cmd.toString().length()) ? p : cmd; >> -??????????? skipShebangTest = shortJavaCmd.toString().length() > 100; >> +??????????? skipShebangTest = shortJavaCmd.toString().length() > 90; >> ????????? } >> >> ????????? log = System.err; >> >> thanks >> Mandy >> [1] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053700.html From xueming.shen at oracle.com Sat Jun 9 19:18:05 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Sat, 09 Jun 2018 12:18:05 -0700 Subject: The store for byte strings In-Reply-To: <87y3fo8das.fsf@mid.deneb.enyo.de> References: <87y3fo8das.fsf@mid.deneb.enyo.de> Message-ID: <5B1C27ED.2020404@oracle.com> On 6/9/18, 3:27 AM, Florian Weimer wrote: > Lately I've been thinking about string representation. The world > turned out not to be UCS-2 or UTF-16, after all, and we often have to > deal with strings generally encoded as ASCII or UTF-8, but we aren't > always encoded this way (and there might not even be a charset > declaration, see the ELF spec). > > (a) byte[] with defensive copies. > Internal storage is byte[], copy is made before returning it to > the caller. Quite common across the JDK. > > (b) byte[] without defensive copies. > Internal storage is byte[], and a reference is returned. In the > past, this could be a security bug, and usually, it was adjusted > to (a) when noticed. Without security requirements, this can be > quite efficient, but there is ample potential for API misuse. > > (c) java.lang.String with ISO-8859-1 decoding/encoding. > Sometimes done by reconfiguring the entire JVM to run with > ISO-8859-1, usually so that it is possible to process malformed > UTF-8. The advantage is that there is rich API support, including > regular expressions, and good optimization. There is also > language support for string literals. > > (d) java.lang.String with UTF-8 decoding/encoding and replacement. > This seems to be very common, but is not completely accurate > and can lead to subtle bugs (or completely non-processible > data). Otherwise has the same advantages as (c). > > (e) Various variants of ByteBuffer. > Have not seen this much in practice (outside binary file format > parsers). In the past, it needed deep defensive copies on input > for security (because there isn't an immutably backed ByteBuffer), > and shallow copies for access. The ByteBuffer objects themselves > are also quite heavy when they can't be optimized away. For that > reason, probably most useful on interfaces, and not for storage. > > (f) Custom, immutable ByteString class. > Quite common, but has cross-library interoperability issues, > and a full complement of support (matching java.lang.String) > is quite hard. > > (g) Something based on VarHandle. > Haven't seen this yet. Probably not useful for storage. > > Anything that I have missed? > > Considering these choices, what is the expected direction on the JDK > side for new code? Option (d) for things generally ASCII/UTF-8, and > (b) for things of a more binary nature? What to do if the choice is > difficult? Hi Florian, Some comments about the j.l.String storage. Ideally I would assume we would want to have a utf-8 internal storage for String, even in theory utf8 is supposed to be used externally and utf16 to be the internal one. I did have a byte[]/utf-8 prototype implementation when we did the compact string for jdk9 but that was finally dropped because of the potential performance regression for index base access, such as the basic String.charAt(int), as you have to count from the beginning to locate the target character each every time. But I think we might want to try it again later, especially for use scenario that index base access performance is not that important/critical and the throughput operation of the String, means input from /output to the external utf-8/byte[] world, is more desired. Given we are heading utf-8 as the default encoding for jvm [1], I think we might want to at least provide some alternative that you can "optionally" do that for String object. The idea might go further (wild, just an idea, not necessary something thing we really want to do :-) for Java String) to other charsets, so you can simply store the byte[] (verified no malformed/unmappable) + charsetId directly when creating a String object. This might be useful and efficient in use scenario that the String object is simply a vehicle to carry a sequence of characters back and forth between a front end server and back end server, the jvm is simply passing them around/through. Defensive copy when getting byte[] in & out of String object seems still inevitable for now, before we can have something like "read-only" byte[], given the nature of its immutability commitment. Regards, Sherman [1] https://bugs.openjdk.java.net/browse/JDK-8187041 From john.r.rose at oracle.com Sat Jun 9 23:52:25 2018 From: john.r.rose at oracle.com (John Rose) Date: Sat, 9 Jun 2018 16:52:25 -0700 Subject: The store for byte strings In-Reply-To: <87y3fo8das.fsf@mid.deneb.enyo.de> References: <87y3fo8das.fsf@mid.deneb.enyo.de> Message-ID: <82A561D3-1A00-47E4-9A9D-C180F2B8AA8E@oracle.com> I'm glad to see you are thinking about this, Florian. You appear to be aiming at a way to compactly store and manipulate series of octets (in an arbitrary encoding) with an emphasis on using those octets to represent strings, in the usual sense of character sequences. Would you agree that this design problem factors well into a generic problem of storing and manipulating octet sequences, plus a detachable upper layer that allows strings (in various encodings) to be extracted from those sequences? I think the sweet spot here is to introduce a "stringy but char-free" API which commits to dealing with chunks of memory (viewed as octet sequences), regardless of how those chunks will be interpreted. In https://bugs.openjdk.java.net/browse/JDK-8161256 I discuss this nascent API under the name "ByteSequence", which is analogous to CharSequence, but doesn't mention the types 'char' or 'String'. By "stringy" I mean that there are natural ways to index or partition an existing sequence of octets, or concatenate multiple sequences into one. I also mean that immutability plays a strong role, enabling algorithms to work without defensive copies. Making it an interface like CharSequence means we can use backing stores like ByteBuffer or byte[], or more exotic things like Panama native memory, interoperably. Here are some uses for an immutable octet sequence type: - manipulation of non-UTF16 character data (which you mention) - zero copy views (slices, modifiable or not) into existing types (ByteBuffer, byte[], etc.) - zero copy views into file images (N.B. requires a 'long' size property, not 'int') - zero copy views to intra-classfile resources (CONSTANT_Bytes) - backing stores for Panama data structures and smart pointers - copy-reduced scatter and gather nodes associated with packet processing - octet-level cursors for parsers, scanners, packet decoders, etc. If the ByteSequence views are value instances, they can be created at a very high rate with little or no GC impact. Generic algorithms would still operate on them A mutable octet sequence class, analogous to StringBuilder, would allow immutable sequences to be built with fewer intermediate copies, just like with StringBuilder. If the API is properly defined it can be inserted directly into existing types like ByteBuffer. Doing this will probably require us to polish ByteBuffer a little, adding immutability as an option and lifting the 32-bit limits. It should be possible to "freeze" a ByteBuffer or array and use it as a backing store that is reliably immutable, so it can be handed to zero-copy algorithms that work with ByteSequences. For some of this see https://bugs.openjdk.java.net/browse/JDK-8180628 . Independently, I want to eventually add frozen arrays, including frozen byte[] arrays, to the JVM, but that doesn't cover zero-copy use cases; it has to be an interface like CharSequence. So the option I prefer is not on your list; it would be: (h) ByteSequence interface with retrofits to ByteBuffer, byte[], etc. This is more flexible than (f) the concrete ByteString class. I think the ByteString you are thinking of would appear as a non-public class created by a ByteSequence factory, analogous to List::of. ? John On Jun 9, 2018, at 3:27 AM, Florian Weimer wrote: > > Lately I've been thinking about string representation. The world > turned out not to be UCS-2 or UTF-16, after all, and we often have to > deal with strings generally encoded as ASCII or UTF-8, but we aren't > always encoded this way (and there might not even be a charset > declaration, see the ELF spec). > > (a) byte[] with defensive copies. > Internal storage is byte[], copy is made before returning it to > the caller. Quite common across the JDK. > > (b) byte[] without defensive copies. > Internal storage is byte[], and a reference is returned. In the > past, this could be a security bug, and usually, it was adjusted > to (a) when noticed. Without security requirements, this can be > quite efficient, but there is ample potential for API misuse. > > (c) java.lang.String with ISO-8859-1 decoding/encoding. > Sometimes done by reconfiguring the entire JVM to run with > ISO-8859-1, usually so that it is possible to process malformed > UTF-8. The advantage is that there is rich API support, including > regular expressions, and good optimization. There is also > language support for string literals. > > (d) java.lang.String with UTF-8 decoding/encoding and replacement. > This seems to be very common, but is not completely accurate > and can lead to subtle bugs (or completely non-processible > data). Otherwise has the same advantages as (c). > > (e) Various variants of ByteBuffer. > Have not seen this much in practice (outside binary file format > parsers). In the past, it needed deep defensive copies on input > for security (because there isn't an immutably backed ByteBuffer), > and shallow copies for access. The ByteBuffer objects themselves > are also quite heavy when they can't be optimized away. For that > reason, probably most useful on interfaces, and not for storage. > > (f) Custom, immutable ByteString class. > Quite common, but has cross-library interoperability issues, > and a full complement of support (matching java.lang.String) > is quite hard. > > (g) Something based on VarHandle. > Haven't seen this yet. Probably not useful for storage. > > Anything that I have missed? > > Considering these choices, what is the expected direction on the JDK > side for new code? Option (d) for things generally ASCII/UTF-8, and > (b) for things of a more binary nature? What to do if the choice is > difficult? From john.r.rose at oracle.com Sun Jun 10 01:48:59 2018 From: john.r.rose at oracle.com (John Rose) Date: Sat, 9 Jun 2018 18:48:59 -0700 Subject: The store for byte strings In-Reply-To: <5B1C27ED.2020404@oracle.com> References: <87y3fo8das.fsf@mid.deneb.enyo.de> <5B1C27ED.2020404@oracle.com> Message-ID: On Jun 9, 2018, at 12:18 PM, Xueming Shen wrote: > > Ideally I would assume we would want to have a utf-8 internal storage for > String, even in theory utf8 is supposed to be used externally and utf16 > to be the internal one. Separately from my point about ByteSequence, I agree that "doubling down" on Utf8 as a standard format for packed strings is a good idea. A reasonable way to prototype right now would be an implementation of CharSequence that is backed by a byte[] (eventually ByteSequence) and has some sort of fast access (probably streaming) to Utf16 code points. To make it pay for itself the Utf8 encoding should be applicable as an overlay in as many places as possible, including slices of byte[] and ByteBuffer objects, and later ByteSequences. > Defensive copy when getting byte[] in & out of String object seems still > inevitable for now, before we can have something like "read-only" byte[], > given the nature of its immutability commitment. We didn't need frozen char[] arrays to avoid defensive copying of String objects, only an immutability invariant on the class. We could pull a similar trick with Utf8 by supplying a ByteSequence view of a String's underlying bytes. If the String has underlying chars (Utf16) a view is also possible, although it is more difficult to get right (as you described). ? John From fw at deneb.enyo.de Sun Jun 10 09:47:11 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 10 Jun 2018 11:47:11 +0200 Subject: The store for byte strings In-Reply-To: <82A561D3-1A00-47E4-9A9D-C180F2B8AA8E@oracle.com> (John Rose's message of "Sat, 9 Jun 2018 16:52:25 -0700") References: <87y3fo8das.fsf@mid.deneb.enyo.de> <82A561D3-1A00-47E4-9A9D-C180F2B8AA8E@oracle.com> Message-ID: <877en73rds.fsf@mid.deneb.enyo.de> * John Rose: > In https://bugs.openjdk.java.net/browse/JDK-8161256 I discuss > this nascent API under the name "ByteSequence", which is analogous > to CharSequence, but doesn't mention the types 'char' or 'String'. Very interesting. What's the specification for toString() and hashCode()? One problem of retrofitting a custom ByteString into a CharSequence is that CharSequence reuses toString() in a fairly central fashion, and it's hard to reconcile this with byte-based length() and charAt() methods unless ISO-8859-1 encoding is used. If this feature is supposed to land before JEP 218? If not, how does ByteSequence differ from List? > If the ByteSequence views are value instances, they can be created > at a very high rate with little or no GC impact. Generic algorithms > would still operate on them I'm not up-to-date with those upcoming changes. Would the nature as value instances be expressed as part of the ByteSequence interface type? > If the API is properly defined it can be inserted directly into > existing types like ByteBuffer. Doing this will probably require us > to polish ByteBuffer a little, adding immutability as an option and > lifting the 32-bit limits. It should be possible to "freeze" a > ByteBuffer or array and use it as a backing store that is reliably > immutable, so it can be handed to zero-copy algorithms that work > with ByteSequences. Such freezing is incompatible with mapped byte buffers, right? Even if the implementation prevents changes at the VM/process level, changes on the file system could well become visible. Do you expect to make freezing an optional operation (probably not a good idea), or copy the contents of the mapping to the heap (which is probably not too bad, considering that a shared byte[] could also result in arbitrarily large copies). > Independently, I want to eventually add frozen arrays, including > frozen byte[] arrays, to the JVM, but that doesn't cover zero-copy use > cases; it has to be an interface like CharSequence. Well, there is already the VarHandle approach for that. But it's not a particularly rich interface and very far away from strings. > So the option I prefer is not on your list; it would be: > > (h) ByteSequence interface with retrofits to ByteBuffer, byte[], etc. > > This is more flexible than (f) the concrete ByteString class. I think > the ByteString you are thinking of would appear as a non-public class > created by a ByteSequence factory, analogous to List::of. Yes, this seems reasonable. It's a bit of a drawback that the immutable nature of a value cannot be expressed in the type system (so that you have to remember to use ByteSequence::of to get a view to an immutable object), but at least it's consistent with collections. From xu.y.yin at oracle.com Mon Jun 11 05:12:07 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 11 Jun 2018 13:12:07 +0800 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> Message-ID: <3C0AFC7B-AA3C-4DE1-AF9A-0DB556A1AEAB@oracle.com> Hi, Mandy Thanks lot for your suggestions, update webrev as below and comment inline, thanks http://cr.openjdk.java.net/~xyin/8201528/webrev.02/ > On 9 Jun 2018, at 4:22 AM, mandy chung wrote: > > > > On 6/8/18 12:07 AM, Chris Yin wrote: >> Hi, Mandy >> Many thanks for your detailed review and comments, updates new webrev >> as below, and comment inline, thanks >> webrev: http://cr.openjdk.java.net/~xyin/8201528/webrev.01/ > > > Thanks for adding the test description, that is very helpful. > This is looking pretty good. I have some suggestion on the test > arguments that one can translate to the command-line easily. > > 69 * PackageFromManifest runJar single > 73 * PackageFromManifest runUrlLoader single > > single means running with "testpack1.jar". What about putting > the JAR file name as the option which makes it very clear what > is being tested. Nit: "testpack1" could be confused with pack200 > and maybe simply test1.jar. How about Fixed, thanks > > @run main PackageFromManifest runJar test1.jar > > 77 * PackageFromManifest runJar 1 > 81 * PackageFromManifest runJar 2 > > These tests load two JAR files and load foo.Foo. What do you > think to express: > > PackageFromManifest runJar test1.jar test2.jar foo.Foo1 > PackageFromManifest runJar test1.jar test2.jar foo.Foo2 > > 85 * PackageFromManifest runUrlLoader 1 > 89 * PackageFromManifest runUrlLoader 2 > > PackageFromManifest runUrlLoader test1.jar test2.jar foo.Foo1 > PackageFromManifest runUrlLoader test1.jar test2.jar foo.Foo2 Used new style as you suggested, that looks exactly match with the description :) > > Nit: I suggest to group the runType together. i.e. move line 34 > to above line 37. "

" not strictly needed as we won't generate > the javadoc. You can consider leaving it a blank line. Sure, grouped runType and removed

(oh, it?s added by IDE during code reformat) Thanks & Regards, Chris > > Mandy > From ivan.gerasimov at oracle.com Mon Jun 11 06:15:17 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sun, 10 Jun 2018 23:15:17 -0700 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> Message-ID: <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> Hi Alan! On 6/6/18 6:57 AM, Alan Bateman wrote: > I think this should be okay but the Windows implementation has a long > history of biting the fingers of anyone that dares touch it. Sometimes > unexpected behavior changes only come to light long after the change. > The recent mails here about Kafka and sparse files is a good example > of that. So I think it's important to check the test coverage before > pushing this change. Specifically I think we need to check that we > have tests that > > 1. Exercise shrinking, extending, and not change the file length > > 2. Check getFilePointer when used with setLength. > > 3. Check how FileChannel behaves when created from a RandomAccessFile > but the file length is changed after the FileChannel is obtained. > I extended the existing reg. test java/io/RandomAccessFile/SetLength.java to cover more cases that involve changing the file size. Also I added another regression test, as you Alan suggested, to check that RandomAccessFile and its FileChannel behave consistently in various scenarios. All the tests, including the new ones, pass on all supported platforms. BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/01/webrev/ Would you please help review the fix? -- With kind regards, Ivan Gerasimov From ecki at zusammenkunft.net Mon Jun 11 07:55:50 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Mon, 11 Jun 2018 07:55:50 +0000 (UTC) Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> Message-ID: <8C274EA3A72EE413.A6ABA911-B6E7-442F-973A-999E61EE9EDD@mail.outlook.com> We had BTW problems (on Windows) with SMB hosted files where appending causes sometimes the filepointer to not increase. We fixed that by closing the file in such conditions. It does however look like a smb Cache and not Java Code problem. But just in case anyone has seen encountered that... Not sure how easy it is to replicate, but will let you know. Bernd -- https://Bernd.eckenfels.net On Mon, Jun 11, 2018 at 9:45 AM +0200, "Ivan Gerasimov" wrote: Hi Alan! On 6/6/18 6:57 AM, Alan Bateman wrote: > I think this should be okay but the Windows implementation has a long > history of biting the fingers of anyone that dares touch it. Sometimes > unexpected behavior changes only come to light long after the change. > The recent mails here about Kafka and sparse files is a good example > of that. So I think it's important to check the test coverage before > pushing this change. Specifically I think we need to check that we > have tests that > > 1. Exercise shrinking, extending, and not change the file length > > 2. Check getFilePointer when used with setLength. > > 3. Check how FileChannel behaves when created from a RandomAccessFile > but the file length is changed after the FileChannel is obtained. > I extended the existing reg. test java/io/RandomAccessFile/SetLength.java to cover more cases that involve changing the file size. Also I added another regression test, as you Alan suggested, to check that RandomAccessFile and its FileChannel behave consistently in various scenarios. All the tests, including the new ones, pass on all supported platforms. BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/01/webrev/ Would you please help review the fix? -- With kind regards, Ivan Gerasimov From robbin.ehn at oracle.com Mon Jun 11 08:07:07 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 11 Jun 2018 10:07:07 +0200 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: Hi Bob, On 06/07/2018 07:43 PM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 Seems okay. Metrics.java "Returns the length of the operating system time slice" Note that is is only true if you are using a batch scheduler. Otherwise this period may be split on multiple 'time slices'. In printSystemMetrics there is no units, maybe intentional? Do we have support now in mach5 for docker jtreg, or do we still run these separate? You can ship it. Thanks for fixing, and super thanks for fixing the bug in PlainRead also! /Robbin > > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory & Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> > From david.holmes at oracle.com Mon Jun 11 08:32:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 11 Jun 2018 18:32:00 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Sorry Bob I haven't had a chance to look at this detail. For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? David On 8/06/2018 3:43 AM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory & Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> > From scolebourne at joda.org Mon Jun 11 09:22:50 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 11 Jun 2018 10:22:50 +0100 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: Finally got to check the logic in convert(Duration), and it looks fine to me. All three look good thanks Stephen (not an OpenJDK reviewer). On 6 June 2018 at 18:03, Martin Buchholz wrote: > OK, here is a RFR for low-hanging changes (some of these changes have > already been reviewed by some of you): > > 8204375: Add TimeUnit#convert(Duration) > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/convertDuration/index.html > https://bugs.openjdk.java.net/browse/JDK-8204375 > > 8204444: java.time cleanup > http://cr.openjdk.java.net/~martin/webrevs/jdk/time-lint/ > https://bugs.openjdk.java.net/browse/JDK-8204444 > > 8204377: Rename Object#wait parameter name from "timeout" to "timeoutMillis" > http://cr.openjdk.java.net/~martin/webrevs/jdk/Object-wait-timeout/ > https://bugs.openjdk.java.net/browse/JDK-8204377 > From peter.levart at gmail.com Mon Jun 11 10:57:57 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 11 Jun 2018 12:57:57 +0200 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap Message-ID: Hi, Those two methods were added in JDK 10 and they are not very optimal. Map.copyOf(Map map) 1st dumps the source map into an array of Map.Entry(s) (map.entrySet().toArray(new Entry[0])), which typically creates new Map.Entry objects, then passes the array to Map.ofEntries(Map.Entry[] entries) factory method that iterates the array and constructs a key/value interleaved array from it which is passed to ImmutableCollections.MapN constructor, which then constructs a linear-probing hash table from it. So each key and value is copied 3 times, while several temporary objects are created in the process. One copying step could be eliminated and construction of temporary Map.Entry objects too: http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.01/ Collecting stream(s) using Collectors.toUnmodifiableMap() 1st collects key/value pairs into a HashMap, then it performs equivalent operations as Map.copyOf(hashMap) at the end. Using Map.copyOf() directly benefits this collection operation too. The following benchmark: http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench.java Shows up to 30% improvement of .copyOf operation with this patch applied: Original: Benchmark?????????????????????????????? (size)? Mode Cnt????? Score???? Error? Units UnmodifiableMapBench.copyOf???????????????? 10? avgt 10??? 403.633 ??? 2.640? ns/op UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 3489.623 ?? 44.590? ns/op UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 40030.572 ? 277.075? ns/op UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10??? 831.221 ??? 3.816? ns/op UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9783.519 ?? 43.097? ns/op UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 96524.536 ? 670.818? ns/op Patched: Benchmark?????????????????????????????? (size)? Mode Cnt????? Score????? Error? Units UnmodifiableMapBench.copyOf???????????????? 10? avgt 10??? 264.172 ???? 1.882? ns/op UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 2318.974 ??? 15.877? ns/op UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 29291.782 ? 3139.737? ns/op UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10??? 771.221 ??? 65.432? ns/op UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9275.016 ?? 725.722? ns/op UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 82204.342 ?? 851.741? ns/op Production of garbage is also reduced, since no Map.Entry temporary objects are constructed: Original: Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10??? 416.001 ????? 0.002??? B/op UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10?? 2936.005 ????? 0.019??? B/op UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10? 28136.059 ????? 0.199??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10? avgt?? 10?? 1368.001 ????? 0.004??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100? avgt?? 10? 10208.139 ????? 0.045??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000? avgt?? 10? 93025.923 ????? 0.573??? B/op Patched: Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10??? 304.000 ????? 0.001??? B/op UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10?? 2464.004 ????? 0.012??? B/op UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10? 24064.040 ????? 0.137??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10? avgt?? 10?? 1256.001 ????? 0.003??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100? avgt?? 10?? 9720.153 ????? 0.055??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000? avgt?? 10? 88905.688 ????? 0.574??? B/op So what do you think? Is this an improvement? Regards, Peter From peter.levart at gmail.com Mon Jun 11 12:39:38 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 11 Jun 2018 14:39:38 +0200 Subject: BiCollector Message-ID: Hi, Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into? groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. I created a little utility Collector implementation that serves the purpose quite well: /** ?* A {@link Collector} implementation taking two delegate Collector(s) and producing result composed ?* of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. ?* ?* @param the type of elements collected ?* @param the type of 1st delegate collector collected result ?* @param tye type of 2nd delegate collector collected result ?*/ public class BiCollector implements Collector, Map.Entry> { ??? private final Collector keyCollector; ??? private final Collector valCollector; ??? @SuppressWarnings("unchecked") ??? public BiCollector(Collector keyCollector, Collector valCollector) { ??????? this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); ??????? this.valCollector = (Collector) Objects.requireNonNull(valCollector); ??? } ??? @Override ??? public Supplier> supplier() { ??????? Supplier keySupplier = keyCollector.supplier(); ??????? Supplier valSupplier = valCollector.supplier(); ??????? return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); ??? } ??? @Override ??? public BiConsumer, T> accumulator() { ??????? BiConsumer keyAccumulator = keyCollector.accumulator(); ??????? BiConsumer valAccumulator = valCollector.accumulator(); ??????? return (accumulation, t) -> { ??????????? keyAccumulator.accept(accumulation.getKey(), t); ??????????? valAccumulator.accept(accumulation.getValue(), t); ??????? }; ??? } ??? @Override ??? public BinaryOperator> combiner() { ??????? BinaryOperator keyCombiner = keyCollector.combiner(); ??????? BinaryOperator valCombiner = valCollector.combiner(); ??????? return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( ??????????? keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), ??????????? valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) ??????? ); ??? } ??? @Override ??? public Function, Map.Entry> finisher() { ??????? Function keyFinisher = keyCollector.finisher(); ??????? Function valFinisher = valCollector.finisher(); ??????? return accumulation -> new AbstractMap.SimpleImmutableEntry<>( ??????????? keyFinisher.apply(accumulation.getKey()), ??????????? valFinisher.apply(accumulation.getValue()) ??????? ); ??? } ??? @Override ??? public Set characteristics() { ??????? EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); ??????? intersection.retainAll(valCollector.characteristics()); ??????? return intersection; ??? } } Do you think this class is general enough to be part of standard Collectors repertoire? For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: ??????? Map map = ... ??????? Map.Entry, List> keys_values = ??????????? map.entrySet() ?????????????? .stream() ?????????????? .collect( ?????????????????? toBoth( ?????????????????????? mapping(Map.Entry::getKey, toList()), ?????????????????????? mapping(Map.Entry::getValue, toList()) ?????????????????? ) ?????????????? ); ??????? Map.Entry, Long> histogram_count = ??????????? ThreadLocalRandom ??????????????? .current() ??????????????? .ints(100, 0, 10) ??????????????? .boxed() ??????????????? .collect( ??????????????????? toBoth( ??????????????????????? groupingBy(Function.identity(), counting()), ??????????????????????? counting() ??????????????????? ) ??????????????? ); Regards, Peter From thomas.stuefe at gmail.com Mon Jun 11 13:13:23 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 11 Jun 2018 15:13:23 +0200 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 Message-ID: Hi, may I please have reviews for this small cleanup. Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ This change removes some native functions from FOS/FIS which are orphaned and unused since "JDK-8187631: Refactor FileDescriptor close implementation". Tests on jdk-submit came back clean save one strange test error on windows which I cannot reproduce locally. I am investigating that one. Thank you, Thomas From bob.vandette at oracle.com Mon Jun 11 14:12:29 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 10:12:29 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Message-ID: > On Jun 11, 2018, at 4:32 AM, David Holmes wrote: > > Sorry Bob I haven't had a chance to look at this detail. > > For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. > > For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? The comments reads: * Returns the length of the operating system time slice, in * milliseconds, for processes within the Isolation Group. The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. Isn?t this sufficient? Thanks, Bob. > > David > > On 8/06/2018 3:43 AM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> Mandy Chung has reviewed this change. >> I?ve run Mach5 hotspot and core lib tests. >> I?ve reviewed the tests which were written by Harsha Wardhana >> I filed a CSR for the command line change and it?s now approved and closed. >> Thanks, >> Bob. >>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>> >>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>> information. See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> Provider: cgroupv1 >>> Effective CPU Count: 4 >>> CPU Period: 100000 >>> CPU Quota: -1 >>> CPU Shares: -1 >>> List of Processors, 4 total: >>> 4 5 6 7 >>> List of Effective Processors, 4 total: >>> 4 5 6 7 >>> List of Memory Nodes, 2 total: >>> 0 1 >>> List of Available Memory Nodes, 2 total: >>> 0 1 >>> CPUSet Memory Pressure Enabled: false >>> Memory Limit: 256.00M >>> Memory Soft Limit: Unlimited >>> Memory & Swap Limit: 512.00M >>> Kernel Memory Limit: Unlimited >>> TCP Memory Limit: Unlimited >>> Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> From roger.riggs at oracle.com Mon Jun 11 15:55:32 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 11 Jun 2018 11:55:32 -0400 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: References: Message-ID: <4528b331-12d0-931d-ed09-6a89ecc95055@oracle.com> On 6/11/18 9:13 AM, Thomas St?fe wrote: > Hi, > > may I please have reviews for this small cleanup. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ > > This change removes some native functions from FOS/FIS which are > orphaned and unused since "JDK-8187631: Refactor FileDescriptor close > implementation". > > Tests on jdk-submit came back clean save one strange test error on > windows which I cannot reproduce locally. I am investigating that one. > > Thank you, Thomas From roger.riggs at oracle.com Mon Jun 11 15:56:18 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 11 Jun 2018 11:56:18 -0400 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: References: Message-ID: <7f4ab1a7-2fbd-7731-dd14-78227b82e207@oracle.com> Hi Thomas, Looks fine.? (And also passed the tests I ran). Thanks, Roger On 6/11/18 9:13 AM, Thomas St?fe wrote: > Hi, > > may I please have reviews for this small cleanup. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ > > This change removes some native functions from FOS/FIS which are > orphaned and unused since "JDK-8187631: Refactor FileDescriptor close > implementation". > > Tests on jdk-submit came back clean save one strange test error on > windows which I cannot reproduce locally. I am investigating that one. > > Thank you, Thomas From thomas.stuefe at gmail.com Mon Jun 11 16:00:19 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 11 Jun 2018 18:00:19 +0200 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: <7f4ab1a7-2fbd-7731-dd14-78227b82e207@oracle.com> References: <7f4ab1a7-2fbd-7731-dd14-78227b82e207@oracle.com> Message-ID: Thank you Roger! I re-ran submission tests and all went well this time. Must have been a windows specific fluke. ..Thomas On Mon, Jun 11, 2018 at 5:56 PM, Roger Riggs wrote: > Hi Thomas, > > Looks fine. (And also passed the tests I ran). > > Thanks, Roger > > > On 6/11/18 9:13 AM, Thomas St?fe wrote: >> >> Hi, >> >> may I please have reviews for this small cleanup. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ >> >> This change removes some native functions from FOS/FIS which are >> orphaned and unused since "JDK-8187631: Refactor FileDescriptor close >> implementation". >> >> Tests on jdk-submit came back clean save one strange test error on >> windows which I cannot reproduce locally. I am investigating that one. >> >> Thank you, Thomas > > From paul.sandoz at oracle.com Mon Jun 11 17:10:53 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 11 Jun 2018 10:10:53 -0700 Subject: BiCollector In-Reply-To: References: Message-ID: Hi Peter, I like it and can see it being useful, thanks for sharing. I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. What are you thoughts on this? FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? Paul. > On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: > > Hi, > > Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. > > I created a little utility Collector implementation that serves the purpose quite well: > > /** > * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed > * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. > * > * @param the type of elements collected > * @param the type of 1st delegate collector collected result > * @param tye type of 2nd delegate collector collected result > */ > public class BiCollector implements Collector, Map.Entry> { > private final Collector keyCollector; > private final Collector valCollector; > > @SuppressWarnings("unchecked") > public BiCollector(Collector keyCollector, Collector valCollector) { > this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); > this.valCollector = (Collector) Objects.requireNonNull(valCollector); > } > > @Override > public Supplier> supplier() { > Supplier keySupplier = keyCollector.supplier(); > Supplier valSupplier = valCollector.supplier(); > return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > } > > @Override > public BiConsumer, T> accumulator() { > BiConsumer keyAccumulator = keyCollector.accumulator(); > BiConsumer valAccumulator = valCollector.accumulator(); > return (accumulation, t) -> { > keyAccumulator.accept(accumulation.getKey(), t); > valAccumulator.accept(accumulation.getValue(), t); > }; > } > > @Override > public BinaryOperator> combiner() { > BinaryOperator keyCombiner = keyCollector.combiner(); > BinaryOperator valCombiner = valCollector.combiner(); > return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( > keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), > valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) > ); > } > > @Override > public Function, Map.Entry> finisher() { > Function keyFinisher = keyCollector.finisher(); > Function valFinisher = valCollector.finisher(); > return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > keyFinisher.apply(accumulation.getKey()), > valFinisher.apply(accumulation.getValue()) > ); > } > > @Override > public Set characteristics() { > EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); > intersection.retainAll(valCollector.characteristics()); > return intersection; > } > } > > > Do you think this class is general enough to be part of standard Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: > > Map map = ... > > Map.Entry, List> keys_values = > map.entrySet() > .stream() > .collect( > toBoth( > mapping(Map.Entry::getKey, toList()), > mapping(Map.Entry::getValue, toList()) > ) > ); > > > Map.Entry, Long> histogram_count = > ThreadLocalRandom > .current() > .ints(100, 0, 10) > .boxed() > .collect( > toBoth( > groupingBy(Function.identity(), counting()), > counting() > ) > ); > > > Regards, Peter > From maurizio.cimadamore at oracle.com Mon Jun 11 17:32:46 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 11 Jun 2018 18:32:46 +0100 Subject: BiCollector In-Reply-To: References: Message-ID: Note also that this has some overlappings with Collectors.partitioningBy - which currently wraps results into a Map, where O is the desired collector output type. Without commenting on the feasibility of its inclusion in the JDK (Paul rules here :-)), I note that BiStream would obviously allow this functionality to be exposed in a more user friendly way. Cheers Maurizio On 11/06/18 13:39, Peter Levart wrote: > Hi, > > Have you ever wanted to perform a collection of the same Stream into > two different targets using two Collectors? Say you wanted to collect > Map.Entry elements into two parallel lists, each of them containing > keys and values respectively. Or you wanted to collect elements into? > groups by some key, but also count them at the same time? Currently > this is not possible to do with a single Stream. You have to create > two identical streams, so you end up passing Supplier to other > methods instead of bare Stream. > > I created a little utility Collector implementation that serves the > purpose quite well: > > /** > ?* A {@link Collector} implementation taking two delegate Collector(s) > and producing result composed > ?* of two results produced by delegating collectors, wrapped in {@link > Map.Entry} object. > ?* > ?* @param the type of elements collected > ?* @param the type of 1st delegate collector collected result > ?* @param tye type of 2nd delegate collector collected result > ?*/ > public class BiCollector implements Collector Map.Entry, Map.Entry> { > ??? private final Collector keyCollector; > ??? private final Collector valCollector; > > ??? @SuppressWarnings("unchecked") > ??? public BiCollector(Collector keyCollector, Collector ?, V> valCollector) { > ??????? this.keyCollector = (Collector) > Objects.requireNonNull(keyCollector); > ??????? this.valCollector = (Collector) > Objects.requireNonNull(valCollector); > ??? } > > ??? @Override > ??? public Supplier> supplier() { > ??????? Supplier keySupplier = keyCollector.supplier(); > ??????? Supplier valSupplier = valCollector.supplier(); > ??????? return () -> new > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > ??? } > > ??? @Override > ??? public BiConsumer, T> accumulator() { > ??????? BiConsumer keyAccumulator = > keyCollector.accumulator(); > ??????? BiConsumer valAccumulator = > valCollector.accumulator(); > ??????? return (accumulation, t) -> { > ??????????? keyAccumulator.accept(accumulation.getKey(), t); > ??????????? valAccumulator.accept(accumulation.getValue(), t); > ??????? }; > ??? } > > ??? @Override > ??? public BinaryOperator> combiner() { > ??????? BinaryOperator keyCombiner = keyCollector.combiner(); > ??????? BinaryOperator valCombiner = valCollector.combiner(); > ??????? return (accumulation1, accumulation2) -> new > AbstractMap.SimpleImmutableEntry<>( > ??????????? keyCombiner.apply(accumulation1.getKey(), > accumulation2.getKey()), > ??????????? valCombiner.apply(accumulation1.getValue(), > accumulation2.getValue()) > ??????? ); > ??? } > > ??? @Override > ??? public Function, Map.Entry> > finisher() { > ??????? Function keyFinisher = keyCollector.finisher(); > ??????? Function valFinisher = valCollector.finisher(); > ??????? return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > ??????????? keyFinisher.apply(accumulation.getKey()), > ??????????? valFinisher.apply(accumulation.getValue()) > ??????? ); > ??? } > > ??? @Override > ??? public Set characteristics() { > ??????? EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > ??????? intersection.retainAll(valCollector.characteristics()); > ??????? return intersection; > ??? } > } > > > Do you think this class is general enough to be part of standard > Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector > coll1, Collector coll2), bi-collection could then be coded simply as: > > ??????? Map map = ... > > ??????? Map.Entry, List> keys_values = > ??????????? map.entrySet() > ?????????????? .stream() > ?????????????? .collect( > ?????????????????? toBoth( > ?????????????????????? mapping(Map.Entry::getKey, toList()), > ?????????????????????? mapping(Map.Entry::getValue, toList()) > ?????????????????? ) > ?????????????? ); > > > ??????? Map.Entry, Long> histogram_count = > ??????????? ThreadLocalRandom > ??????????????? .current() > ??????????????? .ints(100, 0, 10) > ??????????????? .boxed() > ??????????????? .collect( > ??????????????????? toBoth( > ??????????????????????? groupingBy(Function.identity(), counting()), > ??????????????????????? counting() > ??????????????????? ) > ??????????????? ); > > > Regards, Peter > From kirk.pepperdine at gmail.com Mon Jun 11 17:45:33 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Mon, 11 Jun 2018 20:45:33 +0300 Subject: BiCollector In-Reply-To: References: Message-ID: <5E1659A3-9309-4C27-A4F6-37686BA62DF1@gmail.com> Hi, I?ve created one of my own and I?d happily toss it for a standard implementation. ? Kirk > On Jun 11, 2018, at 8:10 PM, Paul Sandoz wrote: > > Hi Peter, > > I like it and can see it being useful, thanks for sharing. > > I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. > > What are you thoughts on this? > > FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? > > Paul. > > > > >> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >> >> Hi, >> >> Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. >> >> I created a little utility Collector implementation that serves the purpose quite well: >> >> /** >> * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed >> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. >> * >> * @param the type of elements collected >> * @param the type of 1st delegate collector collected result >> * @param tye type of 2nd delegate collector collected result >> */ >> public class BiCollector implements Collector, Map.Entry> { >> private final Collector keyCollector; >> private final Collector valCollector; >> >> @SuppressWarnings("unchecked") >> public BiCollector(Collector keyCollector, Collector valCollector) { >> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >> } >> >> @Override >> public Supplier> supplier() { >> Supplier keySupplier = keyCollector.supplier(); >> Supplier valSupplier = valCollector.supplier(); >> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >> } >> >> @Override >> public BiConsumer, T> accumulator() { >> BiConsumer keyAccumulator = keyCollector.accumulator(); >> BiConsumer valAccumulator = valCollector.accumulator(); >> return (accumulation, t) -> { >> keyAccumulator.accept(accumulation.getKey(), t); >> valAccumulator.accept(accumulation.getValue(), t); >> }; >> } >> >> @Override >> public BinaryOperator> combiner() { >> BinaryOperator keyCombiner = keyCollector.combiner(); >> BinaryOperator valCombiner = valCollector.combiner(); >> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >> ); >> } >> >> @Override >> public Function, Map.Entry> finisher() { >> Function keyFinisher = keyCollector.finisher(); >> Function valFinisher = valCollector.finisher(); >> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> keyFinisher.apply(accumulation.getKey()), >> valFinisher.apply(accumulation.getValue()) >> ); >> } >> >> @Override >> public Set characteristics() { >> EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); >> intersection.retainAll(valCollector.characteristics()); >> return intersection; >> } >> } >> >> >> Do you think this class is general enough to be part of standard Collectors repertoire? >> >> For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: >> >> Map map = ... >> >> Map.Entry, List> keys_values = >> map.entrySet() >> .stream() >> .collect( >> toBoth( >> mapping(Map.Entry::getKey, toList()), >> mapping(Map.Entry::getValue, toList()) >> ) >> ); >> >> >> Map.Entry, Long> histogram_count = >> ThreadLocalRandom >> .current() >> .ints(100, 0, 10) >> .boxed() >> .collect( >> toBoth( >> groupingBy(Function.identity(), counting()), >> counting() >> ) >> ); >> >> >> Regards, Peter >> > From mandy.chung at oracle.com Mon Jun 11 17:48:21 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 10:48:21 -0700 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <3C0AFC7B-AA3C-4DE1-AF9A-0DB556A1AEAB@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> <3C0AFC7B-AA3C-4DE1-AF9A-0DB556A1AEAB@oracle.com> Message-ID: <9711e185-356e-9e87-b991-b34f6ccbbc12@oracle.com> On 6/10/18 10:12 PM, Chris Yin wrote: > Hi, Mandy > > Thanks lot for your suggestions, update webrev as below and comment > inline, thanks > > http://cr.openjdk.java.net/~xyin/8201528/webrev.02/ Looks good. Thanks for the update and this is much easier to understand the test cases. Mandy From bob.vandette at oracle.com Mon Jun 11 18:28:05 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 14:28:05 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <54435D5C-5E6B-4174-8344-984CDDF5D46A@oracle.com> > On Jun 11, 2018, at 4:07 AM, Robbin Ehn wrote: > > Hi Bob, > > On 06/07/2018 07:43 PM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > Seems okay. > > Metrics.java > "Returns the length of the operating system time slice" > > Note that is is only true if you are using a batch scheduler. > Otherwise this period may be split on multiple 'time slices?. This is a cgroup metric which uses CFS not the OS time slice. 136 /** 137 * Returns the length of the operating system time slice, in 138 * milliseconds, for processes within the Isolation Group. > > In printSystemMetrics there is no units, maybe intentional? I?ll add ms for the quote/period output. The memory metrics do have units. > > Do we have support now in mach5 for docker jtreg, or do we still run these separate? > > You can ship it. Thanks! Bob. > > Thanks for fixing, and super thanks for fixing the bug in PlainRead also! > > /Robbin > >> Mandy Chung has reviewed this change. >> I?ve run Mach5 hotspot and core lib tests. >> I?ve reviewed the tests which were written by Harsha Wardhana >> I filed a CSR for the command line change and it?s now approved and closed. >> Thanks, >> Bob. >>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>> >>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>> information. See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> Provider: cgroupv1 >>> Effective CPU Count: 4 >>> CPU Period: 100000 >>> CPU Quota: -1 >>> CPU Shares: -1 >>> List of Processors, 4 total: >>> 4 5 6 7 >>> List of Effective Processors, 4 total: >>> 4 5 6 7 >>> List of Memory Nodes, 2 total: >>> 0 1 >>> List of Available Memory Nodes, 2 total: >>> 0 1 >>> CPUSet Memory Pressure Enabled: false >>> Memory Limit: 256.00M >>> Memory Soft Limit: Unlimited >>> Memory & Swap Limit: 512.00M >>> Kernel Memory Limit: Unlimited >>> TCP Memory Limit: Unlimited >>> Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> From peter.levart at gmail.com Mon Jun 11 18:28:36 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 11 Jun 2018 20:28:36 +0200 Subject: BiCollector In-Reply-To: References: Message-ID: Hi Paul, Can you point me to some BiStream code (if it is available publicly)? Thanks, Peter On 06/11/18 19:10, Paul Sandoz wrote: > Hi Peter, > > I like it and can see it being useful, thanks for sharing. > > I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. > > What are you thoughts on this? > > FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? > > Paul. > > > > >> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >> >> Hi, >> >> Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. >> >> I created a little utility Collector implementation that serves the purpose quite well: >> >> /** >> * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed >> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. >> * >> * @param the type of elements collected >> * @param the type of 1st delegate collector collected result >> * @param tye type of 2nd delegate collector collected result >> */ >> public class BiCollector implements Collector, Map.Entry> { >> private final Collector keyCollector; >> private final Collector valCollector; >> >> @SuppressWarnings("unchecked") >> public BiCollector(Collector keyCollector, Collector valCollector) { >> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >> } >> >> @Override >> public Supplier> supplier() { >> Supplier keySupplier = keyCollector.supplier(); >> Supplier valSupplier = valCollector.supplier(); >> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >> } >> >> @Override >> public BiConsumer, T> accumulator() { >> BiConsumer keyAccumulator = keyCollector.accumulator(); >> BiConsumer valAccumulator = valCollector.accumulator(); >> return (accumulation, t) -> { >> keyAccumulator.accept(accumulation.getKey(), t); >> valAccumulator.accept(accumulation.getValue(), t); >> }; >> } >> >> @Override >> public BinaryOperator> combiner() { >> BinaryOperator keyCombiner = keyCollector.combiner(); >> BinaryOperator valCombiner = valCollector.combiner(); >> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >> ); >> } >> >> @Override >> public Function, Map.Entry> finisher() { >> Function keyFinisher = keyCollector.finisher(); >> Function valFinisher = valCollector.finisher(); >> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> keyFinisher.apply(accumulation.getKey()), >> valFinisher.apply(accumulation.getValue()) >> ); >> } >> >> @Override >> public Set characteristics() { >> EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); >> intersection.retainAll(valCollector.characteristics()); >> return intersection; >> } >> } >> >> >> Do you think this class is general enough to be part of standard Collectors repertoire? >> >> For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: >> >> Map map = ... >> >> Map.Entry, List> keys_values = >> map.entrySet() >> .stream() >> .collect( >> toBoth( >> mapping(Map.Entry::getKey, toList()), >> mapping(Map.Entry::getValue, toList()) >> ) >> ); >> >> >> Map.Entry, Long> histogram_count = >> ThreadLocalRandom >> .current() >> .ints(100, 0, 10) >> .boxed() >> .collect( >> toBoth( >> groupingBy(Function.identity(), counting()), >> counting() >> ) >> ); >> >> >> Regards, Peter >> From forax at univ-mlv.fr Mon Jun 11 18:40:33 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 11 Jun 2018 20:40:33 +0200 (CEST) Subject: BiCollector In-Reply-To: References: Message-ID: <1111894492.377996.1528742433755.JavaMail.zimbra@u-pem.fr> Hi Peter, You may find something on lambda-dev, i remember that we discussed about BiStream, and as Paul said we decide to not include it because the performance were not great and it adds another axis to the API making it even harder to retrofitted it when we will introduce specialization. regards, R?mi ----- Mail original ----- > De: "Peter Levart" > ?: "Paul Sandoz" > Cc: "core-libs-dev" > Envoy?: Lundi 11 Juin 2018 20:28:36 > Objet: Re: BiCollector > Hi Paul, > > Can you point me to some BiStream code (if it is available publicly)? > > Thanks, Peter > > On 06/11/18 19:10, Paul Sandoz wrote: >> Hi Peter, >> >> I like it and can see it being useful, thanks for sharing. >> >> I am hesitating a little about it being in the JDK because there is the larger >> abstraction of a BiStream, where a similar form of collection would naturally >> fit (but perhaps without the intersection constraints for the >> characteristics?). We experimented a few times with BiStream and got quite far >> but decided pull back due to the lack of value types and specialized generics. >> So i dunno how this might turn out in the future and if your BiCollector fits >> nicely into such a future model. >> >> What are you thoughts on this? >> >> FWIW i would call it a ?splitting? or ?bisecting" collector e.g. >> ?s.collect(bisecting(?))? >> >> Paul. >> >> >> >> >>> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >>> >>> Hi, >>> >>> Have you ever wanted to perform a collection of the same Stream into two >>> different targets using two Collectors? Say you wanted to collect Map.Entry >>> elements into two parallel lists, each of them containing keys and values >>> respectively. Or you wanted to collect elements into groups by some key, but >>> also count them at the same time? Currently this is not possible to do with a >>> single Stream. You have to create two identical streams, so you end up passing >>> Supplier to other methods instead of bare Stream. >>> >>> I created a little utility Collector implementation that serves the purpose >>> quite well: >>> >>> /** >>> * A {@link Collector} implementation taking two delegate Collector(s) and >>> producing result composed >>> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} >>> object. >>> * >>> * @param the type of elements collected >>> * @param the type of 1st delegate collector collected result >>> * @param tye type of 2nd delegate collector collected result >>> */ >>> public class BiCollector implements Collector>> Object>, Map.Entry> { >>> private final Collector keyCollector; >>> private final Collector valCollector; >>> >>> @SuppressWarnings("unchecked") >>> public BiCollector(Collector keyCollector, Collector >>> valCollector) { >>> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >>> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >>> } >>> >>> @Override >>> public Supplier> supplier() { >>> Supplier keySupplier = keyCollector.supplier(); >>> Supplier valSupplier = valCollector.supplier(); >>> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), >>> valSupplier.get()); >>> } >>> >>> @Override >>> public BiConsumer, T> accumulator() { >>> BiConsumer keyAccumulator = keyCollector.accumulator(); >>> BiConsumer valAccumulator = valCollector.accumulator(); >>> return (accumulation, t) -> { >>> keyAccumulator.accept(accumulation.getKey(), t); >>> valAccumulator.accept(accumulation.getValue(), t); >>> }; >>> } >>> >>> @Override >>> public BinaryOperator> combiner() { >>> BinaryOperator keyCombiner = keyCollector.combiner(); >>> BinaryOperator valCombiner = valCollector.combiner(); >>> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >>> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >>> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >>> ); >>> } >>> >>> @Override >>> public Function, Map.Entry> finisher() { >>> Function keyFinisher = keyCollector.finisher(); >>> Function valFinisher = valCollector.finisher(); >>> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >>> keyFinisher.apply(accumulation.getKey()), >>> valFinisher.apply(accumulation.getValue()) >>> ); >>> } >>> >>> @Override >>> public Set characteristics() { >>> EnumSet intersection = >>> EnumSet.copyOf(keyCollector.characteristics()); >>> intersection.retainAll(valCollector.characteristics()); >>> return intersection; >>> } >>> } >>> >>> >>> Do you think this class is general enough to be part of standard Collectors >>> repertoire? >>> >>> For example, accessed via factory method Collectors.toBoth(Collector coll1, >>> Collector coll2), bi-collection could then be coded simply as: >>> >>> Map map = ... >>> >>> Map.Entry, List> keys_values = >>> map.entrySet() >>> .stream() >>> .collect( >>> toBoth( >>> mapping(Map.Entry::getKey, toList()), >>> mapping(Map.Entry::getValue, toList()) >>> ) >>> ); >>> >>> >>> Map.Entry, Long> histogram_count = >>> ThreadLocalRandom >>> .current() >>> .ints(100, 0, 10) >>> .boxed() >>> .collect( >>> toBoth( >>> groupingBy(Function.identity(), counting()), >>> counting() >>> ) >>> ); >>> >>> >>> Regards, Peter From david.holmes at oracle.com Mon Jun 11 21:21:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 07:21:23 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Message-ID: <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> On 12/06/2018 12:12 AM, Bob Vandette wrote: > >> On Jun 11, 2018, at 4:32 AM, David Holmes > > wrote: >> >> Sorry Bob I haven't had a chance to look at this detail. >> >> For the Java code ... methods that return arrays should return >> zero-length arrays when something is not available rather than null. > > All methods do return zero length arrays except I missed the > getPerCpuUsage. ?I?ll fix that one and correct the javadoc. There are a few more too: 231 * @return An array of available CPUs or null if metric is not available. 232 * 233 */ 234 public int[] getCpuSetCpus(); 242 * @return An array of available and online CPUs or null if the metric 243 * is not available. 244 * 245 */ 246 public int[] getEffectiveCpuSetCpus(); 256 * @return An array of available memory nodes or null if metric is not available. 257 * 258 */ 259 public int[] getCpuSetMems(); 267 * @return An array of available and online nodes or null if the metric 268 * is not available. 269 * 270 */ 271 public int[] getEffectiveCpuSetMems(); > >> >> For getCpuPeriod() the term "operating system time slice" can be >> misconstrued as being related to the scheduler timeslice that may, or >> may not, exist, depending on the scheduler and scheduling policy etc. >> This "timeslice" is something specific to cgroups - no? > > The comments reads: > > * Returns the length of the operating system time slice, in > * milliseconds, for processes within the Isolation Group. > > The comment does infer that it?s process and cgroup (Isolation group) > specific and not the generic os timeslice. > Isn?t this sufficient? The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: * Returns the length of the scheduling period, in * milliseconds, for processes within the Isolation Group. then for quota: * Returns the total available run-time allowed, in milliseconds, * during each scheduling period for all tasks in the Isolation Group. Thanks, David > > Thanks, > Bob. >> >> David >> >> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> I?ve run Mach5 hotspot and core lib tests. >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> I filed a CSR for the command line change and it?s now approved and >>> closed. >>> Thanks, >>> Bob. >>>> On May 30, 2018, at 3:45 PM, Bob Vandette >>> > wrote: >>>> >>>> Please review the following RFE which adds an internal API, along >>>> with jtreg tests that provide >>>> access to Docker container configuration data and metrics. ?In >>>> addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX >>>> Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or host >>>> cgroup confguration >>>> information. ?See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> ???Provider: cgroupv1 >>>> ???Effective CPU Count: 4 >>>> ???CPU Period: 100000 >>>> ???CPU Quota: -1 >>>> ???CPU Shares: -1 >>>> ???List of Processors, 4 total: >>>> ???4 5 6 7 >>>> ???List of Effective Processors, 4 total: >>>> ???4 5 6 7 >>>> ???List of Memory Nodes, 2 total: >>>> ???0 1 >>>> ???List of Available Memory Nodes, 2 total: >>>> ???0 1 >>>> ???CPUSet Memory Pressure Enabled: false >>>> ???Memory Limit: 256.00M >>>> ???Memory Soft Limit: Unlimited >>>> ???Memory & Swap Limit: 512.00M >>>> ???Kernel Memory Limit: Unlimited >>>> ???TCP Memory Limit: Unlimited >>>> ???Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > From bob.vandette at oracle.com Mon Jun 11 23:30:04 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 19:30:04 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> Message-ID: > On Jun 11, 2018, at 5:21 PM, David Holmes wrote: > > On 12/06/2018 12:12 AM, Bob Vandette wrote: >>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>> >>> Sorry Bob I haven't had a chance to look at this detail. >>> >>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. > > There are a few more too: > Those are covered by the function that converts the string range. > 231 * @return An array of available CPUs or null if metric is not available. > 232 * > 233 */ > 234 public int[] getCpuSetCpus(); > > 242 * @return An array of available and online CPUs or null if the metric > 243 * is not available. > 244 * > 245 */ > 246 public int[] getEffectiveCpuSetCpus(); > > 256 * @return An array of available memory nodes or null if metric is not available. > 257 * > 258 */ > 259 public int[] getCpuSetMems(); > > 267 * @return An array of available and online nodes or null if the metric > 268 * is not available. > 269 * > 270 */ > 271 public int[] getEffectiveCpuSetMems(); >>> >>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >> The comments reads: >> * Returns the length of the operating system time slice, in >> * milliseconds, for processes within the Isolation Group. >> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >> Isn?t this sufficient? > > The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: > > * Returns the length of the scheduling period, in > * milliseconds, for processes within the Isolation Group. > > then for quota: > > * Returns the total available run-time allowed, in milliseconds, > * during each scheduling period for all tasks in the Isolation Group. > Ok. I?ll update the docs. Bob > Thanks, > David > >> Thanks, >> Bob. >>> >>> David >>> >>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> Mandy Chung has reviewed this change. >>>> I?ve run Mach5 hotspot and core lib tests. >>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>> I filed a CSR for the command line change and it?s now approved and closed. >>>> Thanks, >>>> Bob. >>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>> >>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>> information. See the sample output below: >>>>> >>>>> RFE: Container Metrics >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>> >>>>> WEBREV: >>>>> >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>> >>>>> >>>>> This commit will also include a fix for the following bug. >>>>> >>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>> >>>>> WEBREV: >>>>> >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>> >>>>> SAMPLE USAGE and OUTPUT: >>>>> >>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>> ./java -XshowSettings:system >>>>> Operating System Metrics: >>>>> Provider: cgroupv1 >>>>> Effective CPU Count: 4 >>>>> CPU Period: 100000 >>>>> CPU Quota: -1 >>>>> CPU Shares: -1 >>>>> List of Processors, 4 total: >>>>> 4 5 6 7 >>>>> List of Effective Processors, 4 total: >>>>> 4 5 6 7 >>>>> List of Memory Nodes, 2 total: >>>>> 0 1 >>>>> List of Available Memory Nodes, 2 total: >>>>> 0 1 >>>>> CPUSet Memory Pressure Enabled: false >>>>> Memory Limit: 256.00M >>>>> Memory Soft Limit: Unlimited >>>>> Memory & Swap Limit: 512.00M >>>>> Kernel Memory Limit: Unlimited >>>>> TCP Memory Limit: Unlimited >>>>> Out Of Memory Killer Enabled: true >>>>> >>>>> TEST RESULTS: >>>>> >>>>> testing runtime container APIs >>>>> Directory "JTwork" not found: creating >>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>> Passed: runtime/containers/docker/TestMisc.java >>>>> Test results: passed: 6 >>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>> >>>>> testing jdk.internal.platform APIs >>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>> Test results: passed: 4 >>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>> >>>>> testing -XshowSettings:system launcher option >>>>> Passed: tools/launcher/Settings.java >>>>> Test results: passed: 1 >>>>> >>>>> >>>>> Bob. >>>>> >>>>> From xu.y.yin at oracle.com Tue Jun 12 00:53:33 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 12 Jun 2018 08:53:33 +0800 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <9711e185-356e-9e87-b991-b34f6ccbbc12@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> <3C0AFC7B-AA3C-4DE1-AF9A-0DB556A1AEAB@oracle.com> <9711e185-356e-9e87-b991-b34f6ccbbc12@oracle.com> Message-ID: <6552B418-CF87-4DFE-A82A-75103B84316C@oracle.com> Thank you, Mandy Regards, Chris > On 12 Jun 2018, at 1:48 AM, mandy chung wrote: > > > > On 6/10/18 10:12 PM, Chris Yin wrote: >> Hi, Mandy >> Thanks lot for your suggestions, update webrev as below and comment inline, thanks >> http://cr.openjdk.java.net/~xyin/8201528/webrev.02/ > > > Looks good. Thanks for the update and this is much easier to understand the test cases. > > Mandy From david.holmes at oracle.com Tue Jun 12 05:12:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 15:12:54 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> Message-ID: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> On 12/06/2018 9:30 AM, Bob Vandette wrote: > > >> On Jun 11, 2018, at 5:21 PM, David Holmes wrote: >> >> On 12/06/2018 12:12 AM, Bob Vandette wrote: >>>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>>> >>>> Sorry Bob I haven't had a chance to look at this detail. >>>> >>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >> >> There are a few more too: >> > > Those are covered by the function that converts the string range. ??? I have no idea what you mean. Java API design style is to return zero-length arrays rather than null. [Ref: Effective Java First Edition, Item 27]. Cheers, David ----- >> 231 * @return An array of available CPUs or null if metric is not available. >> 232 * >> 233 */ >> 234 public int[] getCpuSetCpus(); >> >> 242 * @return An array of available and online CPUs or null if the metric >> 243 * is not available. >> 244 * >> 245 */ >> 246 public int[] getEffectiveCpuSetCpus(); >> >> 256 * @return An array of available memory nodes or null if metric is not available. >> 257 * >> 258 */ >> 259 public int[] getCpuSetMems(); >> >> 267 * @return An array of available and online nodes or null if the metric >> 268 * is not available. >> 269 * >> 270 */ >> 271 public int[] getEffectiveCpuSetMems(); >>>> >>>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >>> The comments reads: >>> * Returns the length of the operating system time slice, in >>> * milliseconds, for processes within the Isolation Group. >>> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >>> Isn?t this sufficient? >> >> The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: >> >> * Returns the length of the scheduling period, in >> * milliseconds, for processes within the Isolation Group. >> >> then for quota: >> >> * Returns the total available run-time allowed, in milliseconds, >> * during each scheduling period for all tasks in the Isolation Group. >> > > Ok. I?ll update the docs. > Bob > >> Thanks, >> David >> >>> Thanks, >>> Bob. >>>> >>>> David >>>> >>>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>> Mandy Chung has reviewed this change. >>>>> I?ve run Mach5 hotspot and core lib tests. >>>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>>> I filed a CSR for the command line change and it?s now approved and closed. >>>>> Thanks, >>>>> Bob. >>>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>>> >>>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>>> information. See the sample output below: >>>>>> >>>>>> RFE: Container Metrics >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>>> >>>>>> WEBREV: >>>>>> >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>> >>>>>> >>>>>> This commit will also include a fix for the following bug. >>>>>> >>>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>>> >>>>>> WEBREV: >>>>>> >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>>> >>>>>> SAMPLE USAGE and OUTPUT: >>>>>> >>>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>>> ./java -XshowSettings:system >>>>>> Operating System Metrics: >>>>>> Provider: cgroupv1 >>>>>> Effective CPU Count: 4 >>>>>> CPU Period: 100000 >>>>>> CPU Quota: -1 >>>>>> CPU Shares: -1 >>>>>> List of Processors, 4 total: >>>>>> 4 5 6 7 >>>>>> List of Effective Processors, 4 total: >>>>>> 4 5 6 7 >>>>>> List of Memory Nodes, 2 total: >>>>>> 0 1 >>>>>> List of Available Memory Nodes, 2 total: >>>>>> 0 1 >>>>>> CPUSet Memory Pressure Enabled: false >>>>>> Memory Limit: 256.00M >>>>>> Memory Soft Limit: Unlimited >>>>>> Memory & Swap Limit: 512.00M >>>>>> Kernel Memory Limit: Unlimited >>>>>> TCP Memory Limit: Unlimited >>>>>> Out Of Memory Killer Enabled: true >>>>>> >>>>>> TEST RESULTS: >>>>>> >>>>>> testing runtime container APIs >>>>>> Directory "JTwork" not found: creating >>>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>>> Passed: runtime/containers/docker/TestMisc.java >>>>>> Test results: passed: 6 >>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>> >>>>>> testing jdk.internal.platform APIs >>>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>>> Test results: passed: 4 >>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>> >>>>>> testing -XshowSettings:system launcher option >>>>>> Passed: tools/launcher/Settings.java >>>>>> Test results: passed: 1 >>>>>> >>>>>> >>>>>> Bob. >>>>>> >>>>>> > From david.holmes at oracle.com Tue Jun 12 05:16:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 15:16:13 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> Message-ID: Here is one further minor update from the CSR discussions: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html Thanks, David On 25/05/2018 3:52 PM, David Holmes wrote: > Here are the further minor updates so far in response to all the review > comments. > > Incremental corelibs webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ > > Full corelibs webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ > > Change summary: > > src/java.base/share/classes/jdk/internal/reflect/Reflection.java > - remove inaccurate pseudo-assertion comment > > test/jdk/java/lang/reflect/Nestmates/SampleNest.java > - code cleanup: <> operator > > test/jdk/java/lang/reflect/Nestmates/TestReflectionAPI.java > - code cleanup: streamify duplicate removals > > test/jdk/java/lang/invoke/PrivateInterfaceCall.java > - use consistent @bug number > > Thanks, > David > > On 22/05/2018 8:15 PM, David Holmes wrote: >> Here are the updates so far in response to all the review comments. >> >> Incremental webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2-incr/ >> >> >> Full webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2/ >> >> Change summary: >> >> src/java.base/share/classes/java/lang/Class.java >> - getNesthost: >> ?? - change "any error" -> "any linkage error" as runtime errors will >> propagate. [This needs ratifying by EG] >> ?? - add clarification that primitive and array classes are not >> explicitly members of any nest and so form singleton nests >> ?? - add clarification that all nestmates are in the same package >> ?? - re-word @return text to exclude the "royal 'we'" >> - fix javadoc cross references >> >> --- >> >> Moved reflection API tests from >> test/hotspot/jtreg/runtime/Nestmates/reflectionAPI/ to >> test/jdk/java/lang/reflect/Nestmates/ >> >> --- >> >> java/lang/reflect/Nestmates/TestReflectionAPI.java >> >> Run tests twice to show that failure reasons remain the same. >> >> --- >> >> test/jdk/jdk/lambda/vm/InterfaceAccessFlagsTest.java >> >> Disable test via annotation rather than commenting out. >> >> --- >> >> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >> >> - Fix indent for nestmate access check. >> - Remove unnecessary local variable >> >> --- >> >> src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >> >> - Replace myassert with proper assert >> >> --- >> >> Thanks, >> David >> >> On 15/05/2018 10:52 AM, David Holmes wrote: >>> This review is being spread across four groups: langtools, core-libs, >>> hotspot and serviceability. This is the specific review thread for >>> core-libs - webrev: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v1/ >>> >>> See below for full details - including annotated full webrev guiding >>> the review. >>> >>> The intent is to have JEP-181 targeted and integrated by the end of >>> this month. >>> >>> Thanks, >>> David >>> ----- >>> >>> The nestmates project (JEP-181) introduces new classfile attributes >>> to identify classes and interfaces in the same nest, so that the VM >>> can perform access control based on those attributes and so allow >>> direct private access between nestmates without requiring javac to >>> generate synthetic accessor methods. These access control changes >>> also extend to core reflection and the MethodHandle.Lookup contexts. >>> >>> Direct private calls between nestmates requires a more general >>> calling context than is permitted by invokespecial, and so the JVMS >>> is updated to allow, and javac updated to use, invokevirtual and >>> invokeinterface for private class and interface method calls >>> respectively. These changed semantics also extend to MethodHandle >>> findXXX operations. >>> >>> At this time we are only concerned with static nest definitions, >>> which map to a top-level class/interface as the nest-host and all its >>> nested types as nest-members. >>> >>> Please see the JEP for further details. >>> >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>> >>> All of the specification changes have been previously been worked out >>> by the Valhalla Project Expert Group, and the implementation reviewed >>> by the various contributors and discussed on the valhalla-dev mailing >>> list. >>> >>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, >>> Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, >>> Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>> >>> Master webrev of all changes: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>> >>> Annotated master webrev index: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>> >>> Performance: this is expected to be performance neutral in a general >>> sense. Benchmarking and performance runs are about to start. >>> >>> Testing Discussion: >>> ------------------ >>> >>> The testing for nestmates can be broken into four main groups: >>> >>> -? New tests specifically related to nestmates and currently in the >>> runtime/Nestmates directory >>> >>> - New tests to complement existing tests by adding in testcases not >>> previously expressible. >>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>> use of invokespecial for private interface methods and performing >>> receiver typechecks, so we add >>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>> invokeinterface. >>> >>> -? New JVM TI tests to verify the spec changes related to nest >>> attributes. >>> >>> -? Existing tests significantly affected by the nestmates changes, >>> primarily: >>> ??? -? runtime/SelectionResolution >>> >>> ??? In most cases the nestmate changes makes certain invocations that >>> were illegal, legal (e.g. not requiring invokespecial to invoke >>> private interface methods; allowing access to private members via >>> reflection/Methodhandles that were previously not allowed). >>> >>> - Existing tests incidentally affected by the nestmate changes >>> >>> ?? This includes tests of things utilising class >>> redefinition/retransformation to alter nested types but which >>> unintentionally alter nest relationships (which is not permitted). >>> >>> There are still a number of tests problem-listed with issues filed >>> against them to have them adapted to work with nestmates. Some of >>> these are intended to be addressed in the short-term, while some >>> (such as the runtime/SelectionResolution test changes) may not >>> eventuate. >>> >>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>> >>> There is also further test work still to be completed (the JNI and >>> JDI invocation tests): >>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>> which will continue in parallel with the main RFR. >>> >>> Pre-integration Testing: >>> ??- General: >>> ???? - Mach5: hs/jdk tier1,2 >>> ???? - Mach5: hs-nightly (tiers 1 -3) >>> ??- Targetted >>> ??? - nashorn (for asm changes) >>> ??? - hotspot: runtime/* >>> ?????????????? serviceability/* >>> ?????????????? compiler/* >>> ?????????????? vmTestbase/* >>> ??? - jdk: java/lang/invoke/* >>> ?????????? java/lang/reflect/* >>> ?????????? java/lang/instrument/* >>> ?????????? java/lang/Class/* >>> ?????????? java/lang/management/* >>> ?? - langtools: tools/javac >>> ??????????????? tools/javap >>> From srinivas.dama at oracle.com Tue Jun 12 05:21:47 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 11 Jun 2018 22:21:47 -0700 (PDT) Subject: RFR: 8196993: Resolve disabled warnings for libunpack In-Reply-To: References: Message-ID: Gentle reminder . May I have review for the below changeset. Regards, Srinivas -----Original Message----- From: Srinivas Dama Sent: Friday, June 08, 2018 6:33 PM To: Core-Libs-Dev Subject: RFR: 8196993: Resolve disabled warnings for libunpack Hi, Please review http://cr.openjdk.java.net/~sdama/8196993/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196993 Regards, Srinivas From srinivas.dama at oracle.com Tue Jun 12 05:23:14 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 11 Jun 2018 22:23:14 -0700 (PDT) Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) In-Reply-To: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> References: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> Message-ID: <6b5d7d2f-d735-4b62-b1c6-8dd09f22e542@default> Gentle reminder. May I have review for the below changeset. Regards, Srinivas -----Original Message----- From: Srinivas Dama Sent: Friday, June 08, 2018 11:47 AM To: core-libs-dev at openjdk.java.net Subject: Re: RFR: 8196988 (Resolve disabled warnings for libjimage) Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: Modified earlier fix to handle warning messages specific to libjimage only. Regards, Srinivas ----- Original Message ----- From: srinivas.dama at oracle.com To: core-libs-dev at openjdk.java.net Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but these warnings are disabled earlier.I have enabled these warnings and fixed in sources. Regards, Srinivas From mandy.chung at oracle.com Tue Jun 12 05:31:05 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 22:31:05 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: On 6/11/18 10:12 PM, David Holmes wrote: >>>>> >>>>> For the Java code ... methods that return arrays should return >>>>> zero-length arrays when something is not available rather than null. >>>> All methods do return zero length arrays except I missed the >>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>> >>> There are a few more too: >>> >> >> Those are covered by the function that converts the string range. > > ??? I have no idea what you mean. I think the methods returning an array calls Subsystem::StringRangeToIntArray which returns an empty array. 171 public static int[] StringRangeToIntArray(String range) { 172 int[] ints = new int[0]; 173 174 if (range == null) return ints; Mandy From mandy.chung at oracle.com Tue Jun 12 05:31:59 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 22:31:59 -0700 Subject: RFR: 8196993: Resolve disabled warnings for libunpack In-Reply-To: References: Message-ID: <3188419b-a919-eea6-eb54-186d660db025@oracle.com> Looks fine. Mandy On 6/11/18 10:21 PM, Srinivas Dama wrote: > Gentle reminder . > > May I have review for the below changeset. > > Regards, > Srinivas > -----Original Message----- > From: Srinivas Dama > Sent: Friday, June 08, 2018 6:33 PM > To: Core-Libs-Dev > Subject: RFR: 8196993: Resolve disabled warnings for libunpack > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196993/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196993 > > Regards, > Srinivas > From mandy.chung at oracle.com Tue Jun 12 05:38:10 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 22:38:10 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> Message-ID: <82fdd320-f5fb-2c34-5bb8-b72f6718fef6@oracle.com> On 6/11/18 10:16 PM, David Holmes wrote: > Here is one further minor update from the CSR discussions: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html "This implementation" is fine, as used in many @implNote. Any reason why it has to be changed to "reference implementation"? Mandy From david.holmes at oracle.com Tue Jun 12 05:43:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 15:43:13 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> On 12/06/2018 3:31 PM, mandy chung wrote: > On 6/11/18 10:12 PM, David Holmes wrote: >>>>>> >>>>>> For the Java code ... methods that return arrays should return >>>>>> zero-length arrays when something is not available rather than null. >>>>> All methods do return zero length arrays except I missed the >>>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>>> >>>> There are a few more too: >>>> >>> >>> Those are covered by the function that converts the string range. >> >> ??? I have no idea what you mean. > > > I think the methods returning an array calls > Subsystem::StringRangeToIntArray which returns an empty array. > > ?171???? public static int[] StringRangeToIntArray(String range) { > ?172???????? int[] ints = new int[0]; > ?173 > ?174???????? if (range == null) return ints; I'm commenting on the specification of the Metrics interface: http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html not any implementation. Cheers, David > > Mandy From mandy.chung at oracle.com Tue Jun 12 06:00:57 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 23:00:57 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> Message-ID: <413aa29c-1a7c-8cd2-ca27-201dc6b53432@oracle.com> On 6/11/18 10:43 PM, David Holmes wrote: > On 12/06/2018 3:31 PM, mandy chung wrote: >> On 6/11/18 10:12 PM, David Holmes wrote: >>>>>>> >>>>>>> For the Java code ... methods that return arrays should return >>>>>>> zero-length arrays when something is not available rather than null. >>>>>> All methods do return zero length arrays except I missed the >>>>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>>>> >>>>> There are a few more too: >>>>> >>>> >>>> Those are covered by the function that converts the string range. >>> >>> ??? I have no idea what you mean. >> >> >> I think the methods returning an array calls >> Subsystem::StringRangeToIntArray which returns an empty array. >> >> ??171???? public static int[] StringRangeToIntArray(String range) { >> ??172???????? int[] ints = new int[0]; >> ??173 >> ??174???????? if (range == null) return ints; > > I'm commenting on the specification of the Metrics interface: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html > not any implementation. The implementation returns empty array, which is good. Yes the javadoc should be updated. Mandy From bhamaram at in.ibm.com Tue Jun 12 09:00:55 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Tue, 12 Jun 2018 09:00:55 +0000 Subject: [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to stdcs In-Reply-To: <5AF5C0FC.5020504@oracle.com> References: <5AF5C0FC.5020504@oracle.com>, <9cecb38d51627bfb0e11afceb0e0ecb4@linux.vnet.ibm.com>, Message-ID: Thank you for review. This is not new charset. This codepage already exist in openJDK and current fix is to move it to standard charset list (only for AIX) as it is default charset for locale zh_TW.IBM-eucTW. This enabling jdk11 on locale zh_TW.IBM-eucTW. May I request someone to create JIRA bug for this issue and sponsor the fix? - Bhaktavatsal Reddy. -----"core-libs-dev" wrote: ----- To: core-libs-dev at openjdk.java.net From: Xueming Shen Sent by: "core-libs-dev" Date: 05/11/2018 09:43PM Subject: Re: [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to stdcs SimpleEUCEncoder is something leftover when I reimplemented the charsets to be map-based-built-during-build-time. I would recommend to leave them as Bhaktavatsal suggested for now. Ideally any new charsets added to the platform needs to be based on the new model (open/make/data/charetmapping), instead of hard-coding the map into the source code. Thanks! Sherman On 5/10/18, 8:20 PM, Bhaktavatsal R Maram wrote: > Hi Ichiroh, > > Yes, moving SimpleEUCEncoder.java to sun.nio.cs package would leave build tools untouched. But, I think that for platforms other than AIX, SimpleEUCEncoder.java in java.base module would not add any value. > > - Bhaktavatsal > > -----Ichiroh Takiguchi wrote: ----- > To: Bhaktavatsal R Maram/India/IBM at IBMIN > From: Ichiroh Takiguchi > Date: 05/11/2018 06:31AM > Cc: Java Core Libs > Subject: Re: [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to stdcs > > Hello Bhaktavatsal. > > I think we should move > from > src/jdk.charsets/share/classes/sun/nio/cs/ext/SimpleEUCEncoder.java > to > src/java.base/share/classes/sun/nio/cs/SimpleEUCEncoder.java > instead of moving to > > src/jdk.charsets/share/classes/sun/nio/cs/ext/SimpleEUCEncoder.java.template > > Then we don't need to touch following files > make/jdk/src/classes/build/tools/charsetmapping/Main.java > make/jdk/src/classes/build/tools/charsetmapping/SRC.java > > I think SimpleEUCEncoder.java is not large file, > so it can be moved to java.base module. > > What do you think ? > > On 2018-05-10 22:35, Bhaktavatsal R Maram wrote: >> Hi All, >> >> In bug 8201540 (Extend the set of supported charsets in java.base on >> AIX)[1] we moved default charsets that are available in OpenJDK to >> java.base module except to IBM964. Charset IBM964 is bit tricky. Its >> Encoder is extended from sun.nio.cs.ext.SimpleEUCEncoder. So, >> SimpleEUCEncoder also should be moved along with IBM964 to sun.nio.cs. >> However, there is another charset IBM33722 which should always in >> sun.nio.cs.ext whose encoder also extends SimpleEUCEncoder. >> >> Hence, the challenge is that sun.nio.cs.ext.IBM33722 should import >> SimpleEUCEncoder from either sun.nio.cs or sun.nio.cs.ext based on the >> location of IBM964. >> >> On AIX, following should be the package locations >> sun.nio.cs.IBM964 >> sun.nio.cs.SimpleEUCEncoder >> sun.nio.cs.ext.IBM33722 imports sun.nio.cs.SimpleEUCEncoder >> >> On Linux (and other platforms), following should be the package >> locations >> sun.nio.cs.ext.IBM964 >> sun.nio.cs.ext.SimpleEUCEncoder >> sun.nio.cs.ext.IBM33722 imports sun.nio.cs.ext.SimpleEUCEncoder >> >> To achieve this, I've changed all source files involved to templates. >> And, I also modified build tools to check if IBM964 is moved to stdcs >> or not and handled import statement of SimpleEUCEncoder in >> sun.nio.cs.ext.IBM33722 accordingly. >> >> Patch can be found here [2]. Please review. After jira bug is opened, >> I'll host it again with bug ID. >> >> With this change, java -version is successful on locale >> zh_TW.IBM-eucTW. >> >> -bash-4.4$ LANG=zh_TW.IBM-eucTW >> ~/openjdk/jdk11/build/aix-ppc64-normal-server-release/jdk/bin/java >> -version >> openjdk version "11-internal" 2018-09-25 >> OpenJDK Runtime Environment (build 11-internal+0-adhoc.bhamaram.jdk11) >> OpenJDK 64-Bit Server VM (build 11-internal+0-adhoc.bhamaram.jdk11, >> mixed mode) >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8201540 >> [2] http://cr.openjdk.java.net/~aleonard/AIXIBM964/webrev.01/ >> >> Thanks, >> Bhaktavatsal Reddy From vivek.theeyarath at oracle.com Tue Jun 12 09:34:22 2018 From: vivek.theeyarath at oracle.com (Vivek Theeyarath) Date: Tue, 12 Jun 2018 02:34:22 -0700 (PDT) Subject: RFR: 8202216: (bf) Add Buffer mismatch() Message-ID: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> Hi All, Please review fix for https://bugs.openjdk.java.net/browse/JDK-8202216 Webrev: http://cr.openjdk.java.net/~vtheeyarath/8202216/webrev.00/ CSR : https://bugs.openjdk.java.net/browse/JDK-8204852 Regards Vivek From peter.levart at gmail.com Tue Jun 12 10:20:52 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 12 Jun 2018 12:20:52 +0200 Subject: RFR: 8202216: (bf) Add Buffer mismatch() In-Reply-To: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> References: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> Message-ID: <18401a8c-8315-97f3-6a22-619976c534ce@gmail.com> Hi, On 06/12/2018 11:34 AM, Vivek Theeyarath wrote: > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8202216 > > > > Webrev: http://cr.openjdk.java.net/~vtheeyarath/8202216/webrev.00/ > > CSR : https://bugs.openjdk.java.net/browse/JDK-8204852 > > > > Regards > > Vivek > > This looks good as is, but would it make sense for this new method to be defined as abstract method on Buffer? Like for example: public abstract class Buffer { ??? public abstract int mismatch(B that); ... public abstract class ByteBuffer ??? extends Buffer ??? implements Comparable { ??? @Override ??? public int mismatch(ByteBuffer that) { ... public abstract class CharBuffer ??? extends Buffer ??? implements Comparable { ??? @Override ??? public int mismatch(CharBuffer that) { ... Regards, Peter From vivek.theeyarath at oracle.com Tue Jun 12 10:45:05 2018 From: vivek.theeyarath at oracle.com (Vivek Theeyarath) Date: Tue, 12 Jun 2018 03:45:05 -0700 (PDT) Subject: RFR: 8204342: CLONE - methods in java.time's TCKZoneRules OpenJDK test miss @Test annotation Message-ID: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> Hi All, Please review fix for https://bugs.openjdk.java.net/browse/JDK-8204342 . The jtreg test runs fine with the changes. http://cr.openjdk.java.net/~vtheeyarath/8204342/webrev.00/ Regards Vivek From bob.vandette at oracle.com Tue Jun 12 10:56:33 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 12 Jun 2018 06:56:33 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: > On Jun 12, 2018, at 1:12 AM, David Holmes wrote: > > On 12/06/2018 9:30 AM, Bob Vandette wrote: >>> On Jun 11, 2018, at 5:21 PM, David Holmes wrote: >>> >>> On 12/06/2018 12:12 AM, Bob Vandette wrote: >>>>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>>>> >>>>> Sorry Bob I haven't had a chance to look at this detail. >>>>> >>>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >>> >>> There are a few more too: >>> >> Those are covered by the function that converts the string range. > > ??? I have no idea what you mean. > > Java API design style is to return zero-length arrays rather than null. [Ref: Effective Java First Edition, Item 27]. Look at line 174 in this file. http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java.html Bob > > Cheers, > David > ----- > >>> 231 * @return An array of available CPUs or null if metric is not available. >>> 232 * >>> 233 */ >>> 234 public int[] getCpuSetCpus(); >>> >>> 242 * @return An array of available and online CPUs or null if the metric >>> 243 * is not available. >>> 244 * >>> 245 */ >>> 246 public int[] getEffectiveCpuSetCpus(); >>> >>> 256 * @return An array of available memory nodes or null if metric is not available. >>> 257 * >>> 258 */ >>> 259 public int[] getCpuSetMems(); >>> >>> 267 * @return An array of available and online nodes or null if the metric >>> 268 * is not available. >>> 269 * >>> 270 */ >>> 271 public int[] getEffectiveCpuSetMems(); >>>>> >>>>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >>>> The comments reads: >>>> * Returns the length of the operating system time slice, in >>>> * milliseconds, for processes within the Isolation Group. >>>> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >>>> Isn?t this sufficient? >>> >>> The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: >>> >>> * Returns the length of the scheduling period, in >>> * milliseconds, for processes within the Isolation Group. >>> >>> then for quota: >>> >>> * Returns the total available run-time allowed, in milliseconds, >>> * during each scheduling period for all tasks in the Isolation Group. >>> >> Ok. I?ll update the docs. >> Bob >>> Thanks, >>> David >>> >>>> Thanks, >>>> Bob. >>>>> >>>>> David >>>>> >>>>>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>>>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>> Mandy Chung has reviewed this change. >>>>>> I?ve run Mach5 hotspot and core lib tests. >>>>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>>>> I filed a CSR for the command line change and it?s now approved and closed. >>>>>> Thanks, >>>>>> Bob. >>>>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>>>> >>>>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>>>> information. See the sample output below: >>>>>>> >>>>>>> RFE: Container Metrics >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>>>> >>>>>>> WEBREV: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>>> >>>>>>> >>>>>>> This commit will also include a fix for the following bug. >>>>>>> >>>>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>>>> >>>>>>> WEBREV: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>>>> >>>>>>> SAMPLE USAGE and OUTPUT: >>>>>>> >>>>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>>>> ./java -XshowSettings:system >>>>>>> Operating System Metrics: >>>>>>> Provider: cgroupv1 >>>>>>> Effective CPU Count: 4 >>>>>>> CPU Period: 100000 >>>>>>> CPU Quota: -1 >>>>>>> CPU Shares: -1 >>>>>>> List of Processors, 4 total: >>>>>>> 4 5 6 7 >>>>>>> List of Effective Processors, 4 total: >>>>>>> 4 5 6 7 >>>>>>> List of Memory Nodes, 2 total: >>>>>>> 0 1 >>>>>>> List of Available Memory Nodes, 2 total: >>>>>>> 0 1 >>>>>>> CPUSet Memory Pressure Enabled: false >>>>>>> Memory Limit: 256.00M >>>>>>> Memory Soft Limit: Unlimited >>>>>>> Memory & Swap Limit: 512.00M >>>>>>> Kernel Memory Limit: Unlimited >>>>>>> TCP Memory Limit: Unlimited >>>>>>> Out Of Memory Killer Enabled: true >>>>>>> >>>>>>> TEST RESULTS: >>>>>>> >>>>>>> testing runtime container APIs >>>>>>> Directory "JTwork" not found: creating >>>>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>>>> Passed: runtime/containers/docker/TestMisc.java >>>>>>> Test results: passed: 6 >>>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>>> >>>>>>> testing jdk.internal.platform APIs >>>>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>>>> Test results: passed: 4 >>>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>>> >>>>>>> testing -XshowSettings:system launcher option >>>>>>> Passed: tools/launcher/Settings.java >>>>>>> Test results: passed: 1 >>>>>>> >>>>>>> >>>>>>> Bob. >>>>>>> >>>>>>> From bob.vandette at oracle.com Tue Jun 12 10:59:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 12 Jun 2018 06:59:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> Message-ID: > On Jun 12, 2018, at 1:43 AM, David Holmes wrote: > >> On 12/06/2018 3:31 PM, mandy chung wrote: >> On 6/11/18 10:12 PM, David Holmes wrote: >>>>>>> >>>>>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>>>>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >>>>> >>>>> There are a few more too: >>>>> >>>> >>>> Those are covered by the function that converts the string range. >>> >>> ??? I have no idea what you mean. >> I think the methods returning an array calls Subsystem::StringRangeToIntArray which returns an empty array. >> 171 public static int[] StringRangeToIntArray(String range) { >> 172 int[] ints = new int[0]; >> 173 >> 174 if (range == null) return ints; > > I'm commenting on the specification of the Metrics interface: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html > > not any implementation. Oh. I previously mentioned that I needed to correct the javadoc comments. I had corrected the implementation but hadn?t fixed the spec. Bob. > > Cheers, > David > >> Mandy From srinivas.dama at oracle.com Tue Jun 12 12:46:51 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Tue, 12 Jun 2018 05:46:51 -0700 (PDT) Subject: RFR: 8204861 : fix for 8196993 has broken the build on linux Message-ID: <71aad77c-0a6a-43f8-95af-5d777d451197@default> Hi, My earlier fix for https://bugs.openjdk.java.net/browse/JDK-8196993 has broken the build on linux. So I am reverting my fix for now. Bug : https://bugs.openjdk.java.net/browse/JDK-8204861 webrev: http://cr.openjdk.java.net/~sdama/8204861/webrev.00/ Regards, Srinivas From stefan.karlsson at oracle.com Tue Jun 12 12:52:21 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 12 Jun 2018 14:52:21 +0200 Subject: RFR: 8204861 : fix for 8196993 has broken the build on linux In-Reply-To: <71aad77c-0a6a-43f8-95af-5d777d451197@default> References: <71aad77c-0a6a-43f8-95af-5d777d451197@default> Message-ID: Looks good to me. Thanks for fixing this. StefanK On 2018-06-12 14:46, Srinivas Dama wrote: > Hi, > > My earlier fix for https://bugs.openjdk.java.net/browse/JDK-8196993 has broken the build on linux. > So I am reverting my fix for now. > > Bug : https://bugs.openjdk.java.net/browse/JDK-8204861 > webrev: http://cr.openjdk.java.net/~sdama/8204861/webrev.00/ > > Regards, > Srinivas > From shade at redhat.com Tue Jun 12 12:56:34 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 12 Jun 2018 14:56:34 +0200 Subject: RFR: 8204861 : fix for 8196993 has broken the build on linux In-Reply-To: <71aad77c-0a6a-43f8-95af-5d777d451197@default> References: <71aad77c-0a6a-43f8-95af-5d777d451197@default> Message-ID: On 06/12/2018 02:46 PM, Srinivas Dama wrote: > Hi, > > My earlier fix for https://bugs.openjdk.java.net/browse/JDK-8196993 has broken the build on linux. > So I am reverting my fix for now. > > Bug : https://bugs.openjdk.java.net/browse/JDK-8204861 > webrev: http://cr.openjdk.java.net/~sdama/8204861/webrev.00/ Please do it. Reversal looks good. Thanks, -Aleksey From roger.riggs at oracle.com Tue Jun 12 13:24:05 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 12 Jun 2018 09:24:05 -0400 Subject: RFR: 8204342: CLONE - methods in java.time's TCKZoneRules OpenJDK test miss @Test annotation In-Reply-To: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> References: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> Message-ID: Hi Vivek, Look fine, Roger On 6/12/18 6:45 AM, Vivek Theeyarath wrote: > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8204342 . The jtreg test runs fine with the changes. > > > > http://cr.openjdk.java.net/~vtheeyarath/8204342/webrev.00/ > > > > Regards > > Vivek From Roger.Riggs at Oracle.com Tue Jun 12 14:17:17 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 12 Jun 2018 10:17:17 -0400 Subject: RFR 8197930: JNI exception pending in initializeEncoding of jni_util.c Message-ID: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> Please review a cleanup to not ignore errors from GetMethodID and GetFieldID while initializing the system encoding. Webrev: ?? http://cr.openjdk.java.net/~rriggs/webrev-jnicheck-8197930/ issue: ?? https://bugs.openjdk.java.net/browse/JDK-8197930 Thanks, Roger From naoto.sato at oracle.com Tue Jun 12 15:06:52 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 12 Jun 2018 08:06:52 -0700 Subject: RFR: 8204342: CLONE - methods in java.time's TCKZoneRules OpenJDK test miss @Test annotation In-Reply-To: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> References: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> Message-ID: <9f7af9b8-93d1-9144-fb80-e7e20da05320@oracle.com> +1 Naoto On 6/12/18 3:45 AM, Vivek Theeyarath wrote: > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8204342 . The jtreg test runs fine with the changes. > > > > http://cr.openjdk.java.net/~vtheeyarath/8204342/webrev.00/ > > > > Regards > > Vivek > From paul.sandoz at oracle.com Tue Jun 12 15:17:32 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 08:17:32 -0700 Subject: BiCollector In-Reply-To: References: Message-ID: <055EBC54-C8D2-40A5-98B7-B9341F7403B9@oracle.com> I don?t have anything publicly available. I?ll see if i can dig something out and send it to you directly. Paul. > On Jun 11, 2018, at 11:28 AM, Peter Levart wrote: > > Hi Paul, > > Can you point me to some BiStream code (if it is available publicly)? > > Thanks, Peter > > On 06/11/18 19:10, Paul Sandoz wrote: >> Hi Peter, >> >> I like it and can see it being useful, thanks for sharing. >> >> I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. >> >> What are you thoughts on this? >> >> FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? >> >> Paul. From paul.sandoz at oracle.com Tue Jun 12 15:39:33 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 08:39:33 -0700 Subject: RFR: 8202216: (bf) Add Buffer mismatch() In-Reply-To: <18401a8c-8315-97f3-6a22-619976c534ce@gmail.com> References: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> <18401a8c-8315-97f3-6a22-619976c534ce@gmail.com> Message-ID: <29E31126-00CA-431C-92B5-B089971E9998@oracle.com> Hi Peter, Buffer is not currently a generic class and making it so likely would cause some source compatibility issues (e.g. lots of raw type warnings/errors). Paul. > On Jun 12, 2018, at 3:20 AM, Peter Levart wrote: > > Hi, > > On 06/12/2018 11:34 AM, Vivek Theeyarath wrote: >> Hi All, >> >> Please review fix for https://bugs.openjdk.java.net/browse/JDK-8202216 >> >> >> Webrev: http://cr.openjdk.java.net/~vtheeyarath/8202216/webrev.00/ >> >> CSR : https://bugs.openjdk.java.net/browse/JDK-8204852 >> >> >> Regards >> >> Vivek >> >> > > This looks good as is, but would it make sense for this new method to be defined as abstract method on Buffer? Like for example: > > public abstract class Buffer { > public abstract int mismatch(B that); > ... > > > public abstract class ByteBuffer > extends Buffer > implements Comparable > { > @Override > public int mismatch(ByteBuffer that) { > ... > > > public abstract class CharBuffer > extends Buffer > implements Comparable > { > @Override > public int mismatch(CharBuffer that) { > ... > > > > Regards, Peter > From thomas.stuefe at gmail.com Tue Jun 12 15:47:13 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 12 Jun 2018 17:47:13 +0200 Subject: RFR 8197930: JNI exception pending in initializeEncoding of jni_util.c In-Reply-To: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> References: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> Message-ID: Looks good. Thanks, Thomas On Tue, Jun 12, 2018 at 4:17 PM, Roger Riggs wrote: > Please review a cleanup to not ignore errors from GetMethodID and GetFieldID > while initializing the system encoding. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-jnicheck-8197930/ > issue: > https://bugs.openjdk.java.net/browse/JDK-8197930 > > Thanks, Roger > From paul.sandoz at oracle.com Tue Jun 12 16:11:21 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 09:11:21 -0700 Subject: RFR: 8202216: (bf) Add Buffer mismatch() In-Reply-To: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> References: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> Message-ID: <5DB14AEA-D496-4BC6-9327-B03E7FD1F163@oracle.com> +1 Paul. > On Jun 12, 2018, at 2:34 AM, Vivek Theeyarath wrote: > > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8202216 > > > > Webrev: http://cr.openjdk.java.net/~vtheeyarath/8202216/webrev.00/ > > CSR : https://bugs.openjdk.java.net/browse/JDK-8204852 > > > > Regards > > Vivek > > From mandy.chung at oracle.com Tue Jun 12 16:30:59 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 12 Jun 2018 09:30:59 -0700 Subject: RFR 8197930: JNI exception pending in initializeEncoding of jni_util.c In-Reply-To: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> References: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> Message-ID: <4bf29fcb-d39c-3dc6-a762-62bb11736447@oracle.com> Looks good. Mandy On 6/12/18 7:17 AM, Roger Riggs wrote: > Please review a cleanup to not ignore errors from GetMethodID and > GetFieldID while initializing the system encoding. > > Webrev: > ?? http://cr.openjdk.java.net/~rriggs/webrev-jnicheck-8197930/ > issue: > ?? https://bugs.openjdk.java.net/browse/JDK-8197930 > > Thanks, Roger > From paul.sandoz at oracle.com Tue Jun 12 16:38:51 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 09:38:51 -0700 Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) In-Reply-To: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> References: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> Message-ID: <3AB1E03A-F47E-48E6-8C9B-DBC698ABA2A1@oracle.com> Looks ok. Including Jim in case he is not listening in? I am guessing the comment selectively disables the warning in this code (rather than using a diagnostic pragma). Thanks, Paul. > On Jun 7, 2018, at 11:17 PM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > > Note: Modified earlier fix to handle warning messages specific to > libjimage only. > > Regards, > Srinivas > > ----- Original Message ----- > From: srinivas.dama at oracle.com > To: core-libs-dev at openjdk.java.net > Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but > these warnings are disabled earlier.I have enabled these warnings and fixed in sources. > > Regards, > Srinivas From mandy.chung at oracle.com Tue Jun 12 16:44:21 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 12 Jun 2018 09:44:21 -0700 Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) In-Reply-To: <6b5d7d2f-d735-4b62-b1c6-8dd09f22e542@default> References: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> <6b5d7d2f-d735-4b62-b1c6-8dd09f22e542@default> Message-ID: <818bdf09-1c95-3672-3381-b47fe5ea44fc@oracle.com> Have you verified the build on all platforms (release and debug)? Mandy On 6/11/18 10:23 PM, Srinivas Dama wrote: > Gentle reminder. > > May I have review for the below changeset. > > Regards, > Srinivas > -----Original Message----- > From: Srinivas Dama > Sent: Friday, June 08, 2018 11:47 AM > To: core-libs-dev at openjdk.java.net > Subject: Re: RFR: 8196988 (Resolve disabled warnings for libjimage) > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > > Note: Modified earlier fix to handle warning messages specific to > libjimage only. > > Regards, > Srinivas > > ----- Original Message ----- > From: srinivas.dama at oracle.com > To: core-libs-dev at openjdk.java.net > Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but > these warnings are disabled earlier.I have enabled these warnings and fixed in sources. > > Regards, > Srinivas > From huizhe.wang at oracle.com Tue Jun 12 16:52:09 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 12 Jun 2018 09:52:09 -0700 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> Message-ID: <5B1FFA39.9000704@oracle.com> Hi all, It's been a while since the 1st round of reviews. Thank you all for the valuable comments and suggestions! Note that Roger's last response went to nio-dev, but not core-libs-dev, I've therefore attached it below. CSR [2]: the CSR is now approved. Note the write method has been changed to writeString. Impl [3]: for performance reason and the different requirement from byte-string conversion for malformed/unmappable character handing, the implementation uses a specific method separate from the existing ones. Both Sherman and Alan preferred specific method than adding parameters to the APIs that might be error prone. Thanks Sherman for the code! JMH benchmark tests showed the new read method performs better than an operation that reads the bytes and convert to string. The gains start to be significant even at quite reasonable sizes (< 10K). [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ Please review. Thanks, Joe On 5/15/18, 7:51 AM, Roger Riggs wrote: > Hi Joe, et. al. > > My $.02 on line separators: > > - We should avoid clever tricks trying to solve problems that are > infrequent such as cross OS file line operations. > Most files will be read/written on a single system so the line > separators will be as expected. > > - Have/use APIs that split lines consistently accepting both line > separators so developer code can be agnostic to line separators. > aka BufferedReader.readLine for developers that are processing the > contents *as lines*. > Those other methods already exist; if there are any gaps in line > oriented processing that's a separate task. > > - These new file methods are defined to handle Charset > encoding/decoding and buffering. > Since there are other methods to deal with files as lines these > methods should not look for or break to lines. > > - Performance: adding code to look for line characters will slow it > down and in most cases would have no impact > since the line endings will match the system. > > - The strings passed to writeString (CharSequence) should have been > constructed using the proper line separators. > There are any number of methods that insert the os specific line > terminator and its easy to write File.separator as needed. > > As for the method naming: > > I'd prefer "writeString" as a familiar term that isn't stretched too > far by making the argument be a CharSequence. > its fine to call a CharSequence a string (with a lower case s). > > $.02, Roger > > > On 4/27/18 6:33 PM, Joe Wang wrote: >> >> >> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>> ----- Mail original ----- >>>> De: "Joe Wang" >>>> ?: "Remi Forax" , "Alan Bateman" >>>> >>>> Cc: "nio-dev" , "core-libs-dev" >>>> >>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>> reading/writing a string from/to a file >>>> Hi R?mi, Alan, >>> Hi Joe, >>> >>>> I'm not sure we'd want to replace system dependent line separators >>>> with >>>> '\n'. There are cases as you described where the replacement makes >>>> sense. However, there are other cases too. For example, the >>>> purpose is >>>> to read, edit a text file and then write it back. If this is on >>>> Windows, >>>> and if line separators are replaced with '\n', that would cause the >>>> resulting file not formatted properly in for example Notepad. There >>>> are >>>> cases where users write text files on Windows using Java, and only >>>> found >>>> the lines were not separated in Notepad. >>> I agree that why the counterpart of readString() that write the >>> string should inserts the platform's line separator. >>> BTW, i just found that those methods are not named writeString, or >>> writeCharSequence, i think they should. >> >> While readString() does not modify the original content (e.g. by >> replacing the platform's line separator with '\n'), write(String) >> won't either, by adding extra characters such as the line separator. >> >> I would think interfaces shall read along with the parameters. >> readString(Path) == read as a String from the specified Path (one >> could argue for readToString, readAsString, but we generally omit the >> preps) >> write(Path, CharSequence) == write the CharSequence to the file, >> since CharSequence is already in the method signature as a parameter, >> we probably don't want to add that to the name, otherwise it would >> read like repeating the word CharSequence. >> >> It is in a similar situation as write(Path, Iterable) where it was >> defined as writeLines(Path, Iterable). >> >>> >>>> Files.write(Path, Iterable) is also specified to terminate each line >>>> with the platform's line separator. If readString does the >>>> replacement, >>>> it would be inconsistent. >>> >>> Anyway, if we look for consistency the methods writeCharSequence >>> should transform the \n in the CharSequence to the platform's line >>> separator. >>> >>> Files.write(Path, Iterable) is is not a counterpart of readString(), >>> it's consistent with Files.lines() or Files.readLines() (or >>> BufferedReader.readLine()) that all suppress the line separators. >>> Anyway, Files.write(path, readString(path)::line) will be consistent >>> if you replace the line separators or not because String.line() >>> suppresses the line separators. >> >> readString pairs with write(String), therefore it's more like >> Files.write(path, readString(path)) than readString(path)::line. The >> use case: >> >> String s = Files.read(path); >> Files.write(path, s.replace("/config/location", "/new/location")); >> >> would then work as expected. >> >> These two methods are one-off (open/read/write/close file) operation. >> write(String) therefore is not intended for adding/appending multiple >> Strings from other operations such as String.line(). If an app needs >> to put the result of String.line() or any other processes into a file >> using this method, it needs to take care of adding the line separator >> itself when necessary. "when necessary" would be a judgement call by >> the developer. >> >> That said, if there's a consensus on the idea of terminating the >> string with a line separator, then readString method would need to >> strip it off to go with the write method. >> >>> >>>> I would think readString behaves like readAllBytes, that would >>>> simply read all content faithfully. >>> readString does an interpolation due to the Charset so it's not the >>> real content, again, the idea is that developers should not have to >>> care about the platform's line separators (they have more important >>> things to think). >>> >>> The other solution is to just not introduce those new methods, after >>> all Files.lines().collect(Collectors.joining("\n")) already does the >>> job, no ? >> >> While there are many ways to do it, none is as straight-forward. >> "Read (entire) file to String"/"Write String to file" are popular >> requests from users. Read to string -> do some String manipulation -> >> write it back is such a simple use case, it really needs to be very >> easy to do as illustrated in the above code snippet. >> >> Cheers, >> Joe >> >>> >>>> Best, >>>> Joe >>> regards, >>> R?mi >>> >>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>> Hi Alan, >>>>> >>>>> People do not read the documentation. >>>>> So adding something in the documentation about when a method >>>>> should be used or >>>>> not is never a solution. >>>>> >>>>> Here the user want a String and provides a charset so you have no >>>>> way but to >>>>> decode the content to substitute the line separator. >>>>> >>>>> cheers, >>>>> R?mi >>>>> >>>>> ----- Mail original ----- >>>>>> De: "Alan Bateman" >>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>> >>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>> >>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>> reading/writing a string from/to a file >>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>> I think that having a readString that includes OS dependent line >>>>>>> separators is a >>>>>>> mistake, >>>>>>> Java does a great job to try to shield the developer from the >>>>>>> kind of things >>>>>>> that makes programs behave differently on different platforms. >>>>>>> >>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>> people will do a call >>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>> hard way. >>>>>>> >>>>>>> raw string literal does the same substitution for the same reason. >>>>>>> >>>>>> Yes, there are several discussion points around this and somewhat >>>>>> timely >>>>>> with multi-string support. >>>>>> >>>>>> One thing that I think Joe will need in this API is some note to >>>>>> make it >>>>>> clearer what the intended usage is (as I think the intention is >>>>>> simple >>>>>> cases with mostly single lines of text). >>>>>> >>>>>> -Alan. >> > From paul.sandoz at oracle.com Tue Jun 12 17:12:39 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 10:12:39 -0700 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> Message-ID: <382F6C6B-D097-4DF2-A8AB-FBCB60C5D3F5@oracle.com> Hi Jonathan, > On Jun 8, 2018, at 3:59 AM, Jonathan Halliday wrote: > > > Hi Paul > > Looks like we're all on the same page regarding the basic approach of using a small API and making the critical bits intrinsic. We perhaps have some way to go on exactly what that API looks like in terms of the classes and methods, but iterating on it by discussion of a JEP seems like the best way forward. The important thing from my perspective is that so far nobody has come forward with a use case that is not covered by the proposed primitives. So it's a small API, but not too small. Yes, a smallish API we can iterate on. > > As far a tweaks go, we have considered making the low level primitive method / intrinsic just a flushCacheline(base_address), since the arithmetic and loop for writing flush(from,to) in terms of that low level op is something that can be optimized fine by the JIT already. Though that does mean exposing the cache line size to the Java layer, whereas currently it's only visible in the C code. > That?s ok. The simpler the intrinsics while relying on Java code + JIT would be my generally preferred pattern. > > My own background and focus is transactions systems, so I'm more about the speed and fault tolerance than the capacity, but I can see long vs. int being of interest to our Infinispan data grid team and likewise for e.g. Oracle Coherence or databases like Cassandra. OTOH it's not uncommon to prefer moderately sized files and shard over them, which sidesteps the issue. > Ok, which is conveniently how developers currently work around the issue of mapping large files :-) > Utility code to assist with fine-grained memory management within the buffer/file may be more useful than support for really large buffers, since they tend to be used with some form of internal block/heap structure anyhow, rather than to hold very large objects. Providing that may be the role of a 3rd party pure Java library like PCJ though, rather than something we want in the JDK itself at this early stage. The researcher in me is kinda interested in how much of the memory allocation and GC code can be re-purposed here though... > > What's the intended timeline on long buffer indexing at present? Unsure, but it's probably something we want to solve soonish. > My feeling is a pmem API JEP is probably targeting around JDK 13, but we're flexible on that. Note that the JEP process can be started before then and JEPs are not targeted to a release until ready, if its ready sooner great! otherwise later. Keeping such a JEP focused on the mapping/flushing of BBs for NVM would be my recommendation rather than expanding its scope. > We may also want to look at related enhancements like unmapping buffers. I think those pieces are sufficient decoupled that they won't be dependencies for the pmem API though, unlike other factors such as the availability of test hardware. > That?s tricky! We have been through many discussions over the years on how to achieve this without much resolution. Andrew Haley came up with an interesting solution which IIRC requires the deallocating/unmapping thread to effectively reach safe point and wait for all other threads to pass through a check point. Project Panama is looking at the explicit scoping of resources, perhaps also resources that are thread confined or owned. My sense is Project Panama will eventually push strongly on this area and that?s where we should focus our efforts. Thanks, Paul. From Roger.Riggs at Oracle.com Tue Jun 12 17:37:56 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 12 Jun 2018 13:37:56 -0400 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B1FFA39.9000704@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> Message-ID: <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> Hi Joe, Looks good to me. Typo: StringCoding.java:1026 "unmappble"? (no new webrev needed) Regards, Roger On 6/12/2018 12:52 PM, Joe Wang wrote: > Hi all, > > It's been a while since the 1st round of reviews. Thank you all for > the valuable comments and suggestions!? Note that Roger's last > response went to nio-dev, but not core-libs-dev, I've therefore > attached it below. > > CSR [2]: the CSR is now approved. Note the write method has been > changed to writeString. > > Impl [3]: for performance reason and the different requirement from > byte-string conversion for malformed/unmappable character handing, the > implementation uses a specific method separate from the existing ones. > Both Sherman and Alan preferred specific method than adding parameters > to the APIs that might be error prone. Thanks Sherman for the code! > > JMH benchmark tests showed the new read method performs better than an > operation that reads the bytes and convert to string. The gains start > to be significant even at quite reasonable sizes (< 10K). > > > [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 > [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 > [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ > > Please review. > > Thanks, > Joe > > On 5/15/18, 7:51 AM, Roger Riggs wrote: >> Hi Joe, et. al. >> >> My $.02 on line separators: >> >> ?- We should avoid clever tricks trying to solve problems that are >> infrequent such as cross OS file line operations. >> ??? Most files will be read/written on a single system so the line >> separators will be as expected. >> >> ?- Have/use APIs that split lines consistently accepting both line >> separators so developer code can be agnostic to line separators. >> ??? aka BufferedReader.readLine for developers that are processing >> the contents *as lines*. >> ?? Those other methods already exist; if there are any gaps in line >> oriented processing that's a separate task. >> >> ?- These new file methods are defined to handle Charset >> encoding/decoding and buffering. >> ??? Since there are other methods to deal with files as lines these >> methods should not look for or break to lines. >> >> ?- Performance: adding code to look for line characters will slow it >> down and in most cases would have no impact >> ??? since the line endings will match the system. >> >> ?- The strings passed to writeString (CharSequence) should have been >> constructed using the proper line separators. >> ??? There are any number of methods that insert the os specific line >> terminator and its easy to write File.separator as needed. >> >> As for the method naming: >> >> I'd prefer "writeString" as a familiar term that isn't stretched too >> far by making the argument be a CharSequence. >> its fine to call a CharSequence a string (with a lower case s). >> >> $.02, Roger >> >> >> On 4/27/18 6:33 PM, Joe Wang wrote: >>> >>> >>> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>>> ----- Mail original ----- >>>>> De: "Joe Wang" >>>>> ?: "Remi Forax" , "Alan Bateman" >>>>> >>>>> Cc: "nio-dev" , "core-libs-dev" >>>>> >>>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>> reading/writing a string from/to a file >>>>> Hi R?mi, Alan, >>>> Hi Joe, >>>> >>>>> I'm not sure we'd want to replace system dependent line separators >>>>> with >>>>> '\n'. There are cases as you described where the replacement makes >>>>> sense. However,? there are other cases too. For example, the >>>>> purpose is >>>>> to read, edit a text file and then write it back. If this is on >>>>> Windows, >>>>> and if line separators are replaced with '\n', that would cause the >>>>> resulting file not formatted properly in for example Notepad. >>>>> There are >>>>> cases where users write text files on Windows using Java, and only >>>>> found >>>>> the lines were not separated in Notepad. >>>> I agree that why the counterpart of readString() that write the >>>> string should inserts the platform's line separator. >>>> BTW, i just found that those methods are not named writeString, or >>>> writeCharSequence, i think they should. >>> >>> While readString() does not modify the original content (e.g. by >>> replacing the platform's line separator with '\n'), write(String) >>> won't either, by adding extra characters such as the line separator. >>> >>> I would think interfaces shall read along with the parameters. >>> ??? readString(Path) == read as a String from the specified Path >>> (one could argue for readToString, readAsString, but we generally >>> omit the preps) >>> ??? write(Path, CharSequence) == write the CharSequence to the file, >>> since CharSequence is already in the method signature as a >>> parameter, we probably don't want to add that to the name, otherwise >>> it would read like repeating the word CharSequence. >>> >>> It is in a similar situation as write(Path, Iterable) where it was >>> defined as writeLines(Path, Iterable). >>> >>>> >>>>> Files.write(Path, Iterable) is also specified to terminate each line >>>>> with the platform's line separator. If readString does the >>>>> replacement, >>>>> it would be inconsistent. >>>> >>>> Anyway, if we look for consistency the methods writeCharSequence >>>> should transform the \n in the CharSequence to the platform's line >>>> separator. >>>> >>>> Files.write(Path, Iterable) is is not a counterpart of >>>> readString(), it's consistent with Files.lines() or >>>> Files.readLines() (or BufferedReader.readLine()) that all suppress >>>> the line separators. Anyway, Files.write(path, >>>> readString(path)::line) will be consistent if you replace the line >>>> separators or not because String.line() suppresses the line >>>> separators. >>> >>> readString pairs with write(String), therefore it's more like >>> Files.write(path, readString(path)) than readString(path)::line. The >>> use case: >>> >>> ??? String s = Files.read(path); >>> ??? Files.write(path, s.replace("/config/location", "/new/location")); >>> >>> would then work as expected. >>> >>> These two methods are one-off (open/read/write/close file) >>> operation. write(String) therefore is not intended for >>> adding/appending multiple Strings from other operations such as >>> String.line(). If an app needs to put the result of String.line() or >>> any other processes into a file using this method, it needs to take >>> care of adding the line separator itself when necessary. "when >>> necessary" would be a judgement call by the developer. >>> >>> That said, if there's a consensus on the idea of terminating the >>> string with a line separator, then readString method would need to >>> strip it off to go with the write method. >>> >>>> >>>>> I would think readString behaves like readAllBytes, that would >>>>> simply read all content faithfully. >>>> readString does an interpolation due to the Charset so it's not the >>>> real content, again, the idea is that developers should not have to >>>> care about the platform's line separators (they have more important >>>> things to think). >>>> >>>> The other solution is to just not introduce those new methods, >>>> after all Files.lines().collect(Collectors.joining("\n")) already >>>> does the job, no ? >>> >>> While there are many ways to do it, none is as straight-forward. >>> "Read (entire) file to String"/"Write String to file" are popular >>> requests from users. Read to string -> do some String manipulation >>> -> write it back is such a simple use case, it really needs to be >>> very easy to do as illustrated in the above code snippet. >>> >>> Cheers, >>> Joe >>> >>>> >>>>> Best, >>>>> Joe >>>> regards, >>>> R?mi >>>> >>>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>>> Hi Alan, >>>>>> >>>>>> People do not read the documentation. >>>>>> So adding something in the documentation about when a method >>>>>> should be used or >>>>>> not is never a solution. >>>>>> >>>>>> Here the user want a String and provides a charset so you have no >>>>>> way but to >>>>>> decode the content to substitute the line separator. >>>>>> >>>>>> cheers, >>>>>> R?mi >>>>>> >>>>>> ----- Mail original ----- >>>>>>> De: "Alan Bateman" >>>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>>> >>>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>>> >>>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>>> reading/writing a string from/to a file >>>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>>> I think that having a readString that includes OS dependent >>>>>>>> line separators is a >>>>>>>> mistake, >>>>>>>> Java does a great job to try to shield the developer from the >>>>>>>> kind of things >>>>>>>> that makes programs behave differently on different platforms. >>>>>>>> >>>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>>> people will do a call >>>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>>> hard way. >>>>>>>> >>>>>>>> raw string literal does the same substitution for the same reason. >>>>>>>> >>>>>>> Yes, there are several discussion points around this and >>>>>>> somewhat timely >>>>>>> with multi-string support. >>>>>>> >>>>>>> One thing that I think Joe will need in this API is some note to >>>>>>> make it >>>>>>> clearer what the intended usage is (as I think the intention is >>>>>>> simple >>>>>>> cases with mostly single lines of text). >>>>>>> >>>>>>> -Alan. >>> >> From aph at redhat.com Tue Jun 12 17:58:00 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 12 Jun 2018 18:58:00 +0100 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: <382F6C6B-D097-4DF2-A8AB-FBCB60C5D3F5@oracle.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> <382F6C6B-D097-4DF2-A8AB-FBCB60C5D3F5@oracle.com> Message-ID: <7defb48e-c77a-4ca5-119e-bacd7cf9aa93@redhat.com> On 06/12/2018 06:12 PM, Paul Sandoz wrote: > >> We may also want to look at related enhancements like unmapping >> buffers. I think those pieces are sufficient decoupled that they >> won't be dependencies for the pmem API though, unlike other factors >> such as the availability of test hardware. >> > That?s tricky! We have been through many discussions over the years > on how to achieve this without much resolution. Andrew Haley came up > with an interesting solution which IIRC requires the > deallocating/unmapping thread to effectively reach safe point and > wait for all other threads to pass through a check point. Project > Panama is looking at the explicit scoping of resources, perhaps also > resources that are thread confined or owned. My sense is Project > Panama will eventually push strongly on this area and that?s where > we should focus our efforts. Yeah, perhaps so. I've been waiting to come up for air to have enough time to handle the ByteBuffer.unmap() bug. I can see the advantage of handling it at a static language level, but the solutions aren't necessarily exclusive. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From joe.darcy at oracle.com Wed Jun 13 00:49:13 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 12 Jun 2018 17:49:13 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <82fdd320-f5fb-2c34-5bb8-b72f6718fef6@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <82fdd320-f5fb-2c34-5bb8-b72f6718fef6@oracle.com> Message-ID: <5B206A09.5000404@oracle.com> On 6/11/2018 10:38 PM, mandy chung wrote: > > > On 6/11/18 10:16 PM, David Holmes wrote: >> Here is one further minor update from the CSR discussions: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html > > > "This implementation" is fine, as used in many @implNote. Any reason > why it has to be changed to "reference implementation"? > To summarize the concern there, the phrase "This implementation..." when used elsewhere has a different meaning. Often the phrasing "This implementation..." or the more commonly used "The default implementation..." is used for text that is part of the contract of a method that can be overridden; that is, used to separate out the contract that is independent of which class or interface provides the implementation from the contract of a particular implementation. One example from an API I work on occurs for the method javax.lang.model.element.ElementVisitor.visitModule. The default method defined in an interface states "The default implementation visits a ModuleElement by calling visitUnknown..." and then various visitor classes define their own behavior for this method while still being able to @inheritDoc the general "visit a module element" contract of the visitModule method. However, java.lang.Class is final so for a particular JDK version there is only one implementation of the method in question. In that context "This implementation" doesn't mean "the implementation in this particular class or interface as opposed to the implementation in an a more specific subtype" it means "the implemetnation for the final method used in a particular JDK release." I think using the term "This implementation" in the latter context is misleading so I suggested the alternative wording David sent out for review. HTH, -Joe From weijun.wang at oracle.com Wed Jun 13 01:59:49 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 13 Jun 2018 09:59:49 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> Hi Roger 1. Should all occurrences of reading of these system properties be updated? For example, the following one is not touched http://hg.openjdk.java.net/jdk/jdk/file/4d2e3f5abb48/src/java.base/share/classes/sun/security/tools/keytool/Main.java#l842 2. I assume that with this change not only there is no use calling System.setProperty() in the application but also setting it on the command line is now useless. Is this right? Do we need to make this clear in the CSR? Thanks Max > On Jun 4, 2018, at 9:32 PM, Roger Riggs wrote: > > Please review a change to make the values of java.home, user.home, user.dir, and user.name > effectively read-only for internal use. The values are cached during initialization and the > cached values are used. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8204235 > > Thanks, Roger > > From weijun.wang at oracle.com Wed Jun 13 02:10:57 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 13 Jun 2018 10:10:57 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> Message-ID: <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> In fact, why is setting user.name and user.home always evil? If I only want to set them on the command line so that a special "user environment" is used, why is it a problem? In fact, we have a test setting user.home to an empty directory to avoid unexpected result because we cannot control the test runner's home directory. Thanks Max > On Jun 13, 2018, at 9:59 AM, Weijun Wang wrote: > > Hi Roger > > 1. Should all occurrences of reading of these system properties be updated? For example, the following one is not touched > http://hg.openjdk.java.net/jdk/jdk/file/4d2e3f5abb48/src/java.base/share/classes/sun/security/tools/keytool/Main.java#l842 > > 2. I assume that with this change not only there is no use calling System.setProperty() in the application but also setting it on the command line is now useless. Is this right? Do we need to make this clear in the CSR? > > Thanks > Max > > >> On Jun 4, 2018, at 9:32 PM, Roger Riggs wrote: >> >> Please review a change to make the values of java.home, user.home, user.dir, and user.name >> effectively read-only for internal use. The values are cached during initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8204235 >> >> Thanks, Roger >> >> > From rachna.goel at oracle.com Wed Jun 13 05:33:49 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Wed, 13 Jun 2018 11:03:49 +0530 Subject: RFR: [11] 8202537: CLDR 33 Message-ID: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> Hi, Kindly review fix for JDK-8202537. Fix is to upgrade Unicode consortium's CLDR data into JDK from current version 29 to 33. For more info : http://cldr.unicode.org/index/downloads/cldr-33 Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 Patch: http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html -- Thanks, Rachna From joe.darcy at oracle.com Wed Jun 13 06:08:55 2018 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 12 Jun 2018 23:08:55 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> Message-ID: <9cfdde9b-5929-a5b0-ea26-e79a6d4f56e6@oracle.com> Hi David, On 5/24/2018 10:52 PM, David Holmes wrote: > Here are the further minor updates so far in response to all the > review comments. > > Incremental corelibs webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ > > > Full corelibs webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ > In Class.java, 3990???????? Class[] members = getNestMembers0(); 3991???????? // Can't actually enable this due to bootstrapping issues 3992???????? // assert(members.length != 1 || members[0] == this); // expected invariant from VM can these checks be enabled unconditionally without using the assert facility, throwing AssertionError or some other kind of error? Thanks, -Joe From amy.lu at oracle.com Wed Jun 13 09:31:23 2018 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 13 Jun 2018 17:31:23 +0800 Subject: [JDK 11] RFR 8204944: Remove java/util/Map/InPlaceOpsCollisions.java from ProblemList.txt Message-ID: <2ed84e62-71db-22fc-e892-f48405985a97@oracle.com> Please review this patch to remove java/util/Map/InPlaceOpsCollisions.java from ProblemList.txt, related bug JDK-8203387 (JDK-8203425) has been fixed. Thanks, Amy --- old/test/jdk/ProblemList.txt??? 2018-06-13 17:23:33.000000000 +0800 +++ new/test/jdk/ProblemList.txt??? 2018-06-13 17:23:32.000000000 +0800 @@ -853,8 +853,6 @@ ?# jdk_util -java/util/Map/InPlaceOpsCollisions.java 8203387 generic-all - ?############################################################################ ?# jdk_instrument From thomas.stuefe at gmail.com Wed Jun 13 09:56:43 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 11:56:43 +0200 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: References: Message-ID: May I have a second review please. Thanks, Thomas On Mon, Jun 11, 2018, 15:13 Thomas St?fe wrote: > Hi, > > may I please have reviews for this small cleanup. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ > > This change removes some native functions from FOS/FIS which are > orphaned and unused since "JDK-8187631: Refactor FileDescriptor close > implementation". > > Tests on jdk-submit came back clean save one strange test error on > windows which I cannot reproduce locally. I am investigating that one. > > Thank you, Thomas > From Roger.Riggs at Oracle.com Wed Jun 13 14:10:02 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 13 Jun 2018 10:10:02 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> Message-ID: <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> Hi Joe, The CSR text is still in draft and got out of sync with the open review and comments. The updated apiNote clarifies that changes made through the System.*property*? methods may not affect the value cached during initialization. The implementation changes affect a very short list of properties. The note on the methods is to raise awareness that individual properties may have different behavior and may be unpredictable. On 6/12/2018 10:10 PM, Weijun Wang wrote: > In fact, why is setting user.name and user.home always evil? If I only want to set them on the command line so that a special "user environment" is used, why is it a problem? There is no change to the ability to set properties on the command line. The concern is with the predictability of (some) properties changing dynamically while the application is running. The property user.home is used for mime-types and mailcap files and user.name is used in some networking cases where a username is needed. > > In fact, we have a test setting user.home to an empty directory to avoid unexpected result because we cannot control the test runner's home directory. Right, there is no problem with that Regards, Roger > > Thanks > Max > >> On Jun 13, 2018, at 9:59 AM, Weijun Wang wrote: >> >> Hi Roger >> >> 1. Should all occurrences of reading of these system properties be updated? For example, the following one is not touched >> http://hg.openjdk.java.net/jdk/jdk/file/4d2e3f5abb48/src/java.base/share/classes/sun/security/tools/keytool/Main.java#l842 That usage was in a tool and I did not modify any tools. >> >> 2. I assume that with this change not only there is no use calling System.setProperty() in the application but also setting it on the command line is now useless. Is this right? Do we need to make this clear in the CSR? No change to command line setting of properties. Roger >> >> Thanks >> Max >> >> >>> On Jun 4, 2018, at 9:32 PM, Roger Riggs wrote: >>> >>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>> effectively read-only for internal use. The values are cached during initialization and the >>> cached values are used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>> >>> Thanks, Roger >>> >>> From Roger.Riggs at Oracle.com Wed Jun 13 14:36:52 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 13 Jun 2018 10:36:52 -0400 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> Message-ID: <045baa5b-ce2a-962d-f480-d95ec9e9b63e@Oracle.com> Hi Rachna, How much of the updates are done mechanically between the CLDR data and the .xml files? Manually reviewing that many files is likely to be error prone. Thanks, Roger On 6/13/2018 1:33 AM, Rachna Goel wrote: > Hi, > > Kindly review fix for JDK-8202537. Fix is to upgrade Unicode > consortium's CLDR data into JDK from current version 29 to 33. > > For more info : http://cldr.unicode.org/index/downloads/cldr-33 > > Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 > > Patch: http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html > From rachna.goel at oracle.com Wed Jun 13 15:07:30 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Wed, 13 Jun 2018 20:37:30 +0530 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: <045baa5b-ce2a-962d-f480-d95ec9e9b63e@Oracle.com> References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> <045baa5b-ce2a-962d-f480-d95ec9e9b63e@Oracle.com> Message-ID: <10896dc3-76df-3abd-e706-1814466badda@oracle.com> Hi Roger, No update is done mechanically between CLDR data and .xml files. CLDR's data in .xml files are generated into resourcebundles at build time by cldrconverter tool. Already existing regression test "test/jdk/sun/text/resources/LocaleDataTest.java" verify that same data is retrieved by APIs. Thanks, Rachna* * On 6/13/18 8:06 PM, Roger Riggs wrote: > Hi Rachna, > > How much of the updates are done mechanically between the CLDR data > and the .xml files? > Manually reviewing that many files is likely to be error prone. > > Thanks, Roger > > > On 6/13/2018 1:33 AM, Rachna Goel wrote: >> Hi, >> >> Kindly review fix for JDK-8202537. Fix is to upgrade Unicode >> consortium's CLDR data into JDK from current version 29 to 33. >> >> For more info : http://cldr.unicode.org/index/downloads/cldr-33 >> >> Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 >> >> Patch: >> http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html >> > -- Thanks, Rachna From paul.sandoz at oracle.com Wed Jun 13 16:19:39 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 13 Jun 2018 09:19:39 -0700 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: References: Message-ID: <35C6B224-6AE8-4DD8-A4B2-36889AB7EB0E@oracle.com> +1 Paul. > On Jun 13, 2018, at 2:56 AM, Thomas St?fe wrote: > > May I have a second review please. > > Thanks, Thomas > > On Mon, Jun 11, 2018, 15:13 Thomas St?fe wrote: > >> Hi, >> >> may I please have reviews for this small cleanup. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ >> >> This change removes some native functions from FOS/FIS which are >> orphaned and unused since "JDK-8187631: Refactor FileDescriptor close >> implementation". >> >> Tests on jdk-submit came back clean save one strange test error on >> windows which I cannot reproduce locally. I am investigating that one. >> >> Thank you, Thomas >> From paul.sandoz at oracle.com Wed Jun 13 16:21:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 13 Jun 2018 09:21:17 -0700 Subject: [JDK 11] RFR 8204944: Remove java/util/Map/InPlaceOpsCollisions.java from ProblemList.txt In-Reply-To: <2ed84e62-71db-22fc-e892-f48405985a97@oracle.com> References: <2ed84e62-71db-22fc-e892-f48405985a97@oracle.com> Message-ID: +1 Paul. > On Jun 13, 2018, at 2:31 AM, Amy Lu wrote: > > Please review this patch to remove java/util/Map/InPlaceOpsCollisions.java from ProblemList.txt, > related bug JDK-8203387 (JDK-8203425) has been fixed. > > Thanks, > Amy > > --- old/test/jdk/ProblemList.txt 2018-06-13 17:23:33.000000000 +0800 > +++ new/test/jdk/ProblemList.txt 2018-06-13 17:23:32.000000000 +0800 > @@ -853,8 +853,6 @@ > > # jdk_util > > -java/util/Map/InPlaceOpsCollisions.java 8203387 generic-all > - > ############################################################################ > > # jdk_instrument > > From srinivas.dama at oracle.com Wed Jun 13 16:57:32 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Wed, 13 Jun 2018 09:57:32 -0700 (PDT) Subject: RFR: 8204967: Resolve disabled warnings for libunpack Message-ID: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> Hi, Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8204967 Regards, Srinivas From naoto.sato at oracle.com Wed Jun 13 17:03:23 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 13 Jun 2018 10:03:23 -0700 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> Message-ID: Hi Rachna, A couple of comments: - NumberingSystemsParseHandler.java Since the code substitutes latin digits for supplementary digits, it can skip line 68-79. - A test should be written for the above substitution. Naoto On 6/12/18 10:33 PM, Rachna Goel wrote: > Hi, > > Kindly review fix for JDK-8202537. Fix is to upgrade Unicode > consortium's CLDR data into JDK from current version 29 to 33. > > For more info : http://cldr.unicode.org/index/downloads/cldr-33 > > Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 > > Patch: http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html > From james.laskey at oracle.com Wed Jun 13 17:07:26 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 13 Jun 2018 14:07:26 -0300 Subject: RFR: 8204967: Resolve disabled warnings for libunpack In-Reply-To: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> References: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> Message-ID: Does -Wimplicit-fallthrough=3 work for gnu c and clang? unpack.cpp redundant comment 3713 // else fall through: +1 ? Jim > On Jun 13, 2018, at 1:57 PM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8204967 > > Regards, > Srinivas From thomas.stuefe at gmail.com Wed Jun 13 17:20:06 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 19:20:06 +0200 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: <35C6B224-6AE8-4DD8-A4B2-36889AB7EB0E@oracle.com> References: <35C6B224-6AE8-4DD8-A4B2-36889AB7EB0E@oracle.com> Message-ID: Thank you Paul! On Wed, Jun 13, 2018 at 6:19 PM, Paul Sandoz wrote: > +1 > Paul. > >> On Jun 13, 2018, at 2:56 AM, Thomas St?fe wrote: >> >> May I have a second review please. >> >> Thanks, Thomas >> >> On Mon, Jun 11, 2018, 15:13 Thomas St?fe wrote: >> >>> Hi, >>> >>> may I please have reviews for this small cleanup. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ >>> >>> This change removes some native functions from FOS/FIS which are >>> orphaned and unused since "JDK-8187631: Refactor FileDescriptor close >>> implementation". >>> >>> Tests on jdk-submit came back clean save one strange test error on >>> windows which I cannot reproduce locally. I am investigating that one. >>> >>> Thank you, Thomas >>> > From james.laskey at oracle.com Wed Jun 13 19:36:18 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 13 Jun 2018 16:36:18 -0300 Subject: RFR: JDK-8204172 Predicate::not should explicitly mention "NullPointerException - if target is null" Message-ID: <05556A68-691F-44F7-8163-D46507B00F95@oracle.com> CSR: https://bugs.openjdk.java.net/browse/JDK-8204959 webrev: http://cr.openjdk.java.net/~jlaskey/8204172/webrev/index.html Thank you, ? Jim From erik.joelsson at oracle.com Wed Jun 13 19:47:52 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 13 Jun 2018 12:47:52 -0700 Subject: RFR: JDK-8204973: Add build support for filtering translations Message-ID: Hello, Oracle will reduce the number of languages that it maintains translations of JDK resources for. The current translations will remain in the source for now, but we need a way to filter out a set of translations at build time so that we only include the ones we support. This patch adds such a configuration option. It also changes how Oracle builds by using the option to exclude all translations except English, Japanese, Simplified Chinese and Traditional Chinese. Anyone else building OpenJDK will by default include all translations present in the source, just as before. I added a test that verifies this for builds with the "IMPLEMENTOR" field in the release file set to "Oracle Corporation". The test will not be run for other OpenJDK builds. I had to modify an existing test for java.logging which used various translations to verify localized log messages to only use translations that Oracle chooses to include. Since this is the second test that specifically verifies build behavior, I moved the previous such test together with this new test into a common top level test directory "build", under the jdk test root. I put these tests in the jdk tier3 test group. I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically looked for tests that use the java.util.Locale class and ran them locally. Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 /Erik From huizhe.wang at oracle.com Wed Jun 13 19:52:30 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 13 Jun 2018 12:52:30 -0700 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> Message-ID: <5B2175FE.8040500@oracle.com> Thanks Roger! I pushed the changeset after s/unmappble/unmappable, it found three of them :-) Best, Joe On 6/12/18, 10:37 AM, Roger Riggs wrote: > Hi Joe, > > Looks good to me. > > Typo: StringCoding.java:1026 "unmappble" (no new webrev needed) > > Regards, Roger > > > On 6/12/2018 12:52 PM, Joe Wang wrote: >> Hi all, >> >> It's been a while since the 1st round of reviews. Thank you all for >> the valuable comments and suggestions! Note that Roger's last >> response went to nio-dev, but not core-libs-dev, I've therefore >> attached it below. >> >> CSR [2]: the CSR is now approved. Note the write method has been >> changed to writeString. >> >> Impl [3]: for performance reason and the different requirement from >> byte-string conversion for malformed/unmappable character handing, >> the implementation uses a specific method separate from the existing >> ones. Both Sherman and Alan preferred specific method than adding >> parameters to the APIs that might be error prone. Thanks Sherman for >> the code! >> >> JMH benchmark tests showed the new read method performs better than >> an operation that reads the bytes and convert to string. The gains >> start to be significant even at quite reasonable sizes (< 10K). >> >> >> [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 >> [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 >> [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ >> >> Please review. >> >> Thanks, >> Joe >> >> On 5/15/18, 7:51 AM, Roger Riggs wrote: >>> Hi Joe, et. al. >>> >>> My $.02 on line separators: >>> >>> - We should avoid clever tricks trying to solve problems that are >>> infrequent such as cross OS file line operations. >>> Most files will be read/written on a single system so the line >>> separators will be as expected. >>> >>> - Have/use APIs that split lines consistently accepting both line >>> separators so developer code can be agnostic to line separators. >>> aka BufferedReader.readLine for developers that are processing >>> the contents *as lines*. >>> Those other methods already exist; if there are any gaps in line >>> oriented processing that's a separate task. >>> >>> - These new file methods are defined to handle Charset >>> encoding/decoding and buffering. >>> Since there are other methods to deal with files as lines these >>> methods should not look for or break to lines. >>> >>> - Performance: adding code to look for line characters will slow it >>> down and in most cases would have no impact >>> since the line endings will match the system. >>> >>> - The strings passed to writeString (CharSequence) should have been >>> constructed using the proper line separators. >>> There are any number of methods that insert the os specific line >>> terminator and its easy to write File.separator as needed. >>> >>> As for the method naming: >>> >>> I'd prefer "writeString" as a familiar term that isn't stretched too >>> far by making the argument be a CharSequence. >>> its fine to call a CharSequence a string (with a lower case s). >>> >>> $.02, Roger >>> >>> >>> On 4/27/18 6:33 PM, Joe Wang wrote: >>>> >>>> >>>> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>>>> ----- Mail original ----- >>>>>> De: "Joe Wang" >>>>>> ?: "Remi Forax" , "Alan Bateman" >>>>>> >>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>> >>>>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>> reading/writing a string from/to a file >>>>>> Hi R?mi, Alan, >>>>> Hi Joe, >>>>> >>>>>> I'm not sure we'd want to replace system dependent line >>>>>> separators with >>>>>> '\n'. There are cases as you described where the replacement makes >>>>>> sense. However, there are other cases too. For example, the >>>>>> purpose is >>>>>> to read, edit a text file and then write it back. If this is on >>>>>> Windows, >>>>>> and if line separators are replaced with '\n', that would cause the >>>>>> resulting file not formatted properly in for example Notepad. >>>>>> There are >>>>>> cases where users write text files on Windows using Java, and >>>>>> only found >>>>>> the lines were not separated in Notepad. >>>>> I agree that why the counterpart of readString() that write the >>>>> string should inserts the platform's line separator. >>>>> BTW, i just found that those methods are not named writeString, or >>>>> writeCharSequence, i think they should. >>>> >>>> While readString() does not modify the original content (e.g. by >>>> replacing the platform's line separator with '\n'), write(String) >>>> won't either, by adding extra characters such as the line separator. >>>> >>>> I would think interfaces shall read along with the parameters. >>>> readString(Path) == read as a String from the specified Path >>>> (one could argue for readToString, readAsString, but we generally >>>> omit the preps) >>>> write(Path, CharSequence) == write the CharSequence to the >>>> file, since CharSequence is already in the method signature as a >>>> parameter, we probably don't want to add that to the name, >>>> otherwise it would read like repeating the word CharSequence. >>>> >>>> It is in a similar situation as write(Path, Iterable) where it was >>>> defined as writeLines(Path, Iterable). >>>> >>>>> >>>>>> Files.write(Path, Iterable) is also specified to terminate each line >>>>>> with the platform's line separator. If readString does the >>>>>> replacement, >>>>>> it would be inconsistent. >>>>> >>>>> Anyway, if we look for consistency the methods writeCharSequence >>>>> should transform the \n in the CharSequence to the platform's line >>>>> separator. >>>>> >>>>> Files.write(Path, Iterable) is is not a counterpart of >>>>> readString(), it's consistent with Files.lines() or >>>>> Files.readLines() (or BufferedReader.readLine()) that all suppress >>>>> the line separators. Anyway, Files.write(path, >>>>> readString(path)::line) will be consistent if you replace the line >>>>> separators or not because String.line() suppresses the line >>>>> separators. >>>> >>>> readString pairs with write(String), therefore it's more like >>>> Files.write(path, readString(path)) than readString(path)::line. >>>> The use case: >>>> >>>> String s = Files.read(path); >>>> Files.write(path, s.replace("/config/location", "/new/location")); >>>> >>>> would then work as expected. >>>> >>>> These two methods are one-off (open/read/write/close file) >>>> operation. write(String) therefore is not intended for >>>> adding/appending multiple Strings from other operations such as >>>> String.line(). If an app needs to put the result of String.line() >>>> or any other processes into a file using this method, it needs to >>>> take care of adding the line separator itself when necessary. "when >>>> necessary" would be a judgement call by the developer. >>>> >>>> That said, if there's a consensus on the idea of terminating the >>>> string with a line separator, then readString method would need to >>>> strip it off to go with the write method. >>>> >>>>> >>>>>> I would think readString behaves like readAllBytes, that would >>>>>> simply read all content faithfully. >>>>> readString does an interpolation due to the Charset so it's not >>>>> the real content, again, the idea is that developers should not >>>>> have to care about the platform's line separators (they have more >>>>> important things to think). >>>>> >>>>> The other solution is to just not introduce those new methods, >>>>> after all Files.lines().collect(Collectors.joining("\n")) already >>>>> does the job, no ? >>>> >>>> While there are many ways to do it, none is as straight-forward. >>>> "Read (entire) file to String"/"Write String to file" are popular >>>> requests from users. Read to string -> do some String manipulation >>>> -> write it back is such a simple use case, it really needs to be >>>> very easy to do as illustrated in the above code snippet. >>>> >>>> Cheers, >>>> Joe >>>> >>>>> >>>>>> Best, >>>>>> Joe >>>>> regards, >>>>> R?mi >>>>> >>>>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>>>> Hi Alan, >>>>>>> >>>>>>> People do not read the documentation. >>>>>>> So adding something in the documentation about when a method >>>>>>> should be used or >>>>>>> not is never a solution. >>>>>>> >>>>>>> Here the user want a String and provides a charset so you have >>>>>>> no way but to >>>>>>> decode the content to substitute the line separator. >>>>>>> >>>>>>> cheers, >>>>>>> R?mi >>>>>>> >>>>>>> ----- Mail original ----- >>>>>>>> De: "Alan Bateman" >>>>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>>>> >>>>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>>>> >>>>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>>>> reading/writing a string from/to a file >>>>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>>>> I think that having a readString that includes OS dependent >>>>>>>>> line separators is a >>>>>>>>> mistake, >>>>>>>>> Java does a great job to try to shield the developer from the >>>>>>>>> kind of things >>>>>>>>> that makes programs behave differently on different platforms. >>>>>>>>> >>>>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>>>> people will do a call >>>>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>>>> hard way. >>>>>>>>> >>>>>>>>> raw string literal does the same substitution for the same >>>>>>>>> reason. >>>>>>>>> >>>>>>>> Yes, there are several discussion points around this and >>>>>>>> somewhat timely >>>>>>>> with multi-string support. >>>>>>>> >>>>>>>> One thing that I think Joe will need in this API is some note >>>>>>>> to make it >>>>>>>> clearer what the intended usage is (as I think the intention is >>>>>>>> simple >>>>>>>> cases with mostly single lines of text). >>>>>>>> >>>>>>>> -Alan. >>>> >>> > From daniel.fuchs at oracle.com Wed Jun 13 19:58:28 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 13 Jun 2018 20:58:28 +0100 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: References: Message-ID: <8eab54d8-d225-18d3-7472-aa663faba527@oracle.com> Hi Erik, The modifications to the logging test look good to me. Caveat: I don't speak chinese nor japanese ;-) cheers, -- daniel On 13/06/18 20:47, Erik Joelsson wrote: > Hello, > > Oracle will reduce the number of languages that it maintains > translations of JDK resources for. The current translations will remain > in the source for now, but we need a way to filter out a set of > translations at build time so that we only include the ones we support. > This patch adds such a configuration option. It also changes how Oracle > builds by using the option to exclude all translations except English, > Japanese, Simplified Chinese and Traditional Chinese. Anyone else > building OpenJDK will by default include all translations present in the > source, just as before. > > I added a test that verifies this for builds with the "IMPLEMENTOR" > field in the release file set to "Oracle Corporation". The test will not > be run for other OpenJDK builds. > > I had to modify an existing test for java.logging which used various > translations to verify localized log messages to only use translations > that Oracle chooses to include. > > Since this is the second test that specifically verifies build behavior, > I moved the previous such test together with this new test into a common > top level test directory "build", under the jdk test root. I put these > tests in the jdk tier3 test group. > > I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically > looked for tests that use the java.util.Locale class and ran them locally. > > Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 > > /Erik > From daniel.fuchs at oracle.com Wed Jun 13 20:01:25 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 13 Jun 2018 21:01:25 +0100 Subject: RFR: JDK-8204172 Predicate::not should explicitly mention "NullPointerException - if target is null" In-Reply-To: <05556A68-691F-44F7-8163-D46507B00F95@oracle.com> References: <05556A68-691F-44F7-8163-D46507B00F95@oracle.com> Message-ID: Hi Jim, Looks good to me. best regards, -- daniel On 13/06/18 20:36, Jim Laskey wrote: > CSR: https://bugs.openjdk.java.net/browse/JDK-8204959 > webrev: http://cr.openjdk.java.net/~jlaskey/8204172/webrev/index.html > > Thank you, > > ? Jim > From patrick at reini.net Wed Jun 13 20:22:26 2018 From: patrick at reini.net (Patrick Reinhart) Date: Wed, 13 Jun 2018 22:22:26 +0200 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior Message-ID: Hi everybody, While looking into the current Reader Spec and the failing methods I found that the ready() method actually does behave wrong and I fixed this. For the case of mark() I would like to revise the specification to align with the Reader?s default behavior that states for the mark method: IOException - If the stream does not support mark(), or if some other I/O error occurs The new spec for those methods would then read like: 70 *

The {@code markSupported()} method returns {@code false}. The 71 * {@code mark()} and {@code reset()} methods throw an {@code IOException}. Here the link to the webrev: http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev -Patrick From brian.burkhalter at oracle.com Wed Jun 13 20:56:36 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jun 2018 13:56:36 -0700 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: References: Message-ID: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> Hi Patrick, Not part of your change, but I noticed that at line 66 of Reader.java there is an extra parenthesis after ready(). In the test, the bug ID at line 39 could simply be appended to line 38. Otherwise looks good although I suppose given the specification update you?ll need an approved CSR before checking it in. Thanks, Brian On Jun 13, 2018, at 1:22 PM, Patrick Reinhart wrote: > While looking into the current Reader Spec and the failing methods I found that the ready() method actually does behave wrong and I fixed this. > > For the case of mark() I would like to revise the specification to align with the Reader?s default behavior that states for the mark method: > > IOException - If the stream does not support mark(), or if some other I/O error occurs > > The new spec for those methods would then read like: > > 70 *

The {@code markSupported()} method returns {@code false}. The > 71 * {@code mark()} and {@code reset()} methods throw an {@code IOException}. > > > Here the link to the webrev: > > http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev From roger.riggs at oracle.com Wed Jun 13 21:06:08 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 13 Jun 2018 17:06:08 -0400 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> Message-ID: Hi Patrick, Yes, looks good to me too. For the CSR, the easiest path to clarify the spec is to withdraw the CSR, update the spec, and add a note on the revised behavior. Then finalize the CSR again.? That's enough to get it reviewed and approved. (Using a new CSR would just spread the behavior over two CSRs). Thanks, Roger On 6/13/18 4:56 PM, Brian Burkhalter wrote: > Hi Patrick, > > Not part of your change, but I noticed that at line 66 of Reader.java there is an extra parenthesis after ready(). > > In the test, the bug ID at line 39 could simply be appended to line 38. > > Otherwise looks good although I suppose given the specification update you?ll need an approved CSR before checking it in. > > Thanks, > > Brian > > On Jun 13, 2018, at 1:22 PM, Patrick Reinhart wrote: > >> While looking into the current Reader Spec and the failing methods I found that the ready() method actually does behave wrong and I fixed this. >> >> For the case of mark() I would like to revise the specification to align with the Reader?s default behavior that states for the mark method: >> >> IOException - If the stream does not support mark(), or if some other I/O error occurs >> >> The new spec for those methods would then read like: >> >> 70 *

The {@code markSupported()} method returns {@code false}. The >> 71 * {@code mark()} and {@code reset()} methods throw an {@code IOException}. >> >> >> Here the link to the webrev: >> >> http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev From mandy.chung at oracle.com Wed Jun 13 21:10:03 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 13 Jun 2018 14:10:03 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <5B206A09.5000404@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <82fdd320-f5fb-2c34-5bb8-b72f6718fef6@oracle.com> <5B206A09.5000404@oracle.com> Message-ID: Thanks for the explanation, Joe. I see the subtle difference on these terminologies. I'm okay with this patch. I like the other option better to remove @apiNote since the spec states that the duplicates may not be removed. Mandy On 6/12/18 5:49 PM, Joseph D. Darcy wrote: > On 6/11/2018 10:38 PM, mandy chung wrote: >> >> >> On 6/11/18 10:16 PM, David Holmes wrote: >>> Here is one further minor update from the CSR discussions: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html >> >> >> >> "This implementation" is fine, as used in many @implNote.? Any reason >> why it has to be changed to "reference implementation"? >> > > To summarize the concern there, the phrase "This implementation..." when > used elsewhere has a different meaning. > > Often the phrasing "This implementation..." or the more commonly used > "The default implementation..." is used for text that is part of the > contract of a method that can be overridden; that is, used to separate > out the contract that is independent of which class or interface > provides the implementation from the contract of a particular > implementation. > > One example from an API I work on occurs for the method > javax.lang.model.element.ElementVisitor.visitModule. The default method > defined in an interface states "The default implementation visits a > ModuleElement by calling visitUnknown..." and then various visitor > classes define their own behavior for this method while still being able > to @inheritDoc the general "visit a module element" contract of the > visitModule method. > > However, java.lang.Class is final so for a particular JDK version there > is only one implementation of the method in question. In that context > "This implementation" doesn't mean "the implementation in this > particular class or interface as opposed to the implementation in an a > more specific subtype" it means "the implemetnation for the final method > used in a particular JDK release." > > I think using the term "This implementation" in the latter context is > misleading so I suggested the alternative wording David sent out for > review. > > HTH, > > -Joe From brian.burkhalter at oracle.com Wed Jun 13 21:16:17 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jun 2018 14:16:17 -0700 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> Message-ID: <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> Hi Roger, Thanks for pointing this out: simpler and cleaner. Brian On Jun 13, 2018, at 2:06 PM, Roger Riggs wrote: > For the CSR, the easiest path to clarify the spec is to withdraw the CSR, update the spec, > and add a note on the revised behavior. > > Then finalize the CSR again. That's enough to get it reviewed and approved. > > (Using a new CSR would just spread the behavior over two CSRs). From brian.goetz at oracle.com Wed Jun 13 21:40:58 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 13 Jun 2018 17:40:58 -0400 Subject: BiCollector In-Reply-To: References: Message-ID: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> I really like how BiCollector can be private, so all we'd have to expose is toBoth(), and the arguments of toBoth() are just ordinary collectors.? So no new types or abstractions; just a multiplexing. The use of Map.Entry as a pair surrogate is unfortunate -- its definitely not an Entry -- though I understand why you did this. I'm not sure if this is fatal or not for a JDK method, but it's pretty bad*.? (You could generalize to n-ary, returning a List or array (you can take a varargs of Collector), at the loss of sharp types for the results.) *Yes, I'm sure one can find precedent of this being done; this has no effect on whether it's bad. On 6/11/2018 8:39 AM, Peter Levart wrote: > Hi, > > Have you ever wanted to perform a collection of the same Stream into > two different targets using two Collectors? Say you wanted to collect > Map.Entry elements into two parallel lists, each of them containing > keys and values respectively. Or you wanted to collect elements into? > groups by some key, but also count them at the same time? Currently > this is not possible to do with a single Stream. You have to create > two identical streams, so you end up passing Supplier to other > methods instead of bare Stream. > > I created a little utility Collector implementation that serves the > purpose quite well: > > /** > ?* A {@link Collector} implementation taking two delegate Collector(s) > and producing result composed > ?* of two results produced by delegating collectors, wrapped in {@link > Map.Entry} object. > ?* > ?* @param the type of elements collected > ?* @param the type of 1st delegate collector collected result > ?* @param tye type of 2nd delegate collector collected result > ?*/ > public class BiCollector implements Collector Map.Entry, Map.Entry> { > ??? private final Collector keyCollector; > ??? private final Collector valCollector; > > ??? @SuppressWarnings("unchecked") > ??? public BiCollector(Collector keyCollector, Collector ?, V> valCollector) { > ??????? this.keyCollector = (Collector) > Objects.requireNonNull(keyCollector); > ??????? this.valCollector = (Collector) > Objects.requireNonNull(valCollector); > ??? } > > ??? @Override > ??? public Supplier> supplier() { > ??????? Supplier keySupplier = keyCollector.supplier(); > ??????? Supplier valSupplier = valCollector.supplier(); > ??????? return () -> new > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > ??? } > > ??? @Override > ??? public BiConsumer, T> accumulator() { > ??????? BiConsumer keyAccumulator = > keyCollector.accumulator(); > ??????? BiConsumer valAccumulator = > valCollector.accumulator(); > ??????? return (accumulation, t) -> { > ??????????? keyAccumulator.accept(accumulation.getKey(), t); > ??????????? valAccumulator.accept(accumulation.getValue(), t); > ??????? }; > ??? } > > ??? @Override > ??? public BinaryOperator> combiner() { > ??????? BinaryOperator keyCombiner = keyCollector.combiner(); > ??????? BinaryOperator valCombiner = valCollector.combiner(); > ??????? return (accumulation1, accumulation2) -> new > AbstractMap.SimpleImmutableEntry<>( > ??????????? keyCombiner.apply(accumulation1.getKey(), > accumulation2.getKey()), > ??????????? valCombiner.apply(accumulation1.getValue(), > accumulation2.getValue()) > ??????? ); > ??? } > > ??? @Override > ??? public Function, Map.Entry> > finisher() { > ??????? Function keyFinisher = keyCollector.finisher(); > ??????? Function valFinisher = valCollector.finisher(); > ??????? return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > ??????????? keyFinisher.apply(accumulation.getKey()), > ??????????? valFinisher.apply(accumulation.getValue()) > ??????? ); > ??? } > > ??? @Override > ??? public Set characteristics() { > ??????? EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > ??????? intersection.retainAll(valCollector.characteristics()); > ??????? return intersection; > ??? } > } > > > Do you think this class is general enough to be part of standard > Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector > coll1, Collector coll2), bi-collection could then be coded simply as: > > ??????? Map map = ... > > ??????? Map.Entry, List> keys_values = > ??????????? map.entrySet() > ?????????????? .stream() > ?????????????? .collect( > ?????????????????? toBoth( > ?????????????????????? mapping(Map.Entry::getKey, toList()), > ?????????????????????? mapping(Map.Entry::getValue, toList()) > ?????????????????? ) > ?????????????? ); > > > ??????? Map.Entry, Long> histogram_count = > ??????????? ThreadLocalRandom > ??????????????? .current() > ??????????????? .ints(100, 0, 10) > ??????????????? .boxed() > ??????????????? .collect( > ??????????????????? toBoth( > ??????????????????????? groupingBy(Function.identity(), counting()), > ??????????????????????? counting() > ??????????????????? ) > ??????????????? ); > > > Regards, Peter > From joe.darcy at oracle.com Thu Jun 14 00:51:13 2018 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 13 Jun 2018 17:51:13 -0700 Subject: JDK 11 RFR of JDK-8205003: Replace selected link tags with linkplain in java.lang.Class Message-ID: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> Hello, Please review the small patch below to address ??? JDK-8205003: Replace selected link tags with linkplain in java.lang.Class Thanks, -Joe diff -r 0742a087710e src/java.base/share/classes/java/lang/Class.java --- a/src/java.base/share/classes/java/lang/Class.java??? Wed Jun 13 13:12:50 2018 -0700 +++ b/src/java.base/share/classes/java/lang/Class.java??? Wed Jun 13 17:50:12 2018 -0700 @@ -820,7 +820,7 @@ ????? * primitive type or void, then the {@code Module} object for the ????? * {@code java.base} module is returned. ????? * -???? * If this class is in an unnamed module then the {@link +???? * If this class is in an unnamed module then the {@linkplain ????? * ClassLoader#getUnnamedModule() unnamed} {@code Module} of the class ????? * loader for this class is returned. ????? * @@ -953,14 +953,14 @@ ????? * empty string if the class is in an unnamed package. ????? * ????? *

If this class is a member class, then this method is equivalent to -???? * invoking {@code getPackageName()} on the {@link #getEnclosingClass +???? * invoking {@code getPackageName()} on the {@linkplain #getEnclosingClass ????? * enclosing class}. ????? * -???? *

If this class is a {@link #isLocalClass local class} or an {@link +???? *

If this class is a {@linkplain #isLocalClass local class} or an {@linkplain ????? * #isAnonymousClass() anonymous class}, then this method is equivalent to -???? * invoking {@code getPackageName()} on the {@link #getDeclaringClass -???? * declaring class} of the {@link #getEnclosingMethod enclosing method} or -???? * {@link #getEnclosingConstructor enclosing constructor}. +???? * invoking {@code getPackageName()} on the {@linkplain #getDeclaringClass +???? * declaring class} of the {@linkplain #getEnclosingMethod enclosing method} or +???? * {@linkplain #getEnclosingConstructor enclosing constructor}. ????? * ????? *

If this class represents an array type then this method returns the ????? * package name of the element type. If this class represents a primitive @@ -2576,7 +2576,7 @@ ????? * @param? name name of the desired resource ????? * @return? A {@link java.io.InputStream} object; {@code null} if no ????? *????????? resource with this name is found, the resource is in a package -???? *????????? that is not {@link Module#isOpen(String, Module) open} to at +???? *????????? that is not {@linkplain Module#isOpen(String, Module) open} to at ????? *????????? least the caller module, or access to the resource is denied ????? *????????? by the security manager. ????? * @throws? NullPointerException If {@code name} is {@code null} @@ -2675,7 +2675,7 @@ ????? * @return A {@link java.net.URL} object; {@code null} if no resource with ????? *???????? this name is found, the resource cannot be located by a URL, the ????? *???????? resource is in a package that is not -???? *???????? {@link Module#isOpen(String, Module) open} to at least the caller +???? *???????? {@linkplain Module#isOpen(String, Module) open} to at least the caller ????? *???????? module, or access to the resource is denied by the security ????? *???????? manager. ????? * @throws NullPointerException If {@code name} is {@code null} From brian.burkhalter at oracle.com Thu Jun 14 00:56:25 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jun 2018 17:56:25 -0700 Subject: JDK 11 RFR of JDK-8205003: Replace selected link tags with linkplain in java.lang.Class In-Reply-To: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> References: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> Message-ID: Hi Joe, +1 Brian On Jun 13, 2018, at 5:51 PM, joe darcy wrote: > Please review the small patch below to address > > JDK-8205003: Replace selected link tags with linkplain in java.lang.Class From mandy.chung at oracle.com Thu Jun 14 01:02:23 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 13 Jun 2018 18:02:23 -0700 Subject: JDK 11 RFR of JDK-8205003: Replace selected link tags with linkplain in java.lang.Class In-Reply-To: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> References: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> Message-ID: +1 Mandy On 6/13/18 5:51 PM, joe darcy wrote: > Hello, > > Please review the small patch below to address > > ??? JDK-8205003: Replace selected link tags with linkplain in > java.lang.Class > > Thanks, > > -Joe > > diff -r 0742a087710e src/java.base/share/classes/java/lang/Class.java > --- a/src/java.base/share/classes/java/lang/Class.java??? Wed Jun 13 > 13:12:50 2018 -0700 > +++ b/src/java.base/share/classes/java/lang/Class.java??? Wed Jun 13 > 17:50:12 2018 -0700 > @@ -820,7 +820,7 @@ > ????? * primitive type or void, then the {@code Module} object for the > ????? * {@code java.base} module is returned. > ????? * > -???? * If this class is in an unnamed module then the {@link > +???? * If this class is in an unnamed module then the {@linkplain > ????? * ClassLoader#getUnnamedModule() unnamed} {@code Module} of the > class > ????? * loader for this class is returned. > ????? * > @@ -953,14 +953,14 @@ > ????? * empty string if the class is in an unnamed package. > ????? * > ????? *

If this class is a member class, then this method is > equivalent to > -???? * invoking {@code getPackageName()} on the {@link #getEnclosingClass > +???? * invoking {@code getPackageName()} on the {@linkplain > #getEnclosingClass > ????? * enclosing class}. > ????? * > -???? *

If this class is a {@link #isLocalClass local class} or an > {@link > +???? *

If this class is a {@linkplain #isLocalClass local class} or > an {@linkplain > ????? * #isAnonymousClass() anonymous class}, then this method is > equivalent to > -???? * invoking {@code getPackageName()} on the {@link #getDeclaringClass > -???? * declaring class} of the {@link #getEnclosingMethod enclosing > method} or > -???? * {@link #getEnclosingConstructor enclosing constructor}. > +???? * invoking {@code getPackageName()} on the {@linkplain > #getDeclaringClass > +???? * declaring class} of the {@linkplain #getEnclosingMethod > enclosing method} or > +???? * {@linkplain #getEnclosingConstructor enclosing constructor}. > ????? * > ????? *

If this class represents an array type then this method > returns the > ????? * package name of the element type. If this class represents a > primitive > @@ -2576,7 +2576,7 @@ > ????? * @param? name name of the desired resource > ????? * @return? A {@link java.io.InputStream} object; {@code null} if no > ????? *????????? resource with this name is found, the resource is in a > package > -???? *????????? that is not {@link Module#isOpen(String, Module) open} > to at > +???? *????????? that is not {@linkplain Module#isOpen(String, Module) > open} to at > ????? *????????? least the caller module, or access to the resource is > denied > ????? *????????? by the security manager. > ????? * @throws? NullPointerException If {@code name} is {@code null} > @@ -2675,7 +2675,7 @@ > ????? * @return A {@link java.net.URL} object; {@code null} if no > resource with > ????? *???????? this name is found, the resource cannot be located by a > URL, the > ????? *???????? resource is in a package that is not > -???? *???????? {@link Module#isOpen(String, Module) open} to at least > the caller > +???? *???????? {@linkplain Module#isOpen(String, Module) open} to at > least the caller > ????? *???????? module, or access to the resource is denied by the > security > ????? *???????? manager. > ????? * @throws NullPointerException If {@code name} is {@code null} > From Lance.Andersen at oracle.com Thu Jun 14 01:45:32 2018 From: Lance.Andersen at oracle.com (Lance Andersen) Date: Wed, 13 Jun 2018 21:45:32 -0400 Subject: JDK 11 RFR of JDK-8205003: Replace selected link tags with linkplain in java.lang.Class In-Reply-To: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> References: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> Message-ID: +1 -- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPhone > On Jun 13, 2018, at 8:51 PM, joe darcy wrote: > > Hello, > > Please review the small patch below to address > > JDK-8205003: Replace selected link tags with linkplain in java.lang.Class > > Thanks, > > -Joe > > diff -r 0742a087710e src/java.base/share/classes/java/lang/Class.java > --- a/src/java.base/share/classes/java/lang/Class.java Wed Jun 13 13:12:50 2018 -0700 > +++ b/src/java.base/share/classes/java/lang/Class.java Wed Jun 13 17:50:12 2018 -0700 > @@ -820,7 +820,7 @@ > * primitive type or void, then the {@code Module} object for the > * {@code java.base} module is returned. > * > - * If this class is in an unnamed module then the {@link > + * If this class is in an unnamed module then the {@linkplain > * ClassLoader#getUnnamedModule() unnamed} {@code Module} of the class > * loader for this class is returned. > * > @@ -953,14 +953,14 @@ > * empty string if the class is in an unnamed package. > * > *

If this class is a member class, then this method is equivalent to > - * invoking {@code getPackageName()} on the {@link #getEnclosingClass > + * invoking {@code getPackageName()} on the {@linkplain #getEnclosingClass > * enclosing class}. > * > - *

If this class is a {@link #isLocalClass local class} or an {@link > + *

If this class is a {@linkplain #isLocalClass local class} or an {@linkplain > * #isAnonymousClass() anonymous class}, then this method is equivalent to > - * invoking {@code getPackageName()} on the {@link #getDeclaringClass > - * declaring class} of the {@link #getEnclosingMethod enclosing method} or > - * {@link #getEnclosingConstructor enclosing constructor}. > + * invoking {@code getPackageName()} on the {@linkplain #getDeclaringClass > + * declaring class} of the {@linkplain #getEnclosingMethod enclosing method} or > + * {@linkplain #getEnclosingConstructor enclosing constructor}. > * > *

If this class represents an array type then this method returns the > * package name of the element type. If this class represents a primitive > @@ -2576,7 +2576,7 @@ > * @param name name of the desired resource > * @return A {@link java.io.InputStream} object; {@code null} if no > * resource with this name is found, the resource is in a package > - * that is not {@link Module#isOpen(String, Module) open} to at > + * that is not {@linkplain Module#isOpen(String, Module) open} to at > * least the caller module, or access to the resource is denied > * by the security manager. > * @throws NullPointerException If {@code name} is {@code null} > @@ -2675,7 +2675,7 @@ > * @return A {@link java.net.URL} object; {@code null} if no resource with > * this name is found, the resource cannot be located by a URL, the > * resource is in a package that is not > - * {@link Module#isOpen(String, Module) open} to at least the caller > + * {@linkplain Module#isOpen(String, Module) open} to at least the caller > * module, or access to the resource is denied by the security > * manager. > * @throws NullPointerException If {@code name} is {@code null} > From weijun.wang at oracle.com Thu Jun 14 01:48:33 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Thu, 14 Jun 2018 09:48:33 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> Message-ID: <3733DE65-8E74-465E-9BE2-CCB1D236CC96@oracle.com> Thanks a lot for the explanation. Everything looks OK to me now. --Max > On Jun 13, 2018, at 10:10 PM, Roger Riggs wrote: > > Hi Joe, > > The CSR text is still in draft and got out of sync with the open review and comments. > > The updated apiNote clarifies that changes made through the System.*property* methods > may not affect the value cached during initialization. > The implementation changes affect a very short list of properties. > > The note on the methods is to raise awareness that individual properties may have > different behavior and may be unpredictable. > > On 6/12/2018 10:10 PM, Weijun Wang wrote: >> In fact, why is setting user.name and user.home always evil? If I only want to set them on the command line so that a special "user environment" is used, why is it a problem? > There is no change to the ability to set properties on the command line. > The concern is with the predictability of (some) properties changing dynamically while the application > is running. > > The property user.home is used for mime-types and mailcap files and user.name is used in some > networking cases where a username is needed. >> >> In fact, we have a test setting user.home to an empty directory to avoid unexpected result because we cannot control the test runner's home directory. >> > Right, there is no problem with that > > Regards, Roger >> >> Thanks >> Max >> >> >>> On Jun 13, 2018, at 9:59 AM, Weijun Wang >>> wrote: >>> >>> Hi Roger >>> >>> 1. Should all occurrences of reading of these system properties be updated? For example, the following one is not touched >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/4d2e3f5abb48/src/java.base/share/classes/sun/security/tools/keytool/Main.java#l842 > That usage was in a tool and I did not modify any tools. >>> >>> 2. I assume that with this change not only there is no use calling System.setProperty() in the application but also setting it on the command line is now useless. Is this right? Do we need to make this clear in the CSR? >>> > No change to command line setting of properties. > > Roger > >>> >>> Thanks >>> Max >>> >>> >>> >>>> On Jun 4, 2018, at 9:32 PM, Roger Riggs >>>> wrote: >>>> >>>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>>> effectively read-only for internal use. The values are cached during initialization and the >>>> cached values are used. >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>>> >>>> >>>> Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>>> >>>> >>>> CSR: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>>> >>>> >>>> Thanks, Roger >>>> >>>> >>>> > From bhamaram at in.ibm.com Thu Jun 14 03:23:44 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Thu, 14 Jun 2018 03:23:44 +0000 Subject: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 In-Reply-To: References: , , <5331ad7ba3c63a066895d41a91bd9593@linux.vnet.ibm.com> Message-ID: Hi Thomas, I've added testcase along with fix. Please find webrev at http://cr.openjdk.java.net/~aleonard/8202329/webrev.01/ Thanks, Bhaktavatsal Reddy -----"core-libs-dev" wrote: ----- To: "Thomas St?fe" From: "Bhaktavatsal R Maram" Sent by: "core-libs-dev" Date: 06/01/2018 01:34PM Cc: Java Core Libs Subject: Re: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 Hi Thomas, Sorry, I was on vacation. I will submit webrev with jtreg testcase. Thanks, Bhaktavatsal -----"Thomas St?fe" wrote: ----- To: Bhaktavatsal R Maram From: "Thomas St?fe" Date: 05/23/2018 12:32AM Cc: Ichiroh Takiguchi , Volker Simonis , Java Core Libs Subject: Re: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 Hi Bhaktavatsal, the fix is fine. Reviewed. Thanks a lot for your work! Could you please (its okay in a future patch) add a regression test for IBM943 and IBM943C, in the form of a jtreg test? I would like to test conversions for both code pages (at least for the crucial tilde/backslash code points). Additionally, that when started on AIX with Ja_JP.IBM-943, we correctly default to IBM943C. As an example, here is an old test I wrote years ago. You won't be able to use it verbatim, so it is just a suggestion. Feel free to do it differently. http://cr.openjdk.java.net/~stuefe/other/IBM943MappingTest.java If you have not written jtreg tests before: http://openjdk.java.net/jtreg/ Also, under /test are many many examples. Best Regards, Thomas On Mon, May 21, 2018 at 9:47 AM, Bhaktavatsal R Maram wrote: > Hi Thomas, > > Is this fix ready to merge? > > Thanks, > Bhaktavatsal > > > > > -----"Thomas St?fe" wrote: ----- > To: Ichiroh Takiguchi , Bhaktavatsal R Maram > From: "Thomas St?fe" > Date: 05/11/2018 11:49AM > Cc: Volker Simonis , Java Core Libs > Subject: Re: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 > > Hi, > > I'll test and review next week. We also have some in-house tests which > I'd like to run. > > You IBM folks should really apply for authorship so that this > contribution process gets streamlined. After all, if something breaks > in this code, you want to be able to fix it, yes? So even if you do > not contribute much else, more patches may be forthcoming. > > Of course I hope these are not your last contributions :) > > Best, Thomas > > > > On Fri, May 11, 2018 at 7:57 AM, Ichiroh Takiguchi > wrote: >> Hi. >> >> I tested this fix on AIX. >> >> I got following results. >> $ LANG=Ja_JP ~/jdk/bin/java PrintDefaultCharset >> Ja_JP x-IBM943C IBM-943C IBM-943C >> $ LANG=Ja_JP.IBM-943 ~/jdk/bin/java PrintDefaultCharset >> Ja_JP.IBM-943 x-IBM943C IBM-943C IBM-943C >> $ LANG=Zh_TW ~/jdk/bin/java PrintDefaultCharset >> Zh_TW x-IBM950 IBM-950 IBM-950 >> $ LANG=Zh_TW.big5 ~/jdk/bin/java PrintDefaultCharset >> Zh_TW.big5 x-IBM950 IBM-950 IBM-950 >> >> Also I reviewed source code, it's fine >> >> Since this testing requires locale installation for Ja_JP and Zh_TW, >> so it's not easy to test it... >> (At least, I think bos.loc.pc.Ja_JP and bos.loc.iso.Zh_TW filesets are >> required) >> >> >> On 2018-05-02 18:32, Volker Simonis wrote: >>> >>> Hi Bhaktavatsal Reddy, >>> >>> your change looks good. I can sponsor it. >>> >>> Just waiting for a second review... >>> >>> Thank you and best regards, >>> Volker >>> >>> >>> On Mon, Apr 30, 2018 at 11:29 AM, Bhaktavatsal R Maram >>> wrote: >>>> >>>> Hi All, >>>> >>>> Please review the fix. >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202329 >>>> webrev: http://cr.openjdk.java.net/~aleonard/8202329/webrev.00/ >>>> >>>> Thanks, >>>> Bhaktavatsal Reddy >>>> >>>> -----"core-libs-dev" wrote: >>>> ----- >>>> To: Volker Simonis >>>> From: "Bhaktavatsal R Maram" >>>> Sent by: "core-libs-dev" >>>> Date: 04/26/2018 09:31PM >>>> Cc: Java Core Libs >>>> Subject: Re: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>> >>>> Hi Volker, >>>> >>>> Thank you. I will address your review comments and send webrev for >>>> review. >>>> >>>> - Bhaktavatsal Reddy >>>> >>>> >>>> >>>> -----Volker Simonis wrote: ----- >>>> To: Bhaktavatsal R Maram >>>> From: Volker Simonis >>>> Date: 04/26/2018 09:12PM >>>> Cc: Java Core Libs >>>> Subject: Re: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>> >>>> Hi Bhaktavatsal Reddy, >>>> >>>> I've opened the following issue for this problem: >>>> >>>> 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8202329 >>>> >>>> Looking at you fix, can you please replace the "#elif AIX" by "#ifdef >>>> AIX" and the original "#else" by "#ifdef __solaris__". The original >>>> else branch contains Solaris-only code anyway and it is an historical >>>> omission that there are still a lot of places in the code where "not >>>> Linux" implicitly means "Solaris", but that's often wrong. >>>> >>>> Regards, >>>> Volker >>>> >>>> >>>> On Thu, Apr 26, 2018 at 4:02 PM, Bhaktavatsal R Maram >>>> wrote: >>>>> >>>>> Oops! Looks like there is problem with attachment (might be because I >>>>> attached .class file as well). I'm pasting the fix and test program here in >>>>> mail. >>>>> >>>>> Test Program: >>>>> >>>>> import java.nio.charset.*; >>>>> class PrintDefaultCharset { >>>>> public static void main(String[] args) { >>>>> System.out.println("LANG = "+System.getenv("LANG")); >>>>> System.out.println("Default charset = >>>>> "+Charset.defaultCharset().name()); >>>>> System.out.println("file.encoding = >>>>> "+System.getProperty("file.encoding")); >>>>> System.out.println("sun.jnu.encoding = >>>>> "+System.getProperty("sun.jnu.encoding")); >>>>> } >>>>> } >>>>> >>>>> >>>>> Fix: >>>>> >>>>> diff --git a/src/java.base/unix/native/libjava/java_props_md.c >>>>> b/src/java.base/unix/native/libjava/java_props_md.c >>>>> --- a/src/java.base/unix/native/libjava/java_props_md.c >>>>> +++ b/src/java.base/unix/native/libjava/java_props_md.c >>>>> @@ -1,5 +1,5 @@ >>>>> /* >>>>> - * Copyright (c) 1998, 2016, Oracle and/or its affiliates. All rights >>>>> reserved. >>>>> + * Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights >>>>> reserved. >>>>> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>>> * >>>>> * This code is free software; you can redistribute it and/or modify it >>>>> @@ -297,6 +297,18 @@ >>>>> if (strcmp(p, "EUC-JP") == 0) { >>>>> *std_encoding = "EUC-JP-LINUX"; >>>>> } >>>>> +#elif defined _AIX >>>>> + if (strcmp(p, "big5") == 0) { >>>>> + /* On AIX Traditional Chinese Big5 codeset is mapped to >>>>> IBM-950 */ >>>>> + *std_encoding = "IBM-950"; >>>>> + } else if (strcmp(p, "IBM-943") == 0) { >>>>> + /* >>>>> + * On AIX, IBM-943 is mapped to IBM-943C in which symbol >>>>> 'yen' and >>>>> + * 'overline' are replaced with 'backslash' and 'tilde' >>>>> from ASCII >>>>> + * making first 96 code points same as ASCII. >>>>> + */ >>>>> + *std_encoding = "IBM-943C"; >>>>> + } >>>>> #else >>>>> if (strcmp(p,"eucJP") == 0) { >>>>> /* For Solaris use customized vendor defined character >>>>> >>>>> >>>>> Thanks, >>>>> Bhaktavatsal Reddy >>>>> >>>>> >>>>> -----"core-libs-dev" wrote: >>>>> ----- >>>>> To: "Java Core Libs" >>>>> From: "Bhaktavatsal R Maram" >>>>> Sent by: "core-libs-dev" >>>>> Date: 04/26/2018 07:26PM >>>>> Subject: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>>> >>>>> Hi All, >>>>> >>>>> This issue is continuation to bug 8201540 (Extend the set of supported >>>>> charsets in java.base on AIX) in which we have moved default charsets of >>>>> most of the locales supported by Operating System to java.base module thus >>>>> enabling OpenJDK on those locales for AIX platform. >>>>> >>>>> As part of that, charsets for locales Ja_JP (IBM-943) and Zh_TW (big5) >>>>> also have been moved. However, corresponding charsets mapped in Java is not >>>>> correct for them on AIX. Following are the details: >>>>> >>>>> 1. IBM-943 [1] for locale Ja_JP should be mapped to IBM-943C [2] >>>>> >>>>> Fundamental difference between IBM-943 and IBM-943C is that IBM-943C is >>>>> ASCII compatible which means code points 'yen' and 'overline' of IBM-943 is >>>>> replaced with 'backslash' and 'tilde' from ASCII character set. >>>>> >>>>> >>>>> 2. Big5 for locale Zh_TW should be mapped to IBM-950 [3] >>>>> >>>>> I've attached simple test program to print the default charset along >>>>> with fix for this issue. When run test program (PrintDefaultCharset) with >>>>> IBM JDK 8 (on AIX) for locales Ja_JP & Zh_TW, following is output. >>>>> >>>>> -bash-4.4$ LANG=Ja_JP ~/JDKs/IBM/80/ON/sdk/jre/bin/java >>>>> PrintDefaultCharset >>>>> LANG = Ja_JP >>>>> Default charset = x-IBM943C >>>>> file.encoding = IBM-943C >>>>> sun.jnu.encoding = IBM-943C >>>>> >>>>> -bash-4.4$ LANG=Zh_TW ~/JDKs/IBM/80/ON/sdk/jre/bin/java >>>>> PrintDefaultCharset >>>>> LANG = Zh_TW >>>>> Default charset = x-IBM950 >>>>> file.encoding = IBM-950 >>>>> sun.jnu.encoding = IBM-950 >>>>> >>>>> >>>>> Same test run with openJDK 11 gives following output >>>>> >>>>> -bash-4.4$ LANG=Ja_JP ~/jdk/bin/java PrintDefaultCharset >>>>> LANG = Ja_JP >>>>> Default charset = x-IBM943 >>>>> file.encoding = IBM-943 >>>>> sun.jnu.encoding = IBM-943 >>>>> >>>>> -bash-4.4$ LANG=Zh_TW ~/jdk/bin/java PrintDefaultCharset >>>>> LANG = Zh_TW >>>>> Default charset = Big5 >>>>> file.encoding = big5 >>>>> sun.jnu.encoding = big5 >>>>> >>>>> I will get webrev hosted in >>>>> http://cr.openjdk.java.net >>>>> for this change and send it for review once JIRA bug is created. >>>>> >>>>> [1] >>>>> http://demo.icu-project.org/icu-bin/convexp?conv=ibm-943_P130-1999&s=JAVA >>>>> [2] >>>>> http://demo.icu-project.org/icu-bin/convexp?conv=ibm-943_P15A-2003&s=ALL >>>>> [3] >>>>> https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.nlsgdrf/big5.htm >>>>> >>>>> >>>>> Thanks, >>>>> Bhaktavatsal Reddy >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >> > > From david.holmes at oracle.com Thu Jun 14 03:29:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Jun 2018 13:29:27 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <9cfdde9b-5929-a5b0-ea26-e79a6d4f56e6@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9cfdde9b-5929-a5b0-ea26-e79a6d4f56e6@oracle.com> Message-ID: On 13/06/2018 4:08 PM, joe darcy wrote: > Hi David, > > On 5/24/2018 10:52 PM, David Holmes wrote: >> Here are the further minor updates so far in response to all the >> review comments. >> >> Incremental corelibs webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ >> >> >> Full corelibs webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ >> > > In Class.java, > > 3990???????? Class[] members = getNestMembers0(); > 3991???????? // Can't actually enable this due to bootstrapping issues > 3992???????? // assert(members.length != 1 || members[0] == this); // > expected invariant from VM > > can these checks be enabled unconditionally without using the assert > facility, throwing AssertionError or some other kind of error? Of course - but we don't want to pay the price of always checking something that would indicate an error on the VM side. There's an equivalent assertion on the VM side. Thanks, David > Thanks, > > -Joe From amaembo at gmail.com Thu Jun 14 04:56:25 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Thu, 14 Jun 2018 11:56:25 +0700 Subject: BiCollector In-Reply-To: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> References: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> Message-ID: Hello! In my StreamEx library I created a "pairing" collector which does similar job, but allows user to decide how to combine the results of two collectors. This adds more flexibility. The signature is like this: public static Collector pairing( Collector c1, Collector c2, BiFunction finisher) Having such collector, The proposed `toBoth(c1, c2)` can be implemented as simple as `pairing(c1, c2, Map::entry)`. OTOH if somebody wants to use their own Pair class, it would be `pairing(c1, c2, Pair::new)`. Sometimes you don't need a pair, but can create compound result object right here. E.g.: Collector summing = Collectors.reducing(BigDecimal.ZERO, BigDecimal::add); Collector averaging = pairing(summing, Collectors.counting(), (sum, count) -> sum.divide(BigDecimal.valueOf(count), RoundingMode.HALF_EVEN)); So locking to Map.Entry is entirely unnecessary. With best regards, Tagir Valeev. On Thu, Jun 14, 2018 at 6:11 AM Brian Goetz wrote: > I really like how BiCollector can be private, so all we'd have to expose > is toBoth(), and the arguments of toBoth() are just ordinary > collectors. So no new types or abstractions; just a multiplexing. > > The use of Map.Entry as a pair surrogate is unfortunate -- its > definitely not an Entry -- though I understand why you did this. I'm not > sure if this is fatal or not for a JDK method, but it's pretty bad*. > (You could generalize to n-ary, returning a List or array (you can take > a varargs of Collector), at the loss of sharp types for the results.) > > > *Yes, I'm sure one can find precedent of this being done; this has no > effect on whether it's bad. > > On 6/11/2018 8:39 AM, Peter Levart wrote: > > Hi, > > > > Have you ever wanted to perform a collection of the same Stream into > > two different targets using two Collectors? Say you wanted to collect > > Map.Entry elements into two parallel lists, each of them containing > > keys and values respectively. Or you wanted to collect elements into > > groups by some key, but also count them at the same time? Currently > > this is not possible to do with a single Stream. You have to create > > two identical streams, so you end up passing Supplier to other > > methods instead of bare Stream. > > > > I created a little utility Collector implementation that serves the > > purpose quite well: > > > > /** > > * A {@link Collector} implementation taking two delegate Collector(s) > > and producing result composed > > * of two results produced by delegating collectors, wrapped in {@link > > Map.Entry} object. > > * > > * @param the type of elements collected > > * @param the type of 1st delegate collector collected result > > * @param tye type of 2nd delegate collector collected result > > */ > > public class BiCollector implements Collector > Map.Entry, Map.Entry> { > > private final Collector keyCollector; > > private final Collector valCollector; > > > > @SuppressWarnings("unchecked") > > public BiCollector(Collector keyCollector, Collector > ?, V> valCollector) { > > this.keyCollector = (Collector) > > Objects.requireNonNull(keyCollector); > > this.valCollector = (Collector) > > Objects.requireNonNull(valCollector); > > } > > > > @Override > > public Supplier> supplier() { > > Supplier keySupplier = keyCollector.supplier(); > > Supplier valSupplier = valCollector.supplier(); > > return () -> new > > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > > } > > > > @Override > > public BiConsumer, T> accumulator() { > > BiConsumer keyAccumulator = > > keyCollector.accumulator(); > > BiConsumer valAccumulator = > > valCollector.accumulator(); > > return (accumulation, t) -> { > > keyAccumulator.accept(accumulation.getKey(), t); > > valAccumulator.accept(accumulation.getValue(), t); > > }; > > } > > > > @Override > > public BinaryOperator> combiner() { > > BinaryOperator keyCombiner = keyCollector.combiner(); > > BinaryOperator valCombiner = valCollector.combiner(); > > return (accumulation1, accumulation2) -> new > > AbstractMap.SimpleImmutableEntry<>( > > keyCombiner.apply(accumulation1.getKey(), > > accumulation2.getKey()), > > valCombiner.apply(accumulation1.getValue(), > > accumulation2.getValue()) > > ); > > } > > > > @Override > > public Function, Map.Entry> > > finisher() { > > Function keyFinisher = keyCollector.finisher(); > > Function valFinisher = valCollector.finisher(); > > return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > > keyFinisher.apply(accumulation.getKey()), > > valFinisher.apply(accumulation.getValue()) > > ); > > } > > > > @Override > > public Set characteristics() { > > EnumSet intersection = > > EnumSet.copyOf(keyCollector.characteristics()); > > intersection.retainAll(valCollector.characteristics()); > > return intersection; > > } > > } > > > > > > Do you think this class is general enough to be part of standard > > Collectors repertoire? > > > > For example, accessed via factory method Collectors.toBoth(Collector > > coll1, Collector coll2), bi-collection could then be coded simply as: > > > > Map map = ... > > > > Map.Entry, List> keys_values = > > map.entrySet() > > .stream() > > .collect( > > toBoth( > > mapping(Map.Entry::getKey, toList()), > > mapping(Map.Entry::getValue, toList()) > > ) > > ); > > > > > > Map.Entry, Long> histogram_count = > > ThreadLocalRandom > > .current() > > .ints(100, 0, 10) > > .boxed() > > .collect( > > toBoth( > > groupingBy(Function.identity(), counting()), > > counting() > > ) > > ); > > > > > > Regards, Peter > > > > From peter.levart at gmail.com Thu Jun 14 06:53:50 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 14 Jun 2018 08:53:50 +0200 Subject: BiCollector In-Reply-To: References: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> Message-ID: <6890a641-6440-6cac-3c4d-80e0348b6cf5@gmail.com> Hi Tagir, Great! I've been thinking about the same trick - just add another function and you don't have to hard code the decision about the choice of imperfect tuple type. But are we just talking of aestetics here? In the absence of value types, every choice is imperfect. Ok, sometimes you may have luck to be able to further reduce the result into a singular type, but how often does that occur? I'm not so concerned about the choice of tuple type but about another aspect. The characteristics of a combined collector can only be an intersection of characteristics of individual collectors (minus IDENTITY_FINISH, when you provide the finisher/combiner of the results). If one of the collectors is CONCURRENT, but the other is not, then both will be used in non-CONCURRENT collection mode, which might be sub-optimal. But such is the nature of "single pass". You can't travel to the destination via two distinct paths at the same time (if you're not doing quantum superposition :-). I was positively surprised that such composition is even possible in such a simple way, which speaks of the well thought out initial design of the API. So I guess that although a very useful tool in practice, pairing/bi-collection will remain in the domain of custom Stream extensions. Or maybe not? Regards, Peter On 06/14/18 06:56, Tagir Valeev wrote: > Hello! > > In my StreamEx library I created a "pairing" collector which does > similar job, but allows user to decide how to combine the results of > two collectors. This adds more flexibility. The signature is like this: > > public static Collector pairing( > ? ? Collector c1, > ? ? Collector c2, > ? ? BiFunction finisher) > > Having such collector, The proposed `toBoth(c1, c2)` can be > implemented as simple as `pairing(c1, c2, Map::entry)`. OTOH if > somebody wants to use their own Pair class, it would be `pairing(c1, > c2, Pair::new)`. > Sometimes you don't need a pair, but can create compound result object > right here. E.g.: > > Collector summing = > Collectors.reducing(BigDecimal.ZERO, BigDecimal::add); > Collector averaging = > ? ? ? ? ? ? pairing(summing, Collectors.counting(), (sum, count) -> > ? ? ? ? ? ? ? ? ? ? sum.divide(BigDecimal.valueOf(count), > RoundingMode.HALF_EVEN)); > > So locking to Map.Entry is entirely unnecessary. > > With best regards, > Tagir Valeev. > > > > On Thu, Jun 14, 2018 at 6:11 AM Brian Goetz > wrote: > > I really like how BiCollector can be private, so all we'd have to > expose > is toBoth(), and the arguments of toBoth() are just ordinary > collectors.? So no new types or abstractions; just a multiplexing. > > The use of Map.Entry as a pair surrogate is unfortunate -- its > definitely not an Entry -- though I understand why you did this. > I'm not > sure if this is fatal or not for a JDK method, but it's pretty bad*. > (You could generalize to n-ary, returning a List or array (you can > take > a varargs of Collector), at the loss of sharp types for the results.) > > > *Yes, I'm sure one can find precedent of this being done; this has no > effect on whether it's bad. > > On 6/11/2018 8:39 AM, Peter Levart wrote: > > Hi, > > > > Have you ever wanted to perform a collection of the same Stream > into > > two different targets using two Collectors? Say you wanted to > collect > > Map.Entry elements into two parallel lists, each of them containing > > keys and values respectively. Or you wanted to collect elements > into > > groups by some key, but also count them at the same time? Currently > > this is not possible to do with a single Stream. You have to create > > two identical streams, so you end up passing Supplier to > other > > methods instead of bare Stream. > > > > I created a little utility Collector implementation that serves the > > purpose quite well: > > > > /** > > ?* A {@link Collector} implementation taking two delegate > Collector(s) > > and producing result composed > > ?* of two results produced by delegating collectors, wrapped in > {@link > > Map.Entry} object. > > ?* > > ?* @param the type of elements collected > > ?* @param the type of 1st delegate collector collected result > > ?* @param tye type of 2nd delegate collector collected result > > ?*/ > > public class BiCollector implements Collector > Map.Entry, Map.Entry> { > > ??? private final Collector keyCollector; > > ??? private final Collector valCollector; > > > > ??? @SuppressWarnings("unchecked") > > ??? public BiCollector(Collector keyCollector, > Collector > ?, V> valCollector) { > > ??????? this.keyCollector = (Collector) > > Objects.requireNonNull(keyCollector); > > ??????? this.valCollector = (Collector) > > Objects.requireNonNull(valCollector); > > ??? } > > > > ??? @Override > > ??? public Supplier> supplier() { > > ??????? Supplier keySupplier = keyCollector.supplier(); > > ??????? Supplier valSupplier = valCollector.supplier(); > > ??????? return () -> new > > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), > valSupplier.get()); > > ??? } > > > > ??? @Override > > ??? public BiConsumer, T> accumulator() { > > ??????? BiConsumer keyAccumulator = > > keyCollector.accumulator(); > > ??????? BiConsumer valAccumulator = > > valCollector.accumulator(); > > ??????? return (accumulation, t) -> { > > ??????????? keyAccumulator.accept(accumulation.getKey(), t); > > valAccumulator.accept(accumulation.getValue(), t); > > ??????? }; > > ??? } > > > > ??? @Override > > ??? public BinaryOperator> combiner() { > > ??????? BinaryOperator keyCombiner = > keyCollector.combiner(); > > ??????? BinaryOperator valCombiner = > valCollector.combiner(); > > ??????? return (accumulation1, accumulation2) -> new > > AbstractMap.SimpleImmutableEntry<>( > > ??????????? keyCombiner.apply(accumulation1.getKey(), > > accumulation2.getKey()), > > ??????????? valCombiner.apply(accumulation1.getValue(), > > accumulation2.getValue()) > > ??????? ); > > ??? } > > > > ??? @Override > > ??? public Function, Map.Entry> > > finisher() { > > ??????? Function keyFinisher = keyCollector.finisher(); > > ??????? Function valFinisher = valCollector.finisher(); > > ??????? return accumulation -> new > AbstractMap.SimpleImmutableEntry<>( > > ??????????? keyFinisher.apply(accumulation.getKey()), > > ??????????? valFinisher.apply(accumulation.getValue()) > > ??????? ); > > ??? } > > > > ??? @Override > > ??? public Set characteristics() { > > ??????? EnumSet intersection = > > EnumSet.copyOf(keyCollector.characteristics()); > > intersection.retainAll(valCollector.characteristics()); > > ??????? return intersection; > > ??? } > > } > > > > > > Do you think this class is general enough to be part of standard > > Collectors repertoire? > > > > For example, accessed via factory method > Collectors.toBoth(Collector > > coll1, Collector coll2), bi-collection could then be coded > simply as: > > > > ??????? Map map = ... > > > > ??????? Map.Entry, List> keys_values = > > ??????????? map.entrySet() > > ?????????????? .stream() > > ?????????????? .collect( > > ?????????????????? toBoth( > > ?????????????????????? mapping(Map.Entry::getKey, toList()), > > ?????????????????????? mapping(Map.Entry::getValue, toList()) > > ?????????????????? ) > > ?????????????? ); > > > > > > ??????? Map.Entry, Long> histogram_count = > > ??????????? ThreadLocalRandom > > ??????????????? .current() > > ??????????????? .ints(100, 0, 10) > > ??????????????? .boxed() > > ??????????????? .collect( > > ??????????????????? toBoth( > > ??????????????????????? groupingBy(Function.identity(), counting()), > > ??????????????????????? counting() > > ??????????????????? ) > > ??????????????? ); > > > > > > Regards, Peter > > > From peter.levart at gmail.com Thu Jun 14 07:29:16 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 14 Jun 2018 09:29:16 +0200 Subject: BiCollector In-Reply-To: References: Message-ID: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> Hi Paul, On 06/11/18 19:10, Paul Sandoz wrote: > Hi Peter, > > I like it and can see it being useful, thanks for sharing. > > I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. > > What are you thoughts on this? Well, I don't see the need to pack the two results into a Map.Entry (or any similar) container as a drawback. It's not a performance drawback for sure, because this is not happening on the stream-element scale, but on the final result or intermediate accumulation results scale (the later only in parallel non-CONCURRENT scenario). In non-parallel scenario, only a single (or two for non-IDENTITY_FINISH) Map.Entry objects are created. I also don't see a larger abstraction like BiStream as a natural fit for a similar thing. As I understand BiStream attempts (maybe I haven't seen the right ones?), they are more about passing pairs of elements down the pipeline. BiCollector OTOH is "splitting" the single element pipeline at the final collection stage, with the purpose of constructing two independent collections from a single pass of the original single-element Stream. This is more about single pass than anything else. And single pass, I think, is inevitably such that it can only execute a single collection strategy (CONCURRENT vs. not CONCURRENT), regardless of the type of the stream (Stream vs. BiStream). Or have you prototyped a combined strategy in BiStream? Regards, Peter > FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? > > Paul. > > > > >> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >> >> Hi, >> >> Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. >> >> I created a little utility Collector implementation that serves the purpose quite well: >> >> /** >> * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed >> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. >> * >> * @param the type of elements collected >> * @param the type of 1st delegate collector collected result >> * @param tye type of 2nd delegate collector collected result >> */ >> public class BiCollector implements Collector, Map.Entry> { >> private final Collector keyCollector; >> private final Collector valCollector; >> >> @SuppressWarnings("unchecked") >> public BiCollector(Collector keyCollector, Collector valCollector) { >> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >> } >> >> @Override >> public Supplier> supplier() { >> Supplier keySupplier = keyCollector.supplier(); >> Supplier valSupplier = valCollector.supplier(); >> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >> } >> >> @Override >> public BiConsumer, T> accumulator() { >> BiConsumer keyAccumulator = keyCollector.accumulator(); >> BiConsumer valAccumulator = valCollector.accumulator(); >> return (accumulation, t) -> { >> keyAccumulator.accept(accumulation.getKey(), t); >> valAccumulator.accept(accumulation.getValue(), t); >> }; >> } >> >> @Override >> public BinaryOperator> combiner() { >> BinaryOperator keyCombiner = keyCollector.combiner(); >> BinaryOperator valCombiner = valCollector.combiner(); >> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >> ); >> } >> >> @Override >> public Function, Map.Entry> finisher() { >> Function keyFinisher = keyCollector.finisher(); >> Function valFinisher = valCollector.finisher(); >> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> keyFinisher.apply(accumulation.getKey()), >> valFinisher.apply(accumulation.getValue()) >> ); >> } >> >> @Override >> public Set characteristics() { >> EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); >> intersection.retainAll(valCollector.characteristics()); >> return intersection; >> } >> } >> >> >> Do you think this class is general enough to be part of standard Collectors repertoire? >> >> For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: >> >> Map map = ... >> >> Map.Entry, List> keys_values = >> map.entrySet() >> .stream() >> .collect( >> toBoth( >> mapping(Map.Entry::getKey, toList()), >> mapping(Map.Entry::getValue, toList()) >> ) >> ); >> >> >> Map.Entry, Long> histogram_count = >> ThreadLocalRandom >> .current() >> .ints(100, 0, 10) >> .boxed() >> .collect( >> toBoth( >> groupingBy(Function.identity(), counting()), >> counting() >> ) >> ); >> >> >> Regards, Peter >> From peter.levart at gmail.com Thu Jun 14 07:46:31 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 14 Jun 2018 09:46:31 +0200 Subject: BiCollector In-Reply-To: References: Message-ID: <496d36b1-9b05-d299-91fc-457f1d5a6481@gmail.com> Hi Maurizio, Collectors.partitioningBy is only superficially similar. It partitions the stream into two disjunct sub-streams of elements and collects each of them using the same collector. BiCollector OTOH collects all the elemets of the original Stream "two times" into two independent collections (in general using two different collectors). So result of partitioningBy and toBoth are usually totally different. The features of toBoth allow constructing an equivalent case of partitioningBy: Stream stream = ...; Predicate predicate = ...; Collector downstreamCollector = ...; stream.collect( ??? partitioningBy(predicate, downstreamCollector) ); gives equivalent results as: stream.collect( ??? toBoth( ??? ??? filtering(predicate, downstreamCollector), ??? ??? filtering(predicate.negate, downstreamCollector) ??? ) } But there's no way to use partitioningBy to simulate all the variants of toBoth... Regards, Peter On 06/11/18 19:32, Maurizio Cimadamore wrote: > Note also that this has some overlappings with > Collectors.partitioningBy - which currently wraps results into a > Map, where O is the desired collector output type. Without > commenting on the feasibility of its inclusion in the JDK (Paul rules > here :-)), I note that BiStream would obviously allow this > functionality to be exposed in a more user friendly way. > > Cheers > Maurizio > > > On 11/06/18 13:39, Peter Levart wrote: >> Hi, >> >> Have you ever wanted to perform a collection of the same Stream into >> two different targets using two Collectors? Say you wanted to collect >> Map.Entry elements into two parallel lists, each of them containing >> keys and values respectively. Or you wanted to collect elements into? >> groups by some key, but also count them at the same time? Currently >> this is not possible to do with a single Stream. You have to create >> two identical streams, so you end up passing Supplier to >> other methods instead of bare Stream. >> >> I created a little utility Collector implementation that serves the >> purpose quite well: >> >> /** >> ?* A {@link Collector} implementation taking two delegate >> Collector(s) and producing result composed >> ?* of two results produced by delegating collectors, wrapped in >> {@link Map.Entry} object. >> ?* >> ?* @param the type of elements collected >> ?* @param the type of 1st delegate collector collected result >> ?* @param tye type of 2nd delegate collector collected result >> ?*/ >> public class BiCollector implements Collector> Map.Entry, Map.Entry> { >> ??? private final Collector keyCollector; >> ??? private final Collector valCollector; >> >> ??? @SuppressWarnings("unchecked") >> ??? public BiCollector(Collector keyCollector, Collector> ?, V> valCollector) { >> ??????? this.keyCollector = (Collector) >> Objects.requireNonNull(keyCollector); >> ??????? this.valCollector = (Collector) >> Objects.requireNonNull(valCollector); >> ??? } >> >> ??? @Override >> ??? public Supplier> supplier() { >> ??????? Supplier keySupplier = keyCollector.supplier(); >> ??????? Supplier valSupplier = valCollector.supplier(); >> ??????? return () -> new >> AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), >> valSupplier.get()); >> ??? } >> >> ??? @Override >> ??? public BiConsumer, T> accumulator() { >> ??????? BiConsumer keyAccumulator = >> keyCollector.accumulator(); >> ??????? BiConsumer valAccumulator = >> valCollector.accumulator(); >> ??????? return (accumulation, t) -> { >> ??????????? keyAccumulator.accept(accumulation.getKey(), t); >> ??????????? valAccumulator.accept(accumulation.getValue(), t); >> ??????? }; >> ??? } >> >> ??? @Override >> ??? public BinaryOperator> combiner() { >> ??????? BinaryOperator keyCombiner = keyCollector.combiner(); >> ??????? BinaryOperator valCombiner = valCollector.combiner(); >> ??????? return (accumulation1, accumulation2) -> new >> AbstractMap.SimpleImmutableEntry<>( >> ??????????? keyCombiner.apply(accumulation1.getKey(), >> accumulation2.getKey()), >> ??????????? valCombiner.apply(accumulation1.getValue(), >> accumulation2.getValue()) >> ??????? ); >> ??? } >> >> ??? @Override >> ??? public Function, Map.Entry> >> finisher() { >> ??????? Function keyFinisher = keyCollector.finisher(); >> ??????? Function valFinisher = valCollector.finisher(); >> ??????? return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> ??????????? keyFinisher.apply(accumulation.getKey()), >> ??????????? valFinisher.apply(accumulation.getValue()) >> ??????? ); >> ??? } >> >> ??? @Override >> ??? public Set characteristics() { >> ??????? EnumSet intersection = >> EnumSet.copyOf(keyCollector.characteristics()); >> ??????? intersection.retainAll(valCollector.characteristics()); >> ??????? return intersection; >> ??? } >> } >> >> >> Do you think this class is general enough to be part of standard >> Collectors repertoire? >> >> For example, accessed via factory method Collectors.toBoth(Collector >> coll1, Collector coll2), bi-collection could then be coded simply as: >> >> ??????? Map map = ... >> >> ??????? Map.Entry, List> keys_values = >> ??????????? map.entrySet() >> ?????????????? .stream() >> ?????????????? .collect( >> ?????????????????? toBoth( >> ?????????????????????? mapping(Map.Entry::getKey, toList()), >> ?????????????????????? mapping(Map.Entry::getValue, toList()) >> ?????????????????? ) >> ?????????????? ); >> >> >> ??????? Map.Entry, Long> histogram_count = >> ??????????? ThreadLocalRandom >> ??????????????? .current() >> ??????????????? .ints(100, 0, 10) >> ??????????????? .boxed() >> ??????????????? .collect( >> ??????????????????? toBoth( >> ??????????????????????? groupingBy(Function.identity(), counting()), >> ??????????????????????? counting() >> ??????????????????? ) >> ??????????????? ); >> >> >> Regards, Peter >> > From magnus.ihse.bursie at oracle.com Thu Jun 14 09:53:21 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 14 Jun 2018 11:53:21 +0200 Subject: RFR: 8204967: Resolve disabled warnings for libunpack In-Reply-To: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> References: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> Message-ID: <677ede96-2e3f-a6a0-19f9-ef8d18246a3c@oracle.com> On 2018-06-13 18:57, Srinivas Dama wrote: > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8204967 Hi Srinivas, In src/jdk.pack/share/native/common-unpack/zip.cpp, you just read and discard the return value from fread to hide the warning. But the purpose of the warning is to point out that this kind of code is incorrect. The proper fix would be to check the return code from fread to make sure that the read did not fail. Otherwise you're just making it impossible for the compiler to point out the badly written code, and if you're not going to fix this properly, it's better to keep the warning (that way we know the code is fishy). /Magnus > > Regards, > Srinivas From patrick at reini.net Thu Jun 14 10:50:34 2018 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 14 Jun 2018 12:50:34 +0200 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> Message-ID: <315f23c1dddb0724908fd22134021316@reini.net> Hi everybody, I re-proposed the CSR with the changed API. Somehow I'm not able to add the comment explaning the change. (Tried both in proposed and draft state) :-( I also updated the webrev to represent the two small changes as proposed from Brian. -Patrick On 2018-06-13 23:16, Brian Burkhalter wrote: > Hi Roger, > > Thanks for pointing this out: simpler and cleaner. > > Brian > > On Jun 13, 2018, at 2:06 PM, Roger Riggs > wrote: > >> For the CSR, the easiest path to clarify the spec is to withdraw the >> CSR, update the spec, >> and add a note on the revised behavior. >> >> Then finalize the CSR again. That's enough to get it reviewed and >> approved. >> >> (Using a new CSR would just spread the behavior over two CSRs). From Alan.Bateman at oracle.com Thu Jun 14 11:03:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 12:03:04 +0100 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <315f23c1dddb0724908fd22134021316@reini.net> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> <315f23c1dddb0724908fd22134021316@reini.net> Message-ID: <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> On 14/06/2018 11:50, Patrick Reinhart wrote: > Hi everybody, > > I re-proposed the CSR with the changed API. The CSR is associated with the original change-set that was pushed a few months so the simplest is to just create a follow-up CSR that is linked to the new bug. > Somehow I'm not able to add the comment explaning the change. (Tried > both in proposed and draft state) :-( Yeah, it's annoying, it seems have come with a recent JIRA update. > > I also updated the webrev to represent the two small changes as > proposed from Brian. From patrick at reini.net Thu Jun 14 11:08:20 2018 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 14 Jun 2018 13:08:20 +0200 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> <315f23c1dddb0724908fd22134021316@reini.net> <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> Message-ID: <67efb7a80aa9c47aa0b7c7c8b1cd4f5e@reini.net> Sorry, everybody it seems, that I completely messed things up :-(( @Alan can you help me getting that back in order again? -Patrick On 2018-06-14 13:03, Alan Bateman wrote: > On 14/06/2018 11:50, Patrick Reinhart wrote: >> Hi everybody, >> >> I re-proposed the CSR with the changed API. > The CSR is associated with the original change-set that was pushed a > few months so the simplest is to just create a follow-up CSR that is > linked to the new bug. > > >> Somehow I'm not able to add the comment explaning the change. (Tried >> both in proposed and draft state) :-( > Yeah, it's annoying, it seems have come with a recent JIRA update. > > >> >> I also updated the webrev to represent the two small changes as >> proposed from Brian. From Alan.Bateman at oracle.com Thu Jun 14 11:30:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 12:30:29 +0100 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B1FFA39.9000704@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> Message-ID: <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> On 12/06/2018 17:52, Joe Wang wrote: > Hi all, > > It's been a while since the 1st round of reviews. Thank you all for > the valuable comments and suggestions!? Note that Roger's last > response went to nio-dev, but not core-libs-dev, I've therefore > attached it below. > > CSR [2]: the CSR is now approved. Note the write method has been > changed to writeString. > > Impl [3]: for performance reason and the different requirement from > byte-string conversion for malformed/unmappable character handing, the > implementation uses a specific method separate from the existing ones. > Both Sherman and Alan preferred specific method than adding parameters > to the APIs that might be error prone. Thanks Sherman for the code! > > JMH benchmark tests showed the new read method performs better than an > operation that reads the bytes and convert to string. The gains start > to be significant even at quite reasonable sizes (< 10K). > > > [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 > [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 > [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ > > Please review. The javadoc looks good. One part of the implementation that could be cleaner is the exception thrown for the malformed or unmappable cases. There are sub-classes of CharacterCodingException defined for this that would be clearer than an IOException with an IllegalArgumentException as cause. A minor nit in Files is that you can import jdk.internal.misc.JavaLangAccess rather than repeating the fully qualified class name. You can also move the definition of JLA up to the top. There's also a few inconsistencies with the existing code that would be goof to fix too (indenting and line length issues mostly). The test looks reasonable. In getData() then then "shouldn't happen" case should throw an exception as a NPE here might be tricky to diagnose there. Another nit is the sb field - can that be removed. -Alan From Alan.Bateman at oracle.com Thu Jun 14 12:39:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 13:39:24 +0100 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <67efb7a80aa9c47aa0b7c7c8b1cd4f5e@reini.net> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> <315f23c1dddb0724908fd22134021316@reini.net> <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> <67efb7a80aa9c47aa0b7c7c8b1cd4f5e@reini.net> Message-ID: <0f336fa8-9cde-1943-58b7-d751234c453d@oracle.com> On 14/06/2018 12:08, Patrick Reinhart wrote: > > Sorry, everybody it seems, that I completely messed things up :-(( > > @Alan can you help me getting that back in order again? Sure. BTW: I should have mentioned that the main workaround that people are using is to edit the CSR as that allows you to add a comment, even if you don't touch other fields. Hopefully the JIRA bug will get fixed soon. From scolebourne at joda.org Thu Jun 14 12:56:14 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 14 Jun 2018 13:56:14 +0100 Subject: BiCollector In-Reply-To: References: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> Message-ID: In this form (not locked to Map.Entry) it seems like a candidate for the JDK. The implementation is private, so it would be one public method that provides a new way to process streams. Stephen On 14 June 2018 at 05:56, Tagir Valeev wrote: > Hello! > > In my StreamEx library I created a "pairing" collector which does similar > job, but allows user to decide how to combine the results of two > collectors. This adds more flexibility. The signature is like this: > > public static Collector pairing( > Collector c1, > Collector c2, > BiFunction finisher) > > Having such collector, The proposed `toBoth(c1, c2)` can be implemented as > simple as `pairing(c1, c2, Map::entry)`. OTOH if somebody wants to use > their own Pair class, it would be `pairing(c1, c2, Pair::new)`. > Sometimes you don't need a pair, but can create compound result object > right here. E.g.: > > Collector summing = > Collectors.reducing(BigDecimal.ZERO, BigDecimal::add); > Collector averaging = > pairing(summing, Collectors.counting(), (sum, count) -> > sum.divide(BigDecimal.valueOf(count), > RoundingMode.HALF_EVEN)); > > So locking to Map.Entry is entirely unnecessary. > > With best regards, > Tagir Valeev. > > > > On Thu, Jun 14, 2018 at 6:11 AM Brian Goetz wrote: > >> I really like how BiCollector can be private, so all we'd have to expose >> is toBoth(), and the arguments of toBoth() are just ordinary >> collectors. So no new types or abstractions; just a multiplexing. >> >> The use of Map.Entry as a pair surrogate is unfortunate -- its >> definitely not an Entry -- though I understand why you did this. I'm not >> sure if this is fatal or not for a JDK method, but it's pretty bad*. >> (You could generalize to n-ary, returning a List or array (you can take >> a varargs of Collector), at the loss of sharp types for the results.) >> >> >> *Yes, I'm sure one can find precedent of this being done; this has no >> effect on whether it's bad. >> >> On 6/11/2018 8:39 AM, Peter Levart wrote: >> > Hi, >> > >> > Have you ever wanted to perform a collection of the same Stream into >> > two different targets using two Collectors? Say you wanted to collect >> > Map.Entry elements into two parallel lists, each of them containing >> > keys and values respectively. Or you wanted to collect elements into >> > groups by some key, but also count them at the same time? Currently >> > this is not possible to do with a single Stream. You have to create >> > two identical streams, so you end up passing Supplier to other >> > methods instead of bare Stream. >> > >> > I created a little utility Collector implementation that serves the >> > purpose quite well: >> > >> > /** >> > * A {@link Collector} implementation taking two delegate Collector(s) >> > and producing result composed >> > * of two results produced by delegating collectors, wrapped in {@link >> > Map.Entry} object. >> > * >> > * @param the type of elements collected >> > * @param the type of 1st delegate collector collected result >> > * @param tye type of 2nd delegate collector collected result >> > */ >> > public class BiCollector implements Collector> > Map.Entry, Map.Entry> { >> > private final Collector keyCollector; >> > private final Collector valCollector; >> > >> > @SuppressWarnings("unchecked") >> > public BiCollector(Collector keyCollector, Collector> > ?, V> valCollector) { >> > this.keyCollector = (Collector) >> > Objects.requireNonNull(keyCollector); >> > this.valCollector = (Collector) >> > Objects.requireNonNull(valCollector); >> > } >> > >> > @Override >> > public Supplier> supplier() { >> > Supplier keySupplier = keyCollector.supplier(); >> > Supplier valSupplier = valCollector.supplier(); >> > return () -> new >> > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >> > } >> > >> > @Override >> > public BiConsumer, T> accumulator() { >> > BiConsumer keyAccumulator = >> > keyCollector.accumulator(); >> > BiConsumer valAccumulator = >> > valCollector.accumulator(); >> > return (accumulation, t) -> { >> > keyAccumulator.accept(accumulation.getKey(), t); >> > valAccumulator.accept(accumulation.getValue(), t); >> > }; >> > } >> > >> > @Override >> > public BinaryOperator> combiner() { >> > BinaryOperator keyCombiner = keyCollector.combiner(); >> > BinaryOperator valCombiner = valCollector.combiner(); >> > return (accumulation1, accumulation2) -> new >> > AbstractMap.SimpleImmutableEntry<>( >> > keyCombiner.apply(accumulation1.getKey(), >> > accumulation2.getKey()), >> > valCombiner.apply(accumulation1.getValue(), >> > accumulation2.getValue()) >> > ); >> > } >> > >> > @Override >> > public Function, Map.Entry> >> > finisher() { >> > Function keyFinisher = keyCollector.finisher(); >> > Function valFinisher = valCollector.finisher(); >> > return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> > keyFinisher.apply(accumulation.getKey()), >> > valFinisher.apply(accumulation.getValue()) >> > ); >> > } >> > >> > @Override >> > public Set characteristics() { >> > EnumSet intersection = >> > EnumSet.copyOf(keyCollector.characteristics()); >> > intersection.retainAll(valCollector.characteristics()); >> > return intersection; >> > } >> > } >> > >> > >> > Do you think this class is general enough to be part of standard >> > Collectors repertoire? >> > >> > For example, accessed via factory method Collectors.toBoth(Collector >> > coll1, Collector coll2), bi-collection could then be coded simply as: >> > >> > Map map = ... >> > >> > Map.Entry, List> keys_values = >> > map.entrySet() >> > .stream() >> > .collect( >> > toBoth( >> > mapping(Map.Entry::getKey, toList()), >> > mapping(Map.Entry::getValue, toList()) >> > ) >> > ); >> > >> > >> > Map.Entry, Long> histogram_count = >> > ThreadLocalRandom >> > .current() >> > .ints(100, 0, 10) >> > .boxed() >> > .collect( >> > toBoth( >> > groupingBy(Function.identity(), counting()), >> > counting() >> > ) >> > ); >> > >> > >> > Regards, Peter >> > >> >> From rachna.goel at oracle.com Thu Jun 14 14:01:55 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Thu, 14 Jun 2018 19:31:55 +0530 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> Message-ID: <2cc825ca-35e3-de95-e233-4444158e32c2@oracle.com> Hi Naoto, Thanks a lot for the review. I have made suggested changes, Kindly have a look at : http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.07/ - Updated NumberingSystemsParseHandler.java - Updated LocaleData.cldr for new test case. Thanks, Rachna On 6/13/18 10:33 PM, naoto.sato at oracle.com wrote: > Hi Rachna, > > A couple of comments: > > - NumberingSystemsParseHandler.java > > Since the code substitutes latin digits for supplementary digits, it > can skip line 68-79. > > - A test should be written for the above substitution. > > Naoto > > On 6/12/18 10:33 PM, Rachna Goel wrote: >> Hi, >> >> Kindly review fix for JDK-8202537. Fix is to upgrade Unicode >> consortium's CLDR data into JDK from current version 29 to 33. >> >> For more info : http://cldr.unicode.org/index/downloads/cldr-33 >> >> Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 >> >> Patch: >> http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html >> -- Thanks, Rachna From paul.sandoz at oracle.com Thu Jun 14 15:36:16 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 14 Jun 2018 08:36:16 -0700 Subject: BiCollector In-Reply-To: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> Message-ID: <1EF1B940-0C42-4075-8B92-596C181F6295@oracle.com> Hi Peter, I am not concerned about performance of Map.Entry. I find its use awkward for similar reasons as Brian outlined. Tagir?s approach using a finisher nicely side steps this at the expense of another function. If in the future we have an officially blessed pair/2-tuple class we can overload the collector factory method. Regarding BitStream. A Stream could be bisected into a BiStream, then merged back into a Stream or bi-collect (something which i did not get time to look at). Having gone back and looked at the prototype work I tend to see a bisecting collector as complementary in this respect since we have similar operations for collection as we do on Stream. Paul. > On Jun 14, 2018, at 12:29 AM, Peter Levart wrote: > > Hi Paul, > > On 06/11/18 19:10, Paul Sandoz wrote: >> Hi Peter, >> >> I like it and can see it being useful, thanks for sharing. >> >> I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. >> >> What are you thoughts on this? > > Well, I don't see the need to pack the two results into a Map.Entry (or any similar) container as a drawback. It's not a performance drawback for sure, because this is not happening on the stream-element scale, but on the final result or intermediate accumulation results scale (the later only in parallel non-CONCURRENT scenario). In non-parallel scenario, only a single (or two for non-IDENTITY_FINISH) Map.Entry objects are created. > > I also don't see a larger abstraction like BiStream as a natural fit for a similar thing. As I understand BiStream attempts (maybe I haven't seen the right ones?), they are more about passing pairs of elements down the pipeline. BiCollector OTOH is "splitting" the single element pipeline at the final collection stage, with the purpose of constructing two independent collections from a single pass of the original single-element Stream. This is more about single pass than anything else. And single pass, I think, is inevitably such that it can only execute a single collection strategy (CONCURRENT vs. not CONCURRENT), regardless of the type of the stream (Stream vs. BiStream). Or have you prototyped a combined strategy in BiStream? > > Regards, Peter > >> FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? >> >> Paul. >> >> >> >> >>> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >>> >>> Hi, >>> >>> Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. >>> >>> I created a little utility Collector implementation that serves the purpose quite well: >>> >>> /** >>> * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed >>> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. >>> * >>> * @param the type of elements collected >>> * @param the type of 1st delegate collector collected result >>> * @param tye type of 2nd delegate collector collected result >>> */ >>> public class BiCollector implements Collector, Map.Entry> { >>> private final Collector keyCollector; >>> private final Collector valCollector; >>> >>> @SuppressWarnings("unchecked") >>> public BiCollector(Collector keyCollector, Collector valCollector) { >>> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >>> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >>> } >>> >>> @Override >>> public Supplier> supplier() { >>> Supplier keySupplier = keyCollector.supplier(); >>> Supplier valSupplier = valCollector.supplier(); >>> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >>> } >>> >>> @Override >>> public BiConsumer, T> accumulator() { >>> BiConsumer keyAccumulator = keyCollector.accumulator(); >>> BiConsumer valAccumulator = valCollector.accumulator(); >>> return (accumulation, t) -> { >>> keyAccumulator.accept(accumulation.getKey(), t); >>> valAccumulator.accept(accumulation.getValue(), t); >>> }; >>> } >>> >>> @Override >>> public BinaryOperator> combiner() { >>> BinaryOperator keyCombiner = keyCollector.combiner(); >>> BinaryOperator valCombiner = valCollector.combiner(); >>> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >>> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >>> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >>> ); >>> } >>> >>> @Override >>> public Function, Map.Entry> finisher() { >>> Function keyFinisher = keyCollector.finisher(); >>> Function valFinisher = valCollector.finisher(); >>> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >>> keyFinisher.apply(accumulation.getKey()), >>> valFinisher.apply(accumulation.getValue()) >>> ); >>> } >>> >>> @Override >>> public Set characteristics() { >>> EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); >>> intersection.retainAll(valCollector.characteristics()); >>> return intersection; >>> } >>> } >>> >>> >>> Do you think this class is general enough to be part of standard Collectors repertoire? >>> >>> For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: >>> >>> Map map = ... >>> >>> Map.Entry, List> keys_values = >>> map.entrySet() >>> .stream() >>> .collect( >>> toBoth( >>> mapping(Map.Entry::getKey, toList()), >>> mapping(Map.Entry::getValue, toList()) >>> ) >>> ); >>> >>> >>> Map.Entry, Long> histogram_count = >>> ThreadLocalRandom >>> .current() >>> .ints(100, 0, 10) >>> .boxed() >>> .collect( >>> toBoth( >>> groupingBy(Function.identity(), counting()), >>> counting() >>> ) >>> ); >>> >>> >>> Regards, Peter >>> > From brian.goetz at oracle.com Thu Jun 14 17:04:18 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 14 Jun 2018 13:04:18 -0400 Subject: BiCollector In-Reply-To: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> Message-ID: <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> > Well, I don't see the need to pack the two results into a Map.Entry > (or any similar) container as a drawback. From an "integrity of the JDK APIs" perspective, it is unquestionably a drawback.? These items are not a Key and an associated Value in a Map; it's merely pretending that Map.Entry really means "Pair".? There's a reason we don't have a Pair class in the JDK (and no, let's not reopen that now); using something else as a Pair proxy that is supposed to have specific semantics is worse. (It's fine to do this in your own code, but not in the JDK. Different standards for code that has different audiences.) Tagir's proposed sidestepping is nice, and it will also play nicely with records, because then you can say: ???? record NameAndCount(String name, int count); ???? stream.collect(pairing(collectName, collectCount, NameAndCount::new)); and get a more properly abstract result out.? And its more in the spirit of existing Collectors.? If you want to use Map.Entry as an _implementation detail_, that's fine. I can support this form. > I also don't see a larger abstraction like BiStream as a natural fit > for a similar thing. I think the BiStream connection is mostly tangential.? We tried hard to support streams of (K,V) pairs when we did streams, as Paul can attest, but it was a huge complexity-inflater and dropping this out paid an enormous simplification payoff. With records, having streams of tuples will be simpler to represent, but no more performant; it will take until we get to value types and specialized generics to get the performance we want out of this. From maurizio.cimadamore at oracle.com Thu Jun 14 17:13:26 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 14 Jun 2018 18:13:26 +0100 Subject: BiCollector In-Reply-To: <496d36b1-9b05-d299-91fc-457f1d5a6481@gmail.com> References: <496d36b1-9b05-d299-91fc-457f1d5a6481@gmail.com> Message-ID: On 14/06/18 08:46, Peter Levart wrote: > But there's no way to use partitioningBy to simulate all the variants > of toBoth... I wasn't suggesting that - more the other way around, as you illustrated Maurizio From vicente.romero at oracle.com Thu Jun 14 17:31:06 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 14 Jun 2018 13:31:06 -0400 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> <5A90B204.4070005@oracle.com> <5A960359.4070507@oracle.com> Message-ID: I'm not an expert in the area, but the patch looks good to me, Vicente On 05/31/2018 08:39 PM, Liam Miller-Cushon wrote: > Hi, > > Are there any concerns with the patch that I can address? I was hoping to > get this in to JDK 11. > > I confirmed that it still builds and passes all tests at head. > > Thanks, > Liam > > On Mon, May 14, 2018 at 10:38 AM Liam Miller-Cushon > wrote: > >> Ping >> >> On Thu, Apr 5, 2018 at 10:57 AM Liam Miller-Cushon >> wrote: >> >>> Hi, >>> >>> Is there any more feedback on the patch? >>> >>> On Wed, Feb 28, 2018 at 4:26 PM Liam Miller-Cushon >>> wrote: >>> >>>> On Wed, Feb 28, 2018 at 4:13 PM, Martin Buchholz >>>> wrote: >>>> >>>>> Follow their lead and rename your method to assertArrayEquals >>>>> >>>> Done: http://cr.openjdk.java.net/~cushon/7183985/webrev.04/ >>>> From paul.sandoz at oracle.com Thu Jun 14 17:41:20 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 14 Jun 2018 10:41:20 -0700 Subject: RFR 8202922 Method reference identity is broken by serialization Message-ID: <89462B7C-BE80-42FA-8D49-DC5737948A01@oracle.com> Hi, Please review an update to the specifications of LambdaMetafactory and SerializedLambda to clarify that the identity of function objects is unpredictable. A similar (informational) clarification was made to the language specification and similar wording is used. Amusingly a quirk of (or issue with) lambda deserialization (we count our blessings it works as it does!) motivated this clarification. http://cr.openjdk.java.net/~psandoz/jdk/JDK-8202922-function-object-identity/webrev/ CSR: https://bugs.openjdk.java.net/browse/JDK-8205060 Thanks, Paul. From erik.joelsson at oracle.com Thu Jun 14 18:52:02 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 14 Jun 2018 11:52:02 -0700 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: References: Message-ID: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> Hello, Here is a new version of the patch: http://cr.openjdk.java.net/~erikj/8204973/webrev.02/ Changes from last time: * Made the regexp for finding locales more correct. It still does not try to match 3 letter language strings because doing so triggers a large amount of false positives in our souce tree. * Added another accepted locale (en_US_POSIX) that is now matched by the improved regexp. * Added more locales to the exclude list as they were now discovered by the improved regexp. /Erik On 2018-06-13 12:47, Erik Joelsson wrote: > Hello, > > Oracle will reduce the number of languages that it maintains > translations of JDK resources for. The current translations will > remain in the source for now, but we need a way to filter out a set of > translations at build time so that we only include the ones we > support. This patch adds such a configuration option. It also changes > how Oracle builds by using the option to exclude all translations > except English, Japanese, Simplified Chinese and Traditional Chinese. > Anyone else building OpenJDK will by default include all translations > present in the source, just as before. > > I added a test that verifies this for builds with the "IMPLEMENTOR" > field in the release file set to "Oracle Corporation". The test will > not be run for other OpenJDK builds. > > I had to modify an existing test for java.logging which used various > translations to verify localized log messages to only use translations > that Oracle chooses to include. > > Since this is the second test that specifically verifies build > behavior, I moved the previous such test together with this new test > into a common top level test directory "build", under the jdk test > root. I put these tests in the jdk tier3 test group. > > I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically > looked for tests that use the java.util.Locale class and ran them > locally. > > Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 > > /Erik > From naoto.sato at oracle.com Thu Jun 14 19:42:54 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 14 Jun 2018 12:42:54 -0700 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: <2cc825ca-35e3-de95-e233-4444158e32c2@oracle.com> References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> <2cc825ca-35e3-de95-e233-4444158e32c2@oracle.com> Message-ID: <7b670a7b-654a-273c-6ef8-50a087006c3b@oracle.com> Looks good to me. Naoto On 6/14/18 7:01 AM, Rachna Goel wrote: > Hi Naoto, > > Thanks a lot for the review. > > I have made suggested changes, Kindly have a look at : > http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.07/ > > - Updated NumberingSystemsParseHandler.java > > - Updated LocaleData.cldr for new test case. > > Thanks, > > Rachna > > > On 6/13/18 10:33 PM, naoto.sato at oracle.com wrote: >> Hi Rachna, >> >> A couple of comments: >> >> - NumberingSystemsParseHandler.java >> >> Since the code substitutes latin digits for supplementary digits, it >> can skip line 68-79. >> >> - A test should be written for the above substitution. >> >> Naoto >> >> On 6/12/18 10:33 PM, Rachna Goel wrote: >>> Hi, >>> >>> Kindly review fix for JDK-8202537. Fix is to upgrade Unicode >>> consortium's CLDR data into JDK from current version 29 to 33. >>> >>> For more info : http://cldr.unicode.org/index/downloads/cldr-33 >>> >>> Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 >>> >>> Patch: >>> http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html >>> > > -- > Thanks, > Rachna > From magnus.ihse.bursie at oracle.com Thu Jun 14 19:49:07 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 14 Jun 2018 21:49:07 +0200 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> References: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> Message-ID: Would it have been to problematic to reverse the the logic, i.e. having a "include translation" rather than an exclude? Feels more brittle; if someone adds a translation the exclude list needs to be updated. Also, a include mechanism could possibly be used, much simpler, by someone who only wants to build e.g. a chinese jdk. But I realize the contraints by the somewhat odd request from the bug report, to deliver just a subset of the available translations. So I'm okay with the patch. /Magnus On 2018-06-14 20:52, Erik Joelsson wrote: > Hello, > > Here is a new version of the patch: > > http://cr.openjdk.java.net/~erikj/8204973/webrev.02/ > > Changes from last time: > > * Made the regexp for finding locales more correct. It still does not > try to match 3 letter language strings because doing so triggers a > large amount of false positives in our souce tree. > > * Added another accepted locale (en_US_POSIX) that is now matched by > the improved regexp. > > * Added more locales to the exclude list as they were now discovered > by the improved regexp. > > /Erik > > > On 2018-06-13 12:47, Erik Joelsson wrote: >> Hello, >> >> Oracle will reduce the number of languages that it maintains >> translations of JDK resources for. The current translations will >> remain in the source for now, but we need a way to filter out a set >> of translations at build time so that we only include the ones we >> support. This patch adds such a configuration option. It also changes >> how Oracle builds by using the option to exclude all translations >> except English, Japanese, Simplified Chinese and Traditional Chinese. >> Anyone else building OpenJDK will by default include all translations >> present in the source, just as before. >> >> I added a test that verifies this for builds with the "IMPLEMENTOR" >> field in the release file set to "Oracle Corporation". The test will >> not be run for other OpenJDK builds. >> >> I had to modify an existing test for java.logging which used various >> translations to verify localized log messages to only use >> translations that Oracle chooses to include. >> >> Since this is the second test that specifically verifies build >> behavior, I moved the previous such test together with this new test >> into a common top level test directory "build", under the jdk test >> root. I put these tests in the jdk tier3 test group. >> >> I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically >> looked for tests that use the java.util.Locale class and ran them >> locally. >> >> Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 >> >> /Erik >> > From naoto.sato at oracle.com Thu Jun 14 20:05:15 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 14 Jun 2018 13:05:15 -0700 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> References: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> Message-ID: Looks good to me. Naoto On 6/14/18 11:52 AM, Erik Joelsson wrote: > Hello, > > Here is a new version of the patch: > > http://cr.openjdk.java.net/~erikj/8204973/webrev.02/ > > Changes from last time: > > * Made the regexp for finding locales more correct. It still does not > try to match 3 letter language strings because doing so triggers a large > amount of false positives in our souce tree. > > * Added another accepted locale (en_US_POSIX) that is now matched by the > improved regexp. > > * Added more locales to the exclude list as they were now discovered by > the improved regexp. > > /Erik > > > On 2018-06-13 12:47, Erik Joelsson wrote: >> Hello, >> >> Oracle will reduce the number of languages that it maintains >> translations of JDK resources for. The current translations will >> remain in the source for now, but we need a way to filter out a set of >> translations at build time so that we only include the ones we >> support. This patch adds such a configuration option. It also changes >> how Oracle builds by using the option to exclude all translations >> except English, Japanese, Simplified Chinese and Traditional Chinese. >> Anyone else building OpenJDK will by default include all translations >> present in the source, just as before. >> >> I added a test that verifies this for builds with the "IMPLEMENTOR" >> field in the release file set to "Oracle Corporation". The test will >> not be run for other OpenJDK builds. >> >> I had to modify an existing test for java.logging which used various >> translations to verify localized log messages to only use translations >> that Oracle chooses to include. >> >> Since this is the second test that specifically verifies build >> behavior, I moved the previous such test together with this new test >> into a common top level test directory "build", under the jdk test >> root. I put these tests in the jdk tier3 test group. >> >> I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically >> looked for tests that use the java.util.Locale class and ran them >> locally. >> >> Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 >> >> /Erik >> > From erik.joelsson at oracle.com Thu Jun 14 20:08:55 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 14 Jun 2018 13:08:55 -0700 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: References: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> Message-ID: <84ffd038-6c14-ef39-6103-686d1e8bb538@oracle.com> On 2018-06-14 12:49, Magnus Ihse Bursie wrote: > Would it have been to problematic to reverse the the logic, i.e. > having a "include translation" rather than an exclude? Feels more > brittle; if someone adds a translation the exclude list needs to be > updated. Also, a include mechanism could possibly be used, much > simpler, by someone who only wants to build e.g. a chinese jdk. > I agree that include would have been easier to use, and I was initially trying to implement it that way, but couldn't come up with an effective solution in make. You need to identify all source files that have locale suffixes and only filter among them. With the exclude, at least the build logic is reasonably straight forward since the user defines the locale strings that define a translation that warrants action. This is why I added a test that uses include logic, hoping that the combination would prove resilient enough to catch such changes. The test is still a bit brittle since it can easily catch false positives. > But I realize the contraints by the somewhat odd request from the bug > report, to deliver just a subset of the available translations. So I'm > okay with the patch. > Thanks! /Erik > /Magnus > > On 2018-06-14 20:52, Erik Joelsson wrote: >> Hello, >> >> Here is a new version of the patch: >> >> http://cr.openjdk.java.net/~erikj/8204973/webrev.02/ >> >> Changes from last time: >> >> * Made the regexp for finding locales more correct. It still does not >> try to match 3 letter language strings because doing so triggers a >> large amount of false positives in our souce tree. >> >> * Added another accepted locale (en_US_POSIX) that is now matched by >> the improved regexp. >> >> * Added more locales to the exclude list as they were now discovered >> by the improved regexp. >> >> /Erik >> >> >> On 2018-06-13 12:47, Erik Joelsson wrote: >>> Hello, >>> >>> Oracle will reduce the number of languages that it maintains >>> translations of JDK resources for. The current translations will >>> remain in the source for now, but we need a way to filter out a set >>> of translations at build time so that we only include the ones we >>> support. This patch adds such a configuration option. It also >>> changes how Oracle builds by using the option to exclude all >>> translations except English, Japanese, Simplified Chinese and >>> Traditional Chinese. Anyone else building OpenJDK will by default >>> include all translations present in the source, just as before. >>> >>> I added a test that verifies this for builds with the "IMPLEMENTOR" >>> field in the release file set to "Oracle Corporation". The test will >>> not be run for other OpenJDK builds. >>> >>> I had to modify an existing test for java.logging which used various >>> translations to verify localized log messages to only use >>> translations that Oracle chooses to include. >>> >>> Since this is the second test that specifically verifies build >>> behavior, I moved the previous such test together with this new test >>> into a common top level test directory "build", under the jdk test >>> root. I put these tests in the jdk tier3 test group. >>> >>> I have run all tier1, 2 and 3 tests in Mach 5 as well as >>> specifically looked for tests that use the java.util.Locale class >>> and ran them locally. >>> >>> Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 >>> >>> /Erik >>> >> > From paul.sandoz at oracle.com Thu Jun 14 21:35:49 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 14 Jun 2018 14:35:49 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal Message-ID: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> Hi, Please review this enhancement to improve the performance of BitSet traversal when using a stream. http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159-bitset-traverse/webrev/ The associated issue started out life referring to the BitSet?s spliterator reporting SIZED, and to report that the cardinality has to be computed. This has some cost but performance analysis (see issue for attached benchmark) has shown that the cost is small compared to the cost of traversal. It is recognized that the cost is higher for smaller BitSets but not unduly so. Thus it was concluded that it was reasonable for the spliterator to report SIZED. The issue was adjusted to focus on improving the performance of the BitSet?s spliterator forEachRemaining method. For large randomized BitSets a 1.6x speedup can be observed, and for smaller sizes a more modest speed up. The prior conclusion about reporting SIZED still holds. Thanks, Paul. From naokikishida at gmail.com Thu Jun 14 12:13:25 2018 From: naokikishida at gmail.com (kishida naoki) Date: Thu, 14 Jun 2018 21:13:25 +0900 Subject: Japanese New Era supporting does not accept Heisei 32 Message-ID: I'm trying to use the Japanese new era supporting and have found that it does not accept Heisei 32. There are many document that include Heisei 32 or later at present. For example, the expiration date of my driver license is Heisei 32 March. JDK10 can accept Heisei 32. I think the change of the behavior make some trouble. Additionally, Japanese government says some system use Heisei after 5/1/2019 for a while. The JDK version that include the new era support can not be used on such system. https://digital.asahi.com/articles/DA3S13491490.html There are 3 ways to support the new era. 1. H32 is not accepted and S90 is not accepted. (current new era support behavior) 2. H32 is accepted and S90 is not accepted. (current JDK behavior) 3. H32 is accepted and S90 is accepted. (to conformity) I prefer 2 not to change the current JDK behavior. With taking 2 or 3, we might have a new API like JapaneseDate.from(JapaneseEra, TemporalAccessor) to specify an era on making JapaneseDate. -- Naoki Kishdia From naoto.sato at oracle.com Thu Jun 14 23:00:45 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 14 Jun 2018 16:00:45 -0700 Subject: Japanese New Era supporting does not accept Heisei 32 In-Reply-To: References: Message-ID: <33afc91d-8a8d-e3fc-0105-9ba4ecd901e8@oracle.com> Hi, Yes, that's the expected behavior with the implementation assuming "Heisei 32" is non-existent. I see some discussion as you mentioned, whether to allow to use Heisei for a while, for transitional purpose, but at the moment there's very little & no official information available. We may consider adding an additional API to convert those dates leniently. Naoto On 6/14/18 5:13 AM, kishida naoki wrote: > I'm trying to use the Japanese new era supporting and have found that it > does not accept Heisei 32. > There are many document that include Heisei 32 or later at present. For > example, the expiration date of my driver license is Heisei 32 March. > JDK10 can accept Heisei 32. > I think the change of the behavior make some trouble. > > Additionally, Japanese government says some system use Heisei after > 5/1/2019 for a while. The JDK version that include the new era support can > not be used on such system. > https://digital.asahi.com/articles/DA3S13491490.html > > There are 3 ways to support the new era. > 1. H32 is not accepted and S90 is not accepted. (current new era support > behavior) > 2. H32 is accepted and S90 is not accepted. (current JDK behavior) > 3. H32 is accepted and S90 is accepted. (to conformity) > > I prefer 2 not to change the current JDK behavior. > > With taking 2 or 3, we might have a new API like > JapaneseDate.from(JapaneseEra, TemporalAccessor) to specify an era on > making JapaneseDate. > From stuart.marks at oracle.com Fri Jun 15 01:02:04 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 14 Jun 2018 18:02:04 -0700 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) Message-ID: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Hi all, I wanted to restart review of the changeset for bug JDK-8060192 [1]. The latest webrev is here: [2]. The previous review was from December 2017: [3]. The only difference between the current changeset and the previous one is the removal of the overriding implementations (see explanation below). Much of the discussion in the previous review thread was around performance and the shape of the API. # Performance I previously had done some benchmarking on this and reported the results: [4]. (I've recently re-done the benchmarks and conclusions are essentially the same.) The upshot is that implementations that use Arrays.copyOf() are the fastest, probably because it's an intrinsic. It can avoid zero-filling the freshly allocated array because it knows the entire array contents will be overwritten. This is true regardless of what the public API looks like. The default implementation calls generator.apply(0) to get a zero-length array and then simply calls toArray(T[]). This goes through the Arrays.copyOf() fast path, so it's essentially the same speed as toArray(new T[0]). The overrides I had previously provided in specific implementation classes like ArrayList actually are slower, because the allocation of the array is done separately from filling it. This necessitates the extra zero-filling step. Thus, I've removed the overrides. # API Shape There was some discussion about whether it would be preferable to have a Class parameter as a type token for the array component type or the array type itself, instead of using an IntFunction generator. Most of this boils down to what people are comfortable with. However, there are a few points that make the generator function approach preferable. One point in favor of using a generator is that Stream already has a similar toArray(generator) method. Comparing this to the type token alternatives, for simple tasks like converting Collection to String[], things are about equal: toArray(String[]::new) toArray(String.class) toArray(String[].class) However, things are hairier if the element type of the collection is generic, such as Collection>. The result type is a generic array List[]. Using class literals or array constructor references will necessitate using an unchecked cast, because none of these can be used to represent a generic type. However, it's possible to use a method reference to provide a properly generic IntFunction. This would enable the toArray(generator) method to be used without any unchecked casts. (The method it refers to might need have an unchecked cast within it, though.) Such a method could also be reused for the corresponding Stream.toArray(generator) method. For these reasons I'd like to proceed with adding toArray(generator) API. Thanks, s'marks [1] https://bugs.openjdk.java.net/browse/JDK-8060192 [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ [3] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html [4] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html From srinivas.dama at oracle.com Fri Jun 15 04:45:52 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 14 Jun 2018 21:45:52 -0700 (PDT) Subject: RFR: 8204967: Resolve disabled warnings for libunpack Message-ID: <0df0b775-3ff8-49fa-81dd-0cfc3b16641e@default> Hi Magnus/Jim, Thank you for the comments. Here is the latest webrev with suggested changes: webrev: http://cr.openjdk.java.net/~sdama/8204967/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8204967 Jim, gnu c supports -Wimplicit-fallthrough=3. clang takes -Wimplicit-fallthrough without any parameters and ignores it without any effect. clang++ works with -Wimplicit-fallthrough along with -std=c++11(it is cpp specific option) Note: My earlier fix http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ broke linux build because some switch case statements which are having implicit-fallthroughs will be enabled only in debug build(src/jdk.pack/share/native/common-unpack/unpack.cpp) Earlier,I missed to fix those so debug build failed in linux. Regards, Srinivas ----- Original Message ----- From: magnus.ihse.bursie at oracle.com To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net Sent: Thursday, 14 June, 2018 3:23:23 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: RFR: 8204967: Resolve disabled warnings for libunpack On 2018-06-13 18:57, Srinivas Dama wrote: > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8204967 Hi Srinivas, In src/jdk.pack/share/native/common-unpack/zip.cpp, you just read and discard the return value from fread to hide the warning. But the purpose of the warning is to point out that this kind of code is incorrect. The proper fix would be to check the return code from fread to make sure that the read did not fail. Otherwise you're just making it impossible for the compiler to point out the badly written code, and if you're not going to fix this properly, it's better to keep the warning (that way we know the code is fishy). /Magnus > > Regards, > Srinivas From patrick at reini.net Fri Jun 15 06:33:17 2018 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 15 Jun 2018 08:33:17 +0200 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> <315f23c1dddb0724908fd22134021316@reini.net> <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> Message-ID: <207c050f941fa12992cfe2fb57d7e98e@reini.net> On 2018-06-14 13:03, Alan Bateman wrote: > On 14/06/2018 11:50, Patrick Reinhart wrote: >> Hi everybody, >> >> I re-proposed the CSR with the changed API. > The CSR is associated with the original change-set that was pushed a > few months so the simplest is to just create a follow-up CSR that is > linked to the new bug. I created a follow-up CSR with relation to the existing one now. In the meantime I got a comment on the initial issue according the specification of the [ ready(), skip(), transferTo() ] where it seen to be unclear what is the expected behavior "if end of stream reached" exactly means. Any suggestions on that part? Should it state that ready() will always return false, skip() and transferTo() always return zero? -Patrick From forax at univ-mlv.fr Fri Jun 15 07:26:59 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 15 Jun 2018 09:26:59 +0200 (CEST) Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: <1255272034.1809068.1529047619362.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stuart Marks" > ?: "core-libs-dev" > Envoy?: Vendredi 15 Juin 2018 03:02:04 > Objet: RFR(s): 8060192: Add default method Collection.toArray(generator) > Hi all, Hi Stuart, > > I wanted to restart review of the changeset for bug JDK-8060192 [1]. The latest > webrev is here: [2]. > > The previous review was from December 2017: [3]. The only difference between the > current changeset and the previous one is the removal of the overriding > implementations (see explanation below). Much of the discussion in the previous > review thread was around performance and the shape of the API. > > # Performance > > I previously had done some benchmarking on this and reported the results: [4]. > (I've recently re-done the benchmarks and conclusions are essentially the same.) > > The upshot is that implementations that use Arrays.copyOf() are the fastest, > probably because it's an intrinsic. It can avoid zero-filling the freshly > allocated array because it knows the entire array contents will be overwritten. > This is true regardless of what the public API looks like. The default > implementation calls generator.apply(0) to get a zero-length array and then > simply calls toArray(T[]). This goes through the Arrays.copyOf() fast path, so > it's essentially the same speed as toArray(new T[0]). > > The overrides I had previously provided in specific implementation classes like > ArrayList actually are slower, because the allocation of the array is done > separately from filling it. This necessitates the extra zero-filling step. Thus, > I've removed the overrides. for ArrayList, you may use a code like this, T[] toArray(IntFunction generator) { return (T[]) Arrays.copyOf(elementData, size, generator.apply(0).getClass()); } so you win only one comparison (yeah !), which can be perfectly predicted, so you should not see any perf difference :) > > # API Shape > > There was some discussion about whether it would be preferable to have a Class > parameter as a type token for the array component type or the array type itself, > instead of using an IntFunction generator. Most of this boils down to what > people are comfortable with. However, there are a few points that make the > generator function approach preferable. > > One point in favor of using a generator is that Stream already has a similar > toArray(generator) method. > > Comparing this to the type token alternatives, for simple tasks like converting > Collection to String[], things are about equal: > > toArray(String[]::new) > toArray(String.class) > toArray(String[].class) > > However, things are hairier if the element type of the collection is generic, > such as Collection>. The result type is a generic array > List[]. Using class literals or array constructor references will > necessitate using an unchecked cast, because none of these can be used to > represent a generic type. > > However, it's possible to use a method reference to provide a properly generic > IntFunction. This would enable the toArray(generator) method to be used without > any unchecked casts. (The method it refers to might need have an unchecked cast > within it, though.) Such a method could also be reused for the corresponding > Stream.toArray(generator) method. List.class or List[].class do not work either. > > For these reasons I'd like to proceed with adding toArray(generator) API. > so thumb up for me ! > Thanks, > > s'marks R?mi > > > [1] https://bugs.openjdk.java.net/browse/JDK-8060192 > > [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ > > [3] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html > > [4] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html From takiguc at linux.vnet.ibm.com Fri Jun 15 07:33:59 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 15 Jun 2018 16:33:59 +0900 Subject: Add x-IBM-1129 charset In-Reply-To: References: Message-ID: <1ebd2f6516ee598a3a0be2f1eee99b0e@linux.vnet.ibm.com> Hello. I tested IBM-1129 charset on AIX platform. It worked fine for default encoding, $ LANG=Vi_VN ~/jdk/bin/java PrintDefaultCharset Vi_VN x-IBM1129 IBM-1129 IBM-1129 $ And also code table is fine. We would appreciate to open a bug/RFE for this change request. Ichiroh Takiguchi IBM Japan, Ltd. On 2018-05-30 10:41, Nasser Ebrahim wrote: > Hello, > > I just realized that I missed to provide the details of the test I have > conducted for the new charset IBM-1129. Please note that I have done > the > following jtreg tests to make sure the new charset is working properly. > > sun.nio.cs.TestCharsetMapping > java.nio.charset.Charset.RegisteredCharsets > sun.nio.cs.CheckHistoricalNamesTest > java.nio.charset.RemovingSunIO.SunioAlias > > Please let me know if I have to run any additional tests. > > Am aware of the suggestion from Alan and Thomas in the mail thread > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053316.html > to move the IBM platform specific charsets to a separate module instead > of > adding to jdk.charsets. We will open another thread to discuss that > topic > as we would like to contribute many more extended charsets. However, I > believe we can still consider integrating this charset without waiting > for > the conclusion of that discussion as this charset is part of default > charset for AIX and needs to be part of the java.base. If we decided to > move the IBM charsets out of jdk.charsets, we can move this charsets as > well for non-AIX platforms. > > Kindly request you to open a bug/RFE for this change request, review > the > fix and provide your feedback. > > Thank you, > Nasser Ebrahim > > > From: Nasser Ebrahim/India/IBM > To: core-libs-dev at openjdk.java.net > Date: 05/19/2018 12:51 PM > Subject: Add x-IBM-1129 charset > > > > Hello, > > With the following three bugs, all the default locale charsets except > two > (Vi_VN.IBM-1129 & ja_JP.IBM-eucJP) are fixed for AIX platform. > > - JDK-8201540: [AIX] Extend the set of supported charsets in java.base > - JDK-8202329: Codepage mappings for IBM-943 and Big5 (aix) > - > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053050.html > : [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to > stdcs > [bug not yet opened]. > > For those fixed charsets, the charsets were existing in the extended > charsets (jdk.charsets) and they were not working with default locale > charset as it did not exist in the standard charset (java.base). The > charsets correspond to the two pending locale (Vi_VN.IBM-1129 & > ja_JP.IBM-eucJP) does not exist in the jdk. They need to be added to > the > extended charsets before adding to stdcs on AIX platform. > > Here, am including the patch to fix the charset IBM-1129 for the > locale > Vi_VN.IBM-1129. We are working on the other missing charset (for > ja_JP.IBM-eucJP) which will be contributed in some time. > > The webrev of the fix is available at > http://cr.openjdk.java.net/~aleonard/IBM1129/webrev.00/ > > Kindly request you to open a bug and review the fix. Please let me know > if > you have any questions. > > Thank you, > Nasser Ebrahim From TOSHIONA at jp.ibm.com Fri Jun 15 09:10:14 2018 From: TOSHIONA at jp.ibm.com (Toshio 5 Nakamura) Date: Fri, 15 Jun 2018 18:10:14 +0900 Subject: JDK-8042131: Proposal of Mapped-values formatter for non-IsoChronology Message-ID: Hello core-libs and i18n folks, We'd like to request to reconsider JDK-8042131, "DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate". The report was posted by our team long time ago, and was closed as not an issue. At that time, the feature was for only IsoChronology. Now, I found the attached patch can activate the mapped-values formatter for non-IsoChronology. Additionally, the Japanese new era will be expected in May 2019. The first year of each era has a special expression in Japanese, "U+5143 U+5e74" (Gannen). java.util.JapaneseImperialCalendar uses this expression. We'd like to use the expression with JapaneseChronology by mapped-values formatter as an option. Can we have a sponsor of this proposal? --sample usage of Japanese new era-- DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); Map yearMap = new HashMap<>(); yearMap.put(1L, "\u5143"); builder.appendText(ChronoField.ERA, TextStyle.FULL) .appendText(ChronoField.YEAR_OF_ERA, yearMap) .appendLiteral("\u5e74"); --------------- Report: https://bugs.openjdk.java.net/browse/JDK-8042131 Patch: --- old/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.489303979 +0900 +++ new/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.157303972 +0900 @@ -793,6 +793,11 @@ final LocaleStore store = new LocaleStore(map); DateTimeTextProvider provider = new DateTimeTextProvider() { @Override + public String getText(Chronology chrono, TemporalField field, + long value, TextStyle style, Locale locale) { + return store.getText(value, style); + } + @Override public String getText(TemporalField field, long value, TextStyle style, Locale locale) { return store.getText(value, style); } --- old/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.664304007 +0900 +++ new/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.298303999 +0900 @@ -67,14 +67,21 @@ import java.time.chrono.Chronology; import java.time.chrono.IsoChronology; import java.time.chrono.JapaneseChronology; +import java.time.chrono.JapaneseEra; import java.time.chrono.MinguoChronology; +import java.time.chrono.ThaiBuddhistChronology; +import java.time.chrono.ChronoLocalDate; import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatterBuilder; import java.time.format.FormatStyle; import java.time.LocalDate; import java.time.temporal.Temporal; +import java.time.temporal.ChronoField; +import static java.time.temporal.ChronoUnit.YEARS; import java.util.Locale; +import java.util.Map; +import java.util.HashMap; import static org.testng.Assert.assertEquals; @@ -115,6 +122,31 @@ } //----------------------------------------------------------------------- + @DataProvider(name="mappedPatterns") + Object[][] localizedMappedPatterns() { + return new Object[][] { + {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, + {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, + 1,1,8), Locale.ENGLISH}, + {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, + {ThaiBuddhistChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, + }; + } + + @Test(dataProvider="mappedPatterns") + public void test_getLocalizedMappedPattern(ChronoLocalDate date, Locale locale) { + final String new1st = "1st"; + Map yearMap = new HashMap<>(); + yearMap.put(1L, new1st); + DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); + builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); + + String actual = date.format(builder.toFormatter(locale)); + assertEquals(actual, new1st); + } + + + //----------------------------------------------------------------------- @DataProvider(name="localePatterns") Object[][] localizedDateTimePatterns() { return new Object[][] { --- Best regards, Toshio Nakamura, IBM Japan From scolebourne at joda.org Fri Jun 15 09:30:44 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 15 Jun 2018 10:30:44 +0100 Subject: JDK-8042131: Proposal of Mapped-values formatter for non-IsoChronology In-Reply-To: References: Message-ID: >From the looks of it, this is a perfectly reasonable enhancement. Sadly I can't sponsor it as I'm not a committer. Stephen On 15 June 2018 at 10:10, Toshio 5 Nakamura wrote: > > Hello core-libs and i18n folks, > > We'd like to request to reconsider JDK-8042131, > "DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate". > The report was posted by our team long time ago, and was closed as not an > issue. > At that time, the feature was for only IsoChronology. > > Now, I found the attached patch can activate the mapped-values formatter > for > non-IsoChronology. > > Additionally, the Japanese new era will be expected in May 2019. The first > year of > each era has a special expression in Japanese, "U+5143 U+5e74" (Gannen). > java.util.JapaneseImperialCalendar uses this expression. > We'd like to use the expression with JapaneseChronology by mapped-values > formatter as an option. > > Can we have a sponsor of this proposal? > > --sample usage of Japanese new era-- > DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); > Map yearMap = new HashMap<>(); > yearMap.put(1L, "\u5143"); > builder.appendText(ChronoField.ERA, TextStyle.FULL) > .appendText(ChronoField.YEAR_OF_ERA, yearMap) > .appendLiteral("\u5e74"); > --------------- > > Report: > https://bugs.openjdk.java.net/browse/JDK-8042131 > > Patch: > --- old/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.489303979 +0900 > +++ new/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.157303972 +0900 > @@ -793,6 +793,11 @@ > final LocaleStore store = new LocaleStore(map); > DateTimeTextProvider provider = new DateTimeTextProvider() { > @Override > + public String getText(Chronology chrono, TemporalField field, > + long value, TextStyle style, Locale locale) { > + return store.getText(value, style); > + } > + @Override > public String getText(TemporalField field, long value, TextStyle style, Locale locale) { > return store.getText(value, style); > } > --- old/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.664304007 +0900 > +++ new/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.298303999 +0900 > @@ -67,14 +67,21 @@ > import java.time.chrono.Chronology; > import java.time.chrono.IsoChronology; > import java.time.chrono.JapaneseChronology; > +import java.time.chrono.JapaneseEra; > import java.time.chrono.MinguoChronology; > +import java.time.chrono.ThaiBuddhistChronology; > +import java.time.chrono.ChronoLocalDate; > import java.time.format.DateTimeFormatter; > import java.time.format.DateTimeFormatterBuilder; > import java.time.format.FormatStyle; > import java.time.LocalDate; > import java.time.temporal.Temporal; > +import java.time.temporal.ChronoField; > +import static java.time.temporal.ChronoUnit.YEARS; > > import java.util.Locale; > +import java.util.Map; > +import java.util.HashMap; > > import static org.testng.Assert.assertEquals; > > @@ -115,6 +122,31 @@ > } > > //----------------------------------------------------------------------- > + @DataProvider(name="mappedPatterns") > + Object[][] localizedMappedPatterns() { > + return new Object[][] { > + {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, > + {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, > + 1,1,8), Locale.ENGLISH}, > + {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, > + {ThaiBuddhistChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, > + }; > + } > + > + @Test(dataProvider="mappedPatterns") > + public void test_getLocalizedMappedPattern(ChronoLocalDate date, Locale locale) { > + final String new1st = "1st"; > + Map yearMap = new HashMap<>(); > + yearMap.put(1L, new1st); > + DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); > + builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); > + > + String actual = date.format(builder.toFormatter(locale)); > + assertEquals(actual, new1st); > + } > + > + > + //----------------------------------------------------------------------- > @DataProvider(name="localePatterns") > Object[][] localizedDateTimePatterns() { > return new Object[][] { > > --- > Best regards, > Toshio Nakamura, IBM Japan From amaembo at gmail.com Fri Jun 15 11:26:15 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Fri, 15 Jun 2018 18:26:15 +0700 Subject: BiCollector In-Reply-To: <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> Message-ID: Hello! I created a preliminary webrev of my own implementation (no testcases yet): http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ If anybody wants to sponsor my implementation, I will happily log an issue and write tests. The name "pairing" was invented by me, but as I'm not a native English speaker I cannot judge whether it's optimal, so better ideas are welcome. Also I decided to remove accumulator types from public type variables. They do not add anything to type signature, only clutter it increasing the number of type parameters from 4 to 6. I think it was a mistake to expose the accumulator type parameter in other cascading collectors like filtering(), collectingAndThen(), groupingBy(), etc. I'm not insisting though, if you feel that conformance to existing collectors is more important than simplicity. With best regards, Tagir Valeev. On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz wrote: > > > Well, I don't see the need to pack the two results into a Map.Entry > > (or any similar) container as a drawback. > > From an "integrity of the JDK APIs" perspective, it is unquestionably a > drawback. These items are not a Key and an associated Value in a Map; > it's merely pretending that Map.Entry really means "Pair". There's a > reason we don't have a Pair class in the JDK (and no, let's not reopen > that now); using something else as a Pair proxy that is supposed to have > specific semantics is worse. (It's fine to do this in your own code, but > not in the JDK. Different standards for code that has different audiences.) > > Tagir's proposed sidestepping is nice, and it will also play nicely with > records, because then you can say: > > record NameAndCount(String name, int count); > > stream.collect(pairing(collectName, collectCount, > NameAndCount::new)); > > and get a more properly abstract result out. And its more in the spirit > of existing Collectors. If you want to use Map.Entry as an > _implementation detail_, that's fine. > > I can support this form. > > > I also don't see a larger abstraction like BiStream as a natural fit > > for a similar thing. > > I think the BiStream connection is mostly tangential. We tried hard to > support streams of (K,V) pairs when we did streams, as Paul can attest, > but it was a huge complexity-inflater and dropping this out paid an > enormous simplification payoff. > > With records, having streams of tuples will be simpler to represent, but > no more performant; it will take until we get to value types and > specialized generics to get the performance we want out of this. > > > From amaembo at gmail.com Fri Jun 15 11:31:51 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Fri, 15 Jun 2018 18:31:51 +0700 Subject: BiCollector In-Reply-To: References: Message-ID: Peter, > EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); > intersection.retainAll(valCollector.characteristics()); > return intersection; Please note that `copyOf` should either receive an EnumSet or non-empty Collection to obtain the enum class. This is not guaranteed by `characteristics()` implementation, thus failure is possible. Testcase: new BiCollector<>(Collectors.counting(), Collectors.toSet()).characteristics(); Fails with Exception in thread "main" java.lang.IllegalArgumentException: Collection is empty at java.base/java.util.EnumSet.copyOf(EnumSet.java:173) at BiCollector.characteristics(BiCollector.java:77) at Main.main(Main.java:21) With best regards, Tagir Valeev. On Mon, Jun 11, 2018 at 7:40 PM Peter Levart wrote: > Hi, > > Have you ever wanted to perform a collection of the same Stream into two > different targets using two Collectors? Say you wanted to collect > Map.Entry elements into two parallel lists, each of them containing keys > and values respectively. Or you wanted to collect elements into groups > by some key, but also count them at the same time? Currently this is not > possible to do with a single Stream. You have to create two identical > streams, so you end up passing Supplier to other methods instead > of bare Stream. > > I created a little utility Collector implementation that serves the > purpose quite well: > > /** > * A {@link Collector} implementation taking two delegate Collector(s) > and producing result composed > * of two results produced by delegating collectors, wrapped in {@link > Map.Entry} object. > * > * @param the type of elements collected > * @param the type of 1st delegate collector collected result > * @param tye type of 2nd delegate collector collected result > */ > public class BiCollector implements Collector Map.Entry, Map.Entry> { > private final Collector keyCollector; > private final Collector valCollector; > > @SuppressWarnings("unchecked") > public BiCollector(Collector keyCollector, Collector V> valCollector) { > this.keyCollector = (Collector) > Objects.requireNonNull(keyCollector); > this.valCollector = (Collector) > Objects.requireNonNull(valCollector); > } > > @Override > public Supplier> supplier() { > Supplier keySupplier = keyCollector.supplier(); > Supplier valSupplier = valCollector.supplier(); > return () -> new > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > } > > @Override > public BiConsumer, T> accumulator() { > BiConsumer keyAccumulator = keyCollector.accumulator(); > BiConsumer valAccumulator = valCollector.accumulator(); > return (accumulation, t) -> { > keyAccumulator.accept(accumulation.getKey(), t); > valAccumulator.accept(accumulation.getValue(), t); > }; > } > > @Override > public BinaryOperator> combiner() { > BinaryOperator keyCombiner = keyCollector.combiner(); > BinaryOperator valCombiner = valCollector.combiner(); > return (accumulation1, accumulation2) -> new > AbstractMap.SimpleImmutableEntry<>( > keyCombiner.apply(accumulation1.getKey(), > accumulation2.getKey()), > valCombiner.apply(accumulation1.getValue(), > accumulation2.getValue()) > ); > } > > @Override > public Function, Map.Entry> > finisher() { > Function keyFinisher = keyCollector.finisher(); > Function valFinisher = valCollector.finisher(); > return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > keyFinisher.apply(accumulation.getKey()), > valFinisher.apply(accumulation.getValue()) > ); > } > > @Override > public Set characteristics() { > EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > intersection.retainAll(valCollector.characteristics()); > return intersection; > } > } > > > Do you think this class is general enough to be part of standard > Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector > coll1, Collector coll2), bi-collection could then be coded simply as: > > Map map = ... > > Map.Entry, List> keys_values = > map.entrySet() > .stream() > .collect( > toBoth( > mapping(Map.Entry::getKey, toList()), > mapping(Map.Entry::getValue, toList()) > ) > ); > > > Map.Entry, Long> histogram_count = > ThreadLocalRandom > .current() > .ints(100, 0, 10) > .boxed() > .collect( > toBoth( > groupingBy(Function.identity(), counting()), > counting() > ) > ); > > > Regards, Peter > > From james.laskey at oracle.com Fri Jun 15 11:56:54 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 15 Jun 2018 08:56:54 -0300 Subject: RFR: 8204967: Resolve disabled warnings for libunpack In-Reply-To: <0df0b775-3ff8-49fa-81dd-0cfc3b16641e@default> References: <0df0b775-3ff8-49fa-81dd-0cfc3b16641e@default> Message-ID: <5CA9827A-76CF-4F8F-8292-058BA4F3E443@oracle.com> +1 > On Jun 15, 2018, at 1:45 AM, Srinivas Dama wrote: > > Hi Magnus/Jim, > > Thank you for the comments. > Here is the latest webrev with suggested changes: > webrev: http://cr.openjdk.java.net/~sdama/8204967/webrev.01/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8204967 > > Jim, > gnu c supports -Wimplicit-fallthrough=3. > clang takes -Wimplicit-fallthrough without any parameters and ignores it without any effect. > clang++ works with -Wimplicit-fallthrough along with -std=c++11(it is cpp specific option) > > Note: My earlier fix http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ broke linux > build because some switch case statements which are having implicit-fallthroughs will be enabled > only in debug build(src/jdk.pack/share/native/common-unpack/unpack.cpp) > Earlier,I missed to fix those so debug build failed in linux. > > Regards, > Srinivas > > ----- Original Message ----- > From: magnus.ihse.bursie at oracle.com > To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net > Sent: Thursday, 14 June, 2018 3:23:23 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: Re: RFR: 8204967: Resolve disabled warnings for libunpack > > On 2018-06-13 18:57, Srinivas Dama wrote: >> Hi, >> >> Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ >> for https://bugs.openjdk.java.net/browse/JDK-8204967 > > Hi Srinivas, > > In src/jdk.pack/share/native/common-unpack/zip.cpp, you just read and > discard the return value from fread to hide the warning. But the purpose > of the warning is to point out that this kind of code is incorrect. The > proper fix would be to check the return code from fread to make sure > that the read did not fail. Otherwise you're just making it impossible > for the compiler to point out the badly written code, and if you're not > going to fix this properly, it's better to keep the warning (that way we > know the code is fishy). > > /Magnus > >> >> Regards, >> Srinivas > From roger.riggs at oracle.com Fri Jun 15 14:38:01 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 15 Jun 2018 10:38:01 -0400 Subject: JDK-8042131: Proposal of Mapped-values formatter for non-IsoChronology In-Reply-To: References: Message-ID: Hi, A good suggestion to reconsider, I re-opened the issue. I have not looked closely to see if/where the spec needs to be changed. Thanks, Roger On 6/15/18 5:30 AM, Stephen Colebourne wrote: > From the looks of it, this is a perfectly reasonable enhancement. > Sadly I can't sponsor it as I'm not a committer. > > Stephen > > > On 15 June 2018 at 10:10, Toshio 5 Nakamura wrote: >> Hello core-libs and i18n folks, >> >> We'd like to request to reconsider JDK-8042131, >> "DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate". >> The report was posted by our team long time ago, and was closed as not an >> issue. >> At that time, the feature was for only IsoChronology. >> >> Now, I found the attached patch can activate the mapped-values formatter >> for >> non-IsoChronology. >> >> Additionally, the Japanese new era will be expected in May 2019. The first >> year of >> each era has a special expression in Japanese, "U+5143 U+5e74" (Gannen). >> java.util.JapaneseImperialCalendar uses this expression. >> We'd like to use the expression with JapaneseChronology by mapped-values >> formatter as an option. >> >> Can we have a sponsor of this proposal? >> >> --sample usage of Japanese new era-- >> DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); >> Map yearMap = new HashMap<>(); >> yearMap.put(1L, "\u5143"); >> builder.appendText(ChronoField.ERA, TextStyle.FULL) >> .appendText(ChronoField.YEAR_OF_ERA, yearMap) >> .appendLiteral("\u5e74"); >> --------------- >> >> Report: >> https://bugs.openjdk.java.net/browse/JDK-8042131 >> >> Patch: >> --- old/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.489303979 +0900 >> +++ new/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.157303972 +0900 >> @@ -793,6 +793,11 @@ >> final LocaleStore store = new LocaleStore(map); >> DateTimeTextProvider provider = new DateTimeTextProvider() { >> @Override >> + public String getText(Chronology chrono, TemporalField field, >> + long value, TextStyle style, Locale locale) { >> + return store.getText(value, style); >> + } >> + @Override >> public String getText(TemporalField field, long value, TextStyle style, Locale locale) { >> return store.getText(value, style); >> } >> --- old/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.664304007 +0900 >> +++ new/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.298303999 +0900 >> @@ -67,14 +67,21 @@ >> import java.time.chrono.Chronology; >> import java.time.chrono.IsoChronology; >> import java.time.chrono.JapaneseChronology; >> +import java.time.chrono.JapaneseEra; >> import java.time.chrono.MinguoChronology; >> +import java.time.chrono.ThaiBuddhistChronology; >> +import java.time.chrono.ChronoLocalDate; >> import java.time.format.DateTimeFormatter; >> import java.time.format.DateTimeFormatterBuilder; >> import java.time.format.FormatStyle; >> import java.time.LocalDate; >> import java.time.temporal.Temporal; >> +import java.time.temporal.ChronoField; >> +import static java.time.temporal.ChronoUnit.YEARS; >> >> import java.util.Locale; >> +import java.util.Map; >> +import java.util.HashMap; >> >> import static org.testng.Assert.assertEquals; >> >> @@ -115,6 +122,31 @@ >> } >> >> //----------------------------------------------------------------------- >> + @DataProvider(name="mappedPatterns") >> + Object[][] localizedMappedPatterns() { >> + return new Object[][] { >> + {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >> + {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, >> + 1,1,8), Locale.ENGLISH}, >> + {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >> + {ThaiBuddhistChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >> + }; >> + } >> + >> + @Test(dataProvider="mappedPatterns") >> + public void test_getLocalizedMappedPattern(ChronoLocalDate date, Locale locale) { >> + final String new1st = "1st"; >> + Map yearMap = new HashMap<>(); >> + yearMap.put(1L, new1st); >> + DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); >> + builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); >> + >> + String actual = date.format(builder.toFormatter(locale)); >> + assertEquals(actual, new1st); >> + } >> + >> + >> + //----------------------------------------------------------------------- >> @DataProvider(name="localePatterns") >> Object[][] localizedDateTimePatterns() { >> return new Object[][] { >> >> --- >> Best regards, >> Toshio Nakamura, IBM Japan From martinrb at google.com Fri Jun 15 14:57:04 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 07:57:04 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> Message-ID: Code looks correct to me, but as usual there are some things I would try to do differently, 1292 while (word != 0 && tz < 63) { I'd try to make this loop simply while (word != 0) --- Probably neither of us is fond of the bug magnet special case for Integer.MAX_VALUE. Maybe just store fence as a long or as a wordindex which can't overflow (while giving up on intra-word splitting?) Here's a fun program demonstrating cardinality overflow: import java.util.BitSet; public class BitSetCardinality { public static void main(String[] args) throws Throwable { BitSet s = new BitSet(); s.flip(0, Integer.MAX_VALUE); System.out.println(s.cardinality()); s.flip(Integer.MAX_VALUE); System.out.println(s.cardinality()); } } ==> java -Xmx20g -esa -ea BitSetCardinality 2147483647 -2147483648 On Thu, Jun 14, 2018 at 2:35 PM, Paul Sandoz wrote: > Hi, > > Please review this enhancement to improve the performance of BitSet > traversal when using a stream. > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159- > bitset-traverse/webrev/ psandoz/jdk/JDK-8170159-bitset-traverse/webrev/> > > The associated issue started out life referring to the BitSet?s > spliterator reporting SIZED, and to report that the cardinality has to be > computed. This has some cost but performance analysis (see issue for > attached benchmark) has shown that the cost is small compared to the cost > of traversal. It is recognized that the cost is higher for smaller BitSets > but not unduly so. Thus it was concluded that it was reasonable for the > spliterator to report SIZED. > > The issue was adjusted to focus on improving the performance of the > BitSet?s spliterator forEachRemaining method. For large randomized BitSets > a 1.6x speedup can be observed, and for smaller sizes a more modest speed > up. The prior conclusion about reporting SIZED still holds. > > Thanks, > Paul. From brent.christian at oracle.com Fri Jun 15 15:44:31 2018 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 15 Jun 2018 08:44:31 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map Message-ID: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Hi, In JDK 9, 8029891[1] refactored java.util.Properties to store its values in an internal ConcurrentHashMap, and removed synchronization from "reader" methods in order to avoid potential hangs/deadlocks during classloading. Claes has noticed that there is the possibility of the new 'map' field being observed with its default value (null), before being set. After looking at the JSR 133 FAQ[2], I agree with Claes that we should make 'map' a field final. Please review my change to do this: Webrev: http://cr.openjdk.java.net/~bchristi/8199435/webrev/ Issue: https://bugs.openjdk.java.net/browse/JDK-8199435 Thanks, -Brent 1. https://bugs.openjdk.java.net/browse/JDK-8029891 2. https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#finalRight From naoto.sato at oracle.com Fri Jun 15 16:04:13 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 15 Jun 2018 09:04:13 -0700 Subject: JDK-8042131: Proposal of Mapped-values formatter for non-IsoChronology In-Reply-To: References: Message-ID: It does seem to be a good change. I will take a look and sponsor it. Naoto On 6/15/18 7:38 AM, Roger Riggs wrote: > Hi, > > A good suggestion to reconsider, I re-opened the issue. > > I have not looked closely to see if/where the spec needs to be changed. > > Thanks, Roger > > > On 6/15/18 5:30 AM, Stephen Colebourne wrote: >> ?From the looks of it, this is a perfectly reasonable enhancement. >> Sadly I can't sponsor it as I'm not a committer. >> >> Stephen >> >> >> On 15 June 2018 at 10:10, Toshio 5 Nakamura wrote: >>> Hello core-libs and i18n folks, >>> >>> We'd like to request to reconsider JDK-8042131, >>> "DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate". >>> The report was posted by our team long time ago, and was closed as >>> not an >>> issue. >>> At that time, the feature was for only IsoChronology. >>> >>> Now, I found the attached patch can activate the mapped-values formatter >>> for >>> non-IsoChronology. >>> >>> Additionally, the Japanese new era will be expected in May 2019. The >>> first >>> year of >>> each era has a special expression in Japanese, "U+5143 U+5e74" (Gannen). >>> java.util.JapaneseImperialCalendar uses this expression. >>> We'd like to use the expression with JapaneseChronology by mapped-values >>> formatter as an option. >>> >>> Can we have a sponsor of this proposal? >>> >>> --sample usage of Japanese new era-- >>> DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); >>> Map yearMap = new HashMap<>(); >>> yearMap.put(1L, "\u5143"); >>> builder.appendText(ChronoField.ERA, TextStyle.FULL) >>> ???????????? .appendText(ChronoField.YEAR_OF_ERA, yearMap) >>> ???????????? .appendLiteral("\u5e74"); >>> --------------- >>> >>> Report: >>> https://bugs.openjdk.java.net/browse/JDK-8042131 >>> >>> Patch: >>> --- >>> old/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java >>> 2018-06-15 17:39:11.489303979 +0900 >>> +++ >>> new/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java >>> 2018-06-15 17:39:11.157303972 +0900 >>> @@ -793,6 +793,11 @@ >>> ????????? final LocaleStore store = new LocaleStore(map); >>> ????????? DateTimeTextProvider provider = new DateTimeTextProvider() { >>> ????????????? @Override >>> +??????????? public String getText(Chronology chrono, TemporalField >>> field, >>> +????????????????????????????????? long value, TextStyle style, >>> Locale locale) { >>> +??????????????? return store.getText(value, style); >>> +??????????? } >>> +??????????? @Override >>> ????????????? public String getText(TemporalField field, long value, >>> TextStyle style, Locale locale) { >>> ????????????????? return store.getText(value, style); >>> ????????????? } >>> --- >>> old/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java >>> 2018-06-15 17:39:12.664304007 +0900 >>> +++ >>> new/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java >>> 2018-06-15 17:39:12.298303999 +0900 >>> @@ -67,14 +67,21 @@ >>> ? import java.time.chrono.Chronology; >>> ? import java.time.chrono.IsoChronology; >>> ? import java.time.chrono.JapaneseChronology; >>> +import java.time.chrono.JapaneseEra; >>> ? import java.time.chrono.MinguoChronology; >>> +import java.time.chrono.ThaiBuddhistChronology; >>> +import java.time.chrono.ChronoLocalDate; >>> ? import java.time.format.DateTimeFormatter; >>> ? import java.time.format.DateTimeFormatterBuilder; >>> ? import java.time.format.FormatStyle; >>> ? import java.time.LocalDate; >>> ? import java.time.temporal.Temporal; >>> +import java.time.temporal.ChronoField; >>> +import static java.time.temporal.ChronoUnit.YEARS; >>> >>> ? import java.util.Locale; >>> +import java.util.Map; >>> +import java.util.HashMap; >>> >>> ? import static org.testng.Assert.assertEquals; >>> >>> @@ -115,6 +122,31 @@ >>> ????? } >>> >>> >>> //----------------------------------------------------------------------- >>> >>> +??? @DataProvider(name="mappedPatterns") >>> +??? Object[][] localizedMappedPatterns() { >>> +??????? return new Object[][] { >>> +??????????? {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >>> +??????????? {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, >>> +????????????????????????????????????????????? 1,1,8), Locale.ENGLISH}, >>> +??????????? {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >>> +??????????? {ThaiBuddhistChronology.INSTANCE.date(1,1,1), >>> Locale.ENGLISH}, >>> +??????? }; >>> +??? } >>> + >>> +??? @Test(dataProvider="mappedPatterns") >>> +??? public void test_getLocalizedMappedPattern(ChronoLocalDate date, >>> Locale locale) { >>> +??????? final String new1st = "1st"; >>> +??????? Map yearMap = new HashMap<>(); >>> +??????? yearMap.put(1L, new1st); >>> +??????? DateTimeFormatterBuilder builder = new >>> DateTimeFormatterBuilder(); >>> +??????? builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); >>> + >>> +??????? String actual = date.format(builder.toFormatter(locale)); >>> +??????? assertEquals(actual, new1st); >>> +??? } >>> + >>> + >>> + >>> //----------------------------------------------------------------------- >>> >>> ????? @DataProvider(name="localePatterns") >>> ????? Object[][] localizedDateTimePatterns() { >>> ????????? return new Object[][] { >>> >>> --- >>> Best regards, >>> Toshio Nakamura, IBM Japan > From paul.sandoz at oracle.com Fri Jun 15 17:17:53 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 10:17:53 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> Message-ID: <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> Hi Martin, Thanks for reviewing. > On Jun 15, 2018, at 7:57 AM, Martin Buchholz wrote: > > Code looks correct to me, but as usual there are some things I would try to do differently, > Indeed, i tried a few different approaches before settling on this code shape. > 1292 while (word != 0 && tz < 63) { > > I'd try to make this loop simply > while (word != 0) > I started out like that but in the end preferred that both loop conditions were upfront, to avoid another break or explicitly setting word to 0, then the only break is the weird one to deal with the annoying bug magnet :-) > --- > > Probably neither of us is fond of the bug magnet special case for Integer.MAX_VALUE. Maybe just store fence as a long or as a wordindex which can't overflow (while giving up on intra-word splitting?) > > Here's a fun program demonstrating cardinality overflow: > > import java.util.BitSet; > > public class BitSetCardinality { > public static void main(String[] args) throws Throwable { > BitSet s = new BitSet(); > s.flip(0, Integer.MAX_VALUE); > System.out.println(s.cardinality()); > s.flip(Integer.MAX_VALUE); > System.out.println(s.cardinality()); > } > } > > > ==> java -Xmx20g -esa -ea BitSetCardinality > 2147483647 > -2147483648 > Oh dear! Thanks, Paul. > > On Thu, Jun 14, 2018 at 2:35 PM, Paul Sandoz > wrote: > Hi, > > Please review this enhancement to improve the performance of BitSet traversal when using a stream. > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159-bitset-traverse/webrev/ > > > The associated issue started out life referring to the BitSet?s spliterator reporting SIZED, and to report that the cardinality has to be computed. This has some cost but performance analysis (see issue for attached benchmark) has shown that the cost is small compared to the cost of traversal. It is recognized that the cost is higher for smaller BitSets but not unduly so. Thus it was concluded that it was reasonable for the spliterator to report SIZED. > > The issue was adjusted to focus on improving the performance of the BitSet?s spliterator forEachRemaining method. For large randomized BitSets a 1.6x speedup can be observed, and for smaller sizes a more modest speed up. The prior conclusion about reporting SIZED still holds. > > Thanks, > Paul. > From cushon at google.com Fri Jun 15 17:21:17 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 15 Jun 2018 10:21:17 -0700 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> <5A90B204.4070005@oracle.com> <5A960359.4070507@oracle.com> Message-ID: Thanks, I pushed it as: http://hg.openjdk.java.net/jdk/jdk/rev/9f4c08c444e8 On Thu, Jun 14, 2018 at 10:31 AM Vicente Romero wrote: > I'm not an expert in the area, but the patch looks good to me, > > Vicente > > On 05/31/2018 08:39 PM, Liam Miller-Cushon wrote: > > Hi, > > > > Are there any concerns with the patch that I can address? I was hoping to > > get this in to JDK 11. > > > > I confirmed that it still builds and passes all tests at head. > > > > Thanks, > > Liam > > > > On Mon, May 14, 2018 at 10:38 AM Liam Miller-Cushon > > wrote: > > > >> Ping > >> > >> On Thu, Apr 5, 2018 at 10:57 AM Liam Miller-Cushon > >> wrote: > >> > >>> Hi, > >>> > >>> Is there any more feedback on the patch? > >>> > >>> On Wed, Feb 28, 2018 at 4:26 PM Liam Miller-Cushon > >>> wrote: > >>> > >>>> On Wed, Feb 28, 2018 at 4:13 PM, Martin Buchholz > > >>>> wrote: > >>>> > >>>>> Follow their lead and rename your method to assertArrayEquals > >>>>> > >>>> Done: http://cr.openjdk.java.net/~cushon/7183985/webrev.04/ > >>>> > > From paul.sandoz at oracle.com Fri Jun 15 17:35:20 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 10:35:20 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: > On Jun 15, 2018, at 8:44 AM, Brent Christian wrote: > > Hi, > > In JDK 9, 8029891[1] refactored java.util.Properties to store its values in an internal ConcurrentHashMap, and removed synchronization from "reader" methods in order to avoid potential hangs/deadlocks during classloading. > > Claes has noticed that there is the possibility of the new 'map' field being observed with its default value (null), before being set. > Yes. I would also publish the map in readHashtable after you have placed in the keys/values. Pail. > After looking at the JSR 133 FAQ[2], I agree with Claes that we should make 'map' a field final. > > Please review my change to do this: > > Webrev: > http://cr.openjdk.java.net/~bchristi/8199435/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199435 > > Thanks, > -Brent > > 1. https://bugs.openjdk.java.net/browse/JDK-8029891 > 2. https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#finalRight From martinrb at google.com Fri Jun 15 17:43:04 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 10:43:04 -0700 Subject: Durations in existing JDK APIs In-Reply-To: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: On Wed, May 30, 2018 at 11:32 AM, Doug Lea
wrote: > > The original rationale for designing j.u.c.TimeUnit using the Flyweight > pattern was to to reduce allocation and GC-related overhead and timing > jitter for methods that otherwise may operate on the order of > nanoseconds. But there are many cases in which this is not much of a > concern (plus JVMs can now sometimes optimize), so people should be > given a choice. It would be a lot of tedious work (and aggregate code > bulk) to retrofit every time-related j.u.c method though, and it's not > clear where to compromise. But at least adding converters should not be > controversial. > Re-reading Doug's assessment, Doug seems reluctant but open to adding at least some Duration overloads. Here's an obvious first candidate in Future (yes, we have a test that discovers get(Duration) and checks that get(null) throws UOE instead of NPE.). (It's a lot of tedious work) a/util/concurrent/CompletableFuture.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/CompletableFuture.java,v retrieving revision 1.212 diff -u -r1.212 CompletableFuture.java --- src/main/java/util/concurrent/CompletableFuture.java 11 Mar 2018 18:00:05 -0000 1.212 +++ src/main/java/util/concurrent/CompletableFuture.java 15 Jun 2018 17:39:09 -0000 @@ -1983,13 +1983,22 @@ * while waiting * @throws TimeoutException if the wait timed out */ - @SuppressWarnings("unchecked") public T get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException { - long nanos = unit.toNanos(timeout); + return getNanos(unit.toNanos(timeout)); + } + + public T get(java.time.Duration timeout) + throws InterruptedException, ExecutionException, TimeoutException { + return getNanos(TimeUnit.NANOSECONDS.convert(timeout)); + } + + @SuppressWarnings("unchecked") + private T getNanos(long timeoutNanos) + throws InterruptedException, ExecutionException, TimeoutException { Object r; if ((r = result) == null) - r = timedGet(nanos); + r = timedGet(timeoutNanos); return (T) reportGet(r); } @@ -2797,6 +2806,8 @@ throw new UnsupportedOperationException(); } @Override public T get(long timeout, TimeUnit unit) { throw new UnsupportedOperationException(); } + @Override public T get(java.time.Duration duration) { + throw new UnsupportedOperationException(); } @Override public T getNow(T valueIfAbsent) { throw new UnsupportedOperationException(); } @Override public T join() { Index: src/main/java/util/concurrent/Future.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/Future.java,v retrieving revision 1.41 diff -u -r1.41 Future.java --- src/main/java/util/concurrent/Future.java 8 Oct 2016 18:52:37 -0000 1.41 +++ src/main/java/util/concurrent/Future.java 15 Jun 2018 17:39:09 -0000 @@ -6,6 +6,10 @@ package java.util.concurrent; +import static java.util.concurrent.TimeUnit.NANOSECONDS; + +import java.time.Duration; + /** * A {@code Future} represents the result of an asynchronous * computation. Methods are provided to check if the computation is @@ -126,7 +130,30 @@ * @throws InterruptedException if the current thread was interrupted * while waiting * @throws TimeoutException if the wait timed out + * @throws NullPointerException if {@code unit} is null */ V get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException; + + /** + * Waits if necessary for at most the given time for the computation + * to complete, and then retrieves its result, if available. + * + *

Equivalent to: 

 {@code
+     * get(NANOSECONDS.convert(timeout), NANOSECONDS)}
+ * + * @param timeout the maximum time to wait + * @return the computed result + * @throws CancellationException if the computation was cancelled + * @throws ExecutionException if the computation threw an + * exception + * @throws InterruptedException if the current thread was interrupted + * while waiting + * @throws TimeoutException if the wait timed out + * @throws NullPointerException if {@code timeout} is null + */ + default V get(Duration timeout) + throws InterruptedException, ExecutionException, TimeoutException { + return get(NANOSECONDS.convert(timeout), NANOSECONDS); + } } From cushon at google.com Fri Jun 15 17:58:07 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 15 Jun 2018 10:58:07 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication Message-ID: Hello, This change is a follow-up to JDK-7183985 which replaces the interior of parseClassArray, parseEnumArray, and parseAnnotationArray with a shared method that is passed a lambda for the type-specific parsing logic. bug: https://bugs.openjdk.java.net/browse/JDK-8198669 webrev: http://cr.openjdk.java.net/~cushon/8198669/webrev.00/ Liam From brent.christian at oracle.com Fri Jun 15 18:01:51 2018 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 15 Jun 2018 11:01:51 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: <9fe90d21-0604-97ff-2abb-a940a0268c87@oracle.com> On 6/15/18 10:35 AM, Paul Sandoz wrote: > > I would also publish the map in readHashtable after you have placed in the keys/values. > Like this? // create CHM of appropriate capacity - map = new ConcurrentHashMap<>(elements); + ConcurrentHashMap tmpMap = new ConcurrentHashMap<>(elements); // Read all the key/value objects for (; elements > 0; elements--) { Object key = s.readObject(); Object value = s.readObject(); - map.put(key, value); + tmpMap.put(key, value); } + + UNSAFE.putObjectVolatile(this, MAP_OFFSET, tmpMap); } -B From martinrb at google.com Fri Jun 15 18:05:42 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 11:05:42 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> Message-ID: It took me a while to realize why you were checking tz < 63. It's because the shift by 64 that might occur below would be a no-op! 1301 action.accept(i++);1302 word &= (WORD_MASK << i); Can we rewrite to simply flip the one bit via word &= ~(1L << i); action.accept(i++); and then we can simplify the tz handling? From paul.sandoz at oracle.com Fri Jun 15 18:37:41 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 11:37:41 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <9fe90d21-0604-97ff-2abb-a940a0268c87@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <9fe90d21-0604-97ff-2abb-a940a0268c87@oracle.com> Message-ID: Yes, like that, thanks, Paul. > On Jun 15, 2018, at 11:01 AM, Brent Christian wrote: > > On 6/15/18 10:35 AM, Paul Sandoz wrote: >> I would also publish the map in readHashtable after you have placed in the keys/values. > > Like this? > > // create CHM of appropriate capacity > - map = new ConcurrentHashMap<>(elements); > + ConcurrentHashMap tmpMap = new ConcurrentHashMap<>(elements); > > // Read all the key/value objects > for (; elements > 0; elements--) { > Object key = s.readObject(); > Object value = s.readObject(); > - map.put(key, value); > + tmpMap.put(key, value); > } > + > + UNSAFE.putObjectVolatile(this, MAP_OFFSET, tmpMap); > } > > -B From paul.sandoz at oracle.com Fri Jun 15 20:13:51 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 13:13:51 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> Message-ID: <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> Thanks! Doh, obvious in hindsight. > On Jun 15, 2018, at 11:05 AM, Martin Buchholz wrote: > > It took me a while to realize why you were checking tz < 63. It's because the shift by 64 that might occur below would be a no-op! Right! > 1301 action.accept(i++); > 1302 word &= (WORD_MASK << i); > Can we rewrite to simply flip the one bit via > > word &= ~(1L << i); > action.accept(i++); > > and then we can simplify the tz handling? > Yes, and there is no need to increment i: http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159-bitset-traverse/webrev/src/java.base/share/classes/java/util/BitSet.java.sdiff.html Paul. From paul.sandoz at oracle.com Fri Jun 15 20:43:55 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 13:43:55 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication In-Reply-To: References: Message-ID: <850CD01B-E498-407A-BC10-4EC7905D6FCC@oracle.com> +1 A nice cleanup. Did you run any tiered tests just to double check that the use of lambdas causes no bootstrapping issues? (i suspect it probably does not). Paul. > On Jun 15, 2018, at 10:58 AM, Liam Miller-Cushon wrote: > > Hello, > > This change is a follow-up to JDK-7183985 which replaces the interior of > parseClassArray, parseEnumArray, and parseAnnotationArray with a shared > method that is passed a lambda for the type-specific parsing logic. > > bug: https://bugs.openjdk.java.net/browse/JDK-8198669 > webrev: http://cr.openjdk.java.net/~cushon/8198669/webrev.00/ > > Liam From stuart.marks at oracle.com Fri Jun 15 21:22:25 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 15 Jun 2018 14:22:25 -0700 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: References: Message-ID: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Hi Peter, Finally getting back to this. I think there's clearly room for improvement in the creation of the unmodifiable collections. Right now the creation paths (mostly) use only public APIs, which can result in unnecessary allocation and copying. Certainly Map.copyOf does this, but there are also other cases as well. For copying from an unknown map, there are a couple approaches: * accumulate keys and values in a single ArrayList, which can be presized as necessary, but which will grow if necessary; then copy elements from a subrange of ArrayList's internal array (similar to your MapN(Object[], len) overload) * accumulate keys and values into a SpinedBuffer, which doesn't require copying to grow, which is preferable if for some reason we can't pre-size accurately; and then copy the elements out of it The Collectors.toUnmodifiableMap implementations are known to create HashMap instances, so they could pass the HashMap directly to a private MapN constructor that in turn could talk directly to HashMap to get the keys and values. This avoids allocation of a full-sized buffer and one copy. Note that these techniques involve creating new interfaces, sometimes that cross package boundaries. It's a bit of an irritant to have to plumb new paths that go through SharedSecrets, but it seems likely to be worthwhile if we can avoid bulk allocation and copying steps. Given these, it doesn't seem to me that the BiBuffer approach helps very much. I think there are many other avenues that would be worthwhile to explore, and that possibly can provide bigger savings. s'marks On 6/11/18 3:57 AM, Peter Levart wrote: > Hi, > > Those two methods were added in JDK 10 and they are not very optimal. > Map.copyOf(Map map) 1st dumps the source map into an array of Map.Entry(s) > (map.entrySet().toArray(new Entry[0])), which typically creates new Map.Entry > objects, then passes the array to Map.ofEntries(Map.Entry[] entries) factory > method that iterates the array and constructs a key/value interleaved array from > it which is passed to ImmutableCollections.MapN constructor, which then > constructs a linear-probing hash table from it. So each key and value is copied > 3 times, while several temporary objects are created in the process. > > One copying step could be eliminated and construction of temporary Map.Entry > objects too: > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.01/ > > Collecting stream(s) using Collectors.toUnmodifiableMap() 1st collects key/value > pairs into a HashMap, then it performs equivalent operations as > Map.copyOf(hashMap) at the end. Using Map.copyOf() directly benefits this > collection operation too. > > The following benchmark: > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench.java > > > > Shows up to 30% improvement of .copyOf operation with this patch applied: > > Original: > > Benchmark?????????????????????????????? (size)? Mode Cnt????? Score Error? Units > UnmodifiableMapBench.copyOf???????????????? 10? avgt 10??? 403.633 ? 2.640? ns/op > UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 3489.623 ? 44.590? ns/op > UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 40030.572 ? 277.075 > ns/op > UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10??? 831.221 ? 3.816? ns/op > UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9783.519 ? 43.097? ns/op > UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 96524.536 ? 670.818 > ns/op > > Patched: > > Benchmark?????????????????????????????? (size)? Mode Cnt????? Score Error? Units > UnmodifiableMapBench.copyOf???????????????? 10? avgt 10??? 264.172 ? 1.882? ns/op > UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 2318.974 ? 15.877? ns/op > UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 29291.782 ? 3139.737 > ns/op > UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10??? 771.221 ? 65.432? ns/op > UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9275.016 ? 725.722? ns/op > UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 82204.342 ? 851.741 > ns/op > > > Production of garbage is also reduced, since no Map.Entry temporary objects are > constructed: > > Original: > > Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10??? 416.001 ? > 0.002??? B/op > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 2936.005 ? > 0.019??? B/op > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 28136.059 ? > 0.199??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10? avgt 10 > 1368.001 ????? 0.004??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100? avgt 10 > 10208.139 ????? 0.045??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000? avgt 10 > 93025.923 ????? 0.573??? B/op > > Patched: > > Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10??? 304.000 ? > 0.001??? B/op > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 2464.004 ? > 0.012??? B/op > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 24064.040 ? > 0.137??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10? avgt 10 > 1256.001 ????? 0.003??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100? avgt 10 > 9720.153 ????? 0.055??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000? avgt 10 > 88905.688 ????? 0.574??? B/op > > > So what do you think? Is this an improvement? > > Regards, Peter > From martinrb at google.com Fri Jun 15 21:24:19 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 14:24:19 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> Message-ID: Still looks correct, but I might keep looking for something more elegant. What bothers me a little now is 1290 long word = words[u] & (WORD_MASK << i); which is part of the initial setup and is not necessary on each iteration. On Fri, Jun 15, 2018 at 1:13 PM, Paul Sandoz wrote: > Thanks! Doh, obvious in hindsight. > > On Jun 15, 2018, at 11:05 AM, Martin Buchholz wrote: > > It took me a while to realize why you were checking tz < 63. It's because > the shift by 64 that might occur below would be a no-op! > > > Right! > > 1301 action.accept(i++);1302 word &= (WORD_MASK << i); > > Can we rewrite to simply flip the one bit via > > word &= ~(1L << i); > action.accept(i++); > > and then we can simplify the tz handling? > > > Yes, and there is no need to increment i: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159- > bitset-traverse/webrev/src/java.base/share/classes/java/ > util/BitSet.java.sdiff.html > > Paul. > From cushon at google.com Fri Jun 15 21:31:10 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 15 Jun 2018 14:31:10 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication In-Reply-To: <850CD01B-E498-407A-BC10-4EC7905D6FCC@oracle.com> References: <850CD01B-E498-407A-BC10-4EC7905D6FCC@oracle.com> Message-ID: On Fri, Jun 15, 2018 at 1:44 PM Paul Sandoz wrote: > A nice cleanup. Did you run any tiered tests just to double check that the > use of lambdas causes no bootstrapping issues? (i suspect it probably does > not). Thanks, I ran `run-test-tier1` and everything looks fine, are there other tests I should check? From paul.sandoz at oracle.com Fri Jun 15 21:52:54 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 14:52:54 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> Message-ID: > On Jun 15, 2018, at 2:24 PM, Martin Buchholz wrote: > > Still looks correct, but I might keep looking for something more elegant. > What bothers me a little now is > > 1290 long word = words[u] & (WORD_MASK << i); > > which is part of the initial setup and is not necessary on each iteration. > I was looking at this too. From a performance perspective i am not very concerned. Aesthetically it bothers me, but at this point i am willing to declare victory and patch later if there is a better way. Paul. From paul.sandoz at oracle.com Fri Jun 15 22:01:55 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 15:01:55 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication In-Reply-To: References: <850CD01B-E498-407A-BC10-4EC7905D6FCC@oracle.com> Message-ID: <9C96329D-192C-46BD-AA37-685B444939E0@oracle.com> > On Jun 15, 2018, at 2:31 PM, Liam Miller-Cushon wrote: > > On Fri, Jun 15, 2018 at 1:44 PM Paul Sandoz > wrote: > A nice cleanup. Did you run any tiered tests just to double check that the use of lambdas causes no bootstrapping issues? (i suspect it probably does not). > > Thanks, I ran `run-test-tier1` and everything looks fine, are there other tests I should check? > tier1 should be ok. Thanks, Paul. From martinrb at google.com Fri Jun 15 22:03:35 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 15:03:35 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> Message-ID: +1 On Fri, Jun 15, 2018 at 2:52 PM, Paul Sandoz wrote: > > > On Jun 15, 2018, at 2:24 PM, Martin Buchholz wrote: > > Still looks correct, but I might keep looking for something more elegant. > What bothers me a little now is > > 1290 long word = words[u] & (WORD_MASK << i); > > > which is part of the initial setup and is not necessary on each iteration. > > > I was looking at this too. From a performance perspective i am not very > concerned. Aesthetically it bothers me, but at this point i am willing to > declare victory and patch later if there is a better way. > > Paul. > From naoto.sato at oracle.com Fri Jun 15 22:14:47 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Fri, 15 Jun 2018 15:14:47 -0700 Subject: [11] RFR: 8042131: DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate Message-ID: <9018eeda-d891-adf8-81b7-ef1d1db08281@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8042131 The proposed change is located at: http://cr.openjdk.java.net/~naoto/8042131/webrev.00/ The fix is originally contributedy by Toshio Nakamura at IBM [1]. I am sponsoring the fix, with some trivial modifications to the test (diff is attached). Naoto [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053830.html -------------- next part -------------- diff -r 9b997bfc60d5 test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java --- a/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java +++ b/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2012, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -64,24 +64,23 @@ */ package test.java.time.format; +import java.time.chrono.ChronoLocalDate; import java.time.chrono.Chronology; import java.time.chrono.IsoChronology; import java.time.chrono.JapaneseChronology; import java.time.chrono.JapaneseEra; import java.time.chrono.MinguoChronology; import java.time.chrono.ThaiBuddhistChronology; -import java.time.chrono.ChronoLocalDate; import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatterBuilder; import java.time.format.FormatStyle; import java.time.LocalDate; +import java.time.temporal.ChronoField; import java.time.temporal.Temporal; -import java.time.temporal.ChronoField; -import static java.time.temporal.ChronoUnit.YEARS; +import java.util.HashMap; import java.util.Locale; import java.util.Map; -import java.util.HashMap; import static org.testng.Assert.assertEquals; @@ -122,23 +121,21 @@ } //----------------------------------------------------------------------- - @DataProvider(name="mappedPatterns") - Object[][] localizedMappedPatterns() { + @DataProvider(name="mapTextLookup") + Object[][] data_mapTextLookup() { return new Object[][] { {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, - {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, - 1,1,8), Locale.ENGLISH}, + {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, 1, 1, 8), Locale.ENGLISH}, {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, {ThaiBuddhistChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, }; } - @Test(dataProvider="mappedPatterns") - public void test_getLocalizedMappedPattern(ChronoLocalDate date, Locale locale) { + @Test(dataProvider="mapTextLookup") + public void test_appendText_mapTextLookup(ChronoLocalDate date, Locale locale) { final String new1st = "1st"; Map yearMap = new HashMap<>(); yearMap.put(1L, new1st); - DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); String actual = date.format(builder.toFormatter(locale)); From paul.sandoz at oracle.com Fri Jun 15 23:36:29 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 16:36:29 -0700 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> Message-ID: <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> Hi Tagir, This is looking good. My current favorite name for the factory method is ?bisecting? since we are dividing the collector into two collectors, the results of which are then merged. Suggested first paragraph of the factory method: "Returns a collector that passes the input elements to two specified collectors and merges their results with the specified merge function.? Paul. > On Jun 15, 2018, at 4:26 AM, Tagir Valeev wrote: > > Hello! > > I created a preliminary webrev of my own implementation (no testcases yet): > http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ > If anybody wants to sponsor my implementation, I will happily log an issue and write tests. > > The name "pairing" was invented by me, but as I'm not a native English speaker I cannot judge whether it's optimal, so better ideas are welcome. > Also I decided to remove accumulator types from public type variables. They do not add anything to type signature, only clutter it > increasing the number of type parameters from 4 to 6. I think it was a mistake to expose the accumulator type parameter in other cascading collectors > like filtering(), collectingAndThen(), groupingBy(), etc. I'm not insisting though, if you feel that conformance to existing collectors is > more important than simplicity. > > With best regards, > Tagir Valeev. > > On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz wrote: > > > Well, I don't see the need to pack the two results into a Map.Entry > > (or any similar) container as a drawback. > > From an "integrity of the JDK APIs" perspective, it is unquestionably a > drawback. These items are not a Key and an associated Value in a Map; > it's merely pretending that Map.Entry really means "Pair". There's a > reason we don't have a Pair class in the JDK (and no, let's not reopen > that now); using something else as a Pair proxy that is supposed to have > specific semantics is worse. (It's fine to do this in your own code, but > not in the JDK. Different standards for code that has different audiences.) > > Tagir's proposed sidestepping is nice, and it will also play nicely with > records, because then you can say: > > record NameAndCount(String name, int count); > > stream.collect(pairing(collectName, collectCount, NameAndCount::new)); > > and get a more properly abstract result out. And its more in the spirit > of existing Collectors. If you want to use Map.Entry as an > _implementation detail_, that's fine. > > I can support this form. > > > I also don't see a larger abstraction like BiStream as a natural fit > > for a similar thing. > > I think the BiStream connection is mostly tangential. We tried hard to > support streams of (K,V) pairs when we did streams, as Paul can attest, > but it was a huge complexity-inflater and dropping this out paid an > enormous simplification payoff. > > With records, having streams of tuples will be simpler to represent, but > no more performant; it will take until we get to value types and > specialized generics to get the performance we want out of this. > > From amaembo at gmail.com Sat Jun 16 04:01:35 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Sat, 16 Jun 2018 11:01:35 +0700 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: Hello! If you are going to create long-living array which is likely to be empty, it's good to deduplicate them to spare the heap space. This can be easily done via existing toArray method like collection.toArray(MyClass.EMPTY_ARRAY) assuming we have an empty array constant. We use this pattern very often in IntelliJ IDEA source code (e. g. think of method which returns an array of Java member annotations; often there are none). New generator methods is much less convenient for this: toArray(s -> s == 0?MyClass.EMPTY_ARRAY:new MyClass[s]). I'm actually missing a method accepting an empty array instead of generator in the Stream API. And I don't think that new collection method is useful. The existing one is perfect. With best regards, Tagir Valeev ??, 15 ???? 2018 ?., 8:03 Stuart Marks : > Hi all, > > I wanted to restart review of the changeset for bug JDK-8060192 [1]. The > latest > webrev is here: [2]. > > The previous review was from December 2017: [3]. The only difference > between the > current changeset and the previous one is the removal of the overriding > implementations (see explanation below). Much of the discussion in the > previous > review thread was around performance and the shape of the API. > > # Performance > > I previously had done some benchmarking on this and reported the results: > [4]. > (I've recently re-done the benchmarks and conclusions are essentially the > same.) > > The upshot is that implementations that use Arrays.copyOf() are the > fastest, > probably because it's an intrinsic. It can avoid zero-filling the freshly > allocated array because it knows the entire array contents will be > overwritten. > This is true regardless of what the public API looks like. The default > implementation calls generator.apply(0) to get a zero-length array and > then > simply calls toArray(T[]). This goes through the Arrays.copyOf() fast > path, so > it's essentially the same speed as toArray(new T[0]). > > The overrides I had previously provided in specific implementation classes > like > ArrayList actually are slower, because the allocation of the array is done > separately from filling it. This necessitates the extra zero-filling step. > Thus, > I've removed the overrides. > > # API Shape > > There was some discussion about whether it would be preferable to have a > Class > parameter as a type token for the array component type or the array type > itself, > instead of using an IntFunction generator. Most of this boils down to what > people are comfortable with. However, there are a few points that make the > generator function approach preferable. > > One point in favor of using a generator is that Stream already has a > similar > toArray(generator) method. > > Comparing this to the type token alternatives, for simple tasks like > converting > Collection to String[], things are about equal: > > toArray(String[]::new) > toArray(String.class) > toArray(String[].class) > > However, things are hairier if the element type of the collection is > generic, > such as Collection>. The result type is a generic array > List[]. Using class literals or array constructor references will > necessitate using an unchecked cast, because none of these can be used to > represent a generic type. > > However, it's possible to use a method reference to provide a properly > generic > IntFunction. This would enable the toArray(generator) method to be used > without > any unchecked casts. (The method it refers to might need have an unchecked > cast > within it, though.) Such a method could also be reused for the > corresponding > Stream.toArray(generator) method. > > For these reasons I'd like to proceed with adding toArray(generator) API. > > Thanks, > > s'marks > > > [1] https://bugs.openjdk.java.net/browse/JDK-8060192 > > [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ > > [3] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html > > [4] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html > > From scolebourne at joda.org Sat Jun 16 09:33:04 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Sat, 16 Jun 2018 10:33:04 +0100 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: Without commenting on the proposed implementation below, Future seems like a good place to start. I suspect that ExecutorService and ScheduledExecutorService would also be good targets to enhance. Stephen On Fri, 15 Jun 2018, 18:43 Martin Buchholz, wrote: > > > On Wed, May 30, 2018 at 11:32 AM, Doug Lea
wrote: > >> >> The original rationale for designing j.u.c.TimeUnit using the Flyweight >> pattern was to to reduce allocation and GC-related overhead and timing >> jitter for methods that otherwise may operate on the order of >> nanoseconds. But there are many cases in which this is not much of a >> concern (plus JVMs can now sometimes optimize), so people should be >> given a choice. It would be a lot of tedious work (and aggregate code >> bulk) to retrofit every time-related j.u.c method though, and it's not >> clear where to compromise. But at least adding converters should not be >> controversial. >> > > Re-reading Doug's assessment, Doug seems reluctant but open to adding at > least some Duration overloads. Here's an obvious first candidate in Future > (yes, we have a test that discovers get(Duration) and checks that > get(null) throws UOE instead of NPE.). > (It's a lot of tedious work) > > a/util/concurrent/CompletableFuture.java > =================================================================== > RCS file: > /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/CompletableFuture.java,v > retrieving revision 1.212 > diff -u -r1.212 CompletableFuture.java > --- src/main/java/util/concurrent/CompletableFuture.java 11 Mar 2018 > 18:00:05 -0000 1.212 > +++ src/main/java/util/concurrent/CompletableFuture.java 15 Jun 2018 > 17:39:09 -0000 > @@ -1983,13 +1983,22 @@ > * while waiting > * @throws TimeoutException if the wait timed out > */ > - @SuppressWarnings("unchecked") > public T get(long timeout, TimeUnit unit) > throws InterruptedException, ExecutionException, TimeoutException > { > - long nanos = unit.toNanos(timeout); > + return getNanos(unit.toNanos(timeout)); > + } > + > + public T get(java.time.Duration timeout) > + throws InterruptedException, ExecutionException, TimeoutException > { > + return getNanos(TimeUnit.NANOSECONDS.convert(timeout)); > + } > + > + @SuppressWarnings("unchecked") > + private T getNanos(long timeoutNanos) > + throws InterruptedException, ExecutionException, TimeoutException > { > Object r; > if ((r = result) == null) > - r = timedGet(nanos); > + r = timedGet(timeoutNanos); > return (T) reportGet(r); > } > > @@ -2797,6 +2806,8 @@ > throw new UnsupportedOperationException(); } > @Override public T get(long timeout, TimeUnit unit) { > throw new UnsupportedOperationException(); } > + @Override public T get(java.time.Duration duration) { > + throw new UnsupportedOperationException(); } > @Override public T getNow(T valueIfAbsent) { > throw new UnsupportedOperationException(); } > @Override public T join() { > Index: src/main/java/util/concurrent/Future.java > =================================================================== > RCS file: > /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/Future.java,v > retrieving revision 1.41 > diff -u -r1.41 Future.java > --- src/main/java/util/concurrent/Future.java 8 Oct 2016 18:52:37 -0000 > 1.41 > +++ src/main/java/util/concurrent/Future.java 15 Jun 2018 17:39:09 -0000 > @@ -6,6 +6,10 @@ > > package java.util.concurrent; > > +import static java.util.concurrent.TimeUnit.NANOSECONDS; > + > +import java.time.Duration; > + > /** > * A {@code Future} represents the result of an asynchronous > * computation. Methods are provided to check if the computation is > @@ -126,7 +130,30 @@ > * @throws InterruptedException if the current thread was interrupted > * while waiting > * @throws TimeoutException if the wait timed out > + * @throws NullPointerException if {@code unit} is null > */ > V get(long timeout, TimeUnit unit) > throws InterruptedException, ExecutionException, TimeoutException; > + > + /** > + * Waits if necessary for at most the given time for the computation > + * to complete, and then retrieves its result, if available. > + * > + *

Equivalent to: 

 {@code
> +     * get(NANOSECONDS.convert(timeout), NANOSECONDS)}
> + * > + * @param timeout the maximum time to wait > + * @return the computed result > + * @throws CancellationException if the computation was cancelled > + * @throws ExecutionException if the computation threw an > + * exception > + * @throws InterruptedException if the current thread was interrupted > + * while waiting > + * @throws TimeoutException if the wait timed out > + * @throws NullPointerException if {@code timeout} is null > + */ > + default V get(Duration timeout) > + throws InterruptedException, ExecutionException, TimeoutException > { > + return get(NANOSECONDS.convert(timeout), NANOSECONDS); > + } > } > > From Alan.Bateman at oracle.com Sat Jun 16 12:42:52 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 16 Jun 2018 13:42:52 +0100 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> Message-ID: <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> On 13/06/2018 15:10, Roger Riggs wrote: > Hi Joe, > > The CSR text is still in draft and got out of sync with the open > review and comments. > > The updated apiNote clarifies that changes made through the > System.*property*? methods > may not affect the value cached during initialization. > The implementation changes affect a very short list of properties. > > The note on the methods is to raise awareness that individual > properties may have > different behavior and may be unpredictable. Just catching up on this. I checked the CSR and the webrev (the latest webrev was generated on June 6 so I hope that is the version we are meant to look at). The updated apiNote (CSR version) looks okay but I think the word "internal" needs to be dropped as it comes with too many questions. "may be cached during initialization or ..." should be okay. The editing has meant the line lengths are a bit inconsistent so I assume they can be fixed up before the change is pushed. I see the original patch to SocksSocketImpl has been mostly reverted. It looks correct now. The other usages look okay to me. -Alan From peter.levart at gmail.com Sun Jun 17 08:50:58 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 10:50:58 +0200 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: <940f6307-34fd-eea0-88e8-c8f5134b6829@gmail.com> Hi Stuart, On 06/15/18 03:02, Stuart Marks wrote: > Comparing this to the type token alternatives, for simple tasks like > converting Collection to String[], things are about equal: > > ??? toArray(String[]::new) > ??? toArray(String.class) > ??? toArray(String[].class) > > However, things are hairier if the element type of the collection is > generic, such as Collection>. The result type is a > generic array List[]. Using class literals or array > constructor references will necessitate using an unchecked cast, > because none of these can be used to represent a generic type. > > However, it's possible to use a method reference to provide a properly > generic IntFunction. This would enable the toArray(generator) method > to be used without any unchecked casts. (The method it refers to might > need have an unchecked cast within it, though.) Such a method could > also be reused for the corresponding Stream.toArray(generator) method. > > For these reasons I'd like to proceed with adding toArray(generator) API. It's a little strange that the generator function is used to construct an empty array (at least in the default method, but overrides will likely do the same if they want to avoid pre-zeroing overhead) in order to just extract the array's type from it. Someone could reasonably expect the provided function to be called with the exact length of needed array. The Collection.toArray(T[]) at least gives user an option to actually fully use the provided array, if user provides the correct length. The argument about using (and re-using) a method so that a method reference can be passed to the method without any unchecked casts is equally true for any of the 3 alternatives shown - the method itself might need unchecked casts, but its usage not: static List[] array(int len) static Class> elementType() static Class[]> arrayType() But I can see that you want to align the API with Stream.toArray, while still providing the optimal implementation. It's just that the API doesn't fit the usecase. The function approach makes more sense in the Stream API where it is explicitly explained that it may be used to construct arrays needed to hold intermediary results of partitioned parallel execution too, but in Collection API it is usually the case to just provide a copy of the underlying data structure in the most optimal way (without pre-zeroing overhead) and for that purpose, 2nd and 3rd alternatives are a better fit. Suppose Stream took a different approach and used the 2nd or 3rd approach (which is universal). Would then Collection API also get the same method? Regards, Peter From peter.levart at gmail.com Sun Jun 17 09:08:58 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 11:08:58 +0200 Subject: BiCollector In-Reply-To: References: Message-ID: <1140f3ba-dad8-4809-97ac-63a461f61d7b@gmail.com> On 06/15/18 13:31, Tagir Valeev wrote: > Peter, > > > ? ? ? ?EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > > ???????? intersection.retainAll(valCollector.characteristics()); > > ???????? return intersection; > > Please note that `copyOf` should either receive an EnumSet or > non-empty Collection to obtain the enum class. This is not guaranteed > by `characteristics()` implementation, thus failure is possible. > > Testcase: > new BiCollector<>(Collectors.counting(), > Collectors.toSet()).characteristics(); > Fails with > Exception in thread "main" java.lang.IllegalArgumentException: > Collection is empty > at java.base/java.util.EnumSet.copyOf(EnumSet.java:173) > at BiCollector.characteristics(BiCollector.java:77) > at Main.main(Main.java:21) Yeah, it came to my mind after I posted the example implementation. A case where API design sometimes misleads even experienced programmers, because it re-uses an idiom that is usually right (for most other Set(s)), but fails for that particular case. EnumSet needs an explicit enum type, so it should have been provided as an explicit argument... Regards, Peter > > > With best regards, > Tagir Valeev. > > On Mon, Jun 11, 2018 at 7:40 PM Peter Levart > wrote: > > Hi, > > Have you ever wanted to perform a collection of the same Stream > into two > different targets using two Collectors? Say you wanted to collect > Map.Entry elements into two parallel lists, each of them > containing keys > and values respectively. Or you wanted to collect elements into? > groups > by some key, but also count them at the same time? Currently this > is not > possible to do with a single Stream. You have to create two identical > streams, so you end up passing Supplier to other methods > instead > of bare Stream. > > I created a little utility Collector implementation that serves the > purpose quite well: > > /** > ??* A {@link Collector} implementation taking two delegate > Collector(s) > and producing result composed > ??* of two results produced by delegating collectors, wrapped in > {@link > Map.Entry} object. > ??* > ??* @param the type of elements collected > ??* @param the type of 1st delegate collector collected result > ??* @param tye type of 2nd delegate collector collected result > ??*/ > public class BiCollector implements Collector Map.Entry, Map.Entry> { > ???? private final Collector keyCollector; > ???? private final Collector valCollector; > > ???? @SuppressWarnings("unchecked") > ???? public BiCollector(Collector keyCollector, > Collector V> valCollector) { > ???????? this.keyCollector = (Collector) > Objects.requireNonNull(keyCollector); > ???????? this.valCollector = (Collector) > Objects.requireNonNull(valCollector); > ???? } > > ???? @Override > ???? public Supplier> supplier() { > ???????? Supplier keySupplier = keyCollector.supplier(); > ???????? Supplier valSupplier = valCollector.supplier(); > ???????? return () -> new > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), > valSupplier.get()); > ???? } > > ???? @Override > ???? public BiConsumer, T> accumulator() { > ???????? BiConsumer keyAccumulator = > keyCollector.accumulator(); > ???????? BiConsumer valAccumulator = > valCollector.accumulator(); > ???????? return (accumulation, t) -> { > ???????????? keyAccumulator.accept(accumulation.getKey(), t); > ???????????? valAccumulator.accept(accumulation.getValue(), t); > ???????? }; > ???? } > > ???? @Override > ???? public BinaryOperator> combiner() { > ???????? BinaryOperator keyCombiner = keyCollector.combiner(); > ???????? BinaryOperator valCombiner = valCollector.combiner(); > ???????? return (accumulation1, accumulation2) -> new > AbstractMap.SimpleImmutableEntry<>( > ???????????? keyCombiner.apply(accumulation1.getKey(), > accumulation2.getKey()), > ???????????? valCombiner.apply(accumulation1.getValue(), > accumulation2.getValue()) > ???????? ); > ???? } > > ???? @Override > ???? public Function, Map.Entry> > finisher() { > ???????? Function keyFinisher = keyCollector.finisher(); > ???????? Function valFinisher = valCollector.finisher(); > ???????? return accumulation -> new > AbstractMap.SimpleImmutableEntry<>( > ???????????? keyFinisher.apply(accumulation.getKey()), > ???????????? valFinisher.apply(accumulation.getValue()) > ???????? ); > ???? } > > ???? @Override > ???? public Set characteristics() { > ???????? EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > intersection.retainAll(valCollector.characteristics()); > ???????? return intersection; > ???? } > } > > > Do you think this class is general enough to be part of standard > Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector > coll1, Collector coll2), bi-collection could then be coded simply as: > > ???????? Map map = ... > > ???????? Map.Entry, List> keys_values = > ???????????? map.entrySet() > ??????????????? .stream() > ??????????????? .collect( > ??????????????????? toBoth( > ??????????????????????? mapping(Map.Entry::getKey, toList()), > ??????????????????????? mapping(Map.Entry::getValue, toList()) > ??????????????????? ) > ??????????????? ); > > > ???????? Map.Entry, Long> histogram_count = > ???????????? ThreadLocalRandom > ???????????????? .current() > ???????????????? .ints(100, 0, 10) > ???????????????? .boxed() > ???????????????? .collect( > ???????????????????? toBoth( > ???????????????????????? groupingBy(Function.identity(), counting()), > ???????????????????????? counting() > ???????????????????? ) > ???????????????? ); > > > Regards, Peter > From forax at univ-mlv.fr Sun Jun 17 09:15:37 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 17 Jun 2018 11:15:37 +0200 (CEST) Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: <396353523.2274681.1529226937272.JavaMail.zimbra@u-pem.fr> Hi Tagir, ----- Mail original ----- > De: "Tagir Valeev" > ?: "Stuart Marks" > Cc: "core-libs-dev" > Envoy?: Samedi 16 Juin 2018 06:01:35 > Objet: Re: RFR(s): 8060192: Add default method Collection.toArray(generator) > Hello! > > If you are going to create long-living array which is likely to be empty, > it's good to deduplicate them to spare the heap space. This can be easily > done via existing toArray method like > collection.toArray(MyClass.EMPTY_ARRAY) assuming we have an empty array > constant. We use this pattern very often in IntelliJ IDEA source code (e. > g. think of method which returns an array of Java member annotations; often > there are none). New generator methods is much less convenient for this: > toArray(s -> s == 0?MyClass.EMPTY_ARRAY:new MyClass[s]). I'm actually > missing a method accepting an empty array instead of generator in the > Stream API. And I don't think that new collection method is useful. The > existing one is perfect. > > With best regards, > Tagir Valeev we can expect the VM to not allocate the array if once inlined the code is new MyClass[0].getClass() if it's not the case, i think it's a better idea to tweak the VM rather than try to change an API based on a pattern that should not be necessary. R?mi > > ??, 15 ???? 2018 ?., 8:03 Stuart Marks : > >> Hi all, >> >> I wanted to restart review of the changeset for bug JDK-8060192 [1]. The >> latest >> webrev is here: [2]. >> >> The previous review was from December 2017: [3]. The only difference >> between the >> current changeset and the previous one is the removal of the overriding >> implementations (see explanation below). Much of the discussion in the >> previous >> review thread was around performance and the shape of the API. >> >> # Performance >> >> I previously had done some benchmarking on this and reported the results: >> [4]. >> (I've recently re-done the benchmarks and conclusions are essentially the >> same.) >> >> The upshot is that implementations that use Arrays.copyOf() are the >> fastest, >> probably because it's an intrinsic. It can avoid zero-filling the freshly >> allocated array because it knows the entire array contents will be >> overwritten. >> This is true regardless of what the public API looks like. The default >> implementation calls generator.apply(0) to get a zero-length array and >> then >> simply calls toArray(T[]). This goes through the Arrays.copyOf() fast >> path, so >> it's essentially the same speed as toArray(new T[0]). >> >> The overrides I had previously provided in specific implementation classes >> like >> ArrayList actually are slower, because the allocation of the array is done >> separately from filling it. This necessitates the extra zero-filling step. >> Thus, >> I've removed the overrides. >> >> # API Shape >> >> There was some discussion about whether it would be preferable to have a >> Class >> parameter as a type token for the array component type or the array type >> itself, >> instead of using an IntFunction generator. Most of this boils down to what >> people are comfortable with. However, there are a few points that make the >> generator function approach preferable. >> >> One point in favor of using a generator is that Stream already has a >> similar >> toArray(generator) method. >> >> Comparing this to the type token alternatives, for simple tasks like >> converting >> Collection to String[], things are about equal: >> >> toArray(String[]::new) >> toArray(String.class) >> toArray(String[].class) >> >> However, things are hairier if the element type of the collection is >> generic, >> such as Collection>. The result type is a generic array >> List[]. Using class literals or array constructor references will >> necessitate using an unchecked cast, because none of these can be used to >> represent a generic type. >> >> However, it's possible to use a method reference to provide a properly >> generic >> IntFunction. This would enable the toArray(generator) method to be used >> without >> any unchecked casts. (The method it refers to might need have an unchecked >> cast >> within it, though.) Such a method could also be reused for the >> corresponding >> Stream.toArray(generator) method. >> >> For these reasons I'd like to proceed with adding toArray(generator) API. >> >> Thanks, >> >> s'marks >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8060192 >> >> [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ >> >> [3] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html >> >> [4] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html >> From peter.levart at gmail.com Sun Jun 17 09:44:10 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 11:44:10 +0200 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <940f6307-34fd-eea0-88e8-c8f5134b6829@gmail.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> <940f6307-34fd-eea0-88e8-c8f5134b6829@gmail.com> Message-ID: <628e3775-02b0-a046-e54c-5480d59ff98c@gmail.com> On 06/17/18 10:50, Peter Levart wrote: > The argument about using (and re-using) a method so that its method > reference can be passed to the toArray method without any unchecked > casts is equally true for any of the 3 alternatives shown - the method > itself might need unchecked casts, but its usage not: > > static List[] array(int len) > static Class> elementType() > static Class[]> arrayType() For that purpose, I would rather get rid of the need to create a method at all. It might have been the case in the past when Java generics were introduced, that class literals like List.class? would just confuse users, because most aspects of such type token would be erased and there were fears that enabling them might limit options for reified generics in the future. But long years have passed and java programmers have generally become acquainted with Java generics and erasure to the point where it doesn't confuse them any more, while reifying Java generics has been explored further in Valhalla to the point where it has become obvious that erasure of reference types is here to stay. Java could enable use of class literals like List.class without fear that such literals would make users write buggy code or that such uses would limit options for Java in the future. Quite the contrary, they would enable users to write type-safer code than what they can write today. In the light of future Java? (valhalla specialized generics), where such class literals make even more sense, enabling generic class literals could be viewed as a stepping stone that has some purpose in the short term too. So what do you think? Regards, Peter From peter.levart at gmail.com Sun Jun 17 10:04:50 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 12:04:50 +0200 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> Message-ID: Hi Tagir, I don't know if this is important, but in your approach the particular functions of the sub-collectors are retrieved eagerly even if they are later not used. This should typically not present a problem as Collector(s) should be prepared to be used in any scenario (parallel or serial). But anyway in my approach, the sub-functions of the given collectors are retrived lazily, each time the Stream implementation needs them - the dynamics of invoking sub-collector methods to retrieve the functions is therefore unchanged when a collector is used directly to collect a Stream or as a sub-collector in the bi-collection scenario. What do you think of that particular detail? Paul? Regards, Peter On 06/15/18 13:26, Tagir Valeev wrote: > Hello! > > I created a preliminary webrev of my own implementation (no testcases > yet): > http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ > > If anybody wants to sponsor my implementation, I will happily log an > issue and write tests. > > The name "pairing" was invented by me, but as I'm not a native English > speaker I cannot judge whether it's optimal, so better ideas are welcome. > Also I decided to remove accumulator types from public type variables. > They do not add anything to type signature, only clutter it > increasing the number of type parameters from 4 to 6. I think it was a > mistake to expose the accumulator type parameter in other cascading > collectors > like filtering(), collectingAndThen(), groupingBy(), etc. I'm not > insisting though, if you feel that conformance to existing collectors is > more important than simplicity. > > With best regards, > Tagir Valeev. > > On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz > wrote: > > > > Well, I don't see the need to pack the two results into a Map.Entry > > (or any similar) container as a drawback. > > ?From an "integrity of the JDK APIs" perspective, it is > unquestionably a > drawback.? These items are not a Key and an associated Value in a > Map; > it's merely pretending that Map.Entry really means "Pair". There's a > reason we don't have a Pair class in the JDK (and no, let's not > reopen > that now); using something else as a Pair proxy that is supposed > to have > specific semantics is worse. (It's fine to do this in your own > code, but > not in the JDK. Different standards for code that has different > audiences.) > > Tagir's proposed sidestepping is nice, and it will also play > nicely with > records, because then you can say: > > ????? record NameAndCount(String name, int count); > > ????? stream.collect(pairing(collectName, collectCount, > NameAndCount::new)); > > and get a more properly abstract result out.? And its more in the > spirit > of existing Collectors.? If you want to use Map.Entry as an > _implementation detail_, that's fine. > > I can support this form. > > > I also don't see a larger abstraction like BiStream as a natural > fit > > for a similar thing. > > I think the BiStream connection is mostly tangential.? We tried > hard to > support streams of (K,V) pairs when we did streams, as Paul can > attest, > but it was a huge complexity-inflater and dropping this out paid an > enormous simplification payoff. > > With records, having streams of tuples will be simpler to > represent, but > no more performant; it will take until we get to value types and > specialized generics to get the performance we want out of this. > > From peter.levart at gmail.com Sun Jun 17 20:56:24 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 22:56:24 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: <75f225b7-e22a-b31c-004b-2c3fab4cfe16@gmail.com> Hi Brent, If 'map' field could be observed as null (which I think is only possible, if Properties object is unsafely published), then so can 'defaults' field. You could make it final as well if it was not protected. So I'm thinking what kind of memory fences would make those fields behave so that they would not be observed uninitialized even when the instance is published via data race. I think that the following would be enough, for example: ??? private Properties(Properties defaults, int initialCapacity) { ??????? // use package-private constructor to ??????? // initialize unused fields with dummy values ??????? super((Void) null); ??????? map = new ConcurrentHashMap<>(initialCapacity); ??????? this.defaults = defaults; ??? ??? VarHandle.storeStoreFence(); ??? } ... ??? public String getProperty(String key) { ??? ??? VarHandle.loadLoadFence(); ??????? Object oval = map.get(key); ??????? String sval = (oval instanceof String) ? (String)oval : null; ??????? return ((sval == null) && (defaults != null)) ? defaults.getProperty(key) : sval; ??? } ...of course, you would have to put loadLoadFence() at the beggining of each method that uses any of those two fields. Accesses to 'defaults' in subclasses wouldn't be included, but at least accesses in Properties class would be covered. Regards, Peter On 06/15/18 17:44, Brent Christian wrote: > Hi, > > In JDK 9, 8029891[1] refactored java.util.Properties to store its > values in an internal ConcurrentHashMap, and removed synchronization > from "reader" methods in order to avoid potential hangs/deadlocks > during classloading. > > Claes has noticed that there is the possibility of the new 'map' field > being observed with its default value (null), before being set. > > After looking at the JSR 133 FAQ[2], I agree with Claes that we should > make 'map' a field final. > > Please review my change to do this: > > Webrev: > http://cr.openjdk.java.net/~bchristi/8199435/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199435 > > Thanks, > -Brent > > 1. https://bugs.openjdk.java.net/browse/JDK-8029891 > 2. > https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#finalRight From peter.levart at gmail.com Sun Jun 17 21:20:08 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 23:20:08 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: Update: the discussion on concurrent-interest about possible future addition of public ThreadLocal.getIfPresent() or ThreadLocal.isPresent() has died out, but it nevertheless reached a point where ThreadLocal.isPresent() was found the least problematic. So I would like to get further with this proposal using the last webrev.04 version of the patch that uses ThreadLocal.isPresent() internally - still package-private at the moment. Here's the webrev (unchanged from the day it was published): http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ Would this version be OK? Regards, Peter On 06/06/18 20:55, Peter Levart wrote: > Ok, here's next webrev: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.03/ > > The changes from webrev.02 are: > > - renamed JdkThreadLocal -> TerminatingThreadLocal > - instead of overriding initialValue(), ThreadLocal.setInitialValue() > calls TerminatingThreadLocal.register() at the right moment > - TerminatingThreadLocal.threadTerminated() now takes a (T value) > parameter > - TerminatingThreadLocal.REGISTRY uses IdentityHashSet instead of > HashSet (if .equals()/.hashCode() are overriden in some > TerminatingThreadLocal subclass) > > David Lloyd has commented on concurrency-interest about > ThreadLocal.getIfPresent(). There is a concern that such new method > would be inconsistent with what ThreadLocal.get() returns from > existing ThreadLocal subclasses that override .get() and possibly > transform the value returned from super.get() on the way. > > An alternative to "T getIfPresent()" is a "boolean isPresent()" > method. Even if it remains package-private, with such method > TerminatingThreadLocal could be implemented as: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ > > If TreadLocal.isPresent() was made public, the code could be further > simplified. > > > Regards, Peter > > > On 06/06/18 18:58, Peter Levart wrote: >> >> >> On 06/06/18 17:41, Tony Printezis wrote: >>> Peter, >>> >>> You?re totally right re: JdkThreadLocal::initialValue being final >>> and cannot be overridden (I had completely missed the final keyword >>> when I first looked at the webrev). But it?d be nice to have the >>> same API in both versions of ThreadLocal. I assume you didn?t want >>> to override get() since you only wanted to update the REGISTRY the >>> first time get() is called by a thread. If getIfPresent() is >>> exposed, would something like the following work? >>> >>> @Override >>> public T get() { >>> ? final T value = getIfPresent(); >>> ? if (value != null) { >>> ? ? ? return value; >>> ? } >>> ? REGISTRY.get().add(this); >>> ? return super.get(); >>> } >> >> This would work, but if mapped value was 'null', it would keep >> calling REGISTRY.get().add(this) for each get(). Logically correct, >> but with useless overhead. >> >> ?Let me try to do something that might satisfy you... (in next webrev). >> >>> >>> One more question re: getIfPresent() (and maybe I?m overthinking >>> this): It returns null to indicate that a value is not present. >>> Isn?t null a valid ThreadLocal value? Would using an Optional here >>> be more appropriate? >> >> The problem with Optional is that it does not provide an additional >> value. Optional.empty() is a replacement for null. There's no >> Optional.of(null). So what would getIfPresent() return when there was >> a mapping present, but that mapping was null? >> >> Regards, Peter >> >>> >>> Tony >>> >>> ????? >>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>> >>> >>> >>> On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com >>> ) wrote: >>> >>>> Hi Tony, Alan, >>>> >>>> On 06/06/2018 04:37 PM, Tony Printezis wrote: >>>>> Alan, >>>>> >>>>> Thanks for your thoughts! >>>>> >>>>> Peter, >>>>> >>>>> Any chance of taking the two suggestions I made in an earlier >>>>> e-mail into account? >>>> >>>> Right, I'll prepare new webrev with that shortly. >>>> >>>>> >>>>> a) It?d be nice if users can override initialValue(), like when >>>>> using the standard ThreadLocal class, instead of >>>>> calculateInitialValue(). (I can?t come up with a clean solution on >>>>> how to do that, though. I?ll think about it for a bit.) >>>> >>>> Your concern was that users might accidentally override >>>> initialValue() instead of calculateInitialValue(), if I understood >>>> you correctly. If that was the concern, they can't, because it is >>>> final. If the concern was that users would want to override >>>> initialValue() because they are used to do so, then perhaps a >>>> javadoc on final initialiValue() pointing to >>>> calculateInitialValue() might help them. Do you agree? This is >>>> currently internal API and consistent naming is not a big concern. >>>> >>>>> b) It?d be very helpful to pass T value to threadTerminated(), as >>>>> I cannot imagine many use-cases where the value will not be needed. >>>> >>>> Agree. Will include that in new webrev. >>>> >>>>> >>>>> Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? >>>> >>>> Hm. Exit Hooks are already something that is used in JVM (Threads >>>> started when VM is about to exit), so this might be confusing for >>>> someone. >>>> >>>> - DisposableThreadLocal >>>> - ThreadLocalWithCleanup >>>> >>>> And my favorite: >>>> >>>> - TerminatingThreadLocal >>>> >>>> >>>>> >>>>> Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be >>>>> very helpful and can avoid completely unnecessary allocations. >>>> >>>> I agree that this would be generally useful functionality. Might be >>>> good to ask for opinion on concurrency-interest mailing list first. >>>> Will do that. >>>> >>>> Regards, Peter >>>> >>>>> >>>>> Tony >>>>> >>>>> >>>>> ????? >>>>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>>>> >>>>> >>>>> >>>>> On June 6, 2018 at 9:38:05 AM, Alan Bateman >>>>> (alan.bateman at oracle.com ) wrote: >>>>> >>>>>> >>>>>> >>>>>> On 30/05/2018 22:16, Peter Levart wrote: >>>>>> > I thought there would be some hint from Alan about which of the two >>>>>> > paths we should take for more refinement. >>>>>> > >>>>>> > The Tony's: >>>>>> > >>>>>> > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ >>>>>> >>>>>> > >>>>>> > Or the Alan's with my mods: >>>>>> > >>>>>> > >>>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ >>>>>> >>>>>> > >>>>>> Finally getting back to this one. >>>>>> >>>>>> I don't think the two approaches are all that different now. Tony's >>>>>> point on the number of hooks vs. number of locals was an >>>>>> important point >>>>>> but with Peter's update to use a thread local registry then I >>>>>> think we >>>>>> have easy to maintain solution in the DBBCache_Cleanup/webrev.02 >>>>>> patch. >>>>>> So I think I prefer that approach. We need to better name for >>>>>> "JdkThreadLocal", something to indicate that it holds resources or it >>>>>> notified when a thread terminates. >>>>>> >>>>>> Also along the way, we touched on exposing getIfPresent and we should >>>>>> look at that again. If it is expose then it fits well with the second >>>>>> approach too. >>>>>> >>>>>> -Alan >>>> >> > From peter.levart at gmail.com Sun Jun 17 23:23:23 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Jun 2018 01:23:23 +0200 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: Hi Stuart, On 06/15/18 23:22, Stuart Marks wrote: > Hi Peter, > > Finally getting back to this. > > I think there's clearly room for improvement in the creation of the > unmodifiable collections. Right now the creation paths (mostly) use > only public APIs, which can result in unnecessary allocation and > copying. Certainly Map.copyOf does this, but there are also other > cases as well. My attempt to optimize the Map.copyOf() was also using just public APIs (with the addition of an internal class). > > For copying from an unknown map, there are a couple approaches: > > * accumulate keys and values in a single ArrayList, which can be > presized as necessary, but which will grow if necessary; then copy > elements from a subrange of ArrayList's internal array (similar to > your MapN(Object[], len) overload) > > * accumulate keys and values into a SpinedBuffer, which doesn't > require copying to grow, which is preferable if for some reason we > can't pre-size accurately; and then copy the elements out of it > > The Collectors.toUnmodifiableMap implementations are known to create > HashMap instances, so they could pass the HashMap directly to a > private MapN constructor that in turn could talk directly to HashMap > to get the keys and values. This avoids allocation of a full-sized > buffer and one copy. > > Note that these techniques involve creating new interfaces, sometimes > that cross package boundaries. It's a bit of an irritant to have to > plumb new paths that go through SharedSecrets, but it seems likely to > be worthwhile if we can avoid bulk allocation and copying steps. > > Given these, it doesn't seem to me that the BiBuffer approach helps > very much. I think there are many other avenues that would be > worthwhile to explore, and that possibly can provide bigger savings. Well, it does speed-up Map.copyOf() by 30%. But I agree there are other approaches that could go even further. In particular when all intermediary copies could be avoided. Here's one approach (which still uses just public APIs) and avoids intermediary copying: http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.02/ Why choosing Map.forEach for dumping the content of the map instead of iteration over entrySet? Because in some Map implementations this is the most direct iteration without producing garbage (for example in IdentityHashMap), but in the worst case (default Map method for example) it is delegated to iteration over entrySet. I'll benchmark it tomorrow when I get back to the computer other benchmarks were run on so the results can be compared. So what do you think of this one? Regards, Peter > > > s'marks > > > > > > On 6/11/18 3:57 AM, Peter Levart wrote: >> Hi, >> >> Those two methods were added in JDK 10 and they are not very optimal. >> Map.copyOf(Map map) 1st dumps the source map into an array of >> Map.Entry(s) (map.entrySet().toArray(new Entry[0])), which typically >> creates new Map.Entry objects, then passes the array to >> Map.ofEntries(Map.Entry[] entries) factory method that iterates the >> array and constructs a key/value interleaved array from it which is >> passed to ImmutableCollections.MapN constructor, which then >> constructs a linear-probing hash table from it. So each key and value >> is copied 3 times, while several temporary objects are created in the >> process. >> >> One copying step could be eliminated and construction of temporary >> Map.Entry objects too: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.01/ >> >> >> Collecting stream(s) using Collectors.toUnmodifiableMap() 1st >> collects key/value pairs into a HashMap, then it performs equivalent >> operations as Map.copyOf(hashMap) at the end. Using Map.copyOf() >> directly benefits this collection operation too. >> >> The following benchmark: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench.java >> >> >> >> Shows up to 30% improvement of .copyOf operation with this patch >> applied: >> >> Original: >> >> Benchmark?????????????????????????????? (size)? Mode Cnt Score Error? >> Units >> UnmodifiableMapBench.copyOf???????????????? 10? avgt 10 403.633 ? >> 2.640? ns/op >> UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 3489.623 ? >> 44.590? ns/op >> UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 40030.572 ? >> 277.075? ns/op >> UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10 831.221 ? >> 3.816? ns/op >> UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9783.519 ? >> 43.097? ns/op >> UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 96524.536 ? >> 670.818? ns/op >> >> Patched: >> >> Benchmark?????????????????????????????? (size)? Mode Cnt Score Error? >> Units >> UnmodifiableMapBench.copyOf???????????????? 10? avgt 10 264.172 ? >> 1.882? ns/op >> UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 2318.974 ? >> 15.877? ns/op >> UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 29291.782 ? >> 3139.737? ns/op >> UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10 771.221 ? >> 65.432? ns/op >> UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9275.016 ? >> 725.722? ns/op >> UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 82204.342 ? >> 851.741? ns/op >> >> >> Production of garbage is also reduced, since no Map.Entry temporary >> objects are constructed: >> >> Original: >> >> Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10 416.001 >> ????? 0.002??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 >> 2936.005 ????? 0.019??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 >> 28136.059 ????? 0.199??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10 avgt >> 10?? 1368.001 ????? 0.004??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100 avgt >> 10? 10208.139 ????? 0.045??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000 avgt >> 10? 93025.923 ????? 0.573??? B/op >> >> Patched: >> >> Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10 304.000 >> ????? 0.001??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 >> 2464.004 ????? 0.012??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 >> 24064.040 ????? 0.137??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10 avgt >> 10?? 1256.001 ????? 0.003??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100 avgt >> 10?? 9720.153 ????? 0.055??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000 avgt >> 10? 88905.688 ????? 0.574??? B/op >> >> >> So what do you think? Is this an improvement? >> >> Regards, Peter >> From srinivas.dama at oracle.com Mon Jun 18 02:35:59 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Sun, 17 Jun 2018 19:35:59 -0700 (PDT) Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) Message-ID: <0ac897d4-d7f2-47a0-b5b1-ec2421880cfe@default> Thank you Paul for the review. >I am guessing the comment selectively disables the warning in this code (rather than using a diagnostic pragma). yes I have updated the patch on latest repo changes. http://cr.openjdk.java.net/~sdama/8196988/webrev.02/ pushing the above changeset. Note: I have run mach5 tests successfully. Regards, Srinivas ----- Original Message ----- From: paul.sandoz at oracle.com To: srinivas.dama at oracle.com Cc: core-libs-dev at openjdk.java.net, james.laskey at oracle.com Sent: Tuesday, 12 June, 2018 10:08:53 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: RFR: 8196988 (Resolve disabled warnings for libjimage) Looks ok. Including Jim in case he is not listening in? I am guessing the comment selectively disables the warning in this code (rather than using a diagnostic pragma). Thanks, Paul. > On Jun 7, 2018, at 11:17 PM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > > Note: Modified earlier fix to handle warning messages specific to > libjimage only. > > Regards, > Srinivas > > ----- Original Message ----- > From: srinivas.dama at oracle.com > To: core-libs-dev at openjdk.java.net > Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but > these warnings are disabled earlier.I have enabled these warnings and fixed in sources. > > Regards, > Srinivas From david.holmes at oracle.com Mon Jun 18 03:45:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Jun 2018 13:45:19 +1000 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: Hi Brent, On 16/06/2018 1:44 AM, Brent Christian wrote: > Hi, > > In JDK 9, 8029891[1] refactored java.util.Properties to store its values > in an internal ConcurrentHashMap, and removed synchronization from > "reader" methods in order to avoid potential hangs/deadlocks during > classloading. > > Claes has noticed that there is the possibility of the new 'map' field > being observed with its default value (null), before being set. > > After looking at the JSR 133 FAQ[2], I agree with Claes that we should > make 'map' a field final. Making it "final" to fix unsafe publication only works with actual publication after construction. You're making it "final" but then not setting it at construction time and instead using UNSAFE.putVolatile to get around the fact that it is final! The "final" serves no purpose in that regard. Further a putVolatile without corresponding getVolatile does not achieve safe publication under the JMM. I think making the actual field(s) volatile is the only JMM compliant way to correctly address this. David ----- > Please review my change to do this: > > Webrev: > http://cr.openjdk.java.net/~bchristi/8199435/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199435 > > Thanks, > -Brent > > 1. https://bugs.openjdk.java.net/browse/JDK-8029891 > 2. > https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#finalRight From claes.redestad at oracle.com Mon Jun 18 10:02:39 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 12:02:39 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: On 2018-06-18 05:45, David Holmes wrote: > Making it "final" to fix unsafe publication only works with actual > publication after construction. You're making it "final" but then not > setting it at construction time and instead using UNSAFE.putVolatile > to get around the fact that it is final! The "final" serves no purpose > in that regard. Further a putVolatile without corresponding > getVolatile does not achieve safe publication under the JMM. Adding a volatile read on every read through Properties is likely to have some performance impact, but it could still be better than before 8029891 where such operations were synchronized. Regarding Peter's comments about defaults, I think accesses to that field was implicitly guarded by the synchronized super.get calls before 8029891 and now needs to be re-examined too. > > I think making the actual field(s) volatile is the only JMM compliant > way to correctly address this. The only issue is serialization-related readHashTable method (clone could be implemented without Unsafe as clone.map.putAll(map)). Is there no sanctioned way (using e.g. VarHandles) to safely construct and publish a transient "final" field in the scope of a readObject? I've been shying away from experimenting with VarHandles in places like Properties as I'm certain it could lead to bootstrap issues, but for something like a deserialization hook it might very well be fine. /Claes From peter.levart at gmail.com Mon Jun 18 11:06:07 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Jun 2018 13:06:07 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> Hi Claes, On 06/18/2018 12:02 PM, Claes Redestad wrote: > > > On 2018-06-18 05:45, David Holmes wrote: >> Making it "final" to fix unsafe publication only works with actual >> publication after construction. You're making it "final" but then not >> setting it at construction time and instead using UNSAFE.putVolatile >> to get around the fact that it is final! The "final" serves no >> purpose in that regard. Further a putVolatile without corresponding >> getVolatile does not achieve safe publication under the JMM. > > Adding a volatile read on every read through Properties is likely to > have some performance impact, On non-Intel architectures in particular. On intel it would just inhibit some JIT optimizations like hoisting the read of field out of loop... > but it could still be better than before 8029891 where such operations > were synchronized. Regarding Peter's comments about defaults, I think > accesses to that field was implicitly guarded by the synchronized > super.get calls before 8029891 and now needs to be re-examined too. This was pre- 8029891 code of getProperty for example: ??? public String getProperty(String key) { ??????? Object oval = super.get(key); ??????? String sval = (oval instanceof String) ? (String)oval : null; ??????? return ((sval == null) && (defaults != null)) ? defaults.getProperty(key) : sval; ??? } So 'defaults' field was accessed out of synchronized super.get() method and getProperty was not synchronized. This is pre-existing issue. But 'map' field is new. The problem with 'defaults' field is also that is is protected and subclasses can access it without synchronization. If it is made volatile, getProperty() method will be penalized. So maybe using weaker memory fences in combination with plain 'property' field can make the field behave like it was final, so at least accesses to field within Properties class can be made non-racy. If this is done for 'properties' then same fences will work for 'map' too and it doesn't have to be final. > >> >> I think making the actual field(s) volatile is the only JMM compliant >> way to correctly address this. > > The only issue is serialization-related readHashTable method (clone > could be implemented without Unsafe as clone.map.putAll(map)). 1st you would have to do: ??? clone.map = new CHM(); and only then: ??? clone.map.putAll(map); or else you would be adding the elements of the original map to the original map (which would actually work with CHM), but then the cloned Properties object would share the map with original one. So you do need to assign to 'map' field in clone(). > Is there no sanctioned way (using e.g. VarHandles) to safely construct > and publish a transient "final" field in the scope of a readObject? > I've been shying away from experimenting with VarHandles in places > like Properties as I'm certain it could lead to bootstrap issues, but > for something like a deserialization hook it might very well be fine. Paul would know for sure, but I think this was deliberately prevented in VarHandle(s). Regards, Peter > > /Claes > From claes.redestad at oracle.com Mon Jun 18 13:27:06 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 15:27:06 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> Message-ID: <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> On 2018-06-18 13:06, Peter Levart wrote: >> Adding a volatile read on every read through Properties is likely to >> have some performance impact, > > On non-Intel architectures in particular. On intel it would just > inhibit some JIT optimizations like hoisting the read of field out of > loop... Right, and coincidentally those platforms are where I'd expect the current implementation to cause bugs (I've not been able to provoke any real error on my Intel-based workstations). I'd be surprised if we'd be much slower than pre-8029891 using volatiles (volatile read vs synchronized - even with biased locking), and we'd still retain the scalability benefits of 8029891. Ignore my remarks about clone - I'm just back from vacation and have apparently forgotten how cloning works. :-) /Claes From claes.redestad at oracle.com Mon Jun 18 13:54:26 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 15:54:26 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> Message-ID: I'd suggest something simple like this to ensure correctness, while keeping the number of volatile reads to a minimum: http://cr.openjdk.java.net/~redestad/8199435.00/ Then file a follow-up RFE to investigate if we can make these fields non-volatile, e.g. using careful fencing as suggested by Peter. /Claes On 2018-06-18 15:27, Claes Redestad wrote: > > > On 2018-06-18 13:06, Peter Levart wrote: >>> Adding a volatile read on every read through Properties is likely to >>> have some performance impact, >> >> On non-Intel architectures in particular. On intel it would just >> inhibit some JIT optimizations like hoisting the read of field out of >> loop... > > Right, and coincidentally those platforms are where I'd expect the > current implementation to cause bugs (I've not been able to provoke > any real error on my Intel-based workstations). > > I'd be surprised if we'd be much slower than pre-8029891 using > volatiles (volatile read vs synchronized - even with biased locking), > and we'd still retain the scalability benefits of 8029891. > > Ignore my remarks about clone - I'm just back from vacation and have > apparently forgotten how cloning works. :-) > > /Claes From martinrb at google.com Mon Jun 18 14:05:24 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 07:05:24 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> Message-ID: There's a long history of cheating and setting final fields in pseudo-constructors readObject and clone, which is well motivated since Java should really support pseudo-constructors in a better way.. Cheating has used Unsafe or reflection with setAccessible (CopyOnWriteArrayList.resetLock()). I haven't had any success actually reproducing any data race failures in constructors - on modern machines it would only happen if the JIT reordered the writes. Doug - would CopyOnWriteArrayList.clone() be slightly safer if it had a releaseFence() ? From martinrb at google.com Mon Jun 18 14:13:01 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 07:13:01 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: I'm ignoring the direct buffers problem (sorry Tony) and wearing my ReentrantReadWriteLock hat. I still like the idea of adding ThreadLocal.compute(Function remappingFunction) and perhaps other such map-inspired methods. RRWL wouldn't benefit much, since it already tries to minimize use of ThreadLocal, but it would at least allow get() and remove() to be coalesced on read-unlock. RRWL never changes from one non-null value to another, it only switches between absent and present. I'd like to avoid the two lookups due to get and remove. Here's a prototype that does not yet help RRWL, but helps callers switching between non-null values, and could probably be extended via surgery to ThreadLocalMap: public T compute( java.util.function.Function remappingFunction) { final Thread currentThread = Thread.currentThread(); final ThreadLocalMap map = getMap(currentThread); final ThreadLocalMap.Entry e = (map == null) ? null : map.getEntry(this); final T oldValue = (e == null) ? null : (T)e.value; final T newValue = remappingFunction.apply(oldValue); if (newValue == null) { if (oldValue != null) { map.remove(this); } } else if (e != null) { e.value = newValue; } else if (map != null) { map.set(this, newValue); } else { createMap(currentThread, newValue); } return newValue; } On Sun, Jun 17, 2018 at 2:20 PM, Peter Levart wrote: > Update: the discussion on concurrent-interest about possible future > addition of public ThreadLocal.getIfPresent() or ThreadLocal.isPresent() > has died out, but it nevertheless reached a point where > ThreadLocal.isPresent() was found the least problematic. So I would like to > get further with this proposal using the last webrev.04 version of the > patch that uses ThreadLocal.isPresent() internally - still package-private > at the moment. > > Here's the webrev (unchanged from the day it was published): > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ > > Would this version be OK? > > Regards, Peter > > > > On 06/06/18 20:55, Peter Levart wrote: > > From dl at cs.oswego.edu Mon Jun 18 14:21:56 2018 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 18 Jun 2018 10:21:56 -0400 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> Message-ID: <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> On 06/18/2018 10:05 AM, Martin Buchholz wrote: > There's a long history of cheating and setting final fields in > pseudo-constructors readObject and clone, which is well motivated since > Java should really support pseudo-constructors in a better way.. > Cheating has used Unsafe or reflection with setAccessible > (CopyOnWriteArrayList.resetLock()). > > Doug - would CopyOnWriteArrayList.clone() be slightly safer if it had a > releaseFence() ? > Or better, lockField.set in resetLock could instead be setRelease. Except it is using reflection, not VarHandles. So it could be preceded with VarHandle.releaseFence(), although it is hard to think of a scenario in which it could matter. -Doug From peter.levart at gmail.com Mon Jun 18 14:23:01 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Jun 2018 16:23:01 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> Message-ID: <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> Hi Claes, On 06/18/2018 03:54 PM, Claes Redestad wrote: > I'd suggest something simple like this to ensure correctness, while > keeping the number of volatile reads to a minimum: > > http://cr.openjdk.java.net/~redestad/8199435.00/ > > Then file a follow-up RFE to investigate if we can make these fields > non-volatile, e.g. using careful fencing as suggested > by Peter. OK, but the constructor will still need a releaseFence at the end to prevent a potential write that unsafely publishes Properties instance to float before the volatile writes of 'map' and 'defaults'... You might want to use Unsafe directly for that since VarHandle could cause bootstrap issues. Regards, Peter > > /Claes > > On 2018-06-18 15:27, Claes Redestad wrote: >> >> >> On 2018-06-18 13:06, Peter Levart wrote: >>>> Adding a volatile read on every read through Properties is likely >>>> to have some performance impact, >>> >>> On non-Intel architectures in particular. On intel it would just >>> inhibit some JIT optimizations like hoisting the read of field out >>> of loop... >> >> Right, and coincidentally those platforms are where I'd expect the >> current implementation to cause bugs (I've not been able to provoke >> any real error on my Intel-based workstations). >> >> I'd be surprised if we'd be much slower than pre-8029891 using >> volatiles (volatile read vs synchronized - even with biased locking), >> and we'd still retain the scalability benefits of 8029891. >> >> Ignore my remarks about clone - I'm just back from vacation and have >> apparently forgotten how cloning works. :-) >> >> /Claes > From Roger.Riggs at Oracle.com Mon Jun 18 14:25:37 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 18 Jun 2018 10:25:37 -0400 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: Hi Stuart, In regard to new SharedSecret interfaces, one option is move shared (but private) implementation classes to a jdk.internal.xx package (not exported).? This only works well if they are not tightly coupled to other package private classes. SpinedBuffer might be a good candidate, I have some IO cases in mind that could benefit from the allocation/reallocation savings.? (ByteArrayOutputStream for 1). Regards, Roger On 6/15/2018 5:22 PM, Stuart Marks wrote: > Hi Peter, > > Finally getting back to this. > > I think there's clearly room for improvement in the creation of the > unmodifiable collections. Right now the creation paths (mostly) use > only public APIs, which can result in unnecessary allocation and > copying. Certainly Map.copyOf does this, but there are also other > cases as well. > > For copying from an unknown map, there are a couple approaches: > > * accumulate keys and values in a single ArrayList, which can be > presized as necessary, but which will grow if necessary; then copy > elements from a subrange of ArrayList's internal array (similar to > your MapN(Object[], len) overload) > > * accumulate keys and values into a SpinedBuffer, which doesn't > require copying to grow, which is preferable if for some reason we > can't pre-size accurately; and then copy the elements out of it > > The Collectors.toUnmodifiableMap implementations are known to create > HashMap instances, so they could pass the HashMap directly to a > private MapN constructor that in turn could talk directly to HashMap > to get the keys and values. This avoids allocation of a full-sized > buffer and one copy. > > Note that these techniques involve creating new interfaces, sometimes > that cross package boundaries. It's a bit of an irritant to have to > plumb new paths that go through SharedSecrets, but it seems likely to > be worthwhile if we can avoid bulk allocation and copying steps. > > Given these, it doesn't seem to me that the BiBuffer approach helps > very much. I think there are many other avenues that would be > worthwhile to explore, and that possibly can provide bigger savings. > > s'marks > > > > > > On 6/11/18 3:57 AM, Peter Levart wrote: >> Hi, >> >> Those two methods were added in JDK 10 and they are not very optimal. >> Map.copyOf(Map map) 1st dumps the source map into an array of >> Map.Entry(s) (map.entrySet().toArray(new Entry[0])), which typically >> creates new Map.Entry objects, then passes the array to >> Map.ofEntries(Map.Entry[] entries) factory method that iterates the >> array and constructs a key/value interleaved array from it which is >> passed to ImmutableCollections.MapN constructor, which then >> constructs a linear-probing hash table from it. So each key and value >> is copied 3 times, while several temporary objects are created in the >> process. >> >> One copying step could be eliminated and construction of temporary >> Map.Entry objects too: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.01/ >> >> >> Collecting stream(s) using Collectors.toUnmodifiableMap() 1st >> collects key/value pairs into a HashMap, then it performs equivalent >> operations as Map.copyOf(hashMap) at the end. Using Map.copyOf() >> directly benefits this collection operation too. >> >> The following benchmark: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench.java >> >> >> >> Shows up to 30% improvement of .copyOf operation with this patch >> applied: >> >> Original: >> >> Benchmark?????????????????????????????? (size)? Mode Cnt Score Error? >> Units >> UnmodifiableMapBench.copyOf???????????????? 10? avgt 10 403.633 ? >> 2.640? ns/op >> UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 3489.623 ? >> 44.590? ns/op >> UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 40030.572 ? >> 277.075? ns/op >> UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10 831.221 ? >> 3.816? ns/op >> UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9783.519 ? >> 43.097? ns/op >> UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 96524.536 ? >> 670.818? ns/op >> >> Patched: >> >> Benchmark?????????????????????????????? (size)? Mode Cnt Score Error? >> Units >> UnmodifiableMapBench.copyOf???????????????? 10? avgt 10 264.172 ? >> 1.882? ns/op >> UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 2318.974 ? >> 15.877? ns/op >> UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 29291.782 ? >> 3139.737? ns/op >> UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10 771.221 ? >> 65.432? ns/op >> UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9275.016 ? >> 725.722? ns/op >> UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 82204.342 ? >> 851.741? ns/op >> >> >> Production of garbage is also reduced, since no Map.Entry temporary >> objects are constructed: >> >> Original: >> >> Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10 416.001 >> ????? 0.002??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 >> 2936.005 ????? 0.019??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 >> 28136.059 ????? 0.199??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10 avgt >> 10?? 1368.001 ????? 0.004??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100 avgt >> 10? 10208.139 ????? 0.045??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000 avgt >> 10? 93025.923 ????? 0.573??? B/op >> >> Patched: >> >> Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10 304.000 >> ????? 0.001??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 >> 2464.004 ????? 0.012??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 >> 24064.040 ????? 0.137??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10 avgt >> 10?? 1256.001 ????? 0.003??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100 avgt >> 10?? 9720.153 ????? 0.055??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000 avgt >> 10? 88905.688 ????? 0.574??? B/op >> >> >> So what do you think? Is this an improvement? >> >> Regards, Peter >> From Roger.Riggs at Oracle.com Mon Jun 18 14:31:45 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 18 Jun 2018 10:31:45 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> Message-ID: <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> Hi Alan, Thanks for the review and suggestion. The CSR and Webrev are updated. webrev: ? http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ csr: ? https://bugs.openjdk.java.net/browse/JDK-8204235 On 6/16/2018 8:42 AM, Alan Bateman wrote: > On 13/06/2018 15:10, Roger Riggs wrote: >> Hi Joe, >> >> The CSR text is still in draft and got out of sync with the open >> review and comments. >> >> The updated apiNote clarifies that changes made through the >> System.*property*? methods >> may not affect the value cached during initialization. >> The implementation changes affect a very short list of properties. >> >> The note on the methods is to raise awareness that individual >> properties may have >> different behavior and may be unpredictable. > Just catching up on this. I checked the CSR and the webrev (the latest > webrev was generated on June 6 so I hope that is the version we are > meant to look at). > > The updated apiNote (CSR version) looks okay but I think the word > "internal" needs to be dropped as it comes with too many questions. > "may be cached during initialization or ..." should be okay. The > editing has meant the line lengths are a bit inconsistent so I assume > they can be fixed up before the change is pushed. > > I see the original patch to SocksSocketImpl has been mostly reverted. > It looks correct now. The other usages look okay to me. > > -Alan From claes.redestad at oracle.com Mon Jun 18 14:47:57 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 16:47:57 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> Message-ID: On 2018-06-18 16:23, Peter Levart wrote: > Hi Claes, > > On 06/18/2018 03:54 PM, Claes Redestad wrote: >> I'd suggest something simple like this to ensure correctness, while >> keeping the number of volatile reads to a minimum: >> >> http://cr.openjdk.java.net/~redestad/8199435.00/ >> >> Then file a follow-up RFE to investigate if we can make these fields >> non-volatile, e.g. using careful fencing as suggested >> by Peter. > > OK, but the constructor will still need a releaseFence at the end to > prevent a potential write that unsafely publishes Properties instance > to float before the volatile writes of 'map' and 'defaults'... > > You might want to use Unsafe directly for that since VarHandle could > cause bootstrap issues. I don't follow.. which constructor needs a fence in the suggested patch? /Claes From Roger.Riggs at Oracle.com Mon Jun 18 14:55:58 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 18 Jun 2018 10:55:58 -0400 Subject: [11] RFR: 8042131: DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate In-Reply-To: <9018eeda-d891-adf8-81b7-ef1d1db08281@oracle.com> References: <9018eeda-d891-adf8-81b7-ef1d1db08281@oracle.com> Message-ID: <8223a2e8-08a6-9118-000f-b9930ac30d09@Oracle.com> Hi Naoto, Looks fine Regards, Roger On 6/15/2018 6:14 PM, Naoto Sato wrote: > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8042131 > > The proposed change is located at: > > http://cr.openjdk.java.net/~naoto/8042131/webrev.00/ > > The fix is originally contributedy by Toshio Nakamura at IBM [1]. I am > sponsoring the fix, with some trivial modifications to the test (diff > is attached). > > Naoto > > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053830.html From scolebourne at joda.org Mon Jun 18 15:04:11 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 18 Jun 2018 16:04:11 +0100 Subject: [11] RFR: 8042131: DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate In-Reply-To: <8223a2e8-08a6-9118-000f-b9930ac30d09@Oracle.com> References: <9018eeda-d891-adf8-81b7-ef1d1db08281@oracle.com> <8223a2e8-08a6-9118-000f-b9930ac30d09@Oracle.com> Message-ID: +1 Stephen On 18 June 2018 at 15:55, Roger Riggs wrote: > Hi Naoto, > > Looks fine > > Regards, Roger > > > > On 6/15/2018 6:14 PM, Naoto Sato wrote: >> >> Hello, >> >> Please review the fix to the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8042131 >> >> The proposed change is located at: >> >> http://cr.openjdk.java.net/~naoto/8042131/webrev.00/ >> >> The fix is originally contributedy by Toshio Nakamura at IBM [1]. I am >> sponsoring the fix, with some trivial modifications to the test (diff is >> attached). >> >> Naoto >> >> [1] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053830.html > > From martinrb at google.com Mon Jun 18 15:22:21 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 08:22:21 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> Message-ID: On Mon, Jun 18, 2018 at 7:21 AM, Doug Lea
wrote: > On 06/18/2018 10:05 AM, Martin Buchholz wrote: > > There's a long history of cheating and setting final fields in > > pseudo-constructors readObject and clone, which is well motivated since > > Java should really support pseudo-constructors in a better way.. > > Cheating has used Unsafe or reflection with setAccessible > > (CopyOnWriteArrayList.resetLock()). > > > > > Doug - would CopyOnWriteArrayList.clone() be slightly safer if it had a > > releaseFence() ? > > > > Or better, lockField.set in resetLock could instead be setRelease. > Except it is using reflection, not VarHandles. So it could be preceded > with VarHandle.releaseFence(), although it is hard to think of a > scenario in which it could matter. > You mean followed by, not preceded by? try { lockField.set(this, new Object()); + java.lang.invoke.VarHandle.releaseFence(); } catch (IllegalAccessException e) { throw new Error(e); } From Alan.Bateman at oracle.com Mon Jun 18 15:41:09 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Jun 2018 16:41:09 +0100 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: On 17/06/2018 22:20, Peter Levart wrote: > Update: the discussion on concurrent-interest about possible future > addition of public ThreadLocal.getIfPresent() or > ThreadLocal.isPresent() has died out, but it nevertheless reached a > point where ThreadLocal.isPresent() was found the least problematic. > So I would like to get further with this proposal using the last > webrev.04 version of the patch that uses ThreadLocal.isPresent() > internally - still package-private at the moment. > > Here's the webrev (unchanged from the day it was published): > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ > > Would this version be OK? I think looks quite good. One small nit is that the update to ThreadLocal.setInitialValue makes it look like all locals are registered when setting the initial value. What would you think about moving the instanceof check from TerminatingThreadLocal.register so that it's a bit more obvious. -Alan From peter.levart at gmail.com Mon Jun 18 16:09:45 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Jun 2018 18:09:45 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> Message-ID: <8474cc34-36d7-fc76-6297-60f48557c432@gmail.com> On 06/18/2018 04:47 PM, Claes Redestad wrote: > > > On 2018-06-18 16:23, Peter Levart wrote: >> Hi Claes, >> >> On 06/18/2018 03:54 PM, Claes Redestad wrote: >>> I'd suggest something simple like this to ensure correctness, while >>> keeping the number of volatile reads to a minimum: >>> >>> http://cr.openjdk.java.net/~redestad/8199435.00/ >>> >>> Then file a follow-up RFE to investigate if we can make these fields >>> non-volatile, e.g. using careful fencing as suggested >>> by Peter. >> >> OK, but the constructor will still need a releaseFence at the end to >> prevent a potential write that unsafely publishes Properties instance >> to float before the volatile writes of 'map' and 'defaults'... >> >> You might want to use Unsafe directly for that since VarHandle could >> cause bootstrap issues. > > I don't follow.. which constructor needs a fence in the suggested patch? > > /Claes The Properties constructor: ??? private Properties(Properties defaults, int initialCapacity) { ??????? // use package-private constructor to ??????? // initialize unused fields with dummy values ??????? super((Void) null); ??????? map = new ConcurrentHashMap<>(initialCapacity); ??????? this.defaults = defaults; ??? ??? UNSAFE.storeFence(); // <-- HERE!! ??? } Final fields have bigger guarantee than volatile fields in this respect. Plain write that follows in program a volatile write, can in theory float before the volatile write. So if you publish a Properties instance via data race (via plain write), the observer may still see uninitialized 'map' and 'defaults' fields. Regards, Peter From claes.redestad at oracle.com Mon Jun 18 16:53:20 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 18:53:20 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <8474cc34-36d7-fc76-6297-60f48557c432@gmail.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> <8474cc34-36d7-fc76-6297-60f48557c432@gmail.com> Message-ID: <17343fde-3017-0bd8-e426-be9f4c17187e@oracle.com> On 2018-06-18 18:09, Peter Levart wrote: > > > On 06/18/2018 04:47 PM, Claes Redestad wrote: >> >> >> On 2018-06-18 16:23, Peter Levart wrote: >>> Hi Claes, >>> >>> On 06/18/2018 03:54 PM, Claes Redestad wrote: >>>> I'd suggest something simple like this to ensure correctness, while >>>> keeping the number of volatile reads to a minimum: >>>> >>>> http://cr.openjdk.java.net/~redestad/8199435.00/ >>>> >>>> Then file a follow-up RFE to investigate if we can make these >>>> fields non-volatile, e.g. using careful fencing as suggested >>>> by Peter. >>> >>> OK, but the constructor will still need a releaseFence at the end to >>> prevent a potential write that unsafely publishes Properties >>> instance to float before the volatile writes of 'map' and 'defaults'... >>> >>> You might want to use Unsafe directly for that since VarHandle could >>> cause bootstrap issues. >> >> I don't follow.. which constructor needs a fence in the suggested patch? >> >> /Claes > > The Properties constructor: > > > ??? private Properties(Properties defaults, int initialCapacity) { > ??????? // use package-private constructor to > ??????? // initialize unused fields with dummy values > ??????? super((Void) null); > ??????? map = new ConcurrentHashMap<>(initialCapacity); > ??????? this.defaults = defaults; > > ??? ??? UNSAFE.storeFence(); // <-- HERE!! > ??? } > > > Final fields have bigger guarantee than volatile fields in this respect. > > Plain write that follows in program a volatile write, can in theory > float before the volatile write. So if you publish a Properties > instance via data race (via plain write), the observer may still see > uninitialized 'map' and 'defaults' fields. > Right http://cr.openjdk.java.net/~redestad/8199435.01/ (Yes, using VarHandle.storeStoreFence would do the exact same thing, but is not usable from Properties as the VarHandle impl needs to read some system properties...) /Claes From patrick at reini.net Mon Jun 18 17:18:09 2018 From: patrick at reini.net (Patrick Reinhart) Date: Mon, 18 Jun 2018 19:18:09 +0200 Subject: RFR: JDK-8205090 Update of the Reader:nullReader() spec for the mark() method Message-ID: Hi Everybody, In the process of solving the issue [1] a part of the proper solution should be a partially change of the Reader.nullReader() specification. I opened a new CSR to address this. The main change would be to change to following lines: |*

The {@code markSupported()} method returns {@code false}. The * {@code mark()} method does nothing, and the {@code reset()} method * throws {@code IOException}.| to this: |*

The {@code markSupported()} method returns {@code false}. The * {@code mark()} and {@code reset()} methods throw an {@code IOException}.| Side note: I got a additional comment from Krushnareddy Ganapureddy stating: > additional Note - spec says "ready()), skip(long), and transferTo() methods all behave as if end of stream has been reached" > But there is not clear say on these methods [ ready(), skip(), transferTo() ] what is the expected behavior "if end of stream reached" We may able to address the issue above too. Any ideas how to address this? -Patrick [1] https://bugs.openjdk.java.net/browse/JDK-8204930 From tprintezis at twitter.com Mon Jun 18 17:21:24 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Mon, 18 Jun 2018 10:21:24 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: Martin, You are forgiven. :-) So, the (valid, I think) issue with getIfPresent(), as originally proposed by Peter, was that if get() was overridden in the subclass to somehow transform the returned value, getIfPresent() wouldn?t apply the same transformation. Doesn?t your compute() method have essentially the same issue? Apart from that, I personally like this proposal as I agree: one look-up is always better than two. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 18, 2018 at 10:13:02 AM, Martin Buchholz (martinrb at google.com) wrote: I'm ignoring the direct buffers problem (sorry Tony) and wearing my ReentrantReadWriteLock hat. I still like the idea of adding ThreadLocal.compute(Function remappingFunction) and perhaps other such map-inspired methods. RRWL wouldn't benefit much, since it already tries to minimize use of ThreadLocal, but it would at least allow get() and remove() to be coalesced on read-unlock. RRWL never changes from one non-null value to another, it only switches between absent and present. I'd like to avoid the two lookups due to get and remove. Here's a prototype that does not yet help RRWL, but helps callers switching between non-null values, and could probably be extended via surgery to ThreadLocalMap: public T compute( java.util.function.Function remappingFunction) { final Thread currentThread = Thread.currentThread(); final ThreadLocalMap map = getMap(currentThread); final ThreadLocalMap.Entry e = (map == null) ? null : map.getEntry(this); final T oldValue = (e == null) ? null : (T)e.value; final T newValue = remappingFunction.apply(oldValue); if (newValue == null) { if (oldValue != null) { map.remove(this); } } else if (e != null) { e.value = newValue; } else if (map != null) { map.set(this, newValue); } else { createMap(currentThread, newValue); } return newValue; } On Sun, Jun 17, 2018 at 2:20 PM, Peter Levart wrote: > Update: the discussion on concurrent-interest about possible future > addition of public ThreadLocal.getIfPresent() or ThreadLocal.isPresent() > has died out, but it nevertheless reached a point where > ThreadLocal.isPresent() was found the least problematic. So I would like to > get further with this proposal using the last webrev.04 version of the > patch that uses ThreadLocal.isPresent() internally - still package-private > at the moment. > > Here's the webrev (unchanged from the day it was published): > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ > > Would this version be OK? > > Regards, Peter > > > > On 06/06/18 20:55, Peter Levart wrote: > > From martinrb at google.com Mon Jun 18 17:53:26 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 10:53:26 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: On Mon, Jun 18, 2018 at 10:21 AM, Tony Printezis wrote: > Martin, > > You are forgiven. :-) > > So, the (valid, I think) issue with getIfPresent(), as originally proposed > by Peter, was that if get() was overridden in the subclass to somehow > transform the returned value, getIfPresent() wouldn?t apply the same > transformation. > > Doesn?t your compute() method have essentially the same issue? Apart from > that, I personally like this proposal as I agree: one look-up is always > better than two. > > A non-prototype implementation would delegate compute into ThreadLocalMap itself, where there is no risk of overriding. > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On June 18, 2018 at 10:13:02 AM, Martin Buchholz (martinrb at google.com) > wrote: > > I'm ignoring the direct buffers problem (sorry Tony) and wearing my > ReentrantReadWriteLock hat. I still like the idea of adding > ThreadLocal.compute(Function remappingFunction) and perhaps other such > map-inspired methods. RRWL wouldn't benefit much, since it already tries to > minimize use of ThreadLocal, but it would at least allow get() and remove() > to be coalesced on read-unlock. > > RRWL never changes from one non-null value to another, it only switches > between absent and present. I'd like to avoid the two lookups due to get > and remove. Here's a prototype that does not yet help RRWL, but helps > callers switching between non-null values, and could probably be extended > via surgery to ThreadLocalMap: > > public T compute( > java.util.function.Function > remappingFunction) { > final Thread currentThread = Thread.currentThread(); > final ThreadLocalMap map = getMap(currentThread); > final ThreadLocalMap.Entry e = (map == null) > ? null > : map.getEntry(this); > final T oldValue = (e == null) ? null : (T)e.value; > final T newValue = remappingFunction.apply(oldValue); > if (newValue == null) { > if (oldValue != null) { > map.remove(this); > } > } else if (e != null) { > e.value = newValue; > } else if (map != null) { > map.set(this, newValue); > } else { > createMap(currentThread, newValue); > } > return newValue; > } > > > > > From ivan.gerasimov at oracle.com Mon Jun 18 18:57:24 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 18 Jun 2018 11:57:24 -0700 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> Message-ID: A gentle ping :) Do you think it's good to go now? With kind regards, Ivan On 6/10/18 11:15 PM, Ivan Gerasimov wrote: > Hi Alan! > > > On 6/6/18 6:57 AM, Alan Bateman wrote: >> I think this should be okay but the Windows implementation has a long >> history of biting the fingers of anyone that dares touch it. >> Sometimes unexpected behavior changes only come to light long after >> the change. The recent mails here about Kafka and sparse files is a >> good example of that. So I think it's important to check the test >> coverage before pushing this change. Specifically I think we need to >> check that we have tests that >> >> 1. Exercise shrinking, extending, and not change the file length >> >> 2. Check getFilePointer when used with setLength. >> >> 3. Check how FileChannel behaves when created from a RandomAccessFile >> but the file length is changed after the FileChannel is obtained. >> > I extended the existing reg. test > java/io/RandomAccessFile/SetLength.java to cover more cases that > involve changing the file size. > > Also I added another regression test, as you Alan suggested, to check > that RandomAccessFile and its FileChannel behave consistently in > various scenarios. > > All the tests, including the new ones, pass on all supported platforms. > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 > WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/01/webrev/ > > Would you please help review the fix? > -- With kind regards, Ivan Gerasimov From dl at cs.oswego.edu Mon Jun 18 20:01:53 2018 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 18 Jun 2018 16:01:53 -0400 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> Message-ID: On 06/18/2018 11:22 AM, Martin Buchholz wrote: > Or better, lockField.set in resetLock could instead be setRelease. > Except it is using reflection, not VarHandles. So it could be preceded > with VarHandle.releaseFence(), although it is hard to think of a > scenario in which it could matter. > > > You mean followed by, not preceded by? > > ? ? ? ? ? try { > ? ? ? ? ? ? ?lockField.set(this, new Object()); > +? ? ? ? ? ? java.lang.invoke.VarHandle.releaseFence(); > ? ? ? ? ?} catch (IllegalAccessException e) { > ? ? ? ? ? ? ?throw new Error(e); > ? ? ? ? ?} > OK. Followed by is a little better in that it orders any field write before any write of this, not just the array copy before lock field write. -Doug From david.lloyd at redhat.com Mon Jun 18 20:45:01 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Mon, 18 Jun 2018 15:45:01 -0500 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: On Mon, Jun 18, 2018 at 12:53 PM, Martin Buchholz wrote: > On Mon, Jun 18, 2018 at 10:21 AM, Tony Printezis > wrote: > >> Martin, >> >> You are forgiven. :-) >> >> So, the (valid, I think) issue with getIfPresent(), as originally proposed >> by Peter, was that if get() was overridden in the subclass to somehow >> transform the returned value, getIfPresent() wouldn?t apply the same >> transformation. >> >> Doesn?t your compute() method have essentially the same issue? Apart from >> that, I personally like this proposal as I agree: one look-up is always >> better than two. >> >> > A non-prototype implementation would delegate compute into ThreadLocalMap > itself, where there is no risk of overriding. I don't think overriding is the risk; the risk would be compute* behaving inconsistently with regards to an overridden get*. -- - DML From brian.goetz at oracle.com Mon Jun 18 21:29:36 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 18 Jun 2018 17:29:36 -0400 Subject: BiCollector In-Reply-To: <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> Message-ID: <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> "bisecting" sounds like it sends half the elements to one collector and half to the other ... "tee" might be a candidate, though it doesn't follow the `ing convention.? "teeing" sounds dumb. On 6/15/2018 7:36 PM, Paul Sandoz wrote: > Hi Tagir, > > This is looking good. > > My current favorite name for the factory method is ?bisecting? since we are dividing the collector into two collectors, the results of which are then merged. > > Suggested first paragraph of the factory method: > > "Returns a collector that passes the input elements to two specified collectors and merges their results with the specified merge function.? > > Paul. > > >> On Jun 15, 2018, at 4:26 AM, Tagir Valeev wrote: >> >> Hello! >> >> I created a preliminary webrev of my own implementation (no testcases yet): >> http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ >> If anybody wants to sponsor my implementation, I will happily log an issue and write tests. >> >> The name "pairing" was invented by me, but as I'm not a native English speaker I cannot judge whether it's optimal, so better ideas are welcome. >> Also I decided to remove accumulator types from public type variables. They do not add anything to type signature, only clutter it >> increasing the number of type parameters from 4 to 6. I think it was a mistake to expose the accumulator type parameter in other cascading collectors >> like filtering(), collectingAndThen(), groupingBy(), etc. I'm not insisting though, if you feel that conformance to existing collectors is >> more important than simplicity. >> >> With best regards, >> Tagir Valeev. >> >> On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz wrote: >> >>> Well, I don't see the need to pack the two results into a Map.Entry >>> (or any similar) container as a drawback. >> From an "integrity of the JDK APIs" perspective, it is unquestionably a >> drawback. These items are not a Key and an associated Value in a Map; >> it's merely pretending that Map.Entry really means "Pair". There's a >> reason we don't have a Pair class in the JDK (and no, let's not reopen >> that now); using something else as a Pair proxy that is supposed to have >> specific semantics is worse. (It's fine to do this in your own code, but >> not in the JDK. Different standards for code that has different audiences.) >> >> Tagir's proposed sidestepping is nice, and it will also play nicely with >> records, because then you can say: >> >> record NameAndCount(String name, int count); >> >> stream.collect(pairing(collectName, collectCount, NameAndCount::new)); >> >> and get a more properly abstract result out. And its more in the spirit >> of existing Collectors. If you want to use Map.Entry as an >> _implementation detail_, that's fine. >> >> I can support this form. >> >>> I also don't see a larger abstraction like BiStream as a natural fit >>> for a similar thing. >> I think the BiStream connection is mostly tangential. We tried hard to >> support streams of (K,V) pairs when we did streams, as Paul can attest, >> but it was a huge complexity-inflater and dropping this out paid an >> enormous simplification payoff. >> >> With records, having streams of tuples will be simpler to represent, but >> no more performant; it will take until we get to value types and >> specialized generics to get the performance we want out of this. >> >> From jhg023 at bucknell.edu Mon Jun 18 21:42:27 2018 From: jhg023 at bucknell.edu (Jacob Glickman) Date: Mon, 18 Jun 2018 17:42:27 -0400 Subject: BiCollector In-Reply-To: <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: Agreed. Not sure if this has been suggested, but what about duplex(ing)? On Mon, Jun 18, 2018 at 5:29 PM Brian Goetz wrote: > "bisecting" sounds like it sends half the elements to one collector and > half to the other ... > > "tee" might be a candidate, though it doesn't follow the `ing > convention. "teeing" sounds dumb. > > > > On 6/15/2018 7:36 PM, Paul Sandoz wrote: > > Hi Tagir, > > > > This is looking good. > > > > My current favorite name for the factory method is ?bisecting? since we > are dividing the collector into two collectors, the results of which are > then merged. > > > > Suggested first paragraph of the factory method: > > > > "Returns a collector that passes the input elements to two specified > collectors and merges their results with the specified merge function.? > > > > Paul. > > > > > >> On Jun 15, 2018, at 4:26 AM, Tagir Valeev wrote: > >> > >> Hello! > >> > >> I created a preliminary webrev of my own implementation (no testcases > yet): > >> http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ > >> If anybody wants to sponsor my implementation, I will happily log an > issue and write tests. > >> > >> The name "pairing" was invented by me, but as I'm not a native English > speaker I cannot judge whether it's optimal, so better ideas are welcome. > >> Also I decided to remove accumulator types from public type variables. > They do not add anything to type signature, only clutter it > >> increasing the number of type parameters from 4 to 6. I think it was a > mistake to expose the accumulator type parameter in other cascading > collectors > >> like filtering(), collectingAndThen(), groupingBy(), etc. I'm not > insisting though, if you feel that conformance to existing collectors is > >> more important than simplicity. > >> > >> With best regards, > >> Tagir Valeev. > >> > >> On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz > wrote: > >> > >>> Well, I don't see the need to pack the two results into a Map.Entry > >>> (or any similar) container as a drawback. > >> From an "integrity of the JDK APIs" perspective, it is unquestionably > a > >> drawback. These items are not a Key and an associated Value in a Map; > >> it's merely pretending that Map.Entry really means "Pair". There's a > >> reason we don't have a Pair class in the JDK (and no, let's not reopen > >> that now); using something else as a Pair proxy that is supposed to have > >> specific semantics is worse. (It's fine to do this in your own code, but > >> not in the JDK. Different standards for code that has different > audiences.) > >> > >> Tagir's proposed sidestepping is nice, and it will also play nicely with > >> records, because then you can say: > >> > >> record NameAndCount(String name, int count); > >> > >> stream.collect(pairing(collectName, collectCount, > NameAndCount::new)); > >> > >> and get a more properly abstract result out. And its more in the spirit > >> of existing Collectors. If you want to use Map.Entry as an > >> _implementation detail_, that's fine. > >> > >> I can support this form. > >> > >>> I also don't see a larger abstraction like BiStream as a natural fit > >>> for a similar thing. > >> I think the BiStream connection is mostly tangential. We tried hard to > >> support streams of (K,V) pairs when we did streams, as Paul can attest, > >> but it was a huge complexity-inflater and dropping this out paid an > >> enormous simplification payoff. > >> > >> With records, having streams of tuples will be simpler to represent, but > >> no more performant; it will take until we get to value types and > >> specialized generics to get the performance we want out of this. > >> > >> > > From james.laskey at oracle.com Mon Jun 18 21:49:32 2018 From: james.laskey at oracle.com (James Laskey) Date: Mon, 18 Jun 2018 18:49:32 -0300 Subject: BiCollector In-Reply-To: <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: <4DB02197-8C11-4352-BA5A-CAEFE989107B@oracle.com> Bifurcate Sent from my iPhone > On Jun 18, 2018, at 6:29 PM, Brian Goetz wrote: > > "bisecting" sounds like it sends half the elements to one collector and half to the other ... > > "tee" might be a candidate, though it doesn't follow the `ing convention. "teeing" sounds dumb. > > > >> On 6/15/2018 7:36 PM, Paul Sandoz wrote: >> Hi Tagir, >> >> This is looking good. >> >> My current favorite name for the factory method is ?bisecting? since we are dividing the collector into two collectors, the results of which are then merged. >> Suggested first paragraph of the factory method: >> >> "Returns a collector that passes the input elements to two specified collectors and merges their results with the specified merge function.? >> >> Paul. >> >>> On Jun 15, 2018, at 4:26 AM, Tagir Valeev wrote: >>> >>> Hello! >>> >>> I created a preliminary webrev of my own implementation (no testcases yet): >>> http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ >>> If anybody wants to sponsor my implementation, I will happily log an issue and write tests. >>> >>> The name "pairing" was invented by me, but as I'm not a native English speaker I cannot judge whether it's optimal, so better ideas are welcome. >>> Also I decided to remove accumulator types from public type variables. They do not add anything to type signature, only clutter it >>> increasing the number of type parameters from 4 to 6. I think it was a mistake to expose the accumulator type parameter in other cascading collectors >>> like filtering(), collectingAndThen(), groupingBy(), etc. I'm not insisting though, if you feel that conformance to existing collectors is >>> more important than simplicity. >>> >>> With best regards, >>> Tagir Valeev. >>> >>>> On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz wrote: >>>> >>>> Well, I don't see the need to pack the two results into a Map.Entry >>>> (or any similar) container as a drawback. >>> From an "integrity of the JDK APIs" perspective, it is unquestionably a >>> drawback. These items are not a Key and an associated Value in a Map; >>> it's merely pretending that Map.Entry really means "Pair". There's a >>> reason we don't have a Pair class in the JDK (and no, let's not reopen >>> that now); using something else as a Pair proxy that is supposed to have >>> specific semantics is worse. (It's fine to do this in your own code, but >>> not in the JDK. Different standards for code that has different audiences.) >>> >>> Tagir's proposed sidestepping is nice, and it will also play nicely with >>> records, because then you can say: >>> >>> record NameAndCount(String name, int count); >>> >>> stream.collect(pairing(collectName, collectCount, NameAndCount::new)); >>> >>> and get a more properly abstract result out. And its more in the spirit >>> of existing Collectors. If you want to use Map.Entry as an >>> _implementation detail_, that's fine. >>> >>> I can support this form. >>> >>>> I also don't see a larger abstraction like BiStream as a natural fit >>>> for a similar thing. >>> I think the BiStream connection is mostly tangential. We tried hard to >>> support streams of (K,V) pairs when we did streams, as Paul can attest, >>> but it was a huge complexity-inflater and dropping this out paid an >>> enormous simplification payoff. >>> >>> With records, having streams of tuples will be simpler to represent, but >>> no more performant; it will take until we get to value types and >>> specialized generics to get the performance we want out of this. >>> >>> > From james.laskey at oracle.com Mon Jun 18 21:59:15 2018 From: james.laskey at oracle.com (James Laskey) Date: Mon, 18 Jun 2018 18:59:15 -0300 Subject: BiCollector In-Reply-To: <4DB02197-8C11-4352-BA5A-CAEFE989107B@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> <4DB02197-8C11-4352-BA5A-CAEFE989107B@oracle.com> Message-ID: <54006B6B-F512-4174-A9E6-67AE900D55BD@oracle.com> Replicator (as in DNA) Sent from my iPhone > On Jun 18, 2018, at 6:49 PM, James Laskey wrote: > > Bifurcate > > Sent from my iPhone > >> On Jun 18, 2018, at 6:29 PM, Brian Goetz wrote: >> >> "bisecting" sounds like it sends half the elements to one collector and half to the other ... >> >> "tee" might be a candidate, though it doesn't follow the `ing convention. "teeing" sounds dumb. >> >> >> >>> On 6/15/2018 7:36 PM, Paul Sandoz wrote: >>> Hi Tagir, >>> >>> This is looking good. >>> >>> My current favorite name for the factory method is ?bisecting? since we are dividing the collector into two collectors, the results of which are then merged. >>> Suggested first paragraph of the factory method: >>> >>> "Returns a collector that passes the input elements to two specified collectors and merges their results with the specified merge function.? >>> >>> Paul. >>> >>>> On Jun 15, 2018, at 4:26 AM, Tagir Valeev wrote: >>>> >>>> Hello! >>>> >>>> I created a preliminary webrev of my own implementation (no testcases yet): >>>> http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ >>>> If anybody wants to sponsor my implementation, I will happily log an issue and write tests. >>>> >>>> The name "pairing" was invented by me, but as I'm not a native English speaker I cannot judge whether it's optimal, so better ideas are welcome. >>>> Also I decided to remove accumulator types from public type variables. They do not add anything to type signature, only clutter it >>>> increasing the number of type parameters from 4 to 6. I think it was a mistake to expose the accumulator type parameter in other cascading collectors >>>> like filtering(), collectingAndThen(), groupingBy(), etc. I'm not insisting though, if you feel that conformance to existing collectors is >>>> more important than simplicity. >>>> >>>> With best regards, >>>> Tagir Valeev. >>>> >>>>> On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz wrote: >>>>> >>>>> Well, I don't see the need to pack the two results into a Map.Entry >>>>> (or any similar) container as a drawback. >>>> From an "integrity of the JDK APIs" perspective, it is unquestionably a >>>> drawback. These items are not a Key and an associated Value in a Map; >>>> it's merely pretending that Map.Entry really means "Pair". There's a >>>> reason we don't have a Pair class in the JDK (and no, let's not reopen >>>> that now); using something else as a Pair proxy that is supposed to have >>>> specific semantics is worse. (It's fine to do this in your own code, but >>>> not in the JDK. Different standards for code that has different audiences.) >>>> >>>> Tagir's proposed sidestepping is nice, and it will also play nicely with >>>> records, because then you can say: >>>> >>>> record NameAndCount(String name, int count); >>>> >>>> stream.collect(pairing(collectName, collectCount, NameAndCount::new)); >>>> >>>> and get a more properly abstract result out. And its more in the spirit >>>> of existing Collectors. If you want to use Map.Entry as an >>>> _implementation detail_, that's fine. >>>> >>>> I can support this form. >>>> >>>>> I also don't see a larger abstraction like BiStream as a natural fit >>>>> for a similar thing. >>>> I think the BiStream connection is mostly tangential. We tried hard to >>>> support streams of (K,V) pairs when we did streams, as Paul can attest, >>>> but it was a huge complexity-inflater and dropping this out paid an >>>> enormous simplification payoff. >>>> >>>> With records, having streams of tuples will be simpler to represent, but >>>> no more performant; it will take until we get to value types and >>>> specialized generics to get the performance we want out of this. >>>> >>>> >> > From chris.hegarty at oracle.com Mon Jun 18 22:04:01 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 18 Jun 2018 23:04:01 +0100 Subject: BiCollector In-Reply-To: <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: <153D2EF8-1820-4059-B374-1CC85D9111AF@oracle.com> > On 18 Jun 2018, at 22:29, Brian Goetz wrote: > > "bisecting" sounds like it sends half the elements to one collector and half to the other ... > > "tee" might be a candidate, though it doesn't follow the `ing convention. "teeing" sounds dumb. Doesn?t follow the convention either, but: fanout fanningOut -Chris. From brian.goetz at oracle.com Mon Jun 18 22:21:09 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 18 Jun 2018 18:21:09 -0400 Subject: BiCollector In-Reply-To: <153D2EF8-1820-4059-B374-1CC85D9111AF@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> <153D2EF8-1820-4059-B374-1CC85D9111AF@oracle.com> Message-ID: <027a9b5b-d4e8-de45-815f-9736c9da737e@oracle.com> distributing? On 6/18/2018 6:04 PM, Chris Hegarty wrote: > >> On 18 Jun 2018, at 22:29, Brian Goetz wrote: >> >> "bisecting" sounds like it sends half the elements to one collector and half to the other ... >> >> "tee" might be a candidate, though it doesn't follow the `ing convention. "teeing" sounds dumb. > Doesn?t follow the convention either, but: > > fanout > fanningOut > > -Chris. From martinrb at google.com Mon Jun 18 22:27:41 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 15:27:41 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: yes, the proposed new methods would use nulls differently. The original get() behavior of creating a mapping was a mistake, inconsistent with Map. Yes, it will cause confusion. But it's already confusing. On Mon, Jun 18, 2018 at 1:45 PM, David Lloyd wrote: > On Mon, Jun 18, 2018 at 12:53 PM, Martin Buchholz > wrote: > > On Mon, Jun 18, 2018 at 10:21 AM, Tony Printezis > > > wrote: > > > >> Martin, > >> > >> You are forgiven. :-) > >> > >> So, the (valid, I think) issue with getIfPresent(), as originally > proposed > >> by Peter, was that if get() was overridden in the subclass to somehow > >> transform the returned value, getIfPresent() wouldn?t apply the same > >> transformation. > >> > >> Doesn?t your compute() method have essentially the same issue? Apart > from > >> that, I personally like this proposal as I agree: one look-up is always > >> better than two. > >> > >> > > A non-prototype implementation would delegate compute into ThreadLocalMap > > itself, where there is no risk of overriding. > > I don't think overriding is the risk; the risk would be compute* > behaving inconsistently with regards to an overridden get*. > > > -- > - DML > From paul.sandoz at oracle.com Mon Jun 18 22:43:06 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 18 Jun 2018 15:43:06 -0700 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> Message-ID: Hi Peter, Existing composing collectors tend to unpack all the functions of the input collector ahead of time, i don?t recall being too concerned about this at the time. It does allow for more robust null checking, something we were less diligent about doing. Paul. > On Jun 17, 2018, at 3:04 AM, Peter Levart wrote: > > Hi Tagir, > > I don't know if this is important, but in your approach the particular functions of the sub-collectors are retrieved eagerly even if they are later not used. This should typically not present a problem as Collector(s) should be prepared to be used in any scenario (parallel or serial). But anyway in my approach, the sub-functions of the given collectors are retrived lazily, each time the Stream implementation needs them - the dynamics of invoking sub-collector methods to retrieve the functions is therefore unchanged when a collector is used directly to collect a Stream or as a sub-collector in the bi-collection scenario. > > What do you think of that particular detail? Paul? > > Regards, Peter > > On 06/15/18 13:26, Tagir Valeev wrote: >> Hello! >> >> I created a preliminary webrev of my own implementation (no testcases yet): >> http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ >> If anybody wants to sponsor my implementation, I will happily log an issue and write tests. >> >> The name "pairing" was invented by me, but as I'm not a native English speaker I cannot judge whether it's optimal, so better ideas are welcome. >> Also I decided to remove accumulator types from public type variables. They do not add anything to type signature, only clutter it >> increasing the number of type parameters from 4 to 6. I think it was a mistake to expose the accumulator type parameter in other cascading collectors >> like filtering(), collectingAndThen(), groupingBy(), etc. I'm not insisting though, if you feel that conformance to existing collectors is >> more important than simplicity. >> >> With best regards, >> Tagir Valeev. >> >> On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz > wrote: >> >> > Well, I don't see the need to pack the two results into a Map.Entry >> > (or any similar) container as a drawback. >> >> From an "integrity of the JDK APIs" perspective, it is unquestionably a >> drawback. These items are not a Key and an associated Value in a Map; >> it's merely pretending that Map.Entry really means "Pair". There's a >> reason we don't have a Pair class in the JDK (and no, let's not reopen >> that now); using something else as a Pair proxy that is supposed to have >> specific semantics is worse. (It's fine to do this in your own code, but >> not in the JDK. Different standards for code that has different audiences.) >> >> Tagir's proposed sidestepping is nice, and it will also play nicely with >> records, because then you can say: >> >> record NameAndCount(String name, int count); >> >> stream.collect(pairing(collectName, collectCount, NameAndCount::new)); >> >> and get a more properly abstract result out. And its more in the spirit >> of existing Collectors. If you want to use Map.Entry as an >> _implementation detail_, that's fine. >> >> I can support this form. >> >> > I also don't see a larger abstraction like BiStream as a natural fit >> > for a similar thing. >> >> I think the BiStream connection is mostly tangential. We tried hard to >> support streams of (K,V) pairs when we did streams, as Paul can attest, >> but it was a huge complexity-inflater and dropping this out paid an >> enormous simplification payoff. >> >> With records, having streams of tuples will be simpler to represent, but >> no more performant; it will take until we get to value types and >> specialized generics to get the performance we want out of this. >> >> > From brent.christian at oracle.com Mon Jun 18 22:53:27 2018 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 18 Jun 2018 15:53:27 -0700 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> Message-ID: <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> Hi, Roger On 6/18/18 7:31 AM, Roger Riggs wrote: > > The CSR and Webrev are updated. > > webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709 * src/java.base/share/classes/java/lang/System.java : Should the @implNote with the list of cached properties be added everywhere the @apiNote is being added ? Right now the @implNote is only added to getProperties(). * src/java.base/share/classes/jdk/internal/util/StaticProperty.java : Nit: 45 private StaticProperty() { 46 47 } Maybe put this all on one line? Otherwise, the change looks good to me. -Brent From stuart.marks at oracle.com Mon Jun 18 23:26:44 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 18 Jun 2018 16:26:44 -0700 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <1255272034.1809068.1529047619362.JavaMail.zimbra@u-pem.fr> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> <1255272034.1809068.1529047619362.JavaMail.zimbra@u-pem.fr> Message-ID: <5aee5c2e-dfbe-b8ae-fd29-e65e8b0c7583@oracle.com> Hi R?mi, On 6/15/18 12:26 AM, Remi Forax wrote: >> The overrides I had previously provided in specific implementation classes like >> ArrayList actually are slower, because the allocation of the array is done >> separately from filling it. This necessitates the extra zero-filling step. Thus, >> I've removed the overrides. > > for ArrayList, you may use a code like this, > T[] toArray(IntFunction generator) { > return (T[]) Arrays.copyOf(elementData, size, generator.apply(0).getClass()); > } > so you win only one comparison (yeah !), which can be perfectly predicted, so you should not see any perf difference :) True, this will probably work better than the previous code (allocate correct size, fill with System.arraycopy) but it doesn't seem likely to be any faster than the default method. > List.class or List[].class do not work either. I think they can be made to work (see other sub-thread with Peter Levart) but I don't see any advantages going in that direction. >> For these reasons I'd like to proceed with adding toArray(generator) API. > > so thumb up for me ! Great! s'marks From stuart.marks at oracle.com Tue Jun 19 00:08:09 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 18 Jun 2018 17:08:09 -0700 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <396353523.2274681.1529226937272.JavaMail.zimbra@u-pem.fr> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> <396353523.2274681.1529226937272.JavaMail.zimbra@u-pem.fr> Message-ID: Tagir Valeev wrote: >> If you are going to create long-living array which is likely to be empty, >> it's good to deduplicate them to spare the heap space. This can be easily >> done via existing toArray method like >> collection.toArray(MyClass.EMPTY_ARRAY) assuming we have an empty array >> constant. We use this pattern very often in IntelliJ IDEA source code (e. >> g. think of method which returns an array of Java member annotations; often >> there are none). New generator methods is much less convenient for this: >> toArray(s -> s == 0?MyClass.EMPTY_ARRAY:new MyClass[s]). I'm actually >> missing a method accepting an empty array instead of generator in the >> Stream API. And I don't think that new collection method is useful. The >> existing one is perfect. R?mi Forax wrote: > we can expect the VM to not allocate the array if once inlined the code is > new MyClass[0].getClass() > > if it's not the case, i think it's a better idea to tweak the VM rather than try to change an API based on a pattern that should not be necessary. I think the two of you are talking about different things. Tagir is concerned about *long-lived* zero-length arrays, whereas R?mi is talking about the possibility of short-circuiting the allocation of a zero-length array if it's replaced by a nonzero-length array and thus has an extremely short life. Tagir, if your use case is that you know you are creating lots of long-lived zero-length arrays, and you want to deduplicate them, then sure, using toArray(MyClass.EMPTY_ARRAY) is a fine thing to do. There are a bunch of assumptions here about the longevity and frequency of creation of such arrays. Having a shared empty array might indeed be the right thing for the IntelliJ IDEA code base, but that doesn't mean it's true in general. The toArray(generator) might be perfectly fine for many code bases even if it isn't suitable for the one you have in mind. You also mention the lack of a T[] Stream.toArray(T[]) method. This would seem to help the case of sharing a zero-length array, but Collection.toArray(T[]) also has odd semantics that we didn't want to replicate in streams. The point of that method is to *reuse* the argument array. The odd semantics are that, when the argument array is longer than necessary, null is added after the last written element. Instead of replicating the semantics of Collection.toArray(T[]) on Stream, we ended up with Stream.toArray(generator) instead. Now maybe the Stream.toArray() overloads aren't sufficient for what you want to do, in which case you might want to propose something. But that sounds like a different discussion. s'marks From brian.goetz at oracle.com Tue Jun 19 00:26:52 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 18 Jun 2018 20:26:52 -0400 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: +1 from me. > On Jun 14, 2018, at 9:02 PM, Stuart Marks wrote: > > Hi all, > > I wanted to restart review of the changeset for bug JDK-8060192 [1]. The latest webrev is here: [2]. > > The previous review was from December 2017: [3]. The only difference between the current changeset and the previous one is the removal of the overriding implementations (see explanation below). Much of the discussion in the previous review thread was around performance and the shape of the API. > > # Performance > > I previously had done some benchmarking on this and reported the results: [4]. (I've recently re-done the benchmarks and conclusions are essentially the same.) > > The upshot is that implementations that use Arrays.copyOf() are the fastest, probably because it's an intrinsic. It can avoid zero-filling the freshly allocated array because it knows the entire array contents will be overwritten. This is true regardless of what the public API looks like. The default implementation calls generator.apply(0) to get a zero-length array and then simply calls toArray(T[]). This goes through the Arrays.copyOf() fast path, so it's essentially the same speed as toArray(new T[0]). > > The overrides I had previously provided in specific implementation classes like ArrayList actually are slower, because the allocation of the array is done separately from filling it. This necessitates the extra zero-filling step. Thus, I've removed the overrides. > > # API Shape > > There was some discussion about whether it would be preferable to have a Class parameter as a type token for the array component type or the array type itself, instead of using an IntFunction generator. Most of this boils down to what people are comfortable with. However, there are a few points that make the generator function approach preferable. > > One point in favor of using a generator is that Stream already has a similar toArray(generator) method. > > Comparing this to the type token alternatives, for simple tasks like converting Collection to String[], things are about equal: > > toArray(String[]::new) > toArray(String.class) > toArray(String[].class) > > However, things are hairier if the element type of the collection is generic, such as Collection>. The result type is a generic array List[]. Using class literals or array constructor references will necessitate using an unchecked cast, because none of these can be used to represent a generic type. > > However, it's possible to use a method reference to provide a properly generic IntFunction. This would enable the toArray(generator) method to be used without any unchecked casts. (The method it refers to might need have an unchecked cast within it, though.) Such a method could also be reused for the corresponding Stream.toArray(generator) method. > > For these reasons I'd like to proceed with adding toArray(generator) API. > > Thanks, > > s'marks > > > [1] https://bugs.openjdk.java.net/browse/JDK-8060192 > > [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ > > [3] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html > > [4] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html > From john.r.rose at oracle.com Tue Jun 19 00:38:09 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 18 Jun 2018 17:38:09 -0700 Subject: BiCollector In-Reply-To: <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: On Jun 18, 2018, at 2:29 PM, Brian Goetz wrote: > > "bisecting" sounds like it sends half the elements to one collector and half to the other ? The main bisection or splitting operation that's relevant to a stream is what a spliterator does, so this is a concern. Nobody has mentioned "unzipping" yet; this is a term of art which applies to streams of tuples. The image of a zipper is relatively clear and unambiguous, and the tradition is pretty strong. https://en.wikipedia.org/wiki/Convolution_(computer_science) The thing we are looking at differs in two ways from classic "unzipping": First, the two collectors themselves convert the same T elements to whatever internal value (T1, T2) is relevant. Second, we are looking at a new terminal operation (a collector) which consolidates the results from both of streams (a notional Stream and Stream, if you like), rather than delivering the streams as a pair of outputs. The classic "unzip" operation applies "fst" and "snd" (or some other conventional set of access functions) to each T-element of the input stream. Since we don't have a privileged 2-tuple type (like Pair) in Java, the user would need to nominate those two functions explicitly, either by folding them into a "mapping" on each collector, or as a utility overloading like this: unzipping( Function f1, // defaults to identity Collector c1, Function f2, // defaults to identity Collector c2, BiFunction finisher) { return toBoth(mapping(f1, c1), mapping(f2, c2)); } > "tee" might be a candidate, though it doesn't follow the `ing convention. "teeing" sounds dumb. "tee" sounds asymmetrical. "diverting" or "detouring" are "*ing" words that might express asymmetrical disposition of derivative streams. An asymmetrical operation might be interesting if it could fork off a stream of its own. It would have to have a side-effecting void-producing terminal operation, so the main (undiverted) stream could continue to progress at the top level of the expression. interface Stream { default Stream diverting(Consumer> tee) { ? } } values.stream().diverting(s2->s2.forEach(System.out::println)).filter(?).collect(?); Or (and this might be a sweet spot) a symmetric stream-tee operation could materialize two sibling streams and rejoin their results with a bifunction: class Collectors { static Stream unzipping( Function, R1> f1, Function, R2> f2, BiFunction finisher) { ? } } values.stream().unzipping( s1->s1.forEach(System.out::println), s2->s2.filter(?).collect(?), (void1, r2)->r2 ); This would allow each "fork child" of the stream to continue to use the Stream API instead of the more restrictive Collector operators. Optimal code generation for forked/unzipped/teed streams would be tricky, requiring simultaneous loop control logic for each stream. To me that's a feature, not a bug, since hand-writing ad hoc simultaneous loops is a pain. My $0.02. ? John From stuart.marks at oracle.com Tue Jun 19 00:53:17 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 18 Jun 2018 17:53:17 -0700 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <940f6307-34fd-eea0-88e8-c8f5134b6829@gmail.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> <940f6307-34fd-eea0-88e8-c8f5134b6829@gmail.com> Message-ID: <0bdb6a71-6bf6-e999-f51f-740e81dce12f@oracle.com> On 6/17/18 1:50 AM, Peter Levart wrote: > It's a little strange that the generator function is used to construct an > empty array (at least in the default method, but overrides will likely do the > same if they want to avoid pre-zeroing overhead) in order to just extract the > array's type from it. Someone could reasonably expect the provided function > to be called with the exact length of needed array. The > Collection.toArray(T[]) at least gives user an option to actually fully use > the provided array, if user provides the correct length. This is actually what we're trying to avoid. The toArray(T[]) API has to deal with the cases where the array isn't the right length, and it reallocs or inserts an extra null in cases where it isn't. To avoid this, people do coll.toArray(new MyClass[coll.size()]) which turns out to be an anti-pattern. We could try to teach people to write coll.toArray(new MyClass[0]) instead, and this works, but it's quite non-obvious. ("Why do I need to create a zero-length array first?") (#include Tagir's comment about caching zero-length instances) Instead we want to direct people to write coll.toArray(MyClass[]::new) which creates an array of the right type and requested length. > The argument about using (and re-using) a method so that a method reference > can be passed to the method without any unchecked casts is equally true for > any of the 3 alternatives shown - the method itself might need unchecked > casts, but its usage not: > > static List[] array(int len) static Class> > elementType() static Class[]> arrayType() In principle all of these are possible. I don't see these as equal, though. It's quite common to have to create arrays of a generic type, either inline with unchecked casts, or possibly refactored into a method. I very rarely see Class objects of generic types. > But I can see that you want to align the API with Stream.toArray, while still > providing the optimal implementation. It's just that the API doesn't fit the > usecase. The function approach makes more sense in the Stream API where it > is explicitly explained that it may be used to construct arrays needed to > hold intermediary results of partitioned parallel execution too, but in > Collection API it is usually the case to just provide a copy of the > underlying data structure in the most optimal way (without pre-zeroing > overhead) and for that purpose, 2nd and 3rd alternatives are a better fit. Sure, the Stream API has additional uses for holding intermediate results. That doesn't imply that Collection.toArray(generator) doesn't meet its use case (which I described above). I also don't see how class type tokens are a better fit. A type token is "natural" if you're thinking of implementing it in terms of Arrays.copyOf() -- which it is right now, but that's an implementation detail. > Suppose Stream took a different approach and used the 2nd or 3rd approach > (which is universal). Would then Collection API also get the same method? I'm not sure where this is headed. I'm pretty sure we considered using type tokens for Stream.toArray() and rejected them in favor of toArray(generator). If there had been some reason to use a type token instead, maybe we would have used them, in which case we'd consider modifying Collection.toArray() would take a type token as well. But I'm not aware of such a reason, so.... > It might have been the case in the past when Java generics were introduced, > that class literals like List.class would just confuse users, > because most aspects of such type token would be erased and there were fears > that enabling them might limit options for reified generics in the future. > But long years have passed and java programmers have generally become > acquainted with Java generics and erasure to the point where it doesn't > confuse them any more, while reifying Java generics has been explored further > in Valhalla to the point where it has become obvious that erasure of > reference types is here to stay. > > Java could enable use of class literals like List.class without fear > that such literals would make users write buggy code or that such uses would > limit options for Java in the future. Quite the contrary, they would enable > users to write type-safer code than what they can write today. In the light > of future Java (valhalla specialized generics), where such class literals > make even more sense, enabling generic class literals could be viewed as a > stepping stone that has some purpose in the short term too. Again, I'm not sure where this is headed. I certainly would not propose changing the language to allow generic class literals in order to support the present use case (creating an array from a collection). As a longer term proposal, generic class literals might or might not be worthwhile on their own merits. It's certainly an irritant not having them; I could imagine some uses for them. But there are probably some downsides (such as unsoundness) that will need to be thought through. s'marks From stuart.marks at oracle.com Tue Jun 19 00:55:29 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 18 Jun 2018 17:55:29 -0700 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: <8a20606f-b1c5-300a-cffb-5f3b25220928@oracle.com> On 6/18/18 7:25 AM, Roger Riggs wrote: > In regard to new SharedSecret interfaces, one option is move shared (but > private) implementation classes > to a jdk.internal.xx package (not exported).? This only works well if they are > not tightly coupled to other > package private classes. > > SpinedBuffer might be a good candidate, I have some IO cases in mind that could > benefit from > the allocation/reallocation savings.? (ByteArrayOutputStream for 1). Yes, SpinedBuffer is a good candidate for this. There's nothing special about it that ties it to streams. It was just put into java.util.stream because streams were its initial (and currently only) users. A SharedSecret would be helpful for things like new private factory methods for the unmodifiable collections, such as ones that might assume ownership of an array instead of copying it. s'marks From joe.darcy at oracle.com Tue Jun 19 01:18:40 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Mon, 18 Jun 2018 18:18:40 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication In-Reply-To: References: Message-ID: <5B2859F0.7050000@oracle.com> Looks good Liam; thanks, -Joe On 6/15/2018 10:58 AM, Liam Miller-Cushon wrote: > Hello, > > This change is a follow-up to JDK-7183985 which replaces the interior of > parseClassArray, parseEnumArray, and parseAnnotationArray with a shared > method that is passed a lambda for the type-specific parsing logic. > > bug: https://bugs.openjdk.java.net/browse/JDK-8198669 > webrev: http://cr.openjdk.java.net/~cushon/8198669/webrev.00/ > > Liam From erik.gahlin at oracle.com Tue Jun 19 02:06:43 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 19 Jun 2018 04:06:43 +0200 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr Message-ID: <5B286533.6070500@oracle.com> Hi, Could I have a review of an enhancement that will make it possible to add JFR events to java.base, and possibly other modules in the JDK, without a compile time dependency on jdk.jfr. Bug: https://bugs.openjdk.java.net/browse/JDK-8203629 Webrev: http://cr.openjdk.java.net/~egahlin/8203629.0 Testing: Tests in test/jdk/jdk/jfr The functionality is a prerequisite for an upcoming enhancement that will add JFR events to the security libraries. In short, jdk.jfr.Event (located in jdk.jfr) extends jdk.internal.event.Event (located in java.base). Metadata for events, such as labels and descriptions, are added using a mirror event class located in jdk.jfr module. If the fields of the mirror class and the event class don't match an InternalError is thrown. To illustrate how the mechanism can be used, see the following example (not meant to be checked in). http://cr.openjdk.java.net/~egahlin/8203629.example Since the implementation of jdk.internal.event.Event is empty, the JIT will typically eliminate all traces of JFR functionality unless Flight Recorder is enabled. Thanks Erik From stuart.marks at oracle.com Tue Jun 19 02:09:42 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 18 Jun 2018 19:09:42 -0700 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: On 6/17/18 4:23 PM, Peter Levart wrote: > My attempt to optimize the Map.copyOf() was also using just public APIs (with > the addition of an internal class). > Well, it does speed-up Map.copyOf() by 30%. But I agree there are other > approaches that could go even further. In particular when all intermediary > copies could be avoided. > > Here's one approach (which still uses just public APIs) and avoids intermediary > copying: You mentioned "using just public APIs" a couple times. Are you viewing that as a constraint? I don't think it is. > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.02/ > > Why choosing Map.forEach for dumping the content of the map instead of iteration > over entrySet? Because in some Map implementations this is the most direct > iteration without producing garbage (for example in IdentityHashMap), but in the > worst case (default Map method for example) it is delegated to iteration over > entrySet. Sure, entrySet().toArray() does a lot of allocation and copying. Using forEach() can avoid this, but it doesn't necessarily preserve the right invariants. In particular, if the source is a concurrent map, the number of times forEach's BiConsumer is called might not match size(), if the map has changed size. So, the forEach approach has to deal with the possibility of it not being called exactly size() times, whereas we know that the array from toArray() can't change size (and that its contents are consistent, for some definition of consistent). This change also publishes a reference to the Map under construction before its constructor has returned. I'm not sure of all the memory model implications of this. This change also hands out an object that has access to the new Map instance's internals. You're aware of this, of course, because you snapshot the current thread and null it out later, saying // prevent naughty Map implementations from modifying MapN after // it is fully constructed So there are potential security vulnerabilities here, which requires some serious thought. What you have *seems* reasonable, but my experience is that things that are subtle but that seem reasonable actually end up having security holes. I'm trying to think of some alternatives. The issue with the forEach() approach on an arbitrary map is that you have to hand out a BiConsumer, and malicious code can call it even after forEach() has returned. What's necessary is to have a way to shut off the BiConsumer before reading out what it's accumulated. I don't know of a simple, fast, and correct way to do this. (Choose any two!) The "enabled" state (whether a Thread instance or just a boolean) will probably have to be a volatile that must be checked every time. What are the performance implications? Anyway, while it seems promising, the forEach() approach seems like it leads off into the weeds. Unless you can verify the trustworthiness of the source. Suppose you check the source map to see if its getClass() == HashMap.class. Then you can be reasonably assured that forEach() won't misuse the BiConsumer. You can probably also rely on size() being stable. (Probably also include LinkedHashMap.) The toUnmodifiableMap collectors can benefit from this if they're changed to use Map::copyOf, as in your current webrev. You could also provide a similar fast path for ConcurrentHashMap. It'd have to be resilient against size changes, but you can trust that it won't misuse the BiConsumer. Finally, other map implementations will just have to suffer through the multi-copy slow path. s'marks From david.holmes at oracle.com Tue Jun 19 04:01:38 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Jun 2018 14:01:38 +1000 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> Message-ID: On 19/06/2018 6:01 AM, Doug Lea wrote: > On 06/18/2018 11:22 AM, Martin Buchholz wrote: > >> Or better, lockField.set in resetLock could instead be setRelease. >> Except it is using reflection, not VarHandles. So it could be preceded >> with VarHandle.releaseFence(), although it is hard to think of a >> scenario in which it could matter. >> >> >> You mean followed by, not preceded by? >> >> ? ? ? ? ? try { >> ? ? ? ? ? ? ?lockField.set(this, new Object()); >> +? ? ? ? ? ? java.lang.invoke.VarHandle.releaseFence(); >> ? ? ? ? ?} catch (IllegalAccessException e) { >> ? ? ? ? ? ? ?throw new Error(e); >> ? ? ? ? ?} >> > > OK. Followed by is a little better in that it orders any field write > before any write of this, not just the array copy before lock field write. Hang on! A releasing store does the "release" before the store not after it. The whole point being if you see the result of the store then you are guaranteed to see all previous stores. The above can reorder the lockField store with whatever stores come before it. David ----- > -Doug > From kirk.pepperdine at gmail.com Tue Jun 19 04:40:35 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 19 Jun 2018 07:40:35 +0300 Subject: BiCollector In-Reply-To: <027a9b5b-d4e8-de45-815f-9736c9da737e@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> <153D2EF8-1820-4059-B374-1CC85D9111AF@oracle.com> <027a9b5b-d4e8-de45-815f-9736c9da737e@oracle.com> Message-ID: > On Jun 19, 2018, at 1:21 AM, Brian Goetz wrote: > > distributing? > > On 6/18/2018 6:04 PM, Chris Hegarty wrote: >> >>> On 18 Jun 2018, at 22:29, Brian Goetz wrote: >>> >>> "bisecting" sounds like it sends half the elements to one collector and half to the other ... >>> >>> "tee" might be a candidate, though it doesn't follow the `ing convention. "teeing" sounds dumb. But teeing (https://en.wikipedia.org/wiki/Tee_(command) ) is sort of what you?re doing so it sounds just fine. .. I?ve always thought it like tapping but this maybe less descriptive than split which of course would be confused with Spliterator. ? Kirk From david.holmes at oracle.com Tue Jun 19 04:41:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Jun 2018 14:41:22 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> Message-ID: <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> Discussions on the CSR request have led to further changes to the documentation involving nests and the new nest-related method. There are no semantic changes here just clearer explanations. Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ (don't worry if you don't see a v6, it didn't really exist). Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v7/ Specdiffs updated in place at: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/ Summary of changes: - The definition of a nest etc is moved to the class-level javadoc of java.lang.Class, along with some other edits provided by Alex Buckley to pave the way for future updates - The nest-related methods are written in a more clear and consistent way Thanks, David ----- On 12/06/2018 3:16 PM, David Holmes wrote: > Here is one further minor update from the CSR discussions: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html > > > Thanks, > David > > On 25/05/2018 3:52 PM, David Holmes wrote: >> Here are the further minor updates so far in response to all the >> review comments. >> >> Incremental corelibs webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ >> >> >> Full corelibs webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ >> >> Change summary: >> >> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >> - remove inaccurate pseudo-assertion comment >> >> test/jdk/java/lang/reflect/Nestmates/SampleNest.java >> - code cleanup: <> operator >> >> test/jdk/java/lang/reflect/Nestmates/TestReflectionAPI.java >> - code cleanup: streamify duplicate removals >> >> test/jdk/java/lang/invoke/PrivateInterfaceCall.java >> - use consistent @bug number >> >> Thanks, >> David >> >> On 22/05/2018 8:15 PM, David Holmes wrote: >>> Here are the updates so far in response to all the review comments. >>> >>> Incremental webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2-incr/ >>> >>> >>> Full webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2/ >>> >>> Change summary: >>> >>> src/java.base/share/classes/java/lang/Class.java >>> - getNesthost: >>> ?? - change "any error" -> "any linkage error" as runtime errors will >>> propagate. [This needs ratifying by EG] >>> ?? - add clarification that primitive and array classes are not >>> explicitly members of any nest and so form singleton nests >>> ?? - add clarification that all nestmates are in the same package >>> ?? - re-word @return text to exclude the "royal 'we'" >>> - fix javadoc cross references >>> >>> --- >>> >>> Moved reflection API tests from >>> test/hotspot/jtreg/runtime/Nestmates/reflectionAPI/ to >>> test/jdk/java/lang/reflect/Nestmates/ >>> >>> --- >>> >>> java/lang/reflect/Nestmates/TestReflectionAPI.java >>> >>> Run tests twice to show that failure reasons remain the same. >>> >>> --- >>> >>> test/jdk/jdk/lambda/vm/InterfaceAccessFlagsTest.java >>> >>> Disable test via annotation rather than commenting out. >>> >>> --- >>> >>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>> >>> - Fix indent for nestmate access check. >>> - Remove unnecessary local variable >>> >>> --- >>> >>> src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >>> >>> - Replace myassert with proper assert >>> >>> --- >>> >>> Thanks, >>> David >>> >>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>> This review is being spread across four groups: langtools, >>>> core-libs, hotspot and serviceability. This is the specific review >>>> thread for core-libs - webrev: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v1/ >>>> >>>> See below for full details - including annotated full webrev guiding >>>> the review. >>>> >>>> The intent is to have JEP-181 targeted and integrated by the end of >>>> this month. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> The nestmates project (JEP-181) introduces new classfile attributes >>>> to identify classes and interfaces in the same nest, so that the VM >>>> can perform access control based on those attributes and so allow >>>> direct private access between nestmates without requiring javac to >>>> generate synthetic accessor methods. These access control changes >>>> also extend to core reflection and the MethodHandle.Lookup contexts. >>>> >>>> Direct private calls between nestmates requires a more general >>>> calling context than is permitted by invokespecial, and so the JVMS >>>> is updated to allow, and javac updated to use, invokevirtual and >>>> invokeinterface for private class and interface method calls >>>> respectively. These changed semantics also extend to MethodHandle >>>> findXXX operations. >>>> >>>> At this time we are only concerned with static nest definitions, >>>> which map to a top-level class/interface as the nest-host and all >>>> its nested types as nest-members. >>>> >>>> Please see the JEP for further details. >>>> >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>> >>>> All of the specification changes have been previously been worked >>>> out by the Valhalla Project Expert Group, and the implementation >>>> reviewed by the various contributors and discussed on the >>>> valhalla-dev mailing list. >>>> >>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>> Kumar Srinivasan >>>> >>>> Master webrev of all changes: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>> >>>> Annotated master webrev index: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>> >>>> Performance: this is expected to be performance neutral in a general >>>> sense. Benchmarking and performance runs are about to start. >>>> >>>> Testing Discussion: >>>> ------------------ >>>> >>>> The testing for nestmates can be broken into four main groups: >>>> >>>> -? New tests specifically related to nestmates and currently in the >>>> runtime/Nestmates directory >>>> >>>> - New tests to complement existing tests by adding in testcases not >>>> previously expressible. >>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>> use of invokespecial for private interface methods and performing >>>> receiver typechecks, so we add >>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>> invokeinterface. >>>> >>>> -? New JVM TI tests to verify the spec changes related to nest >>>> attributes. >>>> >>>> -? Existing tests significantly affected by the nestmates changes, >>>> primarily: >>>> ??? -? runtime/SelectionResolution >>>> >>>> ??? In most cases the nestmate changes makes certain invocations >>>> that were illegal, legal (e.g. not requiring invokespecial to invoke >>>> private interface methods; allowing access to private members via >>>> reflection/Methodhandles that were previously not allowed). >>>> >>>> - Existing tests incidentally affected by the nestmate changes >>>> >>>> ?? This includes tests of things utilising class >>>> redefinition/retransformation to alter nested types but which >>>> unintentionally alter nest relationships (which is not permitted). >>>> >>>> There are still a number of tests problem-listed with issues filed >>>> against them to have them adapted to work with nestmates. Some of >>>> these are intended to be addressed in the short-term, while some >>>> (such as the runtime/SelectionResolution test changes) may not >>>> eventuate. >>>> >>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>> >>>> There is also further test work still to be completed (the JNI and >>>> JDI invocation tests): >>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>> which will continue in parallel with the main RFR. >>>> >>>> Pre-integration Testing: >>>> ??- General: >>>> ???? - Mach5: hs/jdk tier1,2 >>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>> ??- Targetted >>>> ??? - nashorn (for asm changes) >>>> ??? - hotspot: runtime/* >>>> ?????????????? serviceability/* >>>> ?????????????? compiler/* >>>> ?????????????? vmTestbase/* >>>> ??? - jdk: java/lang/invoke/* >>>> ?????????? java/lang/reflect/* >>>> ?????????? java/lang/instrument/* >>>> ?????????? java/lang/Class/* >>>> ?????????? java/lang/management/* >>>> ?? - langtools: tools/javac >>>> ??????????????? tools/javap >>>> From martinrb at google.com Tue Jun 19 05:08:33 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 22:08:33 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> Message-ID: Latest version looks like this: public Object clone() { try { @SuppressWarnings("unchecked") CopyOnWriteArrayList clone = (CopyOnWriteArrayList) super.clone(); clone.resetLock(); + // Unlike in readObject, here we cannot visibility-piggyback on the + // volatile write in setArray(). + VarHandle.releaseFence(); return clone; } catch (CloneNotSupportedException e) { // this shouldn't happen, since we are Cloneable throw new InternalError(); } } The store we worry about is a hypothetical subsequent racy publication of the new object, so we put the release fence at the end of the pseudo-constructor. On Mon, Jun 18, 2018 at 9:01 PM, David Holmes wrote: > On 19/06/2018 6:01 AM, Doug Lea wrote: > > Hang on! A releasing store does the "release" before the store not after > it. The whole point being if you see the result of the store then you are > guaranteed to see all previous stores. The above can reorder the lockField > store with whatever stores come before it. > > David > ----- > > From david.holmes at oracle.com Tue Jun 19 05:19:24 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Jun 2018 15:19:24 +1000 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> Message-ID: On 19/06/2018 3:08 PM, Martin Buchholz wrote: > Latest version looks like this: > > ? ? ?public Object clone() { > ? ? ? ? ?try { > ? ? ? ? ? ? ?@SuppressWarnings("unchecked") > ? ? ? ? ? ? ?CopyOnWriteArrayList clone = > ? ? ? ? ? ? ? ? ?(CopyOnWriteArrayList) super.clone(); > ? ? ? ? ? ? ?clone.resetLock(); > +? ? ? ? ? ? // Unlike in readObject, here we cannot > visibility-piggyback on the > +? ? ? ? ? ? // volatile write in setArray(). > +? ? ? ? ? ? VarHandle.releaseFence(); > ? ? ? ? ? ? ?return clone; > ? ? ? ? ?} catch (CloneNotSupportedException e) { > ? ? ? ? ? ? ?// this shouldn't happen, since we are Cloneable > ? ? ? ? ? ? ?throw new InternalError(); > ? ? ? ? ?} > ? ? ?} > > The store we worry about is a hypothetical subsequent racy publication > of the new object, so we put the release fence at the end of the > pseudo-constructor. Thanks for clarifying. I'm unsure why we're trying to make this particular class unsafe-publication-proof, but I guess it's generally a good thing. David > > On Mon, Jun 18, 2018 at 9:01 PM, David Holmes > wrote: > > On 19/06/2018 6:01 AM, Doug Lea wrote: > > Hang on! A releasing store does the "release" before the store not > after it. The whole point being if you see the result of the store > then you are guaranteed to see all previous stores. The above can > reorder the lockField store with whatever stores come before it. > > David > ----- > > From martinrb at google.com Tue Jun 19 05:34:00 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 22:34:00 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> Message-ID: On Mon, Jun 18, 2018 at 10:19 PM, David Holmes wrote: > > I'm unsure why we're trying to make this particular class > unsafe-publication-proof, but I guess it's generally a good thing. The particular field we're changing is a """final""" field, and it would be pretty embarrassing to grab a stale value of that - the wrong lock! From orionllmain at gmail.com Tue Jun 19 06:11:15 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Tue, 19 Jun 2018 13:11:15 +0700 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: The function you propose is just a binary variant of mapping: Collector mapping( Function mapper, Collector downstream); (omitted '? super' for readability) So, it is logical to use the name biMapping: Collector biMapping( Function mapper1, Function mapper2, Collector downstream1, Collector downstream2, BiFunction finisher); 2018-06-19 7:38 GMT+07:00 John Rose : > On Jun 18, 2018, at 2:29 PM, Brian Goetz wrote: > > > > "bisecting" sounds like it sends half the elements to one collector and > half to the other ? > > The main bisection or splitting operation that's relevant to a stream is > what > a spliterator does, so this is a concern. > > Nobody has mentioned "unzipping" yet; this is a term of art which applies > to streams > of tuples. The image of a zipper is relatively clear and unambiguous, and > the tradition > is pretty strong. https://en.wikipedia.org/wiki/ > Convolution_(computer_science) > > The thing we are looking at differs in two ways from classic "unzipping": > First, the > two collectors themselves convert the same T elements to whatever internal > value > (T1, T2) is relevant. Second, we are looking at a new terminal operation > (a collector) which > consolidates the results from both of streams (a notional Stream and > Stream, > if you like), rather than delivering the streams as a pair of outputs. > > The classic "unzip" operation applies "fst" and "snd" (or some other > conventional > set of access functions) to each T-element of the input stream. Since we > don't > have a privileged 2-tuple type (like Pair) in Java, the user would > need > to nominate those two functions explicitly, either by folding them into a > "mapping" > on each collector, or as a utility overloading like this: > > unzipping( > Function f1, // defaults to identity > Collector c1, > Function f2, // defaults to identity > Collector c2, > BiFunction finisher) { > return toBoth(mapping(f1, c1), mapping(f2, c2)); > } > > > > "tee" might be a candidate, though it doesn't follow the `ing > convention. "teeing" sounds dumb. > > > "tee" sounds asymmetrical. "diverting" or "detouring" are "*ing" words > that might > express asymmetrical disposition of derivative streams. > > An asymmetrical operation might be interesting if it could fork off a > stream of > its own. It would have to have a side-effecting void-producing terminal > operation, > so the main (undiverted) stream could continue to progress at the top > level of > the expression. > > interface Stream { > default Stream diverting(Consumer> tee) { ? } > } > > values.stream().diverting(s2->s2.forEach(System.out:: > println)).filter(?).collect(?); > > Or (and this might be a sweet spot) a symmetric stream-tee operation could > materialize two sibling streams and rejoin their results with a bifunction: > > class Collectors { > static Stream unzipping( > Function, R1> f1, > Function, R2> f2, > BiFunction finisher) > { ? } > } > > values.stream().unzipping( > s1->s1.forEach(System.out::println), > s2->s2.filter(?).collect(?), > (void1, r2)->r2 > ); > > This would allow each "fork child" of the stream to continue to use the > Stream API instead of the more restrictive Collector operators. > > Optimal code generation for forked/unzipped/teed streams would be tricky, > requiring simultaneous loop control logic for each stream. > To me that's a feature, not a bug, since hand-writing ad hoc > simultaneous loops is a pain. > > My $0.02. > > ? John From kirk.pepperdine at gmail.com Tue Jun 19 06:17:56 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 19 Jun 2018 09:17:56 +0300 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: <51E93CAC-A4DF-4B64-BF0A-36C4AA1145E4@gmail.com> > On Jun 19, 2018, at 9:11 AM, Zheka Kozlov wrote: > > The function you propose is just a binary variant of mapping: > > Collector mapping( > Function mapper, > Collector downstream); > > (omitted '? super' for readability) > > So, it is logical to use the name biMapping: > > Collector biMapping( > Function mapper1, > Function mapper2, > Collector downstream1, > Collector downstream2, > BiFunction finisher); +1 > > > 2018-06-19 7:38 GMT+07:00 John Rose : > >> On Jun 18, 2018, at 2:29 PM, Brian Goetz wrote: >>> >>> "bisecting" sounds like it sends half the elements to one collector and >> half to the other ? >> >> The main bisection or splitting operation that's relevant to a stream is >> what >> a spliterator does, so this is a concern. >> >> Nobody has mentioned "unzipping" yet; this is a term of art which applies >> to streams >> of tuples. The image of a zipper is relatively clear and unambiguous, and >> the tradition >> is pretty strong. https://en.wikipedia.org/wiki/ >> Convolution_(computer_science) >> >> The thing we are looking at differs in two ways from classic "unzipping": >> First, the >> two collectors themselves convert the same T elements to whatever internal >> value >> (T1, T2) is relevant. Second, we are looking at a new terminal operation >> (a collector) which >> consolidates the results from both of streams (a notional Stream and >> Stream, >> if you like), rather than delivering the streams as a pair of outputs. >> >> The classic "unzip" operation applies "fst" and "snd" (or some other >> conventional >> set of access functions) to each T-element of the input stream. Since we >> don't >> have a privileged 2-tuple type (like Pair) in Java, the user would >> need >> to nominate those two functions explicitly, either by folding them into a >> "mapping" >> on each collector, or as a utility overloading like this: >> >> unzipping( >> Function f1, // defaults to identity >> Collector c1, >> Function f2, // defaults to identity >> Collector c2, >> BiFunction finisher) { >> return toBoth(mapping(f1, c1), mapping(f2, c2)); >> } >> >> >>> "tee" might be a candidate, though it doesn't follow the `ing >> convention. "teeing" sounds dumb. >> >> >> "tee" sounds asymmetrical. "diverting" or "detouring" are "*ing" words >> that might >> express asymmetrical disposition of derivative streams. >> >> An asymmetrical operation might be interesting if it could fork off a >> stream of >> its own. It would have to have a side-effecting void-producing terminal >> operation, >> so the main (undiverted) stream could continue to progress at the top >> level of >> the expression. >> >> interface Stream { >> default Stream diverting(Consumer> tee) { ? } >> } >> >> values.stream().diverting(s2->s2.forEach(System.out:: >> println)).filter(?).collect(?); >> >> Or (and this might be a sweet spot) a symmetric stream-tee operation could >> materialize two sibling streams and rejoin their results with a bifunction: >> >> class Collectors { >> static Stream unzipping( >> Function, R1> f1, >> Function, R2> f2, >> BiFunction finisher) >> { ? } >> } >> >> values.stream().unzipping( >> s1->s1.forEach(System.out::println), >> s2->s2.filter(?).collect(?), >> (void1, r2)->r2 >> ); >> >> This would allow each "fork child" of the stream to continue to use the >> Stream API instead of the more restrictive Collector operators. >> >> Optimal code generation for forked/unzipped/teed streams would be tricky, >> requiring simultaneous loop control logic for each stream. >> To me that's a feature, not a bug, since hand-writing ad hoc >> simultaneous loops is a pain. >> >> My $0.02. >> >> ? John From peter.levart at gmail.com Tue Jun 19 07:43:14 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 19 Jun 2018 09:43:14 +0200 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: On 06/19/18 02:38, John Rose wrote: > unzipping( > Function f1, // defaults to identity > Collector c1, > Function f2, // defaults to identity > Collector c2, > BiFunction finisher) { > return toBoth(mapping(f1, c1), mapping(f2, c2)); > } You don't need these f1 and f2 as arguments, because we already have a Collectors.mapping(f1, c1) combinator. So you can write: unzipping( ??? mapping(f1, c1), ??? mapping(f2, c2) ??? finisher ) But then unzipping is not really "unzipping". What's wrong with my initial proposal of Collectors.toBoth()? We already have some toXxx() methods (toList, toSet, toCollection, toMap, ...), so toBoth seems to me as a natural extension of that naming principle: toBoth( ??? toMap() ??? toList() ??? combiner ) But I'm open to other naming suggestions. I would also call the 3rd parameter 'combiner' rather than 'finisher', because finisher is known as particular function that is bound to the particular Collector. And this 3rd argument is not the resulting collector's finisher - it is just a part of it. The real finisher of the resulting Collector is composed of that function and finishers of underlying collectors. Regards, Peter From peter.levart at gmail.com Tue Jun 19 07:52:28 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 19 Jun 2018 09:52:28 +0200 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: <3e389a9e-16bc-72a0-632c-623ea17fbcc4@gmail.com> On 06/19/18 09:43, Peter Levart wrote: > We already have some toXxx() methods (toList, toSet, toCollection, > toMap, ...), so toBoth seems to me as a natural extension of that > naming principle: Well, on a second thought, toList, toSet, etc... they all name a type that is a result of the collection (List, Set, etc.). But we don't have a type called Both (yet). So, please, continue with suggestions... Peter From dawid.weiss at gmail.com Tue Jun 19 08:00:17 2018 From: dawid.weiss at gmail.com (Dawid Weiss) Date: Tue, 19 Jun 2018 10:00:17 +0200 Subject: BiCollector In-Reply-To: <3e389a9e-16bc-72a0-632c-623ea17fbcc4@gmail.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> <3e389a9e-16bc-72a0-632c-623ea17fbcc4@gmail.com> Message-ID: Not that it's an important factor but as a non-native English speaker I like the simplest form of "toBoth()" best. I also doubt there will ever be a "Both" class in the JDK to worry about, even if such examples can be found in the wild [1]. Dawid [1] https://github.com/search?q=filename%3ABoth.java On Tue, Jun 19, 2018 at 9:52 AM, Peter Levart wrote: > > > On 06/19/18 09:43, Peter Levart wrote: >> >> We already have some toXxx() methods (toList, toSet, toCollection, toMap, >> ...), so toBoth seems to me as a natural extension of that naming principle: > > > Well, on a second thought, toList, toSet, etc... they all name a type that > is a result of the collection (List, Set, etc.). But we don't have a type > called Both (yet). > > So, please, continue with suggestions... > > Peter > From orionllmain at gmail.com Tue Jun 19 08:09:02 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Tue, 19 Jun 2018 15:09:02 +0700 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: I agree that f1 and f2 are unnecessary. I also noticed that toBoth is a binary form of `collectingAndThen`: public static Collector collectingAndThen( Collector downstream, Function finisher); So what about `collectionToBothAndThen`? public static Collector collectingToBothAndThen( Collector downstream1, Collector downstream2, BiFunction finisher); Another options: collecting2 collectingToBoth biCollecting 2018-06-19 14:43 GMT+07:00 Peter Levart : > > > On 06/19/18 02:38, John Rose wrote: > >> unzipping( >> Function f1, // defaults to identity >> Collector c1, >> Function f2, // defaults to identity >> Collector c2, >> BiFunction finisher) >> { >> return toBoth(mapping(f1, c1), mapping(f2, c2)); >> } >> > > You don't need these f1 and f2 as arguments, because we already have a > Collectors.mapping(f1, c1) combinator. So you can write: > > unzipping( > mapping(f1, c1), > mapping(f2, c2) > finisher > ) > > But then unzipping is not really "unzipping". > > What's wrong with my initial proposal of Collectors.toBoth()? > > We already have some toXxx() methods (toList, toSet, toCollection, toMap, > ...), so toBoth seems to me as a natural extension of that naming principle: > > toBoth( > toMap() > toList() > combiner > ) > > But I'm open to other naming suggestions. > > I would also call the 3rd parameter 'combiner' rather than 'finisher', > because finisher is known as particular function that is bound to the > particular Collector. And this 3rd argument is not the resulting > collector's finisher - it is just a part of it. The real finisher of the > resulting Collector is composed of that function and finishers of > underlying collectors. > > Regards, Peter > > From peter.levart at gmail.com Tue Jun 19 10:34:08 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 19 Jun 2018 12:34:08 +0200 Subject: BiCollector In-Reply-To: <027a9b5b-d4e8-de45-815f-9736c9da737e@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> <153D2EF8-1820-4059-B374-1CC85D9111AF@oracle.com> <027a9b5b-d4e8-de45-815f-9736c9da737e@oracle.com> Message-ID: On 06/19/2018 12:21 AM, Brian Goetz wrote: > distributing? More "replicating" than "distributing" (thinking of replicated vs. distributed caches for example). Peter From peter.levart at gmail.com Tue Jun 19 11:17:09 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 19 Jun 2018 13:17:09 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <17343fde-3017-0bd8-e426-be9f4c17187e@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> <8474cc34-36d7-fc76-6297-60f48557c432@gmail.com> <17343fde-3017-0bd8-e426-be9f4c17187e@oracle.com> Message-ID: <273ce785-d7f3-254e-95fb-48c5fa7e275a@gmail.com> On 06/18/2018 06:53 PM, Claes Redestad wrote: >> Plain write that follows in program a volatile write, can in theory >> float before the volatile write. So if you publish a Properties >> instance via data race (via plain write), the observer may still see >> uninitialized 'map' and 'defaults' fields. >> > > Right > > http://cr.openjdk.java.net/~redestad/8199435.01/ > > (Yes, using VarHandle.storeStoreFence would do the exact same thing, > but is not usable from Properties as the VarHandle impl needs to read > some system properties...) > > /Claes This looks good to me. One might think that the same fence is needed in deserialization too (readHashtable), but ObjectInputStream already makes care of that (see ObjectInputStream.freeze() and the two calls in readObject() and readUnshared()), so this is fine now. Regards, Peter From brian.goetz at oracle.com Tue Jun 19 12:38:43 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 19 Jun 2018 08:38:43 -0400 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: <460f175e-db12-e87f-d53d-647436e7af1f@oracle.com> No. The essence of `mapping` is: apply this function, then pass them down to something else.? In that case, the something else is secondary to the mapping. The essence of this method is: split the stream into _two_ streams, so it can be operated on by two collectors.? Any mapping here is incidental. On 6/19/2018 2:11 AM, Zheka Kozlov wrote: > The function you propose is just a binary variant of mapping: > > Collector mapping( > Function mapper, > Collector downstream); > > (omitted '? super' for readability) > > So, it is logical to use the name biMapping: > > Collector biMapping( > Function mapper1, > Function mapper2, > Collector downstream1, > Collector downstream2, > BiFunction finisher); > > > 2018-06-19 7:38 GMT+07:00 John Rose : > >> On Jun 18, 2018, at 2:29 PM, Brian Goetz wrote: >>> "bisecting" sounds like it sends half the elements to one collector and >> half to the other ? >> >> The main bisection or splitting operation that's relevant to a stream is >> what >> a spliterator does, so this is a concern. >> >> Nobody has mentioned "unzipping" yet; this is a term of art which applies >> to streams >> of tuples. The image of a zipper is relatively clear and unambiguous, and >> the tradition >> is pretty strong. https://en.wikipedia.org/wiki/ >> Convolution_(computer_science) >> >> The thing we are looking at differs in two ways from classic "unzipping": >> First, the >> two collectors themselves convert the same T elements to whatever internal >> value >> (T1, T2) is relevant. Second, we are looking at a new terminal operation >> (a collector) which >> consolidates the results from both of streams (a notional Stream and >> Stream, >> if you like), rather than delivering the streams as a pair of outputs. >> >> The classic "unzip" operation applies "fst" and "snd" (or some other >> conventional >> set of access functions) to each T-element of the input stream. Since we >> don't >> have a privileged 2-tuple type (like Pair) in Java, the user would >> need >> to nominate those two functions explicitly, either by folding them into a >> "mapping" >> on each collector, or as a utility overloading like this: >> >> unzipping( >> Function f1, // defaults to identity >> Collector c1, >> Function f2, // defaults to identity >> Collector c2, >> BiFunction finisher) { >> return toBoth(mapping(f1, c1), mapping(f2, c2)); >> } >> >> >>> "tee" might be a candidate, though it doesn't follow the `ing >> convention. "teeing" sounds dumb. >> >> >> "tee" sounds asymmetrical. "diverting" or "detouring" are "*ing" words >> that might >> express asymmetrical disposition of derivative streams. >> >> An asymmetrical operation might be interesting if it could fork off a >> stream of >> its own. It would have to have a side-effecting void-producing terminal >> operation, >> so the main (undiverted) stream could continue to progress at the top >> level of >> the expression. >> >> interface Stream { >> default Stream diverting(Consumer> tee) { ? } >> } >> >> values.stream().diverting(s2->s2.forEach(System.out:: >> println)).filter(?).collect(?); >> >> Or (and this might be a sweet spot) a symmetric stream-tee operation could >> materialize two sibling streams and rejoin their results with a bifunction: >> >> class Collectors { >> static Stream unzipping( >> Function, R1> f1, >> Function, R2> f2, >> BiFunction finisher) >> { ? } >> } >> >> values.stream().unzipping( >> s1->s1.forEach(System.out::println), >> s2->s2.filter(?).collect(?), >> (void1, r2)->r2 >> ); >> >> This would allow each "fork child" of the stream to continue to use the >> Stream API instead of the more restrictive Collector operators. >> >> Optimal code generation for forked/unzipped/teed streams would be tricky, >> requiring simultaneous loop control logic for each stream. >> To me that's a feature, not a bug, since hand-writing ad hoc >> simultaneous loops is a pain. >> >> My $0.02. >> >> ? John From brian.goetz at oracle.com Tue Jun 19 12:39:59 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 19 Jun 2018 08:39:59 -0400 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: <9a71cefd-698a-cd5d-f1ec-adebb10cb5e7@oracle.com> > We already have some toXxx() methods (toList, toSet, toCollection, > toMap, ...), so toBoth seems to me as a natural extension of that > naming principle: For toXxx methods, the Xxx is terminal, and tied to the result type.? toArray converts to an array; toList to a List; toCollection to a Collection.? This does not convert the elements to a Both, and there's no Both type.? So, I think the analogy falls apart. From brian.goetz at oracle.com Tue Jun 19 12:44:06 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 19 Jun 2018 08:44:06 -0400 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: > collectingToBoth This one is actually both evocative of what the method does, and in the spirit of the existing naming conventions (collectingAndThen.) An n-ary version could just be called `collectingTo`, where it is passed a varargs of Collector.? Could we get away with collectingTo for a binary version as well?? The existence of the "combiner" function might make that a stretch, but I prefer `collectingTo` to `collectingToBoth`. (I still like `distributing` too.) From david.lloyd at redhat.com Tue Jun 19 13:06:31 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 19 Jun 2018 08:06:31 -0500 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: It may be confusing (to some, I guess) but it is consistent: the ThreadLocal subclass author has absolute control over how the value is presented to the user. Adding compute() or many of the other suggested variants will break that guarantee, which seems like kind of a big deal to me. What about introducing a different thread local API that has more modern behavior? Maybe a new subclass of ThreadLocal? On Mon, Jun 18, 2018 at 5:28 PM Martin Buchholz wrote: > > yes, the proposed new methods would use nulls differently. The original get() behavior of creating a mapping was a mistake, inconsistent with Map. Yes, it will cause confusion. But it's already confusing. > > On Mon, Jun 18, 2018 at 1:45 PM, David Lloyd wrote: >> >> On Mon, Jun 18, 2018 at 12:53 PM, Martin Buchholz wrote: >> > On Mon, Jun 18, 2018 at 10:21 AM, Tony Printezis >> > wrote: >> > >> >> Martin, >> >> >> >> You are forgiven. :-) >> >> >> >> So, the (valid, I think) issue with getIfPresent(), as originally proposed >> >> by Peter, was that if get() was overridden in the subclass to somehow >> >> transform the returned value, getIfPresent() wouldn?t apply the same >> >> transformation. >> >> >> >> Doesn?t your compute() method have essentially the same issue? Apart from >> >> that, I personally like this proposal as I agree: one look-up is always >> >> better than two. >> >> >> >> >> > A non-prototype implementation would delegate compute into ThreadLocalMap >> > itself, where there is no risk of overriding. >> >> I don't think overriding is the risk; the risk would be compute* >> behaving inconsistently with regards to an overridden get*. >> >> >> -- >> - DML > > -- - DML From orionllmain at gmail.com Tue Jun 19 14:14:28 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Tue, 19 Jun 2018 21:14:28 +0700 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: I don't like `distributing` for the same reason as `bisecting`: for me, it sounds like a Stream is giving each collector only a part of elements. 2018-06-19 19:44 GMT+07:00 Brian Goetz : > > > collectingToBoth >> > > This one is actually both evocative of what the method does, and in the > spirit of the existing naming conventions (collectingAndThen.) > > An n-ary version could just be called `collectingTo`, where it is passed a > varargs of Collector. Could we get away with collectingTo for a binary > version as well? The existence of the "combiner" function might make that > a stretch, but I prefer `collectingTo` to `collectingToBoth`. > > > (I still like `distributing` too.) > From brian.goetz at oracle.com Tue Jun 19 14:47:23 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 19 Jun 2018 10:47:23 -0400 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: It is "distributing" in the same sense as the distributive law: ??? c*(a+b) = c*a + c*b (Think of the two collectors as the "sum" of a collector, and "distributing" says that you can send the elements to the sum by sending all of the elements to each.) That said, I agree that the less mathematically-inclined might be drawn to the plain-english meaning, which is more like an (imprecise) bisection. On 6/19/2018 10:14 AM, Zheka Kozlov wrote: > I don't like `distributing` for the same reason as `bisecting`: for > me, it sounds like a Stream is giving each collector only a part of > elements. > > 2018-06-19 19:44 GMT+07:00 Brian Goetz >: > > > > collectingToBoth > > > This one is actually both evocative of what the method does, and > in the spirit of the existing naming conventions (collectingAndThen.) > > An n-ary version could just be called `collectingTo`, where it is > passed a varargs of Collector.? Could we get away with > collectingTo for a binary version as well?? The existence of the > "combiner" function might make that a stretch, but I prefer > `collectingTo` to `collectingToBoth`. > > > (I still like `distributing` too.) > > From Roger.Riggs at Oracle.com Tue Jun 19 15:08:09 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 19 Jun 2018 11:08:09 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> Message-ID: <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> Hi Brent, On 6/18/2018 6:53 PM, Brent Christian wrote: > Hi, Roger > > On 6/18/18 7:31 AM, Roger Riggs wrote: >> >> The CSR and Webrev are updated. >> >> webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709 > > * src/java.base/share/classes/java/lang/System.java : > > Should the @implNote with the list of cached properties be added > everywhere the @apiNote is being added ?? Right now the @implNote is > only added to getProperties(). > The repetition was getting tiresome and the base of all the xxxProperties methods is getProperties. ?Joe suggested having one copy of the full information? and referring to that from the individual @apiNotes. > > * src/java.base/share/classes/jdk/internal/util/StaticProperty.java : > > Nit: > > ? 45???? private StaticProperty() { > ? 46 > ? 47???? } > > Maybe put this all on one line? > Will do Thanks, Roger > > Otherwise, the change looks good to me. > > -Brent From Roger.Riggs at Oracle.com Tue Jun 19 15:14:32 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 19 Jun 2018 11:14:32 -0400 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> Message-ID: Hi Vivek, Do you need to concerned that the arrays are zero length? Are there any tests that test the zero length case. (I assume you ran the existing tests). Thanks, Roger On 6/18/2018 7:52 PM, Deshpande, Vivek R wrote: > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' ; Viswanathan, Sandhya > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > From brent.christian at oracle.com Tue Jun 19 15:52:37 2018 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 19 Jun 2018 08:52:37 -0700 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> Message-ID: <5270e866-014c-497b-60db-855338bd7d8b@oracle.com> On 6/19/18 8:08 AM, Roger Riggs wrote: >> >> * src/java.base/share/classes/java/lang/System.java : >> >> Should the @implNote with the list of cached properties be added >> everywhere the @apiNote is being added ?? Right now the @implNote is >> only added to getProperties(). >> > The repetition was getting tiresome and the base of all the > xxxProperties methods is getProperties. > ?Joe suggested having one copy of the full information? and referring > to that from the individual @apiNotes. Fair enough. >> * src/java.base/share/classes/jdk/internal/util/StaticProperty.java : >> >> ? 45???? private StaticProperty() { >> ? 46 >> ? 47???? } >> >> Maybe put this all on one line? >> > Will do Thanks, -Brent From vivek.r.deshpande at intel.com Tue Jun 19 16:36:25 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Tue, 19 Jun 2018 16:36:25 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> Thanks David. Sending it to core-libs-dev. I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Could you please review and sponsor the patch. Link to bug: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ Regards, Vivek -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Monday, June 18, 2018 10:32 PM To: Deshpande, Vivek R ; jdk-dev at openjdk.java.net Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. Thanks, David On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' ; Viswanathan, Sandhya > > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > From paul.sandoz at oracle.com Tue Jun 19 16:42:01 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 19 Jun 2018 09:42:01 -0700 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: Gosh, this is a tricky one to name. collectingTo seems the best so far, although collect(collectingTo(?)) ... One last suggestion from me, ?expanding?, as in the collector expands the number of collectors the input elements are applied to. Paul. > On Jun 19, 2018, at 7:47 AM, Brian Goetz wrote: > > It is "distributing" in the same sense as the distributive law: > > c*(a+b) = c*a + c*b > > (Think of the two collectors as the "sum" of a collector, and "distributing" says that you can send the elements to the sum by sending all of the elements to each.) > > That said, I agree that the less mathematically-inclined might be drawn to the plain-english meaning, which is more like an (imprecise) bisection. > > On 6/19/2018 10:14 AM, Zheka Kozlov wrote: >> I don't like `distributing` for the same reason as `bisecting`: for me, it sounds like a Stream is giving each collector only a part of elements. >> >> 2018-06-19 19:44 GMT+07:00 Brian Goetz >: >> >> >> >> collectingToBoth >> >> >> This one is actually both evocative of what the method does, and >> in the spirit of the existing naming conventions (collectingAndThen.) >> >> An n-ary version could just be called `collectingTo`, where it is >> passed a varargs of Collector. Could we get away with >> collectingTo for a binary version as well? The existence of the >> "combiner" function might make that a stretch, but I prefer >> `collectingTo` to `collectingToBoth`. >> >> >> (I still like `distributing` too.) >> >> > From paul.sandoz at oracle.com Tue Jun 19 16:54:57 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 19 Jun 2018 09:54:57 -0700 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> Message-ID: <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> Hi Vivek, Thanks for investigating this. 164 public static int mismatch(boolean[] a, 165 boolean[] b, 166 int length) { 167 int i = 0; 168 if (a[i] != b[i]) 169 return i; You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. 186 public static int mismatch(boolean[] a, int aFromIndex, 187 boolean[] b, int bFromIndex, 188 int length) { 189 int i = 0; 190 if (a[i] != b[i]) 191 return i; This is incorrect. You need to use aFromIndex and bFromIndex. Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. Paul. > On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R wrote: > > Thanks David. > Sending it to core-libs-dev. > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Could you please review and sponsor the patch. > Link to bug: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ > > Regards, > Vivek > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Monday, June 18, 2018 10:32 PM > To: Deshpande, Vivek R ; jdk-dev at openjdk.java.net > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. > > Thanks, > David > > On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: >> Hi All >> >> Forgot to add the links: >> https://bugs.openjdk.java.net/browse/JDK-8205194 >> webrev: >> http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 >> 0/ >> >> Regards. >> Vivek >> >> From: Deshpande, Vivek R >> Sent: Monday, June 18, 2018 4:50 PM >> To: 'jdk-dev at openjdk.java.net' >> Cc: 'Paul Sandoz' ; Viswanathan, Sandhya >> >> Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. >> >> Hi All >> >> I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. >> This avoids call to vectorizedMismatch method and gives ~80x speed up. >> Please review and sponsor the patch. >> >> Regards, >> Vivek >> From peter.levart at gmail.com Tue Jun 19 17:01:30 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 19 Jun 2018 19:01:30 +0200 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: Hi Stuart, On 06/19/2018 04:09 AM, Stuart Marks wrote: > > > On 6/17/18 4:23 PM, Peter Levart wrote: >> My attempt to optimize the Map.copyOf() was also using just public >> APIs (with the addition of an internal class). > >> Well, it does speed-up Map.copyOf() by 30%. But I agree there are >> other approaches that could go even further. In particular when all >> intermediary copies could be avoided. >> >> Here's one approach (which still uses just public APIs) and avoids >> intermediary copying: > > You mentioned "using just public APIs" a couple times. Are you viewing > that as a constraint? I don't think it is. More a desire than a constraint. If there really is a solution that is better but needs SharedSecrets, then I'm all for it. OTOH, it surely is less maintenance when only public APIs are used, isn't it? I'd like to use Map.forEach() which is already optimized in most JDK internal implementations while the default implementation is not worse than alternatives. > >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.02/ >> >> >> Why choosing Map.forEach for dumping the content of the map instead >> of iteration over entrySet? Because in some Map implementations this >> is the most direct iteration without producing garbage (for example >> in IdentityHashMap), but in the worst case (default Map method for >> example) it is delegated to iteration over entrySet. > > Sure, entrySet().toArray() does a lot of allocation and copying. Using > forEach() can avoid this, but it doesn't necessarily preserve the > right invariants. In particular, if the source is a concurrent map, > the number of times forEach's BiConsumer is called might not match > size(), if the map has changed size. So, the forEach approach has to > deal with the possibility of it not being called exactly size() times, > whereas we know that the array from toArray() can't change size (and > that its contents are consistent, for some definition of consistent). Right, and my last approach detected that and has thrown ConcurrentModificationException. Which might not be desired in a scenario where a live CHM is copied into an unmodifiable Map for example. > > This change also publishes a reference to the Map under construction > before its constructor has returned. I'm not sure of all the memory > model implications of this. It publishes a Builder object that has a controlled access to the constructing map. You can't get a reference to the constructing map from it and it becomes useless before the constructor of the map returns. > > This change also hands out an object that has access to the new Map > instance's internals. Yes, this is the Builder object. > You're aware of this, of course, because you snapshot the current > thread and null it out later, saying > > ??? // prevent naughty Map implementations from modifying MapN after > ??? // it is fully constructed > > So there are potential security vulnerabilities here, which requires > some serious thought. What you have *seems* reasonable, but my > experience is that things that are subtle but that seem reasonable > actually end up having security holes. This is not my invention. The same technique is used in ObjectInputStream / ObjectOutputStream. > > I'm trying to think of some alternatives. > > The issue with the forEach() approach on an arbitrary map is that you > have to hand out a BiConsumer, and malicious code can call it even > after forEach() has returned. What's necessary is to have a way to > shut off the BiConsumer before reading out what it's accumulated. I > don't know of a simple, fast, and correct way to do this. (Choose any > two!) The "enabled" state (whether a Thread instance or just a > boolean) will probably have to be a volatile that must be checked > every time. What are the performance implications? Anyway, while it > seems promising, the forEach() approach seems like it leads off into > the weeds. The technique used in my last webrev and taken from ObjectInputStream / ObjectOutputStream is simple, simple to prove correctness and fast as it needs zero synchronization. It could be called "a single-threaded object" idiom: class SingleThreadedObject { ??? private Thread thread; ??? SingleThreadedObject() { ??? ??? thread = Thread.currentThread(); ??? } ??? void use() { ??? ??? if (Thread.currentThread() != thread) throw new IllegalStateException(); ??? ??? ... anything ... ??? } ??? void close() { ??? ??? if (Thread.currentThread() != thread) throw new IllegalStateException(); ??? ??? thread = null; ??? } } Claim: The thread that constructs the object is the only thread that can use() or close() the object and only until close() is called by the constructing thread. To prove the correctness of this claim, we have consider two scenarios: - object is use()d and finally close()d in the same thread that constructed it - no need to prove anything here as it is obvious that this works by inspecting the code. - object is attempted to be use()d or close()d in some other thread from the thread that constructed it. To prove that such usage is always prevented, we have to ask ourselves what are the possible values of 'thread' field that can be seen by some other thread. There are two possibilities: - null value (uninitialized or already close()d) -> the usage is prevented as null != Thread.currentThread() - non-null value (initialized) -> the usage is prevented as the non-null value is a different thread than current thread which is trying to use() or close() the object This technique doesn't use any synchronization. Thread.currentThread() is fast (I think I heard that current thread reference or something not far from it is kept in a CPU register all the time). > > Unless you can verify the trustworthiness of the source. Suppose you > check the source map to see if its getClass() == HashMap.class. Then > you can be reasonably assured that forEach() won't misuse the > BiConsumer. You can probably also rely on size() being stable. > (Probably also include LinkedHashMap.) > > The toUnmodifiableMap collectors can benefit from this if they're > changed to use Map::copyOf, as in your current webrev. > > You could also provide a similar fast path for ConcurrentHashMap. It'd > have to be resilient against size changes, but you can trust that it > won't misuse the BiConsumer. > > Finally, other map implementations will just have to suffer through > the multi-copy slow path. Not necessarily. Here's another attempt that uses the same technique but relaxes the possible concurrent source map scenario (where number of pairs emitted by Map.forEach() doesn't agree with Map.size()): http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.03/ The construction is moved entirely into a separate MapBuilder object. The builder is used for Map.of(...) methods too. Here are the benchmark results that show improvements in every respect touched by the patch: http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench_results.txt Here are the JMH benchmarks used to produce these results: http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapCollectorBench.java http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapCopyOfBench.java http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapOfBench.java So what do you think of this one? Regards, Peter From vivek.r.deshpande at intel.com Tue Jun 19 17:16:43 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Tue, 19 Jun 2018 17:16:43 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A90748E08@ORSMSX106.amr.corp.intel.com> Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. Regards, Vivek From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Tuesday, June 19, 2018 9:55 AM To: Deshpande, Vivek R Cc: David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Thanks for investigating this. 164 public static int mismatch(boolean[] a, 165 boolean[] b, 166 int length) { 167 int i = 0; 168 if (a[i] != b[i]) 169 return i; You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. 186 public static int mismatch(boolean[] a, int aFromIndex, 187 boolean[] b, int bFromIndex, 188 int length) { 189 int i = 0; 190 if (a[i] != b[i]) 191 return i; This is incorrect. You need to use aFromIndex and bFromIndex. Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. Paul. On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: Thanks David. Sending it to core-libs-dev. I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Could you please review and sponsor the patch. Link to bug: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ Regards, Vivek -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Monday, June 18, 2018 10:32 PM To: Deshpande, Vivek R >; jdk-dev at openjdk.java.net Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. Thanks, David On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: Hi All Forgot to add the links: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 0/ Regards. Vivek From: Deshpande, Vivek R Sent: Monday, June 18, 2018 4:50 PM To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' >; Viswanathan, Sandhya > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi All I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Please review and sponsor the patch. Regards, Vivek From vivek.r.deshpande at intel.com Tue Jun 19 17:57:22 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Tue, 19 Jun 2018 17:57:22 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> Hi Roger I will also test with zero length arrays and let you know. Thanks for the input. Regards, Vivek From: Deshpande, Vivek R Sent: Tuesday, June 19, 2018 10:17 AM To: 'Paul Sandoz' Cc: David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. Regards, Vivek From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Tuesday, June 19, 2018 9:55 AM To: Deshpande, Vivek R > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Thanks for investigating this. 164 public static int mismatch(boolean[] a, 165 boolean[] b, 166 int length) { 167 int i = 0; 168 if (a[i] != b[i]) 169 return i; You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. 186 public static int mismatch(boolean[] a, int aFromIndex, 187 boolean[] b, int bFromIndex, 188 int length) { 189 int i = 0; 190 if (a[i] != b[i]) 191 return i; This is incorrect. You need to use aFromIndex and bFromIndex. Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. Paul. On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: Thanks David. Sending it to core-libs-dev. I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Could you please review and sponsor the patch. Link to bug: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ Regards, Vivek -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Monday, June 18, 2018 10:32 PM To: Deshpande, Vivek R >; jdk-dev at openjdk.java.net Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. Thanks, David On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: Hi All Forgot to add the links: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 0/ Regards. Vivek From: Deshpande, Vivek R Sent: Monday, June 18, 2018 4:50 PM To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' >; Viswanathan, Sandhya > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi All I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Please review and sponsor the patch. Regards, Vivek From tprintezis at twitter.com Tue Jun 19 19:31:19 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 19 Jun 2018 12:31:19 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: Good idea, as long as it re-uses the existing ThreadLocal infrastructure and doesn?t introduce an extra per-thread map. Making it a ThreadLocal subclass would be an excellent start. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 19, 2018 at 9:07:07 AM, David Lloyd (david.lloyd at redhat.com) wrote: It may be confusing (to some, I guess) but it is consistent: the ThreadLocal subclass author has absolute control over how the value is presented to the user. Adding compute() or many of the other suggested variants will break that guarantee, which seems like kind of a big deal to me. What about introducing a different thread local API that has more modern behavior? Maybe a new subclass of ThreadLocal? On Mon, Jun 18, 2018 at 5:28 PM Martin Buchholz wrote: > > yes, the proposed new methods would use nulls differently. The original get() behavior of creating a mapping was a mistake, inconsistent with Map. Yes, it will cause confusion. But it's already confusing. > > On Mon, Jun 18, 2018 at 1:45 PM, David Lloyd wrote: >> >> On Mon, Jun 18, 2018 at 12:53 PM, Martin Buchholz wrote: >> > On Mon, Jun 18, 2018 at 10:21 AM, Tony Printezis < tprintezis at twitter.com> >> > wrote: >> > >> >> Martin, >> >> >> >> You are forgiven. :-) >> >> >> >> So, the (valid, I think) issue with getIfPresent(), as originally proposed >> >> by Peter, was that if get() was overridden in the subclass to somehow >> >> transform the returned value, getIfPresent() wouldn?t apply the same >> >> transformation. >> >> >> >> Doesn?t your compute() method have essentially the same issue? Apart from >> >> that, I personally like this proposal as I agree: one look-up is always >> >> better than two. >> >> >> >> >> > A non-prototype implementation would delegate compute into ThreadLocalMap >> > itself, where there is no risk of overriding. >> >> I don't think overriding is the risk; the risk would be compute* >> behaving inconsistently with regards to an overridden get*. >> >> >> -- >> - DML > > -- - DML From victorwssilva at gmail.com Tue Jun 19 22:47:41 2018 From: victorwssilva at gmail.com (Victor Williams Stafusa da Silva) Date: Tue, 19 Jun 2018 19:47:41 -0300 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> <8ba042c8-8697-d8fa-2632-aa2e2c9bc7f6@oracle.com> Message-ID: What about forking? 2018-06-19 13:42 GMT-03:00 Paul Sandoz : > Gosh, this is a tricky one to name. > > collectingTo seems the best so far, although collect(collectingTo(?)) ... > > One last suggestion from me, ?expanding?, as in the collector expands the > number of collectors the input elements are applied to. > > Paul. > > > > > On Jun 19, 2018, at 7:47 AM, Brian Goetz wrote: > > > > It is "distributing" in the same sense as the distributive law: > > > > c*(a+b) = c*a + c*b > > > > (Think of the two collectors as the "sum" of a collector, and > "distributing" says that you can send the elements to the sum by sending > all of the elements to each.) > > > > That said, I agree that the less mathematically-inclined might be drawn > to the plain-english meaning, which is more like an (imprecise) bisection. > > > > On 6/19/2018 10:14 AM, Zheka Kozlov wrote: > >> I don't like `distributing` for the same reason as `bisecting`: for me, > it sounds like a Stream is giving each collector only a part of elements. > >> > >> 2018-06-19 19:44 GMT+07:00 Brian Goetz brian.goetz at oracle.com>>: > >> > >> > >> > >> collectingToBoth > >> > >> > >> This one is actually both evocative of what the method does, and > >> in the spirit of the existing naming conventions (collectingAndThen.) > >> > >> An n-ary version could just be called `collectingTo`, where it is > >> passed a varargs of Collector. Could we get away with > >> collectingTo for a binary version as well? The existence of the > >> "combiner" function might make that a stretch, but I prefer > >> `collectingTo` to `collectingToBoth`. > >> > >> > >> (I still like `distributing` too.) > >> > >> > > > > From david.holmes at oracle.com Tue Jun 19 23:30:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jun 2018 09:30:23 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> Message-ID: Sorry another update is imminent ... stay tuned. David On 19/06/2018 2:41 PM, David Holmes wrote: > Discussions on the CSR request have led to further changes to the > documentation involving nests and the new nest-related method. There are > no semantic changes here just clearer explanations. > > Incremental webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ > > (don't worry if you don't see a v6, it didn't really exist). > > Full webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v7/ > > Specdiffs updated in place at: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/ > > Summary of changes: > > - The definition of a nest etc is moved to the class-level javadoc of > java.lang.Class, along with some other edits provided by Alex Buckley to > pave the way for future updates > - The nest-related methods are written in a more clear and consistent way > > Thanks, > David > ----- > > On 12/06/2018 3:16 PM, David Holmes wrote: >> Here is one further minor update from the CSR discussions: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html >> >> >> Thanks, >> David >> >> On 25/05/2018 3:52 PM, David Holmes wrote: >>> Here are the further minor updates so far in response to all the >>> review comments. >>> >>> Incremental corelibs webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ >>> >>> >>> Full corelibs webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ >>> >>> Change summary: >>> >>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>> - remove inaccurate pseudo-assertion comment >>> >>> test/jdk/java/lang/reflect/Nestmates/SampleNest.java >>> - code cleanup: <> operator >>> >>> test/jdk/java/lang/reflect/Nestmates/TestReflectionAPI.java >>> - code cleanup: streamify duplicate removals >>> >>> test/jdk/java/lang/invoke/PrivateInterfaceCall.java >>> - use consistent @bug number >>> >>> Thanks, >>> David >>> >>> On 22/05/2018 8:15 PM, David Holmes wrote: >>>> Here are the updates so far in response to all the review comments. >>>> >>>> Incremental webrev: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2-incr/ >>>> >>>> >>>> Full webrev: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2/ >>>> >>>> Change summary: >>>> >>>> src/java.base/share/classes/java/lang/Class.java >>>> - getNesthost: >>>> ?? - change "any error" -> "any linkage error" as runtime errors >>>> will propagate. [This needs ratifying by EG] >>>> ?? - add clarification that primitive and array classes are not >>>> explicitly members of any nest and so form singleton nests >>>> ?? - add clarification that all nestmates are in the same package >>>> ?? - re-word @return text to exclude the "royal 'we'" >>>> - fix javadoc cross references >>>> >>>> --- >>>> >>>> Moved reflection API tests from >>>> test/hotspot/jtreg/runtime/Nestmates/reflectionAPI/ to >>>> test/jdk/java/lang/reflect/Nestmates/ >>>> >>>> --- >>>> >>>> java/lang/reflect/Nestmates/TestReflectionAPI.java >>>> >>>> Run tests twice to show that failure reasons remain the same. >>>> >>>> --- >>>> >>>> test/jdk/jdk/lambda/vm/InterfaceAccessFlagsTest.java >>>> >>>> Disable test via annotation rather than commenting out. >>>> >>>> --- >>>> >>>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>>> >>>> - Fix indent for nestmate access check. >>>> - Remove unnecessary local variable >>>> >>>> --- >>>> >>>> src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >>>> >>>> - Replace myassert with proper assert >>>> >>>> --- >>>> >>>> Thanks, >>>> David >>>> >>>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>>> This review is being spread across four groups: langtools, >>>>> core-libs, hotspot and serviceability. This is the specific review >>>>> thread for core-libs - webrev: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v1/ >>>>> >>>>> See below for full details - including annotated full webrev >>>>> guiding the review. >>>>> >>>>> The intent is to have JEP-181 targeted and integrated by the end of >>>>> this month. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> The nestmates project (JEP-181) introduces new classfile attributes >>>>> to identify classes and interfaces in the same nest, so that the VM >>>>> can perform access control based on those attributes and so allow >>>>> direct private access between nestmates without requiring javac to >>>>> generate synthetic accessor methods. These access control changes >>>>> also extend to core reflection and the MethodHandle.Lookup contexts. >>>>> >>>>> Direct private calls between nestmates requires a more general >>>>> calling context than is permitted by invokespecial, and so the JVMS >>>>> is updated to allow, and javac updated to use, invokevirtual and >>>>> invokeinterface for private class and interface method calls >>>>> respectively. These changed semantics also extend to MethodHandle >>>>> findXXX operations. >>>>> >>>>> At this time we are only concerned with static nest definitions, >>>>> which map to a top-level class/interface as the nest-host and all >>>>> its nested types as nest-members. >>>>> >>>>> Please see the JEP for further details. >>>>> >>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>> >>>>> All of the specification changes have been previously been worked >>>>> out by the Valhalla Project Expert Group, and the implementation >>>>> reviewed by the various contributors and discussed on the >>>>> valhalla-dev mailing list. >>>>> >>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>>> Kumar Srinivasan >>>>> >>>>> Master webrev of all changes: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>> >>>>> Annotated master webrev index: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>> >>>>> Performance: this is expected to be performance neutral in a >>>>> general sense. Benchmarking and performance runs are about to start. >>>>> >>>>> Testing Discussion: >>>>> ------------------ >>>>> >>>>> The testing for nestmates can be broken into four main groups: >>>>> >>>>> -? New tests specifically related to nestmates and currently in the >>>>> runtime/Nestmates directory >>>>> >>>>> - New tests to complement existing tests by adding in testcases not >>>>> previously expressible. >>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>>> use of invokespecial for private interface methods and performing >>>>> receiver typechecks, so we add >>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>>> invokeinterface. >>>>> >>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>> attributes. >>>>> >>>>> -? Existing tests significantly affected by the nestmates changes, >>>>> primarily: >>>>> ??? -? runtime/SelectionResolution >>>>> >>>>> ??? In most cases the nestmate changes makes certain invocations >>>>> that were illegal, legal (e.g. not requiring invokespecial to >>>>> invoke private interface methods; allowing access to private >>>>> members via reflection/Methodhandles that were previously not >>>>> allowed). >>>>> >>>>> - Existing tests incidentally affected by the nestmate changes >>>>> >>>>> ?? This includes tests of things utilising class >>>>> redefinition/retransformation to alter nested types but which >>>>> unintentionally alter nest relationships (which is not permitted). >>>>> >>>>> There are still a number of tests problem-listed with issues filed >>>>> against them to have them adapted to work with nestmates. Some of >>>>> these are intended to be addressed in the short-term, while some >>>>> (such as the runtime/SelectionResolution test changes) may not >>>>> eventuate. >>>>> >>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>> >>>>> There is also further test work still to be completed (the JNI and >>>>> JDI invocation tests): >>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>> which will continue in parallel with the main RFR. >>>>> >>>>> Pre-integration Testing: >>>>> ??- General: >>>>> ???? - Mach5: hs/jdk tier1,2 >>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>> ??- Targetted >>>>> ??? - nashorn (for asm changes) >>>>> ??? - hotspot: runtime/* >>>>> ?????????????? serviceability/* >>>>> ?????????????? compiler/* >>>>> ?????????????? vmTestbase/* >>>>> ??? - jdk: java/lang/invoke/* >>>>> ?????????? java/lang/reflect/* >>>>> ?????????? java/lang/instrument/* >>>>> ?????????? java/lang/Class/* >>>>> ?????????? java/lang/management/* >>>>> ?? - langtools: tools/javac >>>>> ??????????????? tools/javap >>>>> From paul.sandoz at oracle.com Wed Jun 20 00:08:30 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 19 Jun 2018 17:08:30 -0700 Subject: RFR 8195650 Method references to VarHandle accessors Message-ID: <086A1684-4D97-4B6D-94F2-16A1261057B5@oracle.com> Hi, Please review the following fix to ensure method references to VarHandle signature polymorphic methods are supported at runtime (specifically the method handle to a signature polymorphic method can be loaded from the constant pool): http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195650-varhandle-mref/webrev/ I also added a ?belts and braces? test to ensure a constant method handle to MethodHandle::invokeBasic cannot be loaded if outside of the j.l.invoke package. Paul. From gromero at linux.vnet.ibm.com Wed Jun 20 00:59:47 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 19 Jun 2018 21:59:47 -0300 Subject: RFC: Add new JCA provider to support hardware RNGs Message-ID: <993c258a-57ae-d29d-2368-faac5b4d3cb1@linux.vnet.ibm.com> Hi, Please, could I get comments on the following change? Since it's related to security, I would be glad if security experts could also comment on that. webrev: http://cr.openjdk.java.net/~gromero/POWER9/darn/v6_rebased/ It introduces a way to get benefits from hardware instructions in userspace that can delivery a random number and so be used to speed up and avoid blocking in some SecureRandom methods, notably generateSeed() and nextBytes(), without loosing the random number quality. Particularly on Power, the new POWER9 CPUs introduced a hardware instruction called 'darn' that can be used for that purpose. The main idea is to add a new JCA provider called "HWTRNG" (if no better name is suggested), adding new helper methods that can then be intrinsified and that will be used by generateSeed() and nextBytes(). In that sense, this change also adds the proper mechanisms in the Interpreter, C1 Compiler, and C2 compiler (for PPC64, but also paving the way for other platforms) to intrinsify these helper methods that will be used in the end by generateSeed() and nextBytes() methods. The added helpers are: byte[] getRandomSequence0(int) byte[] getRandomSequence1(byte[]) long validRandomLong(void) long randomLong(void) The helpers also take care of checking if the returned random number is valid and attempt 10 times (as recommended by ISA) get a valid random number before falling back to a software alternative (HWTRNG is based mostly on JCA provider NativePRNG and so the fall back mechanism will use the exactly same methods found in NativePRNG and hence behave similarly. Nonetheless, in my experience such a hardware failures (which are the main cause of a returned invalid random number) are quite rare: in my tests I was never able to exhaust the HW RNG and force it to generate an invalid random number). The user, besides having to specify explicitly the use of a non-default provider (HWTRNG), will have to unlock the VM experimental options to effectively use the hardware RNG as an unique random source by passing options "-XX:+UnlockExperimentalVMOptions -XX:+UseRANDOMIntrinsics". On Power, for the Interpreter and C1 Compiler, besides avoiding the blocking effect of a low entropy on /dev/random, I was able to get about 126 Mbps (3x higher than the version without 'darn') on generaSeed() and nextBytes(). The maximum theoretical value on a POWER9 machine is usually 128 Mbps. I'll address the details about the file headers (Copyright, dates, authors, versions, etc) after I get some feedback about this change. Thanks in advance. Best regards, Gustavo From gromero at linux.vnet.ibm.com Wed Jun 20 01:13:02 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 19 Jun 2018 22:13:02 -0300 Subject: RFC: Add new JCA provider to support hardware RNGs In-Reply-To: <993c258a-57ae-d29d-2368-faac5b4d3cb1@linux.vnet.ibm.com> References: <993c258a-57ae-d29d-2368-faac5b4d3cb1@linux.vnet.ibm.com> Message-ID: Sorry for resending it. I missed a few MLs. -- Hi, Please, could I get comments on the following change? Since it's related to security, I would be glad if security experts could also comment on that. webrev: http://cr.openjdk.java.net/~gromero/POWER9/darn/v6_rebased/ It introduces a way to get benefits from hardware instructions in userspace that can delivery a random number and so be used to speed up and avoid blocking in some SecureRandom methods, notably generateSeed() and nextBytes(), without loosing the random number quality. Particularly on Power, the new POWER9 CPUs introduced a hardware instruction called 'darn' that can be used for that purpose. The main idea is to add a new JCA provider called "HWTRNG" (if no better name is suggested), adding new helper methods that can then be intrinsified and that will be used by generateSeed() and nextBytes(). In that sense, this change also adds the proper mechanisms in the Interpreter, C1 Compiler, and C2 compiler (for PPC64, but also paving the way for other platforms) to intrinsify these helper methods that will be used in the end by generateSeed() and nextBytes() methods. The added helpers are: byte[] getRandomSequence0(int) byte[] getRandomSequence1(byte[]) long validRandomLong(void) long randomLong(void) The helpers also take care of checking if the returned random number is valid and attempt 10 times (as recommended by ISA) get a valid random number before falling back to a software alternative (HWTRNG is based mostly on JCA provider NativePRNG and so the fall back mechanism will use the exactly same methods found in NativePRNG and hence behave similarly. Nonetheless, in my experience such a hardware failures (which are the main cause of a returned invalid random number) are quite rare: in my tests I was never able to exhaust the HW RNG and force it to generate an invalid random number). The user, besides having to specify explicitly the use of a non-default provider (HWTRNG), will have to unlock the VM experimental options to effectively use the hardware RNG as an unique random source by passing options "-XX:+UnlockExperimentalVMOptions -XX:+UseRANDOMIntrinsics". On Power, for the Interpreter and C1 Compiler, besides avoiding the blocking effect of a low entropy on /dev/random, I was able to get about 126 Mbps (3x higher than the version without 'darn') on generaSeed() and nextBytes(). The maximum theoretical value on a POWER9 machine is usually 128 Mbps. I'll address the details about the file headers (Copyright, dates, authors, versions, etc) after I get some feedback about this change. Thanks in advance. Best regards, Gustavo From huizhe.wang at oracle.com Wed Jun 20 03:32:45 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 19 Jun 2018 20:32:45 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> Message-ID: <5B29CADD.7090200@oracle.com> Thanks Alan. I created 8205058 to capture your suggestions. Please see below for more details. On 6/14/18, 4:30 AM, Alan Bateman wrote: > On 12/06/2018 17:52, Joe Wang wrote: >> Hi all, >> >> It's been a while since the 1st round of reviews. Thank you all for >> the valuable comments and suggestions! Note that Roger's last >> response went to nio-dev, but not core-libs-dev, I've therefore >> attached it below. >> >> CSR [2]: the CSR is now approved. Note the write method has been >> changed to writeString. >> >> Impl [3]: for performance reason and the different requirement from >> byte-string conversion for malformed/unmappable character handing, >> the implementation uses a specific method separate from the existing >> ones. Both Sherman and Alan preferred specific method than adding >> parameters to the APIs that might be error prone. Thanks Sherman for >> the code! >> >> JMH benchmark tests showed the new read method performs better than >> an operation that reads the bytes and convert to string. The gains >> start to be significant even at quite reasonable sizes (< 10K). >> >> >> [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 >> [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 >> [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ >> >> Please review. > The javadoc looks good. > > One part of the implementation that could be cleaner is the exception > thrown for the malformed or unmappable cases. There are sub-classes of > CharacterCodingException defined for this that would be clearer than > an IOException with an IllegalArgumentException as cause. Changed the internal APIs to throw CCE instead. In the same way as the previous changeset for 8201276, these methods are made specific for the use cases (though they are now for Files.read/writeString only) so that they are not mixed up with existing ones that may inadvertently affect other usages. One thing to note is that MalformedInputException or UnmappableCharacterException will lose one piece of information in comparison to the existing IAE, that is where it happens (offset). Should there be an improvement in the future, we could consider add it back to this part of code. > > A minor nit in Files is that you can import > jdk.internal.misc.JavaLangAccess rather than repeating the fully > qualified class name. You can also move the definition of JLA up to > the top. There's also a few inconsistencies with the existing code > that would be goof to fix too (indenting and line length issues mostly). Moved JLA and others to the top. Fixed also the indentations (mostly method declarations) and a few long lines. > > The test looks reasonable. In getData() then then "shouldn't happen" > case should throw an exception as a NPE here might be tricky to > diagnose there. Another nit is the sb field - can that be removed. Fixed too. JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ Regards, Joe > > -Alan > From david.holmes at oracle.com Wed Jun 20 04:56:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jun 2018 14:56:43 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> Message-ID: Some further adjustments to getNestMembers() was made. Everything updated in place. Thanks, David On 20/06/2018 9:30 AM, David Holmes wrote: > Sorry another update is imminent ... stay tuned. > > David > > On 19/06/2018 2:41 PM, David Holmes wrote: >> Discussions on the CSR request have led to further changes to the >> documentation involving nests and the new nest-related method. There >> are no semantic changes here just clearer explanations. >> >> Incremental webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ >> >> >> (don't worry if you don't see a v6, it didn't really exist). >> >> Full webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v7/ >> >> Specdiffs updated in place at: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/ >> >> Summary of changes: >> >> - The definition of a nest etc is moved to the class-level javadoc of >> java.lang.Class, along with some other edits provided by Alex Buckley >> to pave the way for future updates >> - The nest-related methods are written in a more clear and consistent way >> >> Thanks, >> David >> ----- >> >> On 12/06/2018 3:16 PM, David Holmes wrote: >>> Here is one further minor update from the CSR discussions: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html >>> >>> >>> Thanks, >>> David >>> >>> On 25/05/2018 3:52 PM, David Holmes wrote: >>>> Here are the further minor updates so far in response to all the >>>> review comments. >>>> >>>> Incremental corelibs webrev: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ >>>> >>>> >>>> Full corelibs webrev: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ >>>> >>>> Change summary: >>>> >>>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>>> - remove inaccurate pseudo-assertion comment >>>> >>>> test/jdk/java/lang/reflect/Nestmates/SampleNest.java >>>> - code cleanup: <> operator >>>> >>>> test/jdk/java/lang/reflect/Nestmates/TestReflectionAPI.java >>>> - code cleanup: streamify duplicate removals >>>> >>>> test/jdk/java/lang/invoke/PrivateInterfaceCall.java >>>> - use consistent @bug number >>>> >>>> Thanks, >>>> David >>>> >>>> On 22/05/2018 8:15 PM, David Holmes wrote: >>>>> Here are the updates so far in response to all the review comments. >>>>> >>>>> Incremental webrev: >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2-incr/ >>>>> >>>>> >>>>> Full webrev: >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2/ >>>>> >>>>> Change summary: >>>>> >>>>> src/java.base/share/classes/java/lang/Class.java >>>>> - getNesthost: >>>>> ?? - change "any error" -> "any linkage error" as runtime errors >>>>> will propagate. [This needs ratifying by EG] >>>>> ?? - add clarification that primitive and array classes are not >>>>> explicitly members of any nest and so form singleton nests >>>>> ?? - add clarification that all nestmates are in the same package >>>>> ?? - re-word @return text to exclude the "royal 'we'" >>>>> - fix javadoc cross references >>>>> >>>>> --- >>>>> >>>>> Moved reflection API tests from >>>>> test/hotspot/jtreg/runtime/Nestmates/reflectionAPI/ to >>>>> test/jdk/java/lang/reflect/Nestmates/ >>>>> >>>>> --- >>>>> >>>>> java/lang/reflect/Nestmates/TestReflectionAPI.java >>>>> >>>>> Run tests twice to show that failure reasons remain the same. >>>>> >>>>> --- >>>>> >>>>> test/jdk/jdk/lambda/vm/InterfaceAccessFlagsTest.java >>>>> >>>>> Disable test via annotation rather than commenting out. >>>>> >>>>> --- >>>>> >>>>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>>>> >>>>> - Fix indent for nestmate access check. >>>>> - Remove unnecessary local variable >>>>> >>>>> --- >>>>> >>>>> src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >>>>> >>>>> - Replace myassert with proper assert >>>>> >>>>> --- >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>>>> This review is being spread across four groups: langtools, >>>>>> core-libs, hotspot and serviceability. This is the specific review >>>>>> thread for core-libs - webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v1/ >>>>>> >>>>>> >>>>>> See below for full details - including annotated full webrev >>>>>> guiding the review. >>>>>> >>>>>> The intent is to have JEP-181 targeted and integrated by the end >>>>>> of this month. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>> attributes to identify classes and interfaces in the same nest, so >>>>>> that the VM can perform access control based on those attributes >>>>>> and so allow direct private access between nestmates without >>>>>> requiring javac to generate synthetic accessor methods. These >>>>>> access control changes also extend to core reflection and the >>>>>> MethodHandle.Lookup contexts. >>>>>> >>>>>> Direct private calls between nestmates requires a more general >>>>>> calling context than is permitted by invokespecial, and so the >>>>>> JVMS is updated to allow, and javac updated to use, invokevirtual >>>>>> and invokeinterface for private class and interface method calls >>>>>> respectively. These changed semantics also extend to MethodHandle >>>>>> findXXX operations. >>>>>> >>>>>> At this time we are only concerned with static nest definitions, >>>>>> which map to a top-level class/interface as the nest-host and all >>>>>> its nested types as nest-members. >>>>>> >>>>>> Please see the JEP for further details. >>>>>> >>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>> >>>>>> All of the specification changes have been previously been worked >>>>>> out by the Valhalla Project Expert Group, and the implementation >>>>>> reviewed by the various contributors and discussed on the >>>>>> valhalla-dev mailing list. >>>>>> >>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>>>> Kumar Srinivasan >>>>>> >>>>>> Master webrev of all changes: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>> >>>>>> Annotated master webrev index: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>> >>>>>> Performance: this is expected to be performance neutral in a >>>>>> general sense. Benchmarking and performance runs are about to start. >>>>>> >>>>>> Testing Discussion: >>>>>> ------------------ >>>>>> >>>>>> The testing for nestmates can be broken into four main groups: >>>>>> >>>>>> -? New tests specifically related to nestmates and currently in >>>>>> the runtime/Nestmates directory >>>>>> >>>>>> - New tests to complement existing tests by adding in testcases >>>>>> not previously expressible. >>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>>>> use of invokespecial for private interface methods and performing >>>>>> receiver typechecks, so we add >>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>>>> invokeinterface. >>>>>> >>>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>>> attributes. >>>>>> >>>>>> -? Existing tests significantly affected by the nestmates changes, >>>>>> primarily: >>>>>> ??? -? runtime/SelectionResolution >>>>>> >>>>>> ??? In most cases the nestmate changes makes certain invocations >>>>>> that were illegal, legal (e.g. not requiring invokespecial to >>>>>> invoke private interface methods; allowing access to private >>>>>> members via reflection/Methodhandles that were previously not >>>>>> allowed). >>>>>> >>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>> >>>>>> ?? This includes tests of things utilising class >>>>>> redefinition/retransformation to alter nested types but which >>>>>> unintentionally alter nest relationships (which is not permitted). >>>>>> >>>>>> There are still a number of tests problem-listed with issues filed >>>>>> against them to have them adapted to work with nestmates. Some of >>>>>> these are intended to be addressed in the short-term, while some >>>>>> (such as the runtime/SelectionResolution test changes) may not >>>>>> eventuate. >>>>>> >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>> >>>>>> There is also further test work still to be completed (the JNI and >>>>>> JDI invocation tests): >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>> which will continue in parallel with the main RFR. >>>>>> >>>>>> Pre-integration Testing: >>>>>> ??- General: >>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>> ??- Targetted >>>>>> ??? - nashorn (for asm changes) >>>>>> ??? - hotspot: runtime/* >>>>>> ?????????????? serviceability/* >>>>>> ?????????????? compiler/* >>>>>> ?????????????? vmTestbase/* >>>>>> ??? - jdk: java/lang/invoke/* >>>>>> ?????????? java/lang/reflect/* >>>>>> ?????????? java/lang/instrument/* >>>>>> ?????????? java/lang/Class/* >>>>>> ?????????? java/lang/management/* >>>>>> ?? - langtools: tools/javac >>>>>> ??????????????? tools/javap >>>>>> From mandy.chung at oracle.com Wed Jun 20 05:25:46 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 19 Jun 2018 22:25:46 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> Message-ID: <74a03d23-1e3a-b4c4-4eba-121e0f8ca545@oracle.com> The javadoc update looks good to me. Mandy On 6/19/18 9:56 PM, David Holmes wrote: > Some further adjustments to getNestMembers() was made. Everything > updated in place. > > Thanks, > David > > On 20/06/2018 9:30 AM, David Holmes wrote: >> Sorry another update is imminent ... stay tuned. >> >> David >> >> On 19/06/2018 2:41 PM, David Holmes wrote: >>> Discussions on the CSR request have led to further changes to the >>> documentation involving nests and the new nest-related method. There >>> are no semantic changes here just clearer explanations. >>> >>> Incremental webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ >>> >>> >>> (don't worry if you don't see a v6, it didn't really exist). >>> >>> Full webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v7/ >>> >>> Specdiffs updated in place at: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/ >>> >>> Summary of changes: >>> >>> - The definition of a nest etc is moved to the class-level javadoc of >>> java.lang.Class, along with some other edits provided by Alex Buckley >>> to pave the way for future updates >>> - The nest-related methods are written in a more clear and consistent >>> way >>> >>> Thanks, >>> David From david.holmes at oracle.com Wed Jun 20 06:12:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jun 2018 16:12:31 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <74a03d23-1e3a-b4c4-4eba-121e0f8ca545@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> <74a03d23-1e3a-b4c4-4eba-121e0f8ca545@oracle.com> Message-ID: <279e039f-900c-1c51-8315-c9f23ff55ca7@oracle.com> Thanks Mandy! David On 20/06/2018 3:25 PM, mandy chung wrote: > The javadoc update looks good to me. > > Mandy > > On 6/19/18 9:56 PM, David Holmes wrote: >> Some further adjustments to getNestMembers() was made. Everything >> updated in place. >> >> Thanks, >> David >> >> On 20/06/2018 9:30 AM, David Holmes wrote: >>> Sorry another update is imminent ... stay tuned. >>> >>> David >>> >>> On 19/06/2018 2:41 PM, David Holmes wrote: >>>> Discussions on the CSR request have led to further changes to the >>>> documentation involving nests and the new nest-related method. There >>>> are no semantic changes here just clearer explanations. >>>> >>>> Incremental webrev: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ >>>> >>>> >>>> (don't worry if you don't see a v6, it didn't really exist). >>>> >>>> Full webrev: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v7/ >>>> >>>> Specdiffs updated in place at: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/ >>>> >>>> Summary of changes: >>>> >>>> - The definition of a nest etc is moved to the class-level javadoc >>>> of java.lang.Class, along with some other edits provided by Alex >>>> Buckley to pave the way for future updates >>>> - The nest-related methods are written in a more clear and >>>> consistent way >>>> >>>> Thanks, >>>> David > From Pengfei.Li at arm.com Wed Jun 20 10:08:42 2018 From: Pengfei.Li at arm.com (Pengfei Li) Date: Wed, 20 Jun 2018 10:08:42 +0000 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> Message-ID: Hi I have tried the patch ( http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/052991.html ) on Ubuntu 18.04 machines (x86_64 & aarch64) with glibc=2.26.1 and build is OK. There's a small issue within the following code in src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Unlike readdir_r(), readdir() does not return int value. The value of res is always 0 before #ifdef. 727 /* EINTR not listed as a possible error */ 728 errno = 0; 729 ptr = readdir64(dirp); 730 res = errno; 731 732 #ifdef _AIX 733 /* On AIX, readdir() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ 734 /* directory stream end. Otherwise, 'errno' will contain the error code. */ 735 if (res != 0) { 736 res = (ptr == NULL && res == EBADF) ? 0 : errno; 737 } 738 #endif May I know that if this core-libs change going to be merged recently, or more platforms needs to be explored? -- Thanks, Pengfei > On May 7, 2018, at 11:20 AM, B. Blaser wrote: > > On 7 May 2018 at 14:19, B. Blaser wrote: >> On 6 May 2018 at 18:35, B. Blaser wrote: >>> On 5 May 2018 at 22:26, Kim Barrett wrote: >>>>> On May 5, 2018, at 8:03 AM, B. Blaser wrote: >>>>> >>>>> On 4 May 2018 at 17:42, Alan Bateman wrote: >>>>>> On 04/05/2018 16:31, B. Blaser wrote: >>>>>>> >>>>>>> : >>>>>>> >>>>>>> I'll create a new JBS issue (unless you want to) since the fix >>>>>>> is mostly located in core-libs and only intended to make the >>>>>>> build complete. >>>>>>> But I'll still need someone to review and push the fix, would >>>>>>> you be interested in doing this? >>>>>>> >>>>>> I think someone needs to explore the behavior on other platforms >>>>>> too as the #ifndef __linux__ patches are ugly. >>>>> >>>>> Yes sure but I cannot test it elsewhere myself... and we can >>>>> easily remove them once POSIX officially deprecates 'readdir_r' or >>>>> if someone validates the fix on other systems. >>>> >>>> I'm not sure anyone has the ability to test on all supported. That >>>> doesn't (and really can't) prevent making changes across platforms >>>> to platform-specific code. It just means one needs to get help >>>> with testing. Make the change across platforms, test on those >>>> platforms one has access to, and ask here for help testing on others. >>> >>> OK, I'll provide a cross-platform fix to be evaluated on other systems too. >> >> Here it is, tier1 is successful on Linux with glibc=2.26. >> Any feedback on other systems would be very helpful. > > Some more cleanup? From nishit.jain at oracle.com Wed Jun 20 10:32:15 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Wed, 20 Jun 2018 16:02:15 +0530 Subject: RFR JDK-8204938: Add a test case to automatically check the updated LSR data Message-ID: <342415ad-b407-66f9-85a0-3f5aedc6f996@oracle.com> Hi, Please review the fix for JDK-8204938 Bug: https://bugs.openjdk.java.net/browse/JDK-8204938 Webrev: http://cr.openjdk.java.net/~nishjain/8204938/webrev.02/ Fix: Added a test case to cross check the LSR data generated for the JDK APIs. So, for each lsr data update, the test case need not be updated, it automatically cross checks the updated lsr data. Regards, Nishit Jain From matthias.baesken at sap.com Wed Jun 20 12:09:21 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 20 Jun 2018 12:09:21 +0000 Subject: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return code in Java_java_io_FileCleanable_cleanupClose0 Message-ID: <6b18f48727a249d790ee32f21b9d28b5@sap.com> Please review this small fix for a return code handling of windows function CloseHandle . MSDN documents CloseHandle here : https://msdn.microsoft.com/de-de/library/windows/desktop/ms724211(v=vs.85).aspx .... Return value If the function succeeds, the return value is nonzero. If the function fails, the return value is zero. To get extended error information, call GetLastError. However until this patch, Java_java_io_FileCleanable_cleanupClose0 Checked for return code -1 of CloseHandle in error cases . Bug: https://bugs.openjdk.java.net/browse/JDK-8205416 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8205416/ Thanks, Matthias From Alan.Bateman at oracle.com Wed Jun 20 12:15:59 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 13:15:59 +0100 Subject: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return code in Java_java_io_FileCleanable_cleanupClose0 In-Reply-To: <6b18f48727a249d790ee32f21b9d28b5@sap.com> References: <6b18f48727a249d790ee32f21b9d28b5@sap.com> Message-ID: On 20/06/2018 13:09, Baesken, Matthias wrote: > : > > > Webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205416/ > This looks okay me, probably an oversight when this code was refactored. -Alan From thomas.stuefe at gmail.com Wed Jun 20 12:16:40 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 20 Jun 2018 14:16:40 +0200 Subject: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return code in Java_java_io_FileCleanable_cleanupClose0 In-Reply-To: <6b18f48727a249d790ee32f21b9d28b5@sap.com> References: <6b18f48727a249d790ee32f21b9d28b5@sap.com> Message-ID: Good catch. Could you please change the expression to either CloseHandle() == FALSE (uppercase) or !CloseHandle(). ? which would be the standard windows way of writing this. I am curious whether we now get a bunch of follow up errors, since before we never catched a failing CloseHandle. A typical reason why this could fail would be a double close. ..Thomas On Wed, Jun 20, 2018 at 2:09 PM, Baesken, Matthias wrote: > Please review this small fix for a return code handling of windows function CloseHandle . > > MSDN documents CloseHandle here : https://msdn.microsoft.com/de-de/library/windows/desktop/ms724211(v=vs.85).aspx > .... > Return value > If the function succeeds, the return value is nonzero. > If the function fails, the return value is zero. To get extended error information, call GetLastError. > > > However until this patch, Java_java_io_FileCleanable_cleanupClose0 > Checked for return code -1 of CloseHandle in error cases . > > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8205416 > > > Webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205416/ > > > > Thanks, Matthias From Alan.Bateman at oracle.com Wed Jun 20 12:27:27 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 13:27:27 +0100 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> Message-ID: <869ce7f1-6484-a2fc-3064-0f94c3ff8974@oracle.com> On 11/06/2018 07:15, Ivan Gerasimov wrote: > : > I extended the existing reg. test > java/io/RandomAccessFile/SetLength.java to cover more cases that > involve changing the file size. > > Also I added another regression test, as you Alan suggested, to check > that RandomAccessFile and its FileChannel behave consistently in > various scenarios. > > All the tests, including the new ones, pass on all supported platforms. > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 > WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/01/webrev/ > > Would you please help review the fix? > The expanded test coverage looks good and really important when touching this Windows specific code (as you know, although every change in this area results in some behavioral change that shows up months or years after the fact).? So I think this is good to go although it may be more prudent to wait until after JDK 11 is forked next week. -Alan From Alan.Bateman at oracle.com Wed Jun 20 12:30:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 13:30:11 +0100 Subject: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return code in Java_java_io_FileCleanable_cleanupClose0 In-Reply-To: References: <6b18f48727a249d790ee32f21b9d28b5@sap.com> Message-ID: <0d3abc92-90f7-e87d-710d-30328fa892f4@oracle.com> On 20/06/2018 13:16, Thomas St?fe wrote: > : > > I am curious whether we now get a bunch of follow up errors, since > before we never catched a failing CloseHandle. A typical reason why > this could fail would be a double close. > Yes, and it typically means that something is seriously broken. We had exactly this early in JDK 11 where some refactoring (to make use of Cleaner) introduced a bug that lead to random closing of sockets. So important that any failures like this are diagnosed quickly. -Alan From thomas.stuefe at gmail.com Wed Jun 20 12:34:41 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 20 Jun 2018 14:34:41 +0200 Subject: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return code in Java_java_io_FileCleanable_cleanupClose0 In-Reply-To: <0d3abc92-90f7-e87d-710d-30328fa892f4@oracle.com> References: <6b18f48727a249d790ee32f21b9d28b5@sap.com> <0d3abc92-90f7-e87d-710d-30328fa892f4@oracle.com> Message-ID: On Wed, Jun 20, 2018 at 2:30 PM, Alan Bateman wrote: > > > On 20/06/2018 13:16, Thomas St?fe wrote: >> >> : >> >> I am curious whether we now get a bunch of follow up errors, since >> before we never catched a failing CloseHandle. A typical reason why >> this could fail would be a double close. >> > Yes, and it typically means that something is seriously broken. We had > exactly this early in JDK 11 where some refactoring (to make use of Cleaner) > introduced a bug that lead to random closing of sockets. So important that > any failures like this are diagnosed quickly. > -Alan Even more fun if the double close actually succeeds because of handle reuse :-) Unix is good though, so whatever error we unearth will be windows specific. ..Thomas From matthias.baesken at sap.com Wed Jun 20 12:39:19 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 20 Jun 2018 12:39:19 +0000 Subject: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return code in Java_java_io_FileCleanable_cleanupClose0 In-Reply-To: References: <6b18f48727a249d790ee32f21b9d28b5@sap.com> Message-ID: Hi Thomas and Alan, thanks for the reviews. I adjusted the expression like Thomas suggested and adjusted the file copyright year too : http://cr.openjdk.java.net/~mbaesken/webrevs/8205416.1/ Best regards, Matthias > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Mittwoch, 20. Juni 2018 14:17 > To: Baesken, Matthias > Cc: core-libs-dev at openjdk.java.net > Subject: Re: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return > code in Java_java_io_FileCleanable_cleanupClose0 > > Good catch. > > Could you please change the expression to either > > CloseHandle() == FALSE (uppercase) > > or > > !CloseHandle(). > > ? which would be the standard windows way of writing this. > > I am curious whether we now get a bunch of follow up errors, since > before we never catched a failing CloseHandle. A typical reason why > this could fail would be a double close. > > ..Thomas > > > On Wed, Jun 20, 2018 at 2:09 PM, Baesken, Matthias > wrote: > > Please review this small fix for a return code handling of windows function > CloseHandle . > > > > MSDN documents CloseHandle here : https://msdn.microsoft.com/de- > de/library/windows/desktop/ms724211(v=vs.85).aspx > > .... > > Return value > > If the function succeeds, the return value is nonzero. > > If the function fails, the return value is zero. To get extended error > information, call GetLastError de/library/windows/desktop/ms679360(v=vs.85).aspx>. > > > > > > However until this patch, Java_java_io_FileCleanable_cleanupClose0 > > Checked for return code -1 of CloseHandle in error cases . > > > > > > > > > > Bug: > > > > https://bugs.openjdk.java.net/browse/JDK-8205416 > > > > > > Webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205416/ > > > > > > > > Thanks, Matthias From thomas.stuefe at gmail.com Wed Jun 20 12:44:47 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 20 Jun 2018 14:44:47 +0200 Subject: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return code in Java_java_io_FileCleanable_cleanupClose0 In-Reply-To: References: <6b18f48727a249d790ee32f21b9d28b5@sap.com> Message-ID: Looks good, Matthias. ..Thomas On Wed, Jun 20, 2018 at 2:39 PM, Baesken, Matthias wrote: > Hi Thomas and Alan, thanks for the reviews. > > I adjusted the expression like Thomas suggested and adjusted the file copyright year too : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205416.1/ > > > Best regards, Matthias > > > >> -----Original Message----- >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> Sent: Mittwoch, 20. Juni 2018 14:17 >> To: Baesken, Matthias >> Cc: core-libs-dev at openjdk.java.net >> Subject: Re: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return >> code in Java_java_io_FileCleanable_cleanupClose0 >> >> Good catch. >> >> Could you please change the expression to either >> >> CloseHandle() == FALSE (uppercase) >> >> or >> >> !CloseHandle(). >> >> ? which would be the standard windows way of writing this. >> >> I am curious whether we now get a bunch of follow up errors, since >> before we never catched a failing CloseHandle. A typical reason why >> this could fail would be a double close. >> >> ..Thomas >> >> >> On Wed, Jun 20, 2018 at 2:09 PM, Baesken, Matthias >> wrote: >> > Please review this small fix for a return code handling of windows function >> CloseHandle . >> > >> > MSDN documents CloseHandle here : https://msdn.microsoft.com/de- >> de/library/windows/desktop/ms724211(v=vs.85).aspx >> > .... >> > Return value >> > If the function succeeds, the return value is nonzero. >> > If the function fails, the return value is zero. To get extended error >> information, call GetLastError> de/library/windows/desktop/ms679360(v=vs.85).aspx>. >> > >> > >> > However until this patch, Java_java_io_FileCleanable_cleanupClose0 >> > Checked for return code -1 of CloseHandle in error cases . >> > >> > >> > >> > >> > Bug: >> > >> > https://bugs.openjdk.java.net/browse/JDK-8205416 >> > >> > >> > Webrev : >> > >> > http://cr.openjdk.java.net/~mbaesken/webrevs/8205416/ >> > >> > >> > >> > Thanks, Matthias From dave_hobbs at uk.ibm.com Wed Jun 20 13:22:42 2018 From: dave_hobbs at uk.ibm.com (Dave Hobbs) Date: Wed, 20 Jun 2018 14:22:42 +0100 Subject: Creating a charset provider module for IBM charsets Message-ID: Hi We would like to migrate a number of the IBM charsets (which don't need to be in java.base) to a modular charset provider. However, as written today our charset classes rely on sun.nio.cs classes and definitions, which are no longer visible. For example: sun.nio.cs.HistoricallyNamedCharset; sun.nio.cs.CharsetMapping.UNMAPPABLE_DECODING; sun.nio.cs.CharsetMapping.UNMAPPABLE_ENCODING; sun.nio.cs.ArrayDecoder; sun.nio.cs.ArrayEncoder; sun.nio.cs.Surrogate.Parser; sun.nio.cs.SingleByte.Decoder; sun.nio.cs.ext.DoubleByte.Decoder; sun.nio.cs.SingleByteDecoder; sun.nio.cs.SingleByteEncoder; Are there any alternatives to these classes in the new module system or an alternative approach to creating charsets? re: http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053316.html Regards Dave Hobbs Java Runtimes Development IBM Hursley IBM United Kingdom Ltd Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Alan.Bateman at oracle.com Wed Jun 20 13:26:56 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 14:26:56 +0100 Subject: Creating a charset provider module for IBM charsets In-Reply-To: References: Message-ID: <64173406-ab24-1620-a029-5b90e0acddb3@oracle.com> On 20/06/2018 14:22, Dave Hobbs wrote: > Hi > > We would like to migrate a number of the IBM charsets (which don't need to > be in java.base) to a modular charset provider. > > However, as written today our charset classes rely on sun.nio.cs classes > and definitions, which are no longer visible. For example: > > sun.nio.cs.HistoricallyNamedCharset; > > sun.nio.cs.CharsetMapping.UNMAPPABLE_DECODING; > sun.nio.cs.CharsetMapping.UNMAPPABLE_ENCODING; > > sun.nio.cs.ArrayDecoder; > sun.nio.cs.ArrayEncoder; > > sun.nio.cs.Surrogate.Parser; > > sun.nio.cs.SingleByte.Decoder; > sun.nio.cs.ext.DoubleByte.Decoder; > > sun.nio.cs.SingleByteDecoder; > sun.nio.cs.SingleByteEncoder; > > Are there any alternatives to these classes in the new module system or an > alternative approach to creating charsets? Can you use jdk.charsets as an example? You'll see that java.base exports these packages to the service provider module. -Alan From Alan.Bateman at oracle.com Wed Jun 20 13:41:58 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 14:41:58 +0100 Subject: RFR [XS] : 8205416 : windows: fix checking of CloseHandle return code in Java_java_io_FileCleanable_cleanupClose0 In-Reply-To: References: <6b18f48727a249d790ee32f21b9d28b5@sap.com> Message-ID: On 20/06/2018 13:39, Baesken, Matthias wrote: > Hi Thomas and Alan, thanks for the reviews. > > I adjusted the expression like Thomas suggested and adjusted the file copyright year too : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205416.1/ > > Looks okay to me too. -Alan From peter.levart at gmail.com Wed Jun 20 14:08:46 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 20 Jun 2018 16:08:46 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> On 06/18/2018 05:41 PM, Alan Bateman wrote: > > > On 17/06/2018 22:20, Peter Levart wrote: >> Update: the discussion on concurrent-interest about possible future >> addition of public ThreadLocal.getIfPresent() or >> ThreadLocal.isPresent() has died out, but it nevertheless reached a >> point where ThreadLocal.isPresent() was found the least problematic. >> So I would like to get further with this proposal using the last >> webrev.04 version of the patch that uses ThreadLocal.isPresent() >> internally - still package-private at the moment. >> >> Here's the webrev (unchanged from the day it was published): >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ >> >> Would this version be OK? > I think looks quite good. > > One small nit is that the update to ThreadLocal.setInitialValue makes > it look like all locals are registered when setting the initial value. > What would you think about moving the instanceof check from > TerminatingThreadLocal.register so that it's a bit more obvious. > > -Alan Like the following? http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ Do we need a test which proves that cached direct buffers are released when thread exits or would a unit test for TerminatingThreadLocal be enough? Maybe both? Regards, Peter From tprintezis at twitter.com Wed Jun 20 14:24:15 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 20 Jun 2018 07:24:15 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> References: <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> Message-ID: Hey Peter, I had written a test to test my version of the exit hooks. I can easily rework it to work with yours. Interested? To check that the native buffers are reclaimed promptly, we should be able to amend the test that tests the setting of the max size for the cached buffers (i.e., check that, after all the threads have exited, the total native count / size is 0). Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 20, 2018 at 10:08:48 AM, Peter Levart (peter.levart at gmail.com) wrote: On 06/18/2018 05:41 PM, Alan Bateman wrote: > > > On 17/06/2018 22:20, Peter Levart wrote: >> Update: the discussion on concurrent-interest about possible future >> addition of public ThreadLocal.getIfPresent() or >> ThreadLocal.isPresent() has died out, but it nevertheless reached a >> point where ThreadLocal.isPresent() was found the least problematic. >> So I would like to get further with this proposal using the last >> webrev.04 version of the patch that uses ThreadLocal.isPresent() >> internally - still package-private at the moment. >> >> Here's the webrev (unchanged from the day it was published): >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ >> >> Would this version be OK? > I think looks quite good. > > One small nit is that the update to ThreadLocal.setInitialValue makes > it look like all locals are registered when setting the initial value. > What would you think about moving the instanceof check from > TerminatingThreadLocal.register so that it's a bit more obvious. > > -Alan Like the following? http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ Do we need a test which proves that cached direct buffers are released when thread exits or would a unit test for TerminatingThreadLocal be enough? Maybe both? Regards, Peter From dave_hobbs at uk.ibm.com Wed Jun 20 14:52:13 2018 From: dave_hobbs at uk.ibm.com (Dave Hobbs) Date: Wed, 20 Jun 2018 15:52:13 +0100 Subject: Creating a charset provider module for IBM charsets In-Reply-To: <64173406-ab24-1620-a029-5b90e0acddb3@oracle.com> References: <64173406-ab24-1620-a029-5b90e0acddb3@oracle.com> Message-ID: Hi Alan, My understanding is that java.base only exports sun.nio.cs to jdk.charsets i.e java.base module-info.java has: ... exports sun.nio.cs to java.desktop, jdk.charsets; ... and jdk.charsets has: module jdk.charsets { provides java.nio.charset.spi.CharsetProvider with sun.nio.cs.ext.ExtendedCharsets; } So jdk.charsets can use sun.nio.cs, whereas my module can not. Am I missing something? Dave From: Alan Bateman To: Dave Hobbs , core-libs-dev at openjdk.java.net Date: 20/06/2018 14:26 Subject: Re: Creating a charset provider module for IBM charsets On 20/06/2018 14:22, Dave Hobbs wrote: > Hi > > We would like to migrate a number of the IBM charsets (which don't need to > be in java.base) to a modular charset provider. > > However, as written today our charset classes rely on sun.nio.cs classes > and definitions, which are no longer visible. For example: > > sun.nio.cs.HistoricallyNamedCharset; > > sun.nio.cs.CharsetMapping.UNMAPPABLE_DECODING; > sun.nio.cs.CharsetMapping.UNMAPPABLE_ENCODING; > > sun.nio.cs.ArrayDecoder; > sun.nio.cs.ArrayEncoder; > > sun.nio.cs.Surrogate.Parser; > > sun.nio.cs.SingleByte.Decoder; > sun.nio.cs.ext.DoubleByte.Decoder; > > sun.nio.cs.SingleByteDecoder; > sun.nio.cs.SingleByteEncoder; > > Are there any alternatives to these classes in the new module system or an > alternative approach to creating charsets? Can you use jdk.charsets as an example? You'll see that java.base exports these packages to the service provider module. -Alan Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From bsrbnd at gmail.com Wed Jun 20 16:01:18 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 20 Jun 2018 18:01:18 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> Message-ID: Hi Pengfei, On 20 June 2018 at 12:08, Pengfei Li wrote: > Hi > > I have tried the patch ( http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/052991.html ) on Ubuntu 18.04 machines (x86_64 & aarch64) with glibc=2.26.1 and build is OK. > > There's a small issue within the following code in src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c > Unlike readdir_r(), readdir() does not return int value. The value of res is always 0 before #ifdef. > > 727 /* EINTR not listed as a possible error */ > 728 errno = 0; > 729 ptr = readdir64(dirp); > 730 res = errno; > 731 > 732 #ifdef _AIX > 733 /* On AIX, readdir() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ > 734 /* directory stream end. Otherwise, 'errno' will contain the error code. */ > 735 if (res != 0) { > 736 res = (ptr == NULL && res == EBADF) ? 0 : errno; > 737 } > 738 #endif > > May I know that if this core-libs change going to be merged recently, or more platforms needs to be explored? > > -- > Thanks, > Pengfei Thanks for evaluating this patch, 'readdir()' doesn't return an 'int' value but it sets 'errno' instead which is then assigned to 'res'. So, I guess the fix is OK, or did I miss anything? I've also tested it on Linux/x86_64 but ideally, the patch should be tested on all supported POSIX platforms before being integrated. The JBS issue [1] doesn't mention any fix version, so I'm not sure if it'll be targeted to 11 or 12. But if someone is available to test it on other POSIX platforms and review it, this would be nice? Thanks, Bernard [1] https://bugs.openjdk.java.net/browse/JDK-8202794 From ivan.gerasimov at oracle.com Wed Jun 20 16:48:23 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 20 Jun 2018 09:48:23 -0700 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <869ce7f1-6484-a2fc-3064-0f94c3ff8974@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> <869ce7f1-6484-a2fc-3064-0f94c3ff8974@oracle.com> Message-ID: <38c59cce-d2f8-3b5f-b0d2-b3940c1a866d@oracle.com> Thank you Alan for review! I'll push it in a couple of weeks then. With kind regards, Ivan On 6/20/18 5:27 AM, Alan Bateman wrote: > On 11/06/2018 07:15, Ivan Gerasimov wrote: >> : >> I extended the existing reg. test >> java/io/RandomAccessFile/SetLength.java to cover more cases that >> involve changing the file size. >> >> Also I added another regression test, as you Alan suggested, to check >> that RandomAccessFile and its FileChannel behave consistently in >> various scenarios. >> >> All the tests, including the new ones, pass on all supported platforms. >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/01/webrev/ >> >> Would you please help review the fix? >> > The expanded test coverage looks good and really important when > touching this Windows specific code (as you know, although every > change in this area results in some behavioral change that shows up > months or years after the fact). So I think this is good to go > although it may be more prudent to wait until after JDK 11 is forked > next week. > > -Alan > -- With kind regards, Ivan Gerasimov From markus.gronlund at oracle.com Wed Jun 20 16:54:54 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 20 Jun 2018 09:54:54 -0700 (PDT) Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <5B286533.6070500@oracle.com> References: <5B286533.6070500@oracle.com> Message-ID: Hi Erik, looks good. Very powerful with minimal intrusion. Thanks Markus -----Original Message----- From: Erik Gahlin Sent: den 19 juni 2018 04:07 To: hotspot-jfr-dev at openjdk.java.net; core-libs-dev Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr Hi, Could I have a review of an enhancement that will make it possible to add JFR events to java.base, and possibly other modules in the JDK, without a compile time dependency on jdk.jfr. Bug: https://bugs.openjdk.java.net/browse/JDK-8203629 Webrev: http://cr.openjdk.java.net/~egahlin/8203629.0 Testing: Tests in test/jdk/jdk/jfr The functionality is a prerequisite for an upcoming enhancement that will add JFR events to the security libraries. In short, jdk.jfr.Event (located in jdk.jfr) extends jdk.internal.event.Event (located in java.base). Metadata for events, such as labels and descriptions, are added using a mirror event class located in jdk.jfr module. If the fields of the mirror class and the event class don't match an InternalError is thrown. To illustrate how the mechanism can be used, see the following example (not meant to be checked in). http://cr.openjdk.java.net/~egahlin/8203629.example Since the implementation of jdk.internal.event.Event is empty, the JIT will typically eliminate all traces of JFR functionality unless Flight Recorder is enabled. Thanks Erik From Alan.Bateman at oracle.com Wed Jun 20 17:09:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 18:09:24 +0100 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <5B286533.6070500@oracle.com> References: <5B286533.6070500@oracle.com> Message-ID: <3bce8034-387e-6ca5-d04d-b7171a13f47b@oracle.com> On 19/06/2018 03:06, Erik Gahlin wrote: > Hi, > > Could I have a review of an enhancement that will make it possible to > add JFR events to java.base, and possibly other modules in the JDK, > without a compile time dependency on jdk.jfr. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8203629 > > Webrev: > http://cr.openjdk.java.net/~egahlin/8203629.0 > > Testing: > Tests in test/jdk/jdk/jfr > > The functionality is a prerequisite for an upcoming enhancement that > will add JFR events to the security libraries. > > In short, jdk.jfr.Event (located in jdk.jfr) extends > jdk.internal.event.Event (located in java.base). Metadata for events, > such as labels and descriptions, are added using a mirror event class > located in jdk.jfr module. If the fields of the mirror class and the > event class don't match an InternalError is thrown. > > To illustrate how the mechanism can be used, see the following example > (not meant to be checked in). > > http://cr.openjdk.java.net/~egahlin/8203629.example > > Since the implementation of jdk.internal.event.Event is empty, the JIT > will typically eliminate all traces of JFR functionality unless Flight > Recorder is enabled. > Adding a way to get events from java.base is good but I wonder if jdk.internal.event.Event could be cleaned up before you push this. It would be nice to have a class description and some minimal method descriptions too. Also all the methods are empty which makes me wonder if they should be abstract (as the class is abstract) or whether it should be an interface. Some of the method modifiers are in unusual order and it would be good to get those cleaned up too. -Alan From sean.coffey at oracle.com Wed Jun 20 17:22:26 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 20 Jun 2018 18:22:26 +0100 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <5B286533.6070500@oracle.com> References: <5B286533.6070500@oracle.com> Message-ID: Thanks for implementing this enhancement Erik. I think it'll prove to be popular given the functionality available in JFR recordings. I'd agree with Alan's comment that the Event class could be documented better to indicate its purpose. I'm already using this enhancement to implement some new events in the JDK security libraries. I hope to get a review started real soon on that front. Regards, Sean. On 19/06/18 03:06, Erik Gahlin wrote: > Hi, > > Could I have a review of an enhancement that will make it possible to > add JFR events to java.base, and possibly other modules in the JDK, > without a compile time dependency on jdk.jfr. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8203629 > > Webrev: > http://cr.openjdk.java.net/~egahlin/8203629.0 > > Testing: > Tests in test/jdk/jdk/jfr > > The functionality is a prerequisite for an upcoming enhancement that > will add JFR events to the security libraries. > > In short, jdk.jfr.Event (located in jdk.jfr) extends > jdk.internal.event.Event (located in java.base). Metadata for events, > such as labels and descriptions, are added using a mirror event class > located in jdk.jfr module. If the fields of the mirror class and the > event class don't match an InternalError is thrown. > > To illustrate how the mechanism can be used, see the following example > (not meant to be checked in). > > http://cr.openjdk.java.net/~egahlin/8203629.example > > Since the implementation of jdk.internal.event.Event is empty, the JIT > will typically eliminate all traces of JFR functionality unless Flight > Recorder is enabled. > > Thanks > Erik > From Alan.Bateman at oracle.com Wed Jun 20 17:27:17 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 18:27:17 +0100 Subject: Creating a charset provider module for IBM charsets In-Reply-To: References: <64173406-ab24-1620-a029-5b90e0acddb3@oracle.com> Message-ID: <0465abd4-b087-263c-baf9-eedbdf1d96fb@oracle.com> On 20/06/2018 15:52, Dave Hobbs wrote: > Hi Alan, > > My understanding is that java.base only exports sun.nio.cs to jdk.charsets > i.e java.base module-info.java has: > > ... > exports sun.nio.cs to > java.desktop, > jdk.charsets; > ... > > and jdk.charsets has: > > module jdk.charsets { > provides java.nio.charset.spi.CharsetProvider with > sun.nio.cs.ext.ExtendedCharsets; > } > > So jdk.charsets can use sun.nio.cs, whereas my module can not. > > Am I missing something? > Are you proposing to add your module to the JDK or will it live outside? If the former then just extend the qualified export to add the name of the module. If the latter then you are out of luck as the sun.nio.cs infrastructure is not meant to be used outside of the JDK. -Alan From paul.sandoz at oracle.com Wed Jun 20 17:43:30 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Jun 2018 10:43:30 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <74a03d23-1e3a-b4c4-4eba-121e0f8ca545@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> <74a03d23-1e3a-b4c4-4eba-121e0f8ca545@oracle.com> Message-ID: > On Jun 19, 2018, at 10:25 PM, mandy chung wrote: > > The javadoc update looks good to me. > Same here, +1 Paul. From erik.gahlin at oracle.com Wed Jun 20 17:59:35 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 20 Jun 2018 19:59:35 +0200 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <3bce8034-387e-6ca5-d04d-b7171a13f47b@oracle.com> References: <5B286533.6070500@oracle.com> <3bce8034-387e-6ca5-d04d-b7171a13f47b@oracle.com> Message-ID: <5B2A9607.9010206@oracle.com> On 2018-06-20 19:09, Alan Bateman wrote: > > > On 19/06/2018 03:06, Erik Gahlin wrote: >> Hi, >> >> Could I have a review of an enhancement that will make it possible to >> add JFR events to java.base, and possibly other modules in the JDK, >> without a compile time dependency on jdk.jfr. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8203629 >> >> Webrev: >> http://cr.openjdk.java.net/~egahlin/8203629.0 >> >> Testing: >> Tests in test/jdk/jdk/jfr >> >> The functionality is a prerequisite for an upcoming enhancement that >> will add JFR events to the security libraries. >> >> In short, jdk.jfr.Event (located in jdk.jfr) extends >> jdk.internal.event.Event (located in java.base). Metadata for events, >> such as labels and descriptions, are added using a mirror event class >> located in jdk.jfr module. If the fields of the mirror class and the >> event class don't match an InternalError is thrown. >> >> To illustrate how the mechanism can be used, see the following >> example (not meant to be checked in). >> >> http://cr.openjdk.java.net/~egahlin/8203629.example >> >> Since the implementation of jdk.internal.event.Event is empty, the >> JIT will typically eliminate all traces of JFR functionality unless >> Flight Recorder is enabled. >> > Adding a way to get events from java.base is good but I wonder if > jdk.internal.event.Event could be cleaned up before you push this. It > would be nice to have a class description and some minimal method > descriptions too. I will add some comments. > Also all the methods are empty which makes me wonder if they should be > abstract (as the class is abstract) or whether it should be an interface. Abstract is needed to prevent user from instantiating the class. The methods need to have a body, otherwise event classes in java.base would need to implement the methods, which would be cumbersome. We like to use a class as it simplifies the implementation if we know all event classes have a common ancestor with java.lang.Object as a parent. so we can be guaranteed that the class > Some of the method modifiers are in unusual order and it would be good > to get those cleaned up too. > I will make the class "public abstract". The other modifiers looked OK to me, but please let me know if there are other modifiers you want me to change. Erik From erik.gahlin at oracle.com Wed Jun 20 18:06:52 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 20 Jun 2018 20:06:52 +0200 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: References: <5B286533.6070500@oracle.com> Message-ID: <5B2A97BC.4060101@oracle.com> Sounds good Sean. We will push this RFE when your enhancement is ready and reviewed. We like to avoid pushing this to JDK 11 unless it is being used in "the current train". Erik > Thanks for implementing this enhancement Erik. I think it'll prove to > be popular given the functionality available in JFR recordings. I'd > agree with Alan's comment that the Event class could be documented > better to indicate its purpose. > > I'm already using this enhancement to implement some new events in the > JDK security libraries. I hope to get a review started real soon on > that front. > > Regards, > Sean. > > On 19/06/18 03:06, Erik Gahlin wrote: >> Hi, >> >> Could I have a review of an enhancement that will make it possible to >> add JFR events to java.base, and possibly other modules in the JDK, >> without a compile time dependency on jdk.jfr. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8203629 >> >> Webrev: >> http://cr.openjdk.java.net/~egahlin/8203629.0 >> >> Testing: >> Tests in test/jdk/jdk/jfr >> >> The functionality is a prerequisite for an upcoming enhancement that >> will add JFR events to the security libraries. >> >> In short, jdk.jfr.Event (located in jdk.jfr) extends >> jdk.internal.event.Event (located in java.base). Metadata for events, >> such as labels and descriptions, are added using a mirror event class >> located in jdk.jfr module. If the fields of the mirror class and the >> event class don't match an InternalError is thrown. >> >> To illustrate how the mechanism can be used, see the following >> example (not meant to be checked in). >> >> http://cr.openjdk.java.net/~egahlin/8203629.example >> >> Since the implementation of jdk.internal.event.Event is empty, the >> JIT will typically eliminate all traces of JFR functionality unless >> Flight Recorder is enabled. >> >> Thanks >> Erik >> > From naoto.sato at oracle.com Wed Jun 20 18:27:30 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 20 Jun 2018 11:27:30 -0700 Subject: RFR JDK-8204938: Add a test case to automatically check the updated LSR data In-Reply-To: <342415ad-b407-66f9-85a0-3f5aedc6f996@oracle.com> References: <342415ad-b407-66f9-85a0-3f5aedc6f996@oracle.com> Message-ID: Looks good. Naoto On 6/20/18 3:32 AM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8204938 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204938 > Webrev: http://cr.openjdk.java.net/~nishjain/8204938/webrev.02/ > > Fix: Added a test case to cross check the LSR data generated for the JDK > APIs. So, for each lsr data update, the test case need not be updated, > it automatically cross checks the updated lsr data. > > Regards, > Nishit Jain From Roger.Riggs at Oracle.com Wed Jun 20 18:53:40 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 20 Jun 2018 14:53:40 -0400 Subject: RFR JDK-8204938: Add a test case to automatically check the updated LSR data In-Reply-To: References: <342415ad-b407-66f9-85a0-3f5aedc6f996@oracle.com> Message-ID: <82a5d19f-d89b-6b91-3a11-8100a4aed3a7@Oracle.com> Hi Nishit, Looks ok My only comment would be to put the relative path to the language-subtag-registry.txt in a private static final constant to call attention to the dependency on the source layout instead of burying it in the code. $.02, Roger On 6/20/2018 2:27 PM, naoto.sato at oracle.com wrote: > Looks good. > > Naoto > > On 6/20/18 3:32 AM, Nishit Jain wrote: >> Hi, >> >> Please review the fix for JDK-8204938 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204938 >> Webrev: http://cr.openjdk.java.net/~nishjain/8204938/webrev.02/ >> >> Fix: Added a test case to cross check the LSR data generated for the >> JDK APIs. So, for each lsr data update, the test case need not be >> updated, it automatically cross checks the updated lsr data. >> >> Regards, >> Nishit Jain From ecki at zusammenkunft.net Wed Jun 20 20:05:55 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Wed, 20 Jun 2018 20:05:55 +0000 Subject: RFC: Add new JCA provider to support hardware RNGs In-Reply-To: <993c258a-57ae-d29d-2368-faac5b4d3cb1@linux.vnet.ibm.com> References: <993c258a-57ae-d29d-2368-faac5b4d3cb1@linux.vnet.ibm.com> Message-ID: Just a FYI under Linux when you read from urandom the Linux kernel will always XOR with random bytes generated with x64 rdrand instruction (arch_get_random_lomg() - if supported). Since it is a XOR it does not have to trust the quality of this black box hardware implementation. I would not implement a generator which does not have its own software whitening. (And most likely there is no need for one different than urandom on Linux). If you do implement whitening I would use a DRBG construction and no longer use a (low state) SHA1PRNG. Gruss Bernd Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: core-libs-dev on behalf of Gustavo Romero Sent: Wednesday, June 20, 2018 2:59:47 AM To: core-libs-dev; security-dev Cc: vladimir.kozlov at oracle.com; Doerr, Martin Subject: RFC: Add new JCA provider to support hardware RNGs Hi, Please, could I get comments on the following change? Since it's related to security, I would be glad if security experts could also comment on that. webrev: http://cr.openjdk.java.net/~gromero/POWER9/darn/v6_rebased/ It introduces a way to get benefits from hardware instructions in userspace that can delivery a random number and so be used to speed up and avoid blocking in some SecureRandom methods, notably generateSeed() and nextBytes(), without loosing the random number quality. Particularly on Power, the new POWER9 CPUs introduced a hardware instruction called 'darn' that can be used for that purpose. The main idea is to add a new JCA provider called "HWTRNG" (if no better name is suggested), adding new helper methods that can then be intrinsified and that will be used by generateSeed() and nextBytes(). In that sense, this change also adds the proper mechanisms in the Interpreter, C1 Compiler, and C2 compiler (for PPC64, but also paving the way for other platforms) to intrinsify these helper methods that will be used in the end by generateSeed() and nextBytes() methods. The added helpers are: byte[] getRandomSequence0(int) byte[] getRandomSequence1(byte[]) long validRandomLong(void) long randomLong(void) The helpers also take care of checking if the returned random number is valid and attempt 10 times (as recommended by ISA) get a valid random number before falling back to a software alternative (HWTRNG is based mostly on JCA provider NativePRNG and so the fall back mechanism will use the exactly same methods found in NativePRNG and hence behave similarly. Nonetheless, in my experience such a hardware failures (which are the main cause of a returned invalid random number) are quite rare: in my tests I was never able to exhaust the HW RNG and force it to generate an invalid random number). The user, besides having to specify explicitly the use of a non-default provider (HWTRNG), will have to unlock the VM experimental options to effectively use the hardware RNG as an unique random source by passing options "-XX:+UnlockExperimentalVMOptions -XX:+UseRANDOMIntrinsics". On Power, for the Interpreter and C1 Compiler, besides avoiding the blocking effect of a low entropy on /dev/random, I was able to get about 126 Mbps (3x higher than the version without 'darn') on generaSeed() and nextBytes(). The maximum theoretical value on a POWER9 machine is usually 128 Mbps. I'll address the details about the file headers (Copyright, dates, authors, versions, etc) after I get some feedback about this change. Thanks in advance. Best regards, Gustavo From john.r.rose at oracle.com Wed Jun 20 20:15:57 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 20 Jun 2018 13:15:57 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> Message-ID: <42093BA6-90C6-4FE1-AB3F-1AA13CDC8FC9@oracle.com> Good; I like the care to distinguish "nested" classes (using the word "enclosing") from the newer concept of "nests" and "nestmates", as well as the careful framing of how stuff bubbles up from the classfile. (Kudos to Alex!) ? John On Jun 19, 2018, at 9:56 PM, David Holmes wrote: > > Some further adjustments to getNestMembers() was made. Everything updated in place. > > Thanks, > David > > On 20/06/2018 9:30 AM, David Holmes wrote: >> Sorry another update is imminent ... stay tuned. >> David >> On 19/06/2018 2:41 PM, David Holmes wrote: >>> Discussions on the CSR request have led to further changes to the documentation involving nests and the new nest-related method. There are no semantic changes here just clearer explanations. >>> >>> Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ >>> From vivek.r.deshpande at intel.com Wed Jun 20 20:21:47 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Wed, 20 Jun 2018 20:21:47 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> Hi All I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. Earlier webrev failed the same test. This webrev also modifies the BufferMismatch.java in similar way. Please take a look and let me know what you think. Updated webrev is here: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.01/ I have also modified the bug with updated webrev. Regards, Vivek From: Deshpande, Vivek R Sent: Tuesday, June 19, 2018 10:57 AM To: Paul Sandoz ; Roger.Riggs at oracle.com Cc: David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Roger I will also test with zero length arrays and let you know. Thanks for the input. Regards, Vivek From: Deshpande, Vivek R Sent: Tuesday, June 19, 2018 10:17 AM To: 'Paul Sandoz' > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. Regards, Vivek From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Tuesday, June 19, 2018 9:55 AM To: Deshpande, Vivek R > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Thanks for investigating this. 164 public static int mismatch(boolean[] a, 165 boolean[] b, 166 int length) { 167 int i = 0; 168 if (a[i] != b[i]) 169 return i; You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. 186 public static int mismatch(boolean[] a, int aFromIndex, 187 boolean[] b, int bFromIndex, 188 int length) { 189 int i = 0; 190 if (a[i] != b[i]) 191 return i; This is incorrect. You need to use aFromIndex and bFromIndex. Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. Paul. On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: Thanks David. Sending it to core-libs-dev. I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Could you please review and sponsor the patch. Link to bug: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ Regards, Vivek -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Monday, June 18, 2018 10:32 PM To: Deshpande, Vivek R >; jdk-dev at openjdk.java.net Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. Thanks, David On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: Hi All Forgot to add the links: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 0/ Regards. Vivek From: Deshpande, Vivek R Sent: Monday, June 18, 2018 4:50 PM To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' >; Viswanathan, Sandhya > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi All I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Please review and sponsor the patch. Regards, Vivek From stuart.marks at oracle.com Wed Jun 20 21:18:34 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 20 Jun 2018 14:18:34 -0700 Subject: RFR(s): JDK-8203184 List.copyOf() fails to copy sublists Message-ID: <46803faf-8d26-e407-d446-e4b27e683393@oracle.com> Hi all, Please review this small changeset that restores copy semantics to List.copyOf() when applied to a sublist of an unmodifiable list. Webrev: http://cr.openjdk.java.net/~smarks/reviews/8203184/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8203184 Thanks, s'marks From paul.sandoz at oracle.com Wed Jun 20 21:29:33 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Jun 2018 14:29:33 -0700 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> Message-ID: <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> 459 public static int mismatch(float[] a, int aFromIndex, 460 float[] b, int bFromIndex, 461 int length) { 462 int i = 0; 463 if (length > 1) { 464 if (a[aFromIndex] != b[bFromIndex]) { 465 i = 0; 466 } else { You fold the if/else to; if (a[aFromIndex] == b[bFromIndex]) { ? } same applies in other cases in both source files. > On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R wrote: > > Hi All > > I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. > Earlier webrev failed the same test. Thanks for confirming. > This webrev also modifies the BufferMismatch.java in similar way. Great! Paul. > Please take a look and let me know what you think. > > Updated webrev is here: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.01/ > I have also modified the bug with updated webrev. > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:57 AM > To: Paul Sandoz >; Roger.Riggs at oracle.com > Cc: David Holmes >; core-libs-dev at openjdk.java.net ; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Roger > > I will also test with zero length arrays and let you know. > Thanks for the input. > > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:17 AM > To: 'Paul Sandoz' > > Cc: David Holmes >; core-libs-dev at openjdk.java.net ; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. > I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. > > Regards, > Vivek > > From: Paul Sandoz [mailto:paul.sandoz at oracle.com ] > Sent: Tuesday, June 19, 2018 9:55 AM > To: Deshpande, Vivek R > > Cc: David Holmes >; core-libs-dev at openjdk.java.net ; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Thanks for investigating this. > > 164 public static int mismatch(boolean[] a, > 165 boolean[] b, > 166 int length) { > 167 int i = 0; > 168 if (a[i] != b[i]) > 169 return i; > You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. > > 186 public static int mismatch(boolean[] a, int aFromIndex, > 187 boolean[] b, int bFromIndex, > 188 int length) { > 189 int i = 0; > 190 if (a[i] != b[i]) > 191 return i; > This is incorrect. You need to use aFromIndex and bFromIndex. > > Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. > > Paul. > > > On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: > > Thanks David. > Sending it to core-libs-dev. > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Could you please review and sponsor the patch. > Link to bug: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ > > Regards, > Vivek > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com ] > Sent: Monday, June 18, 2018 10:32 PM > To: Deshpande, Vivek R >; jdk-dev at openjdk.java.net > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. > > Thanks, > David > > On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: > > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net ' > > Cc: 'Paul Sandoz' >; Viswanathan, Sandhya > > > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > From paul.sandoz at oracle.com Wed Jun 20 21:36:55 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Jun 2018 14:36:55 -0700 Subject: RFR(s): JDK-8203184 List.copyOf() fails to copy sublists In-Reply-To: <46803faf-8d26-e407-d446-e4b27e683393@oracle.com> References: <46803faf-8d26-e407-d446-e4b27e683393@oracle.com> Message-ID: +1 Paul. > On Jun 20, 2018, at 2:18 PM, Stuart Marks wrote: > > Hi all, > > Please review this small changeset that restores copy semantics to List.copyOf() when applied to a sublist of an unmodifiable list. > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8203184/webrev.0/ > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8203184 > > Thanks, > > s'marks From karen.kinnear at oracle.com Wed Jun 20 21:42:16 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 20 Jun 2018 17:42:16 -0400 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> Message-ID: Looks good. Can you send the updates to the valhalla spec experts please? We told them this was coming, and minor changes for clarification. thanks, Karen > On Jun 19, 2018, at 12:41 AM, David Holmes wrote: > > Discussions on the CSR request have led to further changes to the documentation involving nests and the new nest-related method. There are no semantic changes here just clearer explanations. > > Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ > > (don't worry if you don't see a v6, it didn't really exist). > > Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v7/ > > Specdiffs updated in place at: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/ > > Summary of changes: > > - The definition of a nest etc is moved to the class-level javadoc of java.lang.Class, along with some other edits provided by Alex Buckley to pave the way for future updates > - The nest-related methods are written in a more clear and consistent way > > Thanks, > David > ----- > > On 12/06/2018 3:16 PM, David Holmes wrote: >> Here is one further minor update from the CSR discussions: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html Thanks, >> David >> On 25/05/2018 3:52 PM, David Holmes wrote: >>> Here are the further minor updates so far in response to all the review comments. >>> >>> Incremental corelibs webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ >>> >>> Full corelibs webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ >>> >>> Change summary: >>> >>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>> - remove inaccurate pseudo-assertion comment >>> >>> test/jdk/java/lang/reflect/Nestmates/SampleNest.java >>> - code cleanup: <> operator >>> >>> test/jdk/java/lang/reflect/Nestmates/TestReflectionAPI.java >>> - code cleanup: streamify duplicate removals >>> >>> test/jdk/java/lang/invoke/PrivateInterfaceCall.java >>> - use consistent @bug number >>> >>> Thanks, >>> David >>> >>> On 22/05/2018 8:15 PM, David Holmes wrote: >>>> Here are the updates so far in response to all the review comments. >>>> >>>> Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2-incr/ >>>> >>>> Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2/ >>>> >>>> Change summary: >>>> >>>> src/java.base/share/classes/java/lang/Class.java >>>> - getNesthost: >>>> - change "any error" -> "any linkage error" as runtime errors will propagate. [This needs ratifying by EG] >>>> - add clarification that primitive and array classes are not explicitly members of any nest and so form singleton nests >>>> - add clarification that all nestmates are in the same package >>>> - re-word @return text to exclude the "royal 'we'" >>>> - fix javadoc cross references >>>> >>>> --- >>>> >>>> Moved reflection API tests from test/hotspot/jtreg/runtime/Nestmates/reflectionAPI/ to test/jdk/java/lang/reflect/Nestmates/ >>>> >>>> --- >>>> >>>> java/lang/reflect/Nestmates/TestReflectionAPI.java >>>> >>>> Run tests twice to show that failure reasons remain the same. >>>> >>>> --- >>>> >>>> test/jdk/jdk/lambda/vm/InterfaceAccessFlagsTest.java >>>> >>>> Disable test via annotation rather than commenting out. >>>> >>>> --- >>>> >>>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>>> >>>> - Fix indent for nestmate access check. >>>> - Remove unnecessary local variable >>>> >>>> --- >>>> >>>> src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >>>> >>>> - Replace myassert with proper assert >>>> >>>> --- >>>> >>>> Thanks, >>>> David >>>> >>>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>>> This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. This is the specific review thread for core-libs - webrev: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v1/ >>>>> >>>>> See below for full details - including annotated full webrev guiding the review. >>>>> >>>>> The intent is to have JEP-181 targeted and integrated by the end of this month. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> The nestmates project (JEP-181) introduces new classfile attributes to identify classes and interfaces in the same nest, so that the VM can perform access control based on those attributes and so allow direct private access between nestmates without requiring javac to generate synthetic accessor methods. These access control changes also extend to core reflection and the MethodHandle.Lookup contexts. >>>>> >>>>> Direct private calls between nestmates requires a more general calling context than is permitted by invokespecial, and so the JVMS is updated to allow, and javac updated to use, invokevirtual and invokeinterface for private class and interface method calls respectively. These changed semantics also extend to MethodHandle findXXX operations. >>>>> >>>>> At this time we are only concerned with static nest definitions, which map to a top-level class/interface as the nest-host and all its nested types as nest-members. >>>>> >>>>> Please see the JEP for further details. >>>>> >>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>> >>>>> All of the specification changes have been previously been worked out by the Valhalla Project Expert Group, and the implementation reviewed by the various contributors and discussed on the valhalla-dev mailing list. >>>>> >>>>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>>>> >>>>> Master webrev of all changes: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>> >>>>> Annotated master webrev index: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>> >>>>> Performance: this is expected to be performance neutral in a general sense. Benchmarking and performance runs are about to start. >>>>> >>>>> Testing Discussion: >>>>> ------------------ >>>>> >>>>> The testing for nestmates can be broken into four main groups: >>>>> >>>>> - New tests specifically related to nestmates and currently in the runtime/Nestmates directory >>>>> >>>>> - New tests to complement existing tests by adding in testcases not previously expressible. >>>>> - For example java/lang/invoke/SpecialInterfaceCall.java tests use of invokespecial for private interface methods and performing receiver typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do similar tests for invokeinterface. >>>>> >>>>> - New JVM TI tests to verify the spec changes related to nest attributes. >>>>> >>>>> - Existing tests significantly affected by the nestmates changes, primarily: >>>>> - runtime/SelectionResolution >>>>> >>>>> In most cases the nestmate changes makes certain invocations that were illegal, legal (e.g. not requiring invokespecial to invoke private interface methods; allowing access to private members via reflection/Methodhandles that were previously not allowed). >>>>> >>>>> - Existing tests incidentally affected by the nestmate changes >>>>> >>>>> This includes tests of things utilising class redefinition/retransformation to alter nested types but which unintentionally alter nest relationships (which is not permitted). >>>>> >>>>> There are still a number of tests problem-listed with issues filed against them to have them adapted to work with nestmates. Some of these are intended to be addressed in the short-term, while some (such as the runtime/SelectionResolution test changes) may not eventuate. >>>>> >>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>> >>>>> There is also further test work still to be completed (the JNI and JDI invocation tests): >>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>> which will continue in parallel with the main RFR. >>>>> >>>>> Pre-integration Testing: >>>>> - General: >>>>> - Mach5: hs/jdk tier1,2 >>>>> - Mach5: hs-nightly (tiers 1 -3) >>>>> - Targetted >>>>> - nashorn (for asm changes) >>>>> - hotspot: runtime/* >>>>> serviceability/* >>>>> compiler/* >>>>> vmTestbase/* >>>>> - jdk: java/lang/invoke/* >>>>> java/lang/reflect/* >>>>> java/lang/instrument/* >>>>> java/lang/Class/* >>>>> java/lang/management/* >>>>> - langtools: tools/javac >>>>> tools/javap >>>>> From vivek.r.deshpande at intel.com Thu Jun 21 00:05:55 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Thu, 21 Jun 2018 00:05:55 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> Hi Paul I have made the change you have suggested. Please find the updated webrev at this location: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.02/ Regards, Vivek From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Wednesday, June 20, 2018 2:30 PM To: Deshpande, Vivek R Cc: Roger.Riggs at oracle.com; David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. 459 public static int mismatch(float[] a, int aFromIndex, 460 float[] b, int bFromIndex, 461 int length) { 462 int i = 0; 463 if (length > 1) { 464 if (a[aFromIndex] != b[bFromIndex]) { 465 i = 0; 466 } else { You fold the if/else to; if (a[aFromIndex] == b[bFromIndex]) { ? } same applies in other cases in both source files. On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R > wrote: Hi All I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. Earlier webrev failed the same test. Thanks for confirming. This webrev also modifies the BufferMismatch.java in similar way. Great! Paul. Please take a look and let me know what you think. Updated webrev is here: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.01/ I have also modified the bug with updated webrev. Regards, Vivek From: Deshpande, Vivek R Sent: Tuesday, June 19, 2018 10:57 AM To: Paul Sandoz >; Roger.Riggs at oracle.com Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Roger I will also test with zero length arrays and let you know. Thanks for the input. Regards, Vivek From: Deshpande, Vivek R Sent: Tuesday, June 19, 2018 10:17 AM To: 'Paul Sandoz' > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. Regards, Vivek From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Tuesday, June 19, 2018 9:55 AM To: Deshpande, Vivek R > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Thanks for investigating this. 164 public static int mismatch(boolean[] a, 165 boolean[] b, 166 int length) { 167 int i = 0; 168 if (a[i] != b[i]) 169 return i; You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. 186 public static int mismatch(boolean[] a, int aFromIndex, 187 boolean[] b, int bFromIndex, 188 int length) { 189 int i = 0; 190 if (a[i] != b[i]) 191 return i; This is incorrect. You need to use aFromIndex and bFromIndex. Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. Paul. On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: Thanks David. Sending it to core-libs-dev. I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Could you please review and sponsor the patch. Link to bug: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ Regards, Vivek -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Monday, June 18, 2018 10:32 PM To: Deshpande, Vivek R >; jdk-dev at openjdk.java.net Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. Thanks, David On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: Hi All Forgot to add the links: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 0/ Regards. Vivek From: Deshpande, Vivek R Sent: Monday, June 18, 2018 4:50 PM To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' >; Viswanathan, Sandhya > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi All I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Please review and sponsor the patch. Regards, Vivek From ivan.gerasimov at oracle.com Thu Jun 21 00:23:39 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 20 Jun 2018 17:23:39 -0700 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> Message-ID: <01512723-0d61-560c-b8bf-6e0409437826@oracle.com> Hi Vivek! I think you don't need this if block: 464 if (a[aFromIndex] != b[bFromIndex]) { 465 i = 0; 466 } i is already 0 anyway, and the following line is just enough. 467 if (a[aFromIndex] == b[bFromIndex]) { The same applies to other float/double variants. With kind regards, Ivan On 6/20/18 5:05 PM, Deshpande, Vivek R wrote: > Hi Paul > > I have made the change you have suggested. > Please find the updated webrev at this location: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.02/ > > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Wednesday, June 20, 2018 2:30 PM > To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } else { > > You fold the if/else to; > > if (a[aFromIndex] == b[bFromIndex]) { > ? > } > > same applies in other cases in both source files. > > > > On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R > wrote: > > Hi All > > I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. > Earlier webrev failed the same test. > > Thanks for confirming. > > > This webrev also modifies the BufferMismatch.java in similar way. > > Great! > > Paul. > > > Please take a look and let me know what you think. > > Updated webrev is here: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.01/ > I have also modified the bug with updated webrev. > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:57 AM > To: Paul Sandoz >; Roger.Riggs at oracle.com > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Roger > > I will also test with zero length arrays and let you know. > Thanks for the input. > > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:17 AM > To: 'Paul Sandoz' > > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. > I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. > > Regards, > Vivek > > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Tuesday, June 19, 2018 9:55 AM > To: Deshpande, Vivek R > > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Thanks for investigating this. > > > 164 public static int mismatch(boolean[] a, > > 165 boolean[] b, > > 166 int length) { > > 167 int i = 0; > > 168 if (a[i] != b[i]) > > 169 return i; > You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. > > > 186 public static int mismatch(boolean[] a, int aFromIndex, > > 187 boolean[] b, int bFromIndex, > > 188 int length) { > > 189 int i = 0; > > 190 if (a[i] != b[i]) > > 191 return i; > This is incorrect. You need to use aFromIndex and bFromIndex. > > Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. > > Paul. > > On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: > > Thanks David. > Sending it to core-libs-dev. > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Could you please review and sponsor the patch. > Link to bug: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ > > Regards, > Vivek > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Monday, June 18, 2018 10:32 PM > To: Deshpande, Vivek R >; jdk-dev at openjdk.java.net > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. > > Thanks, > David > > On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net' > > Cc: 'Paul Sandoz' >; Viswanathan, Sandhya > > > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > -- With kind regards, Ivan Gerasimov From paul.sandoz at oracle.com Thu Jun 21 00:28:40 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Jun 2018 17:28:40 -0700 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> Message-ID: Hi Vivek, 459 public static int mismatch(float[] a, int aFromIndex, 460 float[] b, int bFromIndex, 461 int length) { 462 int i = 0; 463 if (length > 1) { 464 if (a[aFromIndex] != b[bFromIndex]) { 465 i = 0; 466 } 467 if (a[aFromIndex] == b[bFromIndex]) { Sorry, i don?t think i was clear. You can reduce down to a single equality check, so Lines #464-466 are redundant. This does result in taking the slow path if both values are NaN, which i think was the same for your previous patch. If that is important we can mitigate that by performing the negation of the same checks in the slow loop. WDYT? Paul. > On Jun 20, 2018, at 5:05 PM, Deshpande, Vivek R wrote: > > Hi Paul > > I have made the change you have suggested. > Please find the updated webrev at this location: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.02/ > > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com ] > Sent: Wednesday, June 20, 2018 2:30 PM > To: Deshpande, Vivek R > > Cc: Roger.Riggs at oracle.com ; David Holmes >; core-libs-dev at openjdk.java.net ; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } else { > > You fold the if/else to; > > if (a[aFromIndex] == b[bFromIndex]) { > ? > } > > same applies in other cases in both source files. > > > > On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R > wrote: > > Hi All > > I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. > Earlier webrev failed the same test. > > Thanks for confirming. > > > This webrev also modifies the BufferMismatch.java in similar way. > > Great! > > Paul. > > > Please take a look and let me know what you think. > > Updated webrev is here: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.01/ > I have also modified the bug with updated webrev. > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:57 AM > To: Paul Sandoz >; Roger.Riggs at oracle.com > Cc: David Holmes >; core-libs-dev at openjdk.java.net ; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Roger > > I will also test with zero length arrays and let you know. > Thanks for the input. > > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:17 AM > To: 'Paul Sandoz' > > Cc: David Holmes >; core-libs-dev at openjdk.java.net ; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. > I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. > > Regards, > Vivek > > From: Paul Sandoz [mailto:paul.sandoz at oracle.com ] > Sent: Tuesday, June 19, 2018 9:55 AM > To: Deshpande, Vivek R > > Cc: David Holmes >; core-libs-dev at openjdk.java.net ; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Thanks for investigating this. > > 164 public static int mismatch(boolean[] a, > 165 boolean[] b, > 166 int length) { > 167 int i = 0; > 168 if (a[i] != b[i]) > 169 return i; > You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. > > 186 public static int mismatch(boolean[] a, int aFromIndex, > 187 boolean[] b, int bFromIndex, > 188 int length) { > 189 int i = 0; > 190 if (a[i] != b[i]) > 191 return i; > This is incorrect. You need to use aFromIndex and bFromIndex. > > Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. > > Paul. > > > On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: > > Thanks David. > Sending it to core-libs-dev. > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Could you please review and sponsor the patch. > Link to bug: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ > > Regards, > Vivek > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com ] > Sent: Monday, June 18, 2018 10:32 PM > To: Deshpande, Vivek R >; jdk-dev at openjdk.java.net > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. > > Thanks, > David > > On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: > > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net ' > > Cc: 'Paul Sandoz' >; Viswanathan, Sandhya > > > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > From amaembo at gmail.com Thu Jun 21 04:33:51 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Thu, 21 Jun 2018 11:33:51 +0700 Subject: RFR: JDK-8205461 Create Collector which merges results of two other collectors Message-ID: Please review and sponsor: https://bugs.openjdk.java.net/browse/JDK-8205461 http://cr.openjdk.java.net/~tvaleev/webrev/8205461/r1/ See also previous discussion thread at [1]. It seems that we did not reach the final conclusion about the collector name, so in this review it's still named as "pairing" (proposed by me). Other name proposals: bisecting - by Paul Sandoz (sounds like input is split by two parts like in partitioningBy, which is not the case) tee or teeing - by Brian Goetz (IIUC from the T shape, it's assymmetrical operation, while our collector is symmetrical) duplex(ing) - by Jacob Glickman (well, probably ok?) bifurcate - by James Laskey (or bifurcating?) replicator - by James Laskey (as in DNA) replicating - by Peter Levart fanout or fanningOut - Chris Hegarty (sounds cryptic to me, checked Wikipedia [2], does not look like suitable) distributing - by Brian Goetz (like in distributive law a(b+c) = ab+ac; but common meaning of "distributing" is different) tapping - by Kirk Pepperdine (I have no associations; Google images shows some acupuncture procedures) split - by Kirk Pepperdine (may be confused with Spliterator) unzipping - by John Rose biMapping - by Zheka Kozlov (but he also suggests to change signature which is unnecessary) toBoth - by Peter Levart (but usually toXyz shows target container like toArray/toList/toSet, but there's not "Both" container) collectionToBothAndThen - by Zheka Kozlov (but too long) collectingToBoth - by Zheka Kozlov collectingTo - by Brian Goetz (isn't collect(collectingTo(...)) a little bit repititive?) biCollecting - by Zheka Kozlov expanding - by Paul Sandoz (doesn't sound very descriptive to me) forking - by Victor Williams Stafusa da Silva (could be thought that something is parallelized as forking is usually something connected to parallel computations) I think that it's better to concentrate not on the "splitting" part (each element is passed to two collectors), but on the "joining" part (results of two collectors are combined together). So my preference now is "merging" or "combining". If limiting the selection to the suggested options I'd prefer "duplexing" or "bifurcating" or to stay with my original version "pairing". Of course original Stream API author voices have more weight here, so if you insist on particular version, I will follow. By the way looking into CollectorsTest.java I found some minor things to cleanup: 1. `.map(mapper::apply)` and `.flatMap(mapper::apply)` can be replaced with simple `.map(mapper)` and `.flatMap(mapper)` respectively 2. In many methods redundant `throws ReflectiveOperationException` is declared while exception is never thrown 3. The `if (!Map.class.isAssignableFrom(map.getClass()))` looks useless as `map` is already of `Map` type, so this check is always false (we would never enter this method if it's true). Similarly `if (!List.class.isAssignableFrom(value.getClass()))` 4. Deprecated `clazz.newInstance()` is used 5. Test named `testGroupubgByWithReducing` should be renamed to `testGroupingByWithReducing` Should I fix some or all of these issues? As separate changeset or within this one? With best regards, Tagir Valeev. [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053718.html [2] https://en.wikipedia.org/wiki/Fan-out From Alan.Bateman at oracle.com Thu Jun 21 07:42:47 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Jun 2018 08:42:47 +0100 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> References: <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> Message-ID: <15af509e-3b2d-d938-9831-98e78e017685@oracle.com> On 20/06/2018 15:08, Peter Levart wrote: > > Like the following? > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ Yes, I think this looks good. > > Do we need a test which proves that cached direct buffers are released > when thread exits or would a unit test for TerminatingThreadLocal be > enough? Maybe both? It will be tested by code that uses NIO from threads that exit but I agree it would be good to add a unit test for this. -Alan From Alan.Bateman at oracle.com Thu Jun 21 07:52:03 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Jun 2018 08:52:03 +0100 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <5B2A9607.9010206@oracle.com> References: <5B286533.6070500@oracle.com> <3bce8034-387e-6ca5-d04d-b7171a13f47b@oracle.com> <5B2A9607.9010206@oracle.com> Message-ID: <422e546a-4bf4-e2a4-72cf-d4bc5be5abdf@oracle.com> On 20/06/2018 18:59, Erik Gahlin wrote: > : > >> Also all the methods are empty which makes me wonder if they should >> be abstract (as the class is abstract) or whether it should be an >> interface. > Abstract is needed to prevent user from instantiating the class. > > The methods need to have a body, otherwise event classes in java.base > would need to implement the methods, which would be cumbersome. We > like to use a class as it simplifies the implementation if we know all > event classes have a common ancestor with java.lang.Object as a parent. > > so we can be guaranteed that the class I'm not sure that I understand the issues here but just to say that jdk.internal.event is not exported so code outside of the JDK cannot directly instantiate instances of classes in that package. Also interfaces can have default methods which may go to your concern about needing to implement each of the 6 methods. Once Event is cleaned up with some javadoc then it will be easier to argue this one way or the other. -Alan From nishit.jain at oracle.com Thu Jun 21 08:58:46 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Thu, 21 Jun 2018 14:28:46 +0530 Subject: RFR JDK-8204938: Add a test case to automatically check the updated LSR data In-Reply-To: <82a5d19f-d89b-6b91-3a11-8100a4aed3a7@Oracle.com> References: <342415ad-b407-66f9-85a0-3f5aedc6f996@oracle.com> <82a5d19f-d89b-6b91-3a11-8100a4aed3a7@Oracle.com> Message-ID: <4d7ee619-4a3a-91df-a6c4-2fe8ef501749@oracle.com> Thanks Naoto, Roger for the review. Made the suggested changes and pushed to the repo. Regards, Nishit Jain On 21-06-2018 00:23, Roger Riggs wrote: > Hi Nishit, > > Looks ok > > My only comment would be to put the relative path to the > language-subtag-registry.txt in a private static final constant > to call attention to the dependency on the source layout instead of > burying it in the code. > > $.02, Roger > > > On 6/20/2018 2:27 PM, naoto.sato at oracle.com wrote: >> Looks good. >> >> Naoto >> >> On 6/20/18 3:32 AM, Nishit Jain wrote: >>> Hi, >>> >>> Please review the fix for JDK-8204938 >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204938 >>> Webrev: http://cr.openjdk.java.net/~nishjain/8204938/webrev.02/ >>> >>> Fix: Added a test case to cross check the LSR data generated for the >>> JDK APIs. So, for each lsr data update, the test case need not be >>> updated, it automatically cross checks the updated lsr data. >>> >>> Regards, >>> Nishit Jain > From david.holmes at oracle.com Thu Jun 21 09:08:28 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Jun 2018 19:08:28 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> Message-ID: <21cd2888-6c6d-9365-a575-e293c80d8835@oracle.com> Hi Karen, On 21/06/2018 7:42 AM, Karen Kinnear wrote: > Looks good. Thanks. Unfortunately there were a few more minor edits "overnight". Everything updated in place (apologies as I wanted to include a simple patch but overwrote things before realising I now had no means to do so.) Summary: - Class-level nest definition: use nestmates instead of members to avoid potential confusion/conflict with reference to "private members" "A _nest_ is a set of classes and interfaces... . The classes and interfaces are known as _nestmates_. One nestmate acts as the _nest host_, and enumerates the other nestmates which belong to the nest; each of them in turn records it as the nest host. ... - getNestHost - replace record withenumerate when referring to nest members (members record their nest host; nest hosts enumerate their members) -getNestMembers - replace @jvms ref with @see getNestHost > Can you send the updates to the valhalla spec experts please? We told them this was coming, and minor changes for > clarification. I already did: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-June/000710.html Thanks, David > > thanks, > Karen > >> On Jun 19, 2018, at 12:41 AM, David Holmes wrote: >> >> Discussions on the CSR request have led to further changes to the documentation involving nests and the new nest-related method. There are no semantic changes here just clearer explanations. >> >> Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ >> >> (don't worry if you don't see a v6, it didn't really exist). >> >> Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v7/ >> >> Specdiffs updated in place at: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/ >> >> Summary of changes: >> >> - The definition of a nest etc is moved to the class-level javadoc of java.lang.Class, along with some other edits provided by Alex Buckley to pave the way for future updates >> - The nest-related methods are written in a more clear and consistent way >> >> Thanks, >> David >> ----- >> >> On 12/06/2018 3:16 PM, David Holmes wrote: >>> Here is one further minor update from the CSR discussions: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html Thanks, >>> David >>> On 25/05/2018 3:52 PM, David Holmes wrote: >>>> Here are the further minor updates so far in response to all the review comments. >>>> >>>> Incremental corelibs webrev: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ >>>> >>>> Full corelibs webrev: >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ >>>> >>>> Change summary: >>>> >>>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>>> - remove inaccurate pseudo-assertion comment >>>> >>>> test/jdk/java/lang/reflect/Nestmates/SampleNest.java >>>> - code cleanup: <> operator >>>> >>>> test/jdk/java/lang/reflect/Nestmates/TestReflectionAPI.java >>>> - code cleanup: streamify duplicate removals >>>> >>>> test/jdk/java/lang/invoke/PrivateInterfaceCall.java >>>> - use consistent @bug number >>>> >>>> Thanks, >>>> David >>>> >>>> On 22/05/2018 8:15 PM, David Holmes wrote: >>>>> Here are the updates so far in response to all the review comments. >>>>> >>>>> Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2-incr/ >>>>> >>>>> Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2/ >>>>> >>>>> Change summary: >>>>> >>>>> src/java.base/share/classes/java/lang/Class.java >>>>> - getNesthost: >>>>> - change "any error" -> "any linkage error" as runtime errors will propagate. [This needs ratifying by EG] >>>>> - add clarification that primitive and array classes are not explicitly members of any nest and so form singleton nests >>>>> - add clarification that all nestmates are in the same package >>>>> - re-word @return text to exclude the "royal 'we'" >>>>> - fix javadoc cross references >>>>> >>>>> --- >>>>> >>>>> Moved reflection API tests from test/hotspot/jtreg/runtime/Nestmates/reflectionAPI/ to test/jdk/java/lang/reflect/Nestmates/ >>>>> >>>>> --- >>>>> >>>>> java/lang/reflect/Nestmates/TestReflectionAPI.java >>>>> >>>>> Run tests twice to show that failure reasons remain the same. >>>>> >>>>> --- >>>>> >>>>> test/jdk/jdk/lambda/vm/InterfaceAccessFlagsTest.java >>>>> >>>>> Disable test via annotation rather than commenting out. >>>>> >>>>> --- >>>>> >>>>> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >>>>> >>>>> - Fix indent for nestmate access check. >>>>> - Remove unnecessary local variable >>>>> >>>>> --- >>>>> >>>>> src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >>>>> >>>>> - Replace myassert with proper assert >>>>> >>>>> --- >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>>>> This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. This is the specific review thread for core-libs - webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v1/ >>>>>> >>>>>> See below for full details - including annotated full webrev guiding the review. >>>>>> >>>>>> The intent is to have JEP-181 targeted and integrated by the end of this month. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> The nestmates project (JEP-181) introduces new classfile attributes to identify classes and interfaces in the same nest, so that the VM can perform access control based on those attributes and so allow direct private access between nestmates without requiring javac to generate synthetic accessor methods. These access control changes also extend to core reflection and the MethodHandle.Lookup contexts. >>>>>> >>>>>> Direct private calls between nestmates requires a more general calling context than is permitted by invokespecial, and so the JVMS is updated to allow, and javac updated to use, invokevirtual and invokeinterface for private class and interface method calls respectively. These changed semantics also extend to MethodHandle findXXX operations. >>>>>> >>>>>> At this time we are only concerned with static nest definitions, which map to a top-level class/interface as the nest-host and all its nested types as nest-members. >>>>>> >>>>>> Please see the JEP for further details. >>>>>> >>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>> >>>>>> All of the specification changes have been previously been worked out by the Valhalla Project Expert Group, and the implementation reviewed by the various contributors and discussed on the valhalla-dev mailing list. >>>>>> >>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>>>>> >>>>>> Master webrev of all changes: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>> >>>>>> Annotated master webrev index: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>> >>>>>> Performance: this is expected to be performance neutral in a general sense. Benchmarking and performance runs are about to start. >>>>>> >>>>>> Testing Discussion: >>>>>> ------------------ >>>>>> >>>>>> The testing for nestmates can be broken into four main groups: >>>>>> >>>>>> - New tests specifically related to nestmates and currently in the runtime/Nestmates directory >>>>>> >>>>>> - New tests to complement existing tests by adding in testcases not previously expressible. >>>>>> - For example java/lang/invoke/SpecialInterfaceCall.java tests use of invokespecial for private interface methods and performing receiver typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do similar tests for invokeinterface. >>>>>> >>>>>> - New JVM TI tests to verify the spec changes related to nest attributes. >>>>>> >>>>>> - Existing tests significantly affected by the nestmates changes, primarily: >>>>>> - runtime/SelectionResolution >>>>>> >>>>>> In most cases the nestmate changes makes certain invocations that were illegal, legal (e.g. not requiring invokespecial to invoke private interface methods; allowing access to private members via reflection/Methodhandles that were previously not allowed). >>>>>> >>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>> >>>>>> This includes tests of things utilising class redefinition/retransformation to alter nested types but which unintentionally alter nest relationships (which is not permitted). >>>>>> >>>>>> There are still a number of tests problem-listed with issues filed against them to have them adapted to work with nestmates. Some of these are intended to be addressed in the short-term, while some (such as the runtime/SelectionResolution test changes) may not eventuate. >>>>>> >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>> >>>>>> There is also further test work still to be completed (the JNI and JDI invocation tests): >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>> which will continue in parallel with the main RFR. >>>>>> >>>>>> Pre-integration Testing: >>>>>> - General: >>>>>> - Mach5: hs/jdk tier1,2 >>>>>> - Mach5: hs-nightly (tiers 1 -3) >>>>>> - Targetted >>>>>> - nashorn (for asm changes) >>>>>> - hotspot: runtime/* >>>>>> serviceability/* >>>>>> compiler/* >>>>>> vmTestbase/* >>>>>> - jdk: java/lang/invoke/* >>>>>> java/lang/reflect/* >>>>>> java/lang/instrument/* >>>>>> java/lang/Class/* >>>>>> java/lang/management/* >>>>>> - langtools: tools/javac >>>>>> tools/javap >>>>>> > From markus.gronlund at oracle.com Thu Jun 21 10:03:13 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 21 Jun 2018 03:03:13 -0700 (PDT) Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <422e546a-4bf4-e2a4-72cf-d4bc5be5abdf@oracle.com> References: <5B286533.6070500@oracle.com> <3bce8034-387e-6ca5-d04d-b7171a13f47b@oracle.com> <5B2A9607.9010206@oracle.com> <422e546a-4bf4-e2a4-72cf-d4bc5be5abdf@oracle.com> Message-ID: Hi Alan, Some context might be helpful here: The real JFR API is located in module jdk.jfr and the API handle there is this class jdk.jfr.Event [1] There are multiple reasons for choosing an class as the API instead of an interface - most of them are driven by implementation (in the VM) and performance related. As you can see in [1], the methods we are discussing here are there declared "final". This reflects the fact that we don't want user versions of these methods. We do want the signatures and the schema however so we can rework classes and methods in the VM. Using a class hierarchy allows us via induction quickly decide (bit check) if a loading class is related to the API (which cannot be done as performant should the solution be based on an interface) and we want to work with invokevirtual and invokespecial but not so much invokeinterface. So this solution we are discussing here is about how we can extend what we already have in place for JFR to also allow code in java.base to use it. The main challenge here is with solving the module system inversion in that java.base can't have a compile dependency on the real JFR API located in jdk.jfr: With the solution proposed by Erik, we extend our JFR VM machinery to also treat jdk.internal.events.Event in the same way as jdk.jfr.Event. Unfortunately, in order to do this, we now can't make the methods in jdk.internal.events.Event "final" since we will need inherit it from jdk.jfr.Event (which implement these methods already in the API proper). Also, there now needs to be careful means in the VM of avoiding jdk.jfr module related symbols when processing java.base events in certain states (for example, a module read link not having been established (yet or at all) etc). I don?t think we should view jdk.internal.events.Event as an general API but instead it is a means of hooking into the JFR system at runtime, via the VM, avoiding the compile time dependency. "Regular code" should work with the jdk.jfr module proper if possible, but for situations that cannot handle that / don't want to add the module import for some reason, this is a JDK internal hook point, that can be used if needed. This is also the reason why it is not exported to other modules in general, it is mainly intended for java.base, since the inversion leave no other alternatives (except moving all jfr code into java.base which I think is probably not an option). In addition, nothing in jdk.internal.events.Event strongly couple with jdk.jfr which means it may be put to some other use for alternative implementations that do not have a JFR implementation say. Hope this clarifies a bit Thanks Markus [1] https://docs.oracle.com/javase/9/docs/api/jdk/jfr/Event.html -----Original Message----- From: Alan Bateman Sent: den 21 juni 2018 09:52 To: Erik Gahlin ; hotspot-jfr-dev at openjdk.java.net; core-libs-dev Subject: Re: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr On 20/06/2018 18:59, Erik Gahlin wrote: > : > >> Also all the methods are empty which makes me wonder if they should >> be abstract (as the class is abstract) or whether it should be an >> interface. > Abstract is needed to prevent user from instantiating the class. > > The methods need to have a body, otherwise event classes in java.base > would need to implement the methods, which would be cumbersome. We > like to use a class as it simplifies the implementation if we know all > event classes have a common ancestor with java.lang.Object as a parent. > > so we can be guaranteed that the class I'm not sure that I understand the issues here but just to say that jdk.internal.event is not exported so code outside of the JDK cannot directly instantiate instances of classes in that package. Also interfaces can have default methods which may go to your concern about needing to implement each of the 6 methods. Once Event is cleaned up with some javadoc then it will be easier to argue this one way or the other. -Alan From david.holmes at oracle.com Thu Jun 21 10:33:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Jun 2018 20:33:32 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> <74a03d23-1e3a-b4c4-4eba-121e0f8ca545@oracle.com> Message-ID: <9967b9ad-ea5b-5185-a6a4-33b4c2b6ad6e@oracle.com> Thanks Paul! David On 21/06/2018 3:43 AM, Paul Sandoz wrote: > > >> On Jun 19, 2018, at 10:25 PM, mandy chung wrote: >> >> The javadoc update looks good to me. >> > > Same here, +1 > > Paul. > From david.holmes at oracle.com Thu Jun 21 10:34:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Jun 2018 20:34:13 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <42093BA6-90C6-4FE1-AB3F-1AA13CDC8FC9@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9ff7d13a-6b9c-8394-b29b-10796d0c4ac5@oracle.com> <42093BA6-90C6-4FE1-AB3F-1AA13CDC8FC9@oracle.com> Message-ID: <15989e1f-cf8e-ca57-9428-bd2a241e8282@oracle.com> Thanks John! On 21/06/2018 6:15 AM, John Rose wrote: > Good; I like the care to distinguish "nested" classes (using the > word "enclosing") from the newer concept of "nests" and "nestmates", > as well as the careful framing of how stuff bubbles up from the classfile. > > (Kudos to Alex!) Yes this is much improved. David > ? John > > On Jun 19, 2018, at 9:56 PM, David Holmes > wrote: >> >> Some further adjustments to getNestMembers() was made. Everything >> updated in place. >> >> Thanks, >> David >> >> On 20/06/2018 9:30 AM, David Holmes wrote: >>> Sorry another update is imminent ... stay tuned. >>> David >>> On 19/06/2018 2:41 PM, David Holmes wrote: >>>> Discussions on the CSR request have led to further changes to the >>>> documentation involving nests and the new nest-related method. There >>>> are no semantic changes here just clearer explanations. >>>> >>>> Incremental >>>> webrev:http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v7-incr/ >>>> > From dave_hobbs at uk.ibm.com Thu Jun 21 10:41:23 2018 From: dave_hobbs at uk.ibm.com (Dave Hobbs) Date: Thu, 21 Jun 2018 11:41:23 +0100 Subject: Creating a charset provider module for IBM charsets In-Reply-To: <0465abd4-b087-263c-baf9-eedbdf1d96fb@oracle.com> References: <64173406-ab24-1620-a029-5b90e0acddb3@oracle.com> <0465abd4-b087-263c-baf9-eedbdf1d96fb@oracle.com> Message-ID: Yes, the proposal is to create a new module outside the JDK. I was originally responding to your suggestion of not including these charsets in java.base. http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053316.html "Separately, I think we should start a discussion here about moving some or all of the IBM charsets to their own service provider module. I realize the AIX port might want to include some of them in its build of java.base but they aren't interesting to include in java.base, or even jdk.charsets, on most platforms" A service provider module appears to be the recommended approach according the Java api documentation. However writing charset classes is made a lot more difficult by not having the sun.nio.cs classes available. Should we propose adding our charsets in the java.base or jdk.charsets? - Dave From: Alan Bateman To: Dave Hobbs , core-libs-dev at openjdk.java.net Date: 20/06/2018 18:27 Subject: Re: Creating a charset provider module for IBM charsets On 20/06/2018 15:52, Dave Hobbs wrote: > Hi Alan, > > My understanding is that java.base only exports sun.nio.cs to jdk.charsets > i.e java.base module-info.java has: > > ... > exports sun.nio.cs to > java.desktop, > jdk.charsets; > ... > > and jdk.charsets has: > > module jdk.charsets { > provides java.nio.charset.spi.CharsetProvider with > sun.nio.cs.ext.ExtendedCharsets; > } > > So jdk.charsets can use sun.nio.cs, whereas my module can not. > > Am I missing something? > Are you proposing to add your module to the JDK or will it live outside? If the former then just extend the qualified export to add the name of the module. If the latter then you are out of luck as the sun.nio.cs infrastructure is not meant to be used outside of the JDK. -Alan Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Pengfei.Li at arm.com Thu Jun 21 10:57:40 2018 From: Pengfei.Li at arm.com (Pengfei Li) Date: Thu, 21 Jun 2018 10:57:40 +0000 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> Message-ID: Hi Bernard, Yes, your fix is good but would be nicer if the comment in line 733 is modified as well since it might be misleading. > 733 /* On AIX, readdir() returns EBADF ... I only have Linux machines to test. But I suggest that your patch being merged soon as the deprecated use breaks the build on many latest Linux distributions, and it's hard for guys to find other POSIX platforms in a short time. -- Thanks, Pengfei -----Original Message----- From: B. Blaser Sent: Thursday, June 21, 2018 00:01 To: Pengfei Li Cc: kim.barrett at oracle.com; core-libs-dev at openjdk.java.net; nd ; build-dev at openjdk.java.net Subject: Re: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated Hi Pengfei, On 20 June 2018 at 12:08, Pengfei Li wrote: > Hi > > I have tried the patch ( http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/052991.html ) on Ubuntu 18.04 machines (x86_64 & aarch64) with glibc=2.26.1 and build is OK. > > There's a small issue within the following code in > src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c > Unlike readdir_r(), readdir() does not return int value. The value of res is always 0 before #ifdef. > > 727 /* EINTR not listed as a possible error */ > 728 errno = 0; > 729 ptr = readdir64(dirp); > 730 res = errno; > 731 > 732 #ifdef _AIX > 733 /* On AIX, readdir() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ > 734 /* directory stream end. Otherwise, 'errno' will contain the error code. */ > 735 if (res != 0) { > 736 res = (ptr == NULL && res == EBADF) ? 0 : errno; > 737 } > 738 #endif > > May I know that if this core-libs change going to be merged recently, or more platforms needs to be explored? > > -- > Thanks, > Pengfei Thanks for evaluating this patch, 'readdir()' doesn't return an 'int' value but it sets 'errno' instead which is then assigned to 'res'. So, I guess the fix is OK, or did I miss anything? I've also tested it on Linux/x86_64 but ideally, the patch should be tested on all supported POSIX platforms before being integrated. The JBS issue [1] doesn't mention any fix version, so I'm not sure if it'll be targeted to 11 or 12. But if someone is available to test it on other POSIX platforms and review it, this would be nice? Thanks, Bernard [1] https://bugs.openjdk.java.net/browse/JDK-8202794 From peter.levart at gmail.com Thu Jun 21 11:07:56 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Jun 2018 13:07:56 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> Message-ID: <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> On 06/20/2018 04:24 PM, Tony Printezis wrote: > Hey Peter, > > I had written a test to test my version of the exit hooks. I can > easily rework it to work with yours. Interested? Of course. Just give what you got. I can modify it as needed. > > To check that the native buffers are reclaimed promptly, we should be > able to amend the test that tests the setting of the max size for the > cached buffers (i.e., check that, after all the threads have exited, > the total native count / size is 0). Good idea. Do you happen to know which one is it? > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On June 20, 2018 at 10:08:48 AM, Peter Levart (peter.levart at gmail.com > ) wrote: > >> >> On 06/18/2018 05:41 PM, Alan Bateman wrote: >> > >> > >> > On 17/06/2018 22:20, Peter Levart wrote: >> >> Update: the discussion on concurrent-interest about possible future >> >> addition of public ThreadLocal.getIfPresent() or >> >> ThreadLocal.isPresent() has died out, but it nevertheless reached a >> >> point where ThreadLocal.isPresent() was found the least problematic. >> >> So I would like to get further with this proposal using the last >> >> webrev.04 version of the patch that uses ThreadLocal.isPresent() >> >> internally - still package-private at the moment. >> >> >> >> Here's the webrev (unchanged from the day it was published): >> >> >> >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ >> >> >> >> >> >> Would this version be OK? >> > I think looks quite good. >> > >> > One small nit is that the update to ThreadLocal.setInitialValue makes >> > it look like all locals are registered when setting the initial value. >> > What would you think about moving the instanceof check from >> > TerminatingThreadLocal.register so that it's a bit more obvious. >> > >> > -Alan >> >> Like the following? >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ >> >> >> >> >> Do we need a test which proves that cached direct buffers are released >> when thread exits or would a unit test for TerminatingThreadLocal be >> enough? Maybe both? >> >> Regards, Peter >> From oliver at agilechilli.com Thu Jun 21 11:59:27 2018 From: oliver at agilechilli.com (Oliver Kohll) Date: Thu, 21 Jun 2018 04:59:27 -0700 Subject: free(): corrupted unsorted chunks Message-ID: Hi, I'm a Java developer getting a crash and error message 'free(): corrupted unsorted chunks' in the log. this sounds like it could be a low level issue in the core Java libraries so I'm posting here, but if it would be better in another list (or even a different community) please let me know. So I'm requesting help because I don't know how to debug this further. In summary, the issue is We're getting a crash nearly once a day on our production system, at a time of day when it's most busy (mid morning). This crash hasn't been seen on our development environment, which is the same apart from the server has less memory and CPU resources. The environment is: * A VPS host * Ubuntu LTS 18 (Bionic) * The default Java 10 packaged as 11 I believe ( https://askubuntu.com/questions/1037646/why-is-openjdk-10-packaged-as-openjdk-11 ) * However, javac seems to be from JDK8: root at server:~# file /etc/alternatives/java /etc/alternatives/javac /etc/alternatives/java: symbolic link to /usr/lib/jvm/java-11-openjdk-amd64/bin/java /etc/alternatives/javac: symbolic link to /usr/lib/jvm/java-8-openjdk-amd64/bin/javac We're running our webapp on Tomcat 8 (also the Ubuntu default). When this happens, the server stops responding to HTTP requests (in fact the java process has quit) and needs to be restarted. The crash doesn't seem to correspond to a particular activity on the server as far as I can see, though it's hard to tell as many people are logged in at the same time. Does anyone have any pointers of a) how to track the bug further so I can report it, ideally without a big impact on our live server b) more immediately, any workarounds to try In the meantime I will see if I can downgrade to JDK8 on the test server, but I'm not sure if that's possible or whether it would make a difference. Regards Oliver From thomas.stuefe at gmail.com Thu Jun 21 12:28:45 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 21 Jun 2018 14:28:45 +0200 Subject: free(): corrupted unsorted chunks In-Reply-To: References: Message-ID: Hi Oliver, Is there a stack trace on stderr? Thanks, Thomas On Thu, Jun 21, 2018 at 1:59 PM, Oliver Kohll wrote: > Hi, > > I'm a Java developer getting a crash and error message 'free(): corrupted > unsorted chunks' in the log. this sounds like it could be a low level issue > in the core Java libraries so I'm posting here, but if it would be better > in another list (or even a different community) please let me know. > > So I'm requesting help because I don't know how to debug this further. In > summary, the issue is > > We're getting a crash nearly once a day on our production system, at a time > of day when it's most busy (mid morning). This crash hasn't been seen on > our development environment, which is the same apart from the server has > less memory and CPU resources. > > The environment is: > * A VPS host > * Ubuntu LTS 18 (Bionic) > * The default Java 10 packaged as 11 I believe ( > https://askubuntu.com/questions/1037646/why-is-openjdk-10-packaged-as-openjdk-11 > ) > * However, javac seems to be from JDK8: > root at server:~# file /etc/alternatives/java /etc/alternatives/javac > /etc/alternatives/java: symbolic link to > /usr/lib/jvm/java-11-openjdk-amd64/bin/java > /etc/alternatives/javac: symbolic link to > /usr/lib/jvm/java-8-openjdk-amd64/bin/javac > > We're running our webapp on Tomcat 8 (also the Ubuntu default). When this > happens, the server stops responding to HTTP requests (in fact the java > process has quit) and needs to be restarted. > > The crash doesn't seem to correspond to a particular activity on the server > as far as I can see, though it's hard to tell as many people are logged in > at the same time. > > Does anyone have any pointers of > a) how to track the bug further so I can report it, ideally without a big > impact on our live server > b) more immediately, any workarounds to try > > In the meantime I will see if I can downgrade to JDK8 on the test server, > but I'm not sure if that's possible or whether it would make a difference. > > Regards > Oliver From martinrb at google.com Thu Jun 21 14:47:17 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 21 Jun 2018 07:47:17 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: On Tue, Jun 19, 2018 at 6:06 AM, David Lloyd wrote: > It may be confusing (to some, I guess) but it is consistent: the > ThreadLocal subclass author has absolute control over how the value is > presented to the user. Adding compute() or many of the other > suggested variants will break that guarantee, which seems like kind of > a big deal to me. > > Yeah, that's a good point I didn't consider. Any new method tends to expose presence/absence and evade subclass' checking. OTOH I think the same has happened with default methods added to existing interfaces, including Map, and I don't recall complaints about the failure of encapsulation. In this way, again ThreadLocal.compute would be very much like Map.compute. > What about introducing a different thread local API that has more > modern behavior? Maybe a new subclass of ThreadLocal? > > The hard part is coming up with a good name. ThreadLocalNowWithExtraMappiness ? If we create a new class, we can consider giving get() non-mutating behavior consistent with Map. From oliver at agilechilli.com Thu Jun 21 15:06:51 2018 From: oliver at agilechilli.com (Oliver Kohll) Date: Thu, 21 Jun 2018 08:06:51 -0700 Subject: free(): corrupted unsorted chunks In-Reply-To: References: Message-ID: Hi Thomas, Unfortunately not, that was /var/log/tomcat8/catalina.out, a snippet from just before to afterwards when I restarted is below. Everything after 'free(): corrupted unsorted chunks', from 'Note: Picked up JDK_JAVA_OPTIONS' on was only logged on restart. The warnings are normal. The lines before were from my application. However they're different for each crash. Thanks Oliver --- Thu 2018/06/21 09:29:19.913|Warn|Scheduler|View CTG - (Fitting Sync) has already been scheduled again, shouldn't be reset Thu 2018/06/21 09:29:21.639|Info|ReportDownloader|User [username] exporting report GMP from table n0) ff - GMP Thu 2018/06/21 09:29:23.506|Info|Scheduler|Starting action for googlesyncer view b4) Fitting Dates (Remedial Sync) Thu 2018/06/21 09:29:23.911|Info|Scheduler|Starting action for googlesyncer view b5) Survey Dates (Survey Sync) Thu 2018/06/21 09:29:23.914|Warn|GoogleSyncer|1 other calendar syncs taking place, delaying for a minute Thu 2018/06/21 09:29:23.941|Warn|Scheduler|View (Survey Sync) has already been scheduled again, shouldn't be reset Thu 2018/06/21 09:29:24.106|Info|Scheduler|Starting action for workflowrunner view m1.0) Customer Order.orders to lock Thu 2018/06/21 09:29:24.338|Info|Scheduler|Starting action for googlesyncer view a) leads.AGS (Lead Sync) *free(): corrupted unsorted chunks* NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 21-Jun-2018 09:35:56.096 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/common/classes], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.102 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/common], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.102 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/common/classes], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.102 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/common], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.104 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/server/classes], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.104 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/server], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.105 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/server/classes], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.105 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/server], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.105 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/shared/classes], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.105 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/shared], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.106 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/shared/classes], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.106 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/shared], exists: [false], isDirectory: [false], canRead: [false] 21-Jun-2018 09:35:56.795 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property. 21-Jun-2018 09:35:56.806 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context/Realm/Realm} Setting property 'debug' to '99' did not find a matching property. 21-Jun-2018 09:35:56.807 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context/Realm/Realm} Setting property 'digest' to 'MD5' did not find a matching property. 21-Jun-2018 09:35:56.809 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.5.30 (Ubuntu) 21-Jun-2018 09:35:56.810 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: May 30 2018 13:37:13 UTC 21-Jun-2018 09:35:56.810 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number: 8.5.30.0 21-Jun-2018 09:35:56.810 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux 21-Jun-2018 09:35:56.810 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 4.15.13-x86_64-linode106 21-Jun-2018 09:35:56.811 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64 21-Jun-2018 09:35:56.811 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/lib/jvm/java-11-openjdk-amd64 21-Jun-2018 09:35:56.811 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 10.0.1+10-Ubuntu-3ubuntu1 21-Jun-2018 09:35:56.811 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Oracle Corporation 21-Jun-2018 09:35:56.811 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /var/lib/tomcat8 21-Jun-2018 09:35:56.812 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /usr/share/tomcat8 21-Jun-2018 09:35:56.812 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED 21-Jun-2018 09:35:56.812 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED 21-Jun-2018 09:35:56.812 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 21-Jun-2018 09:35:56.812 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/var/lib/tomcat8/conf/logging.properties 21-Jun-2018 09:35:56.813 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 21-Jun-2018 09:35:56.813 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true 21-Jun-2018 09:35:56.813 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx8g 21-Jun-2018 09:35:56.813 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048 21-Jun-2018 09:35:56.813 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources 21-Jun-2018 09:35:56.813 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 21-Jun-2018 09:35:56.813 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs= 21-Jun-2018 09:35:56.814 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/var/lib/tomcat8 21-Jun-2018 09:35:56.814 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/share/tomcat8 21-Jun-2018 09:35:56.814 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/tmp/tomcat8-tomcat8-tmp 21-Jun-2018 09:35:56.814 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded APR based Apache Tomcat Native library [1.2.16] using APR version [1.6.3]. 21-Jun-2018 09:35:56.814 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true]. 21-Jun-2018 09:35:56.814 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true] 21-Jun-2018 09:35:56.818 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 1.1.0g 2 Nov 2017] 21-Jun-2018 09:35:56.858 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"] 21-Jun-2018 09:35:56.876 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read 21-Jun-2018 09:35:56.884 INFO [main] org.apache.coyote.http11.AbstractHttp11Protocol.configureUpgradeProtocol The ["https-openssl-apr-8443"] connector has been configured to support negotiation to [h2] via ALPN 21-Jun-2018 09:35:56.884 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-openssl-apr-8443"] 21-Jun-2018 09:35:56.926 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 773 ms 21-Jun-2018 09:35:56.997 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina] 21-Jun-2018 09:35:56.998 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.30 (Ubuntu) On 21 June 2018 at 15:31:12, Thomas St?fe (thomas.stuefe at gmail.com) wrote: Whereever you got the "free(): corrupted unsorted chunks" from should contain actually more information. For an example, see: https://www.unix.com/red-hat/233292-free-corrupted-unsorted-chunks.html Thanks, Thomas On Thu, Jun 21, 2018 at 4:27 PM, Oliver Kohll wrote: > Not that I can see I'm afraid - there's nothing else in either > /var/log/tomcat8/catalina.out (or any of the other files in that directory) > or /var/log/syslog. I also can't see anything likely in /tmp > > I'm Googling how to get tomcat to create a stack trace to a file on a crash, > but the consensus seems to be they should be in catalina.out by default. > > On 21 June 2018 at 13:28:46, Thomas St?fe (thomas.stuefe at gmail.com) wrote: > > Hi Oliver, > > Is there a stack trace on stderr? > > Thanks, Thomas > > > > > > On Thu, Jun 21, 2018 at 1:59 PM, Oliver Kohll > wrote: >> Hi, >> >> I'm a Java developer getting a crash and error message 'free(): corrupted >> unsorted chunks' in the log. this sounds like it could be a low level >> issue >> in the core Java libraries so I'm posting here, but if it would be better >> in another list (or even a different community) please let me know. >> >> So I'm requesting help because I don't know how to debug this further. In >> summary, the issue is >> >> We're getting a crash nearly once a day on our production system, at a >> time >> of day when it's most busy (mid morning). This crash hasn't been seen on >> our development environment, which is the same apart from the server has >> less memory and CPU resources. >> >> The environment is: >> * A VPS host >> * Ubuntu LTS 18 (Bionic) >> * The default Java 10 packaged as 11 I believe ( >> >> https://askubuntu.com/questions/1037646/why-is-openjdk-10-packaged-as-openjdk-11 >> ) >> * However, javac seems to be from JDK8: >> root at server:~# file /etc/alternatives/java /etc/alternatives/javac >> /etc/alternatives/java: symbolic link to >> /usr/lib/jvm/java-11-openjdk-amd64/bin/java >> /etc/alternatives/javac: symbolic link to >> /usr/lib/jvm/java-8-openjdk-amd64/bin/javac >> >> We're running our webapp on Tomcat 8 (also the Ubuntu default). When this >> happens, the server stops responding to HTTP requests (in fact the java >> process has quit) and needs to be restarted. >> >> The crash doesn't seem to correspond to a particular activity on the >> server >> as far as I can see, though it's hard to tell as many people are logged in >> at the same time. >> >> Does anyone have any pointers of >> a) how to track the bug further so I can report it, ideally without a big >> impact on our live server >> b) more immediately, any workarounds to try >> >> In the meantime I will see if I can downgrade to JDK8 on the test server, >> but I'm not sure if that's possible or whether it would make a difference. >> >> Regards >> Oliver From peter.levart at gmail.com Thu Jun 21 15:53:11 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Jun 2018 17:53:11 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <15af509e-3b2d-d938-9831-98e78e017685@oracle.com> References: <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <15af509e-3b2d-d938-9831-98e78e017685@oracle.com> Message-ID: Hi, On 06/21/2018 09:42 AM, Alan Bateman wrote: > > > On 20/06/2018 15:08, Peter Levart wrote: >> >> Like the following? >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ > Yes, I think this looks good. > >> >> Do we need a test which proves that cached direct buffers are >> released when thread exits or would a unit test for >> TerminatingThreadLocal be enough? Maybe both? > It will be tested by code that uses NIO from threads that exit but I > agree it would be good to add a unit test for this. > > -Alan Here's the same patch, with tests added: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.06/ Is this good to go? Should I submit the code to testing system first? Please point me to instructions (have not done that yet)... Regards, Peter From peter.levart at gmail.com Thu Jun 21 15:58:43 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Jun 2018 17:58:43 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <15af509e-3b2d-d938-9831-98e78e017685@oracle.com> Message-ID: <8a55c1a0-06e3-510d-15ae-634d6ca089df@gmail.com> On 06/21/2018 05:53 PM, Peter Levart wrote: > Hi, > > On 06/21/2018 09:42 AM, Alan Bateman wrote: >> >> >> On 20/06/2018 15:08, Peter Levart wrote: >>> >>> Like the following? >>> >>> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ >> Yes, I think this looks good. >> >>> >>> Do we need a test which proves that cached direct buffers are >>> released when thread exits or would a unit test for >>> TerminatingThreadLocal be enough? Maybe both? >> It will be tested by code that uses NIO from threads that exit but I >> agree it would be good to add a unit test for this. >> >> -Alan > > Here's the same patch, with tests added: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.06/ > > Is this good to go? Should I submit the code to testing system first? > Please point me to instructions (have not done that yet)... > > Regards, Peter > It just occurred to me that TempDirectBuffersReclamation test may fail if there is concurrent activity in the VM during or even before the test (for example if GC/reference processing frees some direct buffers during the test as a consequence of activity before the test). How to cope with that? Would running the test in separate VM be enough? Regards, Peter From tprintezis at twitter.com Thu Jun 21 16:14:24 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 21 Jun 2018 09:14:24 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> References: <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> Message-ID: Peter, Attached TerminatingThreadLocalTest.java. Let me know what you think (and feel free to modify it / discard it if you don?t like it!). Re: The test for the max cached buffer size is: test/jdk/sun/nio/ch/TestMaxCachedBufferSize.java. I looked at it and, unfortunately, ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 7:07:59 AM, Peter Levart (peter.levart at gmail.com) wrote: On 06/20/2018 04:24 PM, Tony Printezis wrote: Hey Peter, I had written a test to test my version of the exit hooks. I can easily rework it to work with yours. Interested? Of course. Just give what you got. I can modify it as needed. To check that the native buffers are reclaimed promptly, we should be able to amend the test that tests the setting of the max size for the cached buffers (i.e., check that, after all the threads have exited, the total native count / size is 0). Good idea. Do you happen to know which one is it? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 20, 2018 at 10:08:48 AM, Peter Levart (peter.levart at gmail.com) wrote: On 06/18/2018 05:41 PM, Alan Bateman wrote: > > > On 17/06/2018 22:20, Peter Levart wrote: >> Update: the discussion on concurrent-interest about possible future >> addition of public ThreadLocal.getIfPresent() or >> ThreadLocal.isPresent() has died out, but it nevertheless reached a >> point where ThreadLocal.isPresent() was found the least problematic. >> So I would like to get further with this proposal using the last >> webrev.04 version of the patch that uses ThreadLocal.isPresent() >> internally - still package-private at the moment. >> >> Here's the webrev (unchanged from the day it was published): >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ >> >> Would this version be OK? > I think looks quite good. > > One small nit is that the update to ThreadLocal.setInitialValue makes > it look like all locals are registered when setting the initial value. > What would you think about moving the instanceof check from > TerminatingThreadLocal.register so that it's a bit more obvious. > > -Alan Like the following? http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ Do we need a test which proves that cached direct buffers are released when thread exits or would a unit test for TerminatingThreadLocal be enough? Maybe both? Regards, Peter From tprintezis at twitter.com Thu Jun 21 16:17:57 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 21 Jun 2018 09:17:57 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> Message-ID: (I unfortunately pressed Send accidentally; apologies) I was saying: I looked at TestMaxCachedBufferSize and, unfortunately, I don?t think the test makes a lot of sense right now as it checks the number / size of direct buffers after all the threads terminate and, with this change, that should always be 0. Let me look into this and I?ll get back to you in a bit. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 12:14:24 PM, Tony Printezis (tprintezis at twitter.com) wrote: Peter, Attached TerminatingThreadLocalTest.java. Let me know what you think (and feel free to modify it / discard it if you don?t like it!). Re: The test for the max cached buffer size is: test/jdk/sun/nio/ch/TestMaxCachedBufferSize.java. I looked at it and, unfortunately, ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 7:07:59 AM, Peter Levart (peter.levart at gmail.com) wrote: On 06/20/2018 04:24 PM, Tony Printezis wrote: Hey Peter, I had written a test to test my version of the exit hooks. I can easily rework it to work with yours. Interested? Of course. Just give what you got. I can modify it as needed. To check that the native buffers are reclaimed promptly, we should be able to amend the test that tests the setting of the max size for the cached buffers (i.e., check that, after all the threads have exited, the total native count / size is 0). Good idea. Do you happen to know which one is it? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 20, 2018 at 10:08:48 AM, Peter Levart (peter.levart at gmail.com) wrote: On 06/18/2018 05:41 PM, Alan Bateman wrote: > > > On 17/06/2018 22:20, Peter Levart wrote: >> Update: the discussion on concurrent-interest about possible future >> addition of public ThreadLocal.getIfPresent() or >> ThreadLocal.isPresent() has died out, but it nevertheless reached a >> point where ThreadLocal.isPresent() was found the least problematic. >> So I would like to get further with this proposal using the last >> webrev.04 version of the patch that uses ThreadLocal.isPresent() >> internally - still package-private at the moment. >> >> Here's the webrev (unchanged from the day it was published): >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ >> >> Would this version be OK? > I think looks quite good. > > One small nit is that the update to ThreadLocal.setInitialValue makes > it look like all locals are registered when setting the initial value. > What would you think about moving the instanceof check from > TerminatingThreadLocal.register so that it's a bit more obvious. > > -Alan Like the following? http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ Do we need a test which proves that cached direct buffers are released when thread exits or would a unit test for TerminatingThreadLocal be enough? Maybe both? Regards, Peter From tprintezis at twitter.com Thu Jun 21 16:21:35 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 21 Jun 2018 09:21:35 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> Message-ID: ?and I also hadn?t attached the test. Sorry, I?m clearly very distracted today! ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 12:17:57 PM, Tony Printezis (tprintezis at twitter.com) wrote: (I unfortunately pressed Send accidentally; apologies) I was saying: I looked at TestMaxCachedBufferSize and, unfortunately, I don?t think the test makes a lot of sense right now as it checks the number / size of direct buffers after all the threads terminate and, with this change, that should always be 0. Let me look into this and I?ll get back to you in a bit. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 12:14:24 PM, Tony Printezis (tprintezis at twitter.com) wrote: Peter, Attached TerminatingThreadLocalTest.java. Let me know what you think (and feel free to modify it / discard it if you don?t like it!). Re: The test for the max cached buffer size is: test/jdk/sun/nio/ch/TestMaxCachedBufferSize.java. I looked at it and, unfortunately, ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 7:07:59 AM, Peter Levart (peter.levart at gmail.com) wrote: On 06/20/2018 04:24 PM, Tony Printezis wrote: Hey Peter, I had written a test to test my version of the exit hooks. I can easily rework it to work with yours. Interested? Of course. Just give what you got. I can modify it as needed. To check that the native buffers are reclaimed promptly, we should be able to amend the test that tests the setting of the max size for the cached buffers (i.e., check that, after all the threads have exited, the total native count / size is 0). Good idea. Do you happen to know which one is it? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 20, 2018 at 10:08:48 AM, Peter Levart (peter.levart at gmail.com) wrote: On 06/18/2018 05:41 PM, Alan Bateman wrote: > > > On 17/06/2018 22:20, Peter Levart wrote: >> Update: the discussion on concurrent-interest about possible future >> addition of public ThreadLocal.getIfPresent() or >> ThreadLocal.isPresent() has died out, but it nevertheless reached a >> point where ThreadLocal.isPresent() was found the least problematic. >> So I would like to get further with this proposal using the last >> webrev.04 version of the patch that uses ThreadLocal.isPresent() >> internally - still package-private at the moment. >> >> Here's the webrev (unchanged from the day it was published): >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ >> >> Would this version be OK? > I think looks quite good. > > One small nit is that the update to ThreadLocal.setInitialValue makes > it look like all locals are registered when setting the initial value. > What would you think about moving the instanceof check from > TerminatingThreadLocal.register so that it's a bit more obvious. > > -Alan Like the following? http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.05/ Do we need a test which proves that cached direct buffers are released when thread exits or would a unit test for TerminatingThreadLocal be enough? Maybe both? Regards, Peter From adinn at redhat.com Thu Jun 21 16:32:45 2018 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 21 Jun 2018 17:32:45 +0100 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> Message-ID: <7bc18707-946b-e4a7-3bfe-0deefa12db57@redhat.com> Hi Paul, Sorry for the delay in responding to this -- holiday and then an urgent bug fix intervened . . . On 08/06/18 01:42, Paul Sandoz wrote: > Sandhya gave an overview to a few of us Oracle folks. I agree with > what Sandhya says regarding the API, a small surface, and on pursuing > an unsafe intrinsic. I like it and would encourage the writing of a > draft JEP, especially to give this visibility. Great! Thanks for your feedback (also to Sandhya). I'll start drafting a JEP staright away. I'll also work on revising the current intrinsic implementation so it is presented via Unsafe (which should be fairly simple to achieve). > It intersects with https://bugs.openjdk.java.net/browse/JDK-8153111 > ((bf) Allocating ByteBuffer on heterogeneous memory), which is > attempting to be more generic. Ok, thanks. I'll have a think about how we night try to integrate these two approaches and see what I can work into the draft JEP. > We might also need to increase the velocity on > https://bugs.openjdk.java.net/browse/JDK-8180628 (retrofit direct > buffer support for size beyond gigabyte scales), and i would be very > interested your views on this, how you might be currently working > around such size limitations, and what buffer enhancements would work > for you. I think Jonathan answered that better than I can in his response. However, if this accelerates delivery of a fix for JDK-8180628 then all to the good. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From vivek.r.deshpande at intel.com Thu Jun 21 16:48:58 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Thu, 21 Jun 2018 16:48:58 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <01512723-0d61-560c-b8bf-6e0409437826@oracle.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> <01512723-0d61-560c-b8bf-6e0409437826@oracle.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A9074A353@ORSMSX106.amr.corp.intel.com> Yes makes sense. Good catch :) Thanks :) Regards, Vivek -----Original Message----- From: Ivan Gerasimov [mailto:ivan.gerasimov at oracle.com] Sent: Wednesday, June 20, 2018 5:24 PM To: Deshpande, Vivek R Cc: Paul Sandoz ; core-libs-dev at openjdk.java.net Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek! I think you don't need this if block: 464 if (a[aFromIndex] != b[bFromIndex]) { 465 i = 0; 466 } i is already 0 anyway, and the following line is just enough. 467 if (a[aFromIndex] == b[bFromIndex]) { The same applies to other float/double variants. With kind regards, Ivan On 6/20/18 5:05 PM, Deshpande, Vivek R wrote: > Hi Paul > > I have made the change you have suggested. > Please find the updated webrev at this location: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 2/ > > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Wednesday, June 20, 2018 2:30 PM > To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } else { > > You fold the if/else to; > > if (a[aFromIndex] == b[bFromIndex]) { > ? > } > > same applies in other cases in both source files. > > > > On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R > wrote: > > Hi All > > I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. > Earlier webrev failed the same test. > > Thanks for confirming. > > > This webrev also modifies the BufferMismatch.java in similar way. > > Great! > > Paul. > > > Please take a look and let me know what you think. > > Updated webrev is here: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 1/ I have also modified the bug with updated webrev. > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:57 AM > To: Paul Sandoz > >; > Roger.Riggs at oracle.com > Cc: David Holmes > >; > core-libs-dev at openjdk.java.net; > Viswanathan, Sandhya > > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Roger > > I will also test with zero length arrays and let you know. > Thanks for the input. > > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:17 AM > To: 'Paul Sandoz' > > > Cc: David Holmes > >; > core-libs-dev at openjdk.java.net; > Viswanathan, Sandhya > > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. > I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. > > Regards, > Vivek > > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Tuesday, June 19, 2018 9:55 AM > To: Deshpande, Vivek R > > > Cc: David Holmes > >; > core-libs-dev at openjdk.java.net; > Viswanathan, Sandhya > > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Thanks for investigating this. > > > 164 public static int mismatch(boolean[] a, > > 165 boolean[] b, > > 166 int length) { > > 167 int i = 0; > > 168 if (a[i] != b[i]) > > 169 return i; > You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. > > > 186 public static int mismatch(boolean[] a, int aFromIndex, > > 187 boolean[] b, int bFromIndex, > > 188 int length) { > > 189 int i = 0; > > 190 if (a[i] != b[i]) > > 191 return i; > This is incorrect. You need to use aFromIndex and bFromIndex. > > Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. > > Paul. > > On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: > > Thanks David. > Sending it to core-libs-dev. > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Could you please review and sponsor the patch. > Link to bug: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards, > Vivek > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Monday, June 18, 2018 10:32 PM > To: Deshpande, Vivek R > >; > jdk-dev at openjdk.java.net > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. > > Thanks, > David > > On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net' > > > Cc: 'Paul Sandoz' > >; Viswanathan, > Sandhya > > > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > -- With kind regards, Ivan Gerasimov From peter.levart at gmail.com Thu Jun 21 16:59:55 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Jun 2018 18:59:55 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> Message-ID: On 06/21/2018 06:17 PM, Tony Printezis wrote: > I was saying: I looked at TestMaxCachedBufferSize and, unfortunately, > I don?t think the test makes a lot of sense right now as it checks the > number / size of direct buffers after all the threads terminate and, > with this change, that should always be 0. You're right. The test makes no sense now. As I understand, the test checked that number/size of allocated temporary buffers did not exceed the estimated calculated size by checking the MXBean immediately after threads terminate. This will never happen after the change as they will all be freed before checking, regardless of how many were and how much was allocated. Perhaps the test should be modified to include a latch so that threads wait until the measurement is made and only then are allowed to exit... Regards, Peter From tprintezis at twitter.com Thu Jun 21 17:01:41 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 21 Jun 2018 10:01:41 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> Message-ID: I?m trying exactly that. :-) ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 12:59:58 PM, Peter Levart (peter.levart at gmail.com) wrote: On 06/21/2018 06:17 PM, Tony Printezis wrote: I was saying: I looked at TestMaxCachedBufferSize and, unfortunately, I don?t think the test makes a lot of sense right now as it checks the number / size of direct buffers after all the threads terminate and, with this change, that should always be 0. You're right. The test makes no sense now. As I understand, the test checked that number/size of allocated temporary buffers did not exceed the estimated calculated size by checking the MXBean immediately after threads terminate. This will never happen after the change as they will all be freed before checking, regardless of how many were and how much was allocated. Perhaps the test should be modified to include a latch so that threads wait until the measurement is made and only then are allowed to exit... Regards, Peter From peter.levart at gmail.com Thu Jun 21 17:29:36 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Jun 2018 19:29:36 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> Message-ID: <36c14123-db0f-600d-10f5-ae8ca82f66c0@gmail.com> On 06/21/2018 07:01 PM, Tony Printezis wrote: > I?m trying exactly that. :-) Sorry, I didn't know. Here's my attempt: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.07/ I also added @run main/othervm to TempDirectBuffersReclamation test. Peter > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On June 21, 2018 at 12:59:58 PM, Peter Levart (peter.levart at gmail.com > ) wrote: > >> >> >> On 06/21/2018 06:17 PM, Tony Printezis wrote: >>> I was saying: I looked at TestMaxCachedBufferSize and, >>> unfortunately, I don?t think the test makes a lot of sense right now >>> as it checks the number / size of direct buffers after all the >>> threads terminate and, with this change, that should always be 0. >> >> You're right. The test makes no sense now. As I understand, the test >> checked that number/size of allocated temporary buffers did not >> exceed the estimated calculated size by checking the MXBean >> immediately after threads terminate. This will never happen after the >> change as they will all be freed before checking, regardless of how >> many were and how much was allocated. >> >> Perhaps the test should be modified to include a latch so that >> threads wait until the measurement is made and only then are allowed >> to exit... >> >> Regards, Peter >> From tprintezis at twitter.com Thu Jun 21 17:39:27 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 21 Jun 2018 10:39:27 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <36c14123-db0f-600d-10f5-ae8ca82f66c0@gmail.com> References: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> <36c14123-db0f-600d-10f5-ae8ca82f66c0@gmail.com> Message-ID: Peter, The changes to TestMaxCachedBufferSize.java look fine. One point though: Why do you need the TmpDirectBuffersReclamation.java test? In TestMaxCachedBufferSize.java you just call checkDirectBuffers(0, 0); after you the main thread calls join() on the workers? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 1:29:38 PM, Peter Levart (peter.levart at gmail.com) wrote: On 06/21/2018 07:01 PM, Tony Printezis wrote: I?m trying exactly that. :-) Sorry, I didn't know. Here's my attempt: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.07/ I also added @run main/othervm to TempDirectBuffersReclamation test. Peter ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 21, 2018 at 12:59:58 PM, Peter Levart (peter.levart at gmail.com) wrote: On 06/21/2018 06:17 PM, Tony Printezis wrote: I was saying: I looked at TestMaxCachedBufferSize and, unfortunately, I don?t think the test makes a lot of sense right now as it checks the number / size of direct buffers after all the threads terminate and, with this change, that should always be 0. You're right. The test makes no sense now. As I understand, the test checked that number/size of allocated temporary buffers did not exceed the estimated calculated size by checking the MXBean immediately after threads terminate. This will never happen after the change as they will all be freed before checking, regardless of how many were and how much was allocated. Perhaps the test should be modified to include a latch so that threads wait until the measurement is made and only then are allowed to exit... Regards, Peter From vivek.r.deshpande at intel.com Thu Jun 21 18:50:35 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Thu, 21 Jun 2018 18:50:35 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A9074A5D1@ORSMSX106.amr.corp.intel.com> Hi Paul The folding of if/else is the right way in this patch which I missed in earlier webrev. http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.03/ I think I would keep the same earlier path if both the values are NaN. Thanks. Regards, Vivek From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Wednesday, June 20, 2018 5:29 PM To: Deshpande, Vivek R Cc: Roger.Riggs at oracle.com; David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, 459 public static int mismatch(float[] a, int aFromIndex, 460 float[] b, int bFromIndex, 461 int length) { 462 int i = 0; 463 if (length > 1) { 464 if (a[aFromIndex] != b[bFromIndex]) { 465 i = 0; 466 } 467 if (a[aFromIndex] == b[bFromIndex]) { Sorry, i don?t think i was clear. You can reduce down to a single equality check, so Lines #464-466 are redundant. This does result in taking the slow path if both values are NaN, which i think was the same for your previous patch. If that is important we can mitigate that by performing the negation of the same checks in the slow loop. WDYT? Paul. On Jun 20, 2018, at 5:05 PM, Deshpande, Vivek R > wrote: Hi Paul I have made the change you have suggested. Please find the updated webrev at this location: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.02/ Regards, Vivek From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Wednesday, June 20, 2018 2:30 PM To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. 459 public static int mismatch(float[] a, int aFromIndex, 460 float[] b, int bFromIndex, 461 int length) { 462 int i = 0; 463 if (length > 1) { 464 if (a[aFromIndex] != b[bFromIndex]) { 465 i = 0; 466 } else { You fold the if/else to; if (a[aFromIndex] == b[bFromIndex]) { ? } same applies in other cases in both source files. On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R > wrote: Hi All I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. Earlier webrev failed the same test. Thanks for confirming. This webrev also modifies the BufferMismatch.java in similar way. Great! Paul. Please take a look and let me know what you think. Updated webrev is here: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.01/ I have also modified the bug with updated webrev. Regards, Vivek From: Deshpande, Vivek R Sent: Tuesday, June 19, 2018 10:57 AM To: Paul Sandoz >; Roger.Riggs at oracle.com Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Roger I will also test with zero length arrays and let you know. Thanks for the input. Regards, Vivek From: Deshpande, Vivek R Sent: Tuesday, June 19, 2018 10:17 AM To: 'Paul Sandoz' > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. Regards, Vivek From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Tuesday, June 19, 2018 9:55 AM To: Deshpande, Vivek R > Cc: David Holmes >; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Thanks for investigating this. 164 public static int mismatch(boolean[] a, 165 boolean[] b, 166 int length) { 167 int i = 0; 168 if (a[i] != b[i]) 169 return i; You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. 186 public static int mismatch(boolean[] a, int aFromIndex, 187 boolean[] b, int bFromIndex, 188 int length) { 189 int i = 0; 190 if (a[i] != b[i]) 191 return i; This is incorrect. You need to use aFromIndex and bFromIndex. Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. Paul. On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R > wrote: Thanks David. Sending it to core-libs-dev. I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Could you please review and sponsor the patch. Link to bug: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ Regards, Vivek -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Monday, June 18, 2018 10:32 PM To: Deshpande, Vivek R >; jdk-dev at openjdk.java.net Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. Thanks, David On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: Hi All Forgot to add the links: https://bugs.openjdk.java.net/browse/JDK-8205194 webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 0/ Regards. Vivek From: Deshpande, Vivek R Sent: Monday, June 18, 2018 4:50 PM To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' >; Viswanathan, Sandhya > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi All I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. This avoids call to vectorizedMismatch method and gives ~80x speed up. Please review and sponsor the patch. Regards, Vivek From Roger.Riggs at Oracle.com Thu Jun 21 18:59:11 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 21 Jun 2018 14:59:11 -0400 Subject: RFR 8202292 : test/jdk/java/io/FileOutputStream/UnreferencedFOSClosesFd.java fails with "raw fd count wrong" Message-ID: Please review a test improvement to avoid spurious errors caused by concurrent changes in open files during the test. The other existing conditions in the test are sufficient to verify the correct finalize behavior. Some additional diagnostics are added and the tests removed from the problem list. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/index.html Issue: ? https://bugs.openjdk.java.net/browse/JDK-8202292 Thanks, Roger From mandy.chung at oracle.com Thu Jun 21 19:25:20 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 21 Jun 2018 12:25:20 -0700 Subject: RFR 8202292 : test/jdk/java/io/FileOutputStream/UnreferencedFOSClosesFd.java fails with "raw fd count wrong" In-Reply-To: References: Message-ID: On 6/21/18 11:59 AM, Roger Riggs wrote: > Please review a test improvement to avoid spurious errors caused by > concurrent changes in open files during the test. > The other existing conditions in the test are sufficient to verify the > correct finalize behavior. > Some additional diagnostics are added and the tests removed from the > problem list. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/index.html The fix looks reasonable. It's good to get this test in service again. Do you want to keep the commented lines in the test? Maybe add a comment instead. Mandy From brian.burkhalter at oracle.com Thu Jun 21 19:25:29 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 21 Jun 2018 12:25:29 -0700 Subject: RFR 8202292 : test/jdk/java/io/FileOutputStream/UnreferencedFOSClosesFd.java fails with "raw fd count wrong" In-Reply-To: References: Message-ID: Hi Roger, Are the three versions of listProcFD() identical? Is so, could this method be put in test/lib/jdk/test/lib/util/FileUtils.java instead? Thanks, Brian On Jun 21, 2018, at 11:59 AM, Roger Riggs wrote: > Please review a test improvement to avoid spurious errors caused by concurrent changes in open files during the test. > The other existing conditions in the test are sufficient to verify the correct finalize behavior. > Some additional diagnostics are added and the tests removed from the problem list. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/index.html > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8202292 From paul.sandoz at oracle.com Thu Jun 21 19:35:14 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 21 Jun 2018 12:35:14 -0700 Subject: RFR 8202922 Method reference identity is broken by serialization In-Reply-To: <89462B7C-BE80-42FA-8D49-DC5737948A01@oracle.com> References: <89462B7C-BE80-42FA-8D49-DC5737948A01@oracle.com> Message-ID: Gentle reminder, Paul. > On Jun 14, 2018, at 10:41 AM, Paul Sandoz wrote: > > Hi, > > Please review an update to the specifications of LambdaMetafactory and SerializedLambda to clarify that the identity of function objects is unpredictable. A similar (informational) clarification was made to the language specification and similar wording is used. Amusingly a quirk of (or issue with) lambda deserialization (we count our blessings it works as it does!) motivated this clarification. > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8202922-function-object-identity/webrev/ > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8205060 > > Thanks, > Paul. From james.laskey at oracle.com Thu Jun 21 19:43:31 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 21 Jun 2018 16:43:31 -0300 Subject: RFR 8202922 Method reference identity is broken by serialization In-Reply-To: <89462B7C-BE80-42FA-8D49-DC5737948A01@oracle.com> References: <89462B7C-BE80-42FA-8D49-DC5737948A01@oracle.com> Message-ID: <788945DD-446D-43C7-826A-6B293AE63C89@oracle.com> +1 > On Jun 14, 2018, at 2:41 PM, Paul Sandoz wrote: > > Hi, > > Please review an update to the specifications of LambdaMetafactory and SerializedLambda to clarify that the identity of function objects is unpredictable. A similar (informational) clarification was made to the language specification and similar wording is used. Amusingly a quirk of (or issue with) lambda deserialization (we count our blessings it works as it does!) motivated this clarification. > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8202922-function-object-identity/webrev/ > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8205060 > > Thanks, > Paul. From mandy.chung at oracle.com Thu Jun 21 19:48:04 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 21 Jun 2018 12:48:04 -0700 Subject: RFR 8202922 Method reference identity is broken by serialization In-Reply-To: <89462B7C-BE80-42FA-8D49-DC5737948A01@oracle.com> References: <89462B7C-BE80-42FA-8D49-DC5737948A01@oracle.com> Message-ID: <8e5e57c6-f4c5-7612-c1c6-d74a6860ed9c@oracle.com> Looks fine. Mandy On 6/14/18 10:41 AM, Paul Sandoz wrote: > Hi, > > Please review an update to the specifications of LambdaMetafactory > and SerializedLambda to clarify that the identity of function objects > is unpredictable. A similar (informational) clarification was made to > the language specification and similar wording is used. Amusingly a > quirk of (or issue with) lambda deserialization (we count our > blessings it works as it does!) motivated this clarification. > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8202922-function-object-identity/webrev/ > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8205060 > > Thanks, Paul. > > From mandy.chung at oracle.com Thu Jun 21 20:26:32 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 21 Jun 2018 13:26:32 -0700 Subject: RFR 8195650 Method references to VarHandle accessors In-Reply-To: <086A1684-4D97-4B6D-94F2-16A1261057B5@oracle.com> References: <086A1684-4D97-4B6D-94F2-16A1261057B5@oracle.com> Message-ID: This looks good to me AFAICT. Mandy On 6/19/18 5:08 PM, Paul Sandoz wrote: > Hi, > > Please review the following fix to ensure method references to > VarHandle signature polymorphic methods are supported at runtime > (specifically the method handle to a signature polymorphic method can > be loaded from the constant pool): > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195650-varhandle-mref/webrev/ > > I also added a ?belts and braces? test to ensure a constant method > handle to MethodHandle::invokeBasic cannot be loaded if outside of > the j.l.invoke package. > > Paul. > From ecki at zusammenkunft.net Thu Jun 21 20:30:54 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 21 Jun 2018 20:30:54 +0000 Subject: free(): corrupted unsorted chunks In-Reply-To: References: Message-ID: Are you using any native libraries in this VM, is there a hs_err or System core file? Can you run ?ldd? on the java launcher binary. Can you try a OpenJDK binary distribution independent of the OS compiled version? Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: core-libs-dev on behalf of Oliver Kohll Sent: Thursday, June 21, 2018 1:59:27 PM To: core-libs-dev at openjdk.java.net Subject: free(): corrupted unsorted chunks Hi, I'm a Java developer getting a crash and error message 'free(): corrupted unsorted chunks' in the log. this sounds like it could be a low level issue in the core Java libraries so I'm posting here, but if it would be better in another list (or even a different community) please let me know. So I'm requesting help because I don't know how to debug this further. In summary, the issue is We're getting a crash nearly once a day on our production system, at a time of day when it's most busy (mid morning). This crash hasn't been seen on our development environment, which is the same apart from the server has less memory and CPU resources. The environment is: * A VPS host * Ubuntu LTS 18 (Bionic) * The default Java 10 packaged as 11 I believe ( https://askubuntu.com/questions/1037646/why-is-openjdk-10-packaged-as-openjdk-11 ) * However, javac seems to be from JDK8: root at server:~# file /etc/alternatives/java /etc/alternatives/javac /etc/alternatives/java: symbolic link to /usr/lib/jvm/java-11-openjdk-amd64/bin/java /etc/alternatives/javac: symbolic link to /usr/lib/jvm/java-8-openjdk-amd64/bin/javac We're running our webapp on Tomcat 8 (also the Ubuntu default). When this happens, the server stops responding to HTTP requests (in fact the java process has quit) and needs to be restarted. The crash doesn't seem to correspond to a particular activity on the server as far as I can see, though it's hard to tell as many people are logged in at the same time. Does anyone have any pointers of a) how to track the bug further so I can report it, ideally without a big impact on our live server b) more immediately, any workarounds to try In the meantime I will see if I can downgrade to JDK8 on the test server, but I'm not sure if that's possible or whether it would make a difference. Regards Oliver From fw at deneb.enyo.de Thu Jun 21 20:13:47 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Thu, 21 Jun 2018 22:13:47 +0200 Subject: Promptly freeing the per-thread cached direct buffers when a thread exits In-Reply-To: (Tony Printezis's message of "Thu, 5 Apr 2018 14:45:48 -0700") References: Message-ID: <8736xfx5hg.fsf@mid.deneb.enyo.de> * Tony Printezis: > There are a few obvious ways to mitigate this, e.g., cause a Full GC / > concurrent GC cycle at regular intervals. However, the best solution IMHO > is to explicitly free any direct buffers that are still in the cache when a > thread exits. Why is this safe? Couldn't these direct byte buffers be used with a custom channel that leaks them to another thread? From gromero at linux.vnet.ibm.com Thu Jun 21 21:37:34 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 21 Jun 2018 18:37:34 -0300 Subject: RFC: Add new JCA provider to support hardware RNGs In-Reply-To: References: <993c258a-57ae-d29d-2368-faac5b4d3cb1@linux.vnet.ibm.com> Message-ID: Hi Bernd, Thanks for your comments. On 06/20/2018 05:05 PM, Bernd Eckenfels wrote: > Just a FYI under Linux when you read from urandom the Linux kernel will always XOR with random bytes generated with x64 rdrand instruction (arch_get_random_lomg() - if supported). Since it is a XOR it does not have to trust the quality of this black box hardware implementation. Yes, I know to some extent how the use of rdrand on Intel by Linux generated debates in the past. > I would not implement a generator which does not have its own software whitening. (And most likely there is no need for one different than urandom on Linux). If you do implement whitening I would use a DRBG construction and no longer use a (low state) SHA1PRNG. My main concern to use 'darn' on Power was to get some additional speed and throughput on obtaining true random numbers (or on cases where /dev/random - which blocks - was preferred to /dev/urandom). That concern seems valid specially when the code is not JITed yet by the C2 compiler (so for Interpreted and C1 Compiler). Anyway, the idea to have a separated provider and that the use of the intrinsic is by no means enabled by default is to let the user to decide if he/she wants to use that kind of generator. I think OpenSSL disabled by default the use of rdrand similarly but let the user to enable it if desired? The idea of DRBG looks valid. I took SHA1PRNG as a fallback. But if DRBG would be used for whitening that would need a seed from /dev/urandom? (I'm trying to think the impact on performance here). Best regards, Gustavo > Gruss > Bernd > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ________________________________ > From: core-libs-dev on behalf of Gustavo Romero > Sent: Wednesday, June 20, 2018 2:59:47 AM > To: core-libs-dev; security-dev > Cc: vladimir.kozlov at oracle.com; Doerr, Martin > Subject: RFC: Add new JCA provider to support hardware RNGs > > Hi, > > Please, could I get comments on the following change? > > Since it's related to security, I would be glad if security > experts could also comment on that. > > webrev: http://cr.openjdk.java.net/~gromero/POWER9/darn/v6_rebased/ > > It introduces a way to get benefits from hardware instructions in userspace > that can delivery a random number and so be used to speed up and avoid > blocking in some SecureRandom methods, notably generateSeed() and > nextBytes(), without loosing the random number quality. Particularly on > Power, the new POWER9 CPUs introduced a hardware instruction called 'darn' > that can be used for that purpose. > > The main idea is to add a new JCA provider called "HWTRNG" (if no better > name is suggested), adding new helper methods that can then be intrinsified > and that will be used by generateSeed() and nextBytes(). In that sense, > this change also adds the proper mechanisms in the Interpreter, > C1 Compiler, and C2 compiler (for PPC64, but also paving the way for other > platforms) to intrinsify these helper methods that will be used in the end > by generateSeed() and nextBytes() methods. The added helpers are: > > byte[] getRandomSequence0(int) > byte[] getRandomSequence1(byte[]) > long validRandomLong(void) > long randomLong(void) > > The helpers also take care of checking if the returned random number is > valid and attempt 10 times (as recommended by ISA) get a valid random > number before falling back to a software alternative (HWTRNG is based > mostly on JCA provider NativePRNG and so the fall back mechanism will use > the exactly same methods found in NativePRNG and hence behave similarly. > Nonetheless, in my experience such a hardware failures (which are the main > cause of a returned invalid random number) are quite rare: in my tests I > was never able to exhaust the HW RNG and force it to generate an invalid > random number). > > The user, besides having to specify explicitly the use of a non-default > provider (HWTRNG), will have to unlock the VM experimental options to > effectively use the hardware RNG as an unique random source by passing > options "-XX:+UnlockExperimentalVMOptions -XX:+UseRANDOMIntrinsics". > > On Power, for the Interpreter and C1 Compiler, besides avoiding the > blocking effect of a low entropy on /dev/random, I was able to get about > 126 Mbps (3x higher than the version without 'darn') on generaSeed() and > nextBytes(). The maximum theoretical value on a POWER9 machine is usually > 128 Mbps. > > I'll address the details about the file headers (Copyright, dates, authors, > versions, etc) after I get some feedback about this change. > > > Thanks in advance. > > Best regards, > Gustavo > From paul.sandoz at oracle.com Thu Jun 21 22:32:23 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 21 Jun 2018 15:32:23 -0700 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <53E8E64DB2403849AFD89B7D4DAC8B2A9074A5D1@ORSMSX106.amr.corp.intel.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A9074A5D1@ORSMSX106.amr.corp.intel.com> Message-ID: <5A374FC1-26FB-40B7-A648-0F1527BC5A01@oracle.com> Hi Vivek, This looks better. I thought a bit more about NaNs. I am concerned about edge case performance regressions if the first two values are NaNs (with the same bit pattern). Here?s a patch on top of your patch that compares the raw bits for float/double elements. If you are ok with this and apply it on top of your patch then i don?t think there is a need for another review round. Thanks, Paul. diff -r ef8fd07e4440 src/java.base/share/classes/java/nio/BufferMismatch.java --- a/src/java.base/share/classes/java/nio/BufferMismatch.java Thu Jun 21 13:43:04 2018 -0700 +++ b/src/java.base/share/classes/java/nio/BufferMismatch.java Thu Jun 21 14:02:19 2018 -0700 @@ -118,7 +118,7 @@ static int mismatch(FloatBuffer a, int aOff, FloatBuffer b, int bOff, int length) { int i = 0; if (length > 1 && a.order() == b.order()) { - if (a.get(aOff) == b.get(bOff)) { + if (Float.floatToRawIntBits(a.get(aOff)) == Float.floatToRawIntBits(b.get(bOff))) { i = ArraysSupport.vectorizedMismatch( a.base(), a.address + (aOff << ArraysSupport.LOG2_ARRAY_FLOAT_INDEX_SCALE), b.base(), b.address + (bOff << ArraysSupport.LOG2_ARRAY_FLOAT_INDEX_SCALE), @@ -175,7 +175,7 @@ static int mismatch(DoubleBuffer a, int aOff, DoubleBuffer b, int bOff, int length) { int i = 0; if (length > 0 && a.order() == b.order()) { - if (a.get(aOff) == b.get(bOff)) { + if (Double.doubleToRawLongBits(a.get(aOff)) == Double.doubleToRawLongBits(b.get(bOff))) { i = ArraysSupport.vectorizedMismatch( a.base(), a.address + (aOff << ArraysSupport.LOG2_ARRAY_DOUBLE_INDEX_SCALE), b.base(), b.address + (bOff << ArraysSupport.LOG2_ARRAY_DOUBLE_INDEX_SCALE), diff -r ef8fd07e4440 src/java.base/share/classes/jdk/internal/util/ArraysSupport.java --- a/src/java.base/share/classes/jdk/internal/util/ArraysSupport.java Thu Jun 21 13:43:04 2018 -0700 +++ b/src/java.base/share/classes/jdk/internal/util/ArraysSupport.java Thu Jun 21 14:02:19 2018 -0700 @@ -461,7 +461,7 @@ int length) { int i = 0; if (length > 1) { - if (a[aFromIndex] == b[bFromIndex]) { + if (Float.floatToRawIntBits(a[aFromIndex]) == Float.floatToRawIntBits(b[bFromIndex])) { int aOffset = Unsafe.ARRAY_FLOAT_BASE_OFFSET + (aFromIndex << LOG2_ARRAY_FLOAT_INDEX_SCALE); int bOffset = Unsafe.ARRAY_FLOAT_BASE_OFFSET + (bFromIndex << LOG2_ARRAY_FLOAT_INDEX_SCALE); i = vectorizedMismatch( @@ -545,7 +545,7 @@ return -1; } int i = 0; - if (a[aFromIndex] == b[bFromIndex]) { + if (Double.doubleToRawLongBits(a[aFromIndex]) == Double.doubleToRawLongBits(b[bFromIndex])) { int aOffset = Unsafe.ARRAY_DOUBLE_BASE_OFFSET + (aFromIndex << LOG2_ARRAY_DOUBLE_INDEX_SCALE); int bOffset = Unsafe.ARRAY_DOUBLE_BASE_OFFSET + (bFromIndex << LOG2_ARRAY_DOUBLE_INDEX_SCALE); i = vectorizedMismatch( > On Jun 21, 2018, at 11:50 AM, Deshpande, Vivek R wrote: > > Hi Paul > > The folding of if/else is the right way in this patch which I missed in earlier webrev. > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.03/ > I think I would keep the same earlier path if both the values are NaN. > > Thanks. > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Wednesday, June 20, 2018 5:29 PM > To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } > 467 if (a[aFromIndex] == b[bFromIndex]) { > > Sorry, i don?t think i was clear. You can reduce down to a single equality check, so Lines #464-466 are redundant. This does result in taking the slow path if both values are NaN, which i think was the same for your previous patch. If that is important we can mitigate that by performing the negation of the same checks in the slow loop. > > WDYT? > > Paul. > > > On Jun 20, 2018, at 5:05 PM, Deshpande, Vivek R wrote: > > Hi Paul > > I have made the change you have suggested. > Please find the updated webrev at this location: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.02/ > > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Wednesday, June 20, 2018 2:30 PM > To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } else { > > You fold the if/else to; > > if (a[aFromIndex] == b[bFromIndex]) { > ? > } > > same applies in other cases in both source files. > > > > > On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R wrote: > > Hi All > > I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. > Earlier webrev failed the same test. > > Thanks for confirming. > > > > This webrev also modifies the BufferMismatch.java in similar way. > > Great! > > Paul. > > > > Please take a look and let me know what you think. > > Updated webrev is here: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.01/ > I have also modified the bug with updated webrev. > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:57 AM > To: Paul Sandoz ; Roger.Riggs at oracle.com > Cc: David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Roger > > I will also test with zero length arrays and let you know. > Thanks for the input. > > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:17 AM > To: 'Paul Sandoz' > Cc: David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. > I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. > > Regards, > Vivek > > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Tuesday, June 19, 2018 9:55 AM > To: Deshpande, Vivek R > Cc: David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Thanks for investigating this. > > 164 public static int mismatch(boolean[] a, > 165 boolean[] b, > 166 int length) { > 167 int i = 0; > 168 if (a[i] != b[i]) > 169 return i; > You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. > > 186 public static int mismatch(boolean[] a, int aFromIndex, > 187 boolean[] b, int bFromIndex, > 188 int length) { > 189 int i = 0; > 190 if (a[i] != b[i]) > 191 return i; > This is incorrect. You need to use aFromIndex and bFromIndex. > > Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. > > Paul. > > > On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R wrote: > > Thanks David. > Sending it to core-libs-dev. > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Could you please review and sponsor the patch. > Link to bug: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.00/ > > Regards, > Vivek > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Monday, June 18, 2018 10:32 PM > To: Deshpande, Vivek R ; jdk-dev at openjdk.java.net > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. > > Thanks, > David > > On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: > > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' ; Viswanathan, Sandhya > > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > From oliver at agilechilli.com Thu Jun 21 22:34:44 2018 From: oliver at agilechilli.com (Oliver Kohll) Date: Thu, 21 Jun 2018 15:34:44 -0700 Subject: free(): corrupted unsorted chunks In-Reply-To: References: Message-ID: Aha, I wonder if it's configured with the APR native library for SSL. I will check out that and those things. Thanks Oliver On 21 June 2018 at 21:43:24, Bernd Eckenfels (ecki at zusammenkunft.net) wrote: Are you using any native libraries in this VM, is there a hs_err or System core file? Can you run ?ldd? on the java launcher binary. Can you try a OpenJDK binary distribution independent of the OS compiled version? Gruss Bernd -- http://bernd.eckenfels.net From vivek.r.deshpande at intel.com Thu Jun 21 23:13:42 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Thu, 21 Jun 2018 23:13:42 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <5A374FC1-26FB-40B7-A648-0F1527BC5A01@oracle.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A9074A5D1@ORSMSX106.amr.corp.intel.com> <5A374FC1-26FB-40B7-A648-0F1527BC5A01@oracle.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A9074AA58@ORSMSX106.amr.corp.intel.com> Thanks Paul for taking some time and thinking about it. I will make the change you have mentioned and send the patch again. Regards, Vivek -----Original Message----- From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Thursday, June 21, 2018 3:32 PM To: Deshpande, Vivek R Cc: Roger.Riggs at oracle.com; David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya ; Ivan Gerasimov Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, This looks better. I thought a bit more about NaNs. I am concerned about edge case performance regressions if the first two values are NaNs (with the same bit pattern). Here?s a patch on top of your patch that compares the raw bits for float/double elements. If you are ok with this and apply it on top of your patch then i don?t think there is a need for another review round. Thanks, Paul. diff -r ef8fd07e4440 src/java.base/share/classes/java/nio/BufferMismatch.java --- a/src/java.base/share/classes/java/nio/BufferMismatch.java Thu Jun 21 13:43:04 2018 -0700 +++ b/src/java.base/share/classes/java/nio/BufferMismatch.java Thu Jun 21 14:02:19 2018 -0700 @@ -118,7 +118,7 @@ static int mismatch(FloatBuffer a, int aOff, FloatBuffer b, int bOff, int length) { int i = 0; if (length > 1 && a.order() == b.order()) { - if (a.get(aOff) == b.get(bOff)) { + if (Float.floatToRawIntBits(a.get(aOff)) == + Float.floatToRawIntBits(b.get(bOff))) { i = ArraysSupport.vectorizedMismatch( a.base(), a.address + (aOff << ArraysSupport.LOG2_ARRAY_FLOAT_INDEX_SCALE), b.base(), b.address + (bOff << ArraysSupport.LOG2_ARRAY_FLOAT_INDEX_SCALE), @@ -175,7 +175,7 @@ static int mismatch(DoubleBuffer a, int aOff, DoubleBuffer b, int bOff, int length) { int i = 0; if (length > 0 && a.order() == b.order()) { - if (a.get(aOff) == b.get(bOff)) { + if (Double.doubleToRawLongBits(a.get(aOff)) == + Double.doubleToRawLongBits(b.get(bOff))) { i = ArraysSupport.vectorizedMismatch( a.base(), a.address + (aOff << ArraysSupport.LOG2_ARRAY_DOUBLE_INDEX_SCALE), b.base(), b.address + (bOff << ArraysSupport.LOG2_ARRAY_DOUBLE_INDEX_SCALE), diff -r ef8fd07e4440 src/java.base/share/classes/jdk/internal/util/ArraysSupport.java --- a/src/java.base/share/classes/jdk/internal/util/ArraysSupport.java Thu Jun 21 13:43:04 2018 -0700 +++ b/src/java.base/share/classes/jdk/internal/util/ArraysSupport.java Thu Jun 21 14:02:19 2018 -0700 @@ -461,7 +461,7 @@ int length) { int i = 0; if (length > 1) { - if (a[aFromIndex] == b[bFromIndex]) { + if (Float.floatToRawIntBits(a[aFromIndex]) == + Float.floatToRawIntBits(b[bFromIndex])) { int aOffset = Unsafe.ARRAY_FLOAT_BASE_OFFSET + (aFromIndex << LOG2_ARRAY_FLOAT_INDEX_SCALE); int bOffset = Unsafe.ARRAY_FLOAT_BASE_OFFSET + (bFromIndex << LOG2_ARRAY_FLOAT_INDEX_SCALE); i = vectorizedMismatch( @@ -545,7 +545,7 @@ return -1; } int i = 0; - if (a[aFromIndex] == b[bFromIndex]) { + if (Double.doubleToRawLongBits(a[aFromIndex]) == + Double.doubleToRawLongBits(b[bFromIndex])) { int aOffset = Unsafe.ARRAY_DOUBLE_BASE_OFFSET + (aFromIndex << LOG2_ARRAY_DOUBLE_INDEX_SCALE); int bOffset = Unsafe.ARRAY_DOUBLE_BASE_OFFSET + (bFromIndex << LOG2_ARRAY_DOUBLE_INDEX_SCALE); i = vectorizedMismatch( > On Jun 21, 2018, at 11:50 AM, Deshpande, Vivek R wrote: > > Hi Paul > > The folding of if/else is the right way in this patch which I missed in earlier webrev. > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 3/ I think I would keep the same earlier path if both the values are > NaN. > > Thanks. > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Wednesday, June 20, 2018 5:29 PM > To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } > 467 if (a[aFromIndex] == b[bFromIndex]) { > > Sorry, i don?t think i was clear. You can reduce down to a single equality check, so Lines #464-466 are redundant. This does result in taking the slow path if both values are NaN, which i think was the same for your previous patch. If that is important we can mitigate that by performing the negation of the same checks in the slow loop. > > WDYT? > > Paul. > > > On Jun 20, 2018, at 5:05 PM, Deshpande, Vivek R wrote: > > Hi Paul > > I have made the change you have suggested. > Please find the updated webrev at this location: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 2/ > > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Wednesday, June 20, 2018 2:30 PM > To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } else { > > You fold the if/else to; > > if (a[aFromIndex] == b[bFromIndex]) { > ? > } > > same applies in other cases in both source files. > > > > > On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R wrote: > > Hi All > > I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. > Earlier webrev failed the same test. > > Thanks for confirming. > > > > This webrev also modifies the BufferMismatch.java in similar way. > > Great! > > Paul. > > > > Please take a look and let me know what you think. > > Updated webrev is here: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 1/ I have also modified the bug with updated webrev. > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:57 AM > To: Paul Sandoz ; Roger.Riggs at oracle.com > Cc: David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Roger > > I will also test with zero length arrays and let you know. > Thanks for the input. > > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:17 AM > To: 'Paul Sandoz' > Cc: David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. > I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. > > Regards, > Vivek > > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Tuesday, June 19, 2018 9:55 AM > To: Deshpande, Vivek R > Cc: David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Thanks for investigating this. > > 164 public static int mismatch(boolean[] a, > 165 boolean[] b, > 166 int length) { > 167 int i = 0; > 168 if (a[i] != b[i]) > 169 return i; > You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. > > 186 public static int mismatch(boolean[] a, int aFromIndex, > 187 boolean[] b, int bFromIndex, > 188 int length) { > 189 int i = 0; > 190 if (a[i] != b[i]) > 191 return i; > This is incorrect. You need to use aFromIndex and bFromIndex. > > Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. > > Paul. > > > On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R wrote: > > Thanks David. > Sending it to core-libs-dev. > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Could you please review and sponsor the patch. > Link to bug: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards, > Vivek > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Monday, June 18, 2018 10:32 PM > To: Deshpande, Vivek R ; > jdk-dev at openjdk.java.net > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. > > Thanks, > David > > On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: > > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' ; Viswanathan, Sandhya > > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > From martinrb at google.com Fri Jun 22 01:28:34 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 21 Jun 2018 18:28:34 -0700 Subject: RFR: jsr166 integration 2018-06 Message-ID: JDK 11 draws nigh. http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html 8202422: value of 'sizeCtl' in ConcurrentHashMap varies with the constructor called http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/sizeCtl/index.html https://bugs.openjdk.java.net/browse/JDK-8202422 8203864: Execution error in Java's Timsort http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/TimSort/index.html https://bugs.openjdk.java.net/browse/JDK-8203864 --- We've been unable to reach agreement on 8203662 - it remains in limbo. 8203662: remove increment of modCount from ArrayList and Vector replaceAll() http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/replaceAll/index.html https://bugs.openjdk.java.net/browse/JDK-8203662 8203681: Miscellaneous changes imported from jsr166 CVS 2018-06 http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html https://bugs.openjdk.java.net/browse/JDK-8203681 From martinrb at google.com Fri Jun 22 03:06:36 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 21 Jun 2018 20:06:36 -0700 Subject: RFR: 8205184: Delegating Iterator implementations that don't delegate forEachRemaining() Message-ID: 8205184: Delegating Iterator implementations that don't delegate forEachRemaining() http://cr.openjdk.java.net/~martin/webrevs/jdk/delegate-forEachRemaining/ https://bugs.openjdk.java.net/browse/JDK-8205184 From takiguc at linux.vnet.ibm.com Fri Jun 22 05:47:01 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 22 Jun 2018 14:47:01 +0900 Subject: COMPOUND_TEXT charset is missing on JDK11 Message-ID: <5b8b543cace0e6c9d58a1870217d16eb@linux.vnet.ibm.com> Hello. JDK8 Linux build has COMPOUND_TEXT charset, but it seems it's missing on JDK11 Linux build. I checked Bug DB, I could not find out the reason why COMPOUND_TEXT charset was missing. Could you port COMPOUND_TEXT charset from JDK8 to JDK11 ? Thanks, Ichiroh Takiguchi IBM Japan, Ltd. From Alan.Bateman at oracle.com Fri Jun 22 07:14:21 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 Jun 2018 08:14:21 +0100 Subject: COMPOUND_TEXT charset is missing on JDK11 In-Reply-To: <5b8b543cace0e6c9d58a1870217d16eb@linux.vnet.ibm.com> References: <5b8b543cace0e6c9d58a1870217d16eb@linux.vnet.ibm.com> Message-ID: On 22/06/2018 06:47, Ichiroh Takiguchi wrote: > Hello. > > JDK8 Linux build has COMPOUND_TEXT charset, but it seems it's missing > on JDK11 Linux build. > I checked Bug DB, I could not find out the reason why COMPOUND_TEXT > charset was missing. This was removed more than 3 years ago during JDK 9 development as part of separating the desktop area into its own module. It was removed, along with X11 charsets as part of JDK-8035302. Much of the need for these charsets was legacy at that point, esp. with Motif going away. Is the context for your question something in the font or postscript printing areas? This may be something that needs to be brought up on 2d-dev. -Alan From dan.z.zhou at oracle.com Fri Jun 22 08:26:17 2018 From: dan.z.zhou at oracle.com (Dora Zhou) Date: Fri, 22 Jun 2018 16:26:17 +0800 Subject: [11]RFR 8194152: sun/security/tools/jarsigner/AltProvider.java failed on de-DE locale Message-ID: <09f92268-fe4d-3c57-0ffb-d93d9703974d@oracle.com> Hello, Please help review the fix for JDK-8194152. Thank you. The test compares output with expected error messages in English, set locale to en-US so that the output are not translated into other languages. Bug: https://bugs.openjdk.java.net/browse/JDK-8194152 Webrev: http://cr.openjdk.java.net/~ljiang/8194152/webrev/ Regards, Dora From andrej.golovnin at gmail.com Fri Jun 22 08:32:38 2018 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Fri, 22 Jun 2018 10:32:38 +0200 Subject: [PATCH] Minor optimisation for Set.copyOf(Collection) Message-ID: Hi all, the current implementation of Set.copyOf(Collection coll) creates an intermediate HashSet object to remove duplicates even when coll is already a Set. The attached patch adds an additional check whether coll is already a Set and if yes, then calls #toArray() directly on coll and passes the result to Set.of(). And when I have already your attention: the constructors of SetN and MapN may throw a NegativeArraySizeException when the length of the input is greater than Integer.MAX_VALUE / 2 + 1. Maybe this should be documented or we should throw IllegalArgumentException and explain what's went wrong. Best regards, Andrej Golovnin From Alan.Bateman at oracle.com Fri Jun 22 08:51:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 Jun 2018 09:51:14 +0100 Subject: Creating a charset provider module for IBM charsets In-Reply-To: References: <64173406-ab24-1620-a029-5b90e0acddb3@oracle.com> <0465abd4-b087-263c-baf9-eedbdf1d96fb@oracle.com> Message-ID: On 21/06/2018 11:41, Dave Hobbs wrote: > Yes, the proposal is to create a new module outside the JDK. > > : > > A service provider module appears to be the recommended approach according > the Java api documentation. However writing charset classes is made a lot > more difficult by not having the sun.nio.cs classes available. > > Should we propose adding our charsets in the java.base or jdk.charsets? > Is the context here additional EBCDIC code pages? I don't think they are interesting to too many applications and as you've found, there has always been a service provider interface add support for additional charsets. Additional CharsetProvider implementation can be deployed on either the class path or module path. Yes, it's a bit more work to develop a standalone CharsetProvider but there isn't anything special about the sun.nio.cs infrastructure except that it makes it easier when adding additional charsets in the JDK. -Alan From andrej.golovnin at gmail.com Fri Jun 22 09:42:21 2018 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Fri, 22 Jun 2018 11:42:21 +0200 Subject: [PATCH] Minor optimisation for Set.copyOf(Collection) In-Reply-To: References: Message-ID: > Hi all, > > the current implementation of Set.copyOf(Collection coll) > creates an intermediate HashSet object to remove duplicates even when > coll is already a Set. The attached patch adds an additional check > whether coll is already a Set and if yes, then calls #toArray() > directly on coll and passes the result to Set.of(). It looks like my patch was stripped. Now inlined: diff --git a/src/java.base/share/classes/java/util/Set.java b/src/java.base/share/classes/java/util/Set.java --- a/src/java.base/share/classes/java/util/Set.java +++ b/src/java.base/share/classes/java/util/Set.java @@ -723,6 +723,8 @@ static Set copyOf(Collection coll) { if (coll instanceof ImmutableCollections.AbstractImmutableSet) { return (Set)coll; + } else if (coll instanceof Set) { + return (Set)Set.of(coll.toArray()); } else { return (Set)Set.of(new HashSet<>(coll).toArray()); } From Alan.Bateman at oracle.com Fri Jun 22 09:50:21 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 Jun 2018 10:50:21 +0100 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <36c14123-db0f-600d-10f5-ae8ca82f66c0@gmail.com> References: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> <36c14123-db0f-600d-10f5-ae8ca82f66c0@gmail.com> Message-ID: On 21/06/2018 18:29, Peter Levart wrote: > > > On 06/21/2018 07:01 PM, Tony Printezis wrote: >> I?m trying exactly that. :-) > > Sorry, I didn't know. Here's my attempt: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.07/ > > I also added @run main/othervm to TempDirectBuffersReclamation test. Right, tests depending on BufferPoolMXBean::getXXX do need to be /othervm to avoid interference from free'ing buffers that were used by previous tests in the same VM. If the name of the direct buffer pool is changed then it would be useful for this test to fail so that it can be updated. For that reason, I think it should fail if "direct" is not found, maybe just add orElseThrow to the pipeline. The rest looks okay to me (and although it took many iterations, I think we got it to a good place). -Alan From Alan.Bateman at oracle.com Fri Jun 22 10:10:51 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 Jun 2018 11:10:51 +0100 Subject: Promptly freeing the per-thread cached direct buffers when a thread exits In-Reply-To: <8736xfx5hg.fsf@mid.deneb.enyo.de> References: <8736xfx5hg.fsf@mid.deneb.enyo.de> Message-ID: <0582f569-195b-c5a5-0a5d-7edbcdb197a9@oracle.com> On 21/06/2018 21:13, Florian Weimer wrote: > * Tony Printezis: > >> There are a few obvious ways to mitigate this, e.g., cause a Full GC / >> concurrent GC cycle at regular intervals. However, the best solution IMHO >> is to explicitly free any direct buffers that are still in the cache when a >> thread exits. > Why is this safe? Couldn't these direct byte buffers be used with a > custom channel that leaks them to another thread? The temporary direct buffer mechanism is internal to java.base so it should never be used with custom channel implementations. There may be some pre-existing issues with the FileChannel transferXXX methods when called with an untrusted source/sink that will need to be audited but this is not something that I can discuss on this mailing list. -Alan From amaembo at gmail.com Fri Jun 22 10:21:59 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Fri, 22 Jun 2018 17:21:59 +0700 Subject: [PATCH] Minor optimisation for Set.copyOf(Collection) In-Reply-To: References: Message-ID: Hello! What if it's TreeSet with custom comparator which may differentiate elements equal by .equals()? Or, even simpler, Collections.newSetFromMap(new IdentityHashMap<>())? In both cases Set.of(array) would throw. With best regards, Tagir Valeev. On Fri, Jun 22, 2018 at 4:43 PM Andrej Golovnin wrote: > > Hi all, > > > > the current implementation of Set.copyOf(Collection coll) > > creates an intermediate HashSet object to remove duplicates even when > > coll is already a Set. The attached patch adds an additional check > > whether coll is already a Set and if yes, then calls #toArray() > > directly on coll and passes the result to Set.of(). > > It looks like my patch was stripped. Now inlined: > > diff --git a/src/java.base/share/classes/java/util/Set.java > b/src/java.base/share/classes/java/util/Set.java > --- a/src/java.base/share/classes/java/util/Set.java > +++ b/src/java.base/share/classes/java/util/Set.java > @@ -723,6 +723,8 @@ > static Set copyOf(Collection coll) { > if (coll instanceof ImmutableCollections.AbstractImmutableSet) { > return (Set)coll; > + } else if (coll instanceof Set) { > + return (Set)Set.of(coll.toArray()); > } else { > return (Set)Set.of(new HashSet<>(coll).toArray()); > } > From andrej.golovnin at gmail.com Fri Jun 22 10:42:22 2018 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Fri, 22 Jun 2018 12:42:22 +0200 Subject: [PATCH] Minor optimisation for Set.copyOf(Collection) In-Reply-To: References: Message-ID: Hi Tagir, > What if it's TreeSet with custom comparator which may differentiate elements > equal by .equals()? Or, even simpler, Collections.newSetFromMap(new > IdentityHashMap<>())? In both cases Set.of(array) would throw. You are right. I haven't thought about this cases. OK, then we should at least add an optimisation for the case where coll is an instance of HashSet: diff --git a/src/java.base/share/classes/java/util/Set.java b/src/java.base/share/classes/java/util/Set.java --- a/src/java.base/share/classes/java/util/Set.java +++ b/src/java.base/share/classes/java/util/Set.java @@ -723,6 +723,8 @@ static Set copyOf(Collection coll) { if (coll instanceof ImmutableCollections.AbstractImmutableSet) { return (Set)coll; + } else if (coll instanceof HashSet) { + return (Set)Set.of(coll.toArray()); } else { return (Set)Set.of(new HashSet<>(coll).toArray()); } Best regards, Andrej Golovnin From takiguc at linux.vnet.ibm.com Fri Jun 22 11:13:30 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 22 Jun 2018 20:13:30 +0900 Subject: COMPOUND_TEXT charset is missing on JDK11 In-Reply-To: References: <5b8b543cace0e6c9d58a1870217d16eb@linux.vnet.ibm.com> Message-ID: <838b0b515f20096edbb6478a29ac963b@linux.vnet.ibm.com> Thanks Alan. Now we can use OpenJDK JDK11 GUI feature on IBM AIX's CJK locales. And AIX's desktop is still CDE/Motif... We may need COMPOUND_TEXT charset for clipboard and DnD feature against CDE/Motif. To 2d-dev: Could you reconsider about COMPOUND_TEXT charset ? Thanks, Ichiroh Takiguchi IBM Japan, Ltd. On 2018-06-22 16:14, Alan Bateman wrote: > On 22/06/2018 06:47, Ichiroh Takiguchi wrote: >> Hello. >> >> JDK8 Linux build has COMPOUND_TEXT charset, but it seems it's missing >> on JDK11 Linux build. >> I checked Bug DB, I could not find out the reason why COMPOUND_TEXT >> charset was missing. > This was removed more than 3 years ago during JDK 9 development as > part of separating the desktop area into its own module. It was > removed, along with X11 charsets as part of JDK-8035302. Much of the > need for these charsets was legacy at that point, esp. with Motif > going away. Is the context for your question something in the font or > postscript printing areas? This may be something that needs to be > brought up on 2d-dev. > > -Alan From peter.levart at gmail.com Fri Jun 22 11:11:50 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Jun 2018 13:11:50 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> <36c14123-db0f-600d-10f5-ae8ca82f66c0@gmail.com> Message-ID: <0d920567-e036-4819-910c-47a35032815a@gmail.com> Hi Tony, On 06/21/2018 07:39 PM, Tony Printezis wrote: > Peter, > > The changes to TestMaxCachedBufferSize.java look fine. One point > though: Why do you need the TmpDirectBuffersReclamation.java test? In > TestMaxCachedBufferSize.java you just call checkDirectBuffers(0, 0); > after you the main thread calls join() on the workers? This could work, but I'd rather keep it like this so that each test tests just its own aspect. TestMaxCachedBufferSize takes a long time and only runs on big machines (@requires sun.arch.data.model == "64") while TmpDirectBuffersReclamation is quick and can run anywhere. Regards, Peter > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On June 21, 2018 at 1:29:38 PM, Peter Levart (peter.levart at gmail.com > ) wrote: > >> >> >> On 06/21/2018 07:01 PM, Tony Printezis wrote: >>> I?m trying exactly that. :-) >> >> Sorry, I didn't know. Here's my attempt: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.07/ >> >> I also added @run main/othervm to TempDirectBuffersReclamation test. >> >> Peter >> >>> >>> >>> ????? >>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>> >>> >>> >>> On June 21, 2018 at 12:59:58 PM, Peter Levart >>> (peter.levart at gmail.com ) wrote: >>> >>>> >>>> >>>> On 06/21/2018 06:17 PM, Tony Printezis wrote: >>>>> I was saying: I looked at TestMaxCachedBufferSize and, >>>>> unfortunately, I don?t think the test makes a lot of sense right >>>>> now as it checks the number / size of direct buffers after all the >>>>> threads terminate and, with this change, that should always be 0. >>>> >>>> You're right. The test makes no sense now. As I understand, the >>>> test checked that number/size of allocated temporary buffers did >>>> not exceed the estimated calculated size by checking the MXBean >>>> immediately after threads terminate. This will never happen after >>>> the change as they will all be freed before checking, regardless of >>>> how many were and how much was allocated. >>>> >>>> Perhaps the test should be modified to include a latch so that >>>> threads wait until the measurement is made and only then are >>>> allowed to exit... >>>> >>>> Regards, Peter >>>> >> From peter.levart at gmail.com Fri Jun 22 12:12:55 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Jun 2018 14:12:55 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <375195b2-3873-4dff-0d94-3f6f02235c14@gmail.com> <89893be3-0561-eac0-645e-122eca3fc2e1@gmail.com> <36c14123-db0f-600d-10f5-ae8ca82f66c0@gmail.com> Message-ID: On 06/22/2018 11:50 AM, Alan Bateman wrote: > On 21/06/2018 18:29, Peter Levart wrote: >> >> >> On 06/21/2018 07:01 PM, Tony Printezis wrote: >>> I?m trying exactly that. :-) >> >> Sorry, I didn't know. Here's my attempt: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.07/ >> >> I also added @run main/othervm to TempDirectBuffersReclamation test. > Right, tests depending on BufferPoolMXBean::getXXX do need to be > /othervm to avoid interference from free'ing buffers that were used by > previous tests in the same VM. If the name of the direct buffer pool > is changed then it would be useful for this test to fail so that it > can be updated. For that reason, I think it should fail if "direct" is > not found, maybe just add orElseThrow to the pipeline. > > The rest looks okay to me (and although it took many iterations, I > think we got it to a good place). > > -Alan > Thanks, Alan and Tony. I will make the test fail when "direct" BufferPoolMXBean is not found. I've just submitted the code to jdk/submit to see if there are any other issues. If all goes well, I'm going to push it. Regards, Peter From rachna.goel at oracle.com Fri Jun 22 13:54:15 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Fri, 22 Jun 2018 19:24:15 +0530 Subject: RFR: JDK-8205158: Update the .md files for 3rd party software Unicode 10.0, ICU 60.1, and CLDR v33 Message-ID: Hi, Kindly review fix to update legal files for Unicode, CLDR and ICU. Issue: https://bugs.openjdk.java.net/browse/JDK-8205158 Patch :http://cr.openjdk.java.net/~rgoel/JDK-8205158/webrev.02/ -- Thanks, Rachna From philip.race at oracle.com Fri Jun 22 14:21:14 2018 From: philip.race at oracle.com (Philip Race) Date: Fri, 22 Jun 2018 07:21:14 -0700 Subject: [OpenJDK 2D-Dev] COMPOUND_TEXT charset is missing on JDK11 In-Reply-To: <838b0b515f20096edbb6478a29ac963b@linux.vnet.ibm.com> References: <5b8b543cace0e6c9d58a1870217d16eb@linux.vnet.ibm.com> <838b0b515f20096edbb6478a29ac963b@linux.vnet.ibm.com> Message-ID: <5B2D05DA.9040108@oracle.com> The DnD & clipboard cases on such a desktop are the only possible use I can think of for this charset, so if it were to exist anywhere it would be in the desktop module, since it depends on all the X decoders that were moved there. But since MToolkit is gone so we don't have direct need for it. Nor does OpenJDK support an environment with a CDE/Motif desktop any more, that ended with the end of Solaris 10 support on JDK 8. So are not likely to be asked to paste compound text from the x clipboard .. Therefore this would need careful consideration, as we would never use it, and have no way to test it and it quite likely needs some work to run on JDK 11. Not just dump + hope. It may be better to remain in a port that needs it and can test it. -phil. On 6/22/18, 4:13 AM, Ichiroh Takiguchi wrote: > Thanks Alan. > > Now we can use OpenJDK JDK11 GUI feature on IBM AIX's CJK locales. > And AIX's desktop is still CDE/Motif... > We may need COMPOUND_TEXT charset for clipboard and DnD feature > against CDE/Motif. > > To 2d-dev: > Could you reconsider about COMPOUND_TEXT charset ? > > Thanks, > Ichiroh Takiguchi > IBM Japan, Ltd. > > On 2018-06-22 16:14, Alan Bateman wrote: >> On 22/06/2018 06:47, Ichiroh Takiguchi wrote: >>> Hello. >>> >>> JDK8 Linux build has COMPOUND_TEXT charset, but it seems it's >>> missing on JDK11 Linux build. >>> I checked Bug DB, I could not find out the reason why COMPOUND_TEXT >>> charset was missing. >> This was removed more than 3 years ago during JDK 9 development as >> part of separating the desktop area into its own module. It was >> removed, along with X11 charsets as part of JDK-8035302. Much of the >> need for these charsets was legacy at that point, esp. with Motif >> going away. Is the context for your question something in the font or >> postscript printing areas? This may be something that needs to be >> brought up on 2d-dev. >> >> -Alan > From tprintezis at twitter.com Fri Jun 22 14:24:14 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Fri, 22 Jun 2018 16:24:14 +0200 Subject: Promptly freeing the per-thread cached direct buffers when a thread exits In-Reply-To: <0582f569-195b-c5a5-0a5d-7edbcdb197a9@oracle.com> References: <8736xfx5hg.fsf@mid.deneb.enyo.de> <0582f569-195b-c5a5-0a5d-7edbcdb197a9@oracle.com> Message-ID: Can I also add that, when a buffer in the cache needs to be replaced with a new (and typically larger) one, the old buffer is explicitly freed. So, the code already assumes that buffers that are in the cache should not be reachable from anywhere else. Explicitly freeing them when the thread exits is consistent (IMHO) with this behavior. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 22, 2018 at 7:11:05 AM, Alan Bateman (alan.bateman at oracle.com) wrote: On 21/06/2018 21:13, Florian Weimer wrote: > * Tony Printezis: > >> There are a few obvious ways to mitigate this, e.g., cause a Full GC / >> concurrent GC cycle at regular intervals. However, the best solution IMHO >> is to explicitly free any direct buffers that are still in the cache when a >> thread exits. > Why is this safe? Couldn't these direct byte buffers be used with a > custom channel that leaks them to another thread? The temporary direct buffer mechanism is internal to java.base so it should never be used with custom channel implementations. There may be some pre-existing issues with the FileChannel transferXXX methods when called with an untrusted source/sink that will need to be audited but this is not something that I can discuss on this mailing list. -Alan From Roger.Riggs at Oracle.com Fri Jun 22 14:49:05 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 22 Jun 2018 10:49:05 -0400 Subject: RFR 8202292 : test/jdk/java/io/FileOutputStream/UnreferencedFOSClosesFd.java fails with "raw fd count wrong" In-Reply-To: References: Message-ID: Hi Brian, Mandy, Listing the open file descriptors can be a useful utility. Originally I thought it was a temporary and throw away code. To be a utility it needed to be a bit more general (sending the output to a particular stream). Thanks for the suggestions. Please review: ? http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/ Thanks, Roger On 6/21/2018 3:25 PM, Brian Burkhalter wrote: > Hi Roger, > > Are the three versions of listProcFD() identical? Is so, could this > method be put in?test/lib/jdk/test/lib/util/FileUtils.java instead? > > Thanks, > > Brian > > On Jun 21, 2018, at 11:59 AM, Roger Riggs > wrote: > >> Please review a test improvement to avoid spurious errors caused by >> concurrent changes in open files during the test. >> The other existing conditions in the test are sufficient to verify >> the correct finalize behavior. >> Some additional diagnostics are added and the tests removed from the >> problem list. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/index.html >> >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8202292 > From sean.coffey at oracle.com Fri Jun 22 14:50:01 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 22 Jun 2018 15:50:01 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest Message-ID: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> Following on from the recent JDK-8203629 code review, I'd like to propose enhancements on how we can record events in security libs. The introduction of the JFR libraries can give us much better ways of examining JDK actions. For the initial phase, I'm looking to enhance some key security library events in JDK 11 so that they can be either recorded to JFR, logged to a traditional logger, or both. Examples of how useful JFR recordings could be can be seen here : http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png securityProp_2.png gives an example of how the JFR recording can be queried to quickly locate events of interest (in this case, code setting the jdk.tls.* Security properties). I still need to clean up the TLSEvents testcase to improve test coverage and hope to do that in coming days. JBS record : * https://bugs.openjdk.java.net/browse/JDK-8148188 webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ -- Regards, Sean. From mandy.chung at oracle.com Fri Jun 22 16:27:35 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 22 Jun 2018 09:27:35 -0700 Subject: RFR: JDK-8205158: Update the .md files for 3rd party software Unicode 10.0, ICU 60.1, and CLDR v33 In-Reply-To: References: Message-ID: <34b3c8c1-f8fc-77ce-7dd9-eac6071bcd97@oracle.com> On 6/22/18 6:54 AM, Rachna Goel wrote: > Hi, > > Kindly review fix to update legal files for Unicode, CLDR and ICU. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8205158 > > Patch :http://cr.openjdk.java.net/~rgoel/JDK-8205158/webrev.02/ Looks okay. What are the issues that upgrades these libraries? Can you link them with JDK-8205158? Mandy From rachna.goel at oracle.com Fri Jun 22 16:28:20 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Fri, 22 Jun 2018 21:58:20 +0530 Subject: RFR: JDK-8205158: Update the .md files for 3rd party software Unicode 10.0, ICU 60.1, and CLDR v33 In-Reply-To: References: Message-ID: <7e48ad45-77f5-0176-246f-5857fce57127@oracle.com> Hi, Kindly ignore patch mentioned in previous mail. patch for review is : http://cr.openjdk.java.net/~rgoel/JDK-8205158/webrev.01/ Thanks, Rachna On 6/22/18 7:24 PM, Rachna Goel wrote: > Hi, > > Kindly review fix to update legal files for Unicode, CLDR and ICU. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8205158 > > Patch :http://cr.openjdk.java.net/~rgoel/JDK-8205158/webrev.02/ > -- Thanks, Rachna From rachna.goel at oracle.com Fri Jun 22 16:34:48 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Fri, 22 Jun 2018 22:04:48 +0530 Subject: RFR: JDK-8205158: Update the .md files for 3rd party software Unicode 10.0, ICU 60.1, and CLDR v33 In-Reply-To: <34b3c8c1-f8fc-77ce-7dd9-eac6071bcd97@oracle.com> References: <34b3c8c1-f8fc-77ce-7dd9-eac6071bcd97@oracle.com> Message-ID: Hi Mandy, Thanks for the review. Unicode 10.0.0 and ICU 60.2 were integrated with : https://bugs.openjdk.java.net/browse/JDK-8191410 and CLDR 33 was: https://bugs.openjdk.java.net/browse/JDK-8202537 Linked mentioned issues with this one. Thanks, Rachna On 6/22/18 9:57 PM, mandy chung wrote: > > > On 6/22/18 6:54 AM, Rachna Goel wrote: >> Hi, >> >> Kindly review fix to update legal files for Unicode, CLDR and ICU. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8205158 >> >> Patch :http://cr.openjdk.java.net/~rgoel/JDK-8205158/webrev.02/ > > Looks okay. > > What are the issues that upgrades these libraries? Can you link them > with JDK-8205158? > > Mandy -- Thanks, Rachna From peter.levart at gmail.com Fri Jun 22 16:51:31 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Jun 2018 18:51:31 +0200 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: Hi Stuart, Do you still fear that "single-threaded-object" idiom is not safe? I would just like to comment on the last approach... On 06/19/2018 07:01 PM, Peter Levart wrote: > Here's another attempt that uses the same technique but relaxes the > possible concurrent source map scenario (where number of pairs emitted > by Map.forEach() doesn't agree with Map.size()): > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.03/ > > > The construction is moved entirely into a separate MapBuilder object. > The builder is used for Map.of(...) methods too. > > Here are the benchmark results that show improvements in every respect > touched by the patch: > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench_results.txt > > > Here are the JMH benchmarks used to produce these results: > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapCollectorBench.java > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapCopyOfBench.java > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapOfBench.java > > > > So what do you think of this one? > > Regards, Peter When this is used to copy concurrently changing concurrent Map, it has approximately the same performance consequence as when using Map.entrySet().toArray(). Either logic uses the estimated size to pre-size the array, but when the size is a moving target, it fails, so it has to do another copy at the end. My approach uses the estimated size to collect elements into the ready-made hash-table. When it fails, it has to do another copy, but in general case when the estimate is correct, it doesn't need an intermediary representation in the form of array, so it can be more optimal. Regards, Peter From mandy.chung at oracle.com Fri Jun 22 16:55:25 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 22 Jun 2018 09:55:25 -0700 Subject: RFR: JDK-8205158: Update the .md files for 3rd party software Unicode 10.0, ICU 60.1, and CLDR v33 In-Reply-To: References: <34b3c8c1-f8fc-77ce-7dd9-eac6071bcd97@oracle.com> Message-ID: <3157f641-64f7-cf45-8f1e-0b628aa4ccbc@oracle.com> webrev.01 looks better and it wraps long lines. Mandy On 6/22/18 9:34 AM, Rachna Goel wrote: > Hi Mandy, > > Thanks for the review. > > Unicode 10.0.0 and ICU 60.2 were integrated with : > https://bugs.openjdk.java.net/browse/JDK-8191410 > > and CLDR 33 was: https://bugs.openjdk.java.net/browse/JDK-8202537 > > Linked mentioned issues with this one. > > Thanks, > > Rachna > > > On 6/22/18 9:57 PM, mandy chung wrote: >> >> >> On 6/22/18 6:54 AM, Rachna Goel wrote: >>> Hi, >>> >>> Kindly review fix to update legal files for Unicode, CLDR and ICU. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8205158 >>> >>> Patch :http://cr.openjdk.java.net/~rgoel/JDK-8205158/webrev.02/ >> >> Looks okay. >> >> What are the issues that upgrades these libraries?? Can you link them >> with JDK-8205158? >> >> Mandy > > -- > Thanks, > Rachna > From naoto.sato at oracle.com Fri Jun 22 17:10:05 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 22 Jun 2018 10:10:05 -0700 Subject: RFR: JDK-8205158: Update the .md files for 3rd party software Unicode 10.0, ICU 60.1, and CLDR v33 In-Reply-To: <3157f641-64f7-cf45-8f1e-0b628aa4ccbc@oracle.com> References: <34b3c8c1-f8fc-77ce-7dd9-eac6071bcd97@oracle.com> <3157f641-64f7-cf45-8f1e-0b628aa4ccbc@oracle.com> Message-ID: +1 Naoto On 6/22/18 9:55 AM, mandy chung wrote: > webrev.01 looks better and it wraps long lines. > > Mandy > > On 6/22/18 9:34 AM, Rachna Goel wrote: >> Hi Mandy, >> >> Thanks for the review. >> >> Unicode 10.0.0 and ICU 60.2 were integrated with : >> https://bugs.openjdk.java.net/browse/JDK-8191410 >> >> and CLDR 33 was: https://bugs.openjdk.java.net/browse/JDK-8202537 >> >> Linked mentioned issues with this one. >> >> Thanks, >> >> Rachna >> >> >> On 6/22/18 9:57 PM, mandy chung wrote: >>> >>> >>> On 6/22/18 6:54 AM, Rachna Goel wrote: >>>> Hi, >>>> >>>> Kindly review fix to update legal files for Unicode, CLDR and ICU. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8205158 >>>> >>>> Patch :http://cr.openjdk.java.net/~rgoel/JDK-8205158/webrev.02/ >>> >>> Looks okay. >>> >>> What are the issues that upgrades these libraries?? Can you link them >>> with JDK-8205158? >>> >>> Mandy >> >> -- >> Thanks, >> Rachna >> From naoto.sato at oracle.com Fri Jun 22 17:26:17 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 22 Jun 2018 10:26:17 -0700 Subject: [11]RFR 8194152: sun/security/tools/jarsigner/AltProvider.java failed on de-DE locale In-Reply-To: <09f92268-fe4d-3c57-0ffb-d93d9703974d@oracle.com> References: <09f92268-fe4d-3c57-0ffb-d93d9703974d@oracle.com> Message-ID: Hi Dora, You could move those two lines that sets the locale to en-US before the for-loop, so that if "args" contains -J-Duser.language/country then it can override the default en-US. The JIRA issue needs noreg-self label. Naoto On 6/22/18 1:26 AM, Dora Zhou wrote: > Hello, > > Please help review the fix for JDK-8194152. Thank you. > > The test compares output with expected error messages in English, set > locale to en-US so that the output are not translated into other languages. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8194152 > Webrev: http://cr.openjdk.java.net/~ljiang/8194152/webrev/ > > > Regards, > Dora From paul.sandoz at oracle.com Fri Jun 22 17:28:45 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 22 Jun 2018 10:28:45 -0700 Subject: RFR: jsr166 integration 2018-06 In-Reply-To: References: Message-ID: > On Jun 21, 2018, at 6:28 PM, Martin Buchholz wrote: > > JDK 11 draws nigh. > Indeed, comments below, Paul. > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > > 8202422: value of 'sizeCtl' in ConcurrentHashMap varies with the constructor called > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/sizeCtl/index.html > https://bugs.openjdk.java.net/browse/JDK-8202422 > +1 > 8203864: Execution error in Java's Timsort > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/TimSort/index.html > https://bugs.openjdk.java.net/browse/JDK-8203864 > 406 if (n > 0 && runLen[n-1] <= runLen[n] + runLen[n+1] || 407 n > 1 && runLen[n-2] <= runLen[n] + runLen[n-1]) { 408 if (runLen[n - 1] < runLen[n + 1]) 409 n--; Minor comment, would be good if consistent style is used for the index expressions. I have a marginal preference to retaining ordering from a ?picture this in my head" kind of thing, specifically: runLen[n - 2] <= runLen[n - 1] + runLen[n] > --- > We've been unable to reach agreement on 8203662 - it remains in limbo. > > 8203662: remove increment of modCount from ArrayList and Vector replaceAll() > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/replaceAll/index.html > https://bugs.openjdk.java.net/browse/JDK-8203662 > I thought there was enough consensus to go forward with this. I accept the outcome. > 8203681: Miscellaneous changes imported from jsr166 CVS 2018-06 > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html > https://bugs.openjdk.java.net/browse/JDK-8203681 > TimeUnit.java ? * is equivalent to * {@code unit.convert(n, NANOSECONDS)}, and * {@code unit.convert(Duration.of(n, unit.toChronoUnit()))} * is equivalent to {@code n} (in the absence of overflow). * + *

This method differs from {@link Duration#toNanos()} in that + * it does not throw {@link ArithmeticException} on numeric overflow. + * Is that intended to be an api note? From brian.burkhalter at oracle.com Fri Jun 22 17:41:35 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 22 Jun 2018 10:41:35 -0700 Subject: RFR 8202292 : test/jdk/java/io/FileOutputStream/UnreferencedFOSClosesFd.java fails with "raw fd count wrong" In-Reply-To: References: Message-ID: <8CA310C6-E710-4B74-88D3-75250F8F2312@oracle.com> Hi Roger, Looks good! Thanks, Brian On Jun 22, 2018, at 7:49 AM, Roger Riggs wrote: > Listing the open file descriptors can be a useful utility. Originally I thought it was a temporary and throw away code. > To be a utility it needed to be a bit more general (sending the output to a particular stream). > Thanks for the suggestions. > > Please review: > http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/ From martinrb at google.com Fri Jun 22 17:45:32 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 22 Jun 2018 10:45:32 -0700 Subject: RFR: jsr166 integration 2018-06 In-Reply-To: References: Message-ID: On Fri, Jun 22, 2018 at 10:28 AM, Paul Sandoz wrote: > > TimeUnit.java > ? > > * is equivalent to > * {@code unit.convert(n, NANOSECONDS)}, and > * {@code unit.convert(Duration.of(n, unit.toChronoUnit()))} > * is equivalent to {@code n} (in the absence of overflow). > * > + *

This method differs from {@link Duration#toNanos()} in that > + * it does not throw {@link ArithmeticException} on numeric overflow. > + * > > Is that intended to be an api note? > > Done. --- src/main/java/util/concurrent/TimeUnit.java 10 Jun 2018 01:18:10 -0000 1.66 +++ src/main/java/util/concurrent/TimeUnit.java 22 Jun 2018 17:42:03 -0000 @@ -173,8 +173,9 @@ * {@code unit.convert(Duration.of(n, unit.toChronoUnit()))} * is equivalent to {@code n} (in the absence of overflow). * - *

This method differs from {@link Duration#toNanos()} in that - * it does not throw {@link ArithmeticException} on numeric overflow. + * @apiNote + * This method differs from {@link Duration#toNanos()} in that it + * does not throw {@link ArithmeticException} on numeric overflow. * * @param duration the time duration * @return the converted duration in this unit, From martinrb at google.com Fri Jun 22 17:51:39 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 22 Jun 2018 10:51:39 -0700 Subject: RFR: jsr166 integration 2018-06 In-Reply-To: References: Message-ID: On Fri, Jun 22, 2018 at 10:28 AM, Paul Sandoz wrote: > > > --- > > We've been unable to reach agreement on 8203662 - it remains in limbo. > > > > 8203662: remove increment of modCount from ArrayList and Vector > replaceAll() > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166- > integration/replaceAll/index.html > > https://bugs.openjdk.java.net/browse/JDK-8203662 > > > > I thought there was enough consensus to go forward with this. I accept the > outcome. > CSR at https://bugs.openjdk.java.net/browse/JDK-8203704 From paul.sandoz at oracle.com Fri Jun 22 17:41:46 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 22 Jun 2018 10:41:46 -0700 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: <7bc18707-946b-e4a7-3bfe-0deefa12db57@redhat.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> <7bc18707-946b-e4a7-3bfe-0deefa12db57@redhat.com> Message-ID: <49D99DF1-9184-4705-9CAD-7DD612B5B4DF@oracle.com> Hi, > On Jun 21, 2018, at 9:32 AM, Andrew Dinn wrote: > > Hi Paul, > > Sorry for the delay in responding to this -- holiday and then an urgent > bug fix intervened . . . > > On 08/06/18 01:42, Paul Sandoz wrote: >> Sandhya gave an overview to a few of us Oracle folks. I agree with >> what Sandhya says regarding the API, a small surface, and on pursuing >> an unsafe intrinsic. I like it and would encourage the writing of a >> draft JEP, especially to give this visibility. > > Great! Thanks for your feedback (also to Sandhya). I'll start drafting a > JEP staright away. I'll also work on revising the current intrinsic > implementation so it is presented via Unsafe (which should be fairly > simple to achieve). > Great! >> It intersects with https://bugs.openjdk.java.net/browse/JDK-8153111 >> ((bf) Allocating ByteBuffer on heterogeneous memory), which is >> attempting to be more generic. > > Ok, thanks. I'll have a think about how we night try to integrate these > two approaches and see what I can work into the draft JEP. > My impressions are that your approach may be a sufficiently good step forward that we don?t need to introduce a new abstraction for buffer allocation. Vivek any views on this? >> We might also need to increase the velocity on >> https://bugs.openjdk.java.net/browse/JDK-8180628 (retrofit direct >> buffer support for size beyond gigabyte scales), and i would be very >> interested your views on this, how you might be currently working >> around such size limitations, and what buffer enhancements would work >> for you. > > I think Jonathan answered that better than I can in his response. > However, if this accelerates delivery of a fix for JDK-8180628 then all > to the good. > Agreed! Paul. From joe.darcy at oracle.com Fri Jun 22 18:00:02 2018 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 22 Jun 2018 11:00:02 -0700 Subject: RFR: jsr166 integration 2018-06 In-Reply-To: References: Message-ID: <69caa47e-9b23-f443-1d81-ac11e8bbdb6f@oracle.com> On 6/22/2018 10:51 AM, Martin Buchholz wrote: > On Fri, Jun 22, 2018 at 10:28 AM, Paul Sandoz > wrote: > >>> --- >>> We've been unable to reach agreement on 8203662 - it remains in limbo. >>> >>> 8203662: remove increment of modCount from ArrayList and Vector >> replaceAll() >>> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166- >> integration/replaceAll/index.html >>> https://bugs.openjdk.java.net/browse/JDK-8203662 >>> >> I thought there was enough consensus to go forward with this. I accept the >> outcome. >> > CSR at https://bugs.openjdk.java.net/browse/JDK-8203704 The CSR has been backed up review other requests ahead of JDK 11 ramp down 1; sorry for the delay. -Joe From patrick at reini.net Fri Jun 22 18:21:44 2018 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 22 Jun 2018 20:21:44 +0200 Subject: RFR JDK-8205090 / JDK-8204930 Message-ID: Hi everybody, The JDK 11 ramp down fase is coming soon and I would like to have that fixed before that. Please review the CSR and the proposed webrev: http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev Thanks for your replies and a sponsoring committer -Patrick From vivek.r.deshpande at intel.com Fri Jun 22 18:48:40 2018 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Fri, 22 Jun 2018 18:48:40 +0000 Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. In-Reply-To: <53E8E64DB2403849AFD89B7D4DAC8B2A9074AA58@ORSMSX106.amr.corp.intel.com> References: <53E8E64DB2403849AFD89B7D4DAC8B2A90748395@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748CE3@ORSMSX106.amr.corp.intel.com> <0821BE35-CAA7-4BAB-8F4E-FCAC90FAA48D@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90748E6A@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A907498AD@ORSMSX106.amr.corp.intel.com> <0EEFE6FB-71B3-4365-8A95-6E571716A1E8@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A90749B04@ORSMSX106.amr.corp.intel.com> <53E8E64DB2403849AFD89B7D4DAC8B2A9074A5D1@ORSMSX106.amr.corp.intel.com> <5A374FC1-26FB-40B7-A648-0F1527BC5A01@oracle.com> <53E8E64DB2403849AFD89B7D4DAC8B2A9074AA58@ORSMSX106.amr.corp.intel.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A9074C41B@ORSMSX106.amr.corp.intel.com> Hi Paul I have updated the patch in this webrev: http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.04/ Will push that soon. Regards, Vivek -----Original Message----- From: Deshpande, Vivek R Sent: Thursday, June 21, 2018 4:14 PM To: Paul Sandoz Cc: Roger.Riggs at oracle.com; David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya ; Ivan Gerasimov Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Thanks Paul for taking some time and thinking about it. I will make the change you have mentioned and send the patch again. Regards, Vivek -----Original Message----- From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Thursday, June 21, 2018 3:32 PM To: Deshpande, Vivek R Cc: Roger.Riggs at oracle.com; David Holmes ; core-libs-dev at openjdk.java.net; Viswanathan, Sandhya ; Ivan Gerasimov Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. Hi Vivek, This looks better. I thought a bit more about NaNs. I am concerned about edge case performance regressions if the first two values are NaNs (with the same bit pattern). Here?s a patch on top of your patch that compares the raw bits for float/double elements. If you are ok with this and apply it on top of your patch then i don?t think there is a need for another review round. Thanks, Paul. diff -r ef8fd07e4440 src/java.base/share/classes/java/nio/BufferMismatch.java --- a/src/java.base/share/classes/java/nio/BufferMismatch.java Thu Jun 21 13:43:04 2018 -0700 +++ b/src/java.base/share/classes/java/nio/BufferMismatch.java Thu Jun 21 14:02:19 2018 -0700 @@ -118,7 +118,7 @@ static int mismatch(FloatBuffer a, int aOff, FloatBuffer b, int bOff, int length) { int i = 0; if (length > 1 && a.order() == b.order()) { - if (a.get(aOff) == b.get(bOff)) { + if (Float.floatToRawIntBits(a.get(aOff)) == + Float.floatToRawIntBits(b.get(bOff))) { i = ArraysSupport.vectorizedMismatch( a.base(), a.address + (aOff << ArraysSupport.LOG2_ARRAY_FLOAT_INDEX_SCALE), b.base(), b.address + (bOff << ArraysSupport.LOG2_ARRAY_FLOAT_INDEX_SCALE), @@ -175,7 +175,7 @@ static int mismatch(DoubleBuffer a, int aOff, DoubleBuffer b, int bOff, int length) { int i = 0; if (length > 0 && a.order() == b.order()) { - if (a.get(aOff) == b.get(bOff)) { + if (Double.doubleToRawLongBits(a.get(aOff)) == + Double.doubleToRawLongBits(b.get(bOff))) { i = ArraysSupport.vectorizedMismatch( a.base(), a.address + (aOff << ArraysSupport.LOG2_ARRAY_DOUBLE_INDEX_SCALE), b.base(), b.address + (bOff << ArraysSupport.LOG2_ARRAY_DOUBLE_INDEX_SCALE), diff -r ef8fd07e4440 src/java.base/share/classes/jdk/internal/util/ArraysSupport.java --- a/src/java.base/share/classes/jdk/internal/util/ArraysSupport.java Thu Jun 21 13:43:04 2018 -0700 +++ b/src/java.base/share/classes/jdk/internal/util/ArraysSupport.java Thu Jun 21 14:02:19 2018 -0700 @@ -461,7 +461,7 @@ int length) { int i = 0; if (length > 1) { - if (a[aFromIndex] == b[bFromIndex]) { + if (Float.floatToRawIntBits(a[aFromIndex]) == + Float.floatToRawIntBits(b[bFromIndex])) { int aOffset = Unsafe.ARRAY_FLOAT_BASE_OFFSET + (aFromIndex << LOG2_ARRAY_FLOAT_INDEX_SCALE); int bOffset = Unsafe.ARRAY_FLOAT_BASE_OFFSET + (bFromIndex << LOG2_ARRAY_FLOAT_INDEX_SCALE); i = vectorizedMismatch( @@ -545,7 +545,7 @@ return -1; } int i = 0; - if (a[aFromIndex] == b[bFromIndex]) { + if (Double.doubleToRawLongBits(a[aFromIndex]) == + Double.doubleToRawLongBits(b[bFromIndex])) { int aOffset = Unsafe.ARRAY_DOUBLE_BASE_OFFSET + (aFromIndex << LOG2_ARRAY_DOUBLE_INDEX_SCALE); int bOffset = Unsafe.ARRAY_DOUBLE_BASE_OFFSET + (bFromIndex << LOG2_ARRAY_DOUBLE_INDEX_SCALE); i = vectorizedMismatch( > On Jun 21, 2018, at 11:50 AM, Deshpande, Vivek R wrote: > > Hi Paul > > The folding of if/else is the right way in this patch which I missed in earlier webrev. > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 3/ I think I would keep the same earlier path if both the values are > NaN. > > Thanks. > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Wednesday, June 20, 2018 5:29 PM > To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } > 467 if (a[aFromIndex] == b[bFromIndex]) { > > Sorry, i don?t think i was clear. You can reduce down to a single equality check, so Lines #464-466 are redundant. This does result in taking the slow path if both values are NaN, which i think was the same for your previous patch. If that is important we can mitigate that by performing the negation of the same checks in the slow loop. > > WDYT? > > Paul. > > > On Jun 20, 2018, at 5:05 PM, Deshpande, Vivek R wrote: > > Hi Paul > > I have made the change you have suggested. > Please find the updated webrev at this location: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 2/ > > Regards, > Vivek > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Wednesday, June 20, 2018 2:30 PM > To: Deshpande, Vivek R > Cc: Roger.Riggs at oracle.com; David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > 459 public static int mismatch(float[] a, int aFromIndex, > 460 float[] b, int bFromIndex, > 461 int length) { > 462 int i = 0; > 463 if (length > 1) { > 464 if (a[aFromIndex] != b[bFromIndex]) { > 465 i = 0; > 466 } else { > > You fold the if/else to; > > if (a[aFromIndex] == b[bFromIndex]) { > ? > } > > same applies in other cases in both source files. > > > > > On Jun 20, 2018, at 1:21 PM, Deshpande, Vivek R wrote: > > Hi All > > I have updated the webrev with all the suggestions, which passes ArraysEqCmpTest.java. > Earlier webrev failed the same test. > > Thanks for confirming. > > > > This webrev also modifies the BufferMismatch.java in similar way. > > Great! > > Paul. > > > > Please take a look and let me know what you think. > > Updated webrev is here: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 1/ I have also modified the bug with updated webrev. > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:57 AM > To: Paul Sandoz ; Roger.Riggs at oracle.com > Cc: David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Roger > > I will also test with zero length arrays and let you know. > Thanks for the input. > > Regards, > Vivek > > From: Deshpande, Vivek R > Sent: Tuesday, June 19, 2018 10:17 AM > To: 'Paul Sandoz' > Cc: David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: RE: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Thanks Paul for quick review. I will work on the things you have mentioned and get back soon. > I will also test this with test/jdk/java/util/Arrays/ArraysEqCmpTest.java. > > Regards, > Vivek > > From: Paul Sandoz [mailto:paul.sandoz at oracle.com] > Sent: Tuesday, June 19, 2018 9:55 AM > To: Deshpande, Vivek R > Cc: David Holmes ; > core-libs-dev at openjdk.java.net; Viswanathan, Sandhya > > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Thanks for investigating this. > > 164 public static int mismatch(boolean[] a, > 165 boolean[] b, > 166 int length) { > 167 int i = 0; > 168 if (a[i] != b[i]) > 169 return i; > You might as well replace the use if i with 0 and i think you can move this to be performed when the length is greater than the threshold, that way you don?t impact small arrays below the threshold. > > 186 public static int mismatch(boolean[] a, int aFromIndex, > 187 boolean[] b, int bFromIndex, > 188 int length) { > 189 int i = 0; > 190 if (a[i] != b[i]) > 191 return i; > This is incorrect. You need to use aFromIndex and bFromIndex. > > Do you run the test test/jdk/java/util/Arrays/ArraysEqCmpTest.java? If that passed then we need to strengthen for the case of a mismatch on the first relative element in each array. > > Paul. > > > On Jun 19, 2018, at 9:36 AM, Deshpande, Vivek R wrote: > > Thanks David. > Sending it to core-libs-dev. > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Could you please review and sponsor the patch. > Link to bug: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards, > Vivek > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Monday, June 18, 2018 10:32 PM > To: Deshpande, Vivek R ; > jdk-dev at openjdk.java.net > Subject: Re: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi Vivek, > > Reviews should take place on the appropriate mailing list for the code being changed, not on the jdk-dev list. Please takes this to core-libs-dev. > > Thanks, > David > > On 19/06/2018 9:52 AM, Deshpande, Vivek R wrote: > > Hi All > > Forgot to add the links: > https://bugs.openjdk.java.net/browse/JDK-8205194 > webrev: > http://cr.openjdk.java.net/~vdeshpande/vectorizedMismatch_jdk/webrev.0 > 0/ > > Regards. > Vivek > > From: Deshpande, Vivek R > Sent: Monday, June 18, 2018 4:50 PM > To: 'jdk-dev at openjdk.java.net' > Cc: 'Paul Sandoz' ; Viswanathan, Sandhya > > Subject: RFR(S): 8205194: Improve the Array Comparison when there is a mismatch at first element. > > Hi All > > I would like to contribute a patch which improves the array comparison when there is a mismatch for the first element. > This avoids call to vectorizedMismatch method and gives ~80x speed up. > Please review and sponsor the patch. > > Regards, > Vivek > From mandy.chung at oracle.com Fri Jun 22 20:24:46 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 22 Jun 2018 13:24:46 -0700 Subject: RFR 8202292 : test/jdk/java/io/FileOutputStream/UnreferencedFOSClosesFd.java fails with "raw fd count wrong" In-Reply-To: References: Message-ID: +1 Mandy On 6/22/18 7:49 AM, Roger Riggs wrote: > Hi Brian, Mandy, > > Listing the open file descriptors can be a useful utility. Originally I > thought it was a temporary and throw away code. > To be a utility it needed to be a bit more general (sending the output > to a particular stream). > Thanks for the suggestions. > > Please review: > ? http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/ > > Thanks, Roger From smita.kamath at intel.com Fri Jun 22 21:49:56 2018 From: smita.kamath at intel.com (Kamath, Smita) Date: Fri, 22 Jun 2018 21:49:56 +0000 Subject: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions In-Reply-To: References: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> Message-ID: <6563F381B547594081EF9DE181D0791277AAD2D4@FMSMSX119.amr.corp.intel.com> Hi Vladimir, Please see my answers to your questions as below: 1) One question so: why you have own copy of base64 charsets and not using one in library: I am using vpgatherdd instruction to fetch multiple values from base64 array in a single instruction with a vector index. Vpgatherdd instruction works on 32 bit array and so I need to define base64 charset in a 32 bit array. I have given reference to gather instruction below. 2) Some indents in new and old Assembler::emit_operand() are off. In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. I'll make the necessary changes and send an updated webrev. 3) This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. I am using vpgatherdd instruction which requires index vector with scale. It uses VSIB addressing where the index register is a zmm register. Please refer to reference manual, volume 2c, page 2211: https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf Also see, section 2.3.12, page 524 for VSIB memory addressing information. 4) Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. I will add test cases as per your suggestion. Please let me know if you have additional questions. Thanks, Smita -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Friday, June 22, 2018 12:29 PM To: Kamath, Smita Cc: hotspot compiler Subject: Re: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions Hi Smita, I CCing to Libs to review code changes in Base64.java. Looks like you changed all need place to implement intrinsic. One question so: why you have own copy of base64 charsets and not using one in library: private int encode0(byte[] src, int off, int end, byte[] dst) { char[] base64 = isURL ? toBase64URL : toBase64; Some indents in new and old Assembler::emit_operand() are off. In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. What testing you did? Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. I know there is test/jdk/java/util/Base64/ tests but they may not trigger intrinsic usage. But you can use them as starting point for new tests. Thanks, Vladimir On 6/22/18 11:47 AM, Kamath, Smita wrote: > Hi Vladimir, > > I'd like to contribute an optimization for Base64 Encoding Algorithm > using AVX512 Instructions. This optimization shows 1.5x improvement on > x86_64 platform(SKL). > > Link to Bug: https://bugs.openjdk.java.net/browse/JDK-8205528 > > Link to webrev: > http://cr.openjdk.java.net/~vdeshpande/Base64/webrev.00/ > > For testing the implementation, I have run tests under > test/jdk/java/util/Base64/ and also ran > test/jdk/com/sun/jndi/ldap/Base64Test.java > > I have also run jtreg. > > Please review and let me know if you have any comments. > > Thanks and Regards, > > Smita > From vladimir.kozlov at oracle.com Fri Jun 22 19:29:16 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 22 Jun 2018 12:29:16 -0700 Subject: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions In-Reply-To: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> References: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> Message-ID: Hi Smita, I CCing to Libs to review code changes in Base64.java. Looks like you changed all need place to implement intrinsic. One question so: why you have own copy of base64 charsets and not using one in library: private int encode0(byte[] src, int off, int end, byte[] dst) { char[] base64 = isURL ? toBase64URL : toBase64; Some indents in new and old Assembler::emit_operand() are off. In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. What testing you did? Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. I know there is test/jdk/java/util/Base64/ tests but they may not trigger intrinsic usage. But you can use them as starting point for new tests. Thanks, Vladimir On 6/22/18 11:47 AM, Kamath, Smita wrote: > Hi Vladimir, > > I?d like to contribute an optimization for Base64 Encoding Algorithm > using AVX512 Instructions. This optimization shows 1.5x improvement on > x86_64 platform(SKL). > > Link to Bug: https://bugs.openjdk.java.net/browse/JDK-8205528 > > Link to webrev: http://cr.openjdk.java.net/~vdeshpande/Base64/webrev.00/ > > For testing the implementation, I have run tests under > test/jdk/java/util/Base64/ and also ran > test/jdk/com/sun/jndi/ldap/Base64Test.java > > I have also run jtreg. > > Please review and let me know if you have any comments. > > Thanks and Regards, > > Smita > From vladimir.kozlov at oracle.com Sat Jun 23 00:17:56 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 22 Jun 2018 17:17:56 -0700 Subject: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions In-Reply-To: References: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> <13665270-28A9-498D-A5BC-0CCCE1402856@oracle.com> <6563F381B547594081EF9DE181D0791277AAD2F6@FMSMSX119.amr.corp.intel.com> <71BBF912-081E-4385-8940-B8FE802F6699@oracle.com> <6563F381B547594081EF9DE181D0791277AAD356@FMSMSX119.amr.corp.intel.com> Message-ID: <3fc63a04-984b-d101-c5ac-a811f285d35c@oracle.com> On 6/22/18 3:58 PM, Paul Sandoz wrote: > Hi Smita, > > I am ok with it if Vladimir is :-) One slight concern is this may be biasing towards the x86 implementation of the intrinsic. Looking on code and it will be a lot of changes in Base64.java. I am concern about that late in JDK 11. I think we should keep duplicated code for x86 intrinsic as Smita suggested. And we can return to this when/if we intrinsify Decoder too. > I dunno if an int[] table is as useful for an AARCH64 intrinsic. We should ask RH to check. But I think SPARC is better operating on 32-bit values than 16-bit (at least it was issue before). Vladimir > > Paul. > >> On Jun 22, 2018, at 3:25 PM, Kamath, Smita wrote: >> >> Hi Paul, >> >> I can change the Java arrays(toBase64URL and toBase64) to int[] and use them in the intrinsic if it is acceptable. Please let me know. >> >> Thanks, >> Smita >> >> -----Original Message----- >> From: Paul Sandoz [mailto:paul.sandoz at oracle.com] >> Sent: Friday, June 22, 2018 3:18 PM >> To: Kamath, Smita >> Cc: Vladimir Kozlov ; hotspot compiler >> Subject: Re: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions >> >> >> >>> On Jun 22, 2018, at 3:00 PM, Kamath, Smita wrote: >>>> Looks like you changed all need place to implement intrinsic. >>>> One question so: why you have own copy of base64 charsets and not using one in library: >>>> >>>> private int encode0(byte[] src, int off, int end, byte[] dst) { >>>> char[] base64 = isURL ? toBase64URL : toBase64; >>>> >>> >>> Yes, especially if we converted those from char[] to byte[] (which might also improve the C2 generated code) and pass the selected byte[] to the intrinsic. >>> Smita>> I need an integer array in order to use vpgatherdd instruction with vector index. Vpgather instruction works on 32 bit array and so I need to define base64 charset in a 32 bit array. >>> >> >> Thanks, I also saw your reply to Vladimir and see why you need this [*]. We could still unify leveraging a Java int[] array at the expense of extra space required on non-intrisified platforms. IMHO the less stub code and the more Java code the better with regards to maintenance. >> >> >>> Naming wise for the Java methods here are some suggestions: >>> >>> generateImplEncode -> encodeBlockWithBoundsCheck implEncode -> >>> encodeBlock >>> >>> Also can generateImplEncode be private? >>> Smita>> I'll make these changes and send an updated webrev. >>> >> >> Ok. >> >> >>> Further. is there is a need to perform bounds checks in generateImplEncode given the public methods calling encode will, i presume, have dominating checks? >>> Smita>> The check is not required. I'll retain encodeBlock and remove encodeBlockWithBoundsCheck. >>> >> >> Ok. >> >> Thanks, >> Paul. >> >> [*] On AVX-512 it's tempting to explore permute/rearrange operations on bytes, if there are any such instructions, since the translation array of bytes (toBase64URL or toBase64) fits neatly into one z register, or for AVX-2 in two y registers if some masked variant, based on ranges, is possible. > From brian.burkhalter at oracle.com Sat Jun 23 00:26:32 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 22 Jun 2018 17:26:32 -0700 Subject: RFR JDK-8205090 / JDK-8204930 In-Reply-To: References: Message-ID: Hi Patrick, I reviewed the CSR with some minor changes as you will have seen. The webrev looks OK. I?ve not been following this thread that closely so perhaps someone else should also review it. I can be the committer if need be. Thanks, Brian On Jun 22, 2018, at 11:21 AM, Patrick Reinhart wrote: > The JDK 11 ramp down fase is coming soon and I would like to have that fixed before that. Please review the CSR and the proposed webrev: > > http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev > > Thanks for your replies and a sponsoring committer From peter.levart at gmail.com Sat Jun 23 10:11:32 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 23 Jun 2018 12:11:32 +0200 Subject: RFR JDK-8205540: hotspot/jtreg/vmTestbase/nsk/jdb/trace/trace001/trace001.java fails with Debuggee did not exit after 15 commands Message-ID: Please review the following tiny change that delays TerminatingThreadLocal class initialization until the 1st thread that actually uses thread locals, exits: http://cr.openjdk.java.net/~plevart/jdk-dev/8205540_TerminatingThreadLocal.opt/webrev.01/ Since JDK-8202788 has been pushed, a hotspot/jdb test started failing: ??? https://bugs.openjdk.java.net/browse/JDK-8205540 ...presumably because it is not expecting so many instructions to be executed at thread exit. This patch might help silence the test. Even if it doesn't silence the test, it is a change that delays one class initialization until it is needed and this is good. Regards, Peter From david.holmes at oracle.com Sat Jun 23 23:10:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Sun, 24 Jun 2018 09:10:23 +1000 Subject: RFR JDK-8205540: hotspot/jtreg/vmTestbase/nsk/jdb/trace/trace001/trace001.java fails with Debuggee did not exit after 15 commands In-Reply-To: References: Message-ID: <54cbb95a-8a34-d239-239f-913ca7a24ba7@oracle.com> Hi Peter, On 23/06/2018 8:11 PM, Peter Levart wrote: > Please review the following tiny change that delays > TerminatingThreadLocal class initialization until the 1st thread that > actually uses thread locals, exits: > > http://cr.openjdk.java.net/~plevart/jdk-dev/8205540_TerminatingThreadLocal.opt/webrev.01/ > > > Since JDK-8202788 has been pushed, a hotspot/jdb test started failing: > > ??? https://bugs.openjdk.java.net/browse/JDK-8205540 > > ...presumably because it is not expecting so many instructions to be > executed at thread exit. This patch might help silence the test. Even if > it doesn't silence the test, it is a change that delays one class > initialization until it is needed and this is good. That seems like a good change to make. Reviewed. Thanks, David > Regards, Peter > From chris.plummer at oracle.com Sun Jun 24 00:11:20 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Sat, 23 Jun 2018 17:11:20 -0700 Subject: RFR JDK-8205540: hotspot/jtreg/vmTestbase/nsk/jdb/trace/trace001/trace001.java fails with Debuggee did not exit after 15 commands Message-ID: <13f04ff7-1eb8-67f4-c8ad-1e8effd6caab@oracle.com> Hi Peter, The changes looks good and also the test passes now. Thanks for fixing this. Chris > Please review the following tiny change that delays > TerminatingThreadLocal class initialization until the 1st thread that > actually uses thread locals, exits: > > http://cr.openjdk.java.net/~plevart/jdk-dev/8205540_TerminatingThreadLocal.opt/webrev.01/ > > > Since JDK-8202788 has been pushed, a hotspot/jdb test started failing: > > https://bugs.openjdk.java.net/browse/JDK-8205540 > > ...presumably because it is not expecting so many instructions to be > executed at thread exit. This patch might help silence the test. Even if > it doesn't silence the test, it is a change that delays one class > initialization until it is needed and this is good. > > Regards, Peter From Alan.Bateman at oracle.com Sun Jun 24 06:16:50 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 24 Jun 2018 07:16:50 +0100 Subject: RFR JDK-8205540: hotspot/jtreg/vmTestbase/nsk/jdb/trace/trace001/trace001.java fails with Debuggee did not exit after 15 commands In-Reply-To: References: Message-ID: On 23/06/2018 11:11, Peter Levart wrote: > Please review the following tiny change that delays > TerminatingThreadLocal class initialization until the 1st thread that > actually uses thread locals, exits: > > http://cr.openjdk.java.net/~plevart/jdk-dev/8205540_TerminatingThreadLocal.opt/webrev.01/ > > > Since JDK-8202788 has been pushed, a hotspot/jdb test started failing: > > ??? https://bugs.openjdk.java.net/browse/JDK-8205540 > > ...presumably because it is not expecting so many instructions to be > executed at thread exit. This patch might help silence the test. Even > if it doesn't silence the test, it is a change that delays one class > initialization until it is needed and this is good. This looks okay. I assume an alternative would have been to change the test. -Alan From erik.gahlin at oracle.com Sun Jun 24 11:42:57 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Sun, 24 Jun 2018 13:42:57 +0200 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <422e546a-4bf4-e2a4-72cf-d4bc5be5abdf@oracle.com> References: <5B286533.6070500@oracle.com> <3bce8034-387e-6ca5-d04d-b7171a13f47b@oracle.com> <5B2A9607.9010206@oracle.com> <422e546a-4bf4-e2a4-72cf-d4bc5be5abdf@oracle.com> Message-ID: <5B2F83C1.2050207@oracle.com> On 2018-06-21 09:52, Alan Bateman wrote: > On 20/06/2018 18:59, Erik Gahlin wrote: >> : >> >>> Also all the methods are empty which makes me wonder if they should >>> be abstract (as the class is abstract) or whether it should be an >>> interface. >> Abstract is needed to prevent user from instantiating the class. >> >> The methods need to have a body, otherwise event classes in java.base >> would need to implement the methods, which would be cumbersome. We >> like to use a class as it simplifies the implementation if we know >> all event classes have a common ancestor with java.lang.Object as a >> parent. >> >> so we can be guaranteed that the class > I'm not sure that I understand the issues here but just to say that > jdk.internal.event is not exported so code outside of the JDK cannot > directly instantiate instances of classes in that package. Also > interfaces can have default methods which may go to your concern about > needing to implement each of the 6 methods. Once Event is cleaned up > with some javadoc then it will be easier to argue this one way or the > other. > > -Alan I have updated jdk.internal.event.Event class with comments. http://cr.openjdk.java.net/~egahlin/8203629.2/ It is basically a copy the comments in jdk.jfr.Event, but without reference to classes in the JFR module. The rationale for using a class instead of an interface has to do with the implementation of JFR. Markus brought up some reasons in his e-mail. I thought about mentioning JFR in the javadoc, and that the purpose of the class is to allow use of JFR without a compile time dependency on the JFR module, but decided not to, since I think the class could have a value on its own for JVMs that do not support JFR, but that want to do to something similar for the JDK. They could, for instance, add an implementation in the empty method bodies. I also removed an import statement in JDKEvents.java that referenced a test class that should not have been there. Erik From erik.gahlin at oracle.com Sun Jun 24 13:21:33 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Sun, 24 Jun 2018 15:21:33 +0200 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> Message-ID: <5B2F9ADD.1040809@oracle.com> Hi Sean, Some of the changes in the webrev belongs to JDK-8203629 and should be removed for clarity. Some initial comments: default.jfc, profile.jfr: The events should not have control="enable-exceptions". The purpose of the control attribute is so to provide parameterized configuration of events for JMC. As it is now, the security events will be enabled when a user turns on the exception events. X509CertEvent: You should use milliseconds since epoch to represent a date instead of a string value, i.e. @Label("Valid From") @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) public long validFrom; @Label("Valid Until") @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) public long validUntil; This following annotation adds little value @Description("Details of Security Property") I would either remove the annotation, or provide information that helps a user understand the event. For instance, what is X509, and what kind of certificates are we talking about? @Category("Java Application") I'm a bit worried that we will pollute the "Java Application" namespace if we add lots of JDK events in that category. We may want to add a new top level category "Java Development Kit", analogous to the "Java Virtual Machine" category, and put all security related events in subcategory, perhaps called "Security". @Label("X509Cert") The label should be human readable name, with spaces, title cased etc. I would recommend "X.509 Certificate". In general, avoid abbreviations like "certs" and instead use labels such as "Certificate Chain". The label should be short and not a full sentence. For details see, https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html I think it would be good to separate testing of JFR and logging into different files / tests. I would prefer that the test name is the same as the event prefixed with "Test", i.e TestX509CertificateEvent, as this is the pattern used by other JFR tests. I reworked one of the tests to how I like to see it: /* * @test * @key jfr * @library /test/lib * @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent */ public class TestX509CertificateEvent { private static final String CERTIFICATE_1 = "..."; private static final String CERTIFICATE_2 = "..."; public static void main(String... args) throws CertificateException { Recording r = new Recording(); r.enable(EventNames.X509Certificate).withoutStackTrace(); r.start(); CertificateFactory cf = CertificateFactory.getInstance("X.509"); cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_1.getBytes())); cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_2.getBytes())); // Make sure only one event per certificate cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_1.getBytes())); cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_2.getBytes())); r.stop(); List events = Events.fromRecording(r); Asserts.assertEquals(events.size(), 2, "Expected two X.509 Certificate events"); assertEvent(events, "1000", "SHA256withRSA", "CN=SSLCertificate, O=SomeCompany", "CN=Intermediate CA Cert, O=SomeCompany", "RSA", 2048); assertEvent(events, "1000", "SHA256withRSA", "CN=SSLCertificate, O=SomeCompany", "CN=Intermediate CA Cert, O=SomeCompany", "RSA", 2048); } private static void assertEvent(List events, String certID, String algId, String subject, String issuer, String keyType, int length) throws Exception { for (RecordedEvent e : events) { if (e.getString("serialNumber").equals(certID)) { Events.assertField(e, "algId").equal(algId); ... return; } } System.out.println(events); throw new Exception("Could not find event with Cert ID: " + certID); } } The reworked example uses the Events.assertField method, which will give context if the assertion fails. Keeping the test simple, means it can be analyzed quickly if it fails in testing. There is no new test framework to learn, or methods to search for, and it is usually not hard to determine if the failure is product, test or infrastructure related, and what component (team) should be assigned the issue. The use of EventNames.X509Certificate means all occurrences of the event can be tracked done in an IDE using find by reference. Thanks Erik > Following on from the recent JDK-8203629 code review, I'd like to > propose enhancements on how we can record events in security libs. The > introduction of the JFR libraries can give us much better ways of > examining JDK actions. For the initial phase, I'm looking to enhance > some key security library events in JDK 11 so that they can be either > recorded to JFR, logged to a traditional logger, or both. > > Examples of how useful JFR recordings could be can be seen here : > > http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png > http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png > http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png > http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png > > securityProp_2.png gives an example of how the JFR recording can be > queried to quickly locate events of interest (in this case, code > setting the jdk.tls.* Security properties). I still need to clean up > the TLSEvents testcase to improve test coverage and hope to do that in > coming days. > > JBS record : > * https://bugs.openjdk.java.net/browse/JDK-8148188 > > webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ > From Alan.Bateman at oracle.com Sun Jun 24 17:00:34 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 24 Jun 2018 18:00:34 +0100 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <5B2F83C1.2050207@oracle.com> References: <5B286533.6070500@oracle.com> <3bce8034-387e-6ca5-d04d-b7171a13f47b@oracle.com> <5B2A9607.9010206@oracle.com> <422e546a-4bf4-e2a4-72cf-d4bc5be5abdf@oracle.com> <5B2F83C1.2050207@oracle.com> Message-ID: <233f6052-644d-5bff-9acd-f77bb0affa2c@oracle.com> On 24/06/2018 12:42, Erik Gahlin wrote: > I have updated jdk.internal.event.Event class with comments. > > http://cr.openjdk.java.net/~egahlin/8203629.2/ > > It is basically a copy the comments in jdk.jfr.Event, but without > reference to classes in the JFR module. > > The rationale for using a class instead of an interface has to do with > the implementation of JFR. Markus brought up some reasons in his e-mail. > > I thought about mentioning JFR in the javadoc, and that the purpose of > the class is to allow use of JFR without a compile time dependency on > the JFR module, but decided not to, since I think the class could have > a value on its own for JVMs that do not support JFR, but that want to > do to something similar for the JDK. They could, for instance, add an > implementation in the empty method bodies. I read the mail from Markus and looked at the updated webrev. It's good to have some javadoc, just a bit strange to have an abstract class without any abstract methods. There isn't enough in the discussion here to understand the original rational for why jdk.jfr.Event is abstract, esp. as the constructor is protected so it can't be directed instantiated outside of the jdk.jfr package. Also an abstract class can extend a non-abstract class so abstract jdk.jfr.Event could extend a non-abstract jdk.internal.event.Event if you want to clean this up a bit. I don't want to get in the way of the current effort but at some point I think another set of eyes on the APIs here might help. BTW: My previous comment about the modifiers was mostly about jdk.jfr.Event. -Alan From Alan.Bateman at oracle.com Sun Jun 24 19:11:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 24 Jun 2018 20:11:04 +0100 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B29CADD.7090200@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> Message-ID: On 20/06/2018 04:32, Joe Wang wrote: > Thanks Alan.? I created 8205058 to capture your suggestions. Please > see below for more details. > > Changed the internal APIs to throw CCE instead. In the same way as the > previous changeset for 8201276, these methods are made specific for > the use cases (though they are now for Files.read/writeString only) so > that they are not mixed up with existing ones that may inadvertently > affect other usages. > > One thing to note is that MalformedInputException or > UnmappableCharacterException will lose one piece of information in > comparison to the existing IAE, that is where it happens (offset). > Should there be an improvement in the future, we could consider add it > back to this part of code. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 > webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ I see your point on MalformedInputException and UnmappableCharacterException so maybe we can submit a new issue to track the follow-on work. The update means a lot of duplication in StringCoding. Did you consider (or measure) having the private encode/decode methods take a parameter to indicate the exception handling? Sherman might have suggestions on this. In the tests, shouldn't testMalformedRead and testMalformedWrite be updated so that expectedExceptions lists the specify exception that is expected? If I read the update correctly then isn't checking for MalformedInputException and UnmappableCharacterException anywhere (it passes if IOException is thrown). -Alan From erik.gahlin at oracle.com Sun Jun 24 19:15:46 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Sun, 24 Jun 2018 21:15:46 +0200 Subject: RFR: 8203629: Produce events in the JDK without a dependency on jdk.jfr In-Reply-To: <233f6052-644d-5bff-9acd-f77bb0affa2c@oracle.com> References: <5B286533.6070500@oracle.com> <3bce8034-387e-6ca5-d04d-b7171a13f47b@oracle.com> <5B2A9607.9010206@oracle.com> <422e546a-4bf4-e2a4-72cf-d4bc5be5abdf@oracle.com> <5B2F83C1.2050207@oracle.com> <233f6052-644d-5bff-9acd-f77bb0affa2c@oracle.com> Message-ID: <5B2FEDE2.4040106@oracle.com> On 2018-06-24 19:00, Alan Bateman wrote: > On 24/06/2018 12:42, Erik Gahlin wrote: >> I have updated jdk.internal.event.Event class with comments. >> >> http://cr.openjdk.java.net/~egahlin/8203629.2/ >> >> It is basically a copy the comments in jdk.jfr.Event, but without >> reference to classes in the JFR module. >> >> The rationale for using a class instead of an interface has to do >> with the implementation of JFR. Markus brought up some reasons in his >> e-mail. >> >> I thought about mentioning JFR in the javadoc, and that the purpose >> of the class is to allow use of JFR without a compile time dependency >> on the JFR module, but decided not to, since I think the class could >> have a value on its own for JVMs that do not support JFR, but that >> want to do to something similar for the JDK. They could, for >> instance, add an implementation in the empty method bodies. > I read the mail from Markus and looked at the updated webrev. It's > good to have some javadoc, just a bit strange to have an abstract > class without any abstract methods. The design stems from jdk.jfr.Event. If methods in jdk.jfr.Event would be abstract, it would not be possible to declare them final and we want them to be final, as users are not supposed to override them. When the jdk.jfr.Event class is loaded, the final modifiers are removed, so we can generate bytecode for subclasses. This is a trick to prevent incorrect use of the API. Why not make the class non-abstract and rely on the protected constructor? We could do that, it would prevent direct instantiation, but the use of the abstract modifier also fits nicely with how the modifier is used for event hierarchies. Imagine the following hierarchy and reuse of the sql field: class SQLEvent extends jdk.jfr.Event { @Label("SQL"); String sql; } class SQLUpdateEvent extends SQLEvent { } class SQLInsertEvent extends SQLEvent { } class SelectEvent extends SQLEvent { } When users connect with JMC, or call FlightRecorder#getEventTypes(), they will get a list of all event types that have been registered in the JVM. In the above example, they would get the SQLEvent, which is not really meant to be an event that users can configure. If they declare SQLEvent abstract, it will be ignored by the JFR system. This reuse mechanism can be seen in jdk.jfr.events.AbstractJDKEvent Why not use @Registered(false) instead of the abstract modifier? We want that annotation to be inherited, so it is possible to use it for large set events in a hierarchy. The abstract modifier only operates on a particular class. If the jdk.jfr.Event class is also declared abstract, it can reuse the same mechanism. It is convenient from an implementation point of view, no special case for jdk.jfr.Event, and it may also hint to a user that this the class is not something they should instantiate. It also prevents instantiation using reflection and setAccessible(true). An interface could also prevent instantiation, but it may open for malicious implementations and it seems wrong to make into an interface when users are not really meant to implement it. It also doesn't work well with the JVM implementation. > There isn't enough in the discussion here to understand the original > rational for why jdk.jfr.Event is abstract, esp. as the constructor is > protected so it can't be directed instantiated outside of the jdk.jfr > package. Also an abstract class can extend a non-abstract class so > abstract jdk.jfr.Event could extend a non-abstract > jdk.internal.event.Event if you want to clean this up a bit. I don't > want to get in the way of the current effort but at some point I think > another set of eyes on the APIs here might help. > Thanks Alan. We can revisit the design later. Erik > BTW: My previous comment about the modifiers was mostly about > jdk.jfr.Event. > > -Alan From dl at cs.oswego.edu Sun Jun 24 20:29:03 2018 From: dl at cs.oswego.edu (Doug Lea) Date: Sun, 24 Jun 2018 16:29:03 -0400 Subject: RFR: jsr166 integration 2018-06 In-Reply-To: References: Message-ID: <302fbfa2-19ea-7216-686a-0f174d58fda1@cs.oswego.edu> On 06/22/2018 01:28 PM, Paul Sandoz wrote: >> 8203864: Execution error in Java's Timsort >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/TimSort/index.html >> https://bugs.openjdk.java.net/browse/JDK-8203864 >> > > 406 if (n > 0 && runLen[n-1] <= runLen[n] + runLen[n+1] || > 407 n > 1 && runLen[n-2] <= runLen[n] + runLen[n-1]) { > 408 if (runLen[n - 1] < runLen[n + 1]) > 409 n--; > > Minor comment, would be good if consistent style is used for the index expressions. I have a marginal preference to retaining ordering from a ?picture this in my head" kind of thing, specifically: > > runLen[n - 2] <= runLen[n - 1] + runLen[n] I marginally agree, but I'm keeping it this way just for boring consistency with published versions. -Doug From david.holmes at oracle.com Mon Jun 25 05:00:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 25 Jun 2018 15:00:51 +1000 Subject: RFR 8202292 : test/jdk/java/io/FileOutputStream/UnreferencedFOSClosesFd.java fails with "raw fd count wrong" In-Reply-To: References: Message-ID: <75daeb17-7dca-68fe-b3de-a8088fec0899@oracle.com> Hi Roger, On 23/06/2018 12:49 AM, Roger Riggs wrote: > Hi Brian, Mandy, > > Listing the open file descriptors can be a useful utility. Originally I > thought it was a temporary and throw away code. > To be a utility it needed to be a bit more general (sending the output > to a particular stream). > Thanks for the suggestions. You might want to look at the code Leo used for a hotspot test recently. It covers more locations for lsof and also supports pfiles for Solaris: http://hg.openjdk.java.net/jdk/jdk/rev/9ba6f5dfbe56 http://hg.openjdk.java.net/jdk/jdk/rev/1372f66e0a17 But there can be other locations like /usr/local/bin for lsof on Solaris too. David > Please review: > ? http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/ > > Thanks, Roger > > On 6/21/2018 3:25 PM, Brian Burkhalter wrote: >> Hi Roger, >> >> Are the three versions of listProcFD() identical? Is so, could this >> method be put in?test/lib/jdk/test/lib/util/FileUtils.java instead? >> >> Thanks, >> >> Brian >> >> On Jun 21, 2018, at 11:59 AM, Roger Riggs > > wrote: >> >>> Please review a test improvement to avoid spurious errors caused by >>> concurrent changes in open files during the test. >>> The other existing conditions in the test are sufficient to verify >>> the correct finalize behavior. >>> Some additional diagnostics are added and the tests removed from the >>> problem list. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/index.html >>> >>> >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8202292 >> > From david.holmes at oracle.com Mon Jun 25 05:19:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 25 Jun 2018 15:19:43 +1000 Subject: RFR JDK-8205540: hotspot/jtreg/vmTestbase/nsk/jdb/trace/trace001/trace001.java fails with Debuggee did not exit after 15 commands In-Reply-To: References: Message-ID: On 24/06/2018 4:16 PM, Alan Bateman wrote: > > > On 23/06/2018 11:11, Peter Levart wrote: >> Please review the following tiny change that delays >> TerminatingThreadLocal class initialization until the 1st thread that >> actually uses thread locals, exits: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/8205540_TerminatingThreadLocal.opt/webrev.01/ >> >> >> Since JDK-8202788 has been pushed, a hotspot/jdb test started failing: >> >> ??? https://bugs.openjdk.java.net/browse/JDK-8205540 >> >> ...presumably because it is not expecting so many instructions to be >> executed at thread exit. This patch might help silence the test. Even >> if it doesn't silence the test, it is a change that delays one class >> initialization until it is needed and this is good. > This looks okay. I assume an alternative would have been to change the > test. Presumably, but this is a much better fix. Termination isn't quite as fragile as initialization but still best not to introduce unnecessary changes. I hadn't noticed this aspect of the new thread-local changes. Cheers, David > -Alan From dan.z.zhou at oracle.com Mon Jun 25 07:04:16 2018 From: dan.z.zhou at oracle.com (Dora Zhou) Date: Mon, 25 Jun 2018 15:04:16 +0800 Subject: [11]RFR 8196213: sun/security/tools/jarsigner/warnings/NoTimestampTest.java test fails on ar_SA locale. Message-ID: <915dfad2-a373-77f7-c5a0-38df52ca9e70@oracle.com> Hello, Please help review the fix for JDK-8196213. Thank you. Set default locale to en-US so that the output display the date using Gregorian Calendar and Latn numbering system(0~9). Bug: https://bugs.openjdk.java.net/browse/JDK-8196213 Webrev: http://cr.openjdk.java.net/~ljiang/8196213/webrev/ Regards, Dora From dan.z.zhou at oracle.com Mon Jun 25 07:37:39 2018 From: dan.z.zhou at oracle.com (Dora Zhou) Date: Mon, 25 Jun 2018 15:37:39 +0800 Subject: [11]RFR 8194152: sun/security/tools/jarsigner/AltProvider.java failed on de-DE locale In-Reply-To: References: Message-ID: <44ed6c3a-4a19-fec3-edff-d9de27882c29@oracle.com> Hi Naoto, Thanks a lot for the review. I have made suggested changes, Kindly have a look at: http://cr.openjdk.java.net/~ljiang/8194152/webrev.01/ Regards, Dora > From: naoto.sato at oracle.com > To: dan.z.zhou at oracle.com, i18n-dev at openjdk.java.net, core-libs-dev at openjdk.java.net, security-dev at openjdk.java.net > Sent: Saturday, June 23, 2018 1:23:50 AM GMT +08:00 Beijing / Chongqing / Hong Kong / Urumqi > Subject: Re: [11]RFR 8194152: sun/security/tools/jarsigner/AltProvider.java failed on de-DE locale > > Hi Dora, > > You could move those two lines that sets the locale to en-US before the > for-loop, so that if "args" contains -J-Duser.language/country then it > can override the default en-US. > > The JIRA issue needs noreg-self label. > > Naoto > > On 6/22/18 1:26 AM, Dora Zhou wrote: >> Hello, >> >> Please help review the fix for JDK-8194152. Thank you. >> >> The test compares output with expected error messages in English, set >> locale to en-US so that the output are not translated into other languages. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8194152 >> Webrev: http://cr.openjdk.java.net/~ljiang/8194152/webrev/ >> >> >> Regards, >> Dora From oliver at agilechilli.com Mon Jun 25 10:55:48 2018 From: oliver at agilechilli.com (Oliver Kohll) Date: Mon, 25 Jun 2018 03:55:48 -0700 Subject: free(): corrupted unsorted chunks In-Reply-To: References: Message-ID: Hi Bernd, Many thanks for your suggestions. It looks like the native APR library was the issue, at least it hasn't crashed yet after removing that library https://tomcat.apache.org/tomcat-8.0-doc/apr.html (the libapr1 package in Ubuntu) and re-configuring the /etc/tomcat8/server.xml to use NIO for SSL, replacing org.apache.coyote.http11.Http11AprProtocol with org.apache.coyote.http11.Http11NioProtocol. I appreciate your help, it saved my bacon! Oliver On 21 June 2018 at 23:34:44, Oliver Kohll (oliver at agilechilli.com) wrote: Aha, I wonder if it's configured with the APR native library for SSL. I will check out that and those things. Thanks Oliver On 21 June 2018 at 21:43:24, Bernd Eckenfels (ecki at zusammenkunft.net) wrote: Are you using any native libraries in this VM, is there a hs_err or System core file? Can you run ?ldd? on the java launcher binary. Can you try a OpenJDK binary distribution independent of the OS compiled version? Gruss Bernd -- http://bernd.eckenfels.net From Roger.Riggs at Oracle.com Mon Jun 25 13:33:05 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 25 Jun 2018 09:33:05 -0400 Subject: RFR 8202292 : test/jdk/java/io/FileOutputStream/UnreferencedFOSClosesFd.java fails with "raw fd count wrong" In-Reply-To: <75daeb17-7dca-68fe-b3de-a8088fec0899@oracle.com> References: <75daeb17-7dca-68fe-b3de-a8088fec0899@oracle.com> Message-ID: <85f11e26-ea22-ff4b-5514-9b80b5878bd5@Oracle.com> Created new issue: https://bugs.openjdk.java.net/browse/JDK-8205610 Thanks for the suggestion David On 6/25/2018 1:00 AM, David Holmes wrote: > Hi Roger, > > On 23/06/2018 12:49 AM, Roger Riggs wrote: >> Hi Brian, Mandy, >> >> Listing the open file descriptors can be a useful utility. Originally >> I thought it was a temporary and throw away code. >> To be a utility it needed to be a bit more general (sending the >> output to a particular stream). >> Thanks for the suggestions. > > You might want to look at the code Leo used for a hotspot test > recently. It covers more locations for lsof and also supports pfiles > for Solaris: > > http://hg.openjdk.java.net/jdk/jdk/rev/9ba6f5dfbe56 > http://hg.openjdk.java.net/jdk/jdk/rev/1372f66e0a17 > > But there can be other locations like /usr/local/bin for lsof on > Solaris too. > > David > >> Please review: >> http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/ >> >> Thanks, Roger >> >> On 6/21/2018 3:25 PM, Brian Burkhalter wrote: >>> Hi Roger, >>> >>> Are the three versions of listProcFD() identical? Is so, could this >>> method be put in?test/lib/jdk/test/lib/util/FileUtils.java instead? >>> >>> Thanks, >>> >>> Brian >>> >>> On Jun 21, 2018, at 11:59 AM, Roger Riggs >> > wrote: >>> >>>> Please review a test improvement to avoid spurious errors caused by >>>> concurrent changes in open files during the test. >>>> The other existing conditions in the test are sufficient to verify >>>> the correct finalize behavior. >>>> Some additional diagnostics are added and the tests removed from >>>> the problem list. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-count-wrong-8202292/index.html >>>> >>>> >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8202292 >>> >> From patrick at reini.net Mon Jun 25 13:37:35 2018 From: patrick at reini.net (Patrick Reinhart) Date: Mon, 25 Jun 2018 15:37:35 +0200 Subject: RFR JDK-8205090 / JDK-8204930 In-Reply-To: References: Message-ID: <5E7BE94F-7BA2-4227-8E06-33F48DF2E778@reini.net> Can anyone take this? -Patrick > Am 23.06.2018 um 02:26 schrieb Brian Burkhalter : > > Hi Patrick, > > I reviewed the CSR with some minor changes as you will have seen. The webrev looks OK. I?ve not been following this thread that closely so perhaps someone else should also review it. I can be the committer if need be. > > Thanks, > > Brian > >> On Jun 22, 2018, at 11:21 AM, Patrick Reinhart wrote: >> >> The JDK 11 ramp down fase is coming soon and I would like to have that fixed before that. Please review the CSR and the proposed webrev: >> >> http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev >> >> Thanks for your replies and a sponsoring committer > From matthias.baesken at sap.com Mon Jun 25 13:55:37 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 25 Jun 2018 13:55:37 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives Message-ID: <0cee5b67a5d1476d8318837b770de382@sap.com> Hello, please review this small change that improve exception messages during manifest parsing of jar archives . Thanks, Matthias Bug : https://bugs.openjdk.java.net/browse/JDK-8205525 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8205525/ From Alan.Bateman at oracle.com Mon Jun 25 14:16:50 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jun 2018 15:16:50 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <0cee5b67a5d1476d8318837b770de382@sap.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> Message-ID: <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> On 25/06/2018 14:55, Baesken, Matthias wrote: > Hello, please review this small change that improve exception messages during manifest parsing of jar archives . > > Thanks, Matthias > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8205525 > > Webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205525/ This potentially leaks sensitive information and will require a security review. There was a similar discussion on net-dev recently related to leaking host names in exceptions. Something similar may be needed here. -Alan. From Roger.Riggs at Oracle.com Mon Jun 25 14:22:43 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 25 Jun 2018 10:22:43 -0400 Subject: RFR JDK-8205090 / JDK-8204930 In-Reply-To: <5E7BE94F-7BA2-4227-8E06-33F48DF2E778@reini.net> References: <5E7BE94F-7BA2-4227-8E06-33F48DF2E778@reini.net> Message-ID: <7833a35f-771e-70a5-a20a-d05faf8c04c3@Oracle.com> Hi Patrick, Looks fine,? thanks for the followup @Brian, please sponsor Thanks, Roger On 6/25/2018 9:37 AM, Patrick Reinhart wrote: > Can anyone take this? > > -Patrick > > Am 23.06.2018 um 02:26 schrieb Brian Burkhalter > >: > >> Hi Patrick, >> >> I reviewed the CSR with some minor changes as you will have seen. The >> webrev looks OK. I?ve not been following this thread that closely so >> perhaps someone else should also review it. I can be the committer if >> need be. >> >> Thanks, >> >> Brian >> >> On Jun 22, 2018, at 11:21 AM, Patrick Reinhart > > wrote: >> >>> The JDK 11 ramp down fase is coming soon and I would like to have >>> that fixed before that. Please review the CSR and the proposed webrev: >>> >>> http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev >>> >>> >>> Thanks for your replies and a sponsoring committer >> From matthias.baesken at sap.com Mon Jun 25 14:29:49 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 25 Jun 2018 14:29:49 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> Message-ID: <34b096660ca44a4cb3f674198cd51d35@sap.com> Hi, do you consider both the file name and line number as sensitive ? > > There was a similar discussion on net-dev recently related to > leaking host names in exceptions. Something similar may be needed here > Do you know the outcome of this discussion ? Best regards, Matthias > -----Original Message----- > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Sent: Montag, 25. Juni 2018 16:17 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net > Subject: Re: [RFR] 8205525 : Improve exception messages during manifest > parsing of jar archives > > On 25/06/2018 14:55, Baesken, Matthias wrote: > > Hello, please review this small change that improve exception messages > during manifest parsing of jar archives . > > > > Thanks, Matthias > > > > Bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8205525 > > > > Webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205525/ > This potentially leaks sensitive information and will require a security > review. There was a similar discussion on net-dev recently related to > leaking host names in exceptions. Something similar may be needed here. > > -Alan. From Alan.Bateman at oracle.com Mon Jun 25 14:51:49 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jun 2018 15:51:49 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <34b096660ca44a4cb3f674198cd51d35@sap.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> Message-ID: On 25/06/2018 15:29, Baesken, Matthias wrote: > Hi, do you consider both the file name and line number as sensitive ? > >> There was a similar discussion on net-dev recently related to >> leaking host names in exceptions. Something similar may be needed here >> > Do you know the outcome of this discussion ? > All the details are in JDK-8204233 and the associated CSR. -Alan From brian.burkhalter at oracle.com Mon Jun 25 15:18:45 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 25 Jun 2018 08:18:45 -0700 Subject: RFR JDK-8205090 / JDK-8204930 In-Reply-To: <7833a35f-771e-70a5-a20a-d05faf8c04c3@Oracle.com> References: <5E7BE94F-7BA2-4227-8E06-33F48DF2E778@reini.net> <7833a35f-771e-70a5-a20a-d05faf8c04c3@Oracle.com> Message-ID: Hi Patrick, Roger, On Jun 25, 2018, at 7:22 AM, Roger Riggs wrote: > Looks fine, thanks for the followup > > @Brian, please sponsor Will do today. Brian From Roger.Riggs at Oracle.com Mon Jun 25 15:24:41 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 25 Jun 2018 11:24:41 -0400 Subject: RFR 8205547 : FileChannel/CleanerTest.java fails due to expected FD count Message-ID: Please review a test improvement to avoid unexpected file opens/closes during the file channel CleanerTest. The test now follows the pattern of the other java/io/File*/UnreferencedXXXClosesFd tests by waiting for the cleaner and file descriptors to be reclaimed by GC. Webrev: ? http://cr.openjdk.java.net/~rriggs/webrev-fd-count-wrong-8205547/ Thanks, Roger From naoto.sato at oracle.com Mon Jun 25 16:03:55 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 25 Jun 2018 09:03:55 -0700 Subject: [11]RFR 8194152: sun/security/tools/jarsigner/AltProvider.java failed on de-DE locale In-Reply-To: <44ed6c3a-4a19-fec3-edff-d9de27882c29@oracle.com> References: <44ed6c3a-4a19-fec3-edff-d9de27882c29@oracle.com> Message-ID: <67097d88-8657-6bf2-8e45-3bdc52a2ffea@oracle.com> Looks good. Naoto On 6/25/18 12:37 AM, Dora Zhou wrote: > Hi Naoto, > > Thanks a lot for the review. > I have made suggested changes, Kindly have a look at: > http://cr.openjdk.java.net/~ljiang/8194152/webrev.01/ > > > Regards, > Dora > >> From: naoto.sato at oracle.com >> To: dan.z.zhou at oracle.com, i18n-dev at openjdk.java.net, >> core-libs-dev at openjdk.java.net, security-dev at openjdk.java.net >> Sent: Saturday, June 23, 2018 1:23:50 AM GMT +08:00 Beijing / >> Chongqing / Hong Kong / Urumqi >> Subject: Re: [11]RFR 8194152: >> sun/security/tools/jarsigner/AltProvider.java failed on de-DE locale >> >> Hi Dora, >> >> You could move those two lines that sets the locale to en-US before the >> for-loop, so that if "args" contains -J-Duser.language/country then it >> can override the default en-US. >> >> The JIRA issue needs noreg-self label. >> >> Naoto >> >> On 6/22/18 1:26 AM, Dora Zhou wrote: >>> Hello, >>> >>> Please help review the fix for JDK-8194152. Thank you. >>> >>> The test compares output with expected error messages in English, set >>> locale to en-US so that the output are not translated into other >>> languages. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8194152 >>> Webrev: http://cr.openjdk.java.net/~ljiang/8194152/webrev/ >>> >>> >>> Regards, >>> Dora > From paul.sandoz at oracle.com Mon Jun 25 16:09:00 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 25 Jun 2018 09:09:00 -0700 Subject: RFR: jsr166 integration 2018-06 In-Reply-To: <302fbfa2-19ea-7216-686a-0f174d58fda1@cs.oswego.edu> References: <302fbfa2-19ea-7216-686a-0f174d58fda1@cs.oswego.edu> Message-ID: > On Jun 24, 2018, at 1:29 PM, Doug Lea

wrote: > > On 06/22/2018 01:28 PM, Paul Sandoz wrote: > >>> 8203864: Execution error in Java's Timsort >>> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/TimSort/index.html >>> https://bugs.openjdk.java.net/browse/JDK-8203864 >>> >> >> 406 if (n > 0 && runLen[n-1] <= runLen[n] + runLen[n+1] || >> 407 n > 1 && runLen[n-2] <= runLen[n] + runLen[n-1]) { >> 408 if (runLen[n - 1] < runLen[n + 1]) >> 409 n--; >> >> Minor comment, would be good if consistent style is used for the index expressions. I have a marginal preference to retaining ordering from a ?picture this in my head" kind of thing, specifically: >> >> runLen[n - 2] <= runLen[n - 1] + runLen[n] > > I marginally agree, but I'm keeping it this way just for boring > consistency with published versions. > Ok, consistency overrules! Paul. From paul.sandoz at oracle.com Mon Jun 25 16:11:14 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 25 Jun 2018 09:11:14 -0700 Subject: RFR 8195650 Method references to VarHandle accessors In-Reply-To: <086A1684-4D97-4B6D-94F2-16A1261057B5@oracle.com> References: <086A1684-4D97-4B6D-94F2-16A1261057B5@oracle.com> Message-ID: <355600B2-AB78-4170-8B2B-4C7F754B6A85@oracle.com> Gentle reminder. I would like to get this reviews and pushed before the ramp down phase one kicks in this week. Paul. > On Jun 19, 2018, at 5:08 PM, Paul Sandoz wrote: > > Hi, > > Please review the following fix to ensure method references to VarHandle signature polymorphic methods are supported at runtime (specifically the method handle to a signature polymorphic method can be loaded from the constant pool): > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195650-varhandle-mref/webrev/ > > I also added a ?belts and braces? test to ensure a constant method handle to MethodHandle::invokeBasic cannot be loaded if outside of the j.l.invoke package. > > Paul. > From xueming.shen at oracle.com Mon Jun 25 16:13:40 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 25 Jun 2018 09:13:40 -0700 Subject: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions In-Reply-To: <6563F381B547594081EF9DE181D0791277AAD2D4@FMSMSX119.amr.corp.intel.com> References: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> <6563F381B547594081EF9DE181D0791277AAD2D4@FMSMSX119.amr.corp.intel.com> Message-ID: <5B3114B4.1090000@oracle.com> Hi Kamath, Instead of throwing an aiobe, should the generateImplEncode() be like void generateImplEncode(byte[] src, int sp, int sl, byte[] dst, int dp) { if (sp < sl) { implEncode(src, sp, sl, dst, dp, ...); } } Any benefit of separating it into its own method? Thanks, Sherman On 6/22/18, 2:49 PM, Kamath, Smita wrote: > Hi Vladimir, > > Please see my answers to your questions as below: > > 1) One question so: why you have own copy of base64 charsets and not using one in library: > I am using vpgatherdd instruction to fetch multiple values from base64 array in a single instruction with a vector index. Vpgatherdd instruction works on 32 bit array and so I need to define base64 charset in a 32 bit array. I have given reference to gather instruction below. > > 2) Some indents in new and old Assembler::emit_operand() are off. In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. > I'll make the necessary changes and send an updated webrev. > > 3) This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. > I am using vpgatherdd instruction which requires index vector with scale. It uses VSIB addressing where the index register is a zmm register. > Please refer to reference manual, volume 2c, page 2211: https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf > Also see, section 2.3.12, page 524 for VSIB memory addressing information. > > 4) Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. > I will add test cases as per your suggestion. > > Please let me know if you have additional questions. > > Thanks, > Smita > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Friday, June 22, 2018 12:29 PM > To: Kamath, Smita > Cc: hotspot compiler > Subject: Re: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions > > Hi Smita, > > I CCing to Libs to review code changes in Base64.java. > > Looks like you changed all need place to implement intrinsic. > One question so: why you have own copy of base64 charsets and not using one in library: > > private int encode0(byte[] src, int off, int end, byte[] dst) { > char[] base64 = isURL ? toBase64URL : toBase64; > > Some indents in new and old Assembler::emit_operand() are off. > > In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. > > This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. > > What testing you did? > > Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. > > I know there is test/jdk/java/util/Base64/ tests but they may not trigger intrinsic usage. But you can use them as starting point for new tests. > > Thanks, > Vladimir > > On 6/22/18 11:47 AM, Kamath, Smita wrote: >> Hi Vladimir, >> >> I'd like to contribute an optimization for Base64 Encoding Algorithm >> using AVX512 Instructions. This optimization shows 1.5x improvement on >> x86_64 platform(SKL). >> >> Link to Bug: https://bugs.openjdk.java.net/browse/JDK-8205528 >> >> Link to webrev: >> http://cr.openjdk.java.net/~vdeshpande/Base64/webrev.00/ >> >> For testing the implementation, I have run tests under >> test/jdk/java/util/Base64/ and also ran >> test/jdk/com/sun/jndi/ldap/Base64Test.java >> >> I have also run jtreg. >> >> Please review and let me know if you have any comments. >> >> Thanks and Regards, >> >> Smita >> From paul.sandoz at oracle.com Mon Jun 25 16:32:18 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 25 Jun 2018 09:32:18 -0700 Subject: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions In-Reply-To: <3fc63a04-984b-d101-c5ac-a811f285d35c@oracle.com> References: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> <13665270-28A9-498D-A5BC-0CCCE1402856@oracle.com> <6563F381B547594081EF9DE181D0791277AAD2F6@FMSMSX119.amr.corp.intel.com> <71BBF912-081E-4385-8940-B8FE802F6699@oracle.com> <6563F381B547594081EF9DE181D0791277AAD356@FMSMSX119.amr.corp.intel.com> <3fc63a04-984b-d101-c5ac-a811f285d35c@oracle.com> Message-ID: <3844C8EE-A299-4434-9341-833E3E5177B1@oracle.com> > On Jun 22, 2018, at 5:17 PM, Vladimir Kozlov wrote: > > On 6/22/18 3:58 PM, Paul Sandoz wrote: >> Hi Smita, >> I am ok with it if Vladimir is :-) One slight concern is this may be biasing towards the x86 implementation of the intrinsic. > > Looking on code and it will be a lot of changes in Base64.java. I am concern about that late in JDK 11. > > I think we should keep duplicated code for x86 intrinsic as Smita suggested. And we can return to this when/if we intrinsify Decoder too. > Agreed. Paul. >> I dunno if an int[] table is as useful for an AARCH64 intrinsic. > > We should ask RH to check. > > But I think SPARC is better operating on 32-bit values than 16-bit (at least it was issue before). > > Vladimir > From naoto.sato at oracle.com Mon Jun 25 16:43:42 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 25 Jun 2018 09:43:42 -0700 Subject: [11]RFR 8196213: sun/security/tools/jarsigner/warnings/NoTimestampTest.java test fails on ar_SA locale. In-Reply-To: <915dfad2-a373-77f7-c5a0-38df52ca9e70@oracle.com> References: <915dfad2-a373-77f7-c5a0-38df52ca9e70@oracle.com> Message-ID: <49bcf97c-0eec-4481-9114-918baa6b4f48@oracle.com> Looks good. Naoto On 6/25/18 12:04 AM, Dora Zhou wrote: > Hello, > > Please help review the fix for JDK-8196213. Thank you. > Set default locale to en-US so that the output display the date using > Gregorian Calendar and Latn numbering system(0~9). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8196213 > Webrev: http://cr.openjdk.java.net/~ljiang/8196213/webrev/ > > > Regards, > Dora > From huizhe.wang at oracle.com Mon Jun 25 16:48:02 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 25 Jun 2018 09:48:02 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> Message-ID: <5B311CC2.2060005@oracle.com> On 6/24/18, 12:11 PM, Alan Bateman wrote: > On 20/06/2018 04:32, Joe Wang wrote: >> Thanks Alan. I created 8205058 to capture your suggestions. Please >> see below for more details. >> >> Changed the internal APIs to throw CCE instead. In the same way as >> the previous changeset for 8201276, these methods are made specific >> for the use cases (though they are now for Files.read/writeString >> only) so that they are not mixed up with existing ones that may >> inadvertently affect other usages. >> >> One thing to note is that MalformedInputException or >> UnmappableCharacterException will lose one piece of information in >> comparison to the existing IAE, that is where it happens (offset). >> Should there be an improvement in the future, we could consider add >> it back to this part of code. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ > > I see your point on MalformedInputException and > UnmappableCharacterException so maybe we can submit a new issue to > track the follow-on work. Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 > > The update means a lot of duplication in StringCoding. Did you > consider (or measure) having the private encode/decode methods take a > parameter to indicate the exception handling? Sherman might have > suggestions on this. I did. I didn't like the code duplication at all. But it would be even messier if we reuse the code since the previous usage didn't declare checked exception, but did capture the position (offset) and length information, which means the new method while unnecessary to capture these information for Files read/writeString would still need to save them in case Exception occurs. Because of the complication, I thought you and Sherman would again prefer a specific methods than adding parameters (need fields as well). Furthermore, in the first round of the review, Sherman mentioned that he's been working on an improvement in this area. But I'll wait till Sherman could comment on this. > > In the tests, shouldn't testMalformedRead and testMalformedWrite be > updated so that expectedExceptions lists the specify exception that is > expected? If I read the update correctly then isn't checking for > MalformedInputException and UnmappableCharacterException anywhere (it > passes if IOException is thrown). MalformedInputException and UnmappableCharacterException are implementation details, the tests only verified what the spec required (IOE). -Joe > > -Alan > From martinrb at google.com Mon Jun 25 17:32:27 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 25 Jun 2018 10:32:27 -0700 Subject: RFR: jsr166 integration 2018-06 In-Reply-To: References: <302fbfa2-19ea-7216-686a-0f174d58fda1@cs.oswego.edu> Message-ID: Committed except for http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html 8203662: remove increment of modCount from ArrayList and Vector replaceAll() http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/replaceAll/index.html https://bugs.openjdk.java.net/browse/JDK-8203662 From paul.sandoz at oracle.com Mon Jun 25 17:41:38 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 25 Jun 2018 10:41:38 -0700 Subject: RFR 8205547 : FileChannel/CleanerTest.java fails due to expected FD count In-Reply-To: References: Message-ID: <869EBD5C-3E2E-400A-A268-A37956CC6892@oracle.com> Hi Roger, Are you missing the throwing of an exception when the fdCount != fdCount0? (and relatedly i could not tell if the print statements were for temporary debugging and you intended to remove them) Since you check before hand for support of UnixOperatingSystemMXBean there appears no need for the method getFdCount can you can reuse the local variable unixMxBean. (If the check fails it suggests the @requires is incorrect, implying the test should fail.) Paul. > On Jun 25, 2018, at 8:24 AM, Roger Riggs wrote: > > Please review a test improvement to avoid unexpected file opens/closes during the file channel CleanerTest. > The test now follows the pattern of the other java/io/File*/UnreferencedXXXClosesFd tests by waiting > for the cleaner and file descriptors to be reclaimed by GC. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-fd-count-wrong-8205547/ > > Thanks, Roger > From vladimir.kozlov at oracle.com Mon Jun 25 17:48:09 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 25 Jun 2018 10:48:09 -0700 Subject: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions In-Reply-To: <6563F381B547594081EF9DE181D0791277AAD2D4@FMSMSX119.amr.corp.intel.com> References: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> <6563F381B547594081EF9DE181D0791277AAD2D4@FMSMSX119.amr.corp.intel.com> Message-ID: <980cfd3b-a6c1-eeda-3add-3165feea4fb9@oracle.com> I forgot to reply to your answers. On 6/22/18 2:49 PM, Kamath, Smita wrote: > Hi Vladimir, > > Please see my answers to your questions as below: > > 1) One question so: why you have own copy of base64 charsets and not using one in library: > I am using vpgatherdd instruction to fetch multiple values from base64 array in a single instruction with a vector index. Vpgatherdd instruction works on 32 bit array and so I need to define base64 charset in a 32 bit array. I have given reference to gather instruction below. As was discussed in an other e-mail lets keep your copy. > > 2) Some indents in new and old Assembler::emit_operand() are off. In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. > I'll make the necessary changes and send an updated webrev. > > 3) This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. > I am using vpgatherdd instruction which requires index vector with scale. It uses VSIB addressing where the index register is a zmm register. > Please refer to reference manual, volume 2c, page 2211: https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf > Also see, section 2.3.12, page 524 for VSIB memory addressing information. Got it. Thanks for document reference. > > 4) Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. > I will add test cases as per your suggestion. Thanks, Vladimir > > Please let me know if you have additional questions. > > Thanks, > Smita > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Friday, June 22, 2018 12:29 PM > To: Kamath, Smita > Cc: hotspot compiler > Subject: Re: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions > > Hi Smita, > > I CCing to Libs to review code changes in Base64.java. > > Looks like you changed all need place to implement intrinsic. > One question so: why you have own copy of base64 charsets and not using one in library: > > private int encode0(byte[] src, int off, int end, byte[] dst) { > char[] base64 = isURL ? toBase64URL : toBase64; > > Some indents in new and old Assembler::emit_operand() are off. > > In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. > > This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. > > What testing you did? > > Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. > > I know there is test/jdk/java/util/Base64/ tests but they may not trigger intrinsic usage. But you can use them as starting point for new tests. > > Thanks, > Vladimir > > On 6/22/18 11:47 AM, Kamath, Smita wrote: >> Hi Vladimir, >> >> I'd like to contribute an optimization for Base64 Encoding Algorithm >> using AVX512 Instructions. This optimization shows 1.5x improvement on >> x86_64 platform(SKL). >> >> Link to Bug: https://bugs.openjdk.java.net/browse/JDK-8205528 >> >> Link to webrev: >> http://cr.openjdk.java.net/~vdeshpande/Base64/webrev.00/ >> >> For testing the implementation, I have run tests under >> test/jdk/java/util/Base64/ and also ran >> test/jdk/com/sun/jndi/ldap/Base64Test.java >> >> I have also run jtreg. >> >> Please review and let me know if you have any comments. >> >> Thanks and Regards, >> >> Smita >> From Roger.Riggs at Oracle.com Mon Jun 25 18:08:42 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 25 Jun 2018 14:08:42 -0400 Subject: RFR 8205547 : FileChannel/CleanerTest.java fails due to expected FD count In-Reply-To: <869EBD5C-3E2E-400A-A268-A37956CC6892@oracle.com> References: <869EBD5C-3E2E-400A-A268-A37956CC6892@oracle.com> Message-ID: Hi Paul, Updated the webrev in place: ? http://cr.openjdk.java.net/~rriggs/webrev-fd-count-wrong-8205547/ On 6/25/2018 1:41 PM, Paul Sandoz wrote: > Hi Roger, > > Are you missing the throwing of an exception when the fdCount != fdCount0? (and relatedly i could not tell if the print statements were for temporary debugging and you intended to remove them) That condition needed to be removed, the number of open file descriptors changes unpredictably during the test.? The check for the reclamation of the 'FileDescriptor' and 'closer' replaces the simple count of open file descriptors. In one case, I spotted a socket that showed up as open during the test (but the code does not use sockets). The failures have only been seen using mach5 and are not otherwise reproducible. I retained and improved the debugging information with some aspiration to determine what file descriptor was appearing or disappearing. > > Since you check before hand for support of UnixOperatingSystemMXBean there appears no need for the method getFdCount can you can reuse the local variable unixMxBean. (If the check fails it suggests the @requires is incorrect, implying the test should fail.) Right, I updated this case to look like the other java/io/FileXXXStream/UnreferencedXXXClosesFd tests that did not have the @requires or the pretest. The direct use of unixMxBean is fine; I reverted the addition of getFdCount(). I'll keep the instanceofUnixOperatingSystemMXBean as belt and suspenders. Thanks, Roger > > Paul. > > >> On Jun 25, 2018, at 8:24 AM, Roger Riggs wrote: >> >> Please review a test improvement to avoid unexpected file opens/closes during the file channel CleanerTest. >> The test now follows the pattern of the other java/io/File*/UnreferencedXXXClosesFd tests by waiting >> for the cleaner and file descriptors to be reclaimed by GC. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-fd-count-wrong-8205547/ >> >> Thanks, Roger >> From paul.sandoz at oracle.com Mon Jun 25 18:17:10 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 25 Jun 2018 11:17:10 -0700 Subject: RFR: 8205184: Delegating Iterator implementations that don't delegate forEachRemaining() In-Reply-To: References: Message-ID: <58D74C95-D0E5-4E08-8098-92D8ED2D4A53@oracle.com> Looks good. Sometimes in these cases i reach for proxy rather than explicit code. It tends to be more robust to certain changes at the expense of reflective complexity. You could probably reduce the test method names by removing the ?forEachRemainingIsDelegated_? given that can be inferred from the class name. Paul. > On Jun 21, 2018, at 8:06 PM, Martin Buchholz wrote: > > 8205184: Delegating Iterator implementations that don't delegate > forEachRemaining() > http://cr.openjdk.java.net/~martin/webrevs/jdk/delegate-forEachRemaining/ > https://bugs.openjdk.java.net/browse/JDK-8205184 From paul.sandoz at oracle.com Mon Jun 25 18:26:22 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 25 Jun 2018 11:26:22 -0700 Subject: RFR 8205547 : FileChannel/CleanerTest.java fails due to expected FD count In-Reply-To: References: <869EBD5C-3E2E-400A-A268-A37956CC6892@oracle.com> Message-ID: > On Jun 25, 2018, at 11:08 AM, Roger Riggs wrote: > > Hi Paul, > > Updated the webrev in place: > http://cr.openjdk.java.net/~rriggs/webrev-fd-count-wrong-8205547/ > > On 6/25/2018 1:41 PM, Paul Sandoz wrote: >> Hi Roger, >> >> Are you missing the throwing of an exception when the fdCount != fdCount0? (and relatedly i could not tell if the print statements were for temporary debugging and you intended to remove them) > That condition needed to be removed, the number of open file descriptors changes unpredictably > during the test. The check for the reclamation of the 'FileDescriptor' and 'closer' replaces > the simple count of open file descriptors. > > In one case, I spotted a socket that showed up as open during the test (but the code does not use sockets). > The failures have only been seen using mach5 and are not otherwise reproducible. > I retained and improved the debugging information with some aspiration to determine > what file descriptor was appearing or disappearing. Ah yes, i see now. Perhaps add a comment or something in the summary stating that failure will result in a time out? (No need for another review). >> >> Since you check before hand for support of UnixOperatingSystemMXBean there appears no need for the method getFdCount can you can reuse the local variable unixMxBean. (If the check fails it suggests the @requires is incorrect, implying the test should fail.) > Right, I updated this case to look like the other java/io/FileXXXStream/UnreferencedXXXClosesFd tests > that did not have the @requires or the pretest. > The direct use of unixMxBean is fine; I reverted the addition of getFdCount(). > > I'll keep the instanceofUnixOperatingSystemMXBean as belt and suspenders. > Ok. Paul. From Roger.Riggs at Oracle.com Mon Jun 25 18:45:08 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 25 Jun 2018 14:45:08 -0400 Subject: RFR 8205547 : FileChannel/CleanerTest.java fails due to expected FD count In-Reply-To: References: <869EBD5C-3E2E-400A-A268-A37956CC6892@oracle.com> Message-ID: <43502458-61f4-bb1b-041e-ecdb678b2787@Oracle.com> Thanks Paul, I'll add a comment. On 6/25/2018 2:26 PM, Paul Sandoz wrote: > > >> On Jun 25, 2018, at 11:08 AM, Roger Riggs > > wrote: >> >> Hi Paul, >> >> Updated the webrev in place: >> http://cr.openjdk.java.net/~rriggs/webrev-fd-count-wrong-8205547/ >> >> On 6/25/2018 1:41 PM, Paul Sandoz wrote: >>> Hi Roger, >>> >>> Are you missing the throwing of an exception when the fdCount != fdCount0? (and relatedly i could not tell if the print statements were for temporary debugging and you intended to remove them) >> That condition needed to be removed, the number of open file >> descriptors changes unpredictably >> during the test.? The check for the reclamation of the >> 'FileDescriptor' and 'closer' replaces >> the simple count of open file descriptors. >> >> In one case, I spotted a socket that showed up as open during the >> test (but the code does not use sockets). >> The failures have only been seen using mach5 and are not otherwise >> reproducible. >> I retained and improved the debugging information with some >> aspiration to determine >> what file descriptor was appearing or disappearing. > > Ah yes, i see now. Perhaps add a comment or something in the summary > stating that failure will result in a time out? (No need for another > review). > > >>> Since you check before hand for support of UnixOperatingSystemMXBean there appears no need for the method getFdCount can you can reuse the local variable unixMxBean. (If the check fails it suggests the @requires is incorrect, implying the test should fail.) >> Right, I updated this case to look like the other >> java/io/FileXXXStream/UnreferencedXXXClosesFd tests >> that did not have the @requires or the pretest. >> The direct use of unixMxBean is fine; I reverted the addition of >> getFdCount(). >> >> I'll keep the instanceofUnixOperatingSystemMXBean as belt and suspenders. >> > > Ok. > > Paul. From mandy.chung at oracle.com Mon Jun 25 18:46:40 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 25 Jun 2018 11:46:40 -0700 Subject: Review Request: JDK-8205623: Replace use of Class::getPackage with Class::getPackageName Message-ID: <863e0948-59c1-560a-b1d2-422d3a56b829@oracle.com> This patch replaces the use of Class::getPackage with Class::getPackageName in jdk.internal.reflect.ReflectionFactory, sun.util.resources.BreakIteratorResourceBundle, and javax.xml.catalog.CatalogMessages. Class::getPackageName avoids the overhead of constructing Package objects. http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8205623/webrev.00/ ReflectionFactory::packageEquals does not specify if the Class object is an array class. In the current implementation it returns true even if the input parameters are both array classes but in two different runtime package. I added an assert instead of retaining the current behavior. Mandy From Alan.Bateman at oracle.com Mon Jun 25 19:38:40 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jun 2018 20:38:40 +0100 Subject: Review Request: JDK-8205623: Replace use of Class::getPackage with Class::getPackageName In-Reply-To: <863e0948-59c1-560a-b1d2-422d3a56b829@oracle.com> References: <863e0948-59c1-560a-b1d2-422d3a56b829@oracle.com> Message-ID: <61fddda4-5ea7-caba-ff58-198a22be7bdd@oracle.com> On 25/06/2018 19:46, mandy chung wrote: > This patch replaces the use of Class::getPackage with > Class::getPackageName in jdk.internal.reflect.ReflectionFactory, > sun.util.resources.BreakIteratorResourceBundle, and > javax.xml.catalog.CatalogMessages.? Class::getPackageName avoids the > overhead of constructing Package objects. > > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8205623/webrev.00/ > > ReflectionFactory::packageEquals does not specify if the Class object > is an array class.? In the current implementation it returns true even > if the input parameters are both array classes but in two different > runtime package.? I added an assert instead of retaining the current > behavior. BreakIteratorResourceBundler and CatalogMessages look okay. For ReflectionFactory.packageEquals then it might be cleaner to put the assert first but what you have is okay too. -Alan From sean.coffey at oracle.com Mon Jun 25 20:22:10 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 25 Jun 2018 21:22:10 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <5B2F9ADD.1040809@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> Message-ID: <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> Many thanks for the review comments Erik. Replies inline. On 24/06/2018 14:21, Erik Gahlin wrote: > Hi Sean, > > Some of the changes in the webrev belongs to JDK-8203629 and should be > removed for clarity. > > Some initial comments: > > default.jfc, profile.jfr: > The events should not have control="enable-exceptions". The purpose of > the control attribute is so to provide parameterized configuration of > events for JMC.? As it is now, the security events will be enabled > when a user turns on the exception events. Makes sense. I'll remove that parameter. > > X509CertEvent: > You should use milliseconds since epoch to represent a date instead of > a string value, i.e. > > ??? @Label("Valid From") > ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) > ??? public long validFrom; > > ??? @Label("Valid Until") > ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) > ??? public long validUntil; > The CertificateValidity class operates on Date Object values. I'll work with the Date.getTime() method then (and update the Logger formatting) > This following annotation adds little value > > @Description("Details of Security Property") > > I would either remove the annotation, or provide information that > helps a user understand the event. For instance, what is X509, and > what kind of certificates are we talking about? Yes - that looks like the wrong Description. I'll review all of these fields. > > @Category("Java Application") > > I'm a bit worried that we will pollute the "Java Application" > namespace if we add lots of JDK events in that category. We may want > to add a new top level category "Java Development Kit", analogous to > the "Java Virtual Machine" category, and put all security related > events in subcategory, perhaps called "Security". Yes - Open to suggestions. "Security" sounds like a good suggestion. > > @Label("X509Cert") > > The label should be human readable name, with spaces, title cased etc. > I would recommend "X.509 Certificate". In general, avoid abbreviations > like "certs" and instead use labels such as "Certificate Chain". The > label should be short and not a full sentence. > > For details see, > https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html > > I think it would be good to separate testing of JFR and logging into > different files / tests. I would prefer that the test name is the same > as the event prefixed with "Test", i.e TestX509CertificateEvent, as > this is the pattern used by other JFR tests. > I'll take a look at that pattern again. I've separated out the current tests into an (a) outer test to analyze the logger output and (b) the inner test which checks for JFR correctness. I did include extra logic to make sure that the EventHelper configuration was working correctly. "Events.assertField" looks very helpful. Thanks for the pointer. Let me take on board the suggestions below and get a new webrev out on Tuesday. regards, Sean. > I reworked one of the tests to how I like to see it: > > /* > ?* @test > ?* @key jfr > ?* @library /test/lib > ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent > ?*/ > public class TestX509CertificateEvent { > > ??? private static final String CERTIFICATE_1 = "..."; > ??? private static final String CERTIFICATE_2 = "..."; > > ??? public static void main(String... args) throws CertificateException { > > ???????? Recording r = new Recording(); > ???????? r.enable(EventNames.X509Certificate).withoutStackTrace(); > ???????? r.start(); > > ???????? CertificateFactory cf = CertificateFactory.getInstance("X.509"); > ???????? cf.generateCertificate(new > ByteArrayInputStream(CERTIFICATE_1.getBytes())); > ???????? cf.generateCertificate(new > ByteArrayInputStream(CERTIFICATE_2.getBytes())); > > ???????? // Make sure only one event per certificate > ???????? cf.generateCertificate(new > ByteArrayInputStream(CERTIFICATE_1.getBytes())); > ???????? cf.generateCertificate(new > ByteArrayInputStream(CERTIFICATE_2.getBytes())); > > ???????? r.stop(); > > ???????? List events = Events.fromRecording(r); > ???????? Asserts.assertEquals(events.size(), 2, "Expected two X.509 > Certificate events"); > > ???????? assertEvent(events, "1000", "SHA256withRSA", > ??????????????????? "CN=SSLCertificate, O=SomeCompany", > ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", > ???????????????????? "RSA", 2048); > ???????? assertEvent(events, "1000", "SHA256withRSA", > ??????????????????? "CN=SSLCertificate, O=SomeCompany", > ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", > ???????????????????? "RSA", 2048); > ??? } > > ??? private static void assertEvent(List events, String > certID, String algId, String subject, > ??????????? String issuer, String keyType, int length) throws Exception { > > ??????? for (RecordedEvent e : events) { > ??????????? if (e.getString("serialNumber").equals(certID)) { > ??????????????? Events.assertField(e, "algId").equal(algId); > ??????????????? ... > ??????????????? return; > ??????????? } > ??????? } > ??????? System.out.println(events); > ??????? throw new Exception("Could not find event with Cert ID: " + > certID); > ??? } > } > > The reworked example uses the Events.assertField method, which will > give context if the assertion fails. Keeping the test simple, means it > can be analyzed quickly if it fails in testing. There is no new test > framework to learn, or methods to search for, and it is usually not > hard to determine if the failure is product, test or infrastructure > related, and what component (team) should be assigned the issue. The > use of EventNames.X509Certificate means all occurrences of the event > can be tracked done in an IDE using find by reference. > > Thanks > Erik > >> Following on from the recent JDK-8203629 code review, I'd like to >> propose enhancements on how we can record events in security libs. >> The introduction of the JFR libraries can give us much better ways of >> examining JDK actions. For the initial phase, I'm looking to enhance >> some key security library events in JDK 11 so that they can be either >> recorded to JFR, logged to a traditional logger, or both. >> >> Examples of how useful JFR recordings could be can be seen here : >> >> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >> >> securityProp_2.png gives an example of how the JFR recording can be >> queried to quickly locate events of interest (in this case, code >> setting the jdk.tls.* Security properties). I still need to clean up >> the TLSEvents testcase to improve test coverage and hope to do that >> in coming days. >> >> JBS record : >> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >> >> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >> > From john.r.rose at oracle.com Mon Jun 25 20:26:37 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 25 Jun 2018 13:26:37 -0700 Subject: RFR 8195650 Method references to VarHandle accessors In-Reply-To: <355600B2-AB78-4170-8B2B-4C7F754B6A85@oracle.com> References: <086A1684-4D97-4B6D-94F2-16A1261057B5@oracle.com> <355600B2-AB78-4170-8B2B-4C7F754B6A85@oracle.com> Message-ID: Good fix. Reviewed. > On Jun 25, 2018, at 9:11 AM, Paul Sandoz wrote: > > Gentle reminder. > > I would like to get this reviews and pushed before the ramp down phase one kicks in this week. > > Paul. > >> On Jun 19, 2018, at 5:08 PM, Paul Sandoz wrote: >> >> Hi, >> >> Please review the following fix to ensure method references to VarHandle signature polymorphic methods are supported at runtime (specifically the method handle to a signature polymorphic method can be loaded from the constant pool): >> >> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195650-varhandle-mref/webrev/ >> >> I also added a ?belts and braces? test to ensure a constant method handle to MethodHandle::invokeBasic cannot be loaded if outside of the j.l.invoke package. >> >> Paul. >> > From karen.kinnear at oracle.com Mon Jun 25 21:17:19 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Mon, 25 Jun 2018 17:17:19 -0400 Subject: RFR 8195650 Method references to VarHandle accessors In-Reply-To: <086A1684-4D97-4B6D-94F2-16A1261057B5@oracle.com> References: <086A1684-4D97-4B6D-94F2-16A1261057B5@oracle.com> Message-ID: <1E2D21BC-58DC-4B74-B2C8-3A430064F330@oracle.com> Looks good. Matches the existing JVMS. Thanks for the tests. thanks, Karen > On Jun 19, 2018, at 8:08 PM, Paul Sandoz wrote: > > Hi, > > Please review the following fix to ensure method references to VarHandle signature polymorphic methods are supported at runtime (specifically the method handle to a signature polymorphic method can be loaded from the constant pool): > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195650-varhandle-mref/webrev/ > > I also added a ?belts and braces? test to ensure a constant method handle to MethodHandle::invokeBasic cannot be loaded if outside of the j.l.invoke package. > > Paul. > From martinrb at google.com Mon Jun 25 21:50:41 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 25 Jun 2018 14:50:41 -0700 Subject: RFR: 8205184: Delegating Iterator implementations that don't delegate forEachRemaining() In-Reply-To: <58D74C95-D0E5-4E08-8098-92D8ED2D4A53@oracle.com> References: <58D74C95-D0E5-4E08-8098-92D8ED2D4A53@oracle.com> Message-ID: On Mon, Jun 25, 2018 at 11:17 AM, Paul Sandoz wrote: > Looks good. Sometimes in these cases i reach for proxy rather than > explicit code. It tends to be more robust to certain changes at the expense > of reflective complexity. > > Hmmm ... Never done that in a test before. > You could probably reduce the test method names by removing the > ?forEachRemainingIsDelegated_? given that can be inferred from the class > name. > test method names shortened. From stuart.marks at oracle.com Mon Jun 25 22:06:02 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 25 Jun 2018 15:06:02 -0700 Subject: RFR(s): 8203670: unmodifiable List iterator() implementations should not be ListIterators Message-ID: Hi all, Please review this small changeset that ensures that calling iterator() on an unmodifiable List (from List.of, et. al.) returns an instance that cannot be downcast to ListIterator and used as such. Bug: https://bugs.openjdk.java.net/browse/JDK-8203670 Webrev: http://cr.openjdk.java.net/~smarks/reviews/8203670/webrev.0/ Thanks, s'marks From huizhe.wang at oracle.com Mon Jun 25 22:41:58 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 25 Jun 2018 15:41:58 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B311CC2.2060005@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> Message-ID: <5B316FB6.7020308@oracle.com> Hi Alan, The test testMalformedRead and testMalformedWrite now expect an UnmappableCharacterException instead of IOE: webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev01/ Thanks, Joe On 6/25/18, 9:48 AM, Joe Wang wrote: > > > On 6/24/18, 12:11 PM, Alan Bateman wrote: >> On 20/06/2018 04:32, Joe Wang wrote: >>> Thanks Alan. I created 8205058 to capture your suggestions. Please >>> see below for more details. >>> >>> Changed the internal APIs to throw CCE instead. In the same way as >>> the previous changeset for 8201276, these methods are made specific >>> for the use cases (though they are now for Files.read/writeString >>> only) so that they are not mixed up with existing ones that may >>> inadvertently affect other usages. >>> >>> One thing to note is that MalformedInputException or >>> UnmappableCharacterException will lose one piece of information in >>> comparison to the existing IAE, that is where it happens (offset). >>> Should there be an improvement in the future, we could consider add >>> it back to this part of code. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ >> >> I see your point on MalformedInputException and >> UnmappableCharacterException so maybe we can submit a new issue to >> track the follow-on work. > > Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 > >> >> The update means a lot of duplication in StringCoding. Did you >> consider (or measure) having the private encode/decode methods take a >> parameter to indicate the exception handling? Sherman might have >> suggestions on this. > > I did. I didn't like the code duplication at all. But it would be even > messier if we reuse the code since the previous usage didn't declare > checked exception, but did capture the position (offset) and length > information, which means the new method while unnecessary to capture > these information for Files read/writeString would still need to save > them in case Exception occurs. Because of the complication, I thought > you and Sherman would again prefer a specific methods than adding > parameters (need fields as well). > > Furthermore, in the first round of the review, Sherman mentioned that > he's been working on an improvement in this area. But I'll wait till > Sherman could comment on this. > >> >> In the tests, shouldn't testMalformedRead and testMalformedWrite be >> updated so that expectedExceptions lists the specify exception that >> is expected? If I read the update correctly then isn't checking for >> MalformedInputException and UnmappableCharacterException anywhere (it >> passes if IOException is thrown). > > MalformedInputException and UnmappableCharacterException are > implementation details, the tests only verified what the spec required > (IOE). > > -Joe > >> >> -Alan >> From ivan.gerasimov at oracle.com Mon Jun 25 23:14:08 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 25 Jun 2018 16:14:08 -0700 Subject: RFR(s): 8203670: unmodifiable List iterator() implementations should not be ListIterators In-Reply-To: References: Message-ID: Hi Stuart! Out of curiosity. What if someone does something like if (it instanceof ListIterator) { // optimized for bidirectional access } else { // slower algorithm } previous(), nextIndex() and previousIndex() methods are not declared to be optional, so is it appropriate to throw UnsupportedOperationException from them? Someone may assume that if an object can be cast to ListIterator interface, non-optional methods should not throw UOE. With kind regards, Ivan On 6/25/18 3:06 PM, Stuart Marks wrote: > Hi all, > > Please review this small changeset that ensures that calling > iterator() on an unmodifiable List (from List.of, et. al.) returns an > instance that cannot be downcast to ListIterator and used as such. > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8203670 > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8203670/webrev.0/ > > Thanks, > > s'marks > -- With kind regards, Ivan Gerasimov From stuart.marks at oracle.com Mon Jun 25 23:44:23 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 25 Jun 2018 16:44:23 -0700 Subject: RFR(xs): JDK-8201610: fix broken link in UnicastRemoteObject Message-ID: <6b4cf35a-4295-d0d0-d512-259f21b84b57@oracle.com> Hi all, Please review this small fix for a broken link. Patch appended below. Bug: https://bugs.openjdk.java.net/browse/JDK-8201610 Thanks, s'marks diff -r 8b98dcf37891 -r 3da3bcc7b28b src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java --- a/src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java Mon Jun 25 16:21:14 2018 -0700 +++ b/src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java Mon Jun 25 16:43:57 2018 -0700 @@ -126,7 +126,7 @@ *
    * *
  • The proxy's class is defined according to the specifications for the - * + * * {@code Proxy} * * class, using the class loader of the remote object's class. From paul.sandoz at oracle.com Mon Jun 25 23:48:50 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 25 Jun 2018 16:48:50 -0700 Subject: RFR(xs): JDK-8201610: fix broken link in UnicastRemoteObject In-Reply-To: <6b4cf35a-4295-d0d0-d512-259f21b84b57@oracle.com> References: <6b4cf35a-4295-d0d0-d512-259f21b84b57@oracle.com> Message-ID: +1 Paul. > On Jun 25, 2018, at 4:44 PM, Stuart Marks wrote: > > Hi all, > > Please review this small fix for a broken link. > > Patch appended below. > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8201610 > > Thanks, > > s'marks > > > > diff -r 8b98dcf37891 -r 3da3bcc7b28b src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java > --- a/src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java Mon Jun 25 16:21:14 2018 -0700 > +++ b/src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java Mon Jun 25 16:43:57 2018 -0700 > @@ -126,7 +126,7 @@ > *
      > * > *
    • The proxy's class is defined according to the specifications for the > - * > + * > * {@code Proxy} > * > * class, using the class loader of the remote object's class. From lance.andersen at oracle.com Mon Jun 25 23:52:34 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 25 Jun 2018 19:52:34 -0400 Subject: RFR(xs): JDK-8201610: fix broken link in UnicastRemoteObject In-Reply-To: <6b4cf35a-4295-d0d0-d512-259f21b84b57@oracle.com> References: <6b4cf35a-4295-d0d0-d512-259f21b84b57@oracle.com> Message-ID: <87BC5662-5668-4CFA-B992-ABB6DFDE09B3@oracle.com> +1 Had one of those myself recently :-) > On Jun 25, 2018, at 7:44 PM, Stuart Marks wrote: > > Hi all, > > Please review this small fix for a broken link. > > Patch appended below. > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8201610 > > Thanks, > > s'marks > > > > diff -r 8b98dcf37891 -r 3da3bcc7b28b src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java > --- a/src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java Mon Jun 25 16:21:14 2018 -0700 > +++ b/src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java Mon Jun 25 16:43:57 2018 -0700 > @@ -126,7 +126,7 @@ > *
        > * > *
      • The proxy's class is defined according to the specifications for the > - * > + * > * {@code Proxy} > * > * class, using the class loader of the remote object's class. Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From stuart.marks at oracle.com Tue Jun 26 01:16:34 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 25 Jun 2018 18:16:34 -0700 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: <78d82d64-1fea-d5d6-c2f9-339d1459250a@oracle.com> Hi Peter, Sorry for the delay again. I got pulled into a P1 escalation. I don't think I'll have time to look at this in any detail until after JDK 11 ramp-down. s'marks On 6/22/18 9:51 AM, Peter Levart wrote: > Hi Stuart, > > Do you still fear that "single-threaded-object" idiom is not safe? > > I would just like to comment on the last approach... > > On 06/19/2018 07:01 PM, Peter Levart wrote: >> Here's another attempt that uses the same technique but relaxes the possible >> concurrent source map scenario (where number of pairs emitted by >> Map.forEach() doesn't agree with Map.size()): >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.03/ >> >> The construction is moved entirely into a separate MapBuilder object. The >> builder is used for Map.of(...) methods too. >> >> Here are the benchmark results that show improvements in every respect >> touched by the patch: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench_results.txt >> >> >> Here are the JMH benchmarks used to produce these results: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapCollectorBench.java >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapCopyOfBench.java >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapOfBench.java >> >> >> >> So what do you think of this one? >> >> Regards, Peter > > When this is used to copy concurrently changing concurrent Map, it has > approximately the same performance consequence as when using > Map.entrySet().toArray(). Either logic uses the estimated size to pre-size the > array, but when the size is a moving target, it fails, so it has to do another > copy at the end. My approach uses the estimated size to collect elements into > the ready-made hash-table. When it fails, it has to do another copy, but in > general case when the estimate is correct, it doesn't need an intermediary > representation in the form of array, so it can be more optimal. > > Regards, Peter > From martinrb at google.com Tue Jun 26 01:22:01 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 25 Jun 2018 18:22:01 -0700 Subject: RFR: 8205184: Delegating Iterator implementations that don't delegate forEachRemaining() In-Reply-To: References: <58D74C95-D0E5-4E08-8098-92D8ED2D4A53@oracle.com> Message-ID: Committed. From huizhe.wang at oracle.com Tue Jun 26 04:50:13 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 25 Jun 2018 21:50:13 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B316FB6.7020308@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> Message-ID: <5B31C605.5070001@oracle.com> Hi Alan, Sherman, Here's a version where we, as Sherman suggested, throw an IAE with CCE as the cause. This approach reduces code duplication in SC, although it complicates the impl a little bit with the added parameter and the different behavior between the existing usages of the methods and the new ones. The existing code paths are kept intact so there's no compatibility issue for the existing code. This version also did not remove the try-catch in Files as Alan suggested earlier. http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ Thanks, Joe On 6/25/18, 3:41 PM, Joe Wang wrote: > Hi Alan, > > The test testMalformedRead and testMalformedWrite now expect an > UnmappableCharacterException instead of IOE: > > webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev01/ > > Thanks, > Joe > > On 6/25/18, 9:48 AM, Joe Wang wrote: >> >> >> On 6/24/18, 12:11 PM, Alan Bateman wrote: >>> On 20/06/2018 04:32, Joe Wang wrote: >>>> Thanks Alan. I created 8205058 to capture your suggestions. Please >>>> see below for more details. >>>> >>>> Changed the internal APIs to throw CCE instead. In the same way as >>>> the previous changeset for 8201276, these methods are made specific >>>> for the use cases (though they are now for Files.read/writeString >>>> only) so that they are not mixed up with existing ones that may >>>> inadvertently affect other usages. >>>> >>>> One thing to note is that MalformedInputException or >>>> UnmappableCharacterException will lose one piece of information in >>>> comparison to the existing IAE, that is where it happens (offset). >>>> Should there be an improvement in the future, we could consider add >>>> it back to this part of code. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ >>> >>> I see your point on MalformedInputException and >>> UnmappableCharacterException so maybe we can submit a new issue to >>> track the follow-on work. >> >> Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 >> >>> >>> The update means a lot of duplication in StringCoding. Did you >>> consider (or measure) having the private encode/decode methods take >>> a parameter to indicate the exception handling? Sherman might have >>> suggestions on this. >> >> I did. I didn't like the code duplication at all. But it would be >> even messier if we reuse the code since the previous usage didn't >> declare checked exception, but did capture the position (offset) and >> length information, which means the new method while unnecessary to >> capture these information for Files read/writeString would still need >> to save them in case Exception occurs. Because of the complication, I >> thought you and Sherman would again prefer a specific methods than >> adding parameters (need fields as well). >> >> Furthermore, in the first round of the review, Sherman mentioned that >> he's been working on an improvement in this area. But I'll wait till >> Sherman could comment on this. >> >>> >>> In the tests, shouldn't testMalformedRead and testMalformedWrite be >>> updated so that expectedExceptions lists the specify exception that >>> is expected? If I read the update correctly then isn't checking for >>> MalformedInputException and UnmappableCharacterException anywhere >>> (it passes if IOException is thrown). >> >> MalformedInputException and UnmappableCharacterException are >> implementation details, the tests only verified what the spec >> required (IOE). >> >> -Joe >> >>> >>> -Alan >>> From smita.kamath at intel.com Tue Jun 26 05:25:25 2018 From: smita.kamath at intel.com (Kamath, Smita) Date: Tue, 26 Jun 2018 05:25:25 +0000 Subject: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions In-Reply-To: <980cfd3b-a6c1-eeda-3add-3165feea4fb9@oracle.com> References: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> <6563F381B547594081EF9DE181D0791277AAD2D4@FMSMSX119.amr.corp.intel.com> <980cfd3b-a6c1-eeda-3add-3165feea4fb9@oracle.com> Message-ID: <6563F381B547594081EF9DE181D0791277AAF5C2@FMSMSX119.amr.corp.intel.com> Hi Vladimir, Please find the updated webrev link. Webrev Link: http://cr.openjdk.java.net/~srukmannagar/Base64/webrev.00/ Bug link: https://bugs.openjdk.java.net/browse/JDK-8205528 Please let me know if you have additional questions. Thanks and Regards, Smita -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Monday, June 25, 2018 10:48 AM To: Kamath, Smita Cc: hotspot compiler ; core-libs-dev at openjdk.java.net; Paul Sandoz Subject: Re: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions I forgot to reply to your answers. On 6/22/18 2:49 PM, Kamath, Smita wrote: > Hi Vladimir, > > Please see my answers to your questions as below: > > 1) One question so: why you have own copy of base64 charsets and not using one in library: > I am using vpgatherdd instruction to fetch multiple values from base64 array in a single instruction with a vector index. Vpgatherdd instruction works on 32 bit array and so I need to define base64 charset in a 32 bit array. I have given reference to gather instruction below. As was discussed in an other e-mail lets keep your copy. > > 2) Some indents in new and old Assembler::emit_operand() are off. In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. > I'll make the necessary changes and send an updated webrev. > > 3) This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. > I am using vpgatherdd instruction which requires index vector with scale. It uses VSIB addressing where the index register is a zmm register. > Please refer to reference manual, volume 2c, page 2211: > https://software.intel.com/sites/default/files/managed/39/c5/325462-sd > m-vol-1-2abcd-3abcd.pdf Also see, section 2.3.12, page 524 for VSIB > memory addressing information. Got it. Thanks for document reference. > > 4) Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. > I will add test cases as per your suggestion. Thanks, Vladimir > > Please let me know if you have additional questions. > > Thanks, > Smita > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Friday, June 22, 2018 12:29 PM > To: Kamath, Smita > Cc: hotspot compiler > Subject: Re: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 > Instructions > > Hi Smita, > > I CCing to Libs to review code changes in Base64.java. > > Looks like you changed all need place to implement intrinsic. > One question so: why you have own copy of base64 charsets and not using one in library: > > private int encode0(byte[] src, int off, int end, byte[] dst) { > char[] base64 = isURL ? toBase64URL : toBase64; > > Some indents in new and old Assembler::emit_operand() are off. > > In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. > > This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. > > What testing you did? > > Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. > > I know there is test/jdk/java/util/Base64/ tests but they may not trigger intrinsic usage. But you can use them as starting point for new tests. > > Thanks, > Vladimir > > On 6/22/18 11:47 AM, Kamath, Smita wrote: >> Hi Vladimir, >> >> I'd like to contribute an optimization for Base64 Encoding Algorithm >> using AVX512 Instructions. This optimization shows 1.5x improvement >> on >> x86_64 platform(SKL). >> >> Link to Bug: https://bugs.openjdk.java.net/browse/JDK-8205528 >> >> Link to webrev: >> http://cr.openjdk.java.net/~vdeshpande/Base64/webrev.00/ >> >> For testing the implementation, I have run tests under >> test/jdk/java/util/Base64/ and also ran >> test/jdk/com/sun/jndi/ldap/Base64Test.java >> >> I have also run jtreg. >> >> Please review and let me know if you have any comments. >> >> Thanks and Regards, >> >> Smita >> From xueming.shen at oracle.com Tue Jun 26 05:28:16 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 25 Jun 2018 22:28:16 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B31C605.5070001@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> Message-ID: <5B31CEF0.2070504@oracle.com> Hi Joe, I would suggest always embed a malformed exception into the iae as showed below, then the extra flag is no longer needed. private static void throwMalformed(int off, int nb) { String msg = "malformed input off : " + off + ", length : " + nb; throw new IllegalArgumentException(msg, new MalformedInputException(nb)); } It's an implementation details, so we might be able to get rid of the iae later if we can figure out the better way to pass the malformed exception for both ZipCoder/Files later, without touch the api. An alternative is to move the iae-catch into StringCoding, with pair of similar methods to throw IOE (to add into the JLA), not sure if it's better though. It makes the Files impl cleaner at least. -Sherman On 6/25/18, 9:50 PM, Joe Wang wrote: > Hi Alan, Sherman, > > Here's a version where we, as Sherman suggested, throw an IAE with CCE > as the cause. This approach reduces code duplication in SC, although > it complicates the impl a little bit with the added parameter and the > different behavior between the existing usages of the methods and the > new ones. The existing code paths are kept intact so there's no > compatibility issue for the existing code. > > This version also did not remove the try-catch in Files as Alan > suggested earlier. > > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ > > Thanks, > Joe > > On 6/25/18, 3:41 PM, Joe Wang wrote: >> Hi Alan, >> >> The test testMalformedRead and testMalformedWrite now expect an >> UnmappableCharacterException instead of IOE: >> >> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev01/ >> >> Thanks, >> Joe >> >> On 6/25/18, 9:48 AM, Joe Wang wrote: >>> >>> >>> On 6/24/18, 12:11 PM, Alan Bateman wrote: >>>> On 20/06/2018 04:32, Joe Wang wrote: >>>>> Thanks Alan. I created 8205058 to capture your suggestions. >>>>> Please see below for more details. >>>>> >>>>> Changed the internal APIs to throw CCE instead. In the same way as >>>>> the previous changeset for 8201276, these methods are made >>>>> specific for the use cases (though they are now for >>>>> Files.read/writeString only) so that they are not mixed up with >>>>> existing ones that may inadvertently affect other usages. >>>>> >>>>> One thing to note is that MalformedInputException or >>>>> UnmappableCharacterException will lose one piece of information in >>>>> comparison to the existing IAE, that is where it happens (offset). >>>>> Should there be an improvement in the future, we could consider >>>>> add it back to this part of code. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >>>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ >>>> >>>> I see your point on MalformedInputException and >>>> UnmappableCharacterException so maybe we can submit a new issue to >>>> track the follow-on work. >>> >>> Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 >>> >>>> >>>> The update means a lot of duplication in StringCoding. Did you >>>> consider (or measure) having the private encode/decode methods take >>>> a parameter to indicate the exception handling? Sherman might have >>>> suggestions on this. >>> >>> I did. I didn't like the code duplication at all. But it would be >>> even messier if we reuse the code since the previous usage didn't >>> declare checked exception, but did capture the position (offset) and >>> length information, which means the new method while unnecessary to >>> capture these information for Files read/writeString would still >>> need to save them in case Exception occurs. Because of the >>> complication, I thought you and Sherman would again prefer a >>> specific methods than adding parameters (need fields as well). >>> >>> Furthermore, in the first round of the review, Sherman mentioned >>> that he's been working on an improvement in this area. But I'll wait >>> till Sherman could comment on this. >>> >>>> >>>> In the tests, shouldn't testMalformedRead and testMalformedWrite be >>>> updated so that expectedExceptions lists the specify exception that >>>> is expected? If I read the update correctly then isn't checking for >>>> MalformedInputException and UnmappableCharacterException anywhere >>>> (it passes if IOException is thrown). >>> >>> MalformedInputException and UnmappableCharacterException are >>> implementation details, the tests only verified what the spec >>> required (IOE). >>> >>> -Joe >>> >>>> >>>> -Alan >>>> From orionllmain at gmail.com Tue Jun 26 06:11:36 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Tue, 26 Jun 2018 13:11:36 +0700 Subject: RFR(s): 8203670: unmodifiable List iterator() implementations should not be ListIterators In-Reply-To: References: Message-ID: I agree with Ivan. Isn't it better to create a standalone implementation of Iterator and return its instance in iterator()? Implementing of Iterator for ImmutableList doesn't look like a big problem. 2018-06-26 6:14 GMT+07:00 Ivan Gerasimov : > Hi Stuart! > > Out of curiosity. What if someone does something like > > if (it instanceof ListIterator) { > // optimized for bidirectional access > } else { > // slower algorithm > } > > previous(), nextIndex() and previousIndex() methods are not declared to be > optional, so is it appropriate to throw UnsupportedOperationException from > them? > > Someone may assume that if an object can be cast to ListIterator > interface, non-optional methods should not throw UOE. > > With kind regards, > > Ivan > > > > On 6/25/18 3:06 PM, Stuart Marks wrote: > >> Hi all, >> >> Please review this small changeset that ensures that calling iterator() >> on an unmodifiable List (from List.of, et. al.) returns an instance that >> cannot be downcast to ListIterator and used as such. >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8203670 >> >> Webrev: >> >> http://cr.openjdk.java.net/~smarks/reviews/8203670/webrev.0/ >> >> Thanks, >> >> s'marks >> >> > -- > With kind regards, > Ivan Gerasimov > > From peter.levart at gmail.com Tue Jun 26 13:50:35 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 26 Jun 2018 15:50:35 +0200 Subject: RFR(s): 8203670: unmodifiable List iterator() implementations should not be ListIterators In-Reply-To: References: Message-ID: <1a3513db-ed34-b2c1-b036-064ef6d934bd@gmail.com> On 06/26/2018 08:11 AM, Zheka Kozlov wrote: > I agree with Ivan. Isn't it better to create a standalone implementation of > Iterator and return its instance in iterator()? Implementing of Iterator > for ImmutableList doesn't look like a big problem. Also, performance would be better if it was a separate class. It could be a superclass of ListItr so that common logic is not duplicated. @Stable annotation on private final boolean isListIterator does not help here unless the returned Iterator object is assigned to a static final field and used from it. Hardly a common use case for Iterator(s) I guess. Another thing about @Stable is that it never has effect on fields that hold a default value. Ok, when isListIterator == false, we don't need speed as we're throwing UOE anyway. The same arguments hold for other two @Stable fields in this class. I don't think those three @Stable annotations have any effect on real Iterator use cases. Regards, Peter > > 2018-06-26 6:14 GMT+07:00 Ivan Gerasimov : > >> Hi Stuart! >> >> Out of curiosity. What if someone does something like >> >> if (it instanceof ListIterator) { >> // optimized for bidirectional access >> } else { >> // slower algorithm >> } >> >> previous(), nextIndex() and previousIndex() methods are not declared to be >> optional, so is it appropriate to throw UnsupportedOperationException from >> them? >> >> Someone may assume that if an object can be cast to ListIterator >> interface, non-optional methods should not throw UOE. >> >> With kind regards, >> >> Ivan >> >> >> >> On 6/25/18 3:06 PM, Stuart Marks wrote: >> >>> Hi all, >>> >>> Please review this small changeset that ensures that calling iterator() >>> on an unmodifiable List (from List.of, et. al.) returns an instance that >>> cannot be downcast to ListIterator and used as such. >>> >>> Bug: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203670 >>> >>> Webrev: >>> >>> http://cr.openjdk.java.net/~smarks/reviews/8203670/webrev.0/ >>> >>> Thanks, >>> >>> s'marks >>> >>> >> -- >> With kind regards, >> Ivan Gerasimov >> >> From Alan.Bateman at oracle.com Tue Jun 26 13:54:50 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jun 2018 14:54:50 +0100 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B31C605.5070001@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> Message-ID: On 26/06/2018 05:50, Joe Wang wrote: > Hi Alan, Sherman, > > Here's a version where we, as Sherman suggested, throw an IAE with CCE > as the cause. This approach reduces code duplication in SC, although > it complicates the impl a little bit with the added parameter and the > different behavior between the existing usages of the methods and the > new ones. The existing code paths are kept intact so there's no > compatibility issue for the existing code. > > This version also did not remove the try-catch in Files as Alan > suggested earlier. > > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ This version looks much better. In StringCoding, do you really need throwCCE? The encode/decode methods do a replace or throw so I assume one flag will do. If combined with Sherman suggestion then it would be minimal changes to StringCoding. It would be nice to get rid of the IAE completely but that is for another day. In Files then you don't need to check if cause is null before testing its type. The update tests to check for UnmappedCharacterException and MalformedInputException look good. -Alan From ivan.gerasimov at oracle.com Tue Jun 26 16:57:19 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 26 Jun 2018 09:57:19 -0700 Subject: RFR(s): 8203670: unmodifiable List iterator() implementations should not be ListIterators In-Reply-To: <1a3513db-ed34-b2c1-b036-064ef6d934bd@gmail.com> References: <1a3513db-ed34-b2c1-b036-064ef6d934bd@gmail.com> Message-ID: On the first place, I'd like to understand why it is bad, if List.of().iterator() in fact returns ListIterator? What kind of problems it may cause? With kind regards, Ivan On 6/26/18 6:50 AM, Peter Levart wrote: > > > On 06/26/2018 08:11 AM, Zheka Kozlov wrote: >> I agree with Ivan. Isn't it better to create a standalone >> implementation of >> Iterator and return its instance in iterator()? Implementing of Iterator >> for ImmutableList doesn't look like a big problem. > > Also, performance would be better if it was a separate class. It could > be a superclass of ListItr so that common logic is not duplicated. > > @Stable annotation on private final boolean isListIterator does not > help here unless the returned Iterator object is assigned to a static > final field and used from it. Hardly a common use case for Iterator(s) > I guess. > > Another thing about @Stable is that it never has effect on fields that > hold a default value. Ok, when isListIterator == false, we don't need > speed as we're throwing UOE anyway. > > The same arguments hold for other two @Stable fields in this class. I > don't think those three @Stable annotations have any effect on real > Iterator use cases. > > Regards, Peter > >> >> 2018-06-26 6:14 GMT+07:00 Ivan Gerasimov : >> >>> Hi Stuart! >>> >>> Out of curiosity. What if someone does something like >>> >>> if (it instanceof ListIterator) { >>> // optimized for bidirectional access >>> } else { >>> // slower algorithm >>> } >>> >>> previous(), nextIndex() and previousIndex() methods are not declared >>> to be >>> optional, so is it appropriate to throw >>> UnsupportedOperationException from >>> them? >>> >>> Someone may assume that if an object can be cast to ListIterator >>> interface, non-optional methods should not throw UOE. >>> >>> With kind regards, >>> >>> Ivan >>> >>> >>> >>> On 6/25/18 3:06 PM, Stuart Marks wrote: >>> >>>> Hi all, >>>> >>>> Please review this small changeset that ensures that calling >>>> iterator() >>>> on an unmodifiable List (from List.of, et. al.) returns an instance >>>> that >>>> cannot be downcast to ListIterator and used as such. >>>> >>>> Bug: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203670 >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~smarks/reviews/8203670/webrev.0/ >>>> >>>> Thanks, >>>> >>>> s'marks >>>> >>>> >>> -- >>> With kind regards, >>> Ivan Gerasimov >>> >>> > > -- With kind regards, Ivan Gerasimov From huizhe.wang at oracle.com Tue Jun 26 17:39:58 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 26 Jun 2018 10:39:58 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B31CEF0.2070504@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B31CEF0.2070504@oracle.com> Message-ID: <5B327A6E.4000107@oracle.com> Hi Sherman, Combining the msg and cause is a great idea! That allowed us to satisfy the existing code without changes/additional parameter and also the new method for a CCE. I also moved the iae-catch into StringCoding, that makes Files clean of such things. Here's an updated webrev with both suggestions: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ Thanks, Joe On 6/25/18, 10:28 PM, Xueming Shen wrote: > Hi Joe, > > I would suggest always embed a malformed exception into the iae as showed > below, then the extra flag is no longer needed. > > private static void throwMalformed(int off, int nb) { > String msg = "malformed input off : " + off + ", length : " > + nb; > throw new IllegalArgumentException(msg, new > MalformedInputException(nb)); > } > > It's an implementation details, so we might be able to get rid of the > iae later if we > can figure out the better way to pass the malformed exception for both > ZipCoder/Files > later, without touch the api. > > An alternative is to move the iae-catch into StringCoding, with pair > of similar methods > to throw IOE (to add into the JLA), not sure if it's better though. It > makes the Files > impl cleaner at least. > > -Sherman > > On 6/25/18, 9:50 PM, Joe Wang wrote: >> Hi Alan, Sherman, >> >> Here's a version where we, as Sherman suggested, throw an IAE with >> CCE as the cause. This approach reduces code duplication in SC, >> although it complicates the impl a little bit with the added >> parameter and the different behavior between the existing usages of >> the methods and the new ones. The existing code paths are kept intact >> so there's no compatibility issue for the existing code. >> >> This version also did not remove the try-catch in Files as Alan >> suggested earlier. >> >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ >> >> Thanks, >> Joe >> >> On 6/25/18, 3:41 PM, Joe Wang wrote: >>> Hi Alan, >>> >>> The test testMalformedRead and testMalformedWrite now expect an >>> UnmappableCharacterException instead of IOE: >>> >>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev01/ >>> >>> Thanks, >>> Joe >>> >>> On 6/25/18, 9:48 AM, Joe Wang wrote: >>>> >>>> >>>> On 6/24/18, 12:11 PM, Alan Bateman wrote: >>>>> On 20/06/2018 04:32, Joe Wang wrote: >>>>>> Thanks Alan. I created 8205058 to capture your suggestions. >>>>>> Please see below for more details. >>>>>> >>>>>> Changed the internal APIs to throw CCE instead. In the same way >>>>>> as the previous changeset for 8201276, these methods are made >>>>>> specific for the use cases (though they are now for >>>>>> Files.read/writeString only) so that they are not mixed up with >>>>>> existing ones that may inadvertently affect other usages. >>>>>> >>>>>> One thing to note is that MalformedInputException or >>>>>> UnmappableCharacterException will lose one piece of information >>>>>> in comparison to the existing IAE, that is where it happens >>>>>> (offset). Should there be an improvement in the future, we could >>>>>> consider add it back to this part of code. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >>>>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ >>>>> >>>>> I see your point on MalformedInputException and >>>>> UnmappableCharacterException so maybe we can submit a new issue to >>>>> track the follow-on work. >>>> >>>> Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 >>>> >>>>> >>>>> The update means a lot of duplication in StringCoding. Did you >>>>> consider (or measure) having the private encode/decode methods >>>>> take a parameter to indicate the exception handling? Sherman might >>>>> have suggestions on this. >>>> >>>> I did. I didn't like the code duplication at all. But it would be >>>> even messier if we reuse the code since the previous usage didn't >>>> declare checked exception, but did capture the position (offset) >>>> and length information, which means the new method while >>>> unnecessary to capture these information for Files read/writeString >>>> would still need to save them in case Exception occurs. Because of >>>> the complication, I thought you and Sherman would again prefer a >>>> specific methods than adding parameters (need fields as well). >>>> >>>> Furthermore, in the first round of the review, Sherman mentioned >>>> that he's been working on an improvement in this area. But I'll >>>> wait till Sherman could comment on this. >>>> >>>>> >>>>> In the tests, shouldn't testMalformedRead and testMalformedWrite >>>>> be updated so that expectedExceptions lists the specify exception >>>>> that is expected? If I read the update correctly then isn't >>>>> checking for MalformedInputException and >>>>> UnmappableCharacterException anywhere (it passes if IOException is >>>>> thrown). >>>> >>>> MalformedInputException and UnmappableCharacterException are >>>> implementation details, the tests only verified what the spec >>>> required (IOE). >>>> >>>> -Joe >>>> >>>>> >>>>> -Alan >>>>> > From huizhe.wang at oracle.com Tue Jun 26 17:41:35 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 26 Jun 2018 10:41:35 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> Message-ID: <5B327ACF.3080804@oracle.com> On 6/26/18, 6:54 AM, Alan Bateman wrote: > On 26/06/2018 05:50, Joe Wang wrote: >> Hi Alan, Sherman, >> >> Here's a version where we, as Sherman suggested, throw an IAE with >> CCE as the cause. This approach reduces code duplication in SC, >> although it complicates the impl a little bit with the added >> parameter and the different behavior between the existing usages of >> the methods and the new ones. The existing code paths are kept intact >> so there's no compatibility issue for the existing code. >> >> This version also did not remove the try-catch in Files as Alan >> suggested earlier. >> >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ > This version looks much better. In StringCoding, do you really need > throwCCE? The encode/decode methods do a replace or throw so I assume > one flag will do. If combined with Sherman suggestion then it would be > minimal changes to StringCoding. It would be nice to get rid of the > IAE completely but that is for another day. In Files then you don't > need to check if cause is null before testing its type. Yes, combined with Sherman's suggestion eliminated the need for the new parameter. Here's the updated webrev: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ > > The update tests to check for UnmappedCharacterException and > MalformedInputException look good. Thanks, Joe > > -Alan > > From stuart.marks at oracle.com Tue Jun 26 17:51:10 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 26 Jun 2018 10:51:10 -0700 Subject: RFR(s): 8203670: unmodifiable List iterator() implementations should not be ListIterators In-Reply-To: References: <1a3513db-ed34-b2c1-b036-064ef6d934bd@gmail.com> Message-ID: <7089b745-ebf8-36d3-96fe-02a9536ebf8e@oracle.com> On 6/26/18 9:57 AM, Ivan Gerasimov wrote: > On the first place, I'd like to understand why it is bad, if > List.of().iterator() in fact returns ListIterator? > What kind of problems it may cause? Basically it leaks information. Somebody might get an iterator, partially advance it, and hand it off to another piece of code, assuming that it can only be advanced. But if that instance can be downcast and used as a ListIterator, that assumption would be violated. I admit it's a narrow case, but it's nonetheless a possibility. > Out of curiosity. What if someone does something like > > if (it instanceof ListIterator) { > // optimized for bidirectional access > } else { > // slower algorithm > } > > previous(), nextIndex() and previousIndex() methods are not declared to be optional, so is it appropriate to throw UnsupportedOperationException from them? > > Someone may assume that if an object can be cast to ListIterator interface, non-optional methods should not throw UOE. I think this usage is out of bounds for the List API. The List specification is: Iterator iterator(); ListIterator listIterator(); ListIterator listIterator(int); That is, you get an Iterator from the iterator() method, and you get a ListIterator from the listIterator() methods. This implementation fulfils all the contracts for the interfaces declared as the return types. In particular, if you ask for a ListIterator, you get a ListIterator implementation that implements all the methods. However, if you call iterator() and get an Iterator, and do an 'instanceof' and downcast, then all bets are off. It's reasonable to rely on the thing returned by iterator() having any behaviors other than those specified by Iterator. Peter Levart wrote: > Also, performance would be better if it was a separate class. It could be a superclass of ListItr so that common logic is not duplicated. This isn't at all obvious. Certainly it's not obvious enough to embark on a refactoring without doing some benchmarking. However, I want to fix this bug in JDK 11. If somebody wants to do some benchmarking, they're welcome to do so, but I don't want to consider this as part of this changeset. > @Stable annotation on private final boolean isListIterator does not help here unless the returned Iterator object is assigned to a static final field and used from it. Hardly a common use case for Iterator(s) I guess. The exact effect of @Stable is Hotspot-specific and changes over time. Here we're using it as a declaration that this final field really can be treated as final, and that it won't be modified via reflection or a varhandle, thus enabling VM more aggressive optimizations. s'marks From claes.redestad at oracle.com Tue Jun 26 18:04:31 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 26 Jun 2018 20:04:31 +0200 Subject: RFR(s): 8203670: unmodifiable List iterator() implementations should not be ListIterators In-Reply-To: <7089b745-ebf8-36d3-96fe-02a9536ebf8e@oracle.com> References: <1a3513db-ed34-b2c1-b036-064ef6d934bd@gmail.com> <7089b745-ebf8-36d3-96fe-02a9536ebf8e@oracle.com> Message-ID: On 2018-06-26 19:51, Stuart Marks wrote: > Peter Levart wrote: > >> Also, performance would be better if it was a separate class. It >> could be a superclass of ListItr so that common logic is not duplicated. > > This isn't at all obvious. Certainly it's not obvious enough to embark > on a refactoring without doing some benchmarking. > > However, I want to fix this bug in JDK 11. If somebody wants to do > some benchmarking, they're welcome to do so, but I don't want to > consider this as part of this changeset. I don't have time to do any benchmarking here, sorry, but could counter-speculate that the tests are likely to be eliminated by the JIT anyway, and that keeping only one implementation class might allow some hypothetical callsite to stay mono- or bimorphic that would otherwise become bi- or megamorphic. So yes, it's not obvious that splitting will improve performance. > >> @Stable annotation on private final boolean isListIterator does not >> help here unless the returned Iterator object is assigned to a static >> final field and used from it. Hardly a common use case for >> Iterator(s) I guess. > > The exact effect of @Stable is Hotspot-specific and changes over time. > Here we're using it as a declaration that this final field really can > be treated as final, and that it won't be modified via reflection or a > varhandle, thus enabling VM more aggressive optimizations. +1, even though I suspect Peter is right for now. :-) The patch at hand looks good to me as-is. Thanks! /Claes From lance.andersen at oracle.com Tue Jun 26 18:24:27 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 26 Jun 2018 14:24:27 -0400 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B327ACF.3080804@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> Message-ID: Hi Joe, Should src/java.base/share/classes/jdk/internal/misc/JavaLangAccess.java ????? /** * Constructs a new {@code String} by decoding the specified subarray of * bytes using the specified {@linkplain java.nio.charset.Charset charset}. * * The caller of this method shall relinquish and transfer the ownership of * the byte array to the callee since the later will not make a copy. * * @param bytes the byte array source * @param cs the Charset * @return the newly created string * @throws IllegalArgumentException for malformed or unmappable bytes */ String newStringNoRepl(byte[] bytes, Charset cs) throws CharacterCodingException; ???????? include an @throws for CharacterCodingException? > On Jun 26, 2018, at 1:41 PM, Joe Wang wrote: > > > > On 6/26/18, 6:54 AM, Alan Bateman wrote: >> On 26/06/2018 05:50, Joe Wang wrote: >>> Hi Alan, Sherman, >>> >>> Here's a version where we, as Sherman suggested, throw an IAE with CCE as the cause. This approach reduces code duplication in SC, although it complicates the impl a little bit with the added parameter and the different behavior between the existing usages of the methods and the new ones. The existing code paths are kept intact so there's no compatibility issue for the existing code. >>> >>> This version also did not remove the try-catch in Files as Alan suggested earlier. >>> >>> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ >> This version looks much better. In StringCoding, do you really need throwCCE? The encode/decode methods do a replace or throw so I assume one flag will do. If combined with Sherman suggestion then it would be minimal changes to StringCoding. It would be nice to get rid of the IAE completely but that is for another day. In Files then you don't need to check if cause is null before testing its type. > > Yes, combined with Sherman's suggestion eliminated the need for the new parameter. Here's the updated webrev: > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ >> >> The update tests to check for UnmappedCharacterException and MalformedInputException look good. > > Thanks, > Joe > >> >> -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From vladimir.kozlov at oracle.com Tue Jun 26 18:48:39 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 26 Jun 2018 11:48:39 -0700 Subject: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions In-Reply-To: <6563F381B547594081EF9DE181D0791277AAF5C2@FMSMSX119.amr.corp.intel.com> References: <6563F381B547594081EF9DE181D0791277AAD1EC@FMSMSX119.amr.corp.intel.com> <6563F381B547594081EF9DE181D0791277AAD2D4@FMSMSX119.amr.corp.intel.com> <980cfd3b-a6c1-eeda-3add-3165feea4fb9@oracle.com> <6563F381B547594081EF9DE181D0791277AAF5C2@FMSMSX119.amr.corp.intel.com> Message-ID: Yes, this looks good to me. Let me test it. Thanks, Vladimir On 6/25/18 10:25 PM, Kamath, Smita wrote: > Hi Vladimir, > > Please find the updated webrev link. > > Webrev Link: http://cr.openjdk.java.net/~srukmannagar/Base64/webrev.00/ > Bug link: https://bugs.openjdk.java.net/browse/JDK-8205528 > > Please let me know if you have additional questions. > > Thanks and Regards, > Smita > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Monday, June 25, 2018 10:48 AM > To: Kamath, Smita > Cc: hotspot compiler ; core-libs-dev at openjdk.java.net; Paul Sandoz > Subject: Re: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 Instructions > > I forgot to reply to your answers. > > On 6/22/18 2:49 PM, Kamath, Smita wrote: >> Hi Vladimir, >> >> Please see my answers to your questions as below: >> >> 1) One question so: why you have own copy of base64 charsets and not using one in library: >> I am using vpgatherdd instruction to fetch multiple values from base64 array in a single instruction with a vector index. Vpgatherdd instruction works on 32 bit array and so I need to define base64 charset in a 32 bit array. I have given reference to gather instruction below. > > As was discussed in an other e-mail lets keep your copy. > >> >> 2) Some indents in new and old Assembler::emit_operand() are off. In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. >> I'll make the necessary changes and send an updated webrev. >> >> 3) This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. >> I am using vpgatherdd instruction which requires index vector with scale. It uses VSIB addressing where the index register is a zmm register. >> Please refer to reference manual, volume 2c, page 2211: >> https://software.intel.com/sites/default/files/managed/39/c5/325462-sd >> m-vol-1-2abcd-3abcd.pdf Also see, section 2.3.12, page 524 for VSIB >> memory addressing information. > > Got it. Thanks for document reference. > >> >> 4) Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. >> I will add test cases as per your suggestion. > > Thanks, > Vladimir > >> >> Please let me know if you have additional questions. >> >> Thanks, >> Smita >> >> -----Original Message----- >> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >> Sent: Friday, June 22, 2018 12:29 PM >> To: Kamath, Smita >> Cc: hotspot compiler >> Subject: Re: RFR(S) JDK-8205528: Base64 Encode Algorithm using AVX512 >> Instructions >> >> Hi Smita, >> >> I CCing to Libs to review code changes in Base64.java. >> >> Looks like you changed all need place to implement intrinsic. >> One question so: why you have own copy of base64 charsets and not using one in library: >> >> private int encode0(byte[] src, int off, int end, byte[] dst) { >> char[] base64 = isURL ? toBase64URL : toBase64; >> >> Some indents in new and old Assembler::emit_operand() are off. >> >> In new Assembler::emit_operand() is better use } else { instead of 'return' in one branch. >> >> This is first time I see that XMM register can be used for index in address. Is it true? Can you point to Intel's document which describes it. >> >> What testing you did? >> >> Please, add tests to test/hotspot/jtreg/compiler/intrinsics/base64 (may be similar to sha or AES in compiler/codegen/aes) to make sure that all flags, intrinsic is used and it produces correct result. >> >> I know there is test/jdk/java/util/Base64/ tests but they may not trigger intrinsic usage. But you can use them as starting point for new tests. >> >> Thanks, >> Vladimir >> >> On 6/22/18 11:47 AM, Kamath, Smita wrote: >>> Hi Vladimir, >>> >>> I'd like to contribute an optimization for Base64 Encoding Algorithm >>> using AVX512 Instructions. This optimization shows 1.5x improvement >>> on >>> x86_64 platform(SKL). >>> >>> Link to Bug: https://bugs.openjdk.java.net/browse/JDK-8205528 >>> >>> Link to webrev: >>> http://cr.openjdk.java.net/~vdeshpande/Base64/webrev.00/ >>> >>> For testing the implementation, I have run tests under >>> test/jdk/java/util/Base64/ and also ran >>> test/jdk/com/sun/jndi/ldap/Base64Test.java >>> >>> I have also run jtreg. >>> >>> Please review and let me know if you have any comments. >>> >>> Thanks and Regards, >>> >>> Smita >>> From Alan.Bateman at oracle.com Tue Jun 26 18:57:17 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jun 2018 19:57:17 +0100 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B327ACF.3080804@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> Message-ID: <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> On 26/06/2018 18:41, Joe Wang wrote: > > : > > Yes, combined with Sherman's suggestion eliminated the need for the > new parameter. Here's the updated webrev: > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ This looks good. Just a minor nit but newStringNoRepl and getBytesNoReply don't need to check if the cause is null. -Alan From huizhe.wang at oracle.com Tue Jun 26 19:14:57 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 26 Jun 2018 12:14:57 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> Message-ID: <5B3290B1.4010500@oracle.com> Thanks Lance! Indeed it has been changed to CCE. I'm still familiarizing myself with IntelliJ. Opened it in NetBeans, it clearly indicates "missing @throws tag for " CCE, but I don't see anything like that in IntelliJ. Updated webrev: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ -Joe On 6/26/18, 11:24 AM, Lance Andersen wrote: > Hi Joe, > > Should src/java.base/share/classes/jdk/internal/misc/JavaLangAccess.java > > ????? > /** > * Constructs a new {@code String} by decoding the specified subarray of > * bytes using the specified {@linkplain java.nio.charset.Charset charset}. > * > * The caller of this method shall relinquish and transfer the ownership of > * the byte array to the callee since the later will not make a copy. > * > * @param bytes the byte array source > * @param cs the Charset > * @return the newly created string > * @throws IllegalArgumentException for malformed or unmappable bytes > */ > String newStringNoRepl(byte[] bytes, Charset cs) throws CharacterCodingException; > > ???????? > include an @throws for CharacterCodingException? > > >> On Jun 26, 2018, at 1:41 PM, Joe Wang > > wrote: >> >> >> >> On 6/26/18, 6:54 AM, Alan Bateman wrote: >>> On 26/06/2018 05:50, Joe Wang wrote: >>>> Hi Alan, Sherman, >>>> >>>> Here's a version where we, as Sherman suggested, throw an IAE with >>>> CCE as the cause. This approach reduces code duplication in SC, >>>> although it complicates the impl a little bit with the added >>>> parameter and the different behavior between the existing usages of >>>> the methods and the new ones. The existing code paths are kept >>>> intact so there's no compatibility issue for the existing code. >>>> >>>> This version also did not remove the try-catch in Files as Alan >>>> suggested earlier. >>>> >>>> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ >>>> >>> This version looks much better. In StringCoding, do you really need >>> throwCCE? The encode/decode methods do a replace or throw so I >>> assume one flag will do. If combined with Sherman suggestion then it >>> would be minimal changes to StringCoding. It would be nice to get >>> rid of the IAE completely but that is for another day. In Files then >>> you don't need to check if cause is null before testing its type. >> >> Yes, combined with Sherman's suggestion eliminated the need for the >> new parameter. Here's the updated webrev: >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ >> >>> >>> The update tests to check for UnmappedCharacterException and >>> MalformedInputException look good. >> >> Thanks, >> Joe >> >>> >>> -Alan > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From huizhe.wang at oracle.com Tue Jun 26 19:49:13 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 26 Jun 2018 12:49:13 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> Message-ID: <5B3298B9.2020603@oracle.com> On 6/26/18, 11:57 AM, Alan Bateman wrote: > > > On 26/06/2018 18:41, Joe Wang wrote: >> >> : >> >> Yes, combined with Sherman's suggestion eliminated the need for the >> new parameter. Here's the updated webrev: >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ > This looks good. > > Just a minor nit but newStringNoRepl and getBytesNoReply don't need to > check if the cause is null. Removed the null check. The internal impl and test guarantees it's not null indeed: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev04/ Thanks, Joe > > -Alan From xueming.shen at oracle.com Tue Jun 26 19:55:22 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 26 Jun 2018 12:55:22 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B3298B9.2020603@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> <5B3298B9.2020603@oracle.com> Message-ID: <5B329A2A.1070305@oracle.com> +1 On 6/26/18, 12:49 PM, Joe Wang wrote: > > > On 6/26/18, 11:57 AM, Alan Bateman wrote: >> >> >> On 26/06/2018 18:41, Joe Wang wrote: >>> >>> : >>> >>> Yes, combined with Sherman's suggestion eliminated the need for the >>> new parameter. Here's the updated webrev: >>> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ >> This looks good. >> >> Just a minor nit but newStringNoRepl and getBytesNoReply don't need >> to check if the cause is null. > > Removed the null check. The internal impl and test guarantees it's not > null indeed: > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev04/ > > Thanks, > Joe > >> >> -Alan From naoto.sato at oracle.com Tue Jun 26 20:27:14 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 26 Jun 2018 13:27:14 -0700 Subject: Japanese New Era supporting does not accept Heisei 32 In-Reply-To: <33afc91d-8a8d-e3fc-0105-9ba4ecd901e8@oracle.com> References: <33afc91d-8a8d-e3fc-0105-9ba4ecd901e8@oracle.com> Message-ID: <26c97380-7825-d7df-b3d8-f656301ee906@oracle.com> Kishida-san, I further looked into this. Although JapaneseDate itself does not accept H32, DateTimeFormatter does support lenient mode, e.g., a formatter created with withResolverStyle(ResolverStyle.LENIENT) does accept dates like "Heisei 32." Date/time formatter in java.text that use java.util.Calendar instances accept lenient dates by default. HTH, Naoto On 6/14/18 4:00 PM, Naoto Sato wrote: > Hi, > > Yes, that's the expected behavior with the implementation assuming > "Heisei 32" is non-existent. I see some discussion as you mentioned, > whether to allow to use Heisei for a while, for transitional purpose, > but at the moment there's very little & no official information > available. We may consider adding an additional API to convert those > dates leniently. > > Naoto > > On 6/14/18 5:13 AM, kishida naoki wrote: >> I'm trying to use the Japanese new era supporting and have found that it >> does not accept Heisei 32. >> There are many document that include Heisei 32 or later at present. For >> example, the expiration date of my driver license is Heisei 32 March. >> JDK10 can accept Heisei 32. >> I think the change of the behavior make some trouble. >> >> Additionally,? Japanese government says some system use Heisei after >> 5/1/2019 for a while. The JDK version that include the new era support >> can >> not be used on such system. >> https://digital.asahi.com/articles/DA3S13491490.html >> >> There are 3 ways to support the new era. >> 1. H32 is not accepted and S90 is not accepted. (current new era support >> behavior) >> 2. H32 is accepted and S90 is not accepted. (current JDK behavior) >> 3. H32 is accepted and S90 is accepted. (to conformity) >> >> I prefer 2 not to change the current JDK behavior. >> >> With taking 2 or 3, we might have a new API like >> JapaneseDate.from(JapaneseEra, TemporalAccessor) to specify an era on >> making JapaneseDate. >> From ivan.gerasimov at oracle.com Tue Jun 26 20:49:24 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 26 Jun 2018 13:49:24 -0700 Subject: RFR(s): 8203670: unmodifiable List iterator() implementations should not be ListIterators In-Reply-To: <7089b745-ebf8-36d3-96fe-02a9536ebf8e@oracle.com> References: <1a3513db-ed34-b2c1-b036-064ef6d934bd@gmail.com> <7089b745-ebf8-36d3-96fe-02a9536ebf8e@oracle.com> Message-ID: <7a888ac6-8e14-d196-1f63-0266fbbfe5ea@oracle.com> Okay. Thanks for explaining! With kind regards, Ivan On 6/26/18 10:51 AM, Stuart Marks wrote: > On 6/26/18 9:57 AM, Ivan Gerasimov wrote: >> On the first place, I'd like to understand why it is bad, if >> List.of().iterator() in fact returns ListIterator? >> What kind of problems it may cause? > > Basically it leaks information. Somebody might get an iterator, > partially advance it, and hand it off to another piece of code, > assuming that it can only be advanced. But if that instance can be > downcast and used as a ListIterator, that assumption would be > violated. I admit it's a narrow case, but it's nonetheless a possibility. > > >> Out of curiosity. What if someone does something like >> >> if (it instanceof ListIterator) { >> // optimized for bidirectional access >> } else { >> // slower algorithm >> } >> >> previous(), nextIndex() and previousIndex() methods are not declared >> to be optional, so is it appropriate to throw >> UnsupportedOperationException from them? >> >> Someone may assume that if an object can be cast to ListIterator >> interface, non-optional methods should not throw UOE. > > I think this usage is out of bounds for the List API. The List > specification is: > > Iterator iterator(); > ListIterator listIterator(); > ListIterator listIterator(int); > > That is, you get an Iterator from the iterator() method, and you get a > ListIterator from the listIterator() methods. This implementation > fulfils all the contracts for the interfaces declared as the return > types. In particular, if you ask for a ListIterator, you get a > ListIterator implementation that implements all the methods. > > However, if you call iterator() and get an Iterator, and do an > 'instanceof' and downcast, then all bets are off. It's reasonable to > rely on the thing returned by iterator() having any behaviors other > than those specified by Iterator. > > > Peter Levart wrote: > >> Also, performance would be better if it was a separate class. It >> could be a superclass of ListItr so that common logic is not duplicated. > > This isn't at all obvious. Certainly it's not obvious enough to embark > on a refactoring without doing some benchmarking. > > However, I want to fix this bug in JDK 11. If somebody wants to do > some benchmarking, they're welcome to do so, but I don't want to > consider this as part of this changeset. > >> @Stable annotation on private final boolean isListIterator does not >> help here unless the returned Iterator object is assigned to a static >> final field and used from it. Hardly a common use case for >> Iterator(s) I guess. > > The exact effect of @Stable is Hotspot-specific and changes over time. > Here we're using it as a declaration that this final field really can > be treated as final, and that it won't be modified via reflection or a > varhandle, thus enabling VM more aggressive optimizations. > > s'marks > > -- With kind regards, Ivan Gerasimov From sean.coffey at oracle.com Tue Jun 26 22:18:45 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Tue, 26 Jun 2018 23:18:45 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> Message-ID: <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> Erik, I rebased the patch with TLS v1.3 integration today. I hadn't realized how much the handshaker code had changed. Hopefully, I'll get a review from security dev team on that front. Regards the JFR semantics, I believe the edits should match majority of requests . I still have a preference for the test infra design for these new logger/JFR tests used in version 1 of webrev. I think it makes sense to keep the test functionality together - no sense in separating them out into different components IMO. Also, some of the edits to the JFR testing were made to test for the new dual log/record functionality. I might catch up with you tomorrow to see what the best arrangement would be. http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ regards, Sean. On 25/06/2018 21:22, Se?n Coffey wrote: > Many thanks for the review comments Erik. Replies inline. > > > On 24/06/2018 14:21, Erik Gahlin wrote: >> Hi Sean, >> >> Some of the changes in the webrev belongs to JDK-8203629 and should >> be removed for clarity. >> >> Some initial comments: >> >> default.jfc, profile.jfr: >> The events should not have control="enable-exceptions". The purpose >> of the control attribute is so to provide parameterized configuration >> of events for JMC.? As it is now, the security events will be enabled >> when a user turns on the exception events. > Makes sense. I'll remove that parameter. >> >> X509CertEvent: >> You should use milliseconds since epoch to represent a date instead >> of a string value, i.e. >> >> ??? @Label("Valid From") >> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >> ??? public long validFrom; >> >> ??? @Label("Valid Until") >> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >> ??? public long validUntil; >> > The CertificateValidity class operates on Date Object values. I'll > work with the Date.getTime() method then (and update the Logger > formatting) >> This following annotation adds little value >> >> @Description("Details of Security Property") >> >> I would either remove the annotation, or provide information that >> helps a user understand the event. For instance, what is X509, and >> what kind of certificates are we talking about? > Yes - that looks like the wrong Description. I'll review all of these > fields. >> >> @Category("Java Application") >> >> I'm a bit worried that we will pollute the "Java Application" >> namespace if we add lots of JDK events in that category. We may want >> to add a new top level category "Java Development Kit", analogous to >> the "Java Virtual Machine" category, and put all security related >> events in subcategory, perhaps called "Security". > Yes - Open to suggestions. "Security" sounds like a good suggestion. >> >> @Label("X509Cert") >> >> The label should be human readable name, with spaces, title cased >> etc. I would recommend "X.509 Certificate". In general, avoid >> abbreviations like "certs" and instead use labels such as >> "Certificate Chain". The label should be short and not a full sentence. >> >> For details see, >> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >> >> I think it would be good to separate testing of JFR and logging into >> different files / tests. I would prefer that the test name is the >> same as the event prefixed with "Test", i.e TestX509CertificateEvent, >> as this is the pattern used by other JFR tests. >> > I'll take a look at that pattern again. I've separated out the current > tests into an (a) outer test to analyze the logger output and (b) the > inner test which checks for JFR correctness. I did include extra logic > to make sure that the EventHelper configuration was working correctly. > "Events.assertField" looks very helpful. Thanks for the pointer. > > Let me take on board the suggestions below and get a new webrev out on > Tuesday. > > regards, > Sean. > >> I reworked one of the tests to how I like to see it: >> >> /* >> ?* @test >> ?* @key jfr >> ?* @library /test/lib >> ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >> ?*/ >> public class TestX509CertificateEvent { >> >> ??? private static final String CERTIFICATE_1 = "..."; >> ??? private static final String CERTIFICATE_2 = "..."; >> >> ??? public static void main(String... args) throws >> CertificateException { >> >> ???????? Recording r = new Recording(); >> r.enable(EventNames.X509Certificate).withoutStackTrace(); >> ???????? r.start(); >> >> ???????? CertificateFactory cf = >> CertificateFactory.getInstance("X.509"); >> ???????? cf.generateCertificate(new >> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >> ???????? cf.generateCertificate(new >> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >> >> ???????? // Make sure only one event per certificate >> ???????? cf.generateCertificate(new >> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >> ???????? cf.generateCertificate(new >> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >> >> ???????? r.stop(); >> >> ???????? List events = Events.fromRecording(r); >> ???????? Asserts.assertEquals(events.size(), 2, "Expected two X.509 >> Certificate events"); >> >> ???????? assertEvent(events, "1000", "SHA256withRSA", >> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >> ???????????????????? "RSA", 2048); >> ???????? assertEvent(events, "1000", "SHA256withRSA", >> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >> ???????????????????? "RSA", 2048); >> ??? } >> >> ??? private static void assertEvent(List events, >> String certID, String algId, String subject, >> ??????????? String issuer, String keyType, int length) throws >> Exception { >> >> ??????? for (RecordedEvent e : events) { >> ??????????? if (e.getString("serialNumber").equals(certID)) { >> ??????????????? Events.assertField(e, "algId").equal(algId); >> ??????????????? ... >> ??????????????? return; >> ??????????? } >> ??????? } >> ??????? System.out.println(events); >> ??????? throw new Exception("Could not find event with Cert ID: " + >> certID); >> ??? } >> } >> >> The reworked example uses the Events.assertField method, which will >> give context if the assertion fails. Keeping the test simple, means >> it can be analyzed quickly if it fails in testing. There is no new >> test framework to learn, or methods to search for, and it is usually >> not hard to determine if the failure is product, test or >> infrastructure related, and what component (team) should be assigned >> the issue. The use of EventNames.X509Certificate means all >> occurrences of the event can be tracked done in an IDE using find by >> reference. >> >> Thanks >> Erik >> >>> Following on from the recent JDK-8203629 code review, I'd like to >>> propose enhancements on how we can record events in security libs. >>> The introduction of the JFR libraries can give us much better ways >>> of examining JDK actions. For the initial phase, I'm looking to >>> enhance some key security library events in JDK 11 so that they can >>> be either recorded to JFR, logged to a traditional logger, or both. >>> >>> Examples of how useful JFR recordings could be can be seen here : >>> >>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>> >>> securityProp_2.png gives an example of how the JFR recording can be >>> queried to quickly locate events of interest (in this case, code >>> setting the jdk.tls.* Security properties). I still need to clean up >>> the TLSEvents testcase to improve test coverage and hope to do that >>> in coming days. >>> >>> JBS record : >>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>> >>> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>> >> > From roger.riggs at oracle.com Wed Jun 27 02:10:39 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 26 Jun 2018 22:10:39 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <5270e866-014c-497b-60db-855338bd7d8b@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> <5270e866-014c-497b-60db-855338bd7d8b@oracle.com> Message-ID: <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> Hi, Updated webrev: http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/index.html Applied changes from prior comments and droped a change no longer needed due to the TLS 1.3 removal of ClientKeyExchangeService.java. The CSR has been approved without possibly confusing @implNote in System.getProperties about caching of specific properties, including java.home, etc. Thanks for any additional comments. Roger On 6/19/18 11:52 AM, Brent Christian wrote: > On 6/19/18 8:08 AM, Roger Riggs wrote: >>> >>> * src/java.base/share/classes/java/lang/System.java : >>> >>> Should the @implNote with the list of cached properties be added >>> everywhere the @apiNote is being added ?? Right now the @implNote is >>> only added to getProperties(). >>> >> The repetition was getting tiresome and the base of all the >> xxxProperties methods is getProperties. >> ??Joe suggested having one copy of the full information? and >> referring to that from the individual @apiNotes. > > Fair enough. > >>> * src/java.base/share/classes/jdk/internal/util/StaticProperty.java : >>> >>> ? 45???? private StaticProperty() { >>> ? 46 >>> ? 47???? } >>> >>> Maybe put this all on one line? >>> >> Will do > > Thanks, > -Brent > From xueming.shen at oracle.com Wed Jun 27 03:14:46 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 26 Jun 2018 20:14:46 -0700 Subject: RFR JDK-8200243: System error message is decoded as invalid encoding in Windows. Message-ID: <5B330126.2040708@oracle.com> Hi, Please help review the change for JDK-8200243 issue: https://bugs.openjdk.java.net/browse/JDK-8200243 webrev: http://cr.openjdk.java.net/~sherman/8200243/webrev This is a regression. The root cause is one of the change we put in jdk9 for JDK-8057777 [1], which is to remove those unused vm interfaces. The webrev [1] indicates that the changeset added a platform dependent version of GetLastErrorString(), which uses utf8, to replace the original jvm version of JVM_GetLastErrorString() (in jni_util.c). However the encoding of the char* interface between jvm and jdk is always interpreted/specified as "native encoding". So if the platform's default encoding is NOT utf-8, it never uses utf8. This is what we have on Windows platform. The fix is to use/copy-paste the "original" vm implementation in hotspot repo for JVM_GetLastErrorString() (from os_windows.cpp). The fix has been verified on a Chinese version of windows. Thanks Felix! Thanks, Sherman [1] https://bugs.openjdk.java.net/browse/JDK-8057777 From mandy.chung at oracle.com Wed Jun 27 05:54:41 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 26 Jun 2018 22:54:41 -0700 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> <5270e866-014c-497b-60db-855338bd7d8b@oracle.com> <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> Message-ID: <9d5359d7-2f93-4b81-7094-a20b782eb88e@oracle.com> Looks good to me. The part that I'm unsure is whether you caught all the callers that are outside doPrivileged to StaticProperty.* will need to do its property permission check. This requires API inspection and testing that I assume you covered that. Mandy On 6/26/18 7:10 PM, Roger Riggs wrote: > Hi, > > Updated webrev: > > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/index.html > > > Applied changes from prior comments and droped a change no longer needed > due > to the TLS 1.3 removal of ClientKeyExchangeService.java. > > The CSR has been approved without possibly confusing @implNote in > System.getProperties > about caching of specific properties, including java.home, etc. > > Thanks for any additional comments. > > Roger > > > > On 6/19/18 11:52 AM, Brent Christian wrote: >> On 6/19/18 8:08 AM, Roger Riggs wrote: >>>> >>>> * src/java.base/share/classes/java/lang/System.java : >>>> >>>> Should the @implNote with the list of cached properties be added >>>> everywhere the @apiNote is being added ?? Right now the @implNote is >>>> only added to getProperties(). >>>> >>> The repetition was getting tiresome and the base of all the >>> xxxProperties methods is getProperties. >>> ??Joe suggested having one copy of the full information? and >>> referring to that from the individual @apiNotes. >> >> Fair enough. >> >>>> * src/java.base/share/classes/jdk/internal/util/StaticProperty.java : >>>> >>>> ? 45???? private StaticProperty() { >>>> ? 46 >>>> ? 47???? } >>>> >>>> Maybe put this all on one line? >>>> >>> Will do >> >> Thanks, >> -Brent >> > From Alan.Bateman at oracle.com Wed Jun 27 07:29:33 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 Jun 2018 08:29:33 +0100 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> <5270e866-014c-497b-60db-855338bd7d8b@oracle.com> <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> Message-ID: <673ba81b-e65a-7cf4-95c5-6c0f47674552@oracle.com> On 27/06/2018 03:10, Roger Riggs wrote: > Hi, > > Updated webrev: > > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/index.html > > > Applied changes from prior comments and droped a change no longer > needed due > to the TLS 1.3 removal of ClientKeyExchangeService.java. > > The CSR has been approved without possibly confusing @implNote in > System.getProperties > about caching of specific properties, including java.home, etc. This looks okay and is a good first step to eliminating the issues caused by code changing these properties in a running VM. -Alan From Alan.Bateman at oracle.com Wed Jun 27 07:39:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 Jun 2018 08:39:14 +0100 Subject: RFR JDK-8200243: System error message is decoded as invalid encoding in Windows. In-Reply-To: <5B330126.2040708@oracle.com> References: <5B330126.2040708@oracle.com> Message-ID: On 27/06/2018 04:14, Xueming Shen wrote: > Hi, > > Please help review the change for JDK-8200243 > > issue: https://bugs.openjdk.java.net/browse/JDK-8200243 > webrev: http://cr.openjdk.java.net/~sherman/8200243/webrev > > This is a regression. The root cause is one of the change we put in > jdk9 for > JDK-8057777 [1], which is to remove those unused vm interfaces. > > The webrev [1] indicates that the changeset added a platform dependent > version of > GetLastErrorString(),? which uses utf8, to replace the original jvm > version of > JVM_GetLastErrorString() (in jni_util.c). > > However the encoding of the char* interface between jvm and jdk is always > interpreted/specified as "native encoding". So if the platform's > default encoding > is NOT utf-8, it never uses? utf8. This is what we have on Windows > platform. The > fix is to use/copy-paste the "original" vm implementation in hotspot > repo for > JVM_GetLastErrorString() (from os_windows.cpp). This looks okay, just not immediately clear if this links to FormatMessageA or FormatMessageW. -Alan From Alan.Bateman at oracle.com Wed Jun 27 07:44:58 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 Jun 2018 08:44:58 +0100 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B3298B9.2020603@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> <5B3298B9.2020603@oracle.com> Message-ID: <858f7537-2e0c-7c38-9c54-06014793540a@oracle.com> On 26/06/2018 20:49, Joe Wang wrote: > : > > Removed the null check. The internal impl and test guarantees it's not > null indeed: > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev04/ Looks good. -Alan From Joerg.Buchberger at pruftechnik.com Wed Jun 27 10:09:06 2018 From: Joerg.Buchberger at pruftechnik.com (Buchberger, Joerg) Date: Wed, 27 Jun 2018 10:09:06 +0000 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool Message-ID: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> Thanks for the info! And thanks for the efforts. [no irony, no sarcasm - I really mean it] But, to sum up my comprehension... anyone who placed their bets on javapackager, starting with last LTS Java 8 will be left in the rain with followup LTS Java 11, because their ain't neither javapackager (anymore), nor jpackager (yet). Is this correct? So, strategic choice boils down to either throw away all work done based on javapackager so far and the associated distribution concepts (reworking everything from scratch) or neglect Java 11 completely, thus placing all bets on jpackager really coming w/ Java 12 or even waiting for Java 14 as next LTS thereafter. Bam(!), I think, I first need a tiny shot now ;-) and let that info sink in ... Cheers J?rg On 5/31/2018 0:10 AM, Rushforth, Kevin wrote: > I would like to propose the following Draft JEP [1] for discussion. > > JDK-8200758: Packaging Tool > > This is intended as a JDK-scope replacement for the existing > javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK > releases), and was delivered as part of the JavaFX build. The > javapackager tool has been removed from Oracle JDK 11 along with the > removal of JavaFX. > > Comments on this JEP are welcome. It is currently not targeted for a > specific release, but we are aiming for JDK 12. > > -- Kevin > > [1] https://bugs.openjdk.java.net/browse/JDK-8200758 > From volker.simonis at gmail.com Wed Jun 27 10:26:14 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 27 Jun 2018 12:26:14 +0200 Subject: RFR(XXS): 8205916: [test] Fix jdk/tools/launcher/RunpathTest to handle both, RPATH and RUNPATH Message-ID: Hi, can I please have a review for the following tiny test fix (I'm actually not sure which would be the appropriate mailing list for this fix so please redirect if necessary): http://cr.openjdk.java.net/~simonis/webrevs/2018/8205916/ https://bugs.openjdk.java.net/browse/JDK-8205916 The test currently only checks for RPATH in the dynamic section of the launcher, but some linkers / Linux distributions (notably SLES) use RUNPATH instead. Following are the gory details: The test jdk/tools/launcher/RunpathTest.java checks that the java launcher on Linux and Solaris has the correct RPATH path baked into the executable. Unfortunately, the situation with RPATH is a little weird: - in order to bake a runtime path into a dynamically linked library or executable one has to use the "-rpath " linker option (from the C/C++ compiler this is usually available as "-Wl,-rpath,"). - depending on the dynamic linker version and Linux distribution, the "-rpath" linker option adds either a "RPATH" entry (Ubuntu 16.04, RHEL 7.4) or a "RUNPATH" entry (SLES 12.1, SLES 15) or both entries together (SLES 11.3) to the dynamic section of the shared library/executable. - the semantics of "RPATH" and "RUNPATH" are slightly different: "RPATH" is evaluated at runtime before LD_LIBRARY_PATH (if "RUNPATH" isn't present) while "RUNPATH" is evaluated after LD_LIBRARY_PATH (i.e. RUNPATH can be overridden at runtime by setting LD_LIBRARY_PATH). - "RPATH" is considered obsolete and should be replaced by "RUNPATH" according to the man-page of "ld.so (8)". - the linker option "--enable-new-dtags"/"--disable-new-dtags" (or the corresponding compiler flags "-Wl,--enable-new-dtags"/"-Wl,--disable-new-dtags") can be used to enforce the generation of "RUNPATH"/"RPATH" respectively (except for systems like SLES 11.3 where "--enable-new-dtags" generates both "RPATH" and "RUNPATH" while "--disable-new-dtags" only generates "RPATH"). But this issue is not about fixing the build so to cut a long story short - the test RunpathTest.java should be able to handle both runtime path variants equally well. This can be easily achieved by extending the match pattern from ".*RPATH.*\\$ORIGIN/../lib.*" to ".*R(UN)?PATH.*\\$ORIGIN/../lib.*" Thank you and best regards, Volker From sean.mullan at oracle.com Wed Jun 27 10:58:53 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Wed, 27 Jun 2018 06:58:53 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> <5270e866-014c-497b-60db-855338bd7d8b@oracle.com> <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> Message-ID: <6d94a319-4904-c6ce-d103-6cb2a1c5cb6c@oracle.com> I think it is worth putting a stronger warning in each of the methods (and not just the class description) of StaticProperty that additional care should be taken when using these methods since there is no SecurityManager check. For example: "{@link SecurityManager#checkPropertyAccess} is NOT checked in this method. The caller of this method should take care to ensure that the returned property is not made accessible to untrusted code." --Sean On 6/26/18 10:10 PM, Roger Riggs wrote: > Hi, > > Updated webrev: > > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/index.html > > > Applied changes from prior comments and droped a change no longer needed > due > to the TLS 1.3 removal of ClientKeyExchangeService.java. > > The CSR has been approved without possibly confusing @implNote in > System.getProperties > about caching of specific properties, including java.home, etc. > > Thanks for any additional comments. > > Roger > > > > On 6/19/18 11:52 AM, Brent Christian wrote: >> On 6/19/18 8:08 AM, Roger Riggs wrote: >>>> >>>> * src/java.base/share/classes/java/lang/System.java : >>>> >>>> Should the @implNote with the list of cached properties be added >>>> everywhere the @apiNote is being added ?? Right now the @implNote is >>>> only added to getProperties(). >>>> >>> The repetition was getting tiresome and the base of all the >>> xxxProperties methods is getProperties. >>> ??Joe suggested having one copy of the full information? and >>> referring to that from the individual @apiNotes. >> >> Fair enough. >> >>>> * src/java.base/share/classes/jdk/internal/util/StaticProperty.java : >>>> >>>> ? 45???? private StaticProperty() { >>>> ? 46 >>>> ? 47???? } >>>> >>>> Maybe put this all on one line? >>>> >>> Will do >> >> Thanks, >> -Brent >> > From Roger.Riggs at Oracle.com Wed Jun 27 13:20:43 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 27 Jun 2018 09:20:43 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <9d5359d7-2f93-4b81-7094-a20b782eb88e@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> <5270e866-014c-497b-60db-855338bd7d8b@oracle.com> <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> <9d5359d7-2f93-4b81-7094-a20b782eb88e@oracle.com> Message-ID: I rechecked, almost all were within doPriv and the others have new explicit SM checks. Thanks, Roger On 6/27/2018 1:54 AM, mandy chung wrote: > Looks good to me.? The part that I'm unsure is whether you caught all > the callers that are outside doPrivileged to StaticProperty.* will > need to do its property permission check.? This requires API > inspection and testing that I assume you covered that. > > Mandy > > On 6/26/18 7:10 PM, Roger Riggs wrote: >> Hi, >> >> Updated webrev: >> >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/index.html >> >> >> Applied changes from prior comments and droped a change no longer >> needed due >> to the TLS 1.3 removal of ClientKeyExchangeService.java. >> >> The CSR has been approved without possibly confusing @implNote in >> System.getProperties >> about caching of specific properties, including java.home, etc. >> >> Thanks for any additional comments. >> >> Roger >> >> >> >> On 6/19/18 11:52 AM, Brent Christian wrote: >>> On 6/19/18 8:08 AM, Roger Riggs wrote: >>>>> >>>>> * src/java.base/share/classes/java/lang/System.java : >>>>> >>>>> Should the @implNote with the list of cached properties be added >>>>> everywhere the @apiNote is being added ?? Right now the @implNote >>>>> is only added to getProperties(). >>>>> >>>> The repetition was getting tiresome and the base of all the >>>> xxxProperties methods is getProperties. >>>> ??Joe suggested having one copy of the full information? and >>>> referring to that from the individual @apiNotes. >>> >>> Fair enough. >>> >>>>> * src/java.base/share/classes/jdk/internal/util/StaticProperty.java : >>>>> >>>>> ? 45???? private StaticProperty() { >>>>> ? 46 >>>>> ? 47???? } >>>>> >>>>> Maybe put this all on one line? >>>>> >>>> Will do >>> >>> Thanks, >>> -Brent >>> >> From Roger.Riggs at Oracle.com Wed Jun 27 13:41:28 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 27 Jun 2018 09:41:28 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <6d94a319-4904-c6ce-d103-6cb2a1c5cb6c@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> <7a84b30b-dcd7-1c15-a705-d93845715f8b@oracle.com> <553500e5-bfaf-f2a7-f03d-964da3676d75@Oracle.com> <5270e866-014c-497b-60db-855338bd7d8b@oracle.com> <7f195ee2-3cc4-65ee-7f4c-97a33be8682d@oracle.com> <6d94a319-4904-c6ce-d103-6cb2a1c5cb6c@oracle.com> Message-ID: Hi Sean, Updated webrev: http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/index.html I added the suggested text and updated the class level warning to match. Thanks, Roger On 6/27/2018 6:58 AM, Sean Mullan wrote: > I think it is worth putting a stronger warning in each of the methods > (and not just the class description) of StaticProperty that additional > care should be taken when using these methods since there is no > SecurityManager check. For example: > > > "{@link SecurityManager#checkPropertyAccess} is NOT checked > in this method. The caller of this method should take care to ensure > that the returned property is not made accessible to untrusted code." > > --Sean > > On 6/26/18 10:10 PM, Roger Riggs wrote: >> Hi, >> >> Updated webrev: >> >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/index.html >> >> >> Applied changes from prior comments and droped a change no longer >> needed due >> to the TLS 1.3 removal of ClientKeyExchangeService.java. >> >> The CSR has been approved without possibly confusing @implNote in >> System.getProperties >> about caching of specific properties, including java.home, etc. >> >> Thanks for any additional comments. >> >> Roger >> >> >> >> On 6/19/18 11:52 AM, Brent Christian wrote: >>> On 6/19/18 8:08 AM, Roger Riggs wrote: >>>>> >>>>> * src/java.base/share/classes/java/lang/System.java : >>>>> >>>>> Should the @implNote with the list of cached properties be added >>>>> everywhere the @apiNote is being added ?? Right now the @implNote >>>>> is only added to getProperties(). >>>>> >>>> The repetition was getting tiresome and the base of all the >>>> xxxProperties methods is getProperties. >>>> ??Joe suggested having one copy of the full information? and >>>> referring to that from the individual @apiNotes. >>> >>> Fair enough. >>> >>>>> * src/java.base/share/classes/jdk/internal/util/StaticProperty.java : >>>>> >>>>> ? 45???? private StaticProperty() { >>>>> ? 46 >>>>> ? 47???? } >>>>> >>>>> Maybe put this all on one line? >>>>> >>>> Will do >>> >>> Thanks, >>> -Brent >>> >> From martinrb at google.com Wed Jun 27 14:45:08 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 27 Jun 2018 07:45:08 -0700 Subject: RFR(XXS): 8205916: [test] Fix jdk/tools/launcher/RunpathTest to handle both, RPATH and RUNPATH In-Reply-To: References: Message-ID: Looks good to me! On Wed, Jun 27, 2018 at 3:26 AM, Volker Simonis wrote: > Hi, > > can I please have a review for the following tiny test fix (I'm > actually not sure which would be the appropriate mailing list for this > fix so please redirect if necessary): > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8205916/ > https://bugs.openjdk.java.net/browse/JDK-8205916 > > The test currently only checks for RPATH in the dynamic section of the > launcher, but some linkers / Linux distributions (notably SLES) use > RUNPATH instead. > > Following are the gory details: > > The test jdk/tools/launcher/RunpathTest.java checks that the java > launcher on Linux and Solaris has the correct RPATH path baked into > the executable. > > Unfortunately, the situation with RPATH is a little weird: > > - in order to bake a runtime path into a dynamically linked library > or executable one has to use the "-rpath " linker option (from > the C/C++ compiler this is usually available as "-Wl,-rpath,"). > - depending on the dynamic linker version and Linux distribution, > the "-rpath" linker option adds either a "RPATH" entry (Ubuntu 16.04, > RHEL 7.4) or a "RUNPATH" entry (SLES 12.1, SLES 15) or both entries > together (SLES 11.3) to the dynamic section of the shared > library/executable. > - the semantics of "RPATH" and "RUNPATH" are slightly different: > "RPATH" is evaluated at runtime before LD_LIBRARY_PATH (if "RUNPATH" > isn't present) while "RUNPATH" is evaluated after LD_LIBRARY_PATH > (i.e. RUNPATH can be overridden at runtime by setting > LD_LIBRARY_PATH). > - "RPATH" is considered obsolete and should be replaced by "RUNPATH" > according to the man-page of "ld.so (8)". > - the linker option "--enable-new-dtags"/"--disable-new-dtags" (or > the corresponding compiler flags > "-Wl,--enable-new-dtags"/"-Wl,--disable-new-dtags") can be used to > enforce the generation of "RUNPATH"/"RPATH" respectively (except for > systems like SLES 11.3 where "--enable-new-dtags" generates both > "RPATH" and "RUNPATH" while "--disable-new-dtags" only generates > "RPATH"). > > But this issue is not about fixing the build so to cut a long story > short - the test RunpathTest.java should be able to handle both > runtime path variants equally well. This can be easily achieved by > extending the match pattern from ".*RPATH.*\\$ORIGIN/../lib.*" to > ".*R(UN)?PATH.*\\$ORIGIN/../lib.*" > > Thank you and best regards, > Volker > From xueming.shen at oracle.com Wed Jun 27 14:53:06 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 27 Jun 2018 07:53:06 -0700 Subject: RFR JDK-8200243: System error message is decoded as invalid encoding in Windows. In-Reply-To: References: <5B330126.2040708@oracle.com> Message-ID: <5B33A4D2.1090202@oracle.com> On 6/27/18, 12:39 AM, Alan Bateman wrote: > On 27/06/2018 04:14, Xueming Shen wrote: >> Hi, >> >> Please help review the change for JDK-8200243 >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8200243 >> webrev: http://cr.openjdk.java.net/~sherman/8200243/webrev >> >> This is a regression. The root cause is one of the change we put in >> jdk9 for >> JDK-8057777 [1], which is to remove those unused vm interfaces. >> >> The webrev [1] indicates that the changeset added a platform >> dependent version of >> GetLastErrorString(), which uses utf8, to replace the original jvm >> version of >> JVM_GetLastErrorString() (in jni_util.c). >> >> However the encoding of the char* interface between jvm and jdk is >> always >> interpreted/specified as "native encoding". So if the platform's >> default encoding >> is NOT utf-8, it never uses utf8. This is what we have on Windows >> platform. The >> fix is to use/copy-paste the "original" vm implementation in hotspot >> repo for >> JVM_GetLastErrorString() (from os_windows.cpp). > This looks okay, just not immediately clear if this links to > FormatMessageA or FormatMessageW. > Without an explicit _UNICODE and UNICODE switch it should always be "A" version. If we have more time maybe I should go through the circle to simply use the "A" explicitly. -Sherman From erik.joelsson at oracle.com Wed Jun 27 15:53:11 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 27 Jun 2018 08:53:11 -0700 Subject: RFR(XXS): 8205916: [test] Fix jdk/tools/launcher/RunpathTest to handle both, RPATH and RUNPATH In-Reply-To: References: Message-ID: Looks ok to me. /Erik On 2018-06-27 03:26, Volker Simonis wrote: > Hi, > > can I please have a review for the following tiny test fix (I'm > actually not sure which would be the appropriate mailing list for this > fix so please redirect if necessary): > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8205916/ > https://bugs.openjdk.java.net/browse/JDK-8205916 > > The test currently only checks for RPATH in the dynamic section of the > launcher, but some linkers / Linux distributions (notably SLES) use > RUNPATH instead. > > Following are the gory details: > > The test jdk/tools/launcher/RunpathTest.java checks that the java > launcher on Linux and Solaris has the correct RPATH path baked into > the executable. > > Unfortunately, the situation with RPATH is a little weird: > > - in order to bake a runtime path into a dynamically linked library > or executable one has to use the "-rpath " linker option (from > the C/C++ compiler this is usually available as "-Wl,-rpath,"). > - depending on the dynamic linker version and Linux distribution, > the "-rpath" linker option adds either a "RPATH" entry (Ubuntu 16.04, > RHEL 7.4) or a "RUNPATH" entry (SLES 12.1, SLES 15) or both entries > together (SLES 11.3) to the dynamic section of the shared > library/executable. > - the semantics of "RPATH" and "RUNPATH" are slightly different: > "RPATH" is evaluated at runtime before LD_LIBRARY_PATH (if "RUNPATH" > isn't present) while "RUNPATH" is evaluated after LD_LIBRARY_PATH > (i.e. RUNPATH can be overridden at runtime by setting > LD_LIBRARY_PATH). > - "RPATH" is considered obsolete and should be replaced by "RUNPATH" > according to the man-page of "ld.so (8)". > - the linker option "--enable-new-dtags"/"--disable-new-dtags" (or > the corresponding compiler flags > "-Wl,--enable-new-dtags"/"-Wl,--disable-new-dtags") can be used to > enforce the generation of "RUNPATH"/"RPATH" respectively (except for > systems like SLES 11.3 where "--enable-new-dtags" generates both > "RPATH" and "RUNPATH" while "--disable-new-dtags" only generates > "RPATH"). > > But this issue is not about fixing the build so to cut a long story > short - the test RunpathTest.java should be able to handle both > runtime path variants equally well. This can be easily achieved by > extending the match pattern from ".*RPATH.*\\$ORIGIN/../lib.*" to > ".*R(UN)?PATH.*\\$ORIGIN/../lib.*" > > Thank you and best regards, > Volker From huizhe.wang at oracle.com Wed Jun 27 16:34:14 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 27 Jun 2018 09:34:14 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <858f7537-2e0c-7c38-9c54-06014793540a@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> <5B3298B9.2020603@oracle.com> <858f7537-2e0c-7c38-9c54-06014793540a@oracle.com> Message-ID: <5B33BC86.4000708@oracle.com> Thanks all! Glad to be able to get this done right before the deadline :-) -Joe On 6/27/18, 12:44 AM, Alan Bateman wrote: > On 26/06/2018 20:49, Joe Wang wrote: >> : >> >> Removed the null check. The internal impl and test guarantees it's >> not null indeed: >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev04/ > Looks good. > > -Alan From xuelei.fan at oracle.com Wed Jun 27 18:57:37 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 27 Jun 2018 11:57:37 -0700 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> Message-ID: <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> Hi Sean, I may reply in several replies. PKIXMasterCertPathValidator.java -------------------------------- + CertChainEvent cce = new CertChainEvent(); + if(cce.isEnabled() || EventHelper.loggingSecurity()) { + String c = reversedCertList.stream() + .map(x -> x.getSerialNumber().toString(16)) + .collect(Collectors.joining(", ")); + EventHelper.commitCertChainEvent(cce, c); + } No matter the event or the JFR mechanism is enabled or not, the above code will create a new instance. Is the return value of cce.isEnabled() dynamically changed or static? Is there a need to support both logging and JFR? I'm new to record events. I did not get the point to use both. Thanks, Xuelei On 6/26/2018 3:18 PM, Se?n Coffey wrote: > Erik, > > I rebased the patch with TLS v1.3 integration today. I hadn't realized > how much the handshaker code had changed. Hopefully, I'll get a review > from security dev team on that front. > > Regards the JFR semantics, I believe the edits should match majority of > requests . I still have a preference for the test infra design for these > new logger/JFR tests used in version 1 of webrev. I think it makes sense > to keep the test functionality together - no sense in separating them > out into different components IMO. Also, some of the edits to the JFR > testing were made to test for the new dual log/record functionality. I > might catch up with you tomorrow to see what the best arrangement would be. > > http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ > > regards, > Sean. > > > On 25/06/2018 21:22, Se?n Coffey wrote: >> Many thanks for the review comments Erik. Replies inline. >> >> >> On 24/06/2018 14:21, Erik Gahlin wrote: >>> Hi Sean, >>> >>> Some of the changes in the webrev belongs to JDK-8203629 and should >>> be removed for clarity. >>> >>> Some initial comments: >>> >>> default.jfc, profile.jfr: >>> The events should not have control="enable-exceptions". The purpose >>> of the control attribute is so to provide parameterized configuration >>> of events for JMC.? As it is now, the security events will be enabled >>> when a user turns on the exception events. >> Makes sense. I'll remove that parameter. >>> >>> X509CertEvent: >>> You should use milliseconds since epoch to represent a date instead >>> of a string value, i.e. >>> >>> ??? @Label("Valid From") >>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>> ??? public long validFrom; >>> >>> ??? @Label("Valid Until") >>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>> ??? public long validUntil; >>> >> The CertificateValidity class operates on Date Object values. I'll >> work with the Date.getTime() method then (and update the Logger >> formatting) >>> This following annotation adds little value >>> >>> @Description("Details of Security Property") >>> >>> I would either remove the annotation, or provide information that >>> helps a user understand the event. For instance, what is X509, and >>> what kind of certificates are we talking about? >> Yes - that looks like the wrong Description. I'll review all of these >> fields. >>> >>> @Category("Java Application") >>> >>> I'm a bit worried that we will pollute the "Java Application" >>> namespace if we add lots of JDK events in that category. We may want >>> to add a new top level category "Java Development Kit", analogous to >>> the "Java Virtual Machine" category, and put all security related >>> events in subcategory, perhaps called "Security". >> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>> >>> @Label("X509Cert") >>> >>> The label should be human readable name, with spaces, title cased >>> etc. I would recommend "X.509 Certificate". In general, avoid >>> abbreviations like "certs" and instead use labels such as >>> "Certificate Chain". The label should be short and not a full sentence. >>> >>> For details see, >>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>> >>> I think it would be good to separate testing of JFR and logging into >>> different files / tests. I would prefer that the test name is the >>> same as the event prefixed with "Test", i.e TestX509CertificateEvent, >>> as this is the pattern used by other JFR tests. >>> >> I'll take a look at that pattern again. I've separated out the current >> tests into an (a) outer test to analyze the logger output and (b) the >> inner test which checks for JFR correctness. I did include extra logic >> to make sure that the EventHelper configuration was working correctly. >> "Events.assertField" looks very helpful. Thanks for the pointer. >> >> Let me take on board the suggestions below and get a new webrev out on >> Tuesday. >> >> regards, >> Sean. >> >>> I reworked one of the tests to how I like to see it: >>> >>> /* >>> ?* @test >>> ?* @key jfr >>> ?* @library /test/lib >>> ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>> ?*/ >>> public class TestX509CertificateEvent { >>> >>> ??? private static final String CERTIFICATE_1 = "..."; >>> ??? private static final String CERTIFICATE_2 = "..."; >>> >>> ??? public static void main(String... args) throws >>> CertificateException { >>> >>> ???????? Recording r = new Recording(); >>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>> ???????? r.start(); >>> >>> ???????? CertificateFactory cf = >>> CertificateFactory.getInstance("X.509"); >>> ???????? cf.generateCertificate(new >>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>> ???????? cf.generateCertificate(new >>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>> >>> ???????? // Make sure only one event per certificate >>> ???????? cf.generateCertificate(new >>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>> ???????? cf.generateCertificate(new >>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>> >>> ???????? r.stop(); >>> >>> ???????? List events = Events.fromRecording(r); >>> ???????? Asserts.assertEquals(events.size(), 2, "Expected two X.509 >>> Certificate events"); >>> >>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>> ???????????????????? "RSA", 2048); >>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>> ???????????????????? "RSA", 2048); >>> ??? } >>> >>> ??? private static void assertEvent(List events, >>> String certID, String algId, String subject, >>> ??????????? String issuer, String keyType, int length) throws >>> Exception { >>> >>> ??????? for (RecordedEvent e : events) { >>> ??????????? if (e.getString("serialNumber").equals(certID)) { >>> ??????????????? Events.assertField(e, "algId").equal(algId); >>> ??????????????? ... >>> ??????????????? return; >>> ??????????? } >>> ??????? } >>> ??????? System.out.println(events); >>> ??????? throw new Exception("Could not find event with Cert ID: " + >>> certID); >>> ??? } >>> } >>> >>> The reworked example uses the Events.assertField method, which will >>> give context if the assertion fails. Keeping the test simple, means >>> it can be analyzed quickly if it fails in testing. There is no new >>> test framework to learn, or methods to search for, and it is usually >>> not hard to determine if the failure is product, test or >>> infrastructure related, and what component (team) should be assigned >>> the issue. The use of EventNames.X509Certificate means all >>> occurrences of the event can be tracked done in an IDE using find by >>> reference. >>> >>> Thanks >>> Erik >>> >>>> Following on from the recent JDK-8203629 code review, I'd like to >>>> propose enhancements on how we can record events in security libs. >>>> The introduction of the JFR libraries can give us much better ways >>>> of examining JDK actions. For the initial phase, I'm looking to >>>> enhance some key security library events in JDK 11 so that they can >>>> be either recorded to JFR, logged to a traditional logger, or both. >>>> >>>> Examples of how useful JFR recordings could be can be seen here : >>>> >>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>> >>>> securityProp_2.png gives an example of how the JFR recording can be >>>> queried to quickly locate events of interest (in this case, code >>>> setting the jdk.tls.* Security properties). I still need to clean up >>>> the TLSEvents testcase to improve test coverage and hope to do that >>>> in coming days. >>>> >>>> JBS record : >>>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>>> >>>> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>> >>> >> > From sean.coffey at oracle.com Wed Jun 27 19:14:04 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 27 Jun 2018 20:14:04 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> Message-ID: On 27/06/2018 19:57, Xuelei Fan wrote: > Hi Sean, > > I may reply in several replies. > > PKIXMasterCertPathValidator.java > -------------------------------- > +? CertChainEvent cce = new CertChainEvent(); > +? if(cce.isEnabled() || EventHelper.loggingSecurity()) { > +????? String c = reversedCertList.stream() > +????????????????? .map(x -> x.getSerialNumber().toString(16)) > +??????????????????????? .collect(Collectors.joining(", ")); > +???? EventHelper.commitCertChainEvent(cce, c); > +?? } > > No matter the event or the JFR mechanism is enabled or not, the above > code will create a new instance.? Is the return value of > cce.isEnabled() dynamically changed or static? This is a requirement from the JFR framework. All their event.isEnabled calls are instance methods and follow a similar pattern. The JFR team assure me that the JIT can optimize away such calls. > > Is there a need to support both logging and JFR?? I'm new to record > events.? I did not get the point to use both. I was initially hoping to concentrate on just JFR events but I got strong feedback to also consider use of Logger (esp. in cases where the jdk.jfr module is not available) regards, Sean. > > Thanks, > Xuelei > > On 6/26/2018 3:18 PM, Se?n Coffey wrote: >> Erik, >> >> I rebased the patch with TLS v1.3 integration today. I hadn't >> realized how much the handshaker code had changed. Hopefully, I'll >> get a review from security dev team on that front. >> >> Regards the JFR semantics, I believe the edits should match majority >> of requests . I still have a preference for the test infra design for >> these new logger/JFR tests used in version 1 of webrev. I think it >> makes sense to keep the test functionality together - no sense in >> separating them out into different components IMO. Also, some of the >> edits to the JFR testing were made to test for the new dual >> log/record functionality. I might catch up with you tomorrow to see >> what the best arrangement would be. >> >> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >> >> regards, >> Sean. >> >> >> On 25/06/2018 21:22, Se?n Coffey wrote: >>> Many thanks for the review comments Erik. Replies inline. >>> >>> >>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>> Hi Sean, >>>> >>>> Some of the changes in the webrev belongs to JDK-8203629 and should >>>> be removed for clarity. >>>> >>>> Some initial comments: >>>> >>>> default.jfc, profile.jfr: >>>> The events should not have control="enable-exceptions". The purpose >>>> of the control attribute is so to provide parameterized >>>> configuration of events for JMC.? As it is now, the security events >>>> will be enabled when a user turns on the exception events. >>> Makes sense. I'll remove that parameter. >>>> >>>> X509CertEvent: >>>> You should use milliseconds since epoch to represent a date instead >>>> of a string value, i.e. >>>> >>>> ??? @Label("Valid From") >>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>> ??? public long validFrom; >>>> >>>> ??? @Label("Valid Until") >>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>> ??? public long validUntil; >>>> >>> The CertificateValidity class operates on Date Object values. I'll >>> work with the Date.getTime() method then (and update the Logger >>> formatting) >>>> This following annotation adds little value >>>> >>>> @Description("Details of Security Property") >>>> >>>> I would either remove the annotation, or provide information that >>>> helps a user understand the event. For instance, what is X509, and >>>> what kind of certificates are we talking about? >>> Yes - that looks like the wrong Description. I'll review all of >>> these fields. >>>> >>>> @Category("Java Application") >>>> >>>> I'm a bit worried that we will pollute the "Java Application" >>>> namespace if we add lots of JDK events in that category. We may >>>> want to add a new top level category "Java Development Kit", >>>> analogous to the "Java Virtual Machine" category, and put all >>>> security related events in subcategory, perhaps called "Security". >>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>> >>>> @Label("X509Cert") >>>> >>>> The label should be human readable name, with spaces, title cased >>>> etc. I would recommend "X.509 Certificate". In general, avoid >>>> abbreviations like "certs" and instead use labels such as >>>> "Certificate Chain". The label should be short and not a full >>>> sentence. >>>> >>>> For details see, >>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>> >>>> I think it would be good to separate testing of JFR and logging >>>> into different files / tests. I would prefer that the test name is >>>> the same as the event prefixed with "Test", i.e >>>> TestX509CertificateEvent, as this is the pattern used by other JFR >>>> tests. >>>> >>> I'll take a look at that pattern again. I've separated out the >>> current tests into an (a) outer test to analyze the logger output >>> and (b) the inner test which checks for JFR correctness. I did >>> include extra logic to make sure that the EventHelper configuration >>> was working correctly. "Events.assertField" looks very helpful. >>> Thanks for the pointer. >>> >>> Let me take on board the suggestions below and get a new webrev out >>> on Tuesday. >>> >>> regards, >>> Sean. >>> >>>> I reworked one of the tests to how I like to see it: >>>> >>>> /* >>>> ?* @test >>>> ?* @key jfr >>>> ?* @library /test/lib >>>> ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>> ?*/ >>>> public class TestX509CertificateEvent { >>>> >>>> ??? private static final String CERTIFICATE_1 = "..."; >>>> ??? private static final String CERTIFICATE_2 = "..."; >>>> >>>> ??? public static void main(String... args) throws >>>> CertificateException { >>>> >>>> ???????? Recording r = new Recording(); >>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>> ???????? r.start(); >>>> >>>> ???????? CertificateFactory cf = >>>> CertificateFactory.getInstance("X.509"); >>>> ???????? cf.generateCertificate(new >>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>> ???????? cf.generateCertificate(new >>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>> >>>> ???????? // Make sure only one event per certificate >>>> ???????? cf.generateCertificate(new >>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>> ???????? cf.generateCertificate(new >>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>> >>>> ???????? r.stop(); >>>> >>>> ???????? List events = Events.fromRecording(r); >>>> ???????? Asserts.assertEquals(events.size(), 2, "Expected two X.509 >>>> Certificate events"); >>>> >>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>> ???????????????????? "RSA", 2048); >>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>> ???????????????????? "RSA", 2048); >>>> ??? } >>>> >>>> ??? private static void assertEvent(List events, >>>> String certID, String algId, String subject, >>>> ??????????? String issuer, String keyType, int length) throws >>>> Exception { >>>> >>>> ??????? for (RecordedEvent e : events) { >>>> ??????????? if (e.getString("serialNumber").equals(certID)) { >>>> ??????????????? Events.assertField(e, "algId").equal(algId); >>>> ??????????????? ... >>>> ??????????????? return; >>>> ??????????? } >>>> ??????? } >>>> ??????? System.out.println(events); >>>> ??????? throw new Exception("Could not find event with Cert ID: " + >>>> certID); >>>> ??? } >>>> } >>>> >>>> The reworked example uses the Events.assertField method, which will >>>> give context if the assertion fails. Keeping the test simple, means >>>> it can be analyzed quickly if it fails in testing. There is no new >>>> test framework to learn, or methods to search for, and it is >>>> usually not hard to determine if the failure is product, test or >>>> infrastructure related, and what component (team) should be >>>> assigned the issue. The use of EventNames.X509Certificate means all >>>> occurrences of the event can be tracked done in an IDE using find >>>> by reference. >>>> >>>> Thanks >>>> Erik >>>> >>>>> Following on from the recent JDK-8203629 code review, I'd like to >>>>> propose enhancements on how we can record events in security libs. >>>>> The introduction of the JFR libraries can give us much better ways >>>>> of examining JDK actions. For the initial phase, I'm looking to >>>>> enhance some key security library events in JDK 11 so that they >>>>> can be either recorded to JFR, logged to a traditional logger, or >>>>> both. >>>>> >>>>> Examples of how useful JFR recordings could be can be seen here : >>>>> >>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>> >>>>> securityProp_2.png gives an example of how the JFR recording can >>>>> be queried to quickly locate events of interest (in this case, >>>>> code setting the jdk.tls.* Security properties). I still need to >>>>> clean up the TLSEvents testcase to improve test coverage and hope >>>>> to do that in coming days. >>>>> >>>>> JBS record : >>>>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>> >>>>> webrev : >>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>> >>>> >>> >> From xuelei.fan at oracle.com Wed Jun 27 19:46:49 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 27 Jun 2018 12:46:49 -0700 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> Message-ID: KeyUtil.java ------------ 167 public static String getKeyAlgorithm(Key key) { I may suggest remove this method and use Key.getAlgorithm() directly. The KeyUtil.getKeyAlgorithm() is incomplete, and may need additional maintenance if new key algorithms are added in the future. For example, in JDK 11, "RSASSA-PSS" and "XDH" are added, but we really forgot that we may need to update this file as well. On 6/27/2018 12:14 PM, Se?n Coffey wrote: > > > On 27/06/2018 19:57, Xuelei Fan wrote: >> Hi Sean, >> >> I may reply in several replies. >> >> PKIXMasterCertPathValidator.java >> -------------------------------- >> +? CertChainEvent cce = new CertChainEvent(); >> +? if(cce.isEnabled() || EventHelper.loggingSecurity()) { >> +????? String c = reversedCertList.stream() >> +????????????????? .map(x -> x.getSerialNumber().toString(16)) >> +??????????????????????? .collect(Collectors.joining(", ")); >> +???? EventHelper.commitCertChainEvent(cce, c); >> +?? } >> >> No matter the event or the JFR mechanism is enabled or not, the above >> code will create a new instance.? Is the return value of >> cce.isEnabled() dynamically changed or static? > This is a requirement from the JFR framework. All their event.isEnabled > calls are instance methods and follow a similar pattern. The JFR team > assure me that the JIT can optimize away such calls. Got it. >> >> Is there a need to support both logging and JFR?? I'm new to record >> events.? I did not get the point to use both. > I was initially hoping to concentrate on just JFR events but I got > strong feedback to also consider use of Logger (esp. in cases where the > jdk.jfr module is not available) > There are three logging mechanisms: JFR, record event log and the component debugging log. It is a little bit overloaded to me. If jdk.jfr is not available, the existing component debugging log may be used instead. I think you must have weight this decision in the past, so I will accept your decision if you want to keep it. Thanks, Xuelei > regards, > Sean. > >> >> Thanks, >> Xuelei >> >> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>> Erik, >>> >>> I rebased the patch with TLS v1.3 integration today. I hadn't >>> realized how much the handshaker code had changed. Hopefully, I'll >>> get a review from security dev team on that front. >>> >>> Regards the JFR semantics, I believe the edits should match majority >>> of requests . I still have a preference for the test infra design for >>> these new logger/JFR tests used in version 1 of webrev. I think it >>> makes sense to keep the test functionality together - no sense in >>> separating them out into different components IMO. Also, some of the >>> edits to the JFR testing were made to test for the new dual >>> log/record functionality. I might catch up with you tomorrow to see >>> what the best arrangement would be. >>> >>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>> >>> regards, >>> Sean. >>> >>> >>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>> Many thanks for the review comments Erik. Replies inline. >>>> >>>> >>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>> Hi Sean, >>>>> >>>>> Some of the changes in the webrev belongs to JDK-8203629 and should >>>>> be removed for clarity. >>>>> >>>>> Some initial comments: >>>>> >>>>> default.jfc, profile.jfr: >>>>> The events should not have control="enable-exceptions". The purpose >>>>> of the control attribute is so to provide parameterized >>>>> configuration of events for JMC.? As it is now, the security events >>>>> will be enabled when a user turns on the exception events. >>>> Makes sense. I'll remove that parameter. >>>>> >>>>> X509CertEvent: >>>>> You should use milliseconds since epoch to represent a date instead >>>>> of a string value, i.e. >>>>> >>>>> ??? @Label("Valid From") >>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>> ??? public long validFrom; >>>>> >>>>> ??? @Label("Valid Until") >>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>> ??? public long validUntil; >>>>> >>>> The CertificateValidity class operates on Date Object values. I'll >>>> work with the Date.getTime() method then (and update the Logger >>>> formatting) >>>>> This following annotation adds little value >>>>> >>>>> @Description("Details of Security Property") >>>>> >>>>> I would either remove the annotation, or provide information that >>>>> helps a user understand the event. For instance, what is X509, and >>>>> what kind of certificates are we talking about? >>>> Yes - that looks like the wrong Description. I'll review all of >>>> these fields. >>>>> >>>>> @Category("Java Application") >>>>> >>>>> I'm a bit worried that we will pollute the "Java Application" >>>>> namespace if we add lots of JDK events in that category. We may >>>>> want to add a new top level category "Java Development Kit", >>>>> analogous to the "Java Virtual Machine" category, and put all >>>>> security related events in subcategory, perhaps called "Security". >>>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>>> >>>>> @Label("X509Cert") >>>>> >>>>> The label should be human readable name, with spaces, title cased >>>>> etc. I would recommend "X.509 Certificate". In general, avoid >>>>> abbreviations like "certs" and instead use labels such as >>>>> "Certificate Chain". The label should be short and not a full >>>>> sentence. >>>>> >>>>> For details see, >>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>> >>>>> I think it would be good to separate testing of JFR and logging >>>>> into different files / tests. I would prefer that the test name is >>>>> the same as the event prefixed with "Test", i.e >>>>> TestX509CertificateEvent, as this is the pattern used by other JFR >>>>> tests. >>>>> >>>> I'll take a look at that pattern again. I've separated out the >>>> current tests into an (a) outer test to analyze the logger output >>>> and (b) the inner test which checks for JFR correctness. I did >>>> include extra logic to make sure that the EventHelper configuration >>>> was working correctly. "Events.assertField" looks very helpful. >>>> Thanks for the pointer. >>>> >>>> Let me take on board the suggestions below and get a new webrev out >>>> on Tuesday. >>>> >>>> regards, >>>> Sean. >>>> >>>>> I reworked one of the tests to how I like to see it: >>>>> >>>>> /* >>>>> ?* @test >>>>> ?* @key jfr >>>>> ?* @library /test/lib >>>>> ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>>> ?*/ >>>>> public class TestX509CertificateEvent { >>>>> >>>>> ??? private static final String CERTIFICATE_1 = "..."; >>>>> ??? private static final String CERTIFICATE_2 = "..."; >>>>> >>>>> ??? public static void main(String... args) throws >>>>> CertificateException { >>>>> >>>>> ???????? Recording r = new Recording(); >>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>> ???????? r.start(); >>>>> >>>>> ???????? CertificateFactory cf = >>>>> CertificateFactory.getInstance("X.509"); >>>>> ???????? cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>> ???????? cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>> >>>>> ???????? // Make sure only one event per certificate >>>>> ???????? cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>> ???????? cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>> >>>>> ???????? r.stop(); >>>>> >>>>> ???????? List events = Events.fromRecording(r); >>>>> ???????? Asserts.assertEquals(events.size(), 2, "Expected two X.509 >>>>> Certificate events"); >>>>> >>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>> ???????????????????? "RSA", 2048); >>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>> ???????????????????? "RSA", 2048); >>>>> ??? } >>>>> >>>>> ??? private static void assertEvent(List events, >>>>> String certID, String algId, String subject, >>>>> ??????????? String issuer, String keyType, int length) throws >>>>> Exception { >>>>> >>>>> ??????? for (RecordedEvent e : events) { >>>>> ??????????? if (e.getString("serialNumber").equals(certID)) { >>>>> ??????????????? Events.assertField(e, "algId").equal(algId); >>>>> ??????????????? ... >>>>> ??????????????? return; >>>>> ??????????? } >>>>> ??????? } >>>>> ??????? System.out.println(events); >>>>> ??????? throw new Exception("Could not find event with Cert ID: " + >>>>> certID); >>>>> ??? } >>>>> } >>>>> >>>>> The reworked example uses the Events.assertField method, which will >>>>> give context if the assertion fails. Keeping the test simple, means >>>>> it can be analyzed quickly if it fails in testing. There is no new >>>>> test framework to learn, or methods to search for, and it is >>>>> usually not hard to determine if the failure is product, test or >>>>> infrastructure related, and what component (team) should be >>>>> assigned the issue. The use of EventNames.X509Certificate means all >>>>> occurrences of the event can be tracked done in an IDE using find >>>>> by reference. >>>>> >>>>> Thanks >>>>> Erik >>>>> >>>>>> Following on from the recent JDK-8203629 code review, I'd like to >>>>>> propose enhancements on how we can record events in security libs. >>>>>> The introduction of the JFR libraries can give us much better ways >>>>>> of examining JDK actions. For the initial phase, I'm looking to >>>>>> enhance some key security library events in JDK 11 so that they >>>>>> can be either recorded to JFR, logged to a traditional logger, or >>>>>> both. >>>>>> >>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>> >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>> >>>>>> securityProp_2.png gives an example of how the JFR recording can >>>>>> be queried to quickly locate events of interest (in this case, >>>>>> code setting the jdk.tls.* Security properties). I still need to >>>>>> clean up the TLSEvents testcase to improve test coverage and hope >>>>>> to do that in coming days. >>>>>> >>>>>> JBS record : >>>>>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>> >>>>>> webrev : >>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>> >>>>> >>>> >>> > From erik.gahlin at oracle.com Wed Jun 27 19:57:28 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 27 Jun 2018 21:57:28 +0200 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> Message-ID: <5B33EC28.6090604@oracle.com> On 2018-06-27 21:14, Se?n Coffey wrote: > > > On 27/06/2018 19:57, Xuelei Fan wrote: >> Hi Sean, >> >> I may reply in several replies. >> >> PKIXMasterCertPathValidator.java >> -------------------------------- >> + CertChainEvent cce = new CertChainEvent(); >> + if(cce.isEnabled() || EventHelper.loggingSecurity()) { >> + String c = reversedCertList.stream() >> + .map(x -> x.getSerialNumber().toString(16)) >> + .collect(Collectors.joining(", ")); >> + EventHelper.commitCertChainEvent(cce, c); >> + } >> >> No matter the event or the JFR mechanism is enabled or not, the above >> code will create a new instance. Is the return value of >> cce.isEnabled() dynamically changed or static? > This is a requirement from the JFR framework. All their > event.isEnabled calls are instance methods and follow a similar > pattern. The JFR team assure me that the JIT can optimize away such > calls. The JIT will most likely not be able to optimize if the event class escapes. From a JFR perspective, this would be the preferred layout: EventA eventA= new EventA(); eventA.value = this.value; eventA.commit(); and then do logging separately: System.Logger.log(DEBUG, this.value) With this layout, the JIT will remove the allocation and dead store. If it is expensive to gather the data for the event, like the CertChainEvent above, the following pattern should be used. EventB eventB= new EventB (); if (eventB.shouldCommit()) { eventB.value = calculateValue(); eventB .commit(); } If JFR is not enabled, shouldCommit returns false and the JIT should be able to remove the allocation. This will be best from a performance point of view, and also in my opinion from a maintenance and readability perspective. Others may disagree. Erik >> >> Is there a need to support both logging and JFR? I'm new to record >> events. I did not get the point to use both. > I was initially hoping to concentrate on just JFR events but I got > strong feedback to also consider use of Logger (esp. in cases where > the jdk.jfr module is not available) > > regards, > Sean. > >> >> Thanks, >> Xuelei >> >> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>> Erik, >>> >>> I rebased the patch with TLS v1.3 integration today. I hadn't >>> realized how much the handshaker code had changed. Hopefully, I'll >>> get a review from security dev team on that front. >>> >>> Regards the JFR semantics, I believe the edits should match majority >>> of requests . I still have a preference for the test infra design >>> for these new logger/JFR tests used in version 1 of webrev. I think >>> it makes sense to keep the test functionality together - no sense in >>> separating them out into different components IMO. Also, some of the >>> edits to the JFR testing were made to test for the new dual >>> log/record functionality. I might catch up with you tomorrow to see >>> what the best arrangement would be. >>> >>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>> >>> regards, >>> Sean. >>> >>> >>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>> Many thanks for the review comments Erik. Replies inline. >>>> >>>> >>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>> Hi Sean, >>>>> >>>>> Some of the changes in the webrev belongs to JDK-8203629 and >>>>> should be removed for clarity. >>>>> >>>>> Some initial comments: >>>>> >>>>> default.jfc, profile.jfr: >>>>> The events should not have control="enable-exceptions". The >>>>> purpose of the control attribute is so to provide parameterized >>>>> configuration of events for JMC. As it is now, the security >>>>> events will be enabled when a user turns on the exception events. >>>> Makes sense. I'll remove that parameter. >>>>> >>>>> X509CertEvent: >>>>> You should use milliseconds since epoch to represent a date >>>>> instead of a string value, i.e. >>>>> >>>>> @Label("Valid From") >>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>> public long validFrom; >>>>> >>>>> @Label("Valid Until") >>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>> public long validUntil; >>>>> >>>> The CertificateValidity class operates on Date Object values. I'll >>>> work with the Date.getTime() method then (and update the Logger >>>> formatting) >>>>> This following annotation adds little value >>>>> >>>>> @Description("Details of Security Property") >>>>> >>>>> I would either remove the annotation, or provide information that >>>>> helps a user understand the event. For instance, what is X509, and >>>>> what kind of certificates are we talking about? >>>> Yes - that looks like the wrong Description. I'll review all of >>>> these fields. >>>>> >>>>> @Category("Java Application") >>>>> >>>>> I'm a bit worried that we will pollute the "Java Application" >>>>> namespace if we add lots of JDK events in that category. We may >>>>> want to add a new top level category "Java Development Kit", >>>>> analogous to the "Java Virtual Machine" category, and put all >>>>> security related events in subcategory, perhaps called "Security". >>>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>>> >>>>> @Label("X509Cert") >>>>> >>>>> The label should be human readable name, with spaces, title cased >>>>> etc. I would recommend "X.509 Certificate". In general, avoid >>>>> abbreviations like "certs" and instead use labels such as >>>>> "Certificate Chain". The label should be short and not a full >>>>> sentence. >>>>> >>>>> For details see, >>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>> >>>>> I think it would be good to separate testing of JFR and logging >>>>> into different files / tests. I would prefer that the test name is >>>>> the same as the event prefixed with "Test", i.e >>>>> TestX509CertificateEvent, as this is the pattern used by other JFR >>>>> tests. >>>>> >>>> I'll take a look at that pattern again. I've separated out the >>>> current tests into an (a) outer test to analyze the logger output >>>> and (b) the inner test which checks for JFR correctness. I did >>>> include extra logic to make sure that the EventHelper configuration >>>> was working correctly. "Events.assertField" looks very helpful. >>>> Thanks for the pointer. >>>> >>>> Let me take on board the suggestions below and get a new webrev out >>>> on Tuesday. >>>> >>>> regards, >>>> Sean. >>>> >>>>> I reworked one of the tests to how I like to see it: >>>>> >>>>> /* >>>>> * @test >>>>> * @key jfr >>>>> * @library /test/lib >>>>> * @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>>> */ >>>>> public class TestX509CertificateEvent { >>>>> >>>>> private static final String CERTIFICATE_1 = "..."; >>>>> private static final String CERTIFICATE_2 = "..."; >>>>> >>>>> public static void main(String... args) throws >>>>> CertificateException { >>>>> >>>>> Recording r = new Recording(); >>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>> r.start(); >>>>> >>>>> CertificateFactory cf = >>>>> CertificateFactory.getInstance("X.509"); >>>>> cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>> cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>> >>>>> // Make sure only one event per certificate >>>>> cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>> cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>> >>>>> r.stop(); >>>>> >>>>> List events = Events.fromRecording(r); >>>>> Asserts.assertEquals(events.size(), 2, "Expected two >>>>> X.509 Certificate events"); >>>>> >>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>> "CN=SSLCertificate, O=SomeCompany", >>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>> "RSA", 2048); >>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>> "CN=SSLCertificate, O=SomeCompany", >>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>> "RSA", 2048); >>>>> } >>>>> >>>>> private static void assertEvent(List events, >>>>> String certID, String algId, String subject, >>>>> String issuer, String keyType, int length) throws >>>>> Exception { >>>>> >>>>> for (RecordedEvent e : events) { >>>>> if (e.getString("serialNumber").equals(certID)) { >>>>> Events.assertField(e, "algId").equal(algId); >>>>> ... >>>>> return; >>>>> } >>>>> } >>>>> System.out.println(events); >>>>> throw new Exception("Could not find event with Cert ID: " >>>>> + certID); >>>>> } >>>>> } >>>>> >>>>> The reworked example uses the Events.assertField method, which >>>>> will give context if the assertion fails. Keeping the test simple, >>>>> means it can be analyzed quickly if it fails in testing. There is >>>>> no new test framework to learn, or methods to search for, and it >>>>> is usually not hard to determine if the failure is product, test >>>>> or infrastructure related, and what component (team) should be >>>>> assigned the issue. The use of EventNames.X509Certificate means >>>>> all occurrences of the event can be tracked done in an IDE using >>>>> find by reference. >>>>> >>>>> Thanks >>>>> Erik >>>>> >>>>>> Following on from the recent JDK-8203629 code review, I'd like to >>>>>> propose enhancements on how we can record events in security >>>>>> libs. The introduction of the JFR libraries can give us much >>>>>> better ways of examining JDK actions. For the initial phase, I'm >>>>>> looking to enhance some key security library events in JDK 11 so >>>>>> that they can be either recorded to JFR, logged to a traditional >>>>>> logger, or both. >>>>>> >>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>> >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>> >>>>>> securityProp_2.png gives an example of how the JFR recording can >>>>>> be queried to quickly locate events of interest (in this case, >>>>>> code setting the jdk.tls.* Security properties). I still need to >>>>>> clean up the TLSEvents testcase to improve test coverage and hope >>>>>> to do that in coming days. >>>>>> >>>>>> JBS record : >>>>>> * https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>> >>>>>> webrev : >>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>> >>>>> >>>> >>> > From xuelei.fan at oracle.com Wed Jun 27 20:11:14 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 27 Jun 2018 13:11:14 -0700 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> Message-ID: Finished.java ------------- In each handshake, Finished messages are delivered twice. One from client, and the other one from the server side. Catch Finished message and use the final one of them should be sufficient. In the Finished.java implementation, the signal of the final Finished message is the handshakeFinished field is set to true. Please move line 582: 582 recordEvent(chc.conContext.conSession); to line 558: 556 // handshake context cleanup. 557 chc.handshakeFinished = true; 558 Please move line 632: 632 recordEvent(shc.conContext.conSession); to line 608: 606 // handshake context cleanup. 607 shc.handshakeFinished = true; 608 Please remove line 838: 838 recordEvent(shc.conContext.conSession); Please remove line 984: 984 recordEvent(chc.conContext.conSession); No more comment. Thanks, Xuelei On 6/27/2018 11:57 AM, Xuelei Fan wrote: > Hi Sean, > > I may reply in several replies. > > PKIXMasterCertPathValidator.java > -------------------------------- > +? CertChainEvent cce = new CertChainEvent(); > +? if(cce.isEnabled() || EventHelper.loggingSecurity()) { > +????? String c = reversedCertList.stream() > +????????????????? .map(x -> x.getSerialNumber().toString(16)) > +??????????????????????? .collect(Collectors.joining(", ")); > +???? EventHelper.commitCertChainEvent(cce, c); > +?? } > > No matter the event or the JFR mechanism is enabled or not, the above > code will create a new instance.? Is the return value of cce.isEnabled() > dynamically changed or static? > > Is there a need to support both logging and JFR?? I'm new to record > events.? I did not get the point to use both. > > Thanks, > Xuelei > > On 6/26/2018 3:18 PM, Se?n Coffey wrote: >> Erik, >> >> I rebased the patch with TLS v1.3 integration today. I hadn't realized >> how much the handshaker code had changed. Hopefully, I'll get a review >> from security dev team on that front. >> >> Regards the JFR semantics, I believe the edits should match majority >> of requests . I still have a preference for the test infra design for >> these new logger/JFR tests used in version 1 of webrev. I think it >> makes sense to keep the test functionality together - no sense in >> separating them out into different components IMO. Also, some of the >> edits to the JFR testing were made to test for the new dual log/record >> functionality. I might catch up with you tomorrow to see what the best >> arrangement would be. >> >> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >> >> regards, >> Sean. >> >> >> On 25/06/2018 21:22, Se?n Coffey wrote: >>> Many thanks for the review comments Erik. Replies inline. >>> >>> >>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>> Hi Sean, >>>> >>>> Some of the changes in the webrev belongs to JDK-8203629 and should >>>> be removed for clarity. >>>> >>>> Some initial comments: >>>> >>>> default.jfc, profile.jfr: >>>> The events should not have control="enable-exceptions". The purpose >>>> of the control attribute is so to provide parameterized >>>> configuration of events for JMC.? As it is now, the security events >>>> will be enabled when a user turns on the exception events. >>> Makes sense. I'll remove that parameter. >>>> >>>> X509CertEvent: >>>> You should use milliseconds since epoch to represent a date instead >>>> of a string value, i.e. >>>> >>>> ??? @Label("Valid From") >>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>> ??? public long validFrom; >>>> >>>> ??? @Label("Valid Until") >>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>> ??? public long validUntil; >>>> >>> The CertificateValidity class operates on Date Object values. I'll >>> work with the Date.getTime() method then (and update the Logger >>> formatting) >>>> This following annotation adds little value >>>> >>>> @Description("Details of Security Property") >>>> >>>> I would either remove the annotation, or provide information that >>>> helps a user understand the event. For instance, what is X509, and >>>> what kind of certificates are we talking about? >>> Yes - that looks like the wrong Description. I'll review all of these >>> fields. >>>> >>>> @Category("Java Application") >>>> >>>> I'm a bit worried that we will pollute the "Java Application" >>>> namespace if we add lots of JDK events in that category. We may want >>>> to add a new top level category "Java Development Kit", analogous to >>>> the "Java Virtual Machine" category, and put all security related >>>> events in subcategory, perhaps called "Security". >>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>> >>>> @Label("X509Cert") >>>> >>>> The label should be human readable name, with spaces, title cased >>>> etc. I would recommend "X.509 Certificate". In general, avoid >>>> abbreviations like "certs" and instead use labels such as >>>> "Certificate Chain". The label should be short and not a full sentence. >>>> >>>> For details see, >>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>> >>>> I think it would be good to separate testing of JFR and logging into >>>> different files / tests. I would prefer that the test name is the >>>> same as the event prefixed with "Test", i.e >>>> TestX509CertificateEvent, as this is the pattern used by other JFR >>>> tests. >>>> >>> I'll take a look at that pattern again. I've separated out the >>> current tests into an (a) outer test to analyze the logger output and >>> (b) the inner test which checks for JFR correctness. I did include >>> extra logic to make sure that the EventHelper configuration was >>> working correctly. "Events.assertField" looks very helpful. Thanks >>> for the pointer. >>> >>> Let me take on board the suggestions below and get a new webrev out >>> on Tuesday. >>> >>> regards, >>> Sean. >>> >>>> I reworked one of the tests to how I like to see it: >>>> >>>> /* >>>> ?* @test >>>> ?* @key jfr >>>> ?* @library /test/lib >>>> ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>> ?*/ >>>> public class TestX509CertificateEvent { >>>> >>>> ??? private static final String CERTIFICATE_1 = "..."; >>>> ??? private static final String CERTIFICATE_2 = "..."; >>>> >>>> ??? public static void main(String... args) throws >>>> CertificateException { >>>> >>>> ???????? Recording r = new Recording(); >>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>> ???????? r.start(); >>>> >>>> ???????? CertificateFactory cf = >>>> CertificateFactory.getInstance("X.509"); >>>> ???????? cf.generateCertificate(new >>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>> ???????? cf.generateCertificate(new >>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>> >>>> ???????? // Make sure only one event per certificate >>>> ???????? cf.generateCertificate(new >>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>> ???????? cf.generateCertificate(new >>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>> >>>> ???????? r.stop(); >>>> >>>> ???????? List events = Events.fromRecording(r); >>>> ???????? Asserts.assertEquals(events.size(), 2, "Expected two X.509 >>>> Certificate events"); >>>> >>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>> ???????????????????? "RSA", 2048); >>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>> ???????????????????? "RSA", 2048); >>>> ??? } >>>> >>>> ??? private static void assertEvent(List events, >>>> String certID, String algId, String subject, >>>> ??????????? String issuer, String keyType, int length) throws >>>> Exception { >>>> >>>> ??????? for (RecordedEvent e : events) { >>>> ??????????? if (e.getString("serialNumber").equals(certID)) { >>>> ??????????????? Events.assertField(e, "algId").equal(algId); >>>> ??????????????? ... >>>> ??????????????? return; >>>> ??????????? } >>>> ??????? } >>>> ??????? System.out.println(events); >>>> ??????? throw new Exception("Could not find event with Cert ID: " + >>>> certID); >>>> ??? } >>>> } >>>> >>>> The reworked example uses the Events.assertField method, which will >>>> give context if the assertion fails. Keeping the test simple, means >>>> it can be analyzed quickly if it fails in testing. There is no new >>>> test framework to learn, or methods to search for, and it is usually >>>> not hard to determine if the failure is product, test or >>>> infrastructure related, and what component (team) should be assigned >>>> the issue. The use of EventNames.X509Certificate means all >>>> occurrences of the event can be tracked done in an IDE using find by >>>> reference. >>>> >>>> Thanks >>>> Erik >>>> >>>>> Following on from the recent JDK-8203629 code review, I'd like to >>>>> propose enhancements on how we can record events in security libs. >>>>> The introduction of the JFR libraries can give us much better ways >>>>> of examining JDK actions. For the initial phase, I'm looking to >>>>> enhance some key security library events in JDK 11 so that they can >>>>> be either recorded to JFR, logged to a traditional logger, or both. >>>>> >>>>> Examples of how useful JFR recordings could be can be seen here : >>>>> >>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>> >>>>> securityProp_2.png gives an example of how the JFR recording can be >>>>> queried to quickly locate events of interest (in this case, code >>>>> setting the jdk.tls.* Security properties). I still need to clean >>>>> up the TLSEvents testcase to improve test coverage and hope to do >>>>> that in coming days. >>>>> >>>>> JBS record : >>>>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>> >>>>> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>> >>>> >>> >> From jonathan.gibbons at oracle.com Wed Jun 27 20:37:26 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 27 Jun 2018 13:37:26 -0700 Subject: RFR: 8205438: Re-enable shebang tests in test/jdk/tools/launchers/SourceMode.java Message-ID: <5B33F586.6080902@oracle.com> Please review a test fix to re-enable shebang tests in test/jdk/tools/launchers/SourceMode.java . The test cases for invoking the source launcher via the shebang mechanism have been disabled because they were adversely affected by the very long pathnames on our internal test infastructure, causing the test cases to overflow the system limit of 128 characters for the first line of a shebang file The fix is to run jlink to create a small image that can be addressed by a short relative path. Some of the test cases on macOS and Solaris remain explicitly disabled because of the inconsistencies in the system-level support for shebang execution. Finally, the use of hard-coded version numbers in the test is changed to avoid having to add this test to the list of tests that need to be modified at the beginning of each release. -- Jon JBS: https://bugs.openjdk.java.net/browse/JDK-8205438 Webrev: http://cr.openjdk.java.net/~jjg/8205438/webrev.00/ From mandy.chung at oracle.com Wed Jun 27 21:02:15 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 27 Jun 2018 14:02:15 -0700 Subject: RFR: 8205438: Re-enable shebang tests in test/jdk/tools/launchers/SourceMode.java In-Reply-To: <5B33F586.6080902@oracle.com> References: <5B33F586.6080902@oracle.com> Message-ID: <326dbf8d-450c-10d5-630b-33e2c4b3430b@oracle.com> Looks good. It's amazing to find out 3 different behavior on these 3 platforms. Mandy On 6/27/18 1:37 PM, Jonathan Gibbons wrote: > Please review a test fix to re-enable shebang tests in > test/jdk/tools/launchers/SourceMode.java . > > The test cases for invoking the source launcher via the shebang > mechanism have been disabled because they were adversely affected by the > very long pathnames on our internal test infastructure, causing the test > cases to overflow the system limit of 128 characters for the first line > of a shebang file > > The fix is to run jlink to create a small image that can be addressed by > a short relative path. > > Some of the test cases on macOS and Solaris remain explicitly disabled > because of the inconsistencies in the system-level support for shebang > execution. > > Finally, the use of hard-coded version numbers in the test is changed to > avoid having to add this test to the list of tests that need to be > modified at the beginning of each release. > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8205438 > Webrev: http://cr.openjdk.java.net/~jjg/8205438/webrev.00/ From james.laskey at oracle.com Wed Jun 27 21:15:40 2018 From: james.laskey at oracle.com (James Laskey) Date: Wed, 27 Jun 2018 18:15:40 -0300 Subject: RFR: 8205438: Re-enable shebang tests in test/jdk/tools/launchers/SourceMode.java In-Reply-To: <326dbf8d-450c-10d5-630b-33e2c4b3430b@oracle.com> References: <5B33F586.6080902@oracle.com> <326dbf8d-450c-10d5-630b-33e2c4b3430b@oracle.com> Message-ID: We went through the same exercise with jjs. Tough coming up with shebang tests that worked on all platforms. Shebang args vs command line args was a nightmare. Sent from my iPhone > On Jun 27, 2018, at 6:02 PM, mandy chung wrote: > > Looks good. It's amazing to find out 3 different behavior on these 3 platforms. > > Mandy > >> On 6/27/18 1:37 PM, Jonathan Gibbons wrote: >> Please review a test fix to re-enable shebang tests in test/jdk/tools/launchers/SourceMode.java . >> The test cases for invoking the source launcher via the shebang mechanism have been disabled because they were adversely affected by the very long pathnames on our internal test infastructure, causing the test cases to overflow the system limit of 128 characters for the first line of a shebang file >> The fix is to run jlink to create a small image that can be addressed by a short relative path. >> Some of the test cases on macOS and Solaris remain explicitly disabled because of the inconsistencies in the system-level support for shebang execution. >> Finally, the use of hard-coded version numbers in the test is changed to avoid having to add this test to the list of tests that need to be modified at the beginning of each release. >> -- Jon >> JBS: https://bugs.openjdk.java.net/browse/JDK-8205438 >> Webrev: http://cr.openjdk.java.net/~jjg/8205438/webrev.00/ > > From kevin.rushforth at oracle.com Wed Jun 27 22:30:32 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Wed, 27 Jun 2018 15:30:32 -0700 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> Message-ID: We're aiming to get this into JDK 12 early enough so that an EA build would be available around the time JDK 11 ships. That will allow you to take a jlinked image with JDK 11 and package it up using (the EA) jpackager. We will create a development branch in the JDK sandbox [1] some time in the next week or so so you can follow the development. Also, thank you to those who have provided feedback. I'll reply to feedback soon and then incorporate it into an updated JEP. -- Kevin On 6/27/2018 3:09 AM, Buchberger, Joerg wrote: > Thanks for the info! And thanks for the efforts. [no irony, no sarcasm - I really mean it] > > But, to sum up my comprehension... > > anyone who placed their bets on javapackager, starting with last LTS Java 8 > will be left in the rain with followup LTS Java 11, because their ain't neither javapackager (anymore), nor jpackager (yet). > > Is this correct? > > So, strategic choice boils down to either throw away all work done based on javapackager so far and the associated distribution concepts (reworking everything from scratch) > or neglect Java 11 completely, thus placing all bets on jpackager really coming w/ Java 12 or even waiting for Java 14 as next LTS thereafter. > > Bam(!), I think, I first need a tiny shot now ;-) and let that info sink in ... > > Cheers > J?rg > > > On 5/31/2018 0:10 AM, Rushforth, Kevin wrote: >> I would like to propose the following Draft JEP [1] for discussion. >> >> JDK-8200758: Packaging Tool >> >> This is intended as a JDK-scope replacement for the existing >> javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK >> releases), and was delivered as part of the JavaFX build. The >> javapackager tool has been removed from Oracle JDK 11 along with the >> removal of JavaFX. >> >> Comments on this JEP are welcome. It is currently not targeted for a >> specific release, but we are aiming for JDK 12. >> >> -- Kevin >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >> From xu.y.yin at oracle.com Thu Jun 28 02:01:49 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Thu, 28 Jun 2018 10:01:49 +0800 Subject: [JDK 11] RFR 8187069: The case auto failed with the "java.lang.ClassNotFoundException: IPv6NameserverPlatformParsingTest" exception Message-ID: <836D8991-E27C-4929-96FC-19444A48E422@oracle.com> Please review below one line change for manual test case com/sun/jndi/dns/Test6991580.java to build test class automatically which will be used in manual steps, thanks bug: https://bugs.openjdk.java.net/browse/JDK-8187069 Review change as below: diff -r 2d3e99a72541 test/jdk/com/sun/jndi/dns/Test6991580.java --- a/test/jdk/com/sun/jndi/dns/Test6991580.java Wed Jun 27 17:02:41 2018 -0700 +++ b/test/jdk/com/sun/jndi/dns/Test6991580.java Thu Jun 28 09:50:37 2018 +0800 @@ -37,6 +37,7 @@ * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException * @modules java.desktop * jdk.naming.dns/com.sun.jndi.dns + * @build IPv6NameserverPlatformParsingTest * @run main/manual Test6991580 */ Regards, Chris From isaac.r.levy at gmail.com Thu Jun 28 03:57:18 2018 From: isaac.r.levy at gmail.com (Isaac Levy) Date: Wed, 27 Jun 2018 23:57:18 -0400 Subject: [PATCH] AbstractStringBuilder.append() Message-ID: AbstractStringBuilder already has append(). This patch adds append(). The new method improves parity between the two classes. In addition, combining StringBuilders is presumably common. append() has a couple insteadof checks, which this new method skips. -Isaac diff --git a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java index 2ef3e53256..1fe89bb92a 100644 --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java @@ -543,6 +543,11 @@ abstract class AbstractStringBuilder implements Appendable, CharSequence { return this.append((AbstractStringBuilder)sb); } + // Documentation in subclasses because of synchro difference + public AbstractStringBuilder append(StringBuilder sb) { + return this.append((AbstractStringBuilder)sb); + } + /** * @since 1.8 */ diff --git a/src/java.base/share/classes/java/lang/StringBuffer.java b/src/java.base/share/classes/java/lang/StringBuffer.java index e597a8112e..613ba90c25 100644 --- a/src/java.base/share/classes/java/lang/StringBuffer.java +++ b/src/java.base/share/classes/java/lang/StringBuffer.java @@ -313,6 +313,33 @@ import jdk.internal.HotSpotIntrinsicCandidate; return this; } + /** + * Appends the specified {@code StringBuilder} to this sequence. + *

        + * The characters of the {@code StringBuilder} argument are appended, + * in order, to the contents of this {@code StringBuffer}, increasing the + * length of this {@code StringBuffer} by the length of the argument. + * If {@code sb} is {@code null}, then the four characters + * {@code "null"} are appended to this {@code StringBuffer}. + *

        + * Let n be the length of the old character sequence, the one + * contained in the {@code StringBuffer} just prior to execution of the + * {@code append} method. Then the character at index k in + * the new character sequence is equal to the character at index k + * in the old character sequence, if k is less than n; + * otherwise, it is equal to the character at index k-n in the + * argument {@code sb}. + *

        + * + * @param sb the {@code StringBuilder} to append. + * @return a reference to this object. + */ + public synchronized StringBuffer append(StringBuilder sb) { + toStringCache = null; + super.append(sb); + return this; + } + /** * Appends the specified {@code StringBuffer} to this sequence. *

        diff --git a/src/java.base/share/classes/java/lang/StringBuilder.java b/src/java.base/share/classes/java/lang/StringBuilder.java index 40da2083c2..5ddd4fb5f9 100644 --- a/src/java.base/share/classes/java/lang/StringBuilder.java +++ b/src/java.base/share/classes/java/lang/StringBuilder.java @@ -199,6 +199,30 @@ public final class StringBuilder return this; } + /** + * Appends the specified {@code StringBuilder} to this sequence. + *

        + * The characters of the {@code StringBuilder} argument are appended, + * in order, to this sequence, increasing the + * length of this sequence by the length of the argument. + * If {@code sb} is {@code null}, then the four characters + * {@code "null"} are appended to this sequence. + *

        + * Let n be the length of this character sequence just prior to + * execution of the {@code append} method. Then the character at index + * k in the new character sequence is equal to the character at + * index k in the old character sequence, if k is less than + * n; otherwise, it is equal to the character at index k-n + * in the argument {@code sb}. + * + * @param sb the {@code StringBuilder} to append. + * @return a reference to this object. + */ + public StringBuilder append(StringBuilder sb) { + super.append(sb); + return this; + } + @Override public StringBuilder append(CharSequence s) { super.append(s); From martinrb at google.com Thu Jun 28 05:22:18 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 27 Jun 2018 22:22:18 -0700 Subject: [PATCH] AbstractStringBuilder.append() In-Reply-To: References: Message-ID: I'm fairly sure the append(StringBuilder) overloads were left out intentionally. On Wed, Jun 27, 2018 at 8:57 PM, Isaac Levy wrote: > AbstractStringBuilder already has append(). This patch > adds append(). > > The new method improves parity between the two classes. In addition, > combining StringBuilders is presumably common. append() > has a couple insteadof checks, which this new method skips. > > -Isaac > > > > > diff --git a/src/java.base/share/classes/java/lang/ > AbstractStringBuilder.java > b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > index 2ef3e53256..1fe89bb92a 100644 > --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > @@ -543,6 +543,11 @@ abstract class AbstractStringBuilder implements > Appendable, CharSequence { > return this.append((AbstractStringBuilder)sb); > } > > + // Documentation in subclasses because of synchro difference > + public AbstractStringBuilder append(StringBuilder sb) { > + return this.append((AbstractStringBuilder)sb); > + } > + > /** > * @since 1.8 > */ > diff --git a/src/java.base/share/classes/java/lang/StringBuffer.java > b/src/java.base/share/classes/java/lang/StringBuffer.java > index e597a8112e..613ba90c25 100644 > --- a/src/java.base/share/classes/java/lang/StringBuffer.java > +++ b/src/java.base/share/classes/java/lang/StringBuffer.java > @@ -313,6 +313,33 @@ import jdk.internal.HotSpotIntrinsicCandidate; > return this; > } > > + /** > + * Appends the specified {@code StringBuilder} to this sequence. > + *

        > + * The characters of the {@code StringBuilder} argument are appended, > + * in order, to the contents of this {@code StringBuffer}, increasing > the > + * length of this {@code StringBuffer} by the length of the argument. > + * If {@code sb} is {@code null}, then the four characters > + * {@code "null"} are appended to this {@code StringBuffer}. > + *

        > + * Let n be the length of the old character sequence, the one > + * contained in the {@code StringBuffer} just prior to execution of > the > + * {@code append} method. Then the character at index k in > + * the new character sequence is equal to the character at index > k > + * in the old character sequence, if k is less than n; > + * otherwise, it is equal to the character at index k-n in the > + * argument {@code sb}. > + *

        > + * > + * @param sb the {@code StringBuilder} to append. > + * @return a reference to this object. > + */ > + public synchronized StringBuffer append(StringBuilder sb) { > + toStringCache = null; > + super.append(sb); > + return this; > + } > + > /** > * Appends the specified {@code StringBuffer} to this sequence. > *

        > diff --git a/src/java.base/share/classes/java/lang/StringBuilder.java > b/src/java.base/share/classes/java/lang/StringBuilder.java > index 40da2083c2..5ddd4fb5f9 100644 > --- a/src/java.base/share/classes/java/lang/StringBuilder.java > +++ b/src/java.base/share/classes/java/lang/StringBuilder.java > @@ -199,6 +199,30 @@ public final class StringBuilder > return this; > } > > + /** > + * Appends the specified {@code StringBuilder} to this sequence. > + *

        > + * The characters of the {@code StringBuilder} argument are appended, > + * in order, to this sequence, increasing the > + * length of this sequence by the length of the argument. > + * If {@code sb} is {@code null}, then the four characters > + * {@code "null"} are appended to this sequence. > + *

        > + * Let n be the length of this character sequence just prior to > + * execution of the {@code append} method. Then the character at index > + * k in the new character sequence is equal to the character at > + * index k in the old character sequence, if k is less > than > + * n; otherwise, it is equal to the character at index > k-n > + * in the argument {@code sb}. > + * > + * @param sb the {@code StringBuilder} to append. > + * @return a reference to this object. > + */ > + public StringBuilder append(StringBuilder sb) { > + super.append(sb); > + return this; > + } > + > @Override > public StringBuilder append(CharSequence s) { > super.append(s); > From forax at univ-mlv.fr Thu Jun 28 06:06:44 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 28 Jun 2018 08:06:44 +0200 (CEST) Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> Message-ID: <1535277973.1155383.1530166004501.JavaMail.zimbra@u-pem.fr> Hi Kevin, Just a small request, can you call it something like jbundler and not jpackager, because usually in build tools the packager step is the step that create the jars, not the one that bundle the whole application ? regards, R?mi ----- Mail original ----- > De: "Kevin Rushforth" > ?: "core-libs-dev" > Envoy?: Jeudi 31 Mai 2018 02:10:48 > Objet: Draft JEP proposal: JDK-8200758: Packaging Tool > I would like to propose the following Draft JEP [1] for discussion. > > JDK-8200758: Packaging Tool > > This is intended as a JDK-scope replacement for the existing > javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK > releases), and was delivered as part of the JavaFX build. The > javapackager tool has been removed from Oracle JDK 11 along with the > removal of JavaFX. > > Comments on this JEP are welcome. It is currently not targeted for a > specific release, but we are aiming for JDK 12. > > -- Kevin > > [1] https://bugs.openjdk.java.net/browse/JDK-8200758 From volker.simonis at gmail.com Thu Jun 28 06:52:57 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 28 Jun 2018 08:52:57 +0200 Subject: RFR(XXS): 8205916: [test] Fix jdk/tools/launcher/RunpathTest to handle both, RPATH and RUNPATH In-Reply-To: References: Message-ID: Hi Martin, Erik, thanks for the reviews! Regards, Volker On Wed, Jun 27, 2018 at 5:53 PM, Erik Joelsson wrote: > Looks ok to me. > > /Erik > > > > On 2018-06-27 03:26, Volker Simonis wrote: >> >> Hi, >> >> can I please have a review for the following tiny test fix (I'm >> actually not sure which would be the appropriate mailing list for this >> fix so please redirect if necessary): >> >> http://cr.openjdk.java.net/~simonis/webrevs/2018/8205916/ >> https://bugs.openjdk.java.net/browse/JDK-8205916 >> >> The test currently only checks for RPATH in the dynamic section of the >> launcher, but some linkers / Linux distributions (notably SLES) use >> RUNPATH instead. >> >> Following are the gory details: >> >> The test jdk/tools/launcher/RunpathTest.java checks that the java >> launcher on Linux and Solaris has the correct RPATH path baked into >> the executable. >> >> Unfortunately, the situation with RPATH is a little weird: >> >> - in order to bake a runtime path into a dynamically linked library >> or executable one has to use the "-rpath " linker option (from >> the C/C++ compiler this is usually available as "-Wl,-rpath,"). >> - depending on the dynamic linker version and Linux distribution, >> the "-rpath" linker option adds either a "RPATH" entry (Ubuntu 16.04, >> RHEL 7.4) or a "RUNPATH" entry (SLES 12.1, SLES 15) or both entries >> together (SLES 11.3) to the dynamic section of the shared >> library/executable. >> - the semantics of "RPATH" and "RUNPATH" are slightly different: >> "RPATH" is evaluated at runtime before LD_LIBRARY_PATH (if "RUNPATH" >> isn't present) while "RUNPATH" is evaluated after LD_LIBRARY_PATH >> (i.e. RUNPATH can be overridden at runtime by setting >> LD_LIBRARY_PATH). >> - "RPATH" is considered obsolete and should be replaced by "RUNPATH" >> according to the man-page of "ld.so (8)". >> - the linker option "--enable-new-dtags"/"--disable-new-dtags" (or >> the corresponding compiler flags >> "-Wl,--enable-new-dtags"/"-Wl,--disable-new-dtags") can be used to >> enforce the generation of "RUNPATH"/"RPATH" respectively (except for >> systems like SLES 11.3 where "--enable-new-dtags" generates both >> "RPATH" and "RUNPATH" while "--disable-new-dtags" only generates >> "RPATH"). >> >> But this issue is not about fixing the build so to cut a long story >> short - the test RunpathTest.java should be able to handle both >> runtime path variants equally well. This can be easily achieved by >> extending the match pattern from ".*RPATH.*\\$ORIGIN/../lib.*" to >> ".*R(UN)?PATH.*\\$ORIGIN/../lib.*" >> >> Thank you and best regards, >> Volker > > From vivek.theeyarath at oracle.com Thu Jun 28 08:27:36 2018 From: vivek.theeyarath at oracle.com (Vivek Theeyarath) Date: Thu, 28 Jun 2018 01:27:36 -0700 (PDT) Subject: RFR: 8177275: IllegalArgumentException when MH would have too many parameters is not specified for several methods Message-ID: <23748ee1-7c78-46d2-bbda-a1a3f43d75a2@default> Hi All, Please review fix for https://bugs.openjdk.java.net/browse/JDK-8177275 . The jtreg test runs fine with the changes. http://cr.openjdk.java.net/~vtheeyarath/8177275/webrev.02/ CSR : https://bugs.openjdk.java.net/browse/JDK-8205917 Regards Vivek From Joerg.Buchberger at pruftechnik.com Thu Jun 28 09:03:42 2018 From: Joerg.Buchberger at pruftechnik.com (Buchberger, Joerg) Date: Thu, 28 Jun 2018 09:03:42 +0000 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> Message-ID: Hi Kevin Thanks for the helpful reply. If you don't mind, I add my feedback here below. My concern is about the options -BserviceHint and -singleton which make javapackager one of the best things since sliced bread. It seems like next to no one is really aware of these valuable features, no wonder since they are by and large undocumented. So, people out there keep fiddling around with all kinds of service wrapper tools and also with strange approaches to ensure an app is started only once, completely unaware of the fact, that JDK 10 supports this out of the box, by the tip of a finger so to say. According to JDK-8200758, for Windows only msi is required deployment objective. Others only optional. Note in this regard that in contrary to exe (innosetup) deployment, msi installer lacks icon support for the installer itself and resulting msi installer is around 35% bigger in size. Documentation is also of concern. Luckily, the source is open to read, so one can for example find out, how to enable and read debug output for WinLauncherSvc using DebugView and that "-BserviceHint" option exists. Yippie ;-) Cheers J?rg -----Original Message----- From: Kevin Rushforth [mailto:kevin.rushforth at oracle.com] Sent: Donnerstag, 28. Juni 2018 00:31 To: Buchberger, Joerg ; core-libs-dev at openjdk.java.net Subject: Re: Draft JEP proposal: JDK-8200758: Packaging Tool We're aiming to get this into JDK 12 early enough so that an EA build would be available around the time JDK 11 ships. That will allow you to take a jlinked image with JDK 11 and package it up using (the EA) jpackager. We will create a development branch in the JDK sandbox [1] some time in the next week or so so you can follow the development. Also, thank you to those who have provided feedback. I'll reply to feedback soon and then incorporate it into an updated JEP. -- Kevin On 6/27/2018 3:09 AM, Buchberger, Joerg wrote: > Thanks for the info! And thanks for the efforts. [no irony, no > sarcasm - I really mean it] > > But, to sum up my comprehension... > > anyone who placed their bets on javapackager, starting with last LTS > Java 8 will be left in the rain with followup LTS Java 11, because their ain't neither javapackager (anymore), nor jpackager (yet). > > Is this correct? > > So, strategic choice boils down to either throw away all work done > based on javapackager so far and the associated distribution concepts (reworking everything from scratch) or neglect Java 11 completely, thus placing all bets on jpackager really coming w/ Java 12 or even waiting for Java 14 as next LTS thereafter. > > Bam(!), I think, I first need a tiny shot now ;-) and let that info sink in ... > > Cheers > J?rg > > > On 5/31/2018 0:10 AM, Rushforth, Kevin wrote: >> I would like to propose the following Draft JEP [1] for discussion. >> >> JDK-8200758: Packaging Tool >> >> This is intended as a JDK-scope replacement for the existing >> javapackager tool that ships with Oracle JDK 10 (and earlier Oracle >> JDK releases), and was delivered as part of the JavaFX build. The >> javapackager tool has been removed from Oracle JDK 11 along with the >> removal of JavaFX. >> >> Comments on this JEP are welcome. It is currently not targeted for a >> specific release, but we are aiming for JDK 12. >> >> -- Kevin >> >> [1] >> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.jav >> a.net_browse_JDK-2D8200758&d=DwIDaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFm >> xRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=k3PBuMbi >> PBU9Ni8nXxqYD_VD9uEULpswQedWmbRiF-4&s=vBSYLYnENsNahwwRNz0r-gPNrs90xST >> Ebm0wFA2iPWs&e= >> From Pengfei.Li at arm.com Thu Jun 28 10:01:08 2018 From: Pengfei.Li at arm.com (Pengfei Li) Date: Thu, 28 Jun 2018 10:01:08 +0000 Subject: RFR: 8202794: Native Unix code should use readdir rather than readdir_r Message-ID: Hi Last month, Bernard proposed a patch to fix the OpenJDK build issue with recent versions of glibc. See http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/052991.html But this fix requires to be tested on all POSIX systems before getting integrated. As recently, more and more guys are complaining about the OpenJDK build failure issue, shall we just add the following workaround to upstream before Bernard's real fix eventually merged? diff --git a/src/java.base/unix/native/libjava/TimeZone_md.c b/src/java.base/unix/native/libjava/TimeZone_md.c index f0bb362afc..e156381acd 100644 --- a/src/java.base/unix/native/libjava/TimeZone_md.c +++ b/src/java.base/unix/native/libjava/TimeZone_md.c @@ -146,7 +146,7 @@ findZoneinfoFile(char *buf, size_t size, const char *dir) (void) closedir(dirp); return NULL; } - +#pragma GCC diagnostic ignored "-Wdeprecated-declarations" while (readdir64_r(dirp, entry, &dp) == 0 && dp != NULL) { /* * Skip '.' and '..' (and possibly other .* files) diff --git a/src/java.base/unix/native/libjava/UnixFileSystem_md.c b/src/java.base/unix/native/libjava/UnixFileSystem_md.c index 315aa92b64..5b9554dd47 100644 --- a/src/java.base/unix/native/libjava/UnixFileSystem_md.c +++ b/src/java.base/unix/native/libjava/UnixFileSystem_md.c @@ -339,6 +339,7 @@ Java_java_io_UnixFileSystem_list(JNIEnv *env, jobject this, if (rv == NULL) goto error; /* Scan the directory */ +#pragma GCC diagnostic ignored "-Wdeprecated-declarations" while ((readdir64_r(dir, ptr, &result) == 0) && (result != NULL)) { jstring name; if (!strcmp(ptr->d_name, ".") || !strcmp(ptr->d_name, "..")) diff --git a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c index 53bb1c1009..72c38eb9a6 100644 --- a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c +++ b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c @@ -731,6 +731,7 @@ Java_sun_nio_fs_UnixNativeDispatcher_readdir(JNIEnv* env, jclass this, jlong val /* EINTR not listed as a possible error */ /* TDB: reentrant version probably not required here */ +#pragma GCC diagnostic ignored "-Wdeprecated-declarations" res = readdir64_r(dirp, ptr, &result); #ifdef _AIX diff --git a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c index 1adeaf7bb5..080cf2a89b 100644 --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c @@ -80,6 +80,7 @@ static struct dirent* read_dir(DIR* dirp, struct dirent* entry) { return dbuf; #else /* __linux__ || _ALLBSD_SOURCE */ struct dirent* p; +#pragma GCC diagnostic ignored "-Wdeprecated-declarations" if (readdir_r(dirp, entry, &p) == 0) { return p; } else { -- Thanks, Pengfei From vyom.tewari at oracle.com Thu Jun 28 11:02:01 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Thu, 28 Jun 2018 16:32:01 +0530 Subject: [JDK 11] RFR 8187069: The case auto failed with the "java.lang.ClassNotFoundException: IPv6NameserverPlatformParsingTest" exception In-Reply-To: <836D8991-E27C-4929-96FC-19444A48E422@oracle.com> References: <836D8991-E27C-4929-96FC-19444A48E422@oracle.com> Message-ID: <28421129-efb5-312e-4473-9c219f6d8b0d@oracle.com> Hi Chris, change looks good to me. My NetBeans always complains about tag order if it is not correct, as you? adding the new tag i will suggest you to please fix the tag order as well. /* ?* @test ?* @bug 6991580 8080108 8133035 ?* @summary IPv6 Nameservers in resolv.conf throws NumberFormatException ?* @modules java.desktop ?*????????? jdk.naming.dns/com.sun.jndi.dns ?* @requires os.family != "windows" ?* @build IPv6NameserverPlatformParsingTest ?* @run main/manual Test6991580 ?*/ Thanks, Vyom On Thursday 28 June 2018 07:31 AM, Chris Yin wrote: > Please review below one line change for manual test case > com/sun/jndi/dns/Test6991580.java to build test class automatically > which will be used in manual steps, thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8187069 > > Review change as below: > > diff -r 2d3e99a72541 test/jdk/com/sun/jndi/dns/Test6991580.java > --- a/test/jdk/com/sun/jndi/dns/Test6991580.javaWed Jun 27 17:02:41 > 2018 -0700 > +++ b/test/jdk/com/sun/jndi/dns/Test6991580.javaThu Jun 28 09:50:37 > 2018 +0800 > @@ -37,6 +37,7 @@ > ??* @summary IPv6 Nameservers in resolv.conf throws NumberFormatException > ??* @modules java.desktop > ??*? ? ? ? ??jdk.naming.dns/com.sun.jndi.dns > + * @build IPv6NameserverPlatformParsingTest > ??* @run main/manual Test6991580 > ? */ > > Regards, > Chris From Alan.Bateman at oracle.com Thu Jun 28 11:49:34 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 28 Jun 2018 12:49:34 +0100 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> Message-ID: <7a13e911-bdde-e710-0331-0892f2995c8e@oracle.com> On 20/06/2018 11:08, Pengfei Li wrote: > Hi > > I have tried the patch ( http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/052991.html ) on Ubuntu 18.04 machines (x86_64 & aarch64) with glibc=2.26.1 and build is OK. > > There's a small issue within the following code in src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c > Unlike readdir_r(), readdir() does not return int value. The value of res is always 0 before #ifdef. > > 727 /* EINTR not listed as a possible error */ > 728 errno = 0; > 729 ptr = readdir64(dirp); > 730 res = errno; > 731 > 732 #ifdef _AIX > 733 /* On AIX, readdir() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ > 734 /* directory stream end. Otherwise, 'errno' will contain the error code. */ > 735 if (res != 0) { > 736 res = (ptr == NULL && res == EBADF) ? 0 : errno; > 737 } > 738 #endif > > May I know that if this core-libs change going to be merged recently, or more platforms needs to be explored? > I see your other mail looking to to add #pragma GCC ... to avoid using configure --disable-warnings-as-errors. I think it would be better to just replace the usage readdir_r usage in that native method. I've tested the patch below on macOS, Linux and Solaris.? The SAP folks maintain the AIX port and would need to test it on that platform as it eliminates the AIX specific code for dealing with end of stream that should be needed when using readdir. -Alan diff -r 9d62da00bf15 -r 5e67e87bd6fa src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c --- a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Sat May 26 06:59:49 2018 +0200 +++ b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Mon Jun 25 14:17:03 2018 +0100 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2008, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2008, 2018, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -72,7 +72,7 @@ ?#define fstat64 fstat ?#define lstat64 lstat ?#define dirent64 dirent -#define readdir64_r readdir_r +#define readdir64 readdir ?#endif ?#include "jni.h" @@ -720,41 +720,23 @@ ?JNIEXPORT jbyteArray JNICALL ?Java_sun_nio_fs_UnixNativeDispatcher_readdir(JNIEnv* env, jclass this, jlong value) { -??? struct dirent64* result; -??? struct { -??????? struct dirent64 buf; -??????? char name_extra[PATH_MAX + 1 - sizeof result->d_name]; -??? } entry; -??? struct dirent64* ptr = &entry.buf; -??? int res; ???? DIR* dirp = jlong_to_ptr(value); +??? struct dirent64* ptr; -??? /* EINTR not listed as a possible error */ -??? /* TDB: reentrant version probably not required here */ -??? res = readdir64_r(dirp, ptr, &result); - -#ifdef _AIX -??? /* On AIX, readdir_r() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ -??? /* directory stream end. Otherwise, 'errno' will contain the error code. */ -??? if (res != 0) { -??????? res = (result == NULL && res == EBADF) ? 0 : errno; -??? } -#endif - -??? if (res != 0) { -??????? throwUnixException(env, res); +??? errno = 0; +??? ptr = readdir64(dirp); +??? if (ptr == NULL) { +??????? if (errno != 0) { +??????????? throwUnixException(env, errno); +??????? } ???????? return NULL; ???? } else { -??????? if (result == NULL) { -??????????? return NULL; -??????? } else { -??????????? jsize len = strlen(ptr->d_name); -??????????? jbyteArray bytes = (*env)->NewByteArray(env, len); -??????????? if (bytes != NULL) { -??????????????? (*env)->SetByteArrayRegion(env, bytes, 0, len, (jbyte*)(ptr->d_name)); -??????????? } -??????????? return bytes; +??????? jsize len = strlen(ptr->d_name); +??????? jbyteArray bytes = (*env)->NewByteArray(env, len); +??????? if (bytes != NULL) { +??????????? (*env)->SetByteArrayRegion(env, bytes, 0, len, (jbyte*)(ptr->d_name)); ???????? } +??????? return bytes; ???? } ?} From sean.coffey at oracle.com Thu Jun 28 12:28:07 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 28 Jun 2018 13:28:07 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> Message-ID: <25b3ae82-5c97-ff18-d0f9-4bfea00c29cb@oracle.com> Thanks for reviewing Xuelei, I do acknowledge that the new TLS v1.3 code has greatly improved the logging output for related operations. I think the main drive with this enhancement is to use the new JFR API to capture interesting events. We can revisit the Logger requirements if there's a strong opinion from users. By default, both the new Logger and JFR output will be switched off. I've just made an edit to the JFR jfc files to disable these events. "I may suggest remove this method and use Key.getAlgorithm() directly." - Yes, great idea. Done. Thanks for feedback on Finished.java edits. I did question to myself whether fewer edits were sufficient given the client/server nature of the handshake. I've refreshed the webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v3/webrev/ some jfr side edits also : * Change label from "X.509 Certificate" to "X509 Certificate" - JFR test fails with "." usage * Move the instance field variable name in CertChainEvent to "certChain" - JFR tests discourage use of? "ID" in "certIDs" regards, Sean. On 27/06/2018 21:11, Xuelei Fan wrote: > Finished.java > ------------- > In each handshake, Finished messages are delivered twice.? One from > client, and the other one from the server side.? Catch Finished > message and use the final one of them should be sufficient. > > In the Finished.java implementation, the signal of the final Finished > message is the handshakeFinished field is set to true. > > Please move line 582: > ?582???????????? recordEvent(chc.conContext.conSession); > to line 558: > ?556???????????????? // handshake context cleanup. > ?557???????????????? chc.handshakeFinished = true; > ?558 > > Please move line 632: > ?632???????????? recordEvent(shc.conContext.conSession); > to line 608: > ?606???????????????? // handshake context cleanup. > ?607???????????????? shc.handshakeFinished = true; > ?608 > > Please remove line 838: > ?838???????????? recordEvent(shc.conContext.conSession); > > Please remove line 984: > ?984???????????? recordEvent(chc.conContext.conSession); > > No more comment. > > Thanks, > Xuelei > > On 6/27/2018 11:57 AM, Xuelei Fan wrote: >> Hi Sean, >> >> I may reply in several replies. >> >> PKIXMasterCertPathValidator.java >> -------------------------------- >> +? CertChainEvent cce = new CertChainEvent(); >> +? if(cce.isEnabled() || EventHelper.loggingSecurity()) { >> +????? String c = reversedCertList.stream() >> +????????????????? .map(x -> x.getSerialNumber().toString(16)) >> +??????????????????????? .collect(Collectors.joining(", ")); >> +???? EventHelper.commitCertChainEvent(cce, c); >> +?? } >> >> No matter the event or the JFR mechanism is enabled or not, the above >> code will create a new instance.? Is the return value of >> cce.isEnabled() dynamically changed or static? >> >> Is there a need to support both logging and JFR?? I'm new to record >> events.? I did not get the point to use both. >> >> Thanks, >> Xuelei >> >> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>> Erik, >>> >>> I rebased the patch with TLS v1.3 integration today. I hadn't >>> realized how much the handshaker code had changed. Hopefully, I'll >>> get a review from security dev team on that front. >>> >>> Regards the JFR semantics, I believe the edits should match majority >>> of requests . I still have a preference for the test infra design >>> for these new logger/JFR tests used in version 1 of webrev. I think >>> it makes sense to keep the test functionality together - no sense in >>> separating them out into different components IMO. Also, some of the >>> edits to the JFR testing were made to test for the new dual >>> log/record functionality. I might catch up with you tomorrow to see >>> what the best arrangement would be. >>> >>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>> >>> regards, >>> Sean. >>> >>> >>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>> Many thanks for the review comments Erik. Replies inline. >>>> >>>> >>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>> Hi Sean, >>>>> >>>>> Some of the changes in the webrev belongs to JDK-8203629 and >>>>> should be removed for clarity. >>>>> >>>>> Some initial comments: >>>>> >>>>> default.jfc, profile.jfr: >>>>> The events should not have control="enable-exceptions". The >>>>> purpose of the control attribute is so to provide parameterized >>>>> configuration of events for JMC.? As it is now, the security >>>>> events will be enabled when a user turns on the exception events. >>>> Makes sense. I'll remove that parameter. >>>>> >>>>> X509CertEvent: >>>>> You should use milliseconds since epoch to represent a date >>>>> instead of a string value, i.e. >>>>> >>>>> ??? @Label("Valid From") >>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>> ??? public long validFrom; >>>>> >>>>> ??? @Label("Valid Until") >>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>> ??? public long validUntil; >>>>> >>>> The CertificateValidity class operates on Date Object values. I'll >>>> work with the Date.getTime() method then (and update the Logger >>>> formatting) >>>>> This following annotation adds little value >>>>> >>>>> @Description("Details of Security Property") >>>>> >>>>> I would either remove the annotation, or provide information that >>>>> helps a user understand the event. For instance, what is X509, and >>>>> what kind of certificates are we talking about? >>>> Yes - that looks like the wrong Description. I'll review all of >>>> these fields. >>>>> >>>>> @Category("Java Application") >>>>> >>>>> I'm a bit worried that we will pollute the "Java Application" >>>>> namespace if we add lots of JDK events in that category. We may >>>>> want to add a new top level category "Java Development Kit", >>>>> analogous to the "Java Virtual Machine" category, and put all >>>>> security related events in subcategory, perhaps called "Security". >>>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>>> >>>>> @Label("X509Cert") >>>>> >>>>> The label should be human readable name, with spaces, title cased >>>>> etc. I would recommend "X.509 Certificate". In general, avoid >>>>> abbreviations like "certs" and instead use labels such as >>>>> "Certificate Chain". The label should be short and not a full >>>>> sentence. >>>>> >>>>> For details see, >>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>> >>>>> I think it would be good to separate testing of JFR and logging >>>>> into different files / tests. I would prefer that the test name is >>>>> the same as the event prefixed with "Test", i.e >>>>> TestX509CertificateEvent, as this is the pattern used by other JFR >>>>> tests. >>>>> >>>> I'll take a look at that pattern again. I've separated out the >>>> current tests into an (a) outer test to analyze the logger output >>>> and (b) the inner test which checks for JFR correctness. I did >>>> include extra logic to make sure that the EventHelper configuration >>>> was working correctly. "Events.assertField" looks very helpful. >>>> Thanks for the pointer. >>>> >>>> Let me take on board the suggestions below and get a new webrev out >>>> on Tuesday. >>>> >>>> regards, >>>> Sean. >>>> >>>>> I reworked one of the tests to how I like to see it: >>>>> >>>>> /* >>>>> ?* @test >>>>> ?* @key jfr >>>>> ?* @library /test/lib >>>>> ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>>> ?*/ >>>>> public class TestX509CertificateEvent { >>>>> >>>>> ??? private static final String CERTIFICATE_1 = "..."; >>>>> ??? private static final String CERTIFICATE_2 = "..."; >>>>> >>>>> ??? public static void main(String... args) throws >>>>> CertificateException { >>>>> >>>>> ???????? Recording r = new Recording(); >>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>> ???????? r.start(); >>>>> >>>>> ???????? CertificateFactory cf = >>>>> CertificateFactory.getInstance("X.509"); >>>>> ???????? cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>> ???????? cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>> >>>>> ???????? // Make sure only one event per certificate >>>>> ???????? cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>> ???????? cf.generateCertificate(new >>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>> >>>>> ???????? r.stop(); >>>>> >>>>> ???????? List events = Events.fromRecording(r); >>>>> ???????? Asserts.assertEquals(events.size(), 2, "Expected two >>>>> X.509 Certificate events"); >>>>> >>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>> ???????????????????? "RSA", 2048); >>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>> ???????????????????? "RSA", 2048); >>>>> ??? } >>>>> >>>>> ??? private static void assertEvent(List events, >>>>> String certID, String algId, String subject, >>>>> ??????????? String issuer, String keyType, int length) throws >>>>> Exception { >>>>> >>>>> ??????? for (RecordedEvent e : events) { >>>>> ??????????? if (e.getString("serialNumber").equals(certID)) { >>>>> ??????????????? Events.assertField(e, "algId").equal(algId); >>>>> ??????????????? ... >>>>> ??????????????? return; >>>>> ??????????? } >>>>> ??????? } >>>>> ??????? System.out.println(events); >>>>> ??????? throw new Exception("Could not find event with Cert ID: " >>>>> + certID); >>>>> ??? } >>>>> } >>>>> >>>>> The reworked example uses the Events.assertField method, which >>>>> will give context if the assertion fails. Keeping the test simple, >>>>> means it can be analyzed quickly if it fails in testing. There is >>>>> no new test framework to learn, or methods to search for, and it >>>>> is usually not hard to determine if the failure is product, test >>>>> or infrastructure related, and what component (team) should be >>>>> assigned the issue. The use of EventNames.X509Certificate means >>>>> all occurrences of the event can be tracked done in an IDE using >>>>> find by reference. >>>>> >>>>> Thanks >>>>> Erik >>>>> >>>>>> Following on from the recent JDK-8203629 code review, I'd like to >>>>>> propose enhancements on how we can record events in security >>>>>> libs. The introduction of the JFR libraries can give us much >>>>>> better ways of examining JDK actions. For the initial phase, I'm >>>>>> looking to enhance some key security library events in JDK 11 so >>>>>> that they can be either recorded to JFR, logged to a traditional >>>>>> logger, or both. >>>>>> >>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>> >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>> >>>>>> securityProp_2.png gives an example of how the JFR recording can >>>>>> be queried to quickly locate events of interest (in this case, >>>>>> code setting the jdk.tls.* Security properties). I still need to >>>>>> clean up the TLSEvents testcase to improve test coverage and hope >>>>>> to do that in coming days. >>>>>> >>>>>> JBS record : >>>>>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>> >>>>>> webrev : >>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>> >>>>> >>>> >>> From swpalmer at gmail.com Thu Jun 28 12:32:13 2018 From: swpalmer at gmail.com (Scott Palmer) Date: Thu, 28 Jun 2018 08:32:13 -0400 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> Message-ID: <2D66E03A-8F66-4535-8578-940049D3985D@gmail.com> Doesn?t jlink require a *fully* modularized application? I.e. no non-module dependencies. The packaging tool should work with all runnable Java applications, not just fully modularized ones. Modularization seems to be a bit of an effort and is one of the main reasons my application(s) are still stuck on Java 8. Scott > On Jun 27, 2018, at 6:30 PM, Kevin Rushforth wrote: > > We're aiming to get this into JDK 12 early enough so that an EA build would be available around the time JDK 11 ships. That will allow you to take a jlinked image with JDK 11 and package it up using (the EA) jpackager. > > We will create a development branch in the JDK sandbox [1] some time in the next week or so so you can follow the development. > > Also, thank you to those who have provided feedback. I'll reply to feedback soon and then incorporate it into an updated JEP. > > -- Kevin > > > On 6/27/2018 3:09 AM, Buchberger, Joerg wrote: >> Thanks for the info! And thanks for the efforts. [no irony, no sarcasm - I really mean it] >> >> But, to sum up my comprehension... >> >> anyone who placed their bets on javapackager, starting with last LTS Java 8 >> will be left in the rain with followup LTS Java 11, because their ain't neither javapackager (anymore), nor jpackager (yet). >> >> Is this correct? >> >> So, strategic choice boils down to either throw away all work done based on javapackager so far and the associated distribution concepts (reworking everything from scratch) >> or neglect Java 11 completely, thus placing all bets on jpackager really coming w/ Java 12 or even waiting for Java 14 as next LTS thereafter. >> >> Bam(!), I think, I first need a tiny shot now ;-) and let that info sink in ... >> >> Cheers >> J?rg >> >> >> On 5/31/2018 0:10 AM, Rushforth, Kevin wrote: >>> I would like to propose the following Draft JEP [1] for discussion. >>> >>> JDK-8200758: Packaging Tool >>> >>> This is intended as a JDK-scope replacement for the existing >>> javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK >>> releases), and was delivered as part of the JavaFX build. The >>> javapackager tool has been removed from Oracle JDK 11 along with the >>> removal of JavaFX. >>> >>> Comments on this JEP are welcome. It is currently not targeted for a >>> specific release, but we are aiming for JDK 12. >>> >>> -- Kevin >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>> > From sean.coffey at oracle.com Thu Jun 28 12:32:16 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 28 Jun 2018 13:32:16 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <5B33EC28.6090604@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> Message-ID: Thanks for the update Erik. By default I'm proposing that the new JFR Events and Logger be disabled. As a result the event class shouldn't escape. If performance metrics highlight an issue, we should revisit. regards, Sean. On 27/06/2018 20:57, Erik Gahlin wrote: > On 2018-06-27 21:14, Se?n Coffey wrote: >> >> >> On 27/06/2018 19:57, Xuelei Fan wrote: >>> Hi Sean, >>> >>> I may reply in several replies. >>> >>> PKIXMasterCertPathValidator.java >>> -------------------------------- >>> +? CertChainEvent cce = new CertChainEvent(); >>> +? if(cce.isEnabled() || EventHelper.loggingSecurity()) { >>> +????? String c = reversedCertList.stream() >>> +????????????????? .map(x -> x.getSerialNumber().toString(16)) >>> +??????????????????????? .collect(Collectors.joining(", ")); >>> +???? EventHelper.commitCertChainEvent(cce, c); >>> +?? } >>> >>> No matter the event or the JFR mechanism is enabled or not, the >>> above code will create a new instance.? Is the return value of >>> cce.isEnabled() dynamically changed or static? >> This is a requirement from the JFR framework. All their >> event.isEnabled calls are instance methods and follow a similar >> pattern. The JFR team assure me that the JIT can optimize away such >> calls. > > The JIT will most likely not be able to optimize if the event class > escapes. > > From a JFR perspective, this would be the preferred layout: > > EventA eventA= new EventA(); > eventA.value = this.value; > eventA.commit(); > > and then do logging separately: > > System.Logger.log(DEBUG, this.value) > > With this layout, the JIT will remove the allocation and dead store. > > If it is expensive to gather the data for the event, like the > CertChainEvent above, the following pattern should be used. > > EventB eventB= new EventB (); > if (eventB.shouldCommit()) { > ?? eventB.value = calculateValue(); > ?? eventB .commit(); > } > > If JFR is not enabled, shouldCommit returns false and the JIT should > be able to remove the allocation.? This will be best from a > performance point of view, and also in my opinion from a maintenance > and readability perspective. Others may disagree. > > Erik > >>> >>> Is there a need to support both logging and JFR?? I'm new to record >>> events.? I did not get the point to use both. >> I was initially hoping to concentrate on just JFR events but I got >> strong feedback to also consider use of Logger (esp. in cases where >> the jdk.jfr module is not available) >> >> regards, >> Sean. >> >>> >>> Thanks, >>> Xuelei >>> >>> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>>> Erik, >>>> >>>> I rebased the patch with TLS v1.3 integration today. I hadn't >>>> realized how much the handshaker code had changed. Hopefully, I'll >>>> get a review from security dev team on that front. >>>> >>>> Regards the JFR semantics, I believe the edits should match >>>> majority of requests . I still have a preference for the test infra >>>> design for these new logger/JFR tests used in version 1 of webrev. >>>> I think it makes sense to keep the test functionality together - no >>>> sense in separating them out into different components IMO. Also, >>>> some of the edits to the JFR testing were made to test for the new >>>> dual log/record functionality. I might catch up with you tomorrow >>>> to see what the best arrangement would be. >>>> >>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>>> >>>> regards, >>>> Sean. >>>> >>>> >>>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>>> Many thanks for the review comments Erik. Replies inline. >>>>> >>>>> >>>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>>> Hi Sean, >>>>>> >>>>>> Some of the changes in the webrev belongs to JDK-8203629 and >>>>>> should be removed for clarity. >>>>>> >>>>>> Some initial comments: >>>>>> >>>>>> default.jfc, profile.jfr: >>>>>> The events should not have control="enable-exceptions". The >>>>>> purpose of the control attribute is so to provide parameterized >>>>>> configuration of events for JMC.? As it is now, the security >>>>>> events will be enabled when a user turns on the exception events. >>>>> Makes sense. I'll remove that parameter. >>>>>> >>>>>> X509CertEvent: >>>>>> You should use milliseconds since epoch to represent a date >>>>>> instead of a string value, i.e. >>>>>> >>>>>> ??? @Label("Valid From") >>>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>> ??? public long validFrom; >>>>>> >>>>>> ??? @Label("Valid Until") >>>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>> ??? public long validUntil; >>>>>> >>>>> The CertificateValidity class operates on Date Object values. I'll >>>>> work with the Date.getTime() method then (and update the Logger >>>>> formatting) >>>>>> This following annotation adds little value >>>>>> >>>>>> @Description("Details of Security Property") >>>>>> >>>>>> I would either remove the annotation, or provide information that >>>>>> helps a user understand the event. For instance, what is X509, >>>>>> and what kind of certificates are we talking about? >>>>> Yes - that looks like the wrong Description. I'll review all of >>>>> these fields. >>>>>> >>>>>> @Category("Java Application") >>>>>> >>>>>> I'm a bit worried that we will pollute the "Java Application" >>>>>> namespace if we add lots of JDK events in that category. We may >>>>>> want to add a new top level category "Java Development Kit", >>>>>> analogous to the "Java Virtual Machine" category, and put all >>>>>> security related events in subcategory, perhaps called "Security". >>>>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>>>> >>>>>> @Label("X509Cert") >>>>>> >>>>>> The label should be human readable name, with spaces, title cased >>>>>> etc. I would recommend "X.509 Certificate". In general, avoid >>>>>> abbreviations like "certs" and instead use labels such as >>>>>> "Certificate Chain". The label should be short and not a full >>>>>> sentence. >>>>>> >>>>>> For details see, >>>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>>> >>>>>> I think it would be good to separate testing of JFR and logging >>>>>> into different files / tests. I would prefer that the test name >>>>>> is the same as the event prefixed with "Test", i.e >>>>>> TestX509CertificateEvent, as this is the pattern used by other >>>>>> JFR tests. >>>>>> >>>>> I'll take a look at that pattern again. I've separated out the >>>>> current tests into an (a) outer test to analyze the logger output >>>>> and (b) the inner test which checks for JFR correctness. I did >>>>> include extra logic to make sure that the EventHelper >>>>> configuration was working correctly. "Events.assertField" looks >>>>> very helpful. Thanks for the pointer. >>>>> >>>>> Let me take on board the suggestions below and get a new webrev >>>>> out on Tuesday. >>>>> >>>>> regards, >>>>> Sean. >>>>> >>>>>> I reworked one of the tests to how I like to see it: >>>>>> >>>>>> /* >>>>>> ?* @test >>>>>> ?* @key jfr >>>>>> ?* @library /test/lib >>>>>> ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>>>> ?*/ >>>>>> public class TestX509CertificateEvent { >>>>>> >>>>>> ??? private static final String CERTIFICATE_1 = "..."; >>>>>> ??? private static final String CERTIFICATE_2 = "..."; >>>>>> >>>>>> ??? public static void main(String... args) throws >>>>>> CertificateException { >>>>>> >>>>>> ???????? Recording r = new Recording(); >>>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>>> ???????? r.start(); >>>>>> >>>>>> ???????? CertificateFactory cf = >>>>>> CertificateFactory.getInstance("X.509"); >>>>>> ???????? cf.generateCertificate(new >>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>> ???????? cf.generateCertificate(new >>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>> >>>>>> ???????? // Make sure only one event per certificate >>>>>> ???????? cf.generateCertificate(new >>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>> ???????? cf.generateCertificate(new >>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>> >>>>>> ???????? r.stop(); >>>>>> >>>>>> ???????? List events = Events.fromRecording(r); >>>>>> ???????? Asserts.assertEquals(events.size(), 2, "Expected two >>>>>> X.509 Certificate events"); >>>>>> >>>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>>> ???????????????????? "RSA", 2048); >>>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>>> ???????????????????? "RSA", 2048); >>>>>> ??? } >>>>>> >>>>>> ??? private static void assertEvent(List events, >>>>>> String certID, String algId, String subject, >>>>>> ??????????? String issuer, String keyType, int length) throws >>>>>> Exception { >>>>>> >>>>>> ??????? for (RecordedEvent e : events) { >>>>>> ??????????? if (e.getString("serialNumber").equals(certID)) { >>>>>> ??????????????? Events.assertField(e, "algId").equal(algId); >>>>>> ??????????????? ... >>>>>> ??????????????? return; >>>>>> ??????????? } >>>>>> ??????? } >>>>>> ??????? System.out.println(events); >>>>>> ??????? throw new Exception("Could not find event with Cert ID: " >>>>>> + certID); >>>>>> ??? } >>>>>> } >>>>>> >>>>>> The reworked example uses the Events.assertField method, which >>>>>> will give context if the assertion fails. Keeping the test >>>>>> simple, means it can be analyzed quickly if it fails in testing. >>>>>> There is no new test framework to learn, or methods to search >>>>>> for, and it is usually not hard to determine if the failure is >>>>>> product, test or infrastructure related, and what component >>>>>> (team) should be assigned the issue. The use of >>>>>> EventNames.X509Certificate means all occurrences of the event can >>>>>> be tracked done in an IDE using find by reference. >>>>>> >>>>>> Thanks >>>>>> Erik >>>>>> >>>>>>> Following on from the recent JDK-8203629 code review, I'd like >>>>>>> to propose enhancements on how we can record events in security >>>>>>> libs. The introduction of the JFR libraries can give us much >>>>>>> better ways of examining JDK actions. For the initial phase, I'm >>>>>>> looking to enhance some key security library events in JDK 11 so >>>>>>> that they can be either recorded to JFR, logged to a traditional >>>>>>> logger, or both. >>>>>>> >>>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>>> >>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>>> >>>>>>> securityProp_2.png gives an example of how the JFR recording can >>>>>>> be queried to quickly locate events of interest (in this case, >>>>>>> code setting the jdk.tls.* Security properties). I still need to >>>>>>> clean up the TLSEvents testcase to improve test coverage and >>>>>>> hope to do that in coming days. >>>>>>> >>>>>>> JBS record : >>>>>>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>>> >>>>>>> webrev : >>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>>> >>>>>> >>>>> >>>> >> > From xuelei.fan at oracle.com Thu Jun 28 12:42:21 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Thu, 28 Jun 2018 05:42:21 -0700 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <25b3ae82-5c97-ff18-d0f9-4bfea00c29cb@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <25b3ae82-5c97-ff18-d0f9-4bfea00c29cb@oracle.com> Message-ID: <1d8326f3-2a2f-2a0b-f6a5-8773a46e259b@oracle.com> Looks fine to me. Thanks! Xuelei On 6/28/2018 5:28 AM, Se?n Coffey wrote: > Thanks for reviewing Xuelei, > > I do acknowledge that the new TLS v1.3 code has greatly improved the > logging output for related operations. I think the main drive with this > enhancement is to use the new JFR API to capture interesting events. We > can revisit the Logger requirements if there's a strong opinion from > users. By default, both the new Logger and JFR output will be switched > off. I've just made an edit to the JFR jfc files to disable these events. > > "I may suggest remove this method and use Key.getAlgorithm() directly." > - Yes, great idea. Done. > > Thanks for feedback on Finished.java edits. I did question to myself > whether fewer edits were sufficient given the client/server nature of > the handshake. > > I've refreshed the webrev : > > http://cr.openjdk.java.net/~coffeys/webrev.8148188.v3/webrev/ > > some jfr side edits also : > > * Change label from "X.509 Certificate" to "X509 Certificate" - JFR test > fails with "." usage > * Move the instance field variable name in CertChainEvent to "certChain" > - JFR tests discourage use of? "ID" in "certIDs" > > regards, > Sean. > > > On 27/06/2018 21:11, Xuelei Fan wrote: >> Finished.java >> ------------- >> In each handshake, Finished messages are delivered twice.? One from >> client, and the other one from the server side.? Catch Finished >> message and use the final one of them should be sufficient. >> >> In the Finished.java implementation, the signal of the final Finished >> message is the handshakeFinished field is set to true. >> >> Please move line 582: >> ?582???????????? recordEvent(chc.conContext.conSession); >> to line 558: >> ?556???????????????? // handshake context cleanup. >> ?557???????????????? chc.handshakeFinished = true; >> ?558 >> >> Please move line 632: >> ?632???????????? recordEvent(shc.conContext.conSession); >> to line 608: >> ?606???????????????? // handshake context cleanup. >> ?607???????????????? shc.handshakeFinished = true; >> ?608 >> >> Please remove line 838: >> ?838???????????? recordEvent(shc.conContext.conSession); >> >> Please remove line 984: >> ?984???????????? recordEvent(chc.conContext.conSession); >> >> No more comment. >> >> Thanks, >> Xuelei >> >> On 6/27/2018 11:57 AM, Xuelei Fan wrote: >>> Hi Sean, >>> >>> I may reply in several replies. >>> >>> PKIXMasterCertPathValidator.java >>> -------------------------------- >>> +? CertChainEvent cce = new CertChainEvent(); >>> +? if(cce.isEnabled() || EventHelper.loggingSecurity()) { >>> +????? String c = reversedCertList.stream() >>> +????????????????? .map(x -> x.getSerialNumber().toString(16)) >>> +??????????????????????? .collect(Collectors.joining(", ")); >>> +???? EventHelper.commitCertChainEvent(cce, c); >>> +?? } >>> >>> No matter the event or the JFR mechanism is enabled or not, the above >>> code will create a new instance.? Is the return value of >>> cce.isEnabled() dynamically changed or static? >>> >>> Is there a need to support both logging and JFR?? I'm new to record >>> events.? I did not get the point to use both. >>> >>> Thanks, >>> Xuelei >>> >>> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>>> Erik, >>>> >>>> I rebased the patch with TLS v1.3 integration today. I hadn't >>>> realized how much the handshaker code had changed. Hopefully, I'll >>>> get a review from security dev team on that front. >>>> >>>> Regards the JFR semantics, I believe the edits should match majority >>>> of requests . I still have a preference for the test infra design >>>> for these new logger/JFR tests used in version 1 of webrev. I think >>>> it makes sense to keep the test functionality together - no sense in >>>> separating them out into different components IMO. Also, some of the >>>> edits to the JFR testing were made to test for the new dual >>>> log/record functionality. I might catch up with you tomorrow to see >>>> what the best arrangement would be. >>>> >>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>>> >>>> regards, >>>> Sean. >>>> >>>> >>>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>>> Many thanks for the review comments Erik. Replies inline. >>>>> >>>>> >>>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>>> Hi Sean, >>>>>> >>>>>> Some of the changes in the webrev belongs to JDK-8203629 and >>>>>> should be removed for clarity. >>>>>> >>>>>> Some initial comments: >>>>>> >>>>>> default.jfc, profile.jfr: >>>>>> The events should not have control="enable-exceptions". The >>>>>> purpose of the control attribute is so to provide parameterized >>>>>> configuration of events for JMC.? As it is now, the security >>>>>> events will be enabled when a user turns on the exception events. >>>>> Makes sense. I'll remove that parameter. >>>>>> >>>>>> X509CertEvent: >>>>>> You should use milliseconds since epoch to represent a date >>>>>> instead of a string value, i.e. >>>>>> >>>>>> ??? @Label("Valid From") >>>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>> ??? public long validFrom; >>>>>> >>>>>> ??? @Label("Valid Until") >>>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>> ??? public long validUntil; >>>>>> >>>>> The CertificateValidity class operates on Date Object values. I'll >>>>> work with the Date.getTime() method then (and update the Logger >>>>> formatting) >>>>>> This following annotation adds little value >>>>>> >>>>>> @Description("Details of Security Property") >>>>>> >>>>>> I would either remove the annotation, or provide information that >>>>>> helps a user understand the event. For instance, what is X509, and >>>>>> what kind of certificates are we talking about? >>>>> Yes - that looks like the wrong Description. I'll review all of >>>>> these fields. >>>>>> >>>>>> @Category("Java Application") >>>>>> >>>>>> I'm a bit worried that we will pollute the "Java Application" >>>>>> namespace if we add lots of JDK events in that category. We may >>>>>> want to add a new top level category "Java Development Kit", >>>>>> analogous to the "Java Virtual Machine" category, and put all >>>>>> security related events in subcategory, perhaps called "Security". >>>>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>>>> >>>>>> @Label("X509Cert") >>>>>> >>>>>> The label should be human readable name, with spaces, title cased >>>>>> etc. I would recommend "X.509 Certificate". In general, avoid >>>>>> abbreviations like "certs" and instead use labels such as >>>>>> "Certificate Chain". The label should be short and not a full >>>>>> sentence. >>>>>> >>>>>> For details see, >>>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>>> >>>>>> I think it would be good to separate testing of JFR and logging >>>>>> into different files / tests. I would prefer that the test name is >>>>>> the same as the event prefixed with "Test", i.e >>>>>> TestX509CertificateEvent, as this is the pattern used by other JFR >>>>>> tests. >>>>>> >>>>> I'll take a look at that pattern again. I've separated out the >>>>> current tests into an (a) outer test to analyze the logger output >>>>> and (b) the inner test which checks for JFR correctness. I did >>>>> include extra logic to make sure that the EventHelper configuration >>>>> was working correctly. "Events.assertField" looks very helpful. >>>>> Thanks for the pointer. >>>>> >>>>> Let me take on board the suggestions below and get a new webrev out >>>>> on Tuesday. >>>>> >>>>> regards, >>>>> Sean. >>>>> >>>>>> I reworked one of the tests to how I like to see it: >>>>>> >>>>>> /* >>>>>> ?* @test >>>>>> ?* @key jfr >>>>>> ?* @library /test/lib >>>>>> ?* @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>>>> ?*/ >>>>>> public class TestX509CertificateEvent { >>>>>> >>>>>> ??? private static final String CERTIFICATE_1 = "..."; >>>>>> ??? private static final String CERTIFICATE_2 = "..."; >>>>>> >>>>>> ??? public static void main(String... args) throws >>>>>> CertificateException { >>>>>> >>>>>> ???????? Recording r = new Recording(); >>>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>>> ???????? r.start(); >>>>>> >>>>>> ???????? CertificateFactory cf = >>>>>> CertificateFactory.getInstance("X.509"); >>>>>> ???????? cf.generateCertificate(new >>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>> ???????? cf.generateCertificate(new >>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>> >>>>>> ???????? // Make sure only one event per certificate >>>>>> ???????? cf.generateCertificate(new >>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>> ???????? cf.generateCertificate(new >>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>> >>>>>> ???????? r.stop(); >>>>>> >>>>>> ???????? List events = Events.fromRecording(r); >>>>>> ???????? Asserts.assertEquals(events.size(), 2, "Expected two >>>>>> X.509 Certificate events"); >>>>>> >>>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>>> ???????????????????? "RSA", 2048); >>>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>>> ???????????????????? "RSA", 2048); >>>>>> ??? } >>>>>> >>>>>> ??? private static void assertEvent(List events, >>>>>> String certID, String algId, String subject, >>>>>> ??????????? String issuer, String keyType, int length) throws >>>>>> Exception { >>>>>> >>>>>> ??????? for (RecordedEvent e : events) { >>>>>> ??????????? if (e.getString("serialNumber").equals(certID)) { >>>>>> ??????????????? Events.assertField(e, "algId").equal(algId); >>>>>> ??????????????? ... >>>>>> ??????????????? return; >>>>>> ??????????? } >>>>>> ??????? } >>>>>> ??????? System.out.println(events); >>>>>> ??????? throw new Exception("Could not find event with Cert ID: " >>>>>> + certID); >>>>>> ??? } >>>>>> } >>>>>> >>>>>> The reworked example uses the Events.assertField method, which >>>>>> will give context if the assertion fails. Keeping the test simple, >>>>>> means it can be analyzed quickly if it fails in testing. There is >>>>>> no new test framework to learn, or methods to search for, and it >>>>>> is usually not hard to determine if the failure is product, test >>>>>> or infrastructure related, and what component (team) should be >>>>>> assigned the issue. The use of EventNames.X509Certificate means >>>>>> all occurrences of the event can be tracked done in an IDE using >>>>>> find by reference. >>>>>> >>>>>> Thanks >>>>>> Erik >>>>>> >>>>>>> Following on from the recent JDK-8203629 code review, I'd like to >>>>>>> propose enhancements on how we can record events in security >>>>>>> libs. The introduction of the JFR libraries can give us much >>>>>>> better ways of examining JDK actions. For the initial phase, I'm >>>>>>> looking to enhance some key security library events in JDK 11 so >>>>>>> that they can be either recorded to JFR, logged to a traditional >>>>>>> logger, or both. >>>>>>> >>>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>>> >>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>>> >>>>>>> securityProp_2.png gives an example of how the JFR recording can >>>>>>> be queried to quickly locate events of interest (in this case, >>>>>>> code setting the jdk.tls.* Security properties). I still need to >>>>>>> clean up the TLSEvents testcase to improve test coverage and hope >>>>>>> to do that in coming days. >>>>>>> >>>>>>> JBS record : >>>>>>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>>> >>>>>>> webrev : >>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>>> >>>>>> >>>>> >>>> > From ecki at zusammenkunft.net Thu Jun 28 15:34:35 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 28 Jun 2018 15:34:35 +0000 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <2D66E03A-8F66-4535-8578-940049D3985D@gmail.com> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> , <2D66E03A-8F66-4535-8578-940049D3985D@gmail.com> Message-ID: you can jlink without any/complete module info files by specifying the module names on the command line (--add-modules)as well. It produces a jre like Directory including Java launcher which allows additions on the classpath. -- https://Bernd.eckenfels.net ________________________________ From: core-libs-dev on behalf of Scott Palmer Sent: Thursday, June 28, 2018 2:32:13 PM To: Kevin Rushforth Cc: core-libs-dev at openjdk.java.net Subject: Re: Draft JEP proposal: JDK-8200758: Packaging Tool Doesn?t jlink require a *fully* modularized application? I.e. no non-module dependencies. The packaging tool should work with all runnable Java applications, not just fully modularized ones. Modularization seems to be a bit of an effort and is one of the main reasons my application(s) are still stuck on Java 8. Scott > On Jun 27, 2018, at 6:30 PM, Kevin Rushforth wrote: > > We're aiming to get this into JDK 12 early enough so that an EA build would be available around the time JDK 11 ships. That will allow you to take a jlinked image with JDK 11 and package it up using (the EA) jpackager. > > We will create a development branch in the JDK sandbox [1] some time in the next week or so so you can follow the development. > > Also, thank you to those who have provided feedback. I'll reply to feedback soon and then incorporate it into an updated JEP. > > -- Kevin > > > On 6/27/2018 3:09 AM, Buchberger, Joerg wrote: >> Thanks for the info! And thanks for the efforts. [no irony, no sarcasm - I really mean it] >> >> But, to sum up my comprehension... >> >> anyone who placed their bets on javapackager, starting with last LTS Java 8 >> will be left in the rain with followup LTS Java 11, because their ain't neither javapackager (anymore), nor jpackager (yet). >> >> Is this correct? >> >> So, strategic choice boils down to either throw away all work done based on javapackager so far and the associated distribution concepts (reworking everything from scratch) >> or neglect Java 11 completely, thus placing all bets on jpackager really coming w/ Java 12 or even waiting for Java 14 as next LTS thereafter. >> >> Bam(!), I think, I first need a tiny shot now ;-) and let that info sink in ... >> >> Cheers >> J?rg >> >> >> On 5/31/2018 0:10 AM, Rushforth, Kevin wrote: >>> I would like to propose the following Draft JEP [1] for discussion. >>> >>> JDK-8200758: Packaging Tool >>> >>> This is intended as a JDK-scope replacement for the existing >>> javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK >>> releases), and was delivered as part of the JavaFX build. The >>> javapackager tool has been removed from Oracle JDK 11 along with the >>> removal of JavaFX. >>> >>> Comments on this JEP are welcome. It is currently not targeted for a >>> specific release, but we are aiming for JDK 12. >>> >>> -- Kevin >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>> > From erik.gahlin at oracle.com Thu Jun 28 16:20:53 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 28 Jun 2018 18:20:53 +0200 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> Message-ID: <5B350AE5.3080401@oracle.com> It's sufficient if an event object escapes to another method (regardless if JFR is enabled or not). Some more feedback: Rename event jdk.CertChain to jdk.CertificateChain Rename event jdk.X509Cert to jdk.X509Certificate Rename field certChain to certificateChain. Rename field serialNum to serialNumber Rename field algId to AlgorithmicId or AlgorithmicID Rename @Label("Ciphersuite") to @Label("Cipher Suite") Rename @Label("Cert Chain") to @Label("Certificate Chain") Rename @Label("Property Name") to "Name" or "Key" if that makes sense in the context? Rename @Label("Property Value") to "Value". Put events in a subcategory, i.e @Category({"Java Development Kit", "Security"}) Make classes final. What is the unit of the key in X509Certificate event? Bits? If that is the case, use @DataAmount(DataAmount.BITS) @Label("Serial numbers forming chain of trust") should not be a sentence. How about "Serial Numbers"? I think the tests are hard to read when two things are tested at the same time. It is also likely that if a test fail due to logging issues, it will be assigned to JFR because of the test name, even thought the issues is not JFR related. If you want to reuse code between tests, I would put it in testlibrary. Erik > Thanks for the update Erik. By default I'm proposing that the new JFR > Events and Logger be disabled. As a result the event class shouldn't > escape. If performance metrics highlight an issue, we should revisit. > > regards, > Sean. > > > On 27/06/2018 20:57, Erik Gahlin wrote: >> On 2018-06-27 21:14, Se?n Coffey wrote: >>> >>> >>> On 27/06/2018 19:57, Xuelei Fan wrote: >>>> Hi Sean, >>>> >>>> I may reply in several replies. >>>> >>>> PKIXMasterCertPathValidator.java >>>> -------------------------------- >>>> + CertChainEvent cce = new CertChainEvent(); >>>> + if(cce.isEnabled() || EventHelper.loggingSecurity()) { >>>> + String c = reversedCertList.stream() >>>> + .map(x -> x.getSerialNumber().toString(16)) >>>> + .collect(Collectors.joining(", ")); >>>> + EventHelper.commitCertChainEvent(cce, c); >>>> + } >>>> >>>> No matter the event or the JFR mechanism is enabled or not, the >>>> above code will create a new instance. Is the return value of >>>> cce.isEnabled() dynamically changed or static? >>> This is a requirement from the JFR framework. All their >>> event.isEnabled calls are instance methods and follow a similar >>> pattern. The JFR team assure me that the JIT can optimize away such >>> calls. >> >> The JIT will most likely not be able to optimize if the event class >> escapes. >> >> From a JFR perspective, this would be the preferred layout: >> >> EventA eventA= new EventA(); >> eventA.value = this.value; >> eventA.commit(); >> >> and then do logging separately: >> >> System.Logger.log(DEBUG, this.value) >> >> With this layout, the JIT will remove the allocation and dead store. >> >> If it is expensive to gather the data for the event, like the >> CertChainEvent above, the following pattern should be used. >> >> EventB eventB= new EventB (); >> if (eventB.shouldCommit()) { >> eventB.value = calculateValue(); >> eventB .commit(); >> } >> >> If JFR is not enabled, shouldCommit returns false and the JIT should >> be able to remove the allocation. This will be best from a >> performance point of view, and also in my opinion from a maintenance >> and readability perspective. Others may disagree. >> >> Erik >> >>>> >>>> Is there a need to support both logging and JFR? I'm new to record >>>> events. I did not get the point to use both. >>> I was initially hoping to concentrate on just JFR events but I got >>> strong feedback to also consider use of Logger (esp. in cases where >>> the jdk.jfr module is not available) >>> >>> regards, >>> Sean. >>> >>>> >>>> Thanks, >>>> Xuelei >>>> >>>> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>>>> Erik, >>>>> >>>>> I rebased the patch with TLS v1.3 integration today. I hadn't >>>>> realized how much the handshaker code had changed. Hopefully, I'll >>>>> get a review from security dev team on that front. >>>>> >>>>> Regards the JFR semantics, I believe the edits should match >>>>> majority of requests . I still have a preference for the test >>>>> infra design for these new logger/JFR tests used in version 1 of >>>>> webrev. I think it makes sense to keep the test functionality >>>>> together - no sense in separating them out into different >>>>> components IMO. Also, some of the edits to the JFR testing were >>>>> made to test for the new dual log/record functionality. I might >>>>> catch up with you tomorrow to see what the best arrangement would be. >>>>> >>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>>>> >>>>> regards, >>>>> Sean. >>>>> >>>>> >>>>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>>>> Many thanks for the review comments Erik. Replies inline. >>>>>> >>>>>> >>>>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>>>> Hi Sean, >>>>>>> >>>>>>> Some of the changes in the webrev belongs to JDK-8203629 and >>>>>>> should be removed for clarity. >>>>>>> >>>>>>> Some initial comments: >>>>>>> >>>>>>> default.jfc, profile.jfr: >>>>>>> The events should not have control="enable-exceptions". The >>>>>>> purpose of the control attribute is so to provide parameterized >>>>>>> configuration of events for JMC. As it is now, the security >>>>>>> events will be enabled when a user turns on the exception events. >>>>>> Makes sense. I'll remove that parameter. >>>>>>> >>>>>>> X509CertEvent: >>>>>>> You should use milliseconds since epoch to represent a date >>>>>>> instead of a string value, i.e. >>>>>>> >>>>>>> @Label("Valid From") >>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>> public long validFrom; >>>>>>> >>>>>>> @Label("Valid Until") >>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>> public long validUntil; >>>>>>> >>>>>> The CertificateValidity class operates on Date Object values. >>>>>> I'll work with the Date.getTime() method then (and update the >>>>>> Logger formatting) >>>>>>> This following annotation adds little value >>>>>>> >>>>>>> @Description("Details of Security Property") >>>>>>> >>>>>>> I would either remove the annotation, or provide information >>>>>>> that helps a user understand the event. For instance, what is >>>>>>> X509, and what kind of certificates are we talking about? >>>>>> Yes - that looks like the wrong Description. I'll review all of >>>>>> these fields. >>>>>>> >>>>>>> @Category("Java Application") >>>>>>> >>>>>>> I'm a bit worried that we will pollute the "Java Application" >>>>>>> namespace if we add lots of JDK events in that category. We may >>>>>>> want to add a new top level category "Java Development Kit", >>>>>>> analogous to the "Java Virtual Machine" category, and put all >>>>>>> security related events in subcategory, perhaps called "Security". >>>>>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>>>>> >>>>>>> @Label("X509Cert") >>>>>>> >>>>>>> The label should be human readable name, with spaces, title >>>>>>> cased etc. I would recommend "X.509 Certificate". In general, >>>>>>> avoid abbreviations like "certs" and instead use labels such as >>>>>>> "Certificate Chain". The label should be short and not a full >>>>>>> sentence. >>>>>>> >>>>>>> For details see, >>>>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>>>> >>>>>>> I think it would be good to separate testing of JFR and logging >>>>>>> into different files / tests. I would prefer that the test name >>>>>>> is the same as the event prefixed with "Test", i.e >>>>>>> TestX509CertificateEvent, as this is the pattern used by other >>>>>>> JFR tests. >>>>>>> >>>>>> I'll take a look at that pattern again. I've separated out the >>>>>> current tests into an (a) outer test to analyze the logger output >>>>>> and (b) the inner test which checks for JFR correctness. I did >>>>>> include extra logic to make sure that the EventHelper >>>>>> configuration was working correctly. "Events.assertField" looks >>>>>> very helpful. Thanks for the pointer. >>>>>> >>>>>> Let me take on board the suggestions below and get a new webrev >>>>>> out on Tuesday. >>>>>> >>>>>> regards, >>>>>> Sean. >>>>>> >>>>>>> I reworked one of the tests to how I like to see it: >>>>>>> >>>>>>> /* >>>>>>> * @test >>>>>>> * @key jfr >>>>>>> * @library /test/lib >>>>>>> * @run main/othervm >>>>>>> jdk.jfr.event.security.TestX509CertificateEvent >>>>>>> */ >>>>>>> public class TestX509CertificateEvent { >>>>>>> >>>>>>> private static final String CERTIFICATE_1 = "..."; >>>>>>> private static final String CERTIFICATE_2 = "..."; >>>>>>> >>>>>>> public static void main(String... args) throws >>>>>>> CertificateException { >>>>>>> >>>>>>> Recording r = new Recording(); >>>>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>>>> r.start(); >>>>>>> >>>>>>> CertificateFactory cf = >>>>>>> CertificateFactory.getInstance("X.509"); >>>>>>> cf.generateCertificate(new >>>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>> cf.generateCertificate(new >>>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>> >>>>>>> // Make sure only one event per certificate >>>>>>> cf.generateCertificate(new >>>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>> cf.generateCertificate(new >>>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>> >>>>>>> r.stop(); >>>>>>> >>>>>>> List events = Events.fromRecording(r); >>>>>>> Asserts.assertEquals(events.size(), 2, "Expected two >>>>>>> X.509 Certificate events"); >>>>>>> >>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>> "RSA", 2048); >>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>> "RSA", 2048); >>>>>>> } >>>>>>> >>>>>>> private static void assertEvent(List events, >>>>>>> String certID, String algId, String subject, >>>>>>> String issuer, String keyType, int length) throws >>>>>>> Exception { >>>>>>> >>>>>>> for (RecordedEvent e : events) { >>>>>>> if (e.getString("serialNumber").equals(certID)) { >>>>>>> Events.assertField(e, "algId").equal(algId); >>>>>>> ... >>>>>>> return; >>>>>>> } >>>>>>> } >>>>>>> System.out.println(events); >>>>>>> throw new Exception("Could not find event with Cert ID: >>>>>>> " + certID); >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> The reworked example uses the Events.assertField method, which >>>>>>> will give context if the assertion fails. Keeping the test >>>>>>> simple, means it can be analyzed quickly if it fails in testing. >>>>>>> There is no new test framework to learn, or methods to search >>>>>>> for, and it is usually not hard to determine if the failure is >>>>>>> product, test or infrastructure related, and what component >>>>>>> (team) should be assigned the issue. The use of >>>>>>> EventNames.X509Certificate means all occurrences of the event >>>>>>> can be tracked done in an IDE using find by reference. >>>>>>> >>>>>>> Thanks >>>>>>> Erik >>>>>>> >>>>>>>> Following on from the recent JDK-8203629 code review, I'd like >>>>>>>> to propose enhancements on how we can record events in security >>>>>>>> libs. The introduction of the JFR libraries can give us much >>>>>>>> better ways of examining JDK actions. For the initial phase, >>>>>>>> I'm looking to enhance some key security library events in JDK >>>>>>>> 11 so that they can be either recorded to JFR, logged to a >>>>>>>> traditional logger, or both. >>>>>>>> >>>>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>>>> >>>>>>>> securityProp_2.png gives an example of how the JFR recording >>>>>>>> can be queried to quickly locate events of interest (in this >>>>>>>> case, code setting the jdk.tls.* Security properties). I still >>>>>>>> need to clean up the TLSEvents testcase to improve test >>>>>>>> coverage and hope to do that in coming days. >>>>>>>> >>>>>>>> JBS record : >>>>>>>> * https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>>>> >>>>>>>> webrev : >>>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>>>> >>>>>>> >>>>>> >>>>> >>> >> > From sean.coffey at oracle.com Thu Jun 28 16:59:02 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 28 Jun 2018 17:59:02 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <5B350AE5.3080401@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> <5B350AE5.3080401@oracle.com> Message-ID: Comments inline. On 28/06/2018 17:20, Erik Gahlin wrote: > It's sufficient if an event object escapes to another method > (regardless if JFR is enabled or not). > > Some more feedback: > > Rename event jdk.CertChain to jdk.CertificateChain > Rename event jdk.X509Cert to jdk.X509Certificate > Rename field certChain to certificateChain. > Rename field serialNum to serialNumber all above done. > Rename field algId to AlgorithmicId or AlgorithmicID maybe "algorithm" works here also ? > Rename @Label("Ciphersuite") to @Label("Cipher Suite") > Rename @Label("Cert Chain") to @Label("Certificate Chain") > Rename @Label("Property Name") to "Name" or "Key" if that makes sense > in the context? > Rename @Label("Property Value") to "Value". > Put events in a subcategory, i.e? @Category({"Java Development Kit", > "Security"}) done. > Make classes final. done. I had thought that the JFR mechanism required non-final classes. > What is the unit of the key in X509Certificate event? Bits? If that is > the case, use @DataAmount(DataAmount.BITS) Yes - it's essentially the bit length of the keys used. Let me look into that annotation usage. > @Label("Serial numbers forming chain of trust") should not be a > sentence. How about "Serial Numbers"? > > I think the tests are hard to read when two things are tested at the > same time. It is also likely that if a test fail due to logging > issues, it will be assigned to JFR because of the test name, even > thought the issues is not JFR related. I think we're always going to have some ownership issues when tests serve a dual purpose. I still think it makes sense to keep the test logic in one place. Any failures in these tests will most likely be owned by security team. (moving the tests to security directory is also an option) > > If you want to reuse code between tests, I would put it in testlibrary. Let me check if there's any common patterns that could be added to the testlibary. Thanks, Sean. > > Erik > >> Thanks for the update Erik. By default I'm proposing that the new JFR >> Events and Logger be disabled. As a result the event class shouldn't >> escape. If performance metrics highlight an issue, we should revisit. >> >> regards, >> Sean. >> >> >> On 27/06/2018 20:57, Erik Gahlin wrote: >>> On 2018-06-27 21:14, Se?n Coffey wrote: >>>> >>>> >>>> On 27/06/2018 19:57, Xuelei Fan wrote: >>>>> Hi Sean, >>>>> >>>>> I may reply in several replies. >>>>> >>>>> PKIXMasterCertPathValidator.java >>>>> -------------------------------- >>>>> +? CertChainEvent cce = new CertChainEvent(); >>>>> +? if(cce.isEnabled() || EventHelper.loggingSecurity()) { >>>>> +????? String c = reversedCertList.stream() >>>>> +????????????????? .map(x -> x.getSerialNumber().toString(16)) >>>>> +??????????????????????? .collect(Collectors.joining(", ")); >>>>> +???? EventHelper.commitCertChainEvent(cce, c); >>>>> +?? } >>>>> >>>>> No matter the event or the JFR mechanism is enabled or not, the >>>>> above code will create a new instance.? Is the return value of >>>>> cce.isEnabled() dynamically changed or static? >>>> This is a requirement from the JFR framework. All their >>>> event.isEnabled calls are instance methods and follow a similar >>>> pattern. The JFR team assure me that the JIT can optimize away such >>>> calls. >>> >>> The JIT will most likely not be able to optimize if the event class >>> escapes. >>> >>> From a JFR perspective, this would be the preferred layout: >>> >>> EventA eventA= new EventA(); >>> eventA.value = this.value; >>> eventA.commit(); >>> >>> and then do logging separately: >>> >>> System.Logger.log(DEBUG, this.value) >>> >>> With this layout, the JIT will remove the allocation and dead store. >>> >>> If it is expensive to gather the data for the event, like the >>> CertChainEvent above, the following pattern should be used. >>> >>> EventB eventB= new EventB (); >>> if (eventB.shouldCommit()) { >>> ?? eventB.value = calculateValue(); >>> ?? eventB .commit(); >>> } >>> >>> If JFR is not enabled, shouldCommit returns false and the JIT should >>> be able to remove the allocation.? This will be best from a >>> performance point of view, and also in my opinion from a maintenance >>> and readability perspective. Others may disagree. >>> >>> Erik >>> >>>>> >>>>> Is there a need to support both logging and JFR?? I'm new to >>>>> record events.? I did not get the point to use both. >>>> I was initially hoping to concentrate on just JFR events but I got >>>> strong feedback to also consider use of Logger (esp. in cases where >>>> the jdk.jfr module is not available) >>>> >>>> regards, >>>> Sean. >>>> >>>>> >>>>> Thanks, >>>>> Xuelei >>>>> >>>>> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>>>>> Erik, >>>>>> >>>>>> I rebased the patch with TLS v1.3 integration today. I hadn't >>>>>> realized how much the handshaker code had changed. Hopefully, >>>>>> I'll get a review from security dev team on that front. >>>>>> >>>>>> Regards the JFR semantics, I believe the edits should match >>>>>> majority of requests . I still have a preference for the test >>>>>> infra design for these new logger/JFR tests used in version 1 of >>>>>> webrev. I think it makes sense to keep the test functionality >>>>>> together - no sense in separating them out into different >>>>>> components IMO. Also, some of the edits to the JFR testing were >>>>>> made to test for the new dual log/record functionality. I might >>>>>> catch up with you tomorrow to see what the best arrangement would >>>>>> be. >>>>>> >>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>>>>> >>>>>> regards, >>>>>> Sean. >>>>>> >>>>>> >>>>>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>>>>> Many thanks for the review comments Erik. Replies inline. >>>>>>> >>>>>>> >>>>>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>>>>> Hi Sean, >>>>>>>> >>>>>>>> Some of the changes in the webrev belongs to JDK-8203629 and >>>>>>>> should be removed for clarity. >>>>>>>> >>>>>>>> Some initial comments: >>>>>>>> >>>>>>>> default.jfc, profile.jfr: >>>>>>>> The events should not have control="enable-exceptions". The >>>>>>>> purpose of the control attribute is so to provide parameterized >>>>>>>> configuration of events for JMC.? As it is now, the security >>>>>>>> events will be enabled when a user turns on the exception events. >>>>>>> Makes sense. I'll remove that parameter. >>>>>>>> >>>>>>>> X509CertEvent: >>>>>>>> You should use milliseconds since epoch to represent a date >>>>>>>> instead of a string value, i.e. >>>>>>>> >>>>>>>> ??? @Label("Valid From") >>>>>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>> ??? public long validFrom; >>>>>>>> >>>>>>>> ??? @Label("Valid Until") >>>>>>>> ??? @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>> ??? public long validUntil; >>>>>>>> >>>>>>> The CertificateValidity class operates on Date Object values. >>>>>>> I'll work with the Date.getTime() method then (and update the >>>>>>> Logger formatting) >>>>>>>> This following annotation adds little value >>>>>>>> >>>>>>>> @Description("Details of Security Property") >>>>>>>> >>>>>>>> I would either remove the annotation, or provide information >>>>>>>> that helps a user understand the event. For instance, what is >>>>>>>> X509, and what kind of certificates are we talking about? >>>>>>> Yes - that looks like the wrong Description. I'll review all of >>>>>>> these fields. >>>>>>>> >>>>>>>> @Category("Java Application") >>>>>>>> >>>>>>>> I'm a bit worried that we will pollute the "Java Application" >>>>>>>> namespace if we add lots of JDK events in that category. We may >>>>>>>> want to add a new top level category "Java Development Kit", >>>>>>>> analogous to the "Java Virtual Machine" category, and put all >>>>>>>> security related events in subcategory, perhaps called "Security". >>>>>>> Yes - Open to suggestions. "Security" sounds like a good >>>>>>> suggestion. >>>>>>>> >>>>>>>> @Label("X509Cert") >>>>>>>> >>>>>>>> The label should be human readable name, with spaces, title >>>>>>>> cased etc. I would recommend "X.509 Certificate". In general, >>>>>>>> avoid abbreviations like "certs" and instead use labels such as >>>>>>>> "Certificate Chain". The label should be short and not a full >>>>>>>> sentence. >>>>>>>> >>>>>>>> For details see, >>>>>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>>>>> >>>>>>>> I think it would be good to separate testing of JFR and logging >>>>>>>> into different files / tests. I would prefer that the test name >>>>>>>> is the same as the event prefixed with "Test", i.e >>>>>>>> TestX509CertificateEvent, as this is the pattern used by other >>>>>>>> JFR tests. >>>>>>>> >>>>>>> I'll take a look at that pattern again. I've separated out the >>>>>>> current tests into an (a) outer test to analyze the logger >>>>>>> output and (b) the inner test which checks for JFR correctness. >>>>>>> I did include extra logic to make sure that the EventHelper >>>>>>> configuration was working correctly. "Events.assertField" looks >>>>>>> very helpful. Thanks for the pointer. >>>>>>> >>>>>>> Let me take on board the suggestions below and get a new webrev >>>>>>> out on Tuesday. >>>>>>> >>>>>>> regards, >>>>>>> Sean. >>>>>>> >>>>>>>> I reworked one of the tests to how I like to see it: >>>>>>>> >>>>>>>> /* >>>>>>>> ?* @test >>>>>>>> ?* @key jfr >>>>>>>> ?* @library /test/lib >>>>>>>> ?* @run main/othervm >>>>>>>> jdk.jfr.event.security.TestX509CertificateEvent >>>>>>>> ?*/ >>>>>>>> public class TestX509CertificateEvent { >>>>>>>> >>>>>>>> ??? private static final String CERTIFICATE_1 = "..."; >>>>>>>> ??? private static final String CERTIFICATE_2 = "..."; >>>>>>>> >>>>>>>> ??? public static void main(String... args) throws >>>>>>>> CertificateException { >>>>>>>> >>>>>>>> ???????? Recording r = new Recording(); >>>>>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>>>>> ???????? r.start(); >>>>>>>> >>>>>>>> ???????? CertificateFactory cf = >>>>>>>> CertificateFactory.getInstance("X.509"); >>>>>>>> ???????? cf.generateCertificate(new >>>>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>> ???????? cf.generateCertificate(new >>>>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>> >>>>>>>> ???????? // Make sure only one event per certificate >>>>>>>> ???????? cf.generateCertificate(new >>>>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>> ???????? cf.generateCertificate(new >>>>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>> >>>>>>>> ???????? r.stop(); >>>>>>>> >>>>>>>> ???????? List events = Events.fromRecording(r); >>>>>>>> ???????? Asserts.assertEquals(events.size(), 2, "Expected two >>>>>>>> X.509 Certificate events"); >>>>>>>> >>>>>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>> ???????????????????? "RSA", 2048); >>>>>>>> ???????? assertEvent(events, "1000", "SHA256withRSA", >>>>>>>> ??????????????????? "CN=SSLCertificate, O=SomeCompany", >>>>>>>> ??????????????????? "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>> ???????????????????? "RSA", 2048); >>>>>>>> ??? } >>>>>>>> >>>>>>>> ??? private static void assertEvent(List events, >>>>>>>> String certID, String algId, String subject, >>>>>>>> ??????????? String issuer, String keyType, int length) throws >>>>>>>> Exception { >>>>>>>> >>>>>>>> ??????? for (RecordedEvent e : events) { >>>>>>>> ??????????? if (e.getString("serialNumber").equals(certID)) { >>>>>>>> ??????????????? Events.assertField(e, "algId").equal(algId); >>>>>>>> ??????????????? ... >>>>>>>> ??????????????? return; >>>>>>>> ??????????? } >>>>>>>> ??????? } >>>>>>>> ??????? System.out.println(events); >>>>>>>> ??????? throw new Exception("Could not find event with Cert ID: >>>>>>>> " + certID); >>>>>>>> ??? } >>>>>>>> } >>>>>>>> >>>>>>>> The reworked example uses the Events.assertField method, which >>>>>>>> will give context if the assertion fails. Keeping the test >>>>>>>> simple, means it can be analyzed quickly if it fails in >>>>>>>> testing. There is no new test framework to learn, or methods to >>>>>>>> search for, and it is usually not hard to determine if the >>>>>>>> failure is product, test or infrastructure related, and what >>>>>>>> component (team) should be assigned the issue. The use of >>>>>>>> EventNames.X509Certificate means all occurrences of the event >>>>>>>> can be tracked done in an IDE using find by reference. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Erik >>>>>>>> >>>>>>>>> Following on from the recent JDK-8203629 code review, I'd like >>>>>>>>> to propose enhancements on how we can record events in >>>>>>>>> security libs. The introduction of the JFR libraries can give >>>>>>>>> us much better ways of examining JDK actions. For the initial >>>>>>>>> phase, I'm looking to enhance some key security library events >>>>>>>>> in JDK 11 so that they can be either recorded to JFR, logged >>>>>>>>> to a traditional logger, or both. >>>>>>>>> >>>>>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>>>>> >>>>>>>>> securityProp_2.png gives an example of how the JFR recording >>>>>>>>> can be queried to quickly locate events of interest (in this >>>>>>>>> case, code setting the jdk.tls.* Security properties). I still >>>>>>>>> need to clean up the TLSEvents testcase to improve test >>>>>>>>> coverage and hope to do that in coming days. >>>>>>>>> >>>>>>>>> JBS record : >>>>>>>>> ?* https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>>>>> >>>>>>>>> webrev : >>>>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> >> > From bsrbnd at gmail.com Thu Jun 28 17:16:14 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Thu, 28 Jun 2018 19:16:14 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: <7a13e911-bdde-e710-0331-0892f2995c8e@oracle.com> References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> <7a13e911-bdde-e710-0331-0892f2995c8e@oracle.com> Message-ID: Hi Alan, On 28 June 2018 at 13:49, Alan Bateman wrote: > On 20/06/2018 11:08, Pengfei Li wrote: >> >> Hi >> >> I have tried the patch ( >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/052991.html ) >> on Ubuntu 18.04 machines (x86_64 & aarch64) with glibc=2.26.1 and build is >> OK. >> >> There's a small issue within the following code in >> src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c >> Unlike readdir_r(), readdir() does not return int value. The value of res >> is always 0 before #ifdef. >> >> 727 /* EINTR not listed as a possible error */ >> 728 errno = 0; >> 729 ptr = readdir64(dirp); >> 730 res = errno; >> 731 >> 732 #ifdef _AIX >> 733 /* On AIX, readdir() returns EBADF (i.e. '9') and sets 'result' >> to NULL for the */ >> 734 /* directory stream end. Otherwise, 'errno' will contain the >> error code. */ >> 735 if (res != 0) { >> 736 res = (ptr == NULL && res == EBADF) ? 0 : errno; >> 737 } >> 738 #endif >> >> May I know that if this core-libs change going to be merged recently, or >> more platforms needs to be explored? >> > I see your other mail looking to to add #pragma GCC ... to avoid using > configure --disable-warnings-as-errors. > > I think it would be better to just replace the usage readdir_r usage in that > native method. I've tested the patch below on macOS, Linux and Solaris. The > SAP folks maintain the AIX port and would need to test it on that platform > as it eliminates the AIX specific code for dealing with end of stream that > should be needed when using readdir. > > -Alan I've checked AIX's 'readdir()' doc [1] and I guess you're right that it never sets 'errno' to 'EBADF' while 'readdir_r()' may return it (see [2]). I missed this on my original patch. So, your update seems fine to me even if I cannot evaluate it on AIX myself. Thanks, Bernard [1] https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.basetrf1/opendir.htm [2] https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.basetrf2/readdir_r.htm > diff -r 9d62da00bf15 -r 5e67e87bd6fa > src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c > --- a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Sat May 26 > 06:59:49 2018 +0200 > +++ b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Mon Jun 25 > 14:17:03 2018 +0100 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2008, 2017, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2008, 2018, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -72,7 +72,7 @@ > #define fstat64 fstat > #define lstat64 lstat > #define dirent64 dirent > -#define readdir64_r readdir_r > +#define readdir64 readdir > #endif > > #include "jni.h" > @@ -720,41 +720,23 @@ > > JNIEXPORT jbyteArray JNICALL > Java_sun_nio_fs_UnixNativeDispatcher_readdir(JNIEnv* env, jclass this, > jlong value) { > - struct dirent64* result; > - struct { > - struct dirent64 buf; > - char name_extra[PATH_MAX + 1 - sizeof result->d_name]; > - } entry; > - struct dirent64* ptr = &entry.buf; > - int res; > DIR* dirp = jlong_to_ptr(value); > + struct dirent64* ptr; > > - /* EINTR not listed as a possible error */ > - /* TDB: reentrant version probably not required here */ > - res = readdir64_r(dirp, ptr, &result); > - > -#ifdef _AIX > - /* On AIX, readdir_r() returns EBADF (i.e. '9') and sets 'result' to > NULL for the */ > - /* directory stream end. Otherwise, 'errno' will contain the error > code. */ > - if (res != 0) { > - res = (result == NULL && res == EBADF) ? 0 : errno; > - } > -#endif > - > - if (res != 0) { > - throwUnixException(env, res); > + errno = 0; > + ptr = readdir64(dirp); > + if (ptr == NULL) { > + if (errno != 0) { > + throwUnixException(env, errno); > + } > return NULL; > } else { > - if (result == NULL) { > - return NULL; > - } else { > - jsize len = strlen(ptr->d_name); > - jbyteArray bytes = (*env)->NewByteArray(env, len); > - if (bytes != NULL) { > - (*env)->SetByteArrayRegion(env, bytes, 0, len, > (jbyte*)(ptr->d_name)); > - } > - return bytes; > + jsize len = strlen(ptr->d_name); > + jbyteArray bytes = (*env)->NewByteArray(env, len); > + if (bytes != NULL) { > + (*env)->SetByteArrayRegion(env, bytes, 0, len, > (jbyte*)(ptr->d_name)); > } > + return bytes; > } > } From paul.sandoz at oracle.com Thu Jun 28 17:17:11 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 28 Jun 2018 10:17:11 -0700 Subject: RFR: 8177275: IllegalArgumentException when MH would have too many parameters is not specified for several methods In-Reply-To: <23748ee1-7c78-46d2-bbda-a1a3f43d75a2@default> References: <23748ee1-7c78-46d2-bbda-a1a3f43d75a2@default> Message-ID: <659B4C6D-2C1B-4255-8100-28FD817D2F89@oracle.com> +1 Paul. > On Jun 28, 2018, at 1:27 AM, Vivek Theeyarath wrote: > > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8177275 . The jtreg test runs fine with the changes. > > > > http://cr.openjdk.java.net/~vtheeyarath/8177275/webrev.02/ > > > > CSR : https://bugs.openjdk.java.net/browse/JDK-8205917 > > > > Regards > > Vivek > > From isaac.r.levy at gmail.com Thu Jun 28 18:59:16 2018 From: isaac.r.levy at gmail.com (Isaac Levy) Date: Thu, 28 Jun 2018 14:59:16 -0400 Subject: [PATCH] AbstractStringBuilder.append() In-Reply-To: References: Message-ID: And can't remove append(StringBuffer) because of binary compatibility? Isaac On Thu, Jun 28, 2018 at 1:22 AM, Martin Buchholz wrote: > I'm fairly sure the append(StringBuilder) overloads were left out > intentionally. > > On Wed, Jun 27, 2018 at 8:57 PM, Isaac Levy wrote: >> >> AbstractStringBuilder already has append(). This patch >> adds append(). >> >> The new method improves parity between the two classes. In addition, >> combining StringBuilders is presumably common. append() >> has a couple insteadof checks, which this new method skips. >> >> -Isaac >> >> >> >> >> diff --git >> a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> index 2ef3e53256..1fe89bb92a 100644 >> --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> @@ -543,6 +543,11 @@ abstract class AbstractStringBuilder implements >> Appendable, CharSequence { >> return this.append((AbstractStringBuilder)sb); >> } >> >> + // Documentation in subclasses because of synchro difference >> + public AbstractStringBuilder append(StringBuilder sb) { >> + return this.append((AbstractStringBuilder)sb); >> + } >> + >> /** >> * @since 1.8 >> */ >> diff --git a/src/java.base/share/classes/java/lang/StringBuffer.java >> b/src/java.base/share/classes/java/lang/StringBuffer.java >> index e597a8112e..613ba90c25 100644 >> --- a/src/java.base/share/classes/java/lang/StringBuffer.java >> +++ b/src/java.base/share/classes/java/lang/StringBuffer.java >> @@ -313,6 +313,33 @@ import jdk.internal.HotSpotIntrinsicCandidate; >> return this; >> } >> >> + /** >> + * Appends the specified {@code StringBuilder} to this sequence. >> + *

        >> + * The characters of the {@code StringBuilder} argument are appended, >> + * in order, to the contents of this {@code StringBuffer}, increasing >> the >> + * length of this {@code StringBuffer} by the length of the argument. >> + * If {@code sb} is {@code null}, then the four characters >> + * {@code "null"} are appended to this {@code StringBuffer}. >> + *

        >> + * Let n be the length of the old character sequence, the one >> + * contained in the {@code StringBuffer} just prior to execution of >> the >> + * {@code append} method. Then the character at index k in >> + * the new character sequence is equal to the character at index >> k >> + * in the old character sequence, if k is less than n; >> + * otherwise, it is equal to the character at index k-n in the >> + * argument {@code sb}. >> + *

        >> + * >> + * @param sb the {@code StringBuilder} to append. >> + * @return a reference to this object. >> + */ >> + public synchronized StringBuffer append(StringBuilder sb) { >> + toStringCache = null; >> + super.append(sb); >> + return this; >> + } >> + >> /** >> * Appends the specified {@code StringBuffer} to this sequence. >> *

        >> diff --git a/src/java.base/share/classes/java/lang/StringBuilder.java >> b/src/java.base/share/classes/java/lang/StringBuilder.java >> index 40da2083c2..5ddd4fb5f9 100644 >> --- a/src/java.base/share/classes/java/lang/StringBuilder.java >> +++ b/src/java.base/share/classes/java/lang/StringBuilder.java >> @@ -199,6 +199,30 @@ public final class StringBuilder >> return this; >> } >> >> + /** >> + * Appends the specified {@code StringBuilder} to this sequence. >> + *

        >> + * The characters of the {@code StringBuilder} argument are appended, >> + * in order, to this sequence, increasing the >> + * length of this sequence by the length of the argument. >> + * If {@code sb} is {@code null}, then the four characters >> + * {@code "null"} are appended to this sequence. >> + *

        >> + * Let n be the length of this character sequence just prior >> to >> + * execution of the {@code append} method. Then the character at >> index >> + * k in the new character sequence is equal to the character >> at >> + * index k in the old character sequence, if k is less >> than >> + * n; otherwise, it is equal to the character at index >> k-n >> + * in the argument {@code sb}. >> + * >> + * @param sb the {@code StringBuilder} to append. >> + * @return a reference to this object. >> + */ >> + public StringBuilder append(StringBuilder sb) { >> + super.append(sb); >> + return this; >> + } >> + >> @Override >> public StringBuilder append(CharSequence s) { >> super.append(s); > > From forax at univ-mlv.fr Thu Jun 28 20:05:48 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 28 Jun 2018 22:05:48 +0200 (CEST) Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <2D66E03A-8F66-4535-8578-940049D3985D@gmail.com> Message-ID: <863334951.1527559.1530216348174.JavaMail.zimbra@u-pem.fr> no you can't, --add-modules requires the module to have a module-info, being an automatic module is not good enough. regards, R?mi ----- Mail original ----- > De: "Bernd Eckenfels" > ?: "core-libs-dev" > Envoy?: Jeudi 28 Juin 2018 17:34:35 > Objet: Re: Draft JEP proposal: JDK-8200758: Packaging Tool > you can jlink without any/complete module info files by specifying the module > names on the command line (--add-modules)as well. It produces a jre like > Directory including Java launcher which allows additions on the classpath. > > -- > https://Bernd.eckenfels.net > ________________________________ > From: core-libs-dev on behalf of Scott > Palmer > Sent: Thursday, June 28, 2018 2:32:13 PM > To: Kevin Rushforth > Cc: core-libs-dev at openjdk.java.net > Subject: Re: Draft JEP proposal: JDK-8200758: Packaging Tool > > Doesn?t jlink require a *fully* modularized application? I.e. no non-module > dependencies. > The packaging tool should work with all runnable Java applications, not just > fully modularized ones. > > Modularization seems to be a bit of an effort and is one of the main reasons my > application(s) are still stuck on Java 8. > > Scott > > > >> On Jun 27, 2018, at 6:30 PM, Kevin Rushforth wrote: >> >> We're aiming to get this into JDK 12 early enough so that an EA build would be >> available around the time JDK 11 ships. That will allow you to take a jlinked >> image with JDK 11 and package it up using (the EA) jpackager. >> >> We will create a development branch in the JDK sandbox [1] some time in the next >> week or so so you can follow the development. >> >> Also, thank you to those who have provided feedback. I'll reply to feedback soon >> and then incorporate it into an updated JEP. >> >> -- Kevin >> >> >> On 6/27/2018 3:09 AM, Buchberger, Joerg wrote: >>> Thanks for the info! And thanks for the efforts. [no irony, no sarcasm - I >>> really mean it] >>> >>> But, to sum up my comprehension... >>> >>> anyone who placed their bets on javapackager, starting with last LTS Java 8 >>> will be left in the rain with followup LTS Java 11, because their ain't neither >>> javapackager (anymore), nor jpackager (yet). >>> >>> Is this correct? >>> >>> So, strategic choice boils down to either throw away all work done based on >>> javapackager so far and the associated distribution concepts (reworking >>> everything from scratch) >>> or neglect Java 11 completely, thus placing all bets on jpackager really coming >>> w/ Java 12 or even waiting for Java 14 as next LTS thereafter. >>> >>> Bam(!), I think, I first need a tiny shot now ;-) and let that info sink in ... >>> >>> Cheers >>> J?rg >>> >>> >>> On 5/31/2018 0:10 AM, Rushforth, Kevin wrote: >>>> I would like to propose the following Draft JEP [1] for discussion. >>>> >>>> JDK-8200758: Packaging Tool >>>> >>>> This is intended as a JDK-scope replacement for the existing >>>> javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK >>>> releases), and was delivered as part of the JavaFX build. The >>>> javapackager tool has been removed from Oracle JDK 11 along with the >>>> removal of JavaFX. >>>> >>>> Comments on this JEP are welcome. It is currently not targeted for a >>>> specific release, but we are aiming for JDK 12. >>>> >>>> -- Kevin >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>>> From ecki at zusammenkunft.net Thu Jun 28 20:47:23 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 28 Jun 2018 22:47:23 +0200 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <863334951.1527559.1530216348174.JavaMail.zimbra@u-pem.fr> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <2D66E03A-8F66-4535-8578-940049D3985D@gmail.com> <863334951.1527559.1530216348174.JavaMail.zimbra@u-pem.fr> Message-ID: <5b35495b.1c69fb81.12892.af72@mx.google.com> You can add-modules from the JDK (only), so if you ??add-modules lang.base,JDK.jcmd,jdk.crypto.mscapi? you get a super compact JRE which still can start your app from the classpath. Gruss Bernd -- http://bernd.eckenfels.net Von: Remi Forax Gesendet: Donnerstag, 28. Juni 2018 22:05 An: Bernd Eckenfels Cc: core-libs-dev Betreff: Re: Draft JEP proposal: JDK-8200758: Packaging Tool no you can't, --add-modules requires the module to have a module-info, being an automatic module is not good enough. regards, R?mi ----- Mail original ----- > De: "Bernd Eckenfels" > ?: "core-libs-dev" > Envoy?: Jeudi 28 Juin 2018 17:34:35 > Objet: Re: Draft JEP proposal: JDK-8200758: Packaging Tool > you can jlink without any/complete module info files by specifying the module > names on the command line (--add-modules)as well. It produces a jre like > Directory including Java launcher which allows additions on the classpath. > > -- > https://Bernd.eckenfels.net > ________________________________ > From: core-libs-dev on behalf of Scott > Palmer > Sent: Thursday, June 28, 2018 2:32:13 PM > To: Kevin Rushforth > Cc: core-libs-dev at openjdk.java.net > Subject: Re: Draft JEP proposal: JDK-8200758: Packaging Tool > > Doesn?t jlink require a *fully* modularized application? I.e. no non-module > dependencies. > The packaging tool should work with all runnable Java applications, not just > fully modularized ones. > > Modularization seems to be a bit of an effort and is one of the main reasons my > application(s) are still stuck on Java 8. > > Scott > > > >> On Jun 27, 2018, at 6:30 PM, Kevin Rushforth wrote: >> >> We're aiming to get this into JDK 12 early enough so that an EA build would be >> available around the time JDK 11 ships. That will allow you to take a jlinked >> image with JDK 11 and package it up using (the EA) jpackager. >> >> We will create a development branch in the JDK sandbox [1] some time in the next >> week or so so you can follow the development. >> >> Also, thank you to those who have provided feedback. I'll reply to feedback soon >> and then incorporate it into an updated JEP. >> >> -- Kevin >> >> >> On 6/27/2018 3:09 AM, Buchberger, Joerg wrote: >>> Thanks for the info! And thanks for the efforts. [no irony, no sarcasm - I >>> really mean it] >>> >>> But, to sum up my comprehension... >>> >>> anyone who placed their bets on javapackager, starting with last LTS Java 8 >>> will be left in the rain with followup LTS Java 11, because their ain't neither >>> javapackager (anymore), nor jpackager (yet). >>> >>> Is this correct? >>> >>> So, strategic choice boils down to either throw away all work done based on >>> javapackager so far and the associated distribution concepts (reworking >>> everything from scratch) >>> or neglect Java 11 completely, thus placing all bets on jpackager really coming >>> w/ Java 12 or even waiting for Java 14 as next LTS thereafter. >>> >>> Bam(!), I think, I first need a tiny shot now ;-) and let that info sink in ... >>> >>> Cheers >>> J?rg >>> >>> >>> On 5/31/2018 0:10 AM, Rushforth, Kevin wrote: >>>> I would like to propose the following Draft JEP [1] for discussion. >>>> >>>> JDK-8200758: Packaging Tool >>>> >>>> This is intended as a JDK-scope replacement for the existing >>>> javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK >>>> releases), and was delivered as part of the JavaFX build. The >>>> javapackager tool has been removed from Oracle JDK 11 along with the >>>> removal of JavaFX. >>>> >>>> Comments on this JEP are welcome. It is currently not targeted for a >>>> specific release, but we are aiming for JDK 12. >>>> >>>> -- Kevin >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>>> From huizhe.wang at oracle.com Thu Jun 28 21:06:04 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 28 Jun 2018 14:06:04 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8190835: Subtraction with two javax.xml.datatype.Duration gives incorrect result Message-ID: <5B354DBC.1080800@oracle.com> Hi, Please review a quick fix for the first of JDK12/JAXP. JBS: https://bugs.openjdk.java.net/browse/JDK-8190835 webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8190835/webrev/ Thanks, Joe From forax at univ-mlv.fr Thu Jun 28 21:42:39 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 28 Jun 2018 23:42:39 +0200 (CEST) Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <5b35495b.1c69fb81.12892.af72@mx.google.com> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <2D66E03A-8F66-4535-8578-940049D3985D@gmail.com> <863334951.1527559.1530216348174.JavaMail.zimbra@u-pem.fr> <5b35495b.1c69fb81.12892.af72@mx.google.com> Message-ID: <2053830113.1537218.1530222159281.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Bernd Eckenfels" > ?: "core-libs-dev" > Envoy?: Jeudi 28 Juin 2018 22:47:23 > Objet: Re: Draft JEP proposal: JDK-8200758: Packaging Tool > You can add-modules from the JDK (only), so if you ??add-modules > lang.base,JDK.jcmd,jdk.crypto.mscapi? you get a super compact JRE which still > can start your app from the classpath. You mean you select by hand the modules of the JDK you need and use the classpath for your application, ok, that's work with jlink :) > > Gruss > Bernd R?mi > -- > http://bernd.eckenfels.net > > Von: Remi Forax > Gesendet: Donnerstag, 28. Juni 2018 22:05 > An: Bernd Eckenfels > Cc: core-libs-dev > Betreff: Re: Draft JEP proposal: JDK-8200758: Packaging Tool > > no you can't, > --add-modules requires the module to have a module-info, being an automatic > module is not good enough. > > regards, > R?mi > > ----- Mail original ----- >> De: "Bernd Eckenfels" >> ?: "core-libs-dev" >> Envoy?: Jeudi 28 Juin 2018 17:34:35 >> Objet: Re: Draft JEP proposal: JDK-8200758: Packaging Tool > >> you can jlink without any/complete module info files by specifying the module >> names on the command line (--add-modules)as well. It produces a jre like >> Directory including Java launcher which allows additions on the classpath. >> >> -- >> https://Bernd.eckenfels.net >> ________________________________ >> From: core-libs-dev on behalf of Scott >> Palmer >> Sent: Thursday, June 28, 2018 2:32:13 PM >> To: Kevin Rushforth >> Cc: core-libs-dev at openjdk.java.net >> Subject: Re: Draft JEP proposal: JDK-8200758: Packaging Tool >> >> Doesn?t jlink require a *fully* modularized application? I.e. no non-module >> dependencies. >> The packaging tool should work with all runnable Java applications, not just >> fully modularized ones. >> >> Modularization seems to be a bit of an effort and is one of the main reasons my >> application(s) are still stuck on Java 8. >> >> Scott >> >> >> >>> On Jun 27, 2018, at 6:30 PM, Kevin Rushforth wrote: >>> >>> We're aiming to get this into JDK 12 early enough so that an EA build would be >>> available around the time JDK 11 ships. That will allow you to take a jlinked >>> image with JDK 11 and package it up using (the EA) jpackager. >>> >>> We will create a development branch in the JDK sandbox [1] some time in the next >>> week or so so you can follow the development. >>> >>> Also, thank you to those who have provided feedback. I'll reply to feedback soon >>> and then incorporate it into an updated JEP. >>> >>> -- Kevin >>> >>> >>> On 6/27/2018 3:09 AM, Buchberger, Joerg wrote: >>>> Thanks for the info! And thanks for the efforts. [no irony, no sarcasm - I >>>> really mean it] >>>> >>>> But, to sum up my comprehension... >>>> >>>> anyone who placed their bets on javapackager, starting with last LTS Java 8 >>>> will be left in the rain with followup LTS Java 11, because their ain't neither >>>> javapackager (anymore), nor jpackager (yet). >>>> >>>> Is this correct? >>>> >>>> So, strategic choice boils down to either throw away all work done based on >>>> javapackager so far and the associated distribution concepts (reworking >>>> everything from scratch) >>>> or neglect Java 11 completely, thus placing all bets on jpackager really coming >>>> w/ Java 12 or even waiting for Java 14 as next LTS thereafter. >>>> >>>> Bam(!), I think, I first need a tiny shot now ;-) and let that info sink in ... >>>> >>>> Cheers >>>> J?rg >>>> >>>> >>>> On 5/31/2018 0:10 AM, Rushforth, Kevin wrote: >>>>> I would like to propose the following Draft JEP [1] for discussion. >>>>> >>>>> JDK-8200758: Packaging Tool >>>>> >>>>> This is intended as a JDK-scope replacement for the existing >>>>> javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK >>>>> releases), and was delivered as part of the JavaFX build. The >>>>> javapackager tool has been removed from Oracle JDK 11 along with the >>>>> removal of JavaFX. >>>>> >>>>> Comments on this JEP are welcome. It is currently not targeted for a >>>>> specific release, but we are aiming for JDK 12. >>>>> >>>>> -- Kevin >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 From alexfoster at hotmail.ca Tue Jun 26 19:04:52 2018 From: alexfoster at hotmail.ca (Alex Foster) Date: Tue, 26 Jun 2018 19:04:52 +0000 Subject: 8143850: retrofit ArrayDeque to implement List Message-ID: Hi, I did this awhile ago and I'd like to submit it. I believe it needs to be a new class because it changes the behavior of equals and hashcode. My implementation is based off of the java 8 version I think and it has changed a lot since then, I could add some of the improvements if you want. Also I'm not sure if we should make a new Deque + List interface for it to implement or not. Code: https://pastebin.com/suXMfMZW Bug: https://bugs.openjdk.java.net/browse/JDK-8143850 Previous discussion: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-February/024938.html Alex From jiangli.zhou at Oracle.COM Thu Jun 28 23:15:07 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Thu, 28 Jun 2018 16:15:07 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules Message-ID: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. Following are the details of system module archiving, which are duplicated in above bug report. --------------------------------------------------------------------------------------------------------------------------- Support archiving system module graph when the initial module is unnamed module from -cp currently. Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. Dump time system module object archiving ================================= At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. private static SystemModules archivedSystemModules; private static ModuleFinder archivedSystemModuleFinder; private static String archivedMainModule; The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. Runtime initialization from archived system module objects ============================================ VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. Thanks, Jiangli From lance.andersen at oracle.com Fri Jun 29 00:22:26 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 28 Jun 2018 20:22:26 -0400 Subject: RFR(JDK12/JAXP/java.xml) 8190835: Subtraction with two javax.xml.datatype.Duration gives incorrect result In-Reply-To: <5B354DBC.1080800@oracle.com> References: <5B354DBC.1080800@oracle.com> Message-ID: <833711D4-B8B5-4DF6-986F-577A7EE38B2D@oracle.com> Looks OK joe. > On Jun 28, 2018, at 5:06 PM, Joe Wang wrote: > > Hi, > > Please review a quick fix for the first of JDK12/JAXP. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8190835 > webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8190835/webrev/ > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From erik.joelsson at oracle.com Fri Jun 29 00:44:12 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 28 Jun 2018 17:44:12 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> Message-ID: <673771d5-0e61-6338-d222-9d1bd7f2826b@oracle.com> Build changes look good. /Erik On 2018-06-28 16:15, Jiangli Zhou wrote: > This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). > > The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. > > The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. > > webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 > > Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. > > Following are the details of system module archiving, which are duplicated in above bug report. > --------------------------------------------------------------------------------------------------------------------------- > Support archiving system module graph when the initial module is unnamed module from -cp currently. > > Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. > > Dump time system module object archiving > ================================= > At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. > > private static SystemModules archivedSystemModules; > private static ModuleFinder archivedSystemModuleFinder; > private static String archivedMainModule; > > The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. > > 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. > 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. > 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. > 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. > 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. > > Runtime initialization from archived system module objects > ============================================ > VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. > > If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. > > In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. > > Thanks, > Jiangli > > From xu.y.yin at oracle.com Fri Jun 29 00:52:20 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Fri, 29 Jun 2018 08:52:20 +0800 Subject: [JDK 11] RFR 8187069: The case auto failed with the "java.lang.ClassNotFoundException: IPv6NameserverPlatformParsingTest" exception In-Reply-To: <28421129-efb5-312e-4473-9c219f6d8b0d@oracle.com> References: <836D8991-E27C-4929-96FC-19444A48E422@oracle.com> <28421129-efb5-312e-4473-9c219f6d8b0d@oracle.com> Message-ID: Hi, Vyom Sure, fixed the tag order as you suggested, thanks New changes: diff -r 1308189b0848 test/jdk/com/sun/jndi/dns/Test6991580.java --- a/test/jdk/com/sun/jndi/dns/Test6991580.java Thu Jun 28 17:45:59 2018 -0700 +++ b/test/jdk/com/sun/jndi/dns/Test6991580.java Fri Jun 29 08:48:05 2018 +0800 @@ -33,10 +33,11 @@ /* * @test * @bug 6991580 8080108 8133035 - * @requires os.family != "windows" * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException * @modules java.desktop * jdk.naming.dns/com.sun.jndi.dns + * @requires os.family != "windows" + * @build IPv6NameserverPlatformParsingTest * @run main/manual Test6991580 */ Regards, Chris > On 28 Jun 2018, at 7:02 PM, vyom tewari wrote: > > Hi Chris, > > change looks good to me. My NetBeans always complains about tag order if it is not correct, as you adding the new tag i will suggest you to please fix the tag order as well. > > /* > * @test > * @bug 6991580 8080108 8133035 > * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException > * @modules java.desktop > * jdk.naming.dns/com.sun.jndi.dns > * @requires os.family != "windows" > * @build IPv6NameserverPlatformParsingTest > * @run main/manual Test6991580 > */ > > Thanks, > > Vyom > On Thursday 28 June 2018 07:31 AM, Chris Yin wrote: >> Please review below one line change for manual test case com/sun/jndi/dns/Test6991580.java to build test class automatically which will be used in manual steps, thanks >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8187069 >> >> Review change as below: >> >> diff -r 2d3e99a72541 test/jdk/com/sun/jndi/dns/Test6991580.java >> --- a/test/jdk/com/sun/jndi/dns/Test6991580.java Wed Jun 27 17:02:41 2018 -0700 >> +++ b/test/jdk/com/sun/jndi/dns/Test6991580.java Thu Jun 28 09:50:37 2018 +0800 >> @@ -37,6 +37,7 @@ >> * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException >> * @modules java.desktop >> * jdk.naming.dns/com.sun.jndi.dns >> + * @build IPv6NameserverPlatformParsingTest >> * @run main/manual Test6991580 >> */ >> >> Regards, >> Chris > From jiangli.zhou at oracle.com Fri Jun 29 01:02:03 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Thu, 28 Jun 2018 18:02:03 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: <673771d5-0e61-6338-d222-9d1bd7f2826b@oracle.com> References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> <673771d5-0e61-6338-d222-9d1bd7f2826b@oracle.com> Message-ID: <8BE8EC92-9624-4D8B-BF09-E7A5D41765DC@oracle.com> Hi Erik, Thank you for the quick review! Jiangli > On Jun 28, 2018, at 5:44 PM, Erik Joelsson wrote: > > Build changes look good. > > /Erik > > > On 2018-06-28 16:15, Jiangli Zhou wrote: >> This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). >> >> The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. >> >> The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. >> >> webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 >> >> Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. >> >> Following are the details of system module archiving, which are duplicated in above bug report. >> --------------------------------------------------------------------------------------------------------------------------- >> Support archiving system module graph when the initial module is unnamed module from -cp currently. >> >> Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. >> >> Dump time system module object archiving >> ================================= >> At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. >> >> private static SystemModules archivedSystemModules; >> private static ModuleFinder archivedSystemModuleFinder; >> private static String archivedMainModule; >> >> The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. >> >> 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. >> 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. >> 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. >> 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. >> 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. >> >> Runtime initialization from archived system module objects >> ============================================ >> VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. >> >> If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. >> >> In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. >> >> Thanks, >> Jiangli >> >> > From martinrb at google.com Fri Jun 29 02:43:55 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 28 Jun 2018 19:43:55 -0700 Subject: [PATCH] AbstractStringBuilder.append() In-Reply-To: References: Message-ID: On Thu, Jun 28, 2018 at 11:59 AM, Isaac Levy wrote: > And can't remove append(StringBuffer) because of binary compatibility? > That seems right. --- Also, the JIT can optijmize away any instanceof checks after inining when it sees append(stringBuilder). And any optimizations here are far less critical than they would be if they applied to each individual char. From vyom.tewari at oracle.com Fri Jun 29 03:13:36 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Fri, 29 Jun 2018 08:43:36 +0530 Subject: [JDK 11] RFR 8187069: The case auto failed with the "java.lang.ClassNotFoundException: IPv6NameserverPlatformParsingTest" exception In-Reply-To: References: <836D8991-E27C-4929-96FC-19444A48E422@oracle.com> <28421129-efb5-312e-4473-9c219f6d8b0d@oracle.com> Message-ID: looks good to me. Vyom On Friday 29 June 2018 06:22 AM, Chris Yin wrote: > Hi, Vyom > > Sure, fixed the tag order as you suggested, thanks > > New changes: > > diff -r 1308189b0848 test/jdk/com/sun/jndi/dns/Test6991580.java > --- a/test/jdk/com/sun/jndi/dns/Test6991580.javaThu Jun 28 17:45:59 > 2018 -0700 > +++ b/test/jdk/com/sun/jndi/dns/Test6991580.javaFri Jun 29 08:48:05 > 2018 +0800 > @@ -33,10 +33,11 @@ > ?/* > ??* @test > ??* @bug 6991580 8080108 8133035 > - * @requires os.family != "windows" > ??* @summary IPv6 Nameservers in resolv.conf throws NumberFormatException > ??* @modules java.desktop > ??*? ? ? ? ??jdk.naming.dns/com.sun.jndi.dns > + * @requires os.family != "windows" > + * @build IPv6NameserverPlatformParsingTest > ??* @run main/manual Test6991580 > ? */ > > Regards, > Chris > >> On 28 Jun 2018, at 7:02 PM, vyom tewari > > wrote: >> >> Hi Chris, >> >> change looks good to me. My NetBeans always complains about tag order >> if it is not correct, as you? adding the new tag i will suggest you >> to please fix the tag order as well. >> >> /* >> ?* @test >> ?* @bug 6991580 8080108 8133035 >> ?* @summary IPv6 Nameservers in resolv.conf throws NumberFormatException >> ?* @modules java.desktop >> ?*????????? jdk.naming.dns/com.sun.jndi.dns >> ?* @requires os.family != "windows" >> ?* @build IPv6NameserverPlatformParsingTest >> ?* @run main/manual Test6991580 >> ?*/ >> >> Thanks, >> >> Vyom >> >> On Thursday 28 June 2018 07:31 AM, Chris Yin wrote: >>> Please review below one line change for manual test case >>> com/sun/jndi/dns/Test6991580.java to build test class automatically >>> which will be used in manual steps, thanks >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8187069 >>> >>> Review change as below: >>> >>> diff -r 2d3e99a72541 test/jdk/com/sun/jndi/dns/Test6991580.java >>> --- a/test/jdk/com/sun/jndi/dns/Test6991580.javaWed Jun 27 17:02:41 >>> 2018 -0700 >>> +++ b/test/jdk/com/sun/jndi/dns/Test6991580.javaThu Jun 28 09:50:37 >>> 2018 +0800 >>> @@ -37,6 +37,7 @@ >>> ??* @summary IPv6 Nameservers in resolv.conf throws >>> NumberFormatException >>> ??* @modules java.desktop >>> ??*? ? ? ? ??jdk.naming.dns/com.sun.jndi.dns >>> + * @build IPv6NameserverPlatformParsingTest >>> ??* @run main/manual Test6991580 >>> ? */ >>> >>> Regards, >>> Chris >> > From martinrb at google.com Fri Jun 29 04:10:25 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 28 Jun 2018 21:10:25 -0700 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: Message-ID: Thanks, Alex. This has been on my todo list for a long time and I'm more overcommitted than ever. The difficult part is designing of the interface, but then there are many tests to write... I sort of liked ArrayDeque.asList() because at least then the API surface to be added is small. ArrayDeque has changed a little between jdk8 and now. Almost all the code could be shared with the List-implementing class - we don't want to introduce (too much) duplication. On Tue, Jun 26, 2018 at 12:04 PM, Alex Foster wrote: > Hi, > > > > > > I did this awhile ago and I'd like to submit it. > > I believe it needs to be a new class because it changes the behavior of > equals and hashcode. > > My implementation is based off of the java 8 version I think and it has > changed a lot since then, I could add some of the improvements if you want. > > Also I'm not sure if we should make a new Deque + List interface for it to > implement or not. > > > Code: > > https://pastebin.com/suXMfMZW > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8143850 > > Previous discussion: > > http:// > mail.openjdk.java.net/pipermail/core-libs-dev/2014-February/024938.html > > > Alex tp://mail.openjdk.java.net/pipermail/core-libs-dev/2014- > February/024938.html> > From xu.y.yin at oracle.com Fri Jun 29 05:48:50 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Fri, 29 Jun 2018 13:48:50 +0800 Subject: [JDK 11] RFR 8187069: The case auto failed with the "java.lang.ClassNotFoundException: IPv6NameserverPlatformParsingTest" exception In-Reply-To: References: <836D8991-E27C-4929-96FC-19444A48E422@oracle.com> <28421129-efb5-312e-4473-9c219f6d8b0d@oracle.com> Message-ID: <410AD2FB-CF0E-41FA-AFE5-C2537FF42E19@oracle.com> Thank you, Vyom Regards, Chris > On 29 Jun 2018, at 11:13 AM, vyom tewari wrote: > > looks good to me. > > Vyom > > On Friday 29 June 2018 06:22 AM, Chris Yin wrote: >> Hi, Vyom >> >> Sure, fixed the tag order as you suggested, thanks >> >> New changes: >> >> diff -r 1308189b0848 test/jdk/com/sun/jndi/dns/Test6991580.java >> --- a/test/jdk/com/sun/jndi/dns/Test6991580.java Thu Jun 28 17:45:59 2018 -0700 >> +++ b/test/jdk/com/sun/jndi/dns/Test6991580.java Fri Jun 29 08:48:05 2018 +0800 >> @@ -33,10 +33,11 @@ >> /* >> * @test >> * @bug 6991580 8080108 8133035 >> - * @requires os.family != "windows" >> * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException >> * @modules java.desktop >> * jdk.naming.dns/com.sun.jndi.dns >> + * @requires os.family != "windows" >> + * @build IPv6NameserverPlatformParsingTest >> * @run main/manual Test6991580 >> */ >> >> Regards, >> Chris >> >>> On 28 Jun 2018, at 7:02 PM, vyom tewari > wrote: >>> >>> Hi Chris, >>> >>> change looks good to me. My NetBeans always complains about tag order if it is not correct, as you adding the new tag i will suggest you to please fix the tag order as well. >>> >>> /* >>> * @test >>> * @bug 6991580 8080108 8133035 >>> * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException >>> * @modules java.desktop >>> * jdk.naming.dns/com.sun.jndi.dns >>> * @requires os.family != "windows" >>> * @build IPv6NameserverPlatformParsingTest >>> * @run main/manual Test6991580 >>> */ >>> >>> Thanks, >>> >>> Vyom >>> On Thursday 28 June 2018 07:31 AM, Chris Yin wrote: >>>> Please review below one line change for manual test case com/sun/jndi/dns/Test6991580.java to build test class automatically which will be used in manual steps, thanks >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8187069 >>>> >>>> Review change as below: >>>> >>>> diff -r 2d3e99a72541 test/jdk/com/sun/jndi/dns/Test6991580.java >>>> --- a/test/jdk/com/sun/jndi/dns/Test6991580.java Wed Jun 27 17:02:41 2018 -0700 >>>> +++ b/test/jdk/com/sun/jndi/dns/Test6991580.java Thu Jun 28 09:50:37 2018 +0800 >>>> @@ -37,6 +37,7 @@ >>>> * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException >>>> * @modules java.desktop >>>> * jdk.naming.dns/com.sun.jndi.dns >>>> + * @build IPv6NameserverPlatformParsingTest >>>> * @run main/manual Test6991580 >>>> */ >>>> >>>> Regards, >>>> Chris >>> >> > From martinrb at google.com Fri Jun 29 14:09:00 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 29 Jun 2018 07:09:00 -0700 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: Message-ID: On Thu, Jun 28, 2018 at 11:42 PM, Alex Foster wrote: > Another question is whether or not it should allow null elements. > ArrayDeque currently doesn't but I think it would be useful to support > them, which makes having an asList() view or sharing code harder. > JDK has moved away from null elements in collections. We want the new collection to implement both List and Deque, and the spec says """ While Deque implementations are not strictly required to prohibit the insertion of null elements, they are strongly encouraged to do so. Users of any Dequeimplementations that do allow null elements are strongly encouraged *not* to take advantage of the ability to insert nulls. This is so because null is used as a special return value by various methods to indicate that the deque is empty. """ --- Another idea is to simply add all of those List-like methods to ArrayDeque without actually implementing List. Then we have maximal efficiency (no wrappers), we don't have to come up with a new name for the class, and a user who wants a genuine 100% List view can do subList(deq, 0, deq.size()). In the past we've been reluctant to do such a strange thing ... From volker.simonis at gmail.com Fri Jun 29 14:53:53 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 29 Jun 2018 16:53:53 +0200 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors Message-ID: Hi, can I please have a review for the following change which saves ExceptionInInitializerError thrown during class initialization and chains them as cause into potential NoClassDefFoundErrors for the same class. We are using this features since years in our commercial SAP JVM and it proved extremely useful for detecting and fixing errors especially in big deployments. This is a follow-up on a discussion previously started by Goetz [1]. His first proposal (which is close to our current, internal implementation) inserted an additional field into java.lang.Class objects to save potential ExceptionInInitializerErrors. This was considered to much overhead in the initial discussion [1]. http://cr.openjdk.java.net/~simonis/webrevs/2018/8203826.v2/ https://bugs.openjdk.java.net/browse/JDK-8203826 So in this change, I've completely re-implemented the feature by using a java.lang.Hashtable which is attached to the ClassLoaderData object. The Hashtable is lazily created when the first ExceptionInInitializerError is thrown and maps the Class which triggered the ExceptionInInitializerError during the execution of its static initializer to the corresponding ExceptionInInitializerError. If the same class will be accessed once again, this will directly lead to a plain NoClassDefFoundError (as per the JVMS, 5.5 Initialization) because the static initializer won't be executed a second time. Until now, this NoClassDefFoundError wasn't linked in any way to the root cause of the problem (i.e. the first ExceptionInInitializerError together with the chained exception that happened during the execution of the static initializer). With this change, the NoClassDefFoundError will now chain the initial ExceptionInInitializerError as cause, making it much easier to detect the problem which lead to the NoClassDefFoundError. Following is an example from the new JTreg tests which comes which this change to demonstrate the feature. Until know, a typical stack trace from a NoClassDefFoundError looked as follows: java.lang.NoClassDefFoundError: Could not initialize class NoClassDefFound$ClassWithFailedInitializer at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:291) at NoClassDefFound.main(NoClassDefFound.java:38) With this change, the same stack trace now looks as follows: java.lang.NoClassDefFoundError: Could not initialize class NoClassDefFound$ClassWithFailedInitializer at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:315) at NoClassDefFound.main(NoClassDefFound.java:38) Caused by: java.lang.ExceptionInInitializerError at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at java.base/java.lang.Class.newInstance(Class.java:584) at NoClassDefFound$ClassWithFailedInitializer.(NoClassDefFound.java:20) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:315) at NoClassDefFound.main(NoClassDefFound.java:30) Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 1 at NoClassDefFound$A.(NoClassDefFound.java:9) ... 9 more As you can see, the reason for the NoClassDefFoundError when accessing the class 'NoClassDefFound$ClassWithFailedInitializer' is actually not even in the class or its static initializer itself, but in the class 'NoClassDefFound$A' which is a base class of 'NoClassDefFound$ClassWithFailedInitializer'. This is not easily detectible from the old, plain NoClassDefFoundError. As I wrote, the only overhead we have with the new implementation is an additional OopHandle field per ClassLoaderData which I think is acceptable. The Hashtable object itself is only created lazily, after the first occurrence of an ExceptionInInitializerError in the corresponding class loader. The whole Hashtable creation and storing/quering of ExceptionInInitializerErrors in ClassLoaderData::record_init_exception()/ClassLoaderData::query_init_exception() is optional in the sense that any errors/exceptions occurring during the execution of these functions are ignored and cleared. Finally, we also take care to recursively convert all native backtraces in the stored ExceptionInInitializerErrors (and their suppressed and chained exceptions) into symbolic stack traces in order to avoid holding references to classes and prevent their unloading. This is implemented in the new private, static method java.lang.Throwable::removeNativeBacktrace() which is called for each ExceptionInInitializerError before it is stored in the Hashtable. Thank you and best regards, Volker [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-June/028310.html From sean.coffey at oracle.com Fri Jun 29 15:34:25 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 29 Jun 2018 16:34:25 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> <5B350AE5.3080401@oracle.com> Message-ID: I've introduced a new test helper class in the jdk/test/lib/jfr directory to help with the dual test nature of the new tests. It's helped alot with test code duplication. Looked at use of @DataAmount(DataAmount.BITS) also. Not sure if it's fits. The output is displayed in units like "KiB" - not the normal when examining key lengths used in X509Certificates. i.e a 2048 bit key gets displayed as "2 KiB" - I'd prefer to keep the 2048 display version. new webrev at: http://cr.openjdk.java.net/~coffeys/webrev.8148188.v4/webrev/ Regards, Sean. On 28/06/18 17:59, Se?n Coffey wrote: > Comments inline. > > > On 28/06/2018 17:20, Erik Gahlin wrote: >> It's sufficient if an event object escapes to another method >> (regardless if JFR is enabled or not). >> >> Some more feedback: >> >> Rename event jdk.CertChain to jdk.CertificateChain >> Rename event jdk.X509Cert to jdk.X509Certificate >> Rename field certChain to certificateChain. >> Rename field serialNum to serialNumber > all above done. >> Rename field algId to AlgorithmicId or AlgorithmicID > maybe "algorithm" works here also ? >> Rename @Label("Ciphersuite") to @Label("Cipher Suite") >> Rename @Label("Cert Chain") to @Label("Certificate Chain") >> Rename @Label("Property Name") to "Name" or "Key" if that makes sense >> in the context? >> Rename @Label("Property Value") to "Value". >> Put events in a subcategory, i.e @Category({"Java Development Kit", >> "Security"}) > done. >> Make classes final. > done. I had thought that the JFR mechanism required non-final classes. >> What is the unit of the key in X509Certificate event? Bits? If that >> is the case, use @DataAmount(DataAmount.BITS) > Yes - it's essentially the bit length of the keys used. Let me look > into that annotation usage. >> @Label("Serial numbers forming chain of trust") should not be a >> sentence. How about "Serial Numbers"? >> >> I think the tests are hard to read when two things are tested at the >> same time. It is also likely that if a test fail due to logging >> issues, it will be assigned to JFR because of the test name, even >> thought the issues is not JFR related. > I think we're always going to have some ownership issues when tests > serve a dual purpose. I still think it makes sense to keep the test > logic in one place. Any failures in these tests will most likely be > owned by security team. (moving the tests to security directory is > also an option) >> >> If you want to reuse code between tests, I would put it in testlibrary. > Let me check if there's any common patterns that could be added to the > testlibary. > > Thanks, > Sean. >> >> Erik >> >>> Thanks for the update Erik. By default I'm proposing that the new >>> JFR Events and Logger be disabled. As a result the event class >>> shouldn't escape. If performance metrics highlight an issue, we >>> should revisit. >>> >>> regards, >>> Sean. >>> >>> >>> On 27/06/2018 20:57, Erik Gahlin wrote: >>>> On 2018-06-27 21:14, Se?n Coffey wrote: >>>>> >>>>> >>>>> On 27/06/2018 19:57, Xuelei Fan wrote: >>>>>> Hi Sean, >>>>>> >>>>>> I may reply in several replies. >>>>>> >>>>>> PKIXMasterCertPathValidator.java >>>>>> -------------------------------- >>>>>> + CertChainEvent cce = new CertChainEvent(); >>>>>> + if(cce.isEnabled() || EventHelper.loggingSecurity()) { >>>>>> + String c = reversedCertList.stream() >>>>>> + .map(x -> x.getSerialNumber().toString(16)) >>>>>> + .collect(Collectors.joining(", ")); >>>>>> + EventHelper.commitCertChainEvent(cce, c); >>>>>> + } >>>>>> >>>>>> No matter the event or the JFR mechanism is enabled or not, the >>>>>> above code will create a new instance. Is the return value of >>>>>> cce.isEnabled() dynamically changed or static? >>>>> This is a requirement from the JFR framework. All their >>>>> event.isEnabled calls are instance methods and follow a similar >>>>> pattern. The JFR team assure me that the JIT can optimize away >>>>> such calls. >>>> >>>> The JIT will most likely not be able to optimize if the event class >>>> escapes. >>>> >>>> From a JFR perspective, this would be the preferred layout: >>>> >>>> EventA eventA= new EventA(); >>>> eventA.value = this.value; >>>> eventA.commit(); >>>> >>>> and then do logging separately: >>>> >>>> System.Logger.log(DEBUG, this.value) >>>> >>>> With this layout, the JIT will remove the allocation and dead store. >>>> >>>> If it is expensive to gather the data for the event, like the >>>> CertChainEvent above, the following pattern should be used. >>>> >>>> EventB eventB= new EventB (); >>>> if (eventB.shouldCommit()) { >>>> eventB.value = calculateValue(); >>>> eventB .commit(); >>>> } >>>> >>>> If JFR is not enabled, shouldCommit returns false and the JIT >>>> should be able to remove the allocation. This will be best from a >>>> performance point of view, and also in my opinion from a >>>> maintenance and readability perspective. Others may disagree. >>>> >>>> Erik >>>> >>>>>> >>>>>> Is there a need to support both logging and JFR? I'm new to >>>>>> record events. I did not get the point to use both. >>>>> I was initially hoping to concentrate on just JFR events but I got >>>>> strong feedback to also consider use of Logger (esp. in cases >>>>> where the jdk.jfr module is not available) >>>>> >>>>> regards, >>>>> Sean. >>>>> >>>>>> >>>>>> Thanks, >>>>>> Xuelei >>>>>> >>>>>> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>>>>>> Erik, >>>>>>> >>>>>>> I rebased the patch with TLS v1.3 integration today. I hadn't >>>>>>> realized how much the handshaker code had changed. Hopefully, >>>>>>> I'll get a review from security dev team on that front. >>>>>>> >>>>>>> Regards the JFR semantics, I believe the edits should match >>>>>>> majority of requests . I still have a preference for the test >>>>>>> infra design for these new logger/JFR tests used in version 1 of >>>>>>> webrev. I think it makes sense to keep the test functionality >>>>>>> together - no sense in separating them out into different >>>>>>> components IMO. Also, some of the edits to the JFR testing were >>>>>>> made to test for the new dual log/record functionality. I might >>>>>>> catch up with you tomorrow to see what the best arrangement >>>>>>> would be. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>>>>>> >>>>>>> regards, >>>>>>> Sean. >>>>>>> >>>>>>> >>>>>>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>>>>>> Many thanks for the review comments Erik. Replies inline. >>>>>>>> >>>>>>>> >>>>>>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>>>>>> Hi Sean, >>>>>>>>> >>>>>>>>> Some of the changes in the webrev belongs to JDK-8203629 and >>>>>>>>> should be removed for clarity. >>>>>>>>> >>>>>>>>> Some initial comments: >>>>>>>>> >>>>>>>>> default.jfc, profile.jfr: >>>>>>>>> The events should not have control="enable-exceptions". The >>>>>>>>> purpose of the control attribute is so to provide >>>>>>>>> parameterized configuration of events for JMC. As it is now, >>>>>>>>> the security events will be enabled when a user turns on the >>>>>>>>> exception events. >>>>>>>> Makes sense. I'll remove that parameter. >>>>>>>>> >>>>>>>>> X509CertEvent: >>>>>>>>> You should use milliseconds since epoch to represent a date >>>>>>>>> instead of a string value, i.e. >>>>>>>>> >>>>>>>>> @Label("Valid From") >>>>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>>> public long validFrom; >>>>>>>>> >>>>>>>>> @Label("Valid Until") >>>>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>>> public long validUntil; >>>>>>>>> >>>>>>>> The CertificateValidity class operates on Date Object values. >>>>>>>> I'll work with the Date.getTime() method then (and update the >>>>>>>> Logger formatting) >>>>>>>>> This following annotation adds little value >>>>>>>>> >>>>>>>>> @Description("Details of Security Property") >>>>>>>>> >>>>>>>>> I would either remove the annotation, or provide information >>>>>>>>> that helps a user understand the event. For instance, what is >>>>>>>>> X509, and what kind of certificates are we talking about? >>>>>>>> Yes - that looks like the wrong Description. I'll review all of >>>>>>>> these fields. >>>>>>>>> >>>>>>>>> @Category("Java Application") >>>>>>>>> >>>>>>>>> I'm a bit worried that we will pollute the "Java Application" >>>>>>>>> namespace if we add lots of JDK events in that category. We >>>>>>>>> may want to add a new top level category "Java Development >>>>>>>>> Kit", analogous to the "Java Virtual Machine" category, and >>>>>>>>> put all security related events in subcategory, perhaps called >>>>>>>>> "Security". >>>>>>>> Yes - Open to suggestions. "Security" sounds like a good >>>>>>>> suggestion. >>>>>>>>> >>>>>>>>> @Label("X509Cert") >>>>>>>>> >>>>>>>>> The label should be human readable name, with spaces, title >>>>>>>>> cased etc. I would recommend "X.509 Certificate". In general, >>>>>>>>> avoid abbreviations like "certs" and instead use labels such >>>>>>>>> as "Certificate Chain". The label should be short and not a >>>>>>>>> full sentence. >>>>>>>>> >>>>>>>>> For details see, >>>>>>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>>>>>> >>>>>>>>> I think it would be good to separate testing of JFR and >>>>>>>>> logging into different files / tests. I would prefer that the >>>>>>>>> test name is the same as the event prefixed with "Test", i.e >>>>>>>>> TestX509CertificateEvent, as this is the pattern used by other >>>>>>>>> JFR tests. >>>>>>>>> >>>>>>>> I'll take a look at that pattern again. I've separated out the >>>>>>>> current tests into an (a) outer test to analyze the logger >>>>>>>> output and (b) the inner test which checks for JFR correctness. >>>>>>>> I did include extra logic to make sure that the EventHelper >>>>>>>> configuration was working correctly. "Events.assertField" looks >>>>>>>> very helpful. Thanks for the pointer. >>>>>>>> >>>>>>>> Let me take on board the suggestions below and get a new webrev >>>>>>>> out on Tuesday. >>>>>>>> >>>>>>>> regards, >>>>>>>> Sean. >>>>>>>> >>>>>>>>> I reworked one of the tests to how I like to see it: >>>>>>>>> >>>>>>>>> /* >>>>>>>>> * @test >>>>>>>>> * @key jfr >>>>>>>>> * @library /test/lib >>>>>>>>> * @run main/othervm >>>>>>>>> jdk.jfr.event.security.TestX509CertificateEvent >>>>>>>>> */ >>>>>>>>> public class TestX509CertificateEvent { >>>>>>>>> >>>>>>>>> private static final String CERTIFICATE_1 = "..."; >>>>>>>>> private static final String CERTIFICATE_2 = "..."; >>>>>>>>> >>>>>>>>> public static void main(String... args) throws >>>>>>>>> CertificateException { >>>>>>>>> >>>>>>>>> Recording r = new Recording(); >>>>>>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>>>>>> r.start(); >>>>>>>>> >>>>>>>>> CertificateFactory cf = >>>>>>>>> CertificateFactory.getInstance("X.509"); >>>>>>>>> cf.generateCertificate(new >>>>>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>>> cf.generateCertificate(new >>>>>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>>> >>>>>>>>> // Make sure only one event per certificate >>>>>>>>> cf.generateCertificate(new >>>>>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>>> cf.generateCertificate(new >>>>>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>>> >>>>>>>>> r.stop(); >>>>>>>>> >>>>>>>>> List events = Events.fromRecording(r); >>>>>>>>> Asserts.assertEquals(events.size(), 2, "Expected two >>>>>>>>> X.509 Certificate events"); >>>>>>>>> >>>>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>>> "RSA", 2048); >>>>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>>> "RSA", 2048); >>>>>>>>> } >>>>>>>>> >>>>>>>>> private static void assertEvent(List >>>>>>>>> events, String certID, String algId, String subject, >>>>>>>>> String issuer, String keyType, int length) throws >>>>>>>>> Exception { >>>>>>>>> >>>>>>>>> for (RecordedEvent e : events) { >>>>>>>>> if (e.getString("serialNumber").equals(certID)) { >>>>>>>>> Events.assertField(e, "algId").equal(algId); >>>>>>>>> ... >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> System.out.println(events); >>>>>>>>> throw new Exception("Could not find event with Cert >>>>>>>>> ID: " + certID); >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> The reworked example uses the Events.assertField method, which >>>>>>>>> will give context if the assertion fails. Keeping the test >>>>>>>>> simple, means it can be analyzed quickly if it fails in >>>>>>>>> testing. There is no new test framework to learn, or methods >>>>>>>>> to search for, and it is usually not hard to determine if the >>>>>>>>> failure is product, test or infrastructure related, and what >>>>>>>>> component (team) should be assigned the issue. The use of >>>>>>>>> EventNames.X509Certificate means all occurrences of the event >>>>>>>>> can be tracked done in an IDE using find by reference. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Erik >>>>>>>>> >>>>>>>>>> Following on from the recent JDK-8203629 code review, I'd >>>>>>>>>> like to propose enhancements on how we can record events in >>>>>>>>>> security libs. The introduction of the JFR libraries can give >>>>>>>>>> us much better ways of examining JDK actions. For the initial >>>>>>>>>> phase, I'm looking to enhance some key security library >>>>>>>>>> events in JDK 11 so that they can be either recorded to JFR, >>>>>>>>>> logged to a traditional logger, or both. >>>>>>>>>> >>>>>>>>>> Examples of how useful JFR recordings could be can be seen >>>>>>>>>> here : >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>>>>>> >>>>>>>>>> securityProp_2.png gives an example of how the JFR recording >>>>>>>>>> can be queried to quickly locate events of interest (in this >>>>>>>>>> case, code setting the jdk.tls.* Security properties). I >>>>>>>>>> still need to clean up the TLSEvents testcase to improve test >>>>>>>>>> coverage and hope to do that in coming days. >>>>>>>>>> >>>>>>>>>> JBS record : >>>>>>>>>> * https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>>>>>> >>>>>>>>>> webrev : >>>>>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> >> > From paul.sandoz at oracle.com Fri Jun 29 16:48:28 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 Jun 2018 09:48:28 -0700 Subject: [PATCH] AbstractStringBuilder.append() In-Reply-To: References: Message-ID: <66BCA06A-D1BC-4F67-9331-BD3874E47493@oracle.com> Hard to reconstruct the culture memory around this. I suspect such a new method was not considered necessary when StringBuilder was added to Java 1.5 given the CharSequence accepting method, and performance was not considered a concern, and I doubt it's a concern now. (I don?t think there is any circularity that would result from bridge methods if such a method was added.) Paul. > On Jun 27, 2018, at 10:22 PM, Martin Buchholz wrote: > > I'm fairly sure the append(StringBuilder) overloads were left out > intentionally. > > On Wed, Jun 27, 2018 at 8:57 PM, Isaac Levy wrote: > >> AbstractStringBuilder already has append(). This patch >> adds append(). >> >> The new method improves parity between the two classes. In addition, >> combining StringBuilders is presumably common. append() >> has a couple insteadof checks, which this new method skips. >> >> -Isaac >> >> >> >> >> diff --git a/src/java.base/share/classes/java/lang/ >> AbstractStringBuilder.java >> b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> index 2ef3e53256..1fe89bb92a 100644 >> --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> @@ -543,6 +543,11 @@ abstract class AbstractStringBuilder implements >> Appendable, CharSequence { >> return this.append((AbstractStringBuilder)sb); >> } >> >> + // Documentation in subclasses because of synchro difference >> + public AbstractStringBuilder append(StringBuilder sb) { >> + return this.append((AbstractStringBuilder)sb); >> + } >> + >> /** >> * @since 1.8 >> */ >> diff --git a/src/java.base/share/classes/java/lang/StringBuffer.java >> b/src/java.base/share/classes/java/lang/StringBuffer.java >> index e597a8112e..613ba90c25 100644 >> --- a/src/java.base/share/classes/java/lang/StringBuffer.java >> +++ b/src/java.base/share/classes/java/lang/StringBuffer.java >> @@ -313,6 +313,33 @@ import jdk.internal.HotSpotIntrinsicCandidate; >> return this; >> } >> >> + /** >> + * Appends the specified {@code StringBuilder} to this sequence. >> + *

        >> + * The characters of the {@code StringBuilder} argument are appended, >> + * in order, to the contents of this {@code StringBuffer}, increasing >> the >> + * length of this {@code StringBuffer} by the length of the argument. >> + * If {@code sb} is {@code null}, then the four characters >> + * {@code "null"} are appended to this {@code StringBuffer}. >> + *

        >> + * Let n be the length of the old character sequence, the one >> + * contained in the {@code StringBuffer} just prior to execution of >> the >> + * {@code append} method. Then the character at index k in >> + * the new character sequence is equal to the character at index >> k >> + * in the old character sequence, if k is less than n; >> + * otherwise, it is equal to the character at index k-n in the >> + * argument {@code sb}. >> + *

        >> + * >> + * @param sb the {@code StringBuilder} to append. >> + * @return a reference to this object. >> + */ >> + public synchronized StringBuffer append(StringBuilder sb) { >> + toStringCache = null; >> + super.append(sb); >> + return this; >> + } >> + >> /** >> * Appends the specified {@code StringBuffer} to this sequence. >> *

        >> diff --git a/src/java.base/share/classes/java/lang/StringBuilder.java >> b/src/java.base/share/classes/java/lang/StringBuilder.java >> index 40da2083c2..5ddd4fb5f9 100644 >> --- a/src/java.base/share/classes/java/lang/StringBuilder.java >> +++ b/src/java.base/share/classes/java/lang/StringBuilder.java >> @@ -199,6 +199,30 @@ public final class StringBuilder >> return this; >> } >> >> + /** >> + * Appends the specified {@code StringBuilder} to this sequence. >> + *

        >> + * The characters of the {@code StringBuilder} argument are appended, >> + * in order, to this sequence, increasing the >> + * length of this sequence by the length of the argument. >> + * If {@code sb} is {@code null}, then the four characters >> + * {@code "null"} are appended to this sequence. >> + *

        >> + * Let n be the length of this character sequence just prior to >> + * execution of the {@code append} method. Then the character at index >> + * k in the new character sequence is equal to the character at >> + * index k in the old character sequence, if k is less >> than >> + * n; otherwise, it is equal to the character at index >> k-n >> + * in the argument {@code sb}. >> + * >> + * @param sb the {@code StringBuilder} to append. >> + * @return a reference to this object. >> + */ >> + public StringBuilder append(StringBuilder sb) { >> + super.append(sb); >> + return this; >> + } >> + >> @Override >> public StringBuilder append(CharSequence s) { >> super.append(s); >> From huizhe.wang at oracle.com Fri Jun 29 17:17:53 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 29 Jun 2018 10:17:53 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8190835: Subtraction with two javax.xml.datatype.Duration gives incorrect result In-Reply-To: <833711D4-B8B5-4DF6-986F-577A7EE38B2D@oracle.com> References: <5B354DBC.1080800@oracle.com> <833711D4-B8B5-4DF6-986F-577A7EE38B2D@oracle.com> Message-ID: <5B3669C1.6090105@oracle.com> Thanks Lance! Pushed. -Joe On 6/28/18, 5:22 PM, Lance Andersen wrote: > Looks OK joe. > > >> On Jun 28, 2018, at 5:06 PM, Joe Wang > > wrote: >> >> Hi, >> >> Please review a quick fix for the first of JDK12/JAXP. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8190835 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8190835/webrev/ >> >> >> Thanks, >> Joe > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From isaac.r.levy at gmail.com Fri Jun 29 20:02:41 2018 From: isaac.r.levy at gmail.com (Isaac Levy) Date: Fri, 29 Jun 2018 16:02:41 -0400 Subject: [PATCH] AbstractStringBuilder.append() In-Reply-To: <66BCA06A-D1BC-4F67-9331-BD3874E47493@oracle.com> References: <66BCA06A-D1BC-4F67-9331-BD3874E47493@oracle.com> Message-ID: Would this snippet end up merging two StringBuilders? String x = "bar" + "foo"; x += "baz" + "biz"; I had trouble reading stringopts.cpp. I was thinking this append might be common due to implicit StringBuilder creation. Isaac On Fri, Jun 29, 2018 at 12:48 PM, Paul Sandoz wrote: > Hard to reconstruct the culture memory around this. I suspect such a new method was not considered necessary when StringBuilder was added to Java 1.5 given the CharSequence accepting method, and performance was not considered a concern, and I doubt it's a concern now. > > (I don?t think there is any circularity that would result from bridge methods if such a method was added.) > > Paul. > >> On Jun 27, 2018, at 10:22 PM, Martin Buchholz wrote: >> >> I'm fairly sure the append(StringBuilder) overloads were left out >> intentionally. >> >> On Wed, Jun 27, 2018 at 8:57 PM, Isaac Levy wrote: >> >>> AbstractStringBuilder already has append(). This patch >>> adds append(). >>> >>> The new method improves parity between the two classes. In addition, >>> combining StringBuilders is presumably common. append() >>> has a couple insteadof checks, which this new method skips. >>> >>> -Isaac >>> >>> >>> >>> >>> diff --git a/src/java.base/share/classes/java/lang/ >>> AbstractStringBuilder.java >>> b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>> index 2ef3e53256..1fe89bb92a 100644 >>> --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>> +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>> @@ -543,6 +543,11 @@ abstract class AbstractStringBuilder implements >>> Appendable, CharSequence { >>> return this.append((AbstractStringBuilder)sb); >>> } >>> >>> + // Documentation in subclasses because of synchro difference >>> + public AbstractStringBuilder append(StringBuilder sb) { >>> + return this.append((AbstractStringBuilder)sb); >>> + } >>> + >>> /** >>> * @since 1.8 >>> */ >>> diff --git a/src/java.base/share/classes/java/lang/StringBuffer.java >>> b/src/java.base/share/classes/java/lang/StringBuffer.java >>> index e597a8112e..613ba90c25 100644 >>> --- a/src/java.base/share/classes/java/lang/StringBuffer.java >>> +++ b/src/java.base/share/classes/java/lang/StringBuffer.java >>> @@ -313,6 +313,33 @@ import jdk.internal.HotSpotIntrinsicCandidate; >>> return this; >>> } >>> >>> + /** >>> + * Appends the specified {@code StringBuilder} to this sequence. >>> + *

        >>> + * The characters of the {@code StringBuilder} argument are appended, >>> + * in order, to the contents of this {@code StringBuffer}, increasing >>> the >>> + * length of this {@code StringBuffer} by the length of the argument. >>> + * If {@code sb} is {@code null}, then the four characters >>> + * {@code "null"} are appended to this {@code StringBuffer}. >>> + *

        >>> + * Let n be the length of the old character sequence, the one >>> + * contained in the {@code StringBuffer} just prior to execution of >>> the >>> + * {@code append} method. Then the character at index k in >>> + * the new character sequence is equal to the character at index >>> k >>> + * in the old character sequence, if k is less than n; >>> + * otherwise, it is equal to the character at index k-n in the >>> + * argument {@code sb}. >>> + *

        >>> + * >>> + * @param sb the {@code StringBuilder} to append. >>> + * @return a reference to this object. >>> + */ >>> + public synchronized StringBuffer append(StringBuilder sb) { >>> + toStringCache = null; >>> + super.append(sb); >>> + return this; >>> + } >>> + >>> /** >>> * Appends the specified {@code StringBuffer} to this sequence. >>> *

        >>> diff --git a/src/java.base/share/classes/java/lang/StringBuilder.java >>> b/src/java.base/share/classes/java/lang/StringBuilder.java >>> index 40da2083c2..5ddd4fb5f9 100644 >>> --- a/src/java.base/share/classes/java/lang/StringBuilder.java >>> +++ b/src/java.base/share/classes/java/lang/StringBuilder.java >>> @@ -199,6 +199,30 @@ public final class StringBuilder >>> return this; >>> } >>> >>> + /** >>> + * Appends the specified {@code StringBuilder} to this sequence. >>> + *

        >>> + * The characters of the {@code StringBuilder} argument are appended, >>> + * in order, to this sequence, increasing the >>> + * length of this sequence by the length of the argument. >>> + * If {@code sb} is {@code null}, then the four characters >>> + * {@code "null"} are appended to this sequence. >>> + *

        >>> + * Let n be the length of this character sequence just prior to >>> + * execution of the {@code append} method. Then the character at index >>> + * k in the new character sequence is equal to the character at >>> + * index k in the old character sequence, if k is less >>> than >>> + * n; otherwise, it is equal to the character at index >>> k-n >>> + * in the argument {@code sb}. >>> + * >>> + * @param sb the {@code StringBuilder} to append. >>> + * @return a reference to this object. >>> + */ >>> + public StringBuilder append(StringBuilder sb) { >>> + super.append(sb); >>> + return this; >>> + } >>> + >>> @Override >>> public StringBuilder append(CharSequence s) { >>> super.append(s); >>> > From jonathan.gibbons at oracle.com Fri Jun 29 21:50:26 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 29 Jun 2018 14:50:26 -0700 Subject: [PATCH] AbstractStringBuilder.append() In-Reply-To: References: <66BCA06A-D1BC-4F67-9331-BD3874E47493@oracle.com> Message-ID: <5B36A9A2.80708@oracle.com> In that specific snippet, I would expect the compiler (javac) to fold the constants together. -- Jon On 06/29/2018 01:02 PM, Isaac Levy wrote: > Would this snippet end up merging two StringBuilders? > > String x = "bar" + "foo"; > x += "baz" + "biz"; > > I had trouble reading stringopts.cpp. I was thinking this append might > be common due to implicit StringBuilder creation. > > Isaac > > > On Fri, Jun 29, 2018 at 12:48 PM, Paul Sandoz wrote: >> Hard to reconstruct the culture memory around this. I suspect such a new method was not considered necessary when StringBuilder was added to Java 1.5 given the CharSequence accepting method, and performance was not considered a concern, and I doubt it's a concern now. >> >> (I don?t think there is any circularity that would result from bridge methods if such a method was added.) >> >> Paul. >> >>> On Jun 27, 2018, at 10:22 PM, Martin Buchholz wrote: >>> >>> I'm fairly sure the append(StringBuilder) overloads were left out >>> intentionally. >>> >>> On Wed, Jun 27, 2018 at 8:57 PM, Isaac Levy wrote: >>> >>>> AbstractStringBuilder already has append(). This patch >>>> adds append(). >>>> >>>> The new method improves parity between the two classes. In addition, >>>> combining StringBuilders is presumably common. append() >>>> has a couple insteadof checks, which this new method skips. >>>> >>>> -Isaac >>>> >>>> >>>> >>>> >>>> diff --git a/src/java.base/share/classes/java/lang/ >>>> AbstractStringBuilder.java >>>> b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>>> index 2ef3e53256..1fe89bb92a 100644 >>>> --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>>> +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>>> @@ -543,6 +543,11 @@ abstract class AbstractStringBuilder implements >>>> Appendable, CharSequence { >>>> return this.append((AbstractStringBuilder)sb); >>>> } >>>> >>>> + // Documentation in subclasses because of synchro difference >>>> + public AbstractStringBuilder append(StringBuilder sb) { >>>> + return this.append((AbstractStringBuilder)sb); >>>> + } >>>> + >>>> /** >>>> * @since 1.8 >>>> */ >>>> diff --git a/src/java.base/share/classes/java/lang/StringBuffer.java >>>> b/src/java.base/share/classes/java/lang/StringBuffer.java >>>> index e597a8112e..613ba90c25 100644 >>>> --- a/src/java.base/share/classes/java/lang/StringBuffer.java >>>> +++ b/src/java.base/share/classes/java/lang/StringBuffer.java >>>> @@ -313,6 +313,33 @@ import jdk.internal.HotSpotIntrinsicCandidate; >>>> return this; >>>> } >>>> >>>> + /** >>>> + * Appends the specified {@code StringBuilder} to this sequence. >>>> + *

        >>>> + * The characters of the {@code StringBuilder} argument are appended, >>>> + * in order, to the contents of this {@code StringBuffer}, increasing >>>> the >>>> + * length of this {@code StringBuffer} by the length of the argument. >>>> + * If {@code sb} is {@code null}, then the four characters >>>> + * {@code "null"} are appended to this {@code StringBuffer}. >>>> + *

        >>>> + * Let n be the length of the old character sequence, the one >>>> + * contained in the {@code StringBuffer} just prior to execution of >>>> the >>>> + * {@code append} method. Then the character at index k in >>>> + * the new character sequence is equal to the character at index >>>> k >>>> + * in the old character sequence, if k is less than n; >>>> + * otherwise, it is equal to the character at index k-n in the >>>> + * argument {@code sb}. >>>> + *

        >>>> + * >>>> + * @param sb the {@code StringBuilder} to append. >>>> + * @return a reference to this object. >>>> + */ >>>> + public synchronized StringBuffer append(StringBuilder sb) { >>>> + toStringCache = null; >>>> + super.append(sb); >>>> + return this; >>>> + } >>>> + >>>> /** >>>> * Appends the specified {@code StringBuffer} to this sequence. >>>> *

        >>>> diff --git a/src/java.base/share/classes/java/lang/StringBuilder.java >>>> b/src/java.base/share/classes/java/lang/StringBuilder.java >>>> index 40da2083c2..5ddd4fb5f9 100644 >>>> --- a/src/java.base/share/classes/java/lang/StringBuilder.java >>>> +++ b/src/java.base/share/classes/java/lang/StringBuilder.java >>>> @@ -199,6 +199,30 @@ public final class StringBuilder >>>> return this; >>>> } >>>> >>>> + /** >>>> + * Appends the specified {@code StringBuilder} to this sequence. >>>> + *

        >>>> + * The characters of the {@code StringBuilder} argument are appended, >>>> + * in order, to this sequence, increasing the >>>> + * length of this sequence by the length of the argument. >>>> + * If {@code sb} is {@code null}, then the four characters >>>> + * {@code "null"} are appended to this sequence. >>>> + *

        >>>> + * Let n be the length of this character sequence just prior to >>>> + * execution of the {@code append} method. Then the character at index >>>> + * k in the new character sequence is equal to the character at >>>> + * index k in the old character sequence, if k is less >>>> than >>>> + * n; otherwise, it is equal to the character at index >>>> k-n >>>> + * in the argument {@code sb}. >>>> + * >>>> + * @param sb the {@code StringBuilder} to append. >>>> + * @return a reference to this object. >>>> + */ >>>> + public StringBuilder append(StringBuilder sb) { >>>> + super.append(sb); >>>> + return this; >>>> + } >>>> + >>>> @Override >>>> public StringBuilder append(CharSequence s) { >>>> super.append(s); >>>> From paul.sandoz at oracle.com Fri Jun 29 22:59:20 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 Jun 2018 15:59:20 -0700 Subject: [PATCH] AbstractStringBuilder.append() In-Reply-To: <5B36A9A2.80708@oracle.com> References: <66BCA06A-D1BC-4F67-9331-BD3874E47493@oracle.com> <5B36A9A2.80708@oracle.com> Message-ID: > On Jun 29, 2018, at 2:50 PM, Jonathan Gibbons wrote: > > In that specific snippet, I would expect the compiler (javac) to fold the constants together. > Yes, and behold the use of indy string concat :-) Classfile /Users/sandoz/Projects/jdk/jdk-test/target/classes/Test.class Last modified Jun 29, 2018; size 873 bytes MD5 checksum b6eb6030a1b0840263ecced5880a52e6 Compiled from "Test.java" public class Test SourceFile: "Test.java" InnerClasses: public static final #37= #36 of #40; //Lookup=class java/lang/invoke/MethodHandles$Lookup of class java/lang/invoke/MethodHandles BootstrapMethods: 0: #24 invokestatic java/lang/invoke/StringConcatFactory.makeConcatWithConstants:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite; Method arguments: #25 bazbiz minor version: 0 major version: 54 flags: ACC_PUBLIC, ACC_SUPER Constant pool: ... #22 = Utf8 barfoo ... #25 = String #30 // bazbiz { public Test(); ... public static void main(java.lang.String[]); descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC, ACC_STATIC Code: stack=1, locals=2, args_size=1 0: ldc #2 // String barfoo 2: astore_1 3: aload_1 4: invokedynamic #3, 0 // InvokeDynamic #0:makeConcatWithConstants:(Ljava/lang/String;)Ljava/lang/String; 9: astore_1 10: return Paul. > -- Jon > > On 06/29/2018 01:02 PM, Isaac Levy wrote: >> Would this snippet end up merging two StringBuilders? >> >> String x = "bar" + "foo"; >> x += "baz" + "biz"; >> >> I had trouble reading stringopts.cpp. I was thinking this append might >> be common due to implicit StringBuilder creation. >> >> Isaac >> >> >> On Fri, Jun 29, 2018 at 12:48 PM, Paul Sandoz wrote: >>> Hard to reconstruct the culture memory around this. I suspect such a new method was not considered necessary when StringBuilder was added to Java 1.5 given the CharSequence accepting method, and performance was not considered a concern, and I doubt it's a concern now. >>> >>> (I don?t think there is any circularity that would result from bridge methods if such a method was added.) >>> >>> Paul. >>> >>>> On Jun 27, 2018, at 10:22 PM, Martin Buchholz wrote: >>>> >>>> I'm fairly sure the append(StringBuilder) overloads were left out >>>> intentionally. >>>> >>>> On Wed, Jun 27, 2018 at 8:57 PM, Isaac Levy wrote: >>>> >>>>> AbstractStringBuilder already has append(). This patch >>>>> adds append(). >>>>> >>>>> The new method improves parity between the two classes. In addition, >>>>> combining StringBuilders is presumably common. append() >>>>> has a couple insteadof checks, which this new method skips. >>>>> >>>>> -Isaac >>>>> >>>>> >>>>> >>>>> >>>>> diff --git a/src/java.base/share/classes/java/lang/ >>>>> AbstractStringBuilder.java >>>>> b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>>>> index 2ef3e53256..1fe89bb92a 100644 >>>>> --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>>>> +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >>>>> @@ -543,6 +543,11 @@ abstract class AbstractStringBuilder implements >>>>> Appendable, CharSequence { >>>>> return this.append((AbstractStringBuilder)sb); >>>>> } >>>>> >>>>> + // Documentation in subclasses because of synchro difference >>>>> + public AbstractStringBuilder append(StringBuilder sb) { >>>>> + return this.append((AbstractStringBuilder)sb); >>>>> + } >>>>> + >>>>> /** >>>>> * @since 1.8 >>>>> */ >>>>> diff --git a/src/java.base/share/classes/java/lang/StringBuffer.java >>>>> b/src/java.base/share/classes/java/lang/StringBuffer.java >>>>> index e597a8112e..613ba90c25 100644 >>>>> --- a/src/java.base/share/classes/java/lang/StringBuffer.java >>>>> +++ b/src/java.base/share/classes/java/lang/StringBuffer.java >>>>> @@ -313,6 +313,33 @@ import jdk.internal.HotSpotIntrinsicCandidate; >>>>> return this; >>>>> } >>>>> >>>>> + /** >>>>> + * Appends the specified {@code StringBuilder} to this sequence. >>>>> + *

        >>>>> + * The characters of the {@code StringBuilder} argument are appended, >>>>> + * in order, to the contents of this {@code StringBuffer}, increasing >>>>> the >>>>> + * length of this {@code StringBuffer} by the length of the argument. >>>>> + * If {@code sb} is {@code null}, then the four characters >>>>> + * {@code "null"} are appended to this {@code StringBuffer}. >>>>> + *

        >>>>> + * Let n be the length of the old character sequence, the one >>>>> + * contained in the {@code StringBuffer} just prior to execution of >>>>> the >>>>> + * {@code append} method. Then the character at index k in >>>>> + * the new character sequence is equal to the character at index >>>>> k >>>>> + * in the old character sequence, if k is less than n; >>>>> + * otherwise, it is equal to the character at index k-n in the >>>>> + * argument {@code sb}. >>>>> + *

        >>>>> + * >>>>> + * @param sb the {@code StringBuilder} to append. >>>>> + * @return a reference to this object. >>>>> + */ >>>>> + public synchronized StringBuffer append(StringBuilder sb) { >>>>> + toStringCache = null; >>>>> + super.append(sb); >>>>> + return this; >>>>> + } >>>>> + >>>>> /** >>>>> * Appends the specified {@code StringBuffer} to this sequence. >>>>> *

        >>>>> diff --git a/src/java.base/share/classes/java/lang/StringBuilder.java >>>>> b/src/java.base/share/classes/java/lang/StringBuilder.java >>>>> index 40da2083c2..5ddd4fb5f9 100644 >>>>> --- a/src/java.base/share/classes/java/lang/StringBuilder.java >>>>> +++ b/src/java.base/share/classes/java/lang/StringBuilder.java >>>>> @@ -199,6 +199,30 @@ public final class StringBuilder >>>>> return this; >>>>> } >>>>> >>>>> + /** >>>>> + * Appends the specified {@code StringBuilder} to this sequence. >>>>> + *

        >>>>> + * The characters of the {@code StringBuilder} argument are appended, >>>>> + * in order, to this sequence, increasing the >>>>> + * length of this sequence by the length of the argument. >>>>> + * If {@code sb} is {@code null}, then the four characters >>>>> + * {@code "null"} are appended to this sequence. >>>>> + *

        >>>>> + * Let n be the length of this character sequence just prior to >>>>> + * execution of the {@code append} method. Then the character at index >>>>> + * k in the new character sequence is equal to the character at >>>>> + * index k in the old character sequence, if k is less >>>>> than >>>>> + * n; otherwise, it is equal to the character at index >>>>> k-n >>>>> + * in the argument {@code sb}. >>>>> + * >>>>> + * @param sb the {@code StringBuilder} to append. >>>>> + * @return a reference to this object. >>>>> + */ >>>>> + public StringBuilder append(StringBuilder sb) { >>>>> + super.append(sb); >>>>> + return this; >>>>> + } >>>>> + >>>>> @Override >>>>> public StringBuilder append(CharSequence s) { >>>>> super.append(s); >>>>> > From ivan.gerasimov at oracle.com Sat Jun 30 02:21:23 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 29 Jun 2018 19:21:23 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements Message-ID: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Hello! The spec states that an ArrayDeque created with the default constructor should be able to hold 16 elements. https://docs.oracle.com/javase/10/docs/api/java/util/ArrayDeque.html#%3Cinit%3E() """ Constructs an empty array deque with an initial capacity sufficient to hold 16 elements. """ In the constructor an array of size 16 is created: elements = new Object[16]; However, one element must always be null: /** * The array in which the elements of the deque are stored. * All array cells not holding deque elements are always null. * The array always has at least one null slot (at tail). */ transient Object[] elements; which leaves us space for only 15 elements, contrarily to what the spec promises. Would you please help review a trivial fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8206123 WEBREV: http://cr.openjdk.java.net/~igerasim/8206123/00/webrev/ Thanks in advance! -- With kind regards, Ivan Gerasimov From forax at univ-mlv.fr Sat Jun 30 10:54:37 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 30 Jun 2018 12:54:37 +0200 (CEST) Subject: ImmutableCollections.MapN optimisation ? Message-ID: <1797691062.1933689.1530356077904.JavaMail.zimbra@u-pem.fr> Playing with the value types, i've remarked that ImmutableCollections.MapN is missing an override for getOrDefault which making get() 2 times faster than getOrDefault on my test. The other problem is more subtle, get() uses probe() and probe() uses an infinite loop and not a counted loop, the result is that c2 generates lot of assembly codes for probe() as a consequence it doesn't inline probe() in get() making get() really slow compared to HashMap.get() which is fully inlined. There are several wys to fix that: - you can try to transform the while(true) loop in probe into 2 subsequents loops from idx to length + from 0 to idx. - you can inline the code of probe() into get() and even peel the first loop body evaluation as HashMap.get() does. - you can do both. - your idea here :) Also, i wonder if it's not better to make Map1 to also take cares of the zero case instead of making MapN to manage that case, because it's trading a size check with a null check and usually nullcheck can be removed by the VM. regards, R?mi From peter.levart at gmail.com Sat Jun 30 11:10:49 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 30 Jun 2018 13:10:49 +0200 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: References: Message-ID: <79da50ef-e725-60af-fa74-cfd123e018a9@gmail.com> Hi Volker, I fully support this change, although I'm not qualified to review it as I'm not so acquainted with VM internals. The code looks reasonable to me though. This change will greatly help track down class initializiation bugs. I don't think that attaching a cause which is not directly an exception caught at the place where new exception is thrown will confuse people much, although I can imagine that they are used to observe such chains most of the time. The stack traces of cause and top exception clearly show where their source is and as you say the type of cause (ExceptionInInitializerError) hints at that too. Another hint that you may consider adding or not, depending on whether it would be possible to implement elegantly, is for the cause to somehow include the name of the thread in which it occurred in its message. When the top exception (NoClassDefFoundError) is caught and logged, the logger usually includes the thread name. If the cause also included thread name and it was different, it would be another hint to the user of what actually happened. Just some comments / questions about the test. This is really quite a complicated play of exceptions, causes and initialization orders, I must say. I'll try to evaluate my reasoning by describing what test does, so let's start with this: ?109???? public static class A { ?110???????? public static final int i = (new Object[1])[2].hashCode(); ?111???? } ?112???? public static class B extends A {} ?113???? public static class C extends B {} Those 3 classes are designed so that their initialization fails. Initialization of A fails because ArrayIndexOutOfBoundsException is thrown in field initializer, initialization of B fails because it is a subclass of A and initialization of C fails because it is a subclass of B. On top of that, the test includes the following class: ?115???? static class ClassWithFailedInitializer { ?116???????? public static final int i; ?117???????? static { ?118???????????? cl = new URLClassLoader(new URL[] { ?119 ClassWithFailedInitializer.class.getProtectionDomain().getCodeSource().getLocation() }, null); ?120???????????? try { ?121 cl.loadClass("NoClassDefFoundErrorTest$C").newInstance(); ?122???????????? } catch (ClassNotFoundException | InstantiationException | IllegalAccessException e) { ?123???????????????? throw new RuntimeException("Class should be found", e); ?124???????????? } ?125???????????? i = 42; ?126???????? } ?127???? } ...and pulls the trigger with: ?137???????????? if (ClassWithFailedInitializer.i == 0) { This triggers initialization of ClassWithFailedInitializer 1st, which constructs new ClassLoader which it then uses to load class C and attempts to instantiate an object from it. You then have the following handler arround the last two actions: ?120???????????? try { ?121 cl.loadClass("NoClassDefFoundErrorTest$C").newInstance(); ?122???????????? } catch (ClassNotFoundException | InstantiationException | IllegalAccessException e) { ?123???????????????? throw new RuntimeException("Class should be found", e); ?124???????????? } ClassNotFoundException should not be thrown as C class should be loadable. That's OK. The other two exceptions that you catch here are a little confusing, because they can only occur after successful class C initialization and because they are impossible exceptions in the concrete setup. But I can understand that you have to handle them somehow because they are checked exceptions. So perhaps you could group them into another catch block and throw RuntimeException with a more descriptive message (like "Impossible situation" or such). Just to be less confusing for someone who would have to read the code in the future... Then we get to the following confusing point: ?136???????? try { ?137???????????? if (ClassWithFailedInitializer.i == 0) { ?138???????????????? throw new RuntimeException("Class initialization succeeded but is designed to fail."); ?139???????????? } ?140???????????? throw new Exception("Expected exception was not thrown."); ?141???????? } ?142???????? catch (ExceptionInInitializerError e) { ?143???????????? e.printStackTrace(); ?144???????????? Asserts.assertNE(e.getCause(), null, "Expecting cause in ExceptionInInitializerError."); ?145???????????? Asserts.assertEQ(e.getCause().getClass(), ArrayIndexOutOfBoundsException.class, "sanity"); ?146???????? } I think that line 138 is not reachable. If ClassWithFailedInitializer.i == 0 evaluated successfully, it would mean that ClassWithFailedInitializer initialization finished normally, but if it finished normally, then field i would contain 42, wouldn't it? I think you wanted the test to be if (ClassWithFailedInitializer.i == 42) and then you don't need line 140. Am I right? The same goes for line 156. Good work. Regards, Peter On 06/29/18 16:53, Volker Simonis wrote: > Hi, > > can I please have a review for the following change which saves > ExceptionInInitializerError thrown during class initialization and > chains them as cause into potential NoClassDefFoundErrors for the same > class. We are using this features since years in our commercial SAP > JVM and it proved extremely useful for detecting and fixing errors > especially in big deployments. > > This is a follow-up on a discussion previously started by Goetz [1]. > His first proposal (which is close to our current, internal > implementation) inserted an additional field into java.lang.Class > objects to save potential ExceptionInInitializerErrors. This was > considered to much overhead in the initial discussion [1]. > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8203826.v2/ > https://bugs.openjdk.java.net/browse/JDK-8203826 > > So in this change, I've completely re-implemented the feature by using > a java.lang.Hashtable which is attached to the ClassLoaderData object. > The Hashtable is lazily created when the first > ExceptionInInitializerError is thrown and maps the Class which > triggered the ExceptionInInitializerError during the execution of its > static initializer to the corresponding ExceptionInInitializerError. > > If the same class will be accessed once again, this will directly lead > to a plain NoClassDefFoundError (as per the JVMS, 5.5 Initialization) > because the static initializer won't be executed a second time. Until > now, this NoClassDefFoundError wasn't linked in any way to the root > cause of the problem (i.e. the first ExceptionInInitializerError > together with the chained exception that happened during the execution > of the static initializer). With this change, the NoClassDefFoundError > will now chain the initial ExceptionInInitializerError as cause, > making it much easier to detect the problem which lead to the > NoClassDefFoundError. > > Following is an example from the new JTreg tests which comes which > this change to demonstrate the feature. Until know, a typical stack > trace from a NoClassDefFoundError looked as follows: > > java.lang.NoClassDefFoundError: Could not initialize class > NoClassDefFound$ClassWithFailedInitializer > at java.base/java.lang.Class.forName0(Native Method) > at java.base/java.lang.Class.forName(Class.java:291) > at NoClassDefFound.main(NoClassDefFound.java:38) > > With this change, the same stack trace now looks as follows: > > java.lang.NoClassDefFoundError: Could not initialize class > NoClassDefFound$ClassWithFailedInitializer > at java.base/java.lang.Class.forName0(Native Method) > at java.base/java.lang.Class.forName(Class.java:315) > at NoClassDefFound.main(NoClassDefFound.java:38) > Caused by: java.lang.ExceptionInInitializerError > at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) > at java.base/java.lang.Class.newInstance(Class.java:584) > at NoClassDefFound$ClassWithFailedInitializer.(NoClassDefFound.java:20) > at java.base/java.lang.Class.forName0(Native Method) > at java.base/java.lang.Class.forName(Class.java:315) > at NoClassDefFound.main(NoClassDefFound.java:30) > Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 2 out of > bounds for length 1 > at NoClassDefFound$A.(NoClassDefFound.java:9) > ... 9 more > > As you can see, the reason for the NoClassDefFoundError when accessing > the class 'NoClassDefFound$ClassWithFailedInitializer' is actually not > even in the class or its static initializer itself, but in the class > 'NoClassDefFound$A' which is a base class of > 'NoClassDefFound$ClassWithFailedInitializer'. This is not easily > detectible from the old, plain NoClassDefFoundError. > > As I wrote, the only overhead we have with the new implementation is > an additional OopHandle field per ClassLoaderData which I think is > acceptable. The Hashtable object itself is only created lazily, after > the first occurrence of an ExceptionInInitializerError in the > corresponding class loader. The whole Hashtable creation and > storing/quering of ExceptionInInitializerErrors in > ClassLoaderData::record_init_exception()/ClassLoaderData::query_init_exception() > is optional in the sense that any errors/exceptions occurring during > the execution of these functions are ignored and cleared. > > Finally, we also take care to recursively convert all native > backtraces in the stored ExceptionInInitializerErrors (and their > suppressed and chained exceptions) into symbolic stack traces in order > to avoid holding references to classes and prevent their unloading. > This is implemented in the new private, static method > java.lang.Throwable::removeNativeBacktrace() which is called for each > ExceptionInInitializerError before it is stored in the Hashtable. > > Thank you and best regards, > Volker > > [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-June/028310.html From martinrb at google.com Sat Jun 30 15:39:39 2018 From: martinrb at google.com (Martin Buchholz) Date: Sat, 30 Jun 2018 08:39:39 -0700 Subject: ImmutableCollections.MapN optimisation ? In-Reply-To: <1797691062.1933689.1530356077904.JavaMail.zimbra@u-pem.fr> References: <1797691062.1933689.1530356077904.JavaMail.zimbra@u-pem.fr> Message-ID: On Sat, Jun 30, 2018 at 3:54 AM, Remi Forax wrote: > The other problem is more subtle, get() uses probe() and probe() uses an > infinite loop and not a counted loop, the result is that c2 generates lot > of assembly codes for probe() as a consequence it doesn't inline probe() in > get() making get() really slow compared to HashMap.get() which is fully > inlined. > There are several wys to fix that: > - you can try to transform the while(true) loop in probe into 2 > subsequents loops from idx to length + from 0 to idx. > In ArrayDeque we had success transforming loops into 2 nested loops, the inner one being hotspot friendly, and it seems like the same approach would work for probe(). E.g. look at ArrayDeque#contains From john.r.rose at oracle.com Sat Jun 30 16:05:11 2018 From: john.r.rose at oracle.com (John Rose) Date: Sat, 30 Jun 2018 09:05:11 -0700 Subject: ImmutableCollections.MapN optimisation ? In-Reply-To: References: <1797691062.1933689.1530356077904.JavaMail.zimbra@u-pem.fr> Message-ID: <4FA0C46E-2C7E-4296-8978-553905D0B7AB@oracle.com> On Jun 30, 2018, at 8:39 AM, Martin Buchholz wrote: > >> On Sat, Jun 30, 2018 at 3:54 AM, Remi Forax wrote: >> >> The other problem is more subtle, get() uses probe() and probe() uses an >> infinite loop and not a counted loop, the result is that c2 generates lot >> of assembly codes for probe() as a consequence it doesn't inline probe() in >> get() making get() really slow compared to HashMap.get() which is fully >> inlined. >> There are several wys to fix that: >> - you can try to transform the while(true) loop in probe into 2 >> subsequents loops from idx to length + from 0 to idx. >> > > In ArrayDeque we had success transforming loops into 2 nested loops, the > inner one being hotspot friendly, and it seems like the same approach would > work for probe(). > E.g. look at ArrayDeque#contains That was my thought too. Best case is that the JIT unrolls both levels of loop. Test it though; YMMV. I like to make the outer loop iterate from false to true, as a style thing. When it unrolls you get two customized inner loops. ? John From martinrb at google.com Sat Jun 30 17:20:05 2018 From: martinrb at google.com (Martin Buchholz) Date: Sat, 30 Jun 2018 10:20:05 -0700 Subject: ImmutableCollections.MapN optimisation ? In-Reply-To: <4FA0C46E-2C7E-4296-8978-553905D0B7AB@oracle.com> References: <1797691062.1933689.1530356077904.JavaMail.zimbra@u-pem.fr> <4FA0C46E-2C7E-4296-8978-553905D0B7AB@oracle.com> Message-ID: On Sat, Jun 30, 2018 at 9:05 AM, John Rose wrote: > On Jun 30, 2018, at 8:39 AM, Martin Buchholz wrote: > > > >> On Sat, Jun 30, 2018 at 3:54 AM, Remi Forax wrote: > >> > >> The other problem is more subtle, get() uses probe() and probe() uses an > >> infinite loop and not a counted loop, the result is that c2 generates > lot > >> of assembly codes for probe() as a consequence it doesn't inline > probe() in > >> get() making get() really slow compared to HashMap.get() which is fully > >> inlined. > >> There are several wys to fix that: > >> - you can try to transform the while(true) loop in probe into 2 > >> subsequents loops from idx to length + from 0 to idx. > >> > > > > In ArrayDeque we had success transforming loops into 2 nested loops, the > > inner one being hotspot friendly, and it seems like the same approach > would > > work for probe(). > > E.g. look at ArrayDeque#contains > > That was my thought too. Best case is that the JIT unrolls both levels of > loop. Test it though; YMMV. > > I like to make the outer loop iterate from false to true, as a style > thing. When it unrolls you get two customized inner loops. > I'd be interested in seeing a model example of such an outer loop that iterates from false to true. In ArrayDeque#contains we do ad-hoc if (to == end) break; since the outer loop does either 1 or 2 iterations, never 0. From martinrb at google.com Sat Jun 30 17:29:58 2018 From: martinrb at google.com (Martin Buchholz) Date: Sat, 30 Jun 2018 10:29:58 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Message-ID: Hi Ivan, Thanks for finding this bug - I'll take the blame. Forgive my pushiness - I'd like to do a friendly takeover of this. As penance I ported the similar WhiteBox test from ConcurrentHashMap to ArrayDeque. I would elide the comment in the default constructor. And we can delete our noreg label. Index: src/main/java/util/ArrayDeque.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/main/java/util/ArrayDeque.java,v retrieving revision 1.133 diff -u -r1.133 ArrayDeque.java --- src/main/java/util/ArrayDeque.java 24 Feb 2018 22:04:18 -0000 1.133 +++ src/main/java/util/ArrayDeque.java 30 Jun 2018 17:07:46 -0000 @@ -181,7 +181,7 @@ * sufficient to hold 16 elements. */ public ArrayDeque() { - elements = new Object[16]; + elements = new Object[16 + 1]; } /** Index: src/test/jtreg/util/ArrayDeque/WhiteBox.java =================================================================== RCS file: src/test/jtreg/util/ArrayDeque/WhiteBox.java diff -N src/test/jtreg/util/ArrayDeque/WhiteBox.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/test/jtreg/util/ArrayDeque/WhiteBox.java 30 Jun 2018 17:07:46 -0000 @@ -0,0 +1,125 @@ +/* + * Written by Martin Buchholz with assistance from members of JCP + * JSR-166 Expert Group and released to the public domain, as + * explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +/* + * @test + * @modules java.base/java.util:open + * @run testng WhiteBox + * @summary White box tests of implementation details + */ + +import static org.testng.Assert.*; +import org.testng.annotations.Test; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.ObjectInputStream; +import java.io.ObjectOutputStream; +import java.lang.invoke.MethodHandles; +import java.lang.invoke.VarHandle; +import java.util.ArrayDeque; +import java.util.concurrent.ThreadLocalRandom; + + at Test +public class WhiteBox { + final ThreadLocalRandom rnd = ThreadLocalRandom.current(); + final VarHandle ELEMENTS, HEAD, TAIL; + + WhiteBox() throws ReflectiveOperationException { + Class klazz = ArrayDeque.class; + MethodHandles.Lookup lookup + = MethodHandles.privateLookupIn(klazz, MethodHandles.lookup()); + ELEMENTS = lookup.findVarHandle(klazz, "elements", Object[].class); + HEAD = lookup.findVarHandle(klazz, "head", int.class); + TAIL = lookup.findVarHandle(klazz, "tail", int.class); + } + + Object[] elements(ArrayDeque d) { return (Object[]) ELEMENTS.get(d); } + int head(ArrayDeque d) { return (int) HEAD.get(d); } + int tail(ArrayDeque d) { return (int) TAIL.get(d); } + + void checkCapacity(ArrayDeque d, int capacity) { + assertTrue(d.isEmpty()); + assertEquals(0, head(d)); + assertEquals(0, tail(d)); + assertInvariants(d); + Object[] initialElements = elements(d); + + assertInvariants(d); + for (int i = capacity; i--> 0; ) { + d.add(rnd.nextInt(42)); + assertSame(elements(d), initialElements); + assertInvariants(d); + } + d.add(rnd.nextInt(42)); + assertNotSame(elements(d), initialElements); + } + + @Test + public void defaultConstructor() { + checkCapacity(new ArrayDeque(), 16); + } + + @Test + public void shouldNotResizeWhenInitialCapacityProvided() { + int initialCapacity = rnd.nextInt(1, 20); + checkCapacity(new ArrayDeque(initialCapacity), initialCapacity); + } + + byte[] serialBytes(Object o) { + try { + ByteArrayOutputStream bos = new ByteArrayOutputStream(); + ObjectOutputStream oos = new ObjectOutputStream(bos); + oos.writeObject(o); + oos.flush(); + oos.close(); + return bos.toByteArray(); + } catch (Exception fail) { + throw new AssertionError(fail); + } + } + + @SuppressWarnings("unchecked") + T serialClone(T o) { + try { + ObjectInputStream ois = new ObjectInputStream + (new ByteArrayInputStream(serialBytes(o))); + T clone = (T) ois.readObject(); + assertNotSame(o, clone); + assertSame(o.getClass(), clone.getClass()); + return clone; + } catch (Exception fail) { + throw new AssertionError(fail); + } + } + + @Test + public void testSerialization() { + ArrayDeque[] ds = { new ArrayDeque(), new ArrayDeque(rnd.nextInt(20)) }; + for (ArrayDeque d : ds) { + if (rnd.nextBoolean()) d.add(99); + ArrayDeque clone = serialClone(d); + assertInvariants(clone); + assertNotSame(elements(d), elements(clone)); + assertEquals(d, clone); + } + } + + /** Checks conditions which should always be true. */ + void assertInvariants(ArrayDeque d) { + final Object[] elements = elements(d); + final int head = head(d); + final int tail = tail(d); + final int capacity = elements.length; + assertTrue(0 <= head && head < capacity); + assertTrue(0 <= tail && tail < capacity); + assertTrue(capacity > 0); + assertTrue(d.size() < capacity); + assertTrue(head == tail || elements[head] != null); + assertNull(elements[tail]); + assertTrue(head == tail || elements[Math.floorMod(tail - 1, capacity)] != null); + } +} On Fri, Jun 29, 2018 at 7:21 PM, Ivan Gerasimov wrote: > Hello! > > The spec states that an ArrayDeque created with the default constructor > should be able to hold 16 elements. > > https://docs.oracle.com/javase/10/docs/api/java/util/ArrayDe > que.html#%3Cinit%3E() > """ > Constructs an empty array deque with an initial capacity sufficient to > hold 16 elements. > """ > > In the constructor an array of size 16 is created: > elements = new Object[16]; > > However, one element must always be null: > /** > * The array in which the elements of the deque are stored. > * All array cells not holding deque elements are always null. > * The array always has at least one null slot (at tail). > */ > transient Object[] elements; > > which leaves us space for only 15 elements, contrarily to what the spec > promises. > > > Would you please help review a trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8206123 > WEBREV: http://cr.openjdk.java.net/~igerasim/8206123/00/webrev/ > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > > From me at yawk.at Sat Jun 30 19:53:25 2018 From: me at yawk.at (Jonas Konrad) Date: Sat, 30 Jun 2018 21:53:25 +0200 Subject: ArrayDeque and List Message-ID: <990881fd-a08b-024a-e57c-18799113ce5f@yawk.at> Hey, The introduction of ArrayDeque in Java 1.6 has basically replaced LinkedList as the "default" implementation of java.util.Queue, which is probably for the best. However, from time to time, I still see use cases where a class that implements both Deque and List is wanted for ease of use. LinkedList is currently the only class that supports this. One example would be sorting the queue in-place. Has there been any discussion on having ArrayDeque implement List? There is no *technical* reason against doing so - the internal structure of ArrayDeque is fully capable of supporting an implementation of List, and there would be no additional fields needed that would change current memory behavior. The existing Deque interface would also remain unaffected. I see two possible reasons against doing this: - Added maintenance effort, a lot of it duplicated from ArrayList. This *could* be done gradually: ArrayDeque only extends AbstractCollection right now so implementations for e.g. subList could be inherited from AbstractList. Performance tweaking would be nice in many places though. - Design - conceptually, a Deque is something very different from a List. LinkedList implementing both could be seen as a mistake from that point of view. When coding to interfaces, which is good practice anyway, this change isn't visible though. Thoughts? - Jonas Konrad From martinrb at google.com Sat Jun 30 20:00:54 2018 From: martinrb at google.com (Martin Buchholz) Date: Sat, 30 Jun 2018 13:00:54 -0700 Subject: ArrayDeque and List In-Reply-To: <990881fd-a08b-024a-e57c-18799113ce5f@yawk.at> References: <990881fd-a08b-024a-e57c-18799113ce5f@yawk.at> Message-ID: https://openjdk.markmail.org/search/?q=8143850#query:8143850%20list%3Anet.java.openjdk.core-libs-dev+page:1+state:facets On Sat, Jun 30, 2018 at 12:53 PM, Jonas Konrad wrote: > Hey, > > The introduction of ArrayDeque in Java 1.6 has basically replaced > LinkedList as the "default" implementation of java.util.Queue, which is > probably for the best. However, from time to time, I still see use cases > where a class that implements both Deque and List is wanted for ease of > use. LinkedList is currently the only class that supports this. One example > would be sorting the queue in-place. > > Has there been any discussion on having ArrayDeque implement List? > > There is no *technical* reason against doing so - the internal structure > of ArrayDeque is fully capable of supporting an implementation of List, and > there would be no additional fields needed that would change current memory > behavior. The existing Deque interface would also remain unaffected. > > I see two possible reasons against doing this: > > - Added maintenance effort, a lot of it duplicated from ArrayList. This > *could* be done gradually: ArrayDeque only extends AbstractCollection right > now so implementations for e.g. subList could be inherited from > AbstractList. Performance tweaking would be nice in many places though. > - Design - conceptually, a Deque is something very different from a List. > LinkedList implementing both could be seen as a mistake from that point of > view. When coding to interfaces, which is good practice anyway, this change > isn't visible though. > > Thoughts? > > - Jonas Konrad > From ivan.gerasimov at oracle.com Sat Jun 30 21:58:58 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sat, 30 Jun 2018 14:58:58 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Message-ID: <99c777d8-35ed-bdf8-2fa6-2d6ddbf13c19@oracle.com> Hi Martin! On 6/30/18 10:29 AM, Martin Buchholz wrote: > Hi Ivan, > > Thanks for finding this bug - I'll take the blame. > > Forgive my pushiness - I'd like to do a friendly takeover of this. As > penance I ported the similar WhiteBox test from ConcurrentHashMap to > ArrayDeque. Sure, no problem! I reassigned the bug to you. > I would elide the comment in the default constructor. And we can > delete our noreg label. > Sounds good! If there is a test, then no need for a comment. With kind regards, Ivan > Index: src/main/java/util/ArrayDeque.java > =================================================================== > RCS file: > /export/home/jsr166/jsr166/jsr166/src/main/java/util/ArrayDeque.java,v > retrieving revision 1.133 > diff -u -r1.133 ArrayDeque.java > --- src/main/java/util/ArrayDeque.java24 Feb 2018 22:04:18 -00001.133 > +++ src/main/java/util/ArrayDeque.java30 Jun 2018 17:07:46 -0000 > @@ -181,7 +181,7 @@ > * sufficient to hold 16 elements. > */ > public ArrayDeque() { > - elements = new Object[16]; > + elements = new Object[16 + 1]; > } > /** > Index: src/test/jtreg/util/ArrayDeque/WhiteBox.java > =================================================================== > RCS file: src/test/jtreg/util/ArrayDeque/WhiteBox.java > diff -N src/test/jtreg/util/ArrayDeque/WhiteBox.java > --- /dev/null1 Jan 1970 00:00:00 -0000 > +++ src/test/jtreg/util/ArrayDeque/WhiteBox.java30 Jun 2018 17:07:46 -0000 > @@ -0,0 +1,125 @@ > +/* > + * Written by Martin Buchholz with assistance from members of JCP > + * JSR-166 Expert Group and released to the public domain, as > + * explained at http://creativecommons.org/publicdomain/zero/1.0/ > + */ > + > +/* > + * @test > + * @modules java.base/java.util:open > + * @run testng WhiteBox > + * @summary White box tests of implementation details > + */ > + > +import static org.testng.Assert.*; > +import org.testng.annotations.Test; > + > +import java.io.ByteArrayInputStream; > +import java.io.ByteArrayOutputStream; > +import java.io.ObjectInputStream; > +import java.io.ObjectOutputStream; > +import java.lang.invoke.MethodHandles; > +import java.lang.invoke.VarHandle; > +import java.util.ArrayDeque; > +import java.util.concurrent.ThreadLocalRandom; > + > + at Test > +public class WhiteBox { > + final ThreadLocalRandom rnd = ThreadLocalRandom.current(); > + final VarHandle ELEMENTS, HEAD, TAIL; > + > + WhiteBox() throws ReflectiveOperationException { > + Class klazz = ArrayDeque.class; > + MethodHandles.Lookup lookup > + = MethodHandles.privateLookupIn(klazz, > MethodHandles.lookup()); > + ELEMENTS = lookup.findVarHandle(klazz, "elements", > Object[].class); > + HEAD = lookup.findVarHandle(klazz, "head", int.class); > + TAIL = lookup.findVarHandle(klazz, "tail", int.class); > + } > + > + Object[] elements(ArrayDeque d) { return (Object[]) > ELEMENTS.get(d); } > + int head(ArrayDeque d) { return (int) HEAD.get(d); } > + int tail(ArrayDeque d) { return (int) TAIL.get(d); } > + > + void checkCapacity(ArrayDeque d, int capacity) { > + assertTrue(d.isEmpty()); > + assertEquals(0, head(d)); > + assertEquals(0, tail(d)); > + assertInvariants(d); > + Object[] initialElements = elements(d); > + > + assertInvariants(d); > + for (int i = capacity; i--> 0; ) { > + d.add(rnd.nextInt(42)); > + assertSame(elements(d), initialElements); > + assertInvariants(d); > + } > + d.add(rnd.nextInt(42)); > + assertNotSame(elements(d), initialElements); > + } > + > + @Test > + public void defaultConstructor() { > + checkCapacity(new ArrayDeque(), 16); > + } > + > + @Test > + public void shouldNotResizeWhenInitialCapacityProvided() { > + int initialCapacity = rnd.nextInt(1, 20); > + checkCapacity(new ArrayDeque(initialCapacity), initialCapacity); > + } > + > + byte[] serialBytes(Object o) { > + try { > + ByteArrayOutputStream bos = new ByteArrayOutputStream(); > + ObjectOutputStream oos = new ObjectOutputStream(bos); > + oos.writeObject(o); > + oos.flush(); > + oos.close(); > + return bos.toByteArray(); > + } catch (Exception fail) { > + throw new AssertionError(fail); > + } > + } > + > + @SuppressWarnings("unchecked") > + T serialClone(T o) { > + try { > + ObjectInputStream ois = new ObjectInputStream > + (new ByteArrayInputStream(serialBytes(o))); > + T clone = (T) ois.readObject(); > + assertNotSame(o, clone); > + assertSame(o.getClass(), clone.getClass()); > + return clone; > + } catch (Exception fail) { > + throw new AssertionError(fail); > + } > + } > + > + @Test > + public void testSerialization() { > + ArrayDeque[] ds = { new ArrayDeque(), new > ArrayDeque(rnd.nextInt(20)) }; > + for (ArrayDeque d : ds) { > + if (rnd.nextBoolean()) d.add(99); > + ArrayDeque clone = serialClone(d); > + assertInvariants(clone); > + assertNotSame(elements(d), elements(clone)); > + assertEquals(d, clone); > + } > + } > + > + /** Checks conditions which should always be true. */ > + void assertInvariants(ArrayDeque d) { > + final Object[] elements = elements(d); > + final int head = head(d); > + final int tail = tail(d); > + final int capacity = elements.length; > + assertTrue(0 <= head && head < capacity); > + assertTrue(0 <= tail && tail < capacity); > + assertTrue(capacity > 0); > + assertTrue(d.size() < capacity); > + assertTrue(head == tail || elements[head] != null); > + assertNull(elements[tail]); > + assertTrue(head == tail || elements[Math.floorMod(tail - 1, > capacity)] != null); > + } > +} > > > On Fri, Jun 29, 2018 at 7:21 PM, Ivan Gerasimov > > wrote: > > Hello! > > The spec states that an ArrayDeque created with the default > constructor should be able to hold 16 elements. > > https://docs.oracle.com/javase/10/docs/api/java/util/ArrayDeque.html#%3Cinit%3E() > > """ > Constructs an empty array deque with an initial capacity > sufficient to hold 16 elements. > """ > > In the constructor an array of size 16 is created: > elements = new Object[16]; > > However, one element must always be null: > /** > * The array in which the elements of the deque are stored. > * All array cells not holding deque elements are always null. > * The array always has at least one null slot (at tail). > */ > transient Object[] elements; > > which leaves us space for only 15 elements, contrarily to what the > spec promises. > > > Would you please help review a trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8206123 > > WEBREV: http://cr.openjdk.java.net/~igerasim/8206123/00/webrev/ > > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > > -- With kind regards, Ivan Gerasimov From alexfoster at hotmail.ca Fri Jun 29 06:42:07 2018 From: alexfoster at hotmail.ca (Alex Foster) Date: Fri, 29 Jun 2018 06:42:07 +0000 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: , Message-ID: Another question is whether or not it should allow null elements. ArrayDeque currently doesn't but I think it would be useful to support them, which makes having an asList() view or sharing code harder.