From paul.sandoz at oracle.com Thu Feb 1 00:28:06 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 31 Jan 2018 16:28:06 -0800 Subject: [11] RFR 8196533 Update CondyNestedTest.java to compile jcod file Message-ID: Hi, Please review an update to a constant dynamic test that updates to compile the jcod file rather than loading a class from class file bytes encoded in a base64 string: http://cr.openjdk.java.net/~psandoz/jdk/JDK-8196533-CondyNestedTest-compile-jcod/webrev/ This will be pushed to the hs repo. Thanks, Paul. From mandy.chung at oracle.com Thu Feb 1 00:44:36 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 31 Jan 2018 16:44:36 -0800 Subject: [11] RFR 8196533 Update CondyNestedTest.java to compile jcod file In-Reply-To: References: Message-ID: Looks okay. Mandy On 1/31/18 4:28 PM, Paul Sandoz wrote: > Hi, > > Please review an update to a constant dynamic test that updates to compile the jcod file rather than loading a class from class file bytes encoded in a base64 string: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8196533-CondyNestedTest-compile-jcod/webrev/ > > This will be pushed to the hs repo. > > Thanks, > Paul. From paul.sandoz at oracle.com Thu Feb 1 00:45:54 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 31 Jan 2018 16:45:54 -0800 Subject: [11] RFR 8195694: ConstantBootstraps.invoke does not preserve variable arity In-Reply-To: <476DA321-6EAB-4C27-A5CE-5776E29F7646@oracle.com> References: <6A5707B7-518C-43C2-98C9-6F52AAF3FE6F@oracle.com> <476DA321-6EAB-4C27-A5CE-5776E29F7646@oracle.com> Message-ID: <6CBF6FD0-66EB-47A0-89E4-92307F45586B@oracle.com> > On Jan 31, 2018, at 3:49 PM, John Rose wrote: > > On second thought, you should also use invokeWithArguments to support jumbo arities. > It does, but non-selectively based on the arity: 245 return handle.invokeWithArguments(args); > This tricky idiom should be put into a utility method, package private for starters. A version of it also appears in BSM invocation code. > Are you in part referring to the approach of switching on the number of arguments and using invoke with unpacking for small cases? If you don?t object i would like to follow up on that with another issue. >> On Jan 31, 2018, at 3:23 PM, John Rose wrote: >> >> If you remove the old asType call it?s good! >> Ah! something went wrong when importing the patch from the amber repo. Updated in place. Paul. >>> On Jan 31, 2018, at 3:15 PM, Paul Sandoz wrote: >>> >>> Hi, >>> >>> Please review this fix to the invoke BSM so that it preserves variable arity, if any: >>> >>> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195694-constant-bsms-invoke-arity/webrev/ >>> >>> This will be pushed to the hs repo. >>> >>> Thanks, >>> Paul. >> > From huizhe.wang at oracle.com Thu Feb 1 05:31:50 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 31 Jan 2018 21:31:50 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> Message-ID: <5A72A646.7040101@oracle.com> Hi Tagir, Thanks for the comment. I will consider adding that to the javadoc emphasizing that the comparison is performed from 0 to length() - 1 of the two sequences. Best, Joe On 1/29/18, 8:07 PM, Tagir Valeev wrote: > Hello! > > An AbstractStringBuilder#compareTo implementation is wrong. You cannot > simply compare the whole byte array. Here's the test-case: > > public class Test { > public static void main(String[] args) { > StringBuilder sb1 = new StringBuilder("test1"); > StringBuilder sb2 = new StringBuilder("test2"); > sb1.setLength(4); > sb2.setLength(4); > System.out.println(sb1.compareTo(sb2)); > System.out.println(sb1.toString().compareTo(sb2.toString())); > } > } > > We truncated the stringbuilders making their content equal, so > sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares > the original content (before the truncation) as truncation, of course, > does not zero the truncated bytes, neither does it reallocate the > array (unless explicitly asked via trimToSize). > > With best regards, > Tagir Valeev. > > > On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang wrote: >> Hi, >> >> Adding methods for comparing CharSequence, StringBuilder, and StringBuffer. >> >> The Comparable implementations for StringBuilder/Buffer are similar to that >> of String, allowing comparison operations between two >> StringBuilders/Buffers, e.g. aStringBuilder.compareTo(anotherStringBuilder). >> For CharSequence however, refer to the comments in JIRA, a static method >> 'compare' is added instead of implementing the Comparable interface. This >> 'compare' method may take CharSequence implementations such as String, >> StringBuilder and StringBuffer, making it possible to perform comparison >> among them. The previous example for example is equivalent to >> CharSequence.compare(aStringBuilder, anotherStringBuilder). >> >> Tests for java.base have been independent from each other. The new tests are >> therefore created to have no dependency on each other or sharing any code. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> Thanks, >> Joe From john.r.rose at oracle.com Thu Feb 1 12:26:59 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 1 Feb 2018 12:26:59 +0000 Subject: [11] RFR 8195694: ConstantBootstraps.invoke does not preserve variable arity In-Reply-To: <6CBF6FD0-66EB-47A0-89E4-92307F45586B@oracle.com> References: <6A5707B7-518C-43C2-98C9-6F52AAF3FE6F@oracle.com> <476DA321-6EAB-4C27-A5CE-5776E29F7646@oracle.com> <6CBF6FD0-66EB-47A0-89E4-92307F45586B@oracle.com> Message-ID: <49545AD4-C5F4-412F-9651-984BC40C1AA1@oracle.com> On Feb 1, 2018, at 12:45 AM, Paul Sandoz wrote: > > > >> On Jan 31, 2018, at 3:49 PM, John Rose wrote: >> >> On second thought, you should also use invokeWithArguments to support jumbo arities. >> > > It does, but non-selectively based on the arity: > > 245 return handle.invokeWithArguments(args); > > >> This tricky idiom should be put into a utility method, package private for starters. A version of it also appears in BSM invocation code. >> > > Are you in part referring to the approach of switching on the number of arguments and using invoke with unpacking for small cases? That might be worth doing as an internal optimization in a reusable library function. But my main concern is correctness: packaging a tricky idiom instead of having users re-derive it by noticing bugs in the near-miss approximations. We put in withVarargs to capture some of the idiom, but we aren?t there yet. The big you are fixing had happened in several places and the root cause is the lack of an advertised best practice. Hence my interest in a utility method, eventually a public one. > > If you don?t object i would like to follow up on that with another issue. Sure. > > >>> On Jan 31, 2018, at 3:23 PM, John Rose wrote: >>> >>> If you remove the old asType call it?s good! >>> > > > Ah! something went wrong when importing the patch from the amber repo. Updated in place. > > Paul. > >>>> On Jan 31, 2018, at 3:15 PM, Paul Sandoz wrote: >>>> >>>> Hi, >>>> >>>> Please review this fix to the invoke BSM so that it preserves variable arity, if any: >>>> >>>> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195694-constant-bsms-invoke-arity/webrev/ >>>> >>>> This will be pushed to the hs repo. >>>> >>>> Thanks, >>>> Paul. >>> >> > From ben_walsh at uk.ibm.com Thu Feb 1 14:55:42 2018 From: ben_walsh at uk.ibm.com (Ben Walsh) Date: Thu, 1 Feb 2018 14:55:42 +0000 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup Message-ID: This contribution forms a partial solution to the problem detailed here - http://thevirtualmachinist.blogspot.ca/2011/07/subtle-issue-of-reachability.html . In this context, this contribution provides "markers" such that a suitably "aware" compiler can reduce the chances of such a problem occurring with each of the DirectBuffer objects and MappedByteBuffer object. The reachabilityFences may prevent crashes / exceptions due to cleaning up the backing memory before the user has finished using the pointer. Any compiler not suitably "aware" could be modified to make use of the "markers". I would like to pair with a sponsor to contribute this patch ... ------------------------------------------------------------------------------------------------------------------- diff -r d51e64840b4f src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -33,15 +33,21 @@ private $type$ get$Type$(long a) { $memtype$ x = UNSAFE.get$Memtype$Unaligned(null, a, bigEndian); - return $fromBits$(x); + $type$ y = $fromBits$(x); + Reference.reachabilityFence(this); + return y; } public $type$ get$Type$() { - return get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); + $type$ y = get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); + Reference.reachabilityFence(this); + return y; } public $type$ get$Type$(int i) { - return get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); + $type$ y = get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); + Reference.reachabilityFence(this); + return y; } #end[rw] @@ -50,6 +56,7 @@ #if[rw] $memtype$ y = $toBits$(x); UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); diff -r d51e64840b4f src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -28,6 +28,7 @@ package java.nio; import java.io.FileDescriptor; +import java.lang.ref.Reference; import jdk.internal.misc.VM; import jdk.internal.ref.Cleaner; import sun.nio.ch.DirectBuffer; @@ -312,6 +313,7 @@ public $Type$Buffer put($type$ x) { #if[rw] UNSAFE.put$Swaptype$(ix(nextPutIndex()), $swap$($toBits$(x))); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); @@ -321,6 +323,7 @@ public $Type$Buffer put(int i, $type$ x) { #if[rw] UNSAFE.put$Swaptype$(ix(checkIndex(i)), $swap$($toBits$(x))); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); @@ -347,6 +350,7 @@ if (srem > rem) throw new BufferOverflowException(); UNSAFE.copyMemory(sb.ix(spos), ix(pos), (long)srem << $LG_BYTES_PER_VALUE$); + Reference.reachabilityFence(this); sb.position(spos + srem); position(pos + srem); } else if (src.hb != null) { @@ -413,6 +417,7 @@ int rem = (pos <= lim ? lim - pos : 0); UNSAFE.copyMemory(ix(pos), ix(0), (long)rem << $LG_BYTES_PER_VALUE$); + Reference.reachabilityFence(this); position(rem); limit(capacity()); discardMark(); @@ -505,6 +510,7 @@ void _put(int i, byte b) { // package-private #if[rw] UNSAFE.putByte(address + i, b); + Reference.reachabilityFence(this); #else[rw] throw new ReadOnlyBufferException(); #end[rw] diff -r d51e64840b4f src/java.base/share/classes/java/nio/MappedByteBuffer.java --- a/src/java.base/share/classes/java/nio/MappedByteBuffer.java Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/MappedByteBuffer.java Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -26,6 +26,7 @@ package java.nio; import java.io.FileDescriptor; +import java.lang.ref.Reference; import jdk.internal.misc.Unsafe; @@ -164,6 +165,7 @@ // is computed as we go along to prevent the compiler from otherwise // considering the loop as dead code. Unsafe unsafe = Unsafe.getUnsafe(); + Reference.reachabilityFence(this); int ps = Bits.pageSize(); int count = Bits.pageCount(length); long a = mappingAddress(offset); ------------------------------------------------------------------------------------------------------------------- Regards, Ben Walsh Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From chris.hegarty at oracle.com Thu Feb 1 15:11:29 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 1 Feb 2018 15:11:29 +0000 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> Message-ID: <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> Hi Roger, > On 31 Jan 2018, at 15:52, Roger Riggs wrote: > > Adding net-dev at openjdk.java.net > > On 1/30/2018 5:08 PM, Roger Riggs wrote: >> Please review changes to replace finalizers in socket, datagram, and multicast networking >> with Cleaner based release of the raw file descriptors. Each FileDescriptor is registered >> for cleanup after the raw fd (or handle) is assigned. Normal calls to close unregister the >> cleaner before using the current logic to close the raw fd/handle. Windows networking >> uses fd's with the Windows socket_ API requiring a special cased Cleanable. >> >> The tests check that the implementation objects including FileDescriptors are reclaimed >> and for Linux that the raw fd counts are reduced as expected. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ I think this is good. One small comment; could SocketCleanup be a top-level package-private class, since it is shared by the three different socket types. I didn?t look too hard at the tests, other than to note that they seem to verify expected behaviour, which is good. -Chris. From huizhe.wang at oracle.com Thu Feb 1 18:07:09 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 01 Feb 2018 10:07:09 -0800 Subject: RFR (JDK11/JAXP) 8193830: Xalan Update: Xalan Java 2.7.2 Message-ID: <5A73574D.3050201@oracle.com> Hi, Please review an update to Xalan 2.7.2 release. JBS: https://bugs.openjdk.java.net/browse/JDK-8193830 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8193830/webrev/index.html Thanks, Joe From patrick at reini.net Thu Feb 1 18:35:11 2018 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 1 Feb 2018 19:35:11 +0100 Subject: RFR 8196298 Add null Reader and Writer Message-ID: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> Hi, Here is my first shot of the actual implementation and tests: http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.00 Thanks for feedback -Patrick From paul.sandoz at oracle.com Thu Feb 1 18:44:10 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 1 Feb 2018 10:44:10 -0800 Subject: [11] 8196583 Update jib and test jtreg version to 4.2 b12 Message-ID: Hi, The constant dynamic test i recently updated and pushed to hs [*] requires a later version of jtreg to execute correctly (it requires an updated version of asmtools bundled with jtreg). Here are the changes: diff -r 11920d5d14a8 make/conf/jib-profiles.js --- a/make/conf/jib-profiles.js Wed Jan 31 17:43:46 2018 -0800 +++ b/make/conf/jib-profiles.js Thu Feb 01 10:39:38 2018 -0800 @@ -829,7 +829,7 @@ jtreg: { server: "javare", revision: "4.2", - build_number: "b11", + build_number: "b12", checksum_file: "MD5_VALUES", file: "jtreg_bin-4.2.zip", environment_name: "JT_HOME", diff -r 11920d5d14a8 test/hotspot/jtreg/TEST.ROOT --- a/test/hotspot/jtreg/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 +++ b/test/hotspot/jtreg/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 @@ -58,7 +58,7 @@ docker.support # Minimum jtreg version -requiredVersion=4.2 b11 +requiredVersion=4.2 b12 # Path to libraries in the topmost test directory. This is needed so @library # does not need ../../../ notation to reach them diff -r 11920d5d14a8 test/jaxp/TEST.ROOT --- a/test/jaxp/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 +++ b/test/jaxp/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 @@ -23,7 +23,7 @@ groups=TEST.groups # Minimum jtreg version -requiredVersion=4.2 b11 +requiredVersion=4.2 b12 # Path to libraries in the topmost test directory. This is needed so @library # does not need ../../ notation to reach them diff -r 11920d5d14a8 test/jdk/TEST.ROOT --- a/test/jdk/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 +++ b/test/jdk/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 @@ -40,7 +40,7 @@ vm.cds # Minimum jtreg version -requiredVersion=4.2 b11 +requiredVersion=4.2 b12 # Path to libraries in the topmost test directory. This is needed so @library # does not need ../../ notation to reach them diff -r 11920d5d14a8 test/langtools/TEST.ROOT --- a/test/langtools/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 +++ b/test/langtools/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 @@ -15,7 +15,7 @@ groups=TEST.groups # Minimum jtreg version -requiredVersion=4.2 b11 +requiredVersion=4.2 b12 # Use new module options useNewOptions=true diff -r 11920d5d14a8 test/nashorn/TEST.ROOT --- a/test/nashorn/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 +++ b/test/nashorn/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 @@ -8,7 +8,7 @@ groups=TEST.groups # Minimum jtreg version -requiredVersion=4.2 b11 +requiredVersion=4.2 b12 # Use new module options useNewOptions=true Paul. [*] http://hg.openjdk.java.net/jdk/hs/rev/11920d5d14a8 From mandy.chung at oracle.com Thu Feb 1 18:48:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 1 Feb 2018 10:48:22 -0800 Subject: [11] 8196583 Update jib and test jtreg version to 4.2 b12 In-Reply-To: References: Message-ID: +1 Mandy On 2/1/18 10:44 AM, Paul Sandoz wrote: > Hi, > > The constant dynamic test i recently updated and pushed to hs [*] requires a later version of jtreg to execute correctly (it requires an updated version of asmtools bundled with jtreg). > > Here are the changes: > > diff -r 11920d5d14a8 make/conf/jib-profiles.js > --- a/make/conf/jib-profiles.js Wed Jan 31 17:43:46 2018 -0800 > +++ b/make/conf/jib-profiles.js Thu Feb 01 10:39:38 2018 -0800 > @@ -829,7 +829,7 @@ > jtreg: { > server: "javare", > revision: "4.2", > - build_number: "b11", > + build_number: "b12", > checksum_file: "MD5_VALUES", > file: "jtreg_bin-4.2.zip", > environment_name: "JT_HOME", > diff -r 11920d5d14a8 test/hotspot/jtreg/TEST.ROOT > --- a/test/hotspot/jtreg/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/hotspot/jtreg/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -58,7 +58,7 @@ > docker.support > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Path to libraries in the topmost test directory. This is needed so @library > # does not need ../../../ notation to reach them > diff -r 11920d5d14a8 test/jaxp/TEST.ROOT > --- a/test/jaxp/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/jaxp/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -23,7 +23,7 @@ > groups=TEST.groups > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Path to libraries in the topmost test directory. This is needed so @library > # does not need ../../ notation to reach them > diff -r 11920d5d14a8 test/jdk/TEST.ROOT > --- a/test/jdk/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/jdk/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -40,7 +40,7 @@ > vm.cds > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Path to libraries in the topmost test directory. This is needed so @library > # does not need ../../ notation to reach them > diff -r 11920d5d14a8 test/langtools/TEST.ROOT > --- a/test/langtools/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/langtools/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -15,7 +15,7 @@ > groups=TEST.groups > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Use new module options > useNewOptions=true > diff -r 11920d5d14a8 test/nashorn/TEST.ROOT > --- a/test/nashorn/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/nashorn/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -8,7 +8,7 @@ > groups=TEST.groups > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Use new module options > useNewOptions=true > > Paul. > > [*] http://hg.openjdk.java.net/jdk/hs/rev/11920d5d14a8 From forax at univ-mlv.fr Thu Feb 1 18:49:43 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 1 Feb 2018 19:49:43 +0100 (CET) Subject: RFR 8196298 Add null Reader and Writer In-Reply-To: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> References: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> Message-ID: <1248919032.842913.1517510983380.JavaMail.zimbra@u-pem.fr> Hi Patrick, I believe read(CharBuffer), like read(char[], int, int), should test if there are remaining chars in the buffer. cheers, R?mi ----- Mail original ----- > De: "Patrick Reinhart" > ?: "core-libs-dev" > Envoy?: Jeudi 1 F?vrier 2018 19:35:11 > Objet: RFR 8196298 Add null Reader and Writer > Hi, > > Here is my first shot of the actual implementation and tests: > > http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.00 > > Thanks for feedback > > -Patrick From lois.foltan at oracle.com Thu Feb 1 19:06:30 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 1 Feb 2018 14:06:30 -0500 Subject: [11] 8196583 Update jib and test jtreg version to 4.2 b12 In-Reply-To: References: Message-ID: <23cb8743-f403-9f6b-bbdb-3c2346301435@oracle.com> Looks good. Lois On 2/1/2018 1:44 PM, Paul Sandoz wrote: > Hi, > > The constant dynamic test i recently updated and pushed to hs [*] requires a later version of jtreg to execute correctly (it requires an updated version of asmtools bundled with jtreg). > > Here are the changes: > > diff -r 11920d5d14a8 make/conf/jib-profiles.js > --- a/make/conf/jib-profiles.js Wed Jan 31 17:43:46 2018 -0800 > +++ b/make/conf/jib-profiles.js Thu Feb 01 10:39:38 2018 -0800 > @@ -829,7 +829,7 @@ > jtreg: { > server: "javare", > revision: "4.2", > - build_number: "b11", > + build_number: "b12", > checksum_file: "MD5_VALUES", > file: "jtreg_bin-4.2.zip", > environment_name: "JT_HOME", > diff -r 11920d5d14a8 test/hotspot/jtreg/TEST.ROOT > --- a/test/hotspot/jtreg/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/hotspot/jtreg/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -58,7 +58,7 @@ > docker.support > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Path to libraries in the topmost test directory. This is needed so @library > # does not need ../../../ notation to reach them > diff -r 11920d5d14a8 test/jaxp/TEST.ROOT > --- a/test/jaxp/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/jaxp/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -23,7 +23,7 @@ > groups=TEST.groups > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Path to libraries in the topmost test directory. This is needed so @library > # does not need ../../ notation to reach them > diff -r 11920d5d14a8 test/jdk/TEST.ROOT > --- a/test/jdk/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/jdk/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -40,7 +40,7 @@ > vm.cds > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Path to libraries in the topmost test directory. This is needed so @library > # does not need ../../ notation to reach them > diff -r 11920d5d14a8 test/langtools/TEST.ROOT > --- a/test/langtools/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/langtools/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -15,7 +15,7 @@ > groups=TEST.groups > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Use new module options > useNewOptions=true > diff -r 11920d5d14a8 test/nashorn/TEST.ROOT > --- a/test/nashorn/TEST.ROOT Wed Jan 31 17:43:46 2018 -0800 > +++ b/test/nashorn/TEST.ROOT Thu Feb 01 10:39:38 2018 -0800 > @@ -8,7 +8,7 @@ > groups=TEST.groups > > # Minimum jtreg version > -requiredVersion=4.2 b11 > +requiredVersion=4.2 b12 > > # Use new module options > useNewOptions=true > > Paul. > > [*] http://hg.openjdk.java.net/jdk/hs/rev/11920d5d14a8 From patrick at reini.net Thu Feb 1 20:01:37 2018 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 1 Feb 2018 21:01:37 +0100 Subject: RFR 8196298 Add null Reader and Writer In-Reply-To: <1248919032.842913.1517510983380.JavaMail.zimbra@u-pem.fr> References: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> <1248919032.842913.1517510983380.JavaMail.zimbra@u-pem.fr> Message-ID: <3F40C729-B9DF-407E-89A8-1363D729BB50@reini.net> Hi R?mi, I also found, that I missed the throw an ReadOnlyBufferException on a passed in read only CharBuffer. > Am 01.02.2018 um 19:49 schrieb Remi Forax : > > Hi Patrick, > I believe read(CharBuffer), like read(char[], int, int), should test if there are remaining chars in the buffer. Does it make sense to check for remaining chars in the buffer, wen the source is basically at the end of it?s data? -Patrick From lance.andersen at oracle.com Thu Feb 1 20:06:35 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 1 Feb 2018 15:06:35 -0500 Subject: RFR (JDK11/JAXP) 8193830: Xalan Update: Xalan Java 2.7.2 In-Reply-To: <5A73574D.3050201@oracle.com> References: <5A73574D.3050201@oracle.com> Message-ID: Hi Joe, Changes look fine. I assume the russian and polish is correct :-) Best Lance > On Feb 1, 2018, at 1:07 PM, Joe Wang wrote: > > Hi, > > Please review an update to Xalan 2.7.2 release. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8193830 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8193830/webrev/index.html > > Thanks, > Joe > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From huizhe.wang at oracle.com Thu Feb 1 20:42:42 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 1 Feb 2018 12:42:42 -0800 Subject: RFR (JDK11/JAXP) 8193830: Xalan Update: Xalan Java 2.7.2 In-Reply-To: References: <5A73574D.3050201@oracle.com> Message-ID: <084d3cd9-6ca9-8d6b-8d5d-ee5720b15086@oracle.com> Thanks Lance! The russian and polish files were the original ones submitted by the bug reporter. I double-checked with this https://en.wikipedia.org/wiki/Russian_alphabet :-) -Joe On 2/1/2018 12:06 PM, Lance Andersen wrote: > Hi Joe, > > Changes look fine. ?I assume the russian and polish is correct :-) > > Best > Lance >> On Feb 1, 2018, at 1:07 PM, Joe Wang > > wrote: >> >> Hi, >> >> Please review an update to Xalan 2.7.2 release. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8193830 >> webrev: >> http://cr.openjdk.java.net/~joehw/jdk11/8193830/webrev/index.html >> >> >> Thanks, >> Joe >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From kak at google.com Thu Feb 1 20:46:32 2018 From: kak at google.com (Kurt Alfred Kluever) Date: Thu, 1 Feb 2018 15:46:32 -0500 Subject: java.time.Instant.getEpochSeconds() javadocs still has a typo (JDK-8167166) Message-ID: Hey folks, I had a quick question about the javadocs of this method. It looks like the `getNanosOfSecond` -> `getNano` mistake was fixed , but the Java9 docs still claim that `getNano()` returns "the nanosecond part of the day" . It should be "the nanosecond part of the *second*". (Apologies if this isn't the correct way to report things like this...) -- kak From Roger.Riggs at Oracle.com Thu Feb 1 21:20:53 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 1 Feb 2018 16:20:53 -0500 Subject: java.time.Instant.getEpochSeconds() javadocs still has a typo In-Reply-To: References: Message-ID: <723b8af7-7efc-b891-247e-e57d6d449654@Oracle.com> Hi, Filed: 8196609 Improve javadoc for java.time.Instant.getEpochSecond Thanks, Roger On 2/1/2018 3:46 PM, Kurt Alfred Kluever wrote: > Hey folks, > > I had a quick question about the javadocs of this method. It looks like the > `getNanosOfSecond` -> `getNano` mistake was fixed > , but the Java9 docs > still claim that `getNano()` returns "the nanosecond part of the day" > > . > > It should be "the nanosecond part of the *second*". > > (Apologies if this isn't the correct way to report things like this...) > From brian.burkhalter at oracle.com Thu Feb 1 21:26:23 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 1 Feb 2018 13:26:23 -0800 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: <33FEB947-39BD-41FE-B28F-2536E6BD687B@reini.net> References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> <29989B8B-48AB-408A-A819-F1A099A71D9A@oracle.com> <01CCBA7D-1815-4A78-A803-A9D18E437B34@oracle.com> <0be48e06-637c-eb04-bb06-0bf0524494f9@oracle.com> <33FEB947-39BD-41FE-B28F-2536E6BD687B@reini.net> Message-ID: On Jan 30, 2018, at 8:52 AM, Patrick Reinhart wrote: >> Am 30.01.2018 um 16:02 schrieb Alan Bateman : >> >> [?] >> >> One other micro detail is that Reader/Writer support implementations specifying the object to use for synchronization. This isn't interesting for the Reader/Writer objects returned by the proposed static methods so a one-line that says that the object used to synchronize operations is not specified might help. It would avoid something synchronizing on the reader or writer and expecting operations read/write methods to block > > Is that a difference from the null InputStream/OutputStream here? Yes: there is no such lock instance variable for those classes. Probably it would not hurt to add a one liner to each method?s doc as Alan suggested. Thanks, Brian From Roger.Riggs at Oracle.com Thu Feb 1 21:29:53 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 1 Feb 2018 16:29:53 -0500 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> Message-ID: Hi Chris, Thanks for the review and suggestion. Webrev updated: ?? http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ * Refactored SocketCleanup into a java.net package private SocketCleanable * Moved register and unregister into static methods in SocketCleanable * Simplified the test by retaining the checking for GC reclaimed objects but removed checking fd counts, since they were unreliable in testing and not consistent across OS's Thanks, Roger On 2/1/2018 10:11 AM, Chris Hegarty wrote: > Hi Roger, > >> On 31 Jan 2018, at 15:52, Roger Riggs wrote: >> >> Adding net-dev at openjdk.java.net >> >> On 1/30/2018 5:08 PM, Roger Riggs wrote: >>> Please review changes to replace finalizers in socket, datagram, and multicast networking >>> with Cleaner based release of the raw file descriptors. Each FileDescriptor is registered >>> for cleanup after the raw fd (or handle) is assigned. Normal calls to close unregister the >>> cleaner before using the current logic to close the raw fd/handle. Windows networking >>> uses fd's with the Windows socket_ API requiring a special cased Cleanable. >>> >>> The tests check that the implementation objects including FileDescriptors are reclaimed >>> and for Linux that the raw fd counts are reduced as expected. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ > I think this is good. One small comment; could SocketCleanup be a > top-level package-private class, since it is shared by the three > different socket types. > > I didn?t look too hard at the tests, other than to note that they seem > to verify expected behaviour, which is good. > > -Chris. From Roger.Riggs at Oracle.com Thu Feb 1 22:04:08 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 1 Feb 2018 17:04:08 -0500 Subject: Clarification for adding classes shipped with JDK to sun.rmi.registry.registryFilter property in java.security file In-Reply-To: References: Message-ID: <07b2ca12-f10f-c7cc-ed80-a0da4ca811d4@Oracle.com> Hi Nasser, There is a tradeoff between convenience and security. The risk can be reduced, in part, by keeping the set of trusted classes as small as possible. White-lists should be application specific, with the selected classes carefully reviewed and only include what is needed. Regards, Roger On 1/29/2018 8:08 AM, Nasser Ebrahim wrote: > Hi Roger, > > Thank you for your inputs. I agree with you that logging the filtering > action can help developers to easily identify the classes which are failed > due to the new filtering mechanism. > > However, it may not be fair to ask the application developers to add the > basic JDK classes when Java introduced the filtering mechanism. For > application classes, it makes sense for developer to use the logging > mechanism to identify the classes to be added to the white list. > > I think JDK should add all its standard classes (at least the base > packages) to white list by default rather than asking each end user to add > to the white list. > > Please clarify whether there is any reason for not adding the standard JDK > classes by default to the white list as they are trusted code. > > Thank you, > Nasser Ebrahim > > > > From: Roger Riggs > To: core-libs-dev at openjdk.java.net > Date: 01/19/2018 08:50 PM > Subject: Re: Clarification for adding classes shipped with JDK to > sun.rmi.registry.registryFilter property in java.security file > Sent by: "core-libs-dev" > > > > Hi Vipin, > > A couple of suggestions to make working with serialization filters easier. > > JEP 290 enabled logging of filtering actions by setting > java.io.serialization.level = FINEST. > Add that to the logging.properties and supply it on the command line with > "-Djava.util.logging.config.file=logging.properties". > If set to FINEST, it will log every call to the filter with the > arguments of class, array size, etc. > If set to FINE, it will log only the cases where the filter rejects > based on its arguments. > > The pattern based filters can allow or disallow entire packages. > For example, "java.net.**" will allow any class in any package with a > "java.net." prefix. > > For development purposes, serialized classes for the registry could be > identified using a filter of "*" > with logging enabled as FINEST and examining the log. > > Expanding the white list for the registry is a possibility with a very > conservative > evaluation of specific cases. > > Regards, Roger > > [1] > https://urldefense.proofpoint.com/v2/url?u=http-3A__openjdk.java.net_jeps_290&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=yiMdG4h7wT1IHGC_YAWYv5USAuyCyINm1bo82D5y5bY&m=12RWeCnDad-pQO_PzN0K_mAliSGWmNyH_hOob40MLlA&s=HSk5hl04g1T39ZWtVoEpYFo0CYAkuLPyfQlKA2eh5Us&e= > > > > On 1/16/2018 7:21 AM, Vipin Mv1 wrote: >> Hi, >> >> After upgrading to oracle 8u121-b12, I get following Exception when > running >> the following testcase. >> >> import java.io.*; >> import java.net.InetAddress; >> import java.rmi.*; >> import java.rmi.registry.*; >> import java.security.Security; >> >> /* @test >> * @run main/othervm RegistryFilterTest >> */ >> >> public class RegistryFilterTest { >> >> private static int port = 12345; >> private static Registry registry; >> >> static class RemoteObj implements Serializable, Remote { >> private static final long serialVersionUID = 01L; >> >> final Object obj; >> >> RemoteObj(Object obj) { >> this.obj = obj; >> } >> >> >> } >> >> >> public static void main(String[] args) throws Exception{ >> >> InetAddress in = InetAddress.getLocalHost(); >> LocateRegistry.createRegistry(port); >> Registry registry = LocateRegistry.getRegistry("localhost", >> port); >> registry.bind("InetAddress", new RemoteObj(in)); >> registry.unbind("InetAddress"); >> System.out.println("RMI Registry Test Passed"); >> >> } >> >> } >> >> TestResults >> ----------------------- >> Jan 12, 2018 5:32:18 PM java.io.ObjectInputStream filterCheck >> INFO: ObjectInputFilter REJECTED: class java.net.InetAddress, array > length: >> -1, nRefs: 5, depth: 2, bytes: 216, ex: n/a >> Caused by: java.io.InvalidClassException: filter status: REJECTED >> at > java.io.ObjectInputStream.filterCheck(ObjectInputStream.java:1406) >> at java.io.ObjectInputStream.readNonProxyDesc >> (ObjectInputStream.java:1997) >> at > java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1877) >> at java.io.ObjectInputStream.readOrdinaryObject >> (ObjectInputStream.java:2170) >> at > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1698) >> at java.io.ObjectInputStream.defaultReadFields >> (ObjectInputStream.java:2415) >> at java.io.ObjectInputStream.readSerialData >> (ObjectInputStream.java:2339) >> at java.io.ObjectInputStream.readOrdinaryObject >> (ObjectInputStream.java:2197) >> at > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1698) >> at > java.io.ObjectInputStream.readObjectImpl(ObjectInputStream.java:540) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:475) >> ... 16 more >> >> I understand it is due to the new serialization Filtering mechanism >> introduced to RMI Registry and Distributed Garbage Collection as a part > of >> the following fixes. >> >> JDK-8160108 Implement Serialization Filtering >> JDK-8156804: Better constraint checking(not public) >> >> As a result, a new default white list has been introduced for > RMIRegistry >> which allows deserialization only to limited set of classes by default. >> Which are >> >> String.class >> java.lang.Number.class >> java.rmi.Remote.class >> java.lang.reflect.Proxy.class >> sun.rmi.server.UnicastRef.class >> java.rmi.server.RMIClientSocketFactory.class >> java.rmi.server.RMIServerSocketFactory.class >> java.rmi.activation.ActivationID.class >> java.rmi.server.UID.class >> and their subclasses. >> >> It also enabled provisions to increase the default white list by >> >> 1. updating property "sun.rmi.registry.registryFilter" in java.security >> file >> 2. adding -Dsun.rmi.registry.registryFilter in java command line >> >> Though the current solution is straight forward, it is difficult from an >> application point of view to identify such classes without running the >> application multiple times to come up with comprehensive list to be >> allowed. This would be a overhead to the developer and should be avoided > at >> least for classes shipped with JDK (for example, java.net.InetAddress as >> used in testcase). >> >> I assume that default list is very limited to reduce the chances of > getting >> exploited by serialization vulnerability. I think, we can spare public >> packages shipped with JDK by adding them to >> "sun.rmi.registry.registryFilter" in java.security file out of the box. >> >> Please let me know your views. >> >> Thanks & Regards, >> Vipin MV > > > > From paul.sandoz at oracle.com Thu Feb 1 22:50:23 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 1 Feb 2018 14:50:23 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: Message-ID: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Hi Ben, I don?t think you require the fence in all the cases you have currently placed it e.g. here for example $memtype$ y = $toBits$(x); UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); + Reference.reachabilityFence(this); return this; since ?this? is being returned it will be kept live during the unsafe access. Would you mind providing a JMH benchmark and results for the performance with and without the fence for say a put and/or a get. I would like to understand the performance impact on HotSpot, this is one reason why we have yet to add such fences as it will likely impact performance. At the moment we are relying on the method not being inlined, which is an expedient technique to make it functional and keep a reference alive but not necessarily optimal for usages in DBB. For more details see: https://bugs.openjdk.java.net/browse/JDK-8169605 https://bugs.openjdk.java.net/browse/JDK-8149610 Thanks, Paul. > On Feb 1, 2018, at 6:55 AM, Ben Walsh wrote: > > This contribution forms a partial solution to the problem detailed here - > http://thevirtualmachinist.blogspot.ca/2011/07/subtle-issue-of-reachability.html > . > > In this context, this contribution provides "markers" such that a suitably > "aware" compiler can reduce the chances of such a problem occurring with > each of the DirectBuffer objects and MappedByteBuffer > object. The reachabilityFences may prevent crashes / exceptions due to > cleaning up the backing memory before the user has finished using the > pointer. > > Any compiler not suitably "aware" could be modified to make use of the > "markers". > > > I would like to pair with a sponsor to contribute this patch ... > > ------------------------------------------------------------------------------------------------------------------- > diff -r d51e64840b4f > src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template > --- > a/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template > Wed Jan 31 12:04:53 2018 +0800 > +++ > b/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template > Thu Feb 01 11:30:10 2018 +0000 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -33,15 +33,21 @@ > > private $type$ get$Type$(long a) { > $memtype$ x = UNSAFE.get$Memtype$Unaligned(null, a, bigEndian); > - return $fromBits$(x); > + $type$ y = $fromBits$(x); > + Reference.reachabilityFence(this); > + return y; > } > > public $type$ get$Type$() { > - return get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); > + $type$ y = get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); > + Reference.reachabilityFence(this); > + return y; > } > > public $type$ get$Type$(int i) { > - return get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); > + $type$ y = get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); > + Reference.reachabilityFence(this); > + return y; > } > > #end[rw] > @@ -50,6 +56,7 @@ > #if[rw] > $memtype$ y = $toBits$(x); > UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); > + Reference.reachabilityFence(this); > return this; > #else[rw] > throw new ReadOnlyBufferException(); > diff -r d51e64840b4f > src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > Wed Jan 31 12:04:53 2018 +0800 > +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > Thu Feb 01 11:30:10 2018 +0000 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -28,6 +28,7 @@ > package java.nio; > > import java.io.FileDescriptor; > +import java.lang.ref.Reference; > import jdk.internal.misc.VM; > import jdk.internal.ref.Cleaner; > import sun.nio.ch.DirectBuffer; > @@ -312,6 +313,7 @@ > public $Type$Buffer put($type$ x) { > #if[rw] > UNSAFE.put$Swaptype$(ix(nextPutIndex()), $swap$($toBits$(x))); > + Reference.reachabilityFence(this); > return this; > #else[rw] > throw new ReadOnlyBufferException(); > @@ -321,6 +323,7 @@ > public $Type$Buffer put(int i, $type$ x) { > #if[rw] > UNSAFE.put$Swaptype$(ix(checkIndex(i)), $swap$($toBits$(x))); > + Reference.reachabilityFence(this); > return this; > #else[rw] > throw new ReadOnlyBufferException(); > @@ -347,6 +350,7 @@ > if (srem > rem) > throw new BufferOverflowException(); > UNSAFE.copyMemory(sb.ix(spos), ix(pos), (long)srem << > $LG_BYTES_PER_VALUE$); > + Reference.reachabilityFence(this); > sb.position(spos + srem); > position(pos + srem); > } else if (src.hb != null) { > @@ -413,6 +417,7 @@ > int rem = (pos <= lim ? lim - pos : 0); > > UNSAFE.copyMemory(ix(pos), ix(0), (long)rem << > $LG_BYTES_PER_VALUE$); > + Reference.reachabilityFence(this); > position(rem); > limit(capacity()); > discardMark(); > @@ -505,6 +510,7 @@ > void _put(int i, byte b) { // package-private > #if[rw] > UNSAFE.putByte(address + i, b); > + Reference.reachabilityFence(this); > #else[rw] > throw new ReadOnlyBufferException(); > #end[rw] > diff -r d51e64840b4f > src/java.base/share/classes/java/nio/MappedByteBuffer.java > --- a/src/java.base/share/classes/java/nio/MappedByteBuffer.java Wed Jan > 31 12:04:53 2018 +0800 > +++ b/src/java.base/share/classes/java/nio/MappedByteBuffer.java Thu Feb > 01 11:30:10 2018 +0000 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -26,6 +26,7 @@ > package java.nio; > > import java.io.FileDescriptor; > +import java.lang.ref.Reference; > import jdk.internal.misc.Unsafe; > > > @@ -164,6 +165,7 @@ > // is computed as we go along to prevent the compiler from > otherwise > // considering the loop as dead code. > Unsafe unsafe = Unsafe.getUnsafe(); > + Reference.reachabilityFence(this); > int ps = Bits.pageSize(); > int count = Bits.pageCount(length); > long a = mappingAddress(offset); > ------------------------------------------------------------------------------------------------------------------- > > > Regards, > Ben Walsh > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > From patrick at reini.net Thu Feb 1 22:49:38 2018 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 1 Feb 2018 23:49:38 +0100 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> <29989B8B-48AB-408A-A819-F1A099A71D9A@oracle.com> <01CCBA7D-1815-4A78-A803-A9D18E437B34@oracle.com> <0be48e06-637c-eb04-bb06-0bf0524494f9@oracle.com> <33FEB947-39BD-41FE-B28F-2536E6BD687B@reini.net> Message-ID: <4DE11B74-0804-4038-82A0-40BFA45EB20A@reini.net> > Am 01.02.2018 um 22:26 schrieb Brian Burkhalter : > > > On Jan 30, 2018, at 8:52 AM, Patrick Reinhart > wrote: > >>> Am 30.01.2018 um 16:02 schrieb Alan Bateman >: >>> >>> [?] >>> >>> One other micro detail is that Reader/Writer support implementations specifying the object to use for synchronization. This isn't interesting for the Reader/Writer objects returned by the proposed static methods so a one-line that says that the object used to synchronize operations is not specified might help. It would avoid something synchronizing on the reader or writer and expecting operations read/write methods to block >> >> Is that a difference from the null InputStream/OutputStream here? > > Yes: there is no such lock instance variable for those classes. Probably it would not hurt to add a one liner to each method?s doc as Alan suggested. > As my english is not that good, I could need some help for that one liner :-) -Patrick From brian.burkhalter at oracle.com Thu Feb 1 23:03:54 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 1 Feb 2018 15:03:54 -0800 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: <4DE11B74-0804-4038-82A0-40BFA45EB20A@reini.net> References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> <29989B8B-48AB-408A-A819-F1A099A71D9A@oracle.com> <01CCBA7D-1815-4A78-A803-A9D18E437B34@oracle.com> <0be48e06-637c-eb04-bb06-0bf0524494f9@oracle.com> <33FEB947-39BD-41FE-B28F-2536E6BD687B@reini.net> <4DE11B74-0804-4038-82A0-40BFA45EB20A@reini.net> Message-ID: On Feb 1, 2018, at 2:49 PM, Patrick Reinhart wrote: >> Am 01.02.2018 um 22:26 schrieb Brian Burkhalter : >>> [?] >> >> Yes: there is no such lock instance variable for those classes. Probably it would not hurt to add a one liner to each method?s doc as Alan suggested. >> > > As my english is not that good, It?s fine! > I could need some help for that one liner :-) How about simply ?The {@link #lock object} used to synchronize operations on the returned {@code Reader} is not specified? and similarly for Writer? I?d suggest to put it as a separate paragraph before the @return doc. Thanks, Brian From brian.burkhalter at oracle.com Thu Feb 1 23:16:18 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 1 Feb 2018 15:16:18 -0800 Subject: RFR 8196298 Add null Reader and Writer In-Reply-To: <3F40C729-B9DF-407E-89A8-1363D729BB50@reini.net> References: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> <1248919032.842913.1517510983380.JavaMail.zimbra@u-pem.fr> <3F40C729-B9DF-407E-89A8-1363D729BB50@reini.net> Message-ID: <8F4348E4-1637-46F8-B273-EB2479F67EE0@oracle.com> On Feb 1, 2018, at 12:01 PM, Patrick Reinhart wrote: >> I believe read(CharBuffer), like read(char[], int, int), should test if there are remaining chars in the buffer. > > Does it make sense to check for remaining chars in the buffer, wen the source is basically at the end of it?s data? read(char[], int, int) returns 0 if len == 0 and -1 otherwise. read(CharBuffer) returns 0 if hasRemaining() is false and -1 otherwise. The two behaviors seem consistent. Brian From hboehm at google.com Fri Feb 2 00:03:31 2018 From: hboehm at google.com (Hans Boehm) Date: Thu, 1 Feb 2018 16:03:31 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: On Thu, Feb 1, 2018 at 2:50 PM, Paul Sandoz wrote: > > Hi Ben, > > I don?t think you require the fence in all the cases you have currently placed it e.g. here for example > > $memtype$ y = $toBits$(x); > UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); > + Reference.reachabilityFence(this); > return this; > > since ?this? is being returned it will be kept live during the unsafe access. I don't see how that's guaranteed by the JLS, specifically 12.6.2. What if this function is inlined into the caller, and the caller drops the result? I would still greatly prefer to see an added mechanism (possibly along the lines of https://docs.google.com/document/d/1yMC4VzZobMQTrVAraMy7xBdWPCJ0p7G5r_43yaqsj-s/edit?usp=sharing) that associates the reachability requirements with fields or classes, rather than putting a reachabilityFence() in every method accessing a field. I'm sure there are cases where reachabilityFence() is the preferred mechanism. But for the code I've looked at recently, I haven't actually found one yet. I don't think it is a good mechanism here, either. > > Would you mind providing a JMH benchmark and results for the performance with and without the fence for say a put and/or a get. I would like to understand the performance impact on HotSpot, this is one reason why we have yet to add such fences as it will likely impact performance. > > At the moment we are relying on the method not being inlined, which is an expedient technique to make it functional and keep a reference alive but not necessarily optimal for usages in DBB. But that should be fixable, right? It shouldn't generate any code beyond potentially preventing register or stack slot reuse; the required effect of reachabilityFence (in a conventional implementation) is to ensure that the argument is considered live at any preceding GC points. > > For more details see: > > https://bugs.openjdk.java.net/browse/JDK-8169605 < https://bugs.openjdk.java.net/browse/JDK-8169605> > https://bugs.openjdk.java.net/browse/JDK-8149610 < https://bugs.openjdk.java.net/browse/JDK-8149610> > > Thanks, > Paul. From paul.sandoz at oracle.com Fri Feb 2 00:23:05 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 1 Feb 2018 16:23:05 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: <2F9C08AE-777D-416F-9C38-257D0B255459@oracle.com> > On Feb 1, 2018, at 4:03 PM, Hans Boehm wrote: > > > On Thu, Feb 1, 2018 at 2:50 PM, Paul Sandoz > wrote: > > > > Hi Ben, > > > > I don?t think you require the fence in all the cases you have currently placed it e.g. here for example > > > > $memtype$ y = $toBits$(x); > > UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); > > + Reference.reachabilityFence(this); > > return this; > > > > since ?this? is being returned it will be kept live during the unsafe access. > > I don't see how that's guaranteed by the JLS, specifically 12.6.2. What if this function is inlined into the caller, and the caller drops the result? > Ah yes, my apologies, i was thinking too narrowly. Your document reminds we should really use the try/finally pattern for any updates DBB code for reasons you suggest even if not strictly needed. > I would still greatly prefer to see an added mechanism (possibly along the lines of https://docs.google.com/document/d/1yMC4VzZobMQTrVAraMy7xBdWPCJ0p7G5r_43yaqsj-s/edit?usp=sharing ) that associates the reachability requirements with fields or classes, rather than putting a reachabilityFence() in every method accessing a field. I'm sure there are cases where reachabilityFence() is the preferred mechanism. But for the code I've looked at recently, I haven't actually found one yet. I don't think it is a good mechanism here, either. > IIRC we discussed such approaches before and concluded reachabilityFence was the most expedient way to get something functional into developers hands if not in the most ideal and expressive way for all cases. > > > > Would you mind providing a JMH benchmark and results for the performance with and without the fence for say a put and/or a get. I would like to understand the performance impact on HotSpot, this is one reason why we have yet to add such fences as it will likely impact performance. > > > > At the moment we are relying on the method not being inlined, which is an expedient technique to make it functional and keep a reference alive but not necessarily optimal for usages in DBB. > > But that should be fixable, right? I believe so, it comes down to prioritizing and expending some compiler engineering effort. Thanks, Paul. > It shouldn't generate any code beyond potentially preventing register or stack slot reuse; the required effect of reachabilityFence (in a conventional implementation) is to ensure that the argument is considered live at any preceding GC points. > > > > > For more details see: > > > > https://bugs.openjdk.java.net/browse/JDK-8169605 > > > https://bugs.openjdk.java.net/browse/JDK-8149610 > > > > > Thanks, > > Paul. From stuart.marks at oracle.com Fri Feb 2 00:55:10 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 1 Feb 2018 16:55:10 -0800 Subject: Collections.addAll: remove outdated performance advice and add a note about atomicity In-Reply-To: References: Message-ID: <8fb4dd69-1cb8-56db-c9c6-b4748d4f1698@oracle.com> Links to existing material in OpenJDK: http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050326.html https://bugs.openjdk.java.net/browse/JDK-8193031 I agree with the removal of the performance advice. We should also remove "identical"; my suggested replacement was "as if". We can even add some hedging about whether the operation is atomic for certain collections (i.e., the synchronized ones). However, I'm not sure if there is anything useful to say about atomicity of bulk updates. They're only atomic for the synchronized collections, which are largely disused. It's pointless to talk about atomicity for non-concurrent collections, and the operations aren't atomic for things like CopyOnWriteArrayList and a Set projection of ConcurrentHashMap. So I'm not sure discussion of atomicity is useful or warranted here. (Also, adding a collection of elements one at a time to a CopyOnWriteArrayList: *shudder*.) I think the most useful thing is to define a new array-reading default method. Each implementation can then override it to use the best technique for that implementation. (I had previously called this "addEach" but I'm flexible on naming.) Adding a new default method is kind of far afield from where this started. If the spec is bothersome, perhaps we could consider a spec cleanup change separately from implementation changes and definition of a new default method. s'marks On 1/30/18 7:07 PM, Martin Buchholz wrote: > I tried to tackle this here: > http://openjdk.markmail.org/thread/eet2zd6ig3pfpv5g > and it's still on my TODO list but not likely to get to top spot soon. > > On Tue, Jan 30, 2018 at 7:00 PM, Tagir Valeev wrote: > >> Hello! >> >> I suggest a patch for java.util.Collections#addAll JavaDoc: >> >> --- Collections.java 2018-01-31 09:39:31.599107500 +0700 >> +++ Collections.java.patched 2018-01-31 09:51:11.929059600 +0700 >> @@ -5406,4 +5406,8 @@ >> * The behavior of this convenience method is identical to that of >> - * {@code c.addAll(Arrays.asList(elements))}, but this method is >> likely >> - * to run significantly faster under most implementations. >> + * {@code c.addAll(Arrays.asList(elements))} except possible >> + * difference in intermediate state visibility for concurrent or >> + * synchronized collections. Calling this method does not guarantee >> + * that the intermediate state (some of elements are added) is >> invisible, >> + * even if the collection itself provides such guarantee for its >> + * {@link Collection#addAll(Collection)} method. >> * >> >> First, currently it says that Collections#addAll is likely to run >> significantly faster. However it's only marginally faster for >> collections which delegate their addAll method to standard >> AbstractCollection#addAll implementation. Also it could be much slower >> for collections which have optimized addAll (like ArrayList, >> CopyOnWriteArrayList, ConcurrentLinkedDeque, etc.). I don't know a >> single example of collection where Collections#addAll is actually >> significantly faster. Also it says that the behavior is identical, >> while it's not. If, e.g. c is a collection returned from >> synchronizedCollection, then intermediate state of >> c.addAll(Arrays.asList(elements)) would not be visible under >> synchronized(c) in another thread. On the other hand, replacing such >> call with Collections.addAll(c, elements) (to make it "significantly >> faster") will lift this guarantee: now you can see partially added >> array. >> >> What do you think? Should I file an issue? >> >> With best regards, >> Tagir Valeev. >> From stuart.marks at oracle.com Fri Feb 2 01:45:12 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 1 Feb 2018 17:45:12 -0800 Subject: RFR 8196298 Add null Reader and Writer In-Reply-To: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> References: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> Message-ID: On 2/1/18 10:35 AM, Patrick Reinhart wrote: > Here is my first shot of the actual implementation and tests: > > http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.00 I looked at the specification of ready() vs the implementation. The spec says that ready(), among other methods, behaves as if the Reader is at EOF. The spec for Reader.ready() says "true if the next read() is guaranteed not to block for input." Thus I'd assume that at EOF, read() will return -1 immediately, without blocking, so ready() should return true. The NullReader implementation inherits ready() from Reader, which always returns false. So, the behavior doesn't seem consistent with my reading of the specs. However, other implementations aren't, either. For example, the spec for CharArrayReader.ready() says "Character-array readers are always ready to be read." However, new CharArrayReader(new char[0]).ready() returns false! A couple other readers I tried (InputStreamReader, FileReader) also have ready() return true until they reach EOF, at which time they return false. But StringReader.ready() always returns true. Hmmmm. Maybe the safest thing to do is to leave the various Readers' behavior of ready() as they are, and leave nullReader's behavior to be consistent with them. No changes are necessary to Reader in this webrev. We should probably take a pass over the ready() methods in the Reader family to see if they're correct. I'd suggest they need adjustment to mean that the next read() won't block *and* data is available. And the CharArrayReader.ready() spec should be fixed. ** On the Writer side, the spec should mention that the three append() methods and five write() methods do nothing if open and throw IOE if the Writer is closed. The Writer.flush() method spec doesn't say what happens if flush() is called after the Writer is closed. However, Writer.close() says that subsequent write() or flush() calls throw IOE. Writer implementations that track open/closed state, such as FileWriter, will throw IOE from flush() after closure. Since it's also tracking open/closed state, I'd expect nullWriter.flush() to do the same. That is, do nothing if open, throw IOE if closed. Tests for flush() should be added. ** Speaking of tests, the tests for proper exception-throwing for the closed cases can probably usefully employ the @Test(expectedExceptions=IOException.class) annotation. The reason I often avoid this construct is that if the same exception class is thrown from an unexpected part of the test, it can mask actual failures. However, since these are one-liners, that reason doesn't apply. Thus: @Test(groups="closed", expectedExceptions=IOException.class) public static void testAppendCharClosed() { closedWriter.append('x'); } You can probably also omit the try/catch block from the open tests, so that instead of @Test(groups="open") public static void testAppendChar() { try { assertSame(openWriter, openWriter.append('x')); } catch (IOException e) { fail("Unexpected IOException"); } } you can write @Test(groups="open") public static void testAppendChar() { assertSame(openWriter, openWriter.append('x')); } since the framework should catch and report errant exceptions, without the need for code to catch exceptions and call fail(). s'marks From srinivas.dama at oracle.com Fri Feb 2 08:53:41 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Fri, 2 Feb 2018 00:53:41 -0800 (PST) Subject: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) In-Reply-To: <5A708DCA.5030006@oracle.com> References: <3558275d-6cd3-47bc-8203-da263835f747@default> <86f58aa7-2805-7c4b-4130-da1b520e47da@oracle.com> <31fd76f5-fba8-418e-9713-e0ae5b5bc601@default> <960de74d-0b48-2c1d-82ce-85cf7bf00d21@oracle.com> <5A708DCA.5030006@oracle.com> Message-ID: <5def6f35-392a-4493-acef-821577656ad8@default> Hi Alan/Sundar, Please review the revised webrev with copyright header change at http://cr.openjdk.java.net/~sdama/8011697/webrev.02/ Regards, srinivas -----Original Message----- From: Sundararajan Athijegannathan Sent: Tuesday, January 30, 2018 8:53 PM To: core-libs-dev at openjdk.java.net Subject: Re: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) +1 (with the copyright header change suggested by Alan) -Sundar On 30/01/18, 8:28 PM, Alan Bateman wrote: > On 30/01/2018 09:17, Srinivas Dama wrote: >> Hi, >> >> Please review the revised webrev at >> http://cr.openjdk.java.net/~sdama/8011697/webrev.01/ for >> https://bugs.openjdk.java.net/browse/JDK-8011697 >> >> > The updated patch to ScriptEngineManager looks okay. > > Can you replace the copyright header in the tests with the GPL version > before you push this? We don't use the GPL + "Classpath" exception > header in tests. > > -Alan From sundararajan.athijegannathan at oracle.com Fri Feb 2 09:10:14 2018 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 02 Feb 2018 14:40:14 +0530 Subject: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) In-Reply-To: <5def6f35-392a-4493-acef-821577656ad8@default> References: <3558275d-6cd3-47bc-8203-da263835f747@default> <86f58aa7-2805-7c4b-4130-da1b520e47da@oracle.com> <31fd76f5-fba8-418e-9713-e0ae5b5bc601@default> <960de74d-0b48-2c1d-82ce-85cf7bf00d21@oracle.com> <5A708DCA.5030006@oracle.com> <5def6f35-392a-4493-acef-821577656ad8@default> Message-ID: <5A742AF6.5030701@oracle.com> Looks good. -Sundar On 02/02/18, 2:23 PM, Srinivas Dama wrote: > Hi Alan/Sundar, > > Please review the revised webrev with copyright header change at > http://cr.openjdk.java.net/~sdama/8011697/webrev.02/ > > Regards, > srinivas > > -----Original Message----- > From: Sundararajan Athijegannathan > Sent: Tuesday, January 30, 2018 8:53 PM > To: core-libs-dev at openjdk.java.net > Subject: Re: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) > > +1 > > (with the copyright header change suggested by Alan) > > -Sundar > > On 30/01/18, 8:28 PM, Alan Bateman wrote: >> On 30/01/2018 09:17, Srinivas Dama wrote: >>> Hi, >>> >>> Please review the revised webrev at >>> http://cr.openjdk.java.net/~sdama/8011697/webrev.01/ for >>> https://bugs.openjdk.java.net/browse/JDK-8011697 >>> >>> >> The updated patch to ScriptEngineManager looks okay. >> >> Can you replace the copyright header in the tests with the GPL version >> before you push this? We don't use the GPL + "Classpath" exception >> header in tests. >> >> -Alan From Alan.Bateman at oracle.com Fri Feb 2 09:10:46 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 2 Feb 2018 09:10:46 +0000 Subject: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) In-Reply-To: <5def6f35-392a-4493-acef-821577656ad8@default> References: <3558275d-6cd3-47bc-8203-da263835f747@default> <86f58aa7-2805-7c4b-4130-da1b520e47da@oracle.com> <31fd76f5-fba8-418e-9713-e0ae5b5bc601@default> <960de74d-0b48-2c1d-82ce-85cf7bf00d21@oracle.com> <5A708DCA.5030006@oracle.com> <5def6f35-392a-4493-acef-821577656ad8@default> Message-ID: <5e1d7cfa-a6f7-4873-851a-f6f9ee895bc8@oracle.com> On 02/02/2018 08:53, Srinivas Dama wrote: > Hi Alan/Sundar, > > Please review the revised webrev with copyright header change at > http://cr.openjdk.java.net/~sdama/8011697/webrev.02/ > > Looks good. From Ulf.Zibis at CoSoCo.de Fri Feb 2 12:07:42 2018 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Fri, 2 Feb 2018 13:07:42 +0100 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: <3bb16c35-183a-6292-d7a6-7b8b73606969@oracle.com> References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> <54b64c89-3cfe-c4f6-e343-cb9c42f814d3@CoSoCo.de> <3bb16c35-183a-6292-d7a6-7b8b73606969@oracle.com> Message-ID: <6cf9a63f-590c-f21c-ce23-4bc85ac4c30a@CoSoCo.de> Hi Claes, sorry for the delay, I didn't catch you message before, because you CC'd me which causes that my email filter misses the ListID tag to catch. Thanks for the update, times change with the compact strings feature. Another approach could be to access the String constant via Unsafe. -Ulf Am 30.01.2018 um 11:12 schrieb Claes Redestad: > Hi, > > not sure what you're suggesting, exactly, but it seems Charset coders > generate > String literals which are then unpacked to char[]/byte[]'s in static > blocks. > > While constructing arrays from String literals might have been a startup > optimization in the past, it is now mainly a workaround to method > bytecode > limits, see https://bugs.openjdk.java.net/browse/JDK-8185104 > > /Claes > > On 2018-01-30 01:10, Ulf Zibis wrote: >> Hi, >> >> you may can construct the lookup table as a static String constant to >> slim down the footprint and treat it as a byte[] as we do in the >> Charset coders. >> >> -Ulf >> >> >> Am 29.01.2018 um 22:04 schrieb Claes Redestad: >>> I'll experiment and analyze a bit more tomorrow to see if I can find a >>> good explanation for the observed difference and/or a solution that >>> allows us to slim down the lookup array. >> > > From amy.lu at oracle.com Fri Feb 2 15:02:00 2018 From: amy.lu at oracle.com (Amy Lu) Date: Fri, 2 Feb 2018 23:02:00 +0800 Subject: [JDK 11] RFR 8195981: Move some tests to OpenJDK for jdk_lang test group Message-ID: <6fe6e272-1196-7614-7bb9-e9b3041530b0@oracle.com> Please review the patch to move some tests to OpenJDK for jdk_lang test group. bug: https://bugs.openjdk.java.net/browse/JDK-8195981 webrev: http://cr.openjdk.java.net/~amlu/8195981/webrev.00/ Thanks, Amy From chris.hegarty at oracle.com Fri Feb 2 16:30:03 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 2 Feb 2018 16:30:03 +0000 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> Message-ID: <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> Roger, On 01/02/18 21:29, Roger Riggs wrote: > Hi Chris, > > Thanks for the review and suggestion. > > Webrev updated: > http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ This looks good to me, just a few small comments: 1) windows SocketImpl.c in the comments: java_net_AbstractPlainSocketImpl -> java_net_SocketCleanable_cleanupClose0 2) SocketCleanable.java needs a copyright header -Chris. > * Refactored SocketCleanup into a java.net package private SocketCleanable > * Moved register and unregister into static methods in SocketCleanable > * Simplified the test by retaining the checking for GC reclaimed > objects but > removed checking fd counts, since they were unreliable in testing > and not consistent across OS's > > Thanks, Roger > > On 2/1/2018 10:11 AM, Chris Hegarty wrote: >> Hi Roger, >> >>> On 31 Jan 2018, at 15:52, Roger Riggs wrote: >>> >>> Addingnet-dev at openjdk.java.net >>> >>> On 1/30/2018 5:08 PM, Roger Riggs wrote: >>>> Please review changes to replace finalizers in socket, datagram, and multicast networking >>>> with Cleaner based release of the raw file descriptors. Each FileDescriptor is registered >>>> for cleanup after the raw fd (or handle) is assigned. Normal calls to close unregister the >>>> cleaner before using the current logic to close the raw fd/handle. Windows networking >>>> uses fd's with the Windows socket_ API requiring a special cased Cleanable. >>>> >>>> The tests check that the implementation objects including FileDescriptors are reclaimed >>>> and for Linux that the raw fd counts are reduced as expected. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >> I think this is good. One small comment; could SocketCleanup be a >> top-level package-private class, since it is shared by the three >> different socket types. >> >> I didn?t look too hard at the tests, other than to note that they seem >> to verify expected behaviour, which is good. >> >> -Chris. > From Roger.Riggs at Oracle.com Fri Feb 2 17:07:04 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 2 Feb 2018 12:07:04 -0500 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> Message-ID: Hi Chris, Updated in place. ? http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ Thanks, Roger On 2/2/2018 11:30 AM, Chris Hegarty wrote: > Roger, > > On 01/02/18 21:29, Roger Riggs wrote: >> Hi Chris, >> >> Thanks for the review and suggestion. >> >> Webrev updated: >> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ > > This looks good to me, just a few small comments: > > 1) windows SocketImpl.c in the comments: > ???? java_net_AbstractPlainSocketImpl -> > java_net_SocketCleanable_cleanupClose0 > > 2) SocketCleanable.java needs a copyright header > > -Chris. > >> ? * Refactored SocketCleanup into a java.net package private >> SocketCleanable >> ? * Moved register and unregister into static methods in SocketCleanable >> ? * Simplified the test by retaining the checking for GC reclaimed >> ??? objects but >> ??? removed checking fd counts, since they were unreliable in testing >> ??? and not consistent across OS's >> >> Thanks, Roger >> >> On 2/1/2018 10:11 AM, Chris Hegarty wrote: >>> Hi Roger, >>> >>>> On 31 Jan 2018, at 15:52, Roger Riggs? wrote: >>>> >>>> Addingnet-dev at openjdk.java.net >>>> >>>> On 1/30/2018 5:08 PM, Roger Riggs wrote: >>>>> Please review changes to replace finalizers in socket, datagram, >>>>> and multicast networking >>>>> with Cleaner based release of the raw file descriptors. Each >>>>> FileDescriptor is registered >>>>> for cleanup after the raw fd (or handle) is assigned. Normal calls >>>>> to close unregister the >>>>> cleaner before using the current logic to close the raw fd/handle. >>>>> Windows networking >>>>> uses fd's with the Windows socket_ API requiring a special cased >>>>> Cleanable. >>>>> >>>>> The tests check that the implementation objects including >>>>> FileDescriptors are reclaimed >>>>> and for Linux that the raw fd counts are reduced as expected. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >>> I think this is good. One small comment; could SocketCleanup be a >>> top-level package-private class, since it is shared by the three >>> different socket types. >>> >>> I didn?t look too hard at the tests, other than to note that they seem >>> to verify expected behaviour, which is good. >>> >>> -Chris. >> From chris.hegarty at oracle.com Fri Feb 2 18:07:21 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 2 Feb 2018 18:07:21 +0000 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> Message-ID: On 02/02/18 17:07, Roger Riggs wrote: > Hi Chris, > > Updated in place. > http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ Looks good to me. Trivially ( no need to re-generate the webrev ), in windows SocketImpl.c java*_net*_java_net_SocketCleanable -Chris. From peter.levart at gmail.com Fri Feb 2 18:16:31 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 2 Feb 2018 19:16:31 +0100 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> Message-ID: <51530bd3-4635-2b24-bfbf-b0aef7742282@gmail.com> Hi Roger, Nice separation of concerns (io vs. net). Is JavaIOFileDescriptorAccess.registerCleanup(FileDescriptor) currently used at all? Although not necessary for this patch, but to make code more symmetric, FileDecriptor.FDCleaner could also be extracted into a package-private top class and equipped with similar static (un)registration methods as SocketCleanable (possibly also renamed to FileCleanable and made hosting the native method). You could then remove the overload of FileDescriptor.registerCleanup() and corresponding JavaIOFileDescriptorAccess method and make calling code use FileCleanable.(un)register methods instead. You could then add checking for non-nullness of PhantomCleanable in FileDescriptor.registerCleanup(PhantomCleanable) or make the null value behave as unregisterCleanup() so that you only need one method. Currently if you mistakenly pass null to this method from socket code, FDCleanup gets registered. As FileDescriptor is a public class, you could also make FileCleanable and SocketCleanable extend PhantomCleanable and make the method have the following signature FileDescriptor.registerCleanup(PhantomCleanable). The code is good as is, but it's a little harder to follow since it is different for two similar use cases (net vs. io). What do you think? Regards, Peter On 02/02/18 18:07, Roger Riggs wrote: > Hi Chris, > > Updated in place. > http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ > > Thanks, Roger > > > On 2/2/2018 11:30 AM, Chris Hegarty wrote: >> Roger, >> >> On 01/02/18 21:29, Roger Riggs wrote: >>> Hi Chris, >>> >>> Thanks for the review and suggestion. >>> >>> Webrev updated: >>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >> >> This looks good to me, just a few small comments: >> >> 1) windows SocketImpl.c in the comments: >> ???? java_net_AbstractPlainSocketImpl -> >> java_net_SocketCleanable_cleanupClose0 >> >> 2) SocketCleanable.java needs a copyright header >> >> -Chris. >> >>> * Refactored SocketCleanup into a java.net package private >>> SocketCleanable >>> ? * Moved register and unregister into static methods in >>> SocketCleanable >>> ? * Simplified the test by retaining the checking for GC reclaimed >>> ??? objects but >>> ??? removed checking fd counts, since they were unreliable in testing >>> ??? and not consistent across OS's >>> >>> Thanks, Roger >>> >>> On 2/1/2018 10:11 AM, Chris Hegarty wrote: >>>> Hi Roger, >>>> >>>>> On 31 Jan 2018, at 15:52, Roger Riggs wrote: >>>>> >>>>> Addingnet-dev at openjdk.java.net >>>>> >>>>> On 1/30/2018 5:08 PM, Roger Riggs wrote: >>>>>> Please review changes to replace finalizers in socket, datagram, >>>>>> and multicast networking >>>>>> with Cleaner based release of the raw file descriptors. Each >>>>>> FileDescriptor is registered >>>>>> for cleanup after the raw fd (or handle) is assigned. Normal >>>>>> calls to close unregister the >>>>>> cleaner before using the current logic to close the raw >>>>>> fd/handle. Windows networking >>>>>> uses fd's with the Windows socket_ API requiring a special cased >>>>>> Cleanable. >>>>>> >>>>>> The tests check that the implementation objects including >>>>>> FileDescriptors are reclaimed >>>>>> and for Linux that the raw fd counts are reduced as expected. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >>>> I think this is good. One small comment; could SocketCleanup be a >>>> top-level package-private class, since it is shared by the three >>>> different socket types. >>>> >>>> I didn?t look too hard at the tests, other than to note that they seem >>>> to verify expected behaviour, which is good. >>>> >>>> -Chris. >>> > From mandy.chung at oracle.com Fri Feb 2 18:36:53 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 2 Feb 2018 10:36:53 -0800 Subject: [JDK 11] RFR 8195981: Move some tests to OpenJDK for jdk_lang test group In-Reply-To: <6fe6e272-1196-7614-7bb9-e9b3041530b0@oracle.com> References: <6fe6e272-1196-7614-7bb9-e9b3041530b0@oracle.com> Message-ID: <0682b6a6-19b6-b7a7-a9ea-641ddd41fd2c@oracle.com> On 2/2/18 7:02 AM, Amy Lu wrote: > Please review the patch to move some tests to OpenJDK for jdk_lang > test group. > > bug: https://bugs.openjdk.java.net/browse/JDK-8195981 > webrev: http://cr.openjdk.java.net/~amlu/8195981/webrev.00/ > > I suggest to merge jdk/vm/misc and jdk/vm/monitor directory and rename to jdk/vm/runtime which avoids "misc" a dumping group and is explicit that this directory is for runtime-related tests. JITClassInit.java seems a JIT related test and maybe moved it to jdk/vm/jit. ExceptionInInit.java Can you reformat the comment block line 59-95 where some lines should be merged into the line above. Bug4404588.java can be renamed to reflect what this test does. What about renaming it to java/lang/Character/UnicodeBlock/SpecialsUnicodeBlock.java? Otherwise, looks okay. Mandy From philipp.kunz at paratix.ch Fri Feb 2 18:52:55 2018 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Fri, 2 Feb 2018 19:52:55 +0100 Subject: JDK-6372077: JarFile.getManifest() should handle manifest attribute names up to 70 bytes In-Reply-To: <9a08b2c0-3659-1833-f113-23aaceca84bc@Oracle.com> References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> <02abcb68-2294-c733-5048-b1f81ee99435@oracle.com> <88b89995-cfe0-d439-d63d-699c628e2315@paratix.ch> <07e62d8d-ae3a-9ffa-d426-81ab548ad252@Oracle.com> <9a08b2c0-3659-1833-f113-23aaceca84bc@Oracle.com> Message-ID: Hi Roger Glad to send the patch. I also tried to write a meaningful and useful test. Please tell me ruthlessly if it makes sense or what not. Looking forward to progress in a bug that has been open for more than 10 years now. Philipp On 22.01.2018 21:03, Roger Riggs wrote: > Hi Philipp, > > I'm tending to agree with the suggestion about line length interpretation. > To meet OpenJDK IP requirements, please attach the .patch file or > include it in the body > of the message. > > Thanks, Roger > > On 12/18/2017 11:17 PM, Philipp Kunz wrote: >> Hi Roger, >> >> My suggested and also preferred approach is to read the manifest >> specification [1] in a way such that the line breaks are not included >> when counting the maximum line length. The specification does not >> state explicitly whether or not to include line breaks within the >> maximum line length limit but the following sentence from the >> specifications gives a hint: >> >> Because header names cannot be continued, the maximum length of a >> header name is 70 bytes (there must be a colon and a SPACE after the >> name). >> >> Above statement can be true only if line breaks are not counted for >> the maximum line length limit. Assuming so would in my opinion allow >> to understand the complete manifest specification without a conflict >> and effectively result in wider manifest files (maximum each line), >> wider by two bytes of a line break. >> >> In the meantime since the mail you replied to, I created a patch [3] >> mentioned in [2] which might be useful provided the manifest >> specification discussion is resolved. >> >> Regards, >> Philipp >> >> >> [1] >> https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Notes_on_Manifest_and_Signature_Files >> / >> https://docs.oracle.com/javase/9/docs/specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files >> [2] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050500.html >> [3] http://files.paratix.ch/jdk/6372077/webrev.01/ >> >> >> >> On 18.12.2017 16:46, Roger Riggs wrote: >>> Hi Phillip, >>> >>> Sorry for the silence... >>> >>> I/we haven't had time to full understand the ramifications of the >>> change you propose. >>> It seems there is a difficult/unresolvable conflict in the >>> specifications between the line length >>> requirements and the header specs. >>> >>> Regards, Roger >>> >>> On 11/21/2017 1:18 AM, Philipp Kunz wrote: >>>> Hi everyone, >>>> >>>> I haven't got any reply now for around three weeks and now i start >>>> to wonder if I just missed it or if I should refine my approach to >>>> find a sponsor or if it helped if I presented a ready patch or if >>>> noone considers this important enough or anything else whatever. >>>> This is only my second contribution hence I don't know the >>>> procedure well. >>>> >>>> One point maybe worth to point out again is that I don't want to >>>> support manifest headers longer than 70 character, just up to 70, >>>> which has always been the intention but has only worked up to 68. >>>> This might have been written confusingly in my last email. >>>> >>>> As far as I understood, I should first get a sponsor. In any case, >>>> is there any suggestion for how to proceed? >>>> >>>> Regards, >>>> Philipp >>>> >>>> >>>> >>>> On 03.11.2017 00:04, Philipp Kunz wrote: >>>>> Hi Sean and Max and all others, >>>>> >>>>> Thank you Sean for the hint about the right mailing list. And >>>>> thanks also for his hint to Max to make smaller portions of changes. >>>>> >>>>> I would like to contribute a fix for JDK-6372077 which is about >>>>> JarFile.getManifest() should handle manifest attribute name[s >>>>> longer than] 70 bytes. >>>>> >>>>> It looks like the bug is caused by Manifest.make72Safe breaking >>>>> lines at 70 bytes instead of 72 despite its comment and name >>>>> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Manifest.java#l176).The >>>>> resulting StringBuffer has then lines of 72 bytes maximum each >>>>> including the following line break. Without the line break that >>>>> leaves 70 bytes of characters per line. On the other hand, header >>>>> names can be up to 70 bytes (only single-byte utf-8 characters) >>>>> and cannot be broken across a line break and need to be followed >>>>> by a colon and a space which must be on the same line too >>>>> according to the specification. When breaking at 70 bytes >>>>> excluding the line break, however, long header names don't fit in >>>>> one line together with the colon space delimiter because there is >>>>> not sufficient space. >>>>> Manifests with names up to 70 bytes long can still be written >>>>> without immediate exception but the resulting manifests are >>>>> illegal in my opinion. When later reading such manifests >>>>> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Attributes.java#l406), >>>>> an error occurs as a consequence of the bad manifest. This is more >>>>> or less the current situation and also what JDK-6372077 already knew. >>>>> >>>>> --> After all, in order to fix it, i'd like to propose to make >>>>> manifest file lines wider by two bytes. >>>>> >>>>> The only proper alternative that came into my mind would be to >>>>> change the manifest specification and reduce the maximum header >>>>> name length there by two and also in the code. If that would break >>>>> any existing code i guess that would affect code only that >>>>> produced invalid manifests and would be acceptable. >>>>> >>>>> Supporting all existing and possibly invalid manifests would mean >>>>> to add support for reading headers the delimiters of which are >>>>> broken onto the next line which I consider too complex with >>>>> respect to the value added and even more so considering that those >>>>> invalid manifest can be assumed to have been detected as such by >>>>> reading them and also because it would be a feature that would be >>>>> used less and less over time if the code to write manifest is >>>>> changed at the same time to produce only valid manifests in the >>>>> discussed respect here. I don't think this should be actually done. >>>>> >>>>> Before I actually do the leg work, i'd like to ask, if there are >>>>> concerns or objections or tips for such a change or if anyone can >>>>> or cannot follow the reasoning and the conclusion to make >>>>> manifests 2 bytes wider or if i missed an important point altogether. >>>>> >>>>> In case there will be a consent about how to solve this, would >>>>> someone volunteer to sponsor? That may be less urgent at the >>>>> moment than the question above about how to proceed. >>>>> >>>>> Philipp >>>>> >>>>> >>>>> On 12.10.2017 22:32, Sean Mullan wrote: >>>>>> Hi Phillip, >>>>>> >>>>>> All of these bugs are in the core-libs area, so I am copying the >>>>>> core-libs-dev list since that is where they should be discussed >>>>>> and reviewed. I have -bcc-ed security-dev (where this was >>>>>> originally posted). >>>>>> >>>>>> --Sean >>>>>> >>>>>> On 10/2/17 1:24 PM, Philipp Kunz wrote: >>>>>>> Hi, >>>>>>> >>>>>>> While fixing JDK-6695402 I came across other related bugs to >>>>>>> manifests such as JDK-6372077, JDK-6202130, JDK-8176843, >>>>>>> JDK-4842483, and JDK-6443578 which all relate to manifest >>>>>>> reading and writing. Somewhere bug 7071055 is mentioned but I >>>>>>> cannot find anywhere. Another group of bugs, JDK-6910466, >>>>>>> JDK-4935610, and JDK-4271239 concern the mandatory manifest main >>>>>>> attributes Manifest-Version or Signature-Version and at first >>>>>>> glance are duplicates. If you know of more known bugs, not >>>>>>> necessarily present in jira, I'd be glad to get notified. >>>>>>> >>>>>>> There are also some comments about utf handling and line >>>>>>> breaking in the code of Manifest: >>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l299 >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l327 >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l370 >>>>>>> >>>>>>> >>>>>>> Furthermore, Attributes.map should declare appropriate type >>>>>>> parameters: >>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l61 >>>>>>> >>>>>>> The specification would also require that `header names must not >>>>>>> start with _, - or "From"` >>>>>>> (http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Section-Specification) >>>>>>> but I would opt not to add this as a hard restriction because I >>>>>>> guess it can be assumed that such names are in use now after >>>>>>> having been working for years. A warning to a logger might be >>>>>>> conceivable such as in >>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l424 >>>>>>> >>>>>>> Attribute values are never checked at all and invalid characters >>>>>>> such as line breaks are never detected except that when reading >>>>>>> the manifest again the values are cut off. >>>>>>> The tab character (U+0008) does not work in manifest values. >>>>>>> I suspect that there are also issues regarding the iteration >>>>>>> order but I haven't got a prove yet unlike for the other points >>>>>>> above: >>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Manifest.java#l54 >>>>>>> >>>>>>> There is duplicated or very similar code in Attributes and >>>>>>> Manifest: Attributes.write-Manifest.write-Attributes.writeMain >>>>>>> and Attributes.read-Manifest.read. >>>>>>> Resolving JDK-6202130 would have the additional benefit to be >>>>>>> able to view manifests with any utf-8 capable editor even if >>>>>>> multi-byte characters break across lines. >>>>>>> >>>>>>> Fixing these issues individually looks like more complicated >>>>>>> work than fixing them all at once, I guess, also because of a >>>>>>> very low current test coverage. So I'd suggest to add some >>>>>>> thorough tests along with fixing these issues. But before I >>>>>>> start I would like to get some guidance, assistance or at least >>>>>>> confirmation on how to proceed. I'm new to open jdk and have >>>>>>> submitted only one patch so far. >>>>>>> >>>>>>> Is it ok to add tests for things that have worked before? >>>>>>> Is it ok to refactor duplicated code just for the added value to >>>>>>> reduce effort for testing? >>>>>>> I assume compatibility to and from existing manifests is the >>>>>>> highest priority, correct? This would also be the hard part in >>>>>>> adding as complete test coverage as possible. What would be >>>>>>> acceptable criteria to meet? >>>>>>> Why does Attributes not extend LinkedHashMap and why does >>>>>>> Manifest not extend HashMap? Any objection? >>>>>>> While I would not want to write code that looks slow or change >>>>>>> more than necessary one can never know before having performance >>>>>>> actually measured. Is there some way this is dealt with or >>>>>>> should I wait for such feedback until after patch submission? >>>>>>> >>>>>>> Would someone sponsor? >>>>>>> >>>>>>> Regards, >>>>>>> Philipp >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> ------------------------------------------------------------------------ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Paratix GmbH >>>>>>> St Peterhofstatt 11 >>>>>>> 8001 Z?rich >>>>>>> >>>>>>> +41 (0)76 397 79 35 >>>>>>> philipp.kunz at paratix.ch >>>>> >>>>> -- >>>>> >>>>> >>>>> Gruss Philipp >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> >>>>> Paratix GmbH >>>>> St Peterhofstatt 11 >>>>> 8001 Z?rich >>>>> >>>>> +41 (0)76 397 79 35 >>>>> philipp.kunz at paratix.ch >>>> >>> >> > -- Gruss Philipp ------------------------------------------------------------------------ Paratix GmbH St Peterhofstatt 11 8001 Z?rich +41 (0)76 397 79 35 philipp.kunz at paratix.ch -------------- next part -------------- A non-text attachment was scrubbed... Name: 6372077.patch Type: text/x-patch Size: 16060 bytes Desc: not available URL: From huizhe.wang at oracle.com Fri Feb 2 19:01:20 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 2 Feb 2018 11:01:20 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A72A646.7040101@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> Message-ID: Hi, Thanks all for comments and suggestions. I've updated the webrev. Please review. JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ Thanks, Joe On 1/31/2018 9:31 PM, Joe Wang wrote: > Hi Tagir, > > Thanks for the comment. I will consider adding that to the javadoc > emphasizing that the comparison is performed from 0 to length() - 1 of > the two sequences. > > Best, > Joe > > On 1/29/18, 8:07 PM, Tagir Valeev wrote: >> Hello! >> >> An AbstractStringBuilder#compareTo implementation is wrong. You cannot >> simply compare the whole byte array. Here's the test-case: >> >> public class Test { >> ?? public static void main(String[] args) { >> ???? StringBuilder sb1 = new StringBuilder("test1"); >> ???? StringBuilder sb2 = new StringBuilder("test2"); >> ???? sb1.setLength(4); >> ???? sb2.setLength(4); >> ???? System.out.println(sb1.compareTo(sb2)); >> System.out.println(sb1.toString().compareTo(sb2.toString())); >> ?? } >> } >> >> We truncated the stringbuilders making their content equal, so >> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >> the original content (before the truncation) as truncation, of course, >> does not zero the truncated bytes, neither does it reallocate the >> array (unless explicitly asked via trimToSize). >> >> With best regards, >> Tagir Valeev. >> >> >> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang? >> wrote: >>> Hi, >>> >>> Adding methods for comparing CharSequence, StringBuilder, and >>> StringBuffer. >>> >>> The Comparable implementations for StringBuilder/Buffer are similar >>> to that >>> of String, allowing comparison operations between two >>> StringBuilders/Buffers, e.g. >>> aStringBuilder.compareTo(anotherStringBuilder). >>> For CharSequence however, refer to the comments in JIRA, a static >>> method >>> 'compare' is added instead of implementing the Comparable interface. >>> This >>> 'compare' method may take CharSequence implementations such as String, >>> StringBuilder and StringBuffer, making it possible to perform >>> comparison >>> among them. The previous example for example is equivalent to >>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>> >>> Tests for java.base have been independent from each other. The new >>> tests are >>> therefore created to have no dependency on each other or sharing any >>> code. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> Thanks, >>> Joe From jason_mehrens at hotmail.com Fri Feb 2 19:22:54 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Fri, 2 Feb 2018 19:22:54 +0000 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com>, Message-ID: Joe, The identity check in CS.compare makes sense. However, it won't be null hostile if we call CS.compare(null, null) and that doesn't seem right. Usually when writing comparator classes I end up with: === if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { return 0; } === Jason ________________________________________ From: core-libs-dev on behalf of Joe Wang Sent: Friday, February 2, 2018 1:01 PM To: core-libs-dev Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer Hi, Thanks all for comments and suggestions. I've updated the webrev. Please review. JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ Thanks, Joe On 1/31/2018 9:31 PM, Joe Wang wrote: > Hi Tagir, > > Thanks for the comment. I will consider adding that to the javadoc > emphasizing that the comparison is performed from 0 to length() - 1 of > the two sequences. > > Best, > Joe > > On 1/29/18, 8:07 PM, Tagir Valeev wrote: >> Hello! >> >> An AbstractStringBuilder#compareTo implementation is wrong. You cannot >> simply compare the whole byte array. Here's the test-case: >> >> public class Test { >> public static void main(String[] args) { >> StringBuilder sb1 = new StringBuilder("test1"); >> StringBuilder sb2 = new StringBuilder("test2"); >> sb1.setLength(4); >> sb2.setLength(4); >> System.out.println(sb1.compareTo(sb2)); >> System.out.println(sb1.toString().compareTo(sb2.toString())); >> } >> } >> >> We truncated the stringbuilders making their content equal, so >> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >> the original content (before the truncation) as truncation, of course, >> does not zero the truncated bytes, neither does it reallocate the >> array (unless explicitly asked via trimToSize). >> >> With best regards, >> Tagir Valeev. >> >> >> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang? >> wrote: >>> Hi, >>> >>> Adding methods for comparing CharSequence, StringBuilder, and >>> StringBuffer. >>> >>> The Comparable implementations for StringBuilder/Buffer are similar >>> to that >>> of String, allowing comparison operations between two >>> StringBuilders/Buffers, e.g. >>> aStringBuilder.compareTo(anotherStringBuilder). >>> For CharSequence however, refer to the comments in JIRA, a static >>> method >>> 'compare' is added instead of implementing the Comparable interface. >>> This >>> 'compare' method may take CharSequence implementations such as String, >>> StringBuilder and StringBuffer, making it possible to perform >>> comparison >>> among them. The previous example for example is equivalent to >>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>> >>> Tests for java.base have been independent from each other. The new >>> tests are >>> therefore created to have no dependency on each other or sharing any >>> code. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> Thanks, >>> Joe From Roger.Riggs at Oracle.com Fri Feb 2 20:22:32 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 2 Feb 2018 15:22:32 -0500 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: <51530bd3-4635-2b24-bfbf-b0aef7742282@gmail.com> References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> <51530bd3-4635-2b24-bfbf-b0aef7742282@gmail.com> Message-ID: Hi Peter, I filed a new issue for the cleanup: ?? https://bugs.openjdk.java.net/browse/JDK-8196716 On 2/2/2018 1:16 PM, Peter Levart wrote: > Hi Roger, > > Nice separation of concerns (io vs. net). > > Is JavaIOFileDescriptorAccess.registerCleanup(FileDescriptor) > currently used at all? I have not gotten around to looking at the channels implementations of freeing files/sockets. and it should be able to use the normal cleanup of raw fd/handles. Windows networking is asymmetric necessitating the alternate cleanup for sockets. > > Although not necessary for this patch, but to make code more > symmetric, FileDecriptor.FDCleaner could also be extracted into a > package-private top class and equipped with similar static > (un)registration methods as SocketCleanable (possibly also renamed to > FileCleanable and made hosting the native method). You could then > remove the overload of FileDescriptor.registerCleanup() and > corresponding JavaIOFileDescriptorAccess method and make calling code > use FileCleanable.(un)register methods instead. You could then add > checking for non-nullness of PhantomCleanable in > FileDescriptor.registerCleanup(PhantomCleanable) or make the null > value behave as unregisterCleanup() so that you only need one method. > Currently if you mistakenly pass null to this method from socket code, > FDCleanup gets registered. > > As FileDescriptor is a public class, you could also make FileCleanable > and SocketCleanable extend PhantomCleanable and make > the method have the following signature > FileDescriptor.registerCleanup(PhantomCleanable). > > The code is good as is, but it's a little harder to follow since it is > different for two similar use cases (net vs. io). > > What do you think? I'll closer look next week. Thanks, Roger > > Regards, Peter > > On 02/02/18 18:07, Roger Riggs wrote: >> Hi Chris, >> >> Updated in place. >> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >> >> Thanks, Roger >> >> >> On 2/2/2018 11:30 AM, Chris Hegarty wrote: >>> Roger, >>> >>> On 01/02/18 21:29, Roger Riggs wrote: >>>> Hi Chris, >>>> >>>> Thanks for the review and suggestion. >>>> >>>> Webrev updated: >>>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >>> >>> This looks good to me, just a few small comments: >>> >>> 1) windows SocketImpl.c in the comments: >>> ???? java_net_AbstractPlainSocketImpl -> >>> java_net_SocketCleanable_cleanupClose0 >>> >>> 2) SocketCleanable.java needs a copyright header >>> >>> -Chris. >>> >>>> * Refactored SocketCleanup into a java.net package private >>>> SocketCleanable >>>> ? * Moved register and unregister into static methods in >>>> SocketCleanable >>>> ? * Simplified the test by retaining the checking for GC reclaimed >>>> ??? objects but >>>> ??? removed checking fd counts, since they were unreliable in testing >>>> ??? and not consistent across OS's >>>> >>>> Thanks, Roger >>>> >>>> On 2/1/2018 10:11 AM, Chris Hegarty wrote: >>>>> Hi Roger, >>>>> >>>>>> On 31 Jan 2018, at 15:52, Roger Riggs wrote: >>>>>> >>>>>> Addingnet-dev at openjdk.java.net >>>>>> >>>>>> On 1/30/2018 5:08 PM, Roger Riggs wrote: >>>>>>> Please review changes to replace finalizers in socket, datagram, >>>>>>> and multicast networking >>>>>>> with Cleaner based release of the raw file descriptors.? Each >>>>>>> FileDescriptor is registered >>>>>>> for cleanup after the raw fd (or handle) is assigned. Normal >>>>>>> calls to close unregister the >>>>>>> cleaner before using the current logic to close the raw >>>>>>> fd/handle. Windows networking >>>>>>> uses fd's with the Windows socket_ API requiring a special cased >>>>>>> Cleanable. >>>>>>> >>>>>>> The tests check that the implementation objects including >>>>>>> FileDescriptors are reclaimed >>>>>>> and for Linux that the raw fd counts are reduced as expected. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >>>>> I think this is good. One small comment; could SocketCleanup be a >>>>> top-level package-private class, since it is shared by the three >>>>> different socket types. >>>>> >>>>> I didn?t look too hard at the tests, other than to note that they >>>>> seem >>>>> to verify expected behaviour, which is good. >>>>> >>>>> -Chris. >>>> >> > From huizhe.wang at oracle.com Fri Feb 2 20:36:18 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 2 Feb 2018 12:36:18 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8196717: remove xmlresolver.md Message-ID: <2849883a-7907-5b36-2bae-62e60ee45f38@oracle.com> Hi, Refer to JDK-8177561[1], XML Commons Resolver was removed from JDK 10. xmlresolver.md therefore needs to be removed as well. Please review: https://bugs.openjdk.java.net/browse/JDK-8196717 http://cr.openjdk.java.net/~joehw/jdk10/8196717/webrev/ [1] https://bugs.openjdk.java.net/browse/JDK-8177561 Thanks, Joe From Roger.Riggs at Oracle.com Fri Feb 2 20:43:35 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 2 Feb 2018 15:43:35 -0500 Subject: RFR (JDK10/JAXP Doc-only) 8196717: remove xmlresolver.md In-Reply-To: <2849883a-7907-5b36-2bae-62e60ee45f38@oracle.com> References: <2849883a-7907-5b36-2bae-62e60ee45f38@oracle.com> Message-ID: +1 On 2/2/2018 3:36 PM, Joe Wang wrote: > Hi, > > Refer to JDK-8177561[1], XML Commons Resolver was removed from JDK 10. > xmlresolver.md therefore needs to be removed as well. > > Please review: > https://bugs.openjdk.java.net/browse/JDK-8196717 > http://cr.openjdk.java.net/~joehw/jdk10/8196717/webrev/ > > [1] https://bugs.openjdk.java.net/browse/JDK-8177561 > > Thanks, > Joe > From lance.andersen at oracle.com Fri Feb 2 20:44:33 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 2 Feb 2018 15:44:33 -0500 Subject: RFR (JDK10/JAXP Doc-only) 8196717: remove xmlresolver.md In-Reply-To: <2849883a-7907-5b36-2bae-62e60ee45f38@oracle.com> References: <2849883a-7907-5b36-2bae-62e60ee45f38@oracle.com> Message-ID: go for it > On Feb 2, 2018, at 3:36 PM, Joe Wang wrote: > > Hi, > > Refer to JDK-8177561[1], XML Commons Resolver was removed from JDK 10. xmlresolver.md therefore needs to be removed as well. > > Please review: > https://bugs.openjdk.java.net/browse/JDK-8196717 > http://cr.openjdk.java.net/~joehw/jdk10/8196717/webrev/ > > [1] https://bugs.openjdk.java.net/browse/JDK-8177561 > > Thanks, > Joe > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From huizhe.wang at oracle.com Fri Feb 2 20:46:08 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 2 Feb 2018 12:46:08 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> Message-ID: <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> Thanks Jason. Will update that accordingly. Best, Joe On 2/2/2018 11:22 AM, Jason Mehrens wrote: > Joe, > > The identity check in CS.compare makes sense. However, it won't be null hostile if we call CS.compare(null, null) and that doesn't seem right. > Usually when writing comparator classes I end up with: > === > if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { > return 0; > } > === > > Jason > ________________________________________ > From: core-libs-dev on behalf of Joe Wang > Sent: Friday, February 2, 2018 1:01 PM > To: core-libs-dev > Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer > > Hi, > > Thanks all for comments and suggestions. I've updated the webrev. Please > review. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Thanks, > Joe > > On 1/31/2018 9:31 PM, Joe Wang wrote: >> Hi Tagir, >> >> Thanks for the comment. I will consider adding that to the javadoc >> emphasizing that the comparison is performed from 0 to length() - 1 of >> the two sequences. >> >> Best, >> Joe >> >> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>> Hello! >>> >>> An AbstractStringBuilder#compareTo implementation is wrong. You cannot >>> simply compare the whole byte array. Here's the test-case: >>> >>> public class Test { >>> public static void main(String[] args) { >>> StringBuilder sb1 = new StringBuilder("test1"); >>> StringBuilder sb2 = new StringBuilder("test2"); >>> sb1.setLength(4); >>> sb2.setLength(4); >>> System.out.println(sb1.compareTo(sb2)); >>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>> } >>> } >>> >>> We truncated the stringbuilders making their content equal, so >>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >>> the original content (before the truncation) as truncation, of course, >>> does not zero the truncated bytes, neither does it reallocate the >>> array (unless explicitly asked via trimToSize). >>> >>> With best regards, >>> Tagir Valeev. >>> >>> >>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>> wrote: >>>> Hi, >>>> >>>> Adding methods for comparing CharSequence, StringBuilder, and >>>> StringBuffer. >>>> >>>> The Comparable implementations for StringBuilder/Buffer are similar >>>> to that >>>> of String, allowing comparison operations between two >>>> StringBuilders/Buffers, e.g. >>>> aStringBuilder.compareTo(anotherStringBuilder). >>>> For CharSequence however, refer to the comments in JIRA, a static >>>> method >>>> 'compare' is added instead of implementing the Comparable interface. >>>> This >>>> 'compare' method may take CharSequence implementations such as String, >>>> StringBuilder and StringBuffer, making it possible to perform >>>> comparison >>>> among them. The previous example for example is equivalent to >>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>> >>>> Tests for java.base have been independent from each other. The new >>>> tests are >>>> therefore created to have no dependency on each other or sharing any >>>> code. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>> >>>> Thanks, >>>> Joe From huizhe.wang at oracle.com Fri Feb 2 20:47:07 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 2 Feb 2018 12:47:07 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8196717: remove xmlresolver.md In-Reply-To: References: <2849883a-7907-5b36-2bae-62e60ee45f38@oracle.com> Message-ID: <6932c0c7-84a9-e2c5-6cce-e5409b0a906f@oracle.com> Thanks Lance, Roger! Best, Joe On 2/2/2018 12:44 PM, Lance Andersen wrote: > go for it > >> On Feb 2, 2018, at 3:36 PM, Joe Wang > > wrote: >> >> Hi, >> >> Refer to JDK-8177561[1], XML Commons Resolver was removed from JDK >> 10. xmlresolver.md therefore needs to be removed as well. >> >> Please review: >> https://bugs.openjdk.java.net/browse/JDK-8196717 >> http://cr.openjdk.java.net/~joehw/jdk10/8196717/webrev/ >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8177561 >> >> Thanks, >> Joe >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From martinrb at google.com Fri Feb 2 21:14:21 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 2 Feb 2018 13:14:21 -0800 Subject: RFR: 8196609: Improve javadoc for java.time.Instant.getEpochSecond Message-ID: The nanos *are* hard to describe non-confusingly - resisting the urge to fix more globally. --- a/src/java.base/share/classes/java/time/Instant.java +++ b/src/java.base/share/classes/java/time/Instant.java @@ -610,7 +610,7 @@ *

* The epoch second count is a simple incrementing count of seconds where * second 0 is 1970-01-01T00:00:00Z. - * The nanosecond part of the day is returned by {@link #getNano}. + * The nanosecond part is returned by {@link #getNano}. * * @return the seconds from the epoch of 1970-01-01T00:00:00Z */ From hboehm at google.com Fri Feb 2 21:24:50 2018 From: hboehm at google.com (Hans Boehm) Date: Fri, 2 Feb 2018 13:24:50 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: <2F9C08AE-777D-416F-9C38-257D0B255459@oracle.com> References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> <2F9C08AE-777D-416F-9C38-257D0B255459@oracle.com> Message-ID: I agree that reachabilityFence() is the most expedient approach here. But my impression is increasingly that this is a pervasive and very subtle problem, and reachabilityFence() isn't enough. I don't think there are any perfect solutions available. But marking fields that are reclaimed/invalidated by cleaners seems significantly more intuitive than marking all the places they're used. Unfortunately, I didn't fully understand the practical issues here until recently, either. On Thu, Feb 1, 2018 at 4:23 PM, Paul Sandoz wrote: > > > On Feb 1, 2018, at 4:03 PM, Hans Boehm wrote: > > > On Thu, Feb 1, 2018 at 2:50 PM, Paul Sandoz > wrote: > > > > Hi Ben, > > > > I don?t think you require the fence in all the cases you have currently > placed it e.g. here for example > > > > $memtype$ y = $toBits$(x); > > UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); > > + Reference.reachabilityFence(this); > > return this; > > > > since ?this? is being returned it will be kept live during the unsafe > access. > > I don't see how that's guaranteed by the JLS, specifically 12.6.2. What if > this function is inlined into the caller, and the caller drops the result? > > > Ah yes, my apologies, i was thinking too narrowly. Your document reminds > we should really use the try/finally pattern for any updates DBB code for > reasons you suggest even if not strictly needed. > > > I would still greatly prefer to see an added mechanism (possibly along the > lines of https://docs.google.com/document/d/1yMC4VzZobMQTrVAraMy7xBdWPCJ0p > 7G5r_43yaqsj-s/edit?usp=sharing) that associates the reachability > requirements with fields or classes, rather than putting a > reachabilityFence() in every method accessing a field. I'm sure there are > cases where reachabilityFence() is the preferred mechanism. But for the > code I've looked at recently, I haven't actually found one yet. I don't > think it is a good mechanism here, either. > > > IIRC we discussed such approaches before and concluded reachabilityFence > was the most expedient way to get something functional into developers > hands if not in the most ideal and expressive way for all cases. > > > > > > Would you mind providing a JMH benchmark and results for the performance > with and without the fence for say a put and/or a get. I would like to > understand the performance impact on HotSpot, this is one reason why we > have yet to add such fences as it will likely impact performance. > > > > At the moment we are relying on the method not being inlined, which is > an expedient technique to make it functional and keep a reference alive but > not necessarily optimal for usages in DBB. > > But that should be fixable, right? > > > I believe so, it comes down to prioritizing and expending some compiler > engineering effort. > > Thanks, > Paul. > > > It shouldn't generate any code beyond potentially preventing register or > stack slot reuse; the required effect of reachabilityFence (in a > conventional implementation) is to ensure that the argument is considered > live at any preceding GC points. > > > > > For more details see: > > > > https://bugs.openjdk.java.net/browse/JDK-8169605 < > https://bugs.openjdk.java.net/browse/JDK-8169605> > > https://bugs.openjdk.java.net/browse/JDK-8149610 < > https://bugs.openjdk.java.net/browse/JDK-8149610> > > > > Thanks, > > Paul. > > > From hboehm at google.com Fri Feb 2 21:27:05 2018 From: hboehm at google.com (Hans Boehm) Date: Fri, 2 Feb 2018 13:27:05 -0800 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: <51530bd3-4635-2b24-bfbf-b0aef7742282@gmail.com> References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> <51530bd3-4635-2b24-bfbf-b0aef7742282@gmail.com> Message-ID: It would be great if this code eventually grew some comments about how we prevent cleanup code from running while the (low level integer) file descriptors are in use. AFAICT, this is probably mostly OK here, due to some combination of descriptor access bottoming out in non-static JNI methods and synchronized accesses. But the reasoning behind that seems unpleasantly global. E.g. why is FileDescriptor.getAppend() safe? In this case we know that the only caller is a constructor, which kind of requires write to this "afterwards", though that doesn't actually quite seem to follow 12.6.2 and earlier rules either. On Fri, Feb 2, 2018 at 10:16 AM, Peter Levart wrote: > Hi Roger, > > Nice separation of concerns (io vs. net). > > Is JavaIOFileDescriptorAccess.registerCleanup(FileDescriptor) currently > used at all? > > Although not necessary for this patch, but to make code more symmetric, > FileDecriptor.FDCleaner could also be extracted into a package-private top > class and equipped with similar static (un)registration methods as > SocketCleanable (possibly also renamed to FileCleanable and made hosting > the native method). You could then remove the overload of > FileDescriptor.registerCleanup() and corresponding > JavaIOFileDescriptorAccess method and make calling code use > FileCleanable.(un)register methods instead. You could then add checking for > non-nullness of PhantomCleanable in FileDescriptor.registerCleanup(PhantomCleanable) > or make the null value behave as unregisterCleanup() so that you only need > one method. Currently if you mistakenly pass null to this method from > socket code, FDCleanup gets registered. > > As FileDescriptor is a public class, you could also make FileCleanable and > SocketCleanable extend PhantomCleanable and make the > method have the following signature FileDescriptor.registerCleanup > (PhantomCleanable). > > The code is good as is, but it's a little harder to follow since it is > different for two similar use cases (net vs. io). > > What do you think? > > Regards, Peter > > > On 02/02/18 18:07, Roger Riggs wrote: > >> Hi Chris, >> >> Updated in place. >> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >> >> Thanks, Roger >> >> >> On 2/2/2018 11:30 AM, Chris Hegarty wrote: >> >>> Roger, >>> >>> On 01/02/18 21:29, Roger Riggs wrote: >>> >>>> Hi Chris, >>>> >>>> Thanks for the review and suggestion. >>>> >>>> Webrev updated: >>>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >>>> >>> >>> This looks good to me, just a few small comments: >>> >>> 1) windows SocketImpl.c in the comments: >>> java_net_AbstractPlainSocketImpl -> java_net_SocketCleanable_clean >>> upClose0 >>> >>> 2) SocketCleanable.java needs a copyright header >>> >>> -Chris. >>> >>> * Refactored SocketCleanup into a java.net package private >>>> SocketCleanable >>>> * Moved register and unregister into static methods in SocketCleanable >>>> * Simplified the test by retaining the checking for GC reclaimed >>>> objects but >>>> removed checking fd counts, since they were unreliable in testing >>>> and not consistent across OS's >>>> >>>> Thanks, Roger >>>> >>>> On 2/1/2018 10:11 AM, Chris Hegarty wrote: >>>> >>>>> Hi Roger, >>>>> >>>>> On 31 Jan 2018, at 15:52, Roger Riggs wrote: >>>>>> >>>>>> Addingnet-dev at openjdk.java.net >>>>>> >>>>>> On 1/30/2018 5:08 PM, Roger Riggs wrote: >>>>>> >>>>>>> Please review changes to replace finalizers in socket, datagram, and >>>>>>> multicast networking >>>>>>> with Cleaner based release of the raw file descriptors. Each >>>>>>> FileDescriptor is registered >>>>>>> for cleanup after the raw fd (or handle) is assigned. Normal calls >>>>>>> to close unregister the >>>>>>> cleaner before using the current logic to close the raw fd/handle. >>>>>>> Windows networking >>>>>>> uses fd's with the Windows socket_ API requiring a special cased >>>>>>> Cleanable. >>>>>>> >>>>>>> The tests check that the implementation objects including >>>>>>> FileDescriptors are reclaimed >>>>>>> and for Linux that the raw fd counts are reduced as expected. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ >>>>>>> >>>>>> I think this is good. One small comment; could SocketCleanup be a >>>>> top-level package-private class, since it is shared by the three >>>>> different socket types. >>>>> >>>>> I didn?t look too hard at the tests, other than to note that they seem >>>>> to verify expected behaviour, which is good. >>>>> >>>>> -Chris. >>>>> >>>> >>>> >> > From daniel.fuchs at oracle.com Fri Feb 2 21:39:47 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 2 Feb 2018 21:39:47 +0000 Subject: RFR: 8196609: Improve javadoc for java.time.Instant.getEpochSecond In-Reply-To: References: Message-ID: <8a9e706f-d598-7071-ffd7-256824fcc0d8@oracle.com> Hi Martin, Looks good to me! cheers, -- daniel On 02/02/18 21:14, Martin Buchholz wrote: > The nanos *are* hard to describe non-confusingly - resisting the urge to > fix more globally. > > --- a/src/java.base/share/classes/java/time/Instant.java > +++ b/src/java.base/share/classes/java/time/Instant.java > @@ -610,7 +610,7 @@ > *

> * The epoch second count is a simple incrementing count of seconds > where > * second 0 is 1970-01-01T00:00:00Z. > - * The nanosecond part of the day is returned by {@link #getNano}. > + * The nanosecond part is returned by {@link #getNano}. > * > * @return the seconds from the epoch of 1970-01-01T00:00:00Z > */ > From indu.bhagat at oracle.com Sat Feb 3 00:12:08 2018 From: indu.bhagat at oracle.com (Indu Bhagat) Date: Fri, 2 Feb 2018 16:12:08 -0800 Subject: [PATCH] Fix compiler warnings with newer GCC 7.2.1 Message-ID: This patch fixes a few of the compiler warnings when using gcc 7.2.1. In general there are many other warnings (a majority of which are implicit fallthrough warnings in jdk.crypto.ec package's ecp_*.c files and java.desktop package's splashscreen_gfx_impl.h file). I can post them if there is interest. Most warnings, however, look benign (modulo some warnings about unintialized access that I cannot comment on; they need that the code be thoroughly understood). libfdlibm misleading indentation warnings are already knownhttp://mail.openjdk.java.net/pipermail/jdk9-dev/2016-June/004494.html Thanks ---------------------------------------------- The patch fixes the following warnings : 1. /jdk/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c: In function ?mask_s2u?: /jdk/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:389:22: warning: left shift of negative value [-Wshift-negative-value] *mask = htonl(-1 << (32 - m)); ^ 2. /jdk/src/java.desktop/share/native/libjavajpeg/jdphuff.c: In function ?decode_mcu_AC_refine?: /jdk/src/java.desktop/share/native/libjavajpeg/jdphuff.c:505:17: warning: left shift of negative value [-Wshift-negative-value] int m1 = (-1) << cinfo->Al; /* -1 in the bit position being coded */ ^~ 3. /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc: In function ?void position_mark(const hb_ot_shape_plan_t*, hb_font_t*, hb_buffer_t*, hb_glyph_extents_t&, unsigned int, unsigned int)?: /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:223:14: warning: this statement may fall through [-Wimplicit-fallthrough=] } else if (buffer->props.direction == HB_DIRECTION_RTL) { ^~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:229:5: note: here default: ^~~~~~~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:261:27: warning: this statement may fall through [-Wimplicit-fallthrough=] base_extents.height -= y_gap; ~~~~~~~~~~~~~~~~~~~~^~~~~~~~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:264:5: note: here case HB_UNICODE_COMBINING_CLASS_ATTACHED_BELOW_LEFT: ^~~~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:282:27: warning: this statement may fall through [-Wimplicit-fallthrough=] base_extents.height -= y_gap; ~~~~~~~~~~~~~~~~~~~~^~~~~~~~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:285:5: note: here case HB_UNICODE_COMBINING_CLASS_ATTACHED_ABOVE: ^~~~ ----------- diff --git a/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh b/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh --- a/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh +++ b/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh @@ -158,6 +158,9 @@ */ # include # define HB_FALLTHROUGH __fallthrough +#elif __GNUC__ >= 7 + /* GNU fallthrough attribute is available from GCC7 */ +# define HB_FALLTHROUGH __attribute__((fallthrough)) #else # define HB_FALLTHROUGH /* FALLTHROUGH */ #endif diff --git a/src/java.desktop/share/native/libjavajpeg/jdphuff.c b/src/java.desktop/share/native/libjavajpeg/jdphuff.c --- a/src/java.desktop/share/native/libjavajpeg/jdphuff.c +++ b/src/java.desktop/share/native/libjavajpeg/jdphuff.c @@ -501,8 +501,10 @@ { phuff_entropy_ptr entropy = (phuff_entropy_ptr) cinfo->entropy; int Se = cinfo->Se; - int p1 = 1 << cinfo->Al; /* 1 in the bit position being coded */ - int m1 = (-1) << cinfo->Al; /* -1 in the bit position being coded */ + /* 1 in the bit position being coded */ + int p1 = 1 << cinfo->Al; + /* -1 in the bit position being coded */ + int m1 = (int)((unsigned)(~0) << cinfo->Al); register int s, k, r; unsigned int EOBRUN; JBLOCKROW block; diff --git a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c @@ -386,7 +386,7 @@ return instr; } - *mask = htonl(-1 << (32 - m)); + *mask = htonl((uint32_t)(~0) << (32 - m)); return s; } From peter.levart at gmail.com Sat Feb 3 01:04:41 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 3 Feb 2018 02:04:41 +0100 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: <604f59a2-a3f2-5c3f-067b-2002665b9441@gmail.com> Hi, I have one question, maybe stupid. I'm wondering about one situation. Suppose you have this Java code: void m() { ??? // code before... ??? Objects.requireNonNull(this); } Of course, the non-null check will never throw NPE. The check will most likely even be optimized away by JIT. But is JIT allowed to optimize it away so that the "code before" observes the effects of 'this' being unreachable? Wouldn't that violate the semantics of the program as it would 1st observe that some object is not reachable and after that it would observer that 'this' still points to some object which by definition must be it? If JIT is not allowed to optimize the null check so that the object becomes unreachable, then Objects.requireNonNull could be used as a replacement for Reference.reachabilityFence. It might even perform better. What do you think? Regards, Peter On 02/01/18 23:50, Paul Sandoz wrote: > Hi Ben, > > I don?t think you require the fence in all the cases you have currently placed it e.g. here for example > > $memtype$ y = $toBits$(x); > UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); > + Reference.reachabilityFence(this); > return this; > > since ?this? is being returned it will be kept live during the unsafe access. > > Would you mind providing a JMH benchmark and results for the performance with and without the fence for say a put and/or a get. I would like to understand the performance impact on HotSpot, this is one reason why we have yet to add such fences as it will likely impact performance. > > At the moment we are relying on the method not being inlined, which is an expedient technique to make it functional and keep a reference alive but not necessarily optimal for usages in DBB. > > For more details see: > > https://bugs.openjdk.java.net/browse/JDK-8169605 > https://bugs.openjdk.java.net/browse/JDK-8149610 > > Thanks, > Paul. > >> On Feb 1, 2018, at 6:55 AM, Ben Walsh wrote: >> >> This contribution forms a partial solution to the problem detailed here - >> http://thevirtualmachinist.blogspot.ca/2011/07/subtle-issue-of-reachability.html >> . >> >> In this context, this contribution provides "markers" such that a suitably >> "aware" compiler can reduce the chances of such a problem occurring with >> each of the DirectBuffer objects and MappedByteBuffer >> object. The reachabilityFences may prevent crashes / exceptions due to >> cleaning up the backing memory before the user has finished using the >> pointer. >> >> Any compiler not suitably "aware" could be modified to make use of the >> "markers". >> >> >> I would like to pair with a sponsor to contribute this patch ... >> >> ------------------------------------------------------------------------------------------------------------------- >> diff -r d51e64840b4f >> src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template >> --- >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template >> Wed Jan 31 12:04:53 2018 +0800 >> +++ >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template >> Thu Feb 01 11:30:10 2018 +0000 >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights >> reserved. >> + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights >> reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> @@ -33,15 +33,21 @@ >> >> private $type$ get$Type$(long a) { >> $memtype$ x = UNSAFE.get$Memtype$Unaligned(null, a, bigEndian); >> - return $fromBits$(x); >> + $type$ y = $fromBits$(x); >> + Reference.reachabilityFence(this); >> + return y; >> } >> >> public $type$ get$Type$() { >> - return get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); >> + $type$ y = get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); >> + Reference.reachabilityFence(this); >> + return y; >> } >> >> public $type$ get$Type$(int i) { >> - return get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); >> + $type$ y = get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); >> + Reference.reachabilityFence(this); >> + return y; >> } >> >> #end[rw] >> @@ -50,6 +56,7 @@ >> #if[rw] >> $memtype$ y = $toBits$(x); >> UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); >> + Reference.reachabilityFence(this); >> return this; >> #else[rw] >> throw new ReadOnlyBufferException(); >> diff -r d51e64840b4f >> src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> Wed Jan 31 12:04:53 2018 +0800 >> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> Thu Feb 01 11:30:10 2018 +0000 >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights >> reserved. >> + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights >> reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> @@ -28,6 +28,7 @@ >> package java.nio; >> >> import java.io.FileDescriptor; >> +import java.lang.ref.Reference; >> import jdk.internal.misc.VM; >> import jdk.internal.ref.Cleaner; >> import sun.nio.ch.DirectBuffer; >> @@ -312,6 +313,7 @@ >> public $Type$Buffer put($type$ x) { >> #if[rw] >> UNSAFE.put$Swaptype$(ix(nextPutIndex()), $swap$($toBits$(x))); >> + Reference.reachabilityFence(this); >> return this; >> #else[rw] >> throw new ReadOnlyBufferException(); >> @@ -321,6 +323,7 @@ >> public $Type$Buffer put(int i, $type$ x) { >> #if[rw] >> UNSAFE.put$Swaptype$(ix(checkIndex(i)), $swap$($toBits$(x))); >> + Reference.reachabilityFence(this); >> return this; >> #else[rw] >> throw new ReadOnlyBufferException(); >> @@ -347,6 +350,7 @@ >> if (srem > rem) >> throw new BufferOverflowException(); >> UNSAFE.copyMemory(sb.ix(spos), ix(pos), (long)srem << >> $LG_BYTES_PER_VALUE$); >> + Reference.reachabilityFence(this); >> sb.position(spos + srem); >> position(pos + srem); >> } else if (src.hb != null) { >> @@ -413,6 +417,7 @@ >> int rem = (pos <= lim ? lim - pos : 0); >> >> UNSAFE.copyMemory(ix(pos), ix(0), (long)rem << >> $LG_BYTES_PER_VALUE$); >> + Reference.reachabilityFence(this); >> position(rem); >> limit(capacity()); >> discardMark(); >> @@ -505,6 +510,7 @@ >> void _put(int i, byte b) { // package-private >> #if[rw] >> UNSAFE.putByte(address + i, b); >> + Reference.reachabilityFence(this); >> #else[rw] >> throw new ReadOnlyBufferException(); >> #end[rw] >> diff -r d51e64840b4f >> src/java.base/share/classes/java/nio/MappedByteBuffer.java >> --- a/src/java.base/share/classes/java/nio/MappedByteBuffer.java Wed Jan >> 31 12:04:53 2018 +0800 >> +++ b/src/java.base/share/classes/java/nio/MappedByteBuffer.java Thu Feb >> 01 11:30:10 2018 +0000 >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights >> reserved. >> + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights >> reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> @@ -26,6 +26,7 @@ >> package java.nio; >> >> import java.io.FileDescriptor; >> +import java.lang.ref.Reference; >> import jdk.internal.misc.Unsafe; >> >> >> @@ -164,6 +165,7 @@ >> // is computed as we go along to prevent the compiler from >> otherwise >> // considering the loop as dead code. >> Unsafe unsafe = Unsafe.getUnsafe(); >> + Reference.reachabilityFence(this); >> int ps = Bits.pageSize(); >> int count = Bits.pageCount(length); >> long a = mappingAddress(offset); >> ------------------------------------------------------------------------------------------------------------------- >> >> >> Regards, >> Ben Walsh >> >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >> From ivan.gerasimov at oracle.com Sat Feb 3 01:14:51 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 2 Feb 2018 17:14:51 -0800 Subject: RFR 8196740 : Character.digit(int, int) returns wrong value for out of range radix Message-ID: Hello! The recent fix for JDK-8196331 [1] appeared to slightly change the behavior. Specifically, if the radix is greater than MAX_RADIX then the method may return a non-negative value, even though the spec requires it must return -1 [2]. [1] https://bugs.openjdk.java.net/browse/JDK-8196331 [2] https://docs.oracle.com/javase/9/docs/api/java/lang/Character.html#digit-int-int- Would you please help review the fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8196740 WEBREV: http://cr.openjdk.java.net/~igerasim/8196740/00/webrev/ I suspect that this version may even work a tiny bit faster, as we perform less comparisons in a common case. Thanks in advance! -- With kind regards, Ivan Gerasimov From claes.redestad at oracle.com Sat Feb 3 01:47:36 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 3 Feb 2018 02:47:36 +0100 Subject: RFR 8196740 : Character.digit(int,int) returns wrong value for out of range radix In-Reply-To: References: Message-ID: Hi Ivan, On 2018-02-03 02:14, Ivan Gerasimov wrote: > Would you please help review the fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8196740 > WEBREV: http://cr.openjdk.java.net/~igerasim/8196740/00/webrev/ yes, an obvious error in hindsight! > > > I suspect that this version may even work a tiny bit faster, as we > perform less comparisons in a common case. Counter-intuitively it actually seems the compiler generates faster code in the end when keeping the value >= 0 test around. Your version: Benchmark Mode? Cnt???? Score?? Error? Units UUIDBenchmark.benchmarkUUIDFromString avgt??? 4???? 0.326 ? 0.017? us/op UUIDBenchmark.benchmarkUUIDFromString:branches avgt??????? 289.750?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:cycles avgt?????? 1044.087?????????? #/op With: ??????? int value = DIGITS[ch]; ??????? return (value >= 0 && value < radix && radix >= Character.MIN_RADIX ??????????????? && radix <= Character.MAX_RADIX) ? value : -1; Benchmark Mode? Cnt???? Score?? Error? Units UUIDBenchmark.benchmarkUUIDFromString avgt??? 4???? 0.310 ? 0.018? us/op UUIDBenchmark.benchmarkUUIDFromString:branches avgt??????? 256.489?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:cycles avgt??????? 988.998?????????? #/op /Claes From ivan.gerasimov at oracle.com Sat Feb 3 02:36:23 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 2 Feb 2018 18:36:23 -0800 Subject: RFR 8196740 : Character.digit(int,int) returns wrong value for out of range radix In-Reply-To: References: Message-ID: <10653089-6a16-2a6e-a695-36a012f0f41e@oracle.com> On 2/2/18 5:47 PM, Claes Redestad wrote: > Hi Ivan, > > On 2018-02-03 02:14, Ivan Gerasimov wrote: >> Would you please help review the fix? >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8196740 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8196740/00/webrev/ > > yes, an obvious error in hindsight! > >> >> >> I suspect that this version may even work a tiny bit faster, as we >> perform less comparisons in a common case. > > Counter-intuitively it actually seems the compiler generates faster > code in the end when keeping the > value >= 0 test around. > > Your version: > > Benchmark Mode Cnt Score Error Units > UUIDBenchmark.benchmarkUUIDFromString avgt 4 0.326 ? 0.017 us/op > UUIDBenchmark.benchmarkUUIDFromString:branches avgt 289.750 > #/op > UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 1044.087 #/op > > With: > int value = DIGITS[ch]; > return (value >= 0 && value < radix && radix >= > Character.MIN_RADIX > && radix <= Character.MAX_RADIX) ? value : -1; > > Benchmark Mode Cnt Score Error Units > UUIDBenchmark.benchmarkUUIDFromString avgt 4 0.310 ? 0.018 us/op > UUIDBenchmark.benchmarkUUIDFromString:branches avgt 256.489 > #/op > UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 988.998 #/op > > /Claes > Hm. Interesting. Maybe it helps that you placed the check (value < radix) before testing the radix? So that when value == -1 we skip the two other checks. Could you please try your benchmark with this variant: return (value < radix && radix >= Character.MIN_RADIX && radix <= Character.MAX_RADIX) ? value : -1; Thanks! -- With kind regards, Ivan Gerasimov From ivan.gerasimov at oracle.com Sat Feb 3 03:16:40 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 2 Feb 2018 19:16:40 -0800 Subject: RFR 8196740 : Character.digit(int,int) returns wrong value for out of range radix In-Reply-To: <10653089-6a16-2a6e-a695-36a012f0f41e@oracle.com> References: <10653089-6a16-2a6e-a695-36a012f0f41e@oracle.com> Message-ID: <8480c013-675d-eca3-eacc-653c7721992b@oracle.com> On 2/2/18 6:36 PM, Ivan Gerasimov wrote: > > > On 2/2/18 5:47 PM, Claes Redestad wrote: >> Hi Ivan, >> >> On 2018-02-03 02:14, Ivan Gerasimov wrote: >>> Would you please help review the fix? >>> >>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8196740 >>> WEBREV: http://cr.openjdk.java.net/~igerasim/8196740/00/webrev/ >> >> yes, an obvious error in hindsight! >> >>> >>> >>> I suspect that this version may even work a tiny bit faster, as we >>> perform less comparisons in a common case. >> >> Counter-intuitively it actually seems the compiler generates faster >> code in the end when keeping the >> value >= 0 test around. >> >> Your version: >> >> Benchmark Mode Cnt Score Error Units >> UUIDBenchmark.benchmarkUUIDFromString avgt 4 0.326 ? 0.017 us/op >> UUIDBenchmark.benchmarkUUIDFromString:branches avgt 289.750 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 1044.087 >> #/op >> >> With: >> int value = DIGITS[ch]; >> return (value >= 0 && value < radix && radix >= >> Character.MIN_RADIX >> && radix <= Character.MAX_RADIX) ? value : -1; >> >> Benchmark Mode Cnt Score Error Units >> UUIDBenchmark.benchmarkUUIDFromString avgt 4 0.310 ? 0.018 us/op >> UUIDBenchmark.benchmarkUUIDFromString:branches avgt 256.489 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 988.998 #/op >> >> /Claes >> > Hm. Interesting. > Maybe it helps that you placed the check (value < radix) before > testing the radix? > So that when value == -1 we skip the two other checks. > > Could you please try your benchmark with this variant: > return (value < radix && radix >= Character.MIN_RADIX > && radix <= Character.MAX_RADIX) ? value : -1; > Not sure why I thought that for (value == -1) the two other checks will be skipped. But still, could you please check that variant with the checks reordered? > Thanks! -- With kind regards, Ivan Gerasimov From ivan.gerasimov at oracle.com Sat Feb 3 06:51:29 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 2 Feb 2018 22:51:29 -0800 Subject: RFR 8196740 : Character.digit(int,int) returns wrong value for out of range radix In-Reply-To: References: Message-ID: After some experimentation, I couldn't find anything faster than the variant with the check (value >= 0), so let's go ahead with it. Here's the updated webrev: http://cr.openjdk.java.net/~igerasim/8196740/01/webrev/ With kind regards, Ivan On 2/2/18 5:47 PM, Claes Redestad wrote: > Hi Ivan, > > On 2018-02-03 02:14, Ivan Gerasimov wrote: >> Would you please help review the fix? >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8196740 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8196740/00/webrev/ > > yes, an obvious error in hindsight! > >> >> >> I suspect that this version may even work a tiny bit faster, as we >> perform less comparisons in a common case. > > Counter-intuitively it actually seems the compiler generates faster > code in the end when keeping the > value >= 0 test around. > > Your version: > > Benchmark Mode Cnt Score Error Units > UUIDBenchmark.benchmarkUUIDFromString avgt 4 0.326 ? 0.017 us/op > UUIDBenchmark.benchmarkUUIDFromString:branches avgt 289.750 > #/op > UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 1044.087 #/op > > With: > int value = DIGITS[ch]; > return (value >= 0 && value < radix && radix >= > Character.MIN_RADIX > && radix <= Character.MAX_RADIX) ? value : -1; > > Benchmark Mode Cnt Score Error Units > UUIDBenchmark.benchmarkUUIDFromString avgt 4 0.310 ? 0.018 us/op > UUIDBenchmark.benchmarkUUIDFromString:branches avgt 256.489 > #/op > UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 988.998 #/op > > /Claes > -- With kind regards, Ivan Gerasimov From patrick at reini.net Sat Feb 3 17:05:36 2018 From: patrick at reini.net (Patrick Reinhart) Date: Sat, 3 Feb 2018 18:05:36 +0100 Subject: RFR 8196298 Add null Reader and Writer In-Reply-To: References: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> Message-ID: > Am 02.02.2018 um 02:45 schrieb Stuart Marks : > > > > On 2/1/18 10:35 AM, Patrick Reinhart wrote: >> Here is my first shot of the actual implementation and tests: >> http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.00 > > I looked at the specification of ready() vs the implementation. The spec says that ready(), among other methods, behaves as if the Reader is at EOF. The spec for Reader.ready() says "true if the next read() is guaranteed not to block for input." Thus I'd assume that at EOF, read() will return -1 immediately, without blocking, so ready() should return true. > > The NullReader implementation inherits ready() from Reader, which always returns false. > > So, the behavior doesn't seem consistent with my reading of the specs. However, other implementations aren't, either. For example, the spec for CharArrayReader.ready() says "Character-array readers are always ready to be read." However, > > new CharArrayReader(new char[0]).ready() > > returns false! > > A couple other readers I tried (InputStreamReader, FileReader) also have ready() return true until they reach EOF, at which time they return false. But StringReader.ready() always returns true. Hmmmm. > > Maybe the safest thing to do is to leave the various Readers' behavior of ready() as they are, and leave nullReader's behavior to be consistent with them. No changes are necessary to Reader in this webrev. > > We should probably take a pass over the ready() methods in the Reader family to see if they're correct. I'd suggest they need adjustment to mean that the next read() won't block *and* data is available. And the CharArrayReader.ready() spec should be fixed. > > ** > > On the Writer side, the spec should mention that the three append() methods and five write() methods do nothing if open and throw IOE if the Writer is closed. > > The Writer.flush() method spec doesn't say what happens if flush() is called after the Writer is closed. However, Writer.close() says that subsequent write() or flush() calls throw IOE. Writer implementations that track open/closed state, such as FileWriter, will throw IOE from flush() after closure. Since it's also tracking open/closed state, I'd expect nullWriter.flush() to do the same. That is, do nothing if open, throw IOE if closed. > > Tests for flush() should be added. > I stated in the nullWriter() method that flush() will have no action. I certainly can implement that behavior, so in consequence I will change the API description of the nullWriter() method accordingly. > ** > > Speaking of tests, the tests for proper exception-throwing for the closed cases can probably usefully employ the > > @Test(expectedExceptions=IOException.class) > > annotation. The reason I often avoid this construct is that if the same exception class is thrown from an unexpected part of the test, it can mask actual failures. However, since these are one-liners, that reason doesn't apply. Thus: > > @Test(groups="closed", expectedExceptions=IOException.class) > public static void testAppendCharClosed() { > closedWriter.append('x'); > } > > You can probably also omit the try/catch block from the open tests, so that instead of > > @Test(groups="open") > public static void testAppendChar() { > try { > assertSame(openWriter, openWriter.append('x')); > } catch (IOException e) { > fail("Unexpected IOException"); > } > } > > you can write > > @Test(groups="open") > public static void testAppendChar() { > assertSame(openWriter, openWriter.append('x')); > } > > since the framework should catch and report errant exceptions, without the need for code to catch exceptions and call fail(). > > s?marks > I reworked the tests and Writer implementation accordingly http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.01 -Patrick From claes.redestad at oracle.com Sat Feb 3 19:36:19 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 3 Feb 2018 20:36:19 +0100 Subject: RFR 8196740 : Character.digit(int,int) returns wrong value for out of range radix In-Reply-To: References: Message-ID: <0212e391-80c5-036e-e20c-74c48b37d495@oracle.com> On 2018-02-03 07:51, Ivan Gerasimov wrote: > After some experimentation, I couldn't find anything faster than the > variant with the check (value >= 0), so let's go ahead with it. > > Here's the updated webrev: > > http://cr.openjdk.java.net/~igerasim/8196740/01/webrev/ Looks good to me! /Claes From fw at deneb.enyo.de Sun Feb 4 21:17:14 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 04 Feb 2018 22:17:14 +0100 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: (Roger Riggs's message of "Fri, 2 Feb 2018 12:07:04 -0500") References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> Message-ID: <87k1vs315x.fsf@mid.deneb.enyo.de> * Roger Riggs: > Updated in place. > ? http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ Doesn't this leak the file descriptor of SocketCleanable.register throws? From philipp.kunz at paratix.ch Sun Feb 4 21:38:51 2018 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Sun, 4 Feb 2018 22:38:51 +0100 Subject: Cannot add attribute into main attributes of a jar if there is no version In-Reply-To: References: <79D7D305-6B0E-4298-8950-2986B54C6F52@oracle.com> <8F0CFC56-CCBA-4A3E-8B93-7299820E4F00@oracle.com> <5c562c18-4f9a-2809-a120-2194ab4c3d21@oracle.com> Message-ID: <76598dd7-bc24-6ec2-ebc0-baaf402431db@paratix.ch> duplicate of https://bugs.openjdk.java.net/browse/JDK-6910466? On 30.01.2018 02:44, Weijun Wang wrote: > >> On Jan 30, 2018, at 8:57 AM, mandy chung wrote: >> >> >> >> On 1/29/18 4:22 PM, Weijun Wang wrote: >>> Ping again. >>> >>> >>>> On Jan 22, 2018, at 1:12 PM, Weijun Wang >>>> wrote: >>>> >>>> src/java.base/share/classes/java/util/jar/Attributes.java: >>>> >>>> 329 @SuppressWarnings("deprecation") >>>> 330 void writeMain(DataOutputStream out) throws IOException >>>> 331 { >>>> 332 // write out the *-Version header first, if it exists >>>> 333 String vername = Name.MANIFEST_VERSION.toString(); >>>> 334 String version = getValue(vername); >>>> 335 if (version == null) { >>>> 336 vername = Name.SIGNATURE_VERSION.toString(); >>>> 337 version = getValue(vername); >>>> 338 } >>>> 339 >>>> 340 if (version != null) { >>>> 341 out.writeBytes(vername+": "+version+"\r\n"); >>>> 342 } >>>> 343 >>>> 344 // write out all attributes except for the version >>>> 345 // we wrote out earlier >>>> 346 for (Entry e : entrySet()) { >>>> 347 String name = ((Name) e.getKey()).toString(); >>>> 348 if ((version != null) && !(name.equalsIgnoreCase(vername))) { >>>> >>>> So, if there is no existing MANIFEST_VERSION or SIGNATURE_VERSION, then version is null and the check above will be false for ever and any other attribute cannot be written out. >>>> >>>> Is this intended? If so, we can exit with an else block after line 342. >>>> >> From code inspection, I agree that this method is a nop if there is no Manifest-Version attribute or Signature-Attribute. This can return quickly without iterating the entry set. I don't see any issue to make it an else block. > On the other hand, if this is not intended we should fix line 348 and remove the version != null check. I cannot find a place saying a MANIFEST_VERSION or SIGNATURE_VERSION must be provided. Even if so, this should be an error and it's not a good idea to silently drop all other attributes in the main section. > > Anyway, I filed https://bugs.openjdk.java.net/browse/JDK-8196371. > > --Max > >> This method is only called from Manifest::write method which does not mention Signature-Version but I don't have the history to tell if this is intended or not. >> >> Mandy >> >> From amy.lu at oracle.com Mon Feb 5 03:23:18 2018 From: amy.lu at oracle.com (Amy Lu) Date: Mon, 5 Feb 2018 11:23:18 +0800 Subject: [JDK 11] RFR 8195981: Move some tests to OpenJDK for jdk_lang test group In-Reply-To: <0682b6a6-19b6-b7a7-a9ea-641ddd41fd2c@oracle.com> References: <6fe6e272-1196-7614-7bb9-e9b3041530b0@oracle.com> <0682b6a6-19b6-b7a7-a9ea-641ddd41fd2c@oracle.com> Message-ID: <2e90f821-9a21-efc5-61bb-f3c1e3cdad35@oracle.com> On 03/02/2018 2:36 AM, mandy chung wrote: > > On 2/2/18 7:02 AM, Amy Lu wrote: >> Please review the patch to move some tests to OpenJDK for jdk_lang >> test group. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8195981 >> webrev: http://cr.openjdk.java.net/~amlu/8195981/webrev.00/ >> >> > I suggest to merge jdk/vm/misc and jdk/vm/monitor directory and > rename to jdk/vm/runtime which avoids "misc" a dumping group > and is explicit that this directory is for runtime-related tests. > JITClassInit.java seems a JIT related test and maybe moved it > to jdk/vm/jit. > > ExceptionInInit.java > Can you reformat the comment block line 59-95 where some lines > should be merged into the line above. Thank you Mandy for reviewing. Pushed with changes as suggested. > > Bug4404588.java can be renamed to reflect what this test does. > What about renaming it to > java/lang/Character/UnicodeBlock/SpecialsUnicodeBlock.java? Forgot this one. Will do in separate bugid later. Thanks, Amy > > Otherwise, looks okay. > > Mandy From xu.y.yin at oracle.com Mon Feb 5 07:54:10 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 5 Feb 2018 15:54:10 +0800 Subject: RFR: 8195976: Add JNDI test javax/naming/dns/AttributeTests/GetAny.java In-Reply-To: References: <9CAEA585-EB5C-4F55-88D3-124E589778EB@oracle.com> <441789eb-a0ed-5eee-1d74-af5a888e9e82@oracle.com> Message-ID: A reminder that this need a reviewer's approval, many thanks Regards, Chris > On 23 Jan 2018, at 6:14 PM, Chris Yin wrote: > > Thank you Alan, I just moved it to com/sun/jndi/dns/ as you suggested and removed unused "@modules jdk.naming.dns/com.sun.jndi.dns?, updated webrev as below, thanks > > http://cr.openjdk.java.net/~xiaofeya/8195976/webrev.00/ > > Regards, > Chris > >> On 23 Jan 2018, at 3:53 PM, Alan Bateman wrote: >> >> >> >> On 23/01/2018 07:01, Chris Yin wrote: >>> Please review the added JNDI test javax/naming/dns/AttributeTests/GetAny.java, thanks >>> >>> >> You may want to move it to com/sun/jndi/dns so that it's with the other tests for the DNS provider (as there is no javax.naming.dns API). Also I suspect you don't need "@modules jdk.naming.dns/com.sun.jndi.dns" as it doesn't appear to make direct use of the classes in the implementation. >> >> -Alan > From xu.y.yin at oracle.com Mon Feb 5 08:15:45 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 5 Feb 2018 16:15:45 +0800 Subject: RFR: 8196770: Add JNDI test com/sun/jndi/ldap/blits/AddTests/AddNewEntry.java Message-ID: <7C7F5163-0680-4796-BAA7-AA8D6EEB3635@oracle.com> Please review the added JNDI test com/sun/jndi/ldap/blits/AddTests/AddNewEntry.java, thanks It?s to verify capability to add a new entry to the directory using the ADD operation. This test will use LDAPServer to playback dump ldap message to simulate test environment. bug: https://bugs.openjdk.java.net/browse/JDK-8196770 webrev: http://cr.openjdk.java.net/~xiaofeya/8196770/webrev.00/ Regards Chris From Randy.Crihfield at Oracle.com Mon Feb 5 14:59:12 2018 From: Randy.Crihfield at Oracle.com (Randy Crihfield) Date: Mon, 05 Feb 2018 09:59:12 -0500 Subject: RFR: 8194493 test for release file checking needs to be tier1 Message-ID: <5A787140.8000107@Oracle.com> I need to add the negative test created for JDK-8192837 to the TEST.groups file. The related bug is https://bugs.openjdk.java.net/browse/JDK-8194493 The diff is very short and included below. Any comments/suggestions are welcome, also I will need a sponsor for it at the end? Randy randyc at pollux -> hg diff TEST.groups diff -r 3a52333a5e57 test/jdk/TEST.groups --- a/test/jdk/TEST.groups +++ b/test/jdk/TEST.groups @@ -32,7 +32,8 @@ java/nio/Buffer \ com/sun/crypto/provider/Cipher \ :jdk_math \ - tools/pack200 + tools/pack200 \ I need to add the negative test created for JDK-8192837 to the TEST.groups file. The related bug is https://bugs.openjdk.java.net/browse/JDK-8194493 The diff is very short and included below. Any comments/suggestions are welcome, also I will need a sponsor for it at the end? Randy randyc at pollux -> hg diff TEST.groups diff -r 3a52333a5e57 test/jdk/TEST.groups --- a/test/jdk/TEST.groups +++ b/test/jdk/TEST.groups @@ -32,7 +32,8 @@ java/nio/Buffer \ com/sun/crypto/provider/Cipher \ :jdk_math \ - tools/pack200 + tools/pack200 \ + :build_sanity tier2 = \ :jdk_io \ @@ -60,6 +61,10 @@ # Other test definitions; generally smaller granularity than tiers # +# Build source checking +build_sanity = \ + sanity/releaseFile + # java.lang package and VM runtime support jdk_lang = \ java/lang \ randyc at pollux -> + :build_sanity tier2 = \ :jdk_io \ @@ -60,6 +61,10 @@ # Other test definitions; generally smaller granularity than tiers # +# Build source checking +build_sanity = \ + sanity/releaseFile + # java.lang package and VM runtime support jdk_lang = \ java/lang \ randyc at pollux -> From aleksej.efimov at oracle.com Mon Feb 5 16:51:30 2018 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Mon, 5 Feb 2018 16:51:30 +0000 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> Message-ID: Hi Roger, Jason, I've tried to avoid doing the same stuff as I did for XPathException to minimize the changes to the SAXException class. But I was naive to think so: Tried to keep the exception field in place to avoid modifications required to keep serial form intact (similar as it was done in JDK-8009581 bug mentioned by Jason), but I missed the 'initCause' method (spotted by Roger, thank you!) that can make cause/exception values inconsistent. To avoid the API modification and for the sake of clarity I proceeded with removal of the 'exception' field. The new version of the fix: - Removes 'exception' field - Maintains the serial form of the SAXException class - Handles the difference in SAXException:exception and Throwable:cause types 'SAXException::getException', i.e. SAXException:exception is Exception typed and Throwable:cause is Throwable typed. - Avoids any changes to API and serial form of the class, but maintaining it similar to JDK-8009581) - There is a copy of SAXException class in java.base module that is required for j.u.Properties::loadFromXML, it has been update with the same changes. New regression test has been added to test: - Serial compatibility with legacy JDKs (similar to JDK-8009581) - usages of SAXException::initCause/getException Webrev with the fix and new regression: http://cr.openjdk.java.net/~aefimov/6857903/dev/01/ Testing shows no JCK/regression tests failures with the proposed fix. Best Regards, Aleksei On 12/21/2017 07:00 PM, Jason Mehrens wrote: > Aleksei, > > You have to override all of the constructors to always disable initCause. Actually a similar issue was covered in: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html > > Pretty much everything was hashed out in those threads. > > Here is the final webrev for that: > > http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html > > That should be a good cookbook for changes and tests required for this issue. Note that it gets rid of the Exception field and maintains serial compatibility. > Looking back on that change, I would have changed it so the InvalidClassException added the double cause as suppressed exceptions. > > Jason > > ________________________________________ > From: core-libs-dev on behalf of Aleks Efimov > Sent: Thursday, December 21, 2017 11:27 AM > To: core-libs-dev > Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception > > Hello, > > Please, help to review the fix for jaxp bug that fixes SAXException to > correctly set exception cause with 'setCause' method: > http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ > I've tried to keep the fix miminal with respect to serial form of this > API class, i.e. kept private 'exception' field. > > The new test case was added to IssueTracker56Test unit test. Testing > showed no related JCK/JTREG failures. > > With Best Regards, > Aleksei > > From ben_walsh at uk.ibm.com Mon Feb 5 16:52:04 2018 From: ben_walsh at uk.ibm.com (Ben Walsh) Date: Mon, 5 Feb 2018 16:52:04 +0000 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: Running with the following test under the JMH benchmark framework (never used this before, so please do point out any issues with the test) ... ------------------------------------------------------------------------------------------------------------------- package org.sample; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import java.nio.ByteBuffer; public class ByteBufferBenchmark { @State(Scope.Benchmark) public static class ByteBufferContainer { ByteBuffer bb; @Setup(Level.Invocation) public void initByteBuffer() { bb = ByteBuffer.allocateDirect(1); } ByteBuffer getByteBuffer() { return bb; } } @Benchmark public void benchmark_byte_buffer_put(ByteBufferContainer bbC) { bbC.getByteBuffer().put((byte)42); } } ------------------------------------------------------------------------------------------------------------------- ... I get the following results with and without the changes ... [9walsbp at dolphin ~]$ ./build_with/jdk/bin/java -jar benchmarks.jar WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/home/9walsbp/benchmarks.jar) to field java.io.Console.cs WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release # JMH version: 1.20 # VM version: JDK 10-internal, VM 10-internal+0-adhoc.walshbp.jdk # VM invoker: /home/9walsbp/build_with/jdk/bin/java # VM options: # Warmup: 20 iterations, 1 s each # Measurement: 20 iterations, 1 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put # Run progress: 0.00% complete, ETA 00:06:40 # Fork: 1 of 10 # Warmup Iteration 1: 28572294.700 ops/s # Warmup Iteration 2: 33102015.934 ops/s # Warmup Iteration 3: 32883031.968 ops/s # Warmup Iteration 4: 32875302.586 ops/s # Warmup Iteration 5: 33708018.941 ops/s # Warmup Iteration 6: 34107603.885 ops/s # Warmup Iteration 7: 31615123.362 ops/s # Warmup Iteration 8: 34306874.876 ops/s # Warmup Iteration 9: 32615070.232 ops/s # Warmup Iteration 10: 34268641.323 ops/s # Warmup Iteration 11: 32209257.766 ops/s # Warmup Iteration 12: 34111408.999 ops/s # Warmup Iteration 13: 33527360.187 ops/s # Warmup Iteration 14: 34331726.136 ops/s # Warmup Iteration 15: 34325119.218 ops/s # Warmup Iteration 16: 31562168.904 ops/s # Warmup Iteration 17: 33227878.358 ops/s # Warmup Iteration 18: 34151154.273 ops/s # Warmup Iteration 19: 29264381.793 ops/s # Warmup Iteration 20: 30308058.560 ops/s Iteration 1: 33502237.659 ops/s Iteration 2: 33591469.068 ops/s Iteration 3: 29330487.674 ops/s Iteration 4: 31401627.173 ops/s Iteration 5: 32346391.980 ops/s Iteration 6: 34106932.868 ops/s Iteration 7: 29961218.721 ops/s Iteration 8: 31166503.731 ops/s Iteration 9: 34180826.531 ops/s Iteration 10: 34436432.202 ops/s Iteration 11: 28003272.167 ops/s Iteration 12: 30947688.218 ops/s Iteration 13: 33727404.696 ops/s Iteration 14: 33794727.241 ops/s Iteration 15: 34089932.945 ops/s Iteration 16: 30229615.627 ops/s Iteration 17: 32747950.250 ops/s Iteration 18: 34195332.494 ops/s Iteration 19: 34225966.793 ops/s Iteration 20: 30253949.366 ops/s # Run progress: 10.00% complete, ETA 00:27:45 # Fork: 2 of 10 # Warmup Iteration 1: 28189458.524 ops/s # Warmup Iteration 2: 33385943.637 ops/s # Warmup Iteration 3: 31474459.740 ops/s # Warmup Iteration 4: 32994555.969 ops/s # Warmup Iteration 5: 33580210.619 ops/s # Warmup Iteration 6: 34030937.323 ops/s # Warmup Iteration 7: 31891882.950 ops/s # Warmup Iteration 8: 34277935.152 ops/s # Warmup Iteration 9: 31636968.061 ops/s # Warmup Iteration 10: 34376216.983 ops/s # Warmup Iteration 11: 31790184.297 ops/s # Warmup Iteration 12: 34069806.835 ops/s # Warmup Iteration 13: 31737398.413 ops/s # Warmup Iteration 14: 34450863.223 ops/s # Warmup Iteration 15: 34568522.458 ops/s # Warmup Iteration 16: 30029081.882 ops/s # Warmup Iteration 17: 32092783.620 ops/s # Warmup Iteration 18: 33063366.197 ops/s # Warmup Iteration 19: 27694130.000 ops/s # Warmup Iteration 20: 31264525.792 ops/s Iteration 1: 33330874.247 ops/s Iteration 2: 34150690.011 ops/s Iteration 3: 28497705.592 ops/s Iteration 4: 31877639.021 ops/s Iteration 5: 33367324.280 ops/s Iteration 6: 34472176.807 ops/s Iteration 7: 29032738.714 ops/s Iteration 8: 31544864.097 ops/s Iteration 9: 33752028.095 ops/s Iteration 10: 34473732.353 ops/s Iteration 11: 29982974.569 ops/s Iteration 12: 30603336.272 ops/s Iteration 13: 32481117.249 ops/s Iteration 14: 33393570.191 ops/s Iteration 15: 34358979.713 ops/s Iteration 16: 27617909.839 ops/s Iteration 17: 30833802.394 ops/s Iteration 18: 32004461.408 ops/s Iteration 19: 34074910.495 ops/s Iteration 20: 34346053.320 ops/s # Run progress: 20.00% complete, ETA 00:24:26 # Fork: 3 of 10 # Warmup Iteration 1: 28503366.982 ops/s # Warmup Iteration 2: 33475519.627 ops/s # Warmup Iteration 3: 34588321.205 ops/s # Warmup Iteration 4: 35947256.417 ops/s # Warmup Iteration 5: 37638512.766 ops/s # Warmup Iteration 6: 38208797.422 ops/s # Warmup Iteration 7: 32824617.800 ops/s # Warmup Iteration 8: 35767442.077 ops/s # Warmup Iteration 9: 36548126.754 ops/s # Warmup Iteration 10: 38337143.976 ops/s # Warmup Iteration 11: 35379051.550 ops/s # Warmup Iteration 12: 38381933.847 ops/s # Warmup Iteration 13: 36543582.434 ops/s # Warmup Iteration 14: 38178008.316 ops/s # Warmup Iteration 15: 38213974.988 ops/s # Warmup Iteration 16: 32176964.179 ops/s # Warmup Iteration 17: 35487842.133 ops/s # Warmup Iteration 18: 38381350.345 ops/s # Warmup Iteration 19: 31504263.357 ops/s # Warmup Iteration 20: 37831665.793 ops/s Iteration 1: 37530637.288 ops/s Iteration 2: 38628685.180 ops/s Iteration 3: 35170239.750 ops/s Iteration 4: 32397535.287 ops/s Iteration 5: 38643366.565 ops/s Iteration 6: 35723555.410 ops/s Iteration 7: 37073023.139 ops/s Iteration 8: 38332509.859 ops/s Iteration 9: 36047299.714 ops/s Iteration 10: 38666580.432 ops/s Iteration 11: 38827527.980 ops/s Iteration 12: 38885170.177 ops/s Iteration 13: 38552899.367 ops/s Iteration 14: 38231144.659 ops/s Iteration 15: 26785494.819 ops/s Iteration 16: 35979289.874 ops/s Iteration 17: 36233311.409 ops/s Iteration 18: 37322531.784 ops/s Iteration 19: 30917505.079 ops/s Iteration 20: 36602131.217 ops/s # Run progress: 30.00% complete, ETA 00:19:56 # Fork: 4 of 10 # Warmup Iteration 1: 28201486.347 ops/s # Warmup Iteration 2: 33954913.216 ops/s # Warmup Iteration 3: 33614818.543 ops/s # Warmup Iteration 4: 36075462.255 ops/s # Warmup Iteration 5: 35868466.813 ops/s # Warmup Iteration 6: 37539151.611 ops/s # Warmup Iteration 7: 35791698.290 ops/s # Warmup Iteration 8: 37804228.841 ops/s # Warmup Iteration 9: 36209952.096 ops/s # Warmup Iteration 10: 37995654.523 ops/s # Warmup Iteration 11: 36279556.807 ops/s # Warmup Iteration 12: 37876471.909 ops/s # Warmup Iteration 13: 35472820.096 ops/s # Warmup Iteration 14: 38192862.745 ops/s # Warmup Iteration 15: 38214750.954 ops/s # Warmup Iteration 16: 33616490.113 ops/s # Warmup Iteration 17: 37147216.192 ops/s # Warmup Iteration 18: 37759898.719 ops/s # Warmup Iteration 19: 31762576.627 ops/s # Warmup Iteration 20: 35935371.693 ops/s Iteration 1: 37612237.054 ops/s Iteration 2: 31182849.622 ops/s Iteration 3: 35771979.960 ops/s Iteration 4: 37617025.341 ops/s Iteration 5: 31274273.020 ops/s Iteration 6: 36798699.672 ops/s Iteration 7: 37923256.148 ops/s Iteration 8: 38143822.112 ops/s Iteration 9: 29781444.210 ops/s Iteration 10: 34202436.254 ops/s Iteration 11: 36538073.378 ops/s Iteration 12: 37944072.150 ops/s Iteration 13: 30172512.287 ops/s Iteration 14: 34294490.289 ops/s Iteration 15: 37096818.141 ops/s Iteration 16: 37884541.579 ops/s Iteration 17: 31627891.962 ops/s Iteration 18: 34458196.904 ops/s Iteration 19: 37949147.371 ops/s Iteration 20: 38084720.812 ops/s # Run progress: 40.00% complete, ETA 00:17:39 # Fork: 5 of 10 # Warmup Iteration 1: 26430556.314 ops/s # Warmup Iteration 2: 28702874.225 ops/s # Warmup Iteration 3: 28012568.331 ops/s # Warmup Iteration 4: 30857741.012 ops/s # Warmup Iteration 5: 30671764.934 ops/s # Warmup Iteration 6: 31729981.637 ops/s # Warmup Iteration 7: 30171277.707 ops/s # Warmup Iteration 8: 29817290.663 ops/s # Warmup Iteration 9: 31275042.135 ops/s # Warmup Iteration 10: 32028867.062 ops/s # Warmup Iteration 11: 30066034.279 ops/s # Warmup Iteration 12: 30772389.223 ops/s # Warmup Iteration 13: 30066233.182 ops/s # Warmup Iteration 14: 32035909.235 ops/s # Warmup Iteration 15: 32192119.140 ops/s # Warmup Iteration 16: 30496387.577 ops/s # Warmup Iteration 17: 30957378.046 ops/s # Warmup Iteration 18: 31420314.443 ops/s # Warmup Iteration 19: 31981518.074 ops/s # Warmup Iteration 20: 26748312.384 ops/s Iteration 1: 28740606.592 ops/s Iteration 2: 30156532.719 ops/s Iteration 3: 31775304.195 ops/s Iteration 4: 26843600.302 ops/s Iteration 5: 30284709.884 ops/s Iteration 6: 32124615.948 ops/s Iteration 7: 32123075.530 ops/s Iteration 8: 26454319.079 ops/s Iteration 9: 31913402.742 ops/s Iteration 10: 32437412.737 ops/s Iteration 11: 32164716.512 ops/s Iteration 12: 25373082.559 ops/s Iteration 13: 30293078.502 ops/s Iteration 14: 31128369.793 ops/s Iteration 15: 31415561.253 ops/s Iteration 16: 32120636.468 ops/s Iteration 17: 25984394.598 ops/s Iteration 18: 30199573.863 ops/s Iteration 19: 32556187.120 ops/s Iteration 20: 32636129.190 ops/s # Run progress: 50.00% complete, ETA 00:14:30 # Fork: 6 of 10 # Warmup Iteration 1: 28150411.806 ops/s # Warmup Iteration 2: 28054601.733 ops/s # Warmup Iteration 3: 33178455.042 ops/s # Warmup Iteration 4: 33339230.141 ops/s # Warmup Iteration 5: 33332791.104 ops/s # Warmup Iteration 6: 33624536.190 ops/s # Warmup Iteration 7: 31299303.684 ops/s # Warmup Iteration 8: 33037261.267 ops/s # Warmup Iteration 9: 33356211.045 ops/s # Warmup Iteration 10: 33861930.201 ops/s # Warmup Iteration 11: 33270425.776 ops/s # Warmup Iteration 12: 33692415.570 ops/s # Warmup Iteration 13: 33297727.491 ops/s # Warmup Iteration 14: 33820015.956 ops/s # Warmup Iteration 15: 33808521.564 ops/s # Warmup Iteration 16: 31213861.694 ops/s # Warmup Iteration 17: 32149452.446 ops/s # Warmup Iteration 18: 33684868.956 ops/s # Warmup Iteration 19: 30155197.410 ops/s # Warmup Iteration 20: 33566808.050 ops/s Iteration 1: 33654501.647 ops/s Iteration 2: 29509826.671 ops/s Iteration 3: 29610518.398 ops/s Iteration 4: 32646220.404 ops/s Iteration 5: 33773773.299 ops/s Iteration 6: 27873055.802 ops/s Iteration 7: 30681259.230 ops/s Iteration 8: 32717634.647 ops/s Iteration 9: 33758469.696 ops/s Iteration 10: 27232349.464 ops/s Iteration 11: 30616039.535 ops/s Iteration 12: 31682354.340 ops/s Iteration 13: 32580910.390 ops/s Iteration 14: 33811388.262 ops/s Iteration 15: 27191817.186 ops/s Iteration 16: 29873209.932 ops/s Iteration 17: 32087557.824 ops/s Iteration 18: 32644007.761 ops/s Iteration 19: 33799135.934 ops/s Iteration 20: 26758101.989 ops/s # Run progress: 60.00% complete, ETA 00:11:43 # Fork: 7 of 10 # Warmup Iteration 1: 26592644.866 ops/s # Warmup Iteration 2: 33131524.362 ops/s # Warmup Iteration 3: 30127771.163 ops/s # Warmup Iteration 4: 31924144.092 ops/s # Warmup Iteration 5: 32001599.069 ops/s # Warmup Iteration 6: 33648812.429 ops/s # Warmup Iteration 7: 30874748.951 ops/s # Warmup Iteration 8: 33802650.338 ops/s # Warmup Iteration 9: 31054184.796 ops/s # Warmup Iteration 10: 33484369.763 ops/s # Warmup Iteration 11: 32834442.289 ops/s # Warmup Iteration 12: 34217782.915 ops/s # Warmup Iteration 13: 33036828.248 ops/s # Warmup Iteration 14: 34368855.165 ops/s # Warmup Iteration 15: 34415523.035 ops/s # Warmup Iteration 16: 31187588.079 ops/s # Warmup Iteration 17: 32594818.974 ops/s # Warmup Iteration 18: 34086802.016 ops/s # Warmup Iteration 19: 34340272.384 ops/s # Warmup Iteration 20: 28259357.798 ops/s Iteration 1: 31757196.070 ops/s Iteration 2: 33991747.455 ops/s Iteration 3: 34024018.852 ops/s Iteration 4: 33996716.542 ops/s Iteration 5: 29979159.027 ops/s Iteration 6: 34012243.975 ops/s Iteration 7: 33990813.479 ops/s Iteration 8: 34083707.765 ops/s Iteration 9: 34107629.820 ops/s Iteration 10: 27896785.113 ops/s Iteration 11: 30772172.806 ops/s Iteration 12: 33765139.270 ops/s Iteration 13: 34369680.171 ops/s Iteration 14: 34465158.746 ops/s Iteration 15: 28118201.187 ops/s Iteration 16: 30669149.170 ops/s Iteration 17: 32676396.546 ops/s Iteration 18: 34190840.882 ops/s Iteration 19: 34238377.372 ops/s Iteration 20: 27737131.008 ops/s # Run progress: 70.00% complete, ETA 00:08:38 # Fork: 8 of 10 # Warmup Iteration 1: 28630824.119 ops/s # Warmup Iteration 2: 33818762.541 ops/s # Warmup Iteration 3: 35038007.951 ops/s # Warmup Iteration 4: 36304928.826 ops/s # Warmup Iteration 5: 36354056.932 ops/s # Warmup Iteration 6: 37772679.687 ops/s # Warmup Iteration 7: 35227674.534 ops/s # Warmup Iteration 8: 38029506.671 ops/s # Warmup Iteration 9: 34585248.627 ops/s # Warmup Iteration 10: 37248219.973 ops/s # Warmup Iteration 11: 34593434.702 ops/s # Warmup Iteration 12: 35464181.287 ops/s # Warmup Iteration 13: 36696009.835 ops/s # Warmup Iteration 14: 38316197.354 ops/s # Warmup Iteration 15: 38266037.386 ops/s # Warmup Iteration 16: 37189662.171 ops/s # Warmup Iteration 17: 36523113.799 ops/s # Warmup Iteration 18: 37988022.876 ops/s # Warmup Iteration 19: 38051736.146 ops/s # Warmup Iteration 20: 31182088.185 ops/s Iteration 1: 37190441.731 ops/s Iteration 2: 37956882.826 ops/s Iteration 3: 38140733.603 ops/s Iteration 4: 37252516.227 ops/s Iteration 5: 31725393.553 ops/s Iteration 6: 36471814.199 ops/s Iteration 7: 38114054.355 ops/s Iteration 8: 32616409.003 ops/s Iteration 9: 37786024.918 ops/s Iteration 10: 37992346.638 ops/s Iteration 11: 38123237.062 ops/s Iteration 12: 38287723.965 ops/s Iteration 13: 38345880.203 ops/s Iteration 14: 31130022.209 ops/s Iteration 15: 37986966.666 ops/s Iteration 16: 36960266.429 ops/s Iteration 17: 38008103.179 ops/s Iteration 18: 38014088.784 ops/s Iteration 19: 38110627.648 ops/s Iteration 20: 38072572.537 ops/s # Run progress: 80.00% complete, ETA 00:05:39 # Fork: 9 of 10 # Warmup Iteration 1: 28298513.112 ops/s # Warmup Iteration 2: 32838632.035 ops/s # Warmup Iteration 3: 30181316.633 ops/s # Warmup Iteration 4: 32909381.568 ops/s # Warmup Iteration 5: 31944847.556 ops/s # Warmup Iteration 6: 33397277.722 ops/s # Warmup Iteration 7: 30750924.273 ops/s # Warmup Iteration 8: 33913172.836 ops/s # Warmup Iteration 9: 31744562.260 ops/s # Warmup Iteration 10: 33605243.253 ops/s # Warmup Iteration 11: 31510842.700 ops/s # Warmup Iteration 12: 33828656.623 ops/s # Warmup Iteration 13: 31525600.401 ops/s # Warmup Iteration 14: 34036366.024 ops/s # Warmup Iteration 15: 34031912.703 ops/s # Warmup Iteration 16: 29092574.272 ops/s # Warmup Iteration 17: 32539488.819 ops/s # Warmup Iteration 18: 32600231.822 ops/s # Warmup Iteration 19: 32928686.312 ops/s # Warmup Iteration 20: 33536586.546 ops/s Iteration 1: 30083475.241 ops/s Iteration 2: 32341395.220 ops/s Iteration 3: 33803703.325 ops/s Iteration 4: 33915545.687 ops/s Iteration 5: 30320419.165 ops/s Iteration 6: 33472727.881 ops/s Iteration 7: 33757353.957 ops/s Iteration 8: 28339704.610 ops/s Iteration 9: 33045545.359 ops/s Iteration 10: 33815271.934 ops/s Iteration 11: 33793968.698 ops/s Iteration 12: 33715714.353 ops/s Iteration 13: 33979632.042 ops/s Iteration 14: 29129416.976 ops/s Iteration 15: 32352103.774 ops/s Iteration 16: 32349089.070 ops/s Iteration 17: 32198286.479 ops/s Iteration 18: 29763083.673 ops/s Iteration 19: 30985401.866 ops/s Iteration 20: 30704515.609 ops/s # Run progress: 90.00% complete, ETA 00:02:45 # Fork: 10 of 10 # Warmup Iteration 1: 25845863.121 ops/s # Warmup Iteration 2: 33218974.229 ops/s # Warmup Iteration 3: 32576422.877 ops/s # Warmup Iteration 4: 33624698.333 ops/s # Warmup Iteration 5: 32881377.229 ops/s # Warmup Iteration 6: 33675931.296 ops/s # Warmup Iteration 7: 31678171.139 ops/s # Warmup Iteration 8: 31865457.116 ops/s # Warmup Iteration 9: 32035721.100 ops/s # Warmup Iteration 10: 33759626.899 ops/s # Warmup Iteration 11: 32020679.489 ops/s # Warmup Iteration 12: 33497805.239 ops/s # Warmup Iteration 13: 31818025.906 ops/s # Warmup Iteration 14: 34086770.330 ops/s # Warmup Iteration 15: 34168967.286 ops/s # Warmup Iteration 16: 28807710.513 ops/s # Warmup Iteration 17: 31152856.399 ops/s # Warmup Iteration 18: 33772983.915 ops/s # Warmup Iteration 19: 34020496.828 ops/s # Warmup Iteration 20: 28814678.720 ops/s Iteration 1: 31022028.693 ops/s Iteration 2: 33609915.911 ops/s Iteration 3: 32123372.543 ops/s Iteration 4: 27813625.332 ops/s Iteration 5: 31379592.611 ops/s Iteration 6: 32644747.339 ops/s Iteration 7: 33793411.987 ops/s Iteration 8: 30422277.460 ops/s Iteration 9: 32626659.385 ops/s Iteration 10: 33719480.304 ops/s Iteration 11: 33980011.600 ops/s Iteration 12: 28107750.520 ops/s Iteration 13: 31126083.092 ops/s Iteration 14: 32649007.164 ops/s Iteration 15: 33744248.267 ops/s Iteration 16: 34026461.866 ops/s Iteration 17: 27717690.095 ops/s Iteration 18: 31491088.409 ops/s Iteration 19: 32894429.409 ops/s Iteration 20: 33722538.163 ops/s Result "org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put": 33100911.857 ?(99.9%) 747461.951 ops/s [Average] (min, avg, max) = (25373082.559, 33100911.857, 38885170.177), stdev = 3164800.705 CI (99.9%): [32353449.906, 33848373.808] (assumes normal distribution) # Run complete. Total time: 00:27:27 Benchmark Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 33100911.857 ? 747461.951 ops/s [9walsbp at dolphin ~]$ ./build_without/jdk/bin/java -jar benchmarks.jar WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/home/9walsbp/benchmarks.jar) to field java.io.Console.cs WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release # JMH version: 1.20 # VM version: JDK 10-internal, VM 10-internal+0-adhoc.walshbp.jdk # VM invoker: /home/9walsbp/build_without/jdk/bin/java # VM options: # Warmup: 20 iterations, 1 s each # Measurement: 20 iterations, 1 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put # Run progress: 0.00% complete, ETA 00:06:40 # Fork: 1 of 10 # Warmup Iteration 1: 29113219.200 ops/s # Warmup Iteration 2: 38390854.347 ops/s # Warmup Iteration 3: 37576196.805 ops/s # Warmup Iteration 4: 38517882.972 ops/s # Warmup Iteration 5: 38838678.528 ops/s # Warmup Iteration 6: 38722019.753 ops/s # Warmup Iteration 7: 30279515.339 ops/s # Warmup Iteration 8: 37301591.985 ops/s # Warmup Iteration 9: 38937014.242 ops/s # Warmup Iteration 10: 38446665.693 ops/s # Warmup Iteration 11: 35988668.446 ops/s # Warmup Iteration 12: 37681609.419 ops/s # Warmup Iteration 13: 38993187.956 ops/s # Warmup Iteration 14: 37414379.929 ops/s # Warmup Iteration 15: 38753712.247 ops/s # Warmup Iteration 16: 38753551.189 ops/s # Warmup Iteration 17: 33767765.399 ops/s # Warmup Iteration 18: 37450537.087 ops/s # Warmup Iteration 19: 39118737.402 ops/s # Warmup Iteration 20: 39031101.838 ops/s Iteration 1: 32912027.168 ops/s Iteration 2: 37597695.079 ops/s Iteration 3: 37116802.899 ops/s Iteration 4: 37425452.741 ops/s Iteration 5: 31759009.838 ops/s Iteration 6: 37428294.033 ops/s Iteration 7: 38607838.742 ops/s Iteration 8: 34858025.193 ops/s Iteration 9: 36273653.347 ops/s Iteration 10: 38886985.073 ops/s Iteration 11: 38684265.699 ops/s Iteration 12: 30418949.886 ops/s Iteration 13: 36510488.388 ops/s Iteration 14: 37732064.835 ops/s Iteration 15: 37907332.810 ops/s Iteration 16: 38640267.003 ops/s Iteration 17: 31545863.662 ops/s Iteration 18: 36326351.514 ops/s Iteration 19: 37504107.281 ops/s Iteration 20: 36925861.540 ops/s # Run progress: 10.00% complete, ETA 00:23:50 # Fork: 2 of 10 # Warmup Iteration 1: 31027078.683 ops/s # Warmup Iteration 2: 32638372.801 ops/s # Warmup Iteration 3: 32343711.957 ops/s # Warmup Iteration 4: 33981414.940 ops/s # Warmup Iteration 5: 34883411.692 ops/s # Warmup Iteration 6: 35725521.918 ops/s # Warmup Iteration 7: 33357134.144 ops/s # Warmup Iteration 8: 33753068.420 ops/s # Warmup Iteration 9: 35403231.692 ops/s # Warmup Iteration 10: 35235976.314 ops/s # Warmup Iteration 11: 35321898.883 ops/s # Warmup Iteration 12: 34820537.854 ops/s # Warmup Iteration 13: 35920876.086 ops/s # Warmup Iteration 14: 35251249.967 ops/s # Warmup Iteration 15: 34446183.776 ops/s # Warmup Iteration 16: 33048031.622 ops/s # Warmup Iteration 17: 31531379.814 ops/s # Warmup Iteration 18: 33034792.981 ops/s # Warmup Iteration 19: 34738150.009 ops/s # Warmup Iteration 20: 30077393.793 ops/s Iteration 1: 31766138.031 ops/s Iteration 2: 32851639.920 ops/s Iteration 3: 33660705.695 ops/s Iteration 4: 30490177.412 ops/s Iteration 5: 33922213.740 ops/s Iteration 6: 35233366.402 ops/s Iteration 7: 35626761.045 ops/s Iteration 8: 30340609.672 ops/s Iteration 9: 34019673.519 ops/s Iteration 10: 36014726.921 ops/s Iteration 11: 35974210.612 ops/s Iteration 12: 30201358.306 ops/s Iteration 13: 31810853.163 ops/s Iteration 14: 33132733.085 ops/s Iteration 15: 34555927.009 ops/s Iteration 16: 35932316.359 ops/s Iteration 17: 29234047.193 ops/s Iteration 18: 33989744.306 ops/s Iteration 19: 33949712.258 ops/s Iteration 20: 35135848.150 ops/s # Run progress: 20.00% complete, ETA 00:21:55 # Fork: 3 of 10 # Warmup Iteration 1: 30980353.496 ops/s # Warmup Iteration 2: 32972554.880 ops/s # Warmup Iteration 3: 33057281.401 ops/s # Warmup Iteration 4: 34633426.034 ops/s # Warmup Iteration 5: 34517376.033 ops/s # Warmup Iteration 6: 35702523.091 ops/s # Warmup Iteration 7: 34365724.187 ops/s # Warmup Iteration 8: 36032644.388 ops/s # Warmup Iteration 9: 35136119.559 ops/s # Warmup Iteration 10: 36140189.976 ops/s # Warmup Iteration 11: 33882357.986 ops/s # Warmup Iteration 12: 35778782.987 ops/s # Warmup Iteration 13: 33902794.405 ops/s # Warmup Iteration 14: 36155738.838 ops/s # Warmup Iteration 15: 36206714.982 ops/s # Warmup Iteration 16: 32084735.481 ops/s # Warmup Iteration 17: 34564731.167 ops/s # Warmup Iteration 18: 35168148.516 ops/s # Warmup Iteration 19: 30421286.177 ops/s # Warmup Iteration 20: 31997040.084 ops/s Iteration 1: 35802127.743 ops/s Iteration 2: 30223432.040 ops/s Iteration 3: 33523343.257 ops/s Iteration 4: 35660309.083 ops/s Iteration 5: 35997090.209 ops/s Iteration 6: 31239191.252 ops/s Iteration 7: 36147747.729 ops/s Iteration 8: 36147969.988 ops/s Iteration 9: 36208488.301 ops/s Iteration 10: 29792833.489 ops/s Iteration 11: 34580451.949 ops/s Iteration 12: 36064305.066 ops/s Iteration 13: 36041893.088 ops/s Iteration 14: 35855446.086 ops/s Iteration 15: 31661873.490 ops/s Iteration 16: 35036383.811 ops/s Iteration 17: 35857209.919 ops/s Iteration 18: 36029402.370 ops/s Iteration 19: 35930595.692 ops/s Iteration 20: 31699203.984 ops/s # Run progress: 30.00% complete, ETA 00:19:36 # Fork: 4 of 10 # Warmup Iteration 1: 31296595.515 ops/s # Warmup Iteration 2: 37908633.020 ops/s # Warmup Iteration 3: 35250565.078 ops/s # Warmup Iteration 4: 37752285.871 ops/s # Warmup Iteration 5: 37914294.026 ops/s # Warmup Iteration 6: 38377635.946 ops/s # Warmup Iteration 7: 32733864.039 ops/s # Warmup Iteration 8: 38064547.309 ops/s # Warmup Iteration 9: 36544704.445 ops/s # Warmup Iteration 10: 33882227.479 ops/s # Warmup Iteration 11: 36836752.434 ops/s # Warmup Iteration 12: 38446392.157 ops/s # Warmup Iteration 13: 36364073.955 ops/s # Warmup Iteration 14: 38706702.104 ops/s # Warmup Iteration 15: 38652050.082 ops/s # Warmup Iteration 16: 34322809.290 ops/s # Warmup Iteration 17: 37700500.798 ops/s # Warmup Iteration 18: 38906944.697 ops/s # Warmup Iteration 19: 33195845.787 ops/s # Warmup Iteration 20: 35821942.775 ops/s Iteration 1: 39417175.657 ops/s Iteration 2: 39325998.402 ops/s Iteration 3: 39056076.106 ops/s Iteration 4: 32921279.546 ops/s Iteration 5: 39054221.339 ops/s Iteration 6: 36515839.874 ops/s Iteration 7: 32172960.935 ops/s Iteration 8: 37976219.446 ops/s Iteration 9: 38929526.685 ops/s Iteration 10: 38894878.237 ops/s Iteration 11: 31978400.534 ops/s Iteration 12: 36903626.167 ops/s Iteration 13: 39131870.806 ops/s Iteration 14: 39345626.043 ops/s Iteration 15: 39026205.926 ops/s Iteration 16: 36806664.950 ops/s Iteration 17: 38630575.119 ops/s Iteration 18: 37854064.857 ops/s Iteration 19: 39440319.418 ops/s Iteration 20: 39524804.534 ops/s # Run progress: 40.00% complete, ETA 00:16:54 # Fork: 5 of 10 # Warmup Iteration 1: 31110038.023 ops/s # Warmup Iteration 2: 34412040.058 ops/s # Warmup Iteration 3: 31235963.893 ops/s # Warmup Iteration 4: 30767353.395 ops/s # Warmup Iteration 5: 34439323.107 ops/s # Warmup Iteration 6: 36266948.529 ops/s # Warmup Iteration 7: 36055758.564 ops/s # Warmup Iteration 8: 34585869.673 ops/s # Warmup Iteration 9: 36567490.492 ops/s # Warmup Iteration 10: 35504554.003 ops/s # Warmup Iteration 11: 36179962.205 ops/s # Warmup Iteration 12: 32394762.264 ops/s # Warmup Iteration 13: 35653335.954 ops/s # Warmup Iteration 14: 35620586.786 ops/s # Warmup Iteration 15: 35863809.190 ops/s # Warmup Iteration 16: 36259837.483 ops/s # Warmup Iteration 17: 36302387.266 ops/s # Warmup Iteration 18: 33192738.788 ops/s # Warmup Iteration 19: 36523702.645 ops/s # Warmup Iteration 20: 36629186.036 ops/s Iteration 1: 36694296.125 ops/s Iteration 2: 36680567.770 ops/s Iteration 3: 36495812.469 ops/s Iteration 4: 32343259.943 ops/s Iteration 5: 33671386.030 ops/s Iteration 6: 35555005.646 ops/s Iteration 7: 35211358.656 ops/s Iteration 8: 36081526.871 ops/s Iteration 9: 36107058.585 ops/s Iteration 10: 32729883.053 ops/s Iteration 11: 35724907.378 ops/s Iteration 12: 36499143.866 ops/s Iteration 13: 36603461.237 ops/s Iteration 14: 36631899.844 ops/s Iteration 15: 36121361.065 ops/s Iteration 16: 33671874.478 ops/s Iteration 17: 36083864.464 ops/s Iteration 18: 36667481.469 ops/s Iteration 19: 36722951.627 ops/s Iteration 20: 36566197.534 ops/s # Run progress: 50.00% complete, ETA 00:13:22 # Fork: 6 of 10 # Warmup Iteration 1: 31029872.095 ops/s # Warmup Iteration 2: 37230506.713 ops/s # Warmup Iteration 3: 38809151.616 ops/s # Warmup Iteration 4: 38329927.142 ops/s # Warmup Iteration 5: 38143210.031 ops/s # Warmup Iteration 6: 38809118.052 ops/s # Warmup Iteration 7: 33967885.816 ops/s # Warmup Iteration 8: 38749239.488 ops/s # Warmup Iteration 9: 38579178.021 ops/s # Warmup Iteration 10: 36703103.973 ops/s # Warmup Iteration 11: 38954258.462 ops/s # Warmup Iteration 12: 39112773.031 ops/s # Warmup Iteration 13: 38741746.259 ops/s # Warmup Iteration 14: 39076601.751 ops/s # Warmup Iteration 15: 38986191.092 ops/s # Warmup Iteration 16: 37138531.823 ops/s # Warmup Iteration 17: 37857790.291 ops/s # Warmup Iteration 18: 38843053.603 ops/s # Warmup Iteration 19: 38787113.306 ops/s # Warmup Iteration 20: 35306823.192 ops/s Iteration 1: 37055062.962 ops/s Iteration 2: 38855425.772 ops/s Iteration 3: 35013411.175 ops/s Iteration 4: 35274085.229 ops/s Iteration 5: 38493671.894 ops/s Iteration 6: 36760520.168 ops/s Iteration 7: 37653602.024 ops/s Iteration 8: 37387676.588 ops/s Iteration 9: 38555022.030 ops/s Iteration 10: 28687868.812 ops/s Iteration 11: 27046401.908 ops/s Iteration 12: 29259494.625 ops/s Iteration 13: 26667525.465 ops/s Iteration 14: 25558172.378 ops/s Iteration 15: 35911579.186 ops/s Iteration 16: 36880307.369 ops/s Iteration 17: 38333722.473 ops/s Iteration 18: 34790538.914 ops/s Iteration 19: 36686520.151 ops/s Iteration 20: 37382730.439 ops/s # Run progress: 60.00% complete, ETA 00:10:48 # Fork: 7 of 10 # Warmup Iteration 1: 31057306.449 ops/s # Warmup Iteration 2: 35840428.213 ops/s # Warmup Iteration 3: 34143557.080 ops/s # Warmup Iteration 4: 36667927.955 ops/s # Warmup Iteration 5: 36452958.037 ops/s # Warmup Iteration 6: 38232808.327 ops/s # Warmup Iteration 7: 35551071.320 ops/s # Warmup Iteration 8: 38541440.865 ops/s # Warmup Iteration 9: 36172684.586 ops/s # Warmup Iteration 10: 38633066.168 ops/s # Warmup Iteration 11: 36349123.423 ops/s # Warmup Iteration 12: 38523683.054 ops/s # Warmup Iteration 13: 36124569.851 ops/s # Warmup Iteration 14: 38950543.533 ops/s # Warmup Iteration 15: 38869447.272 ops/s # Warmup Iteration 16: 31093331.577 ops/s # Warmup Iteration 17: 37392337.846 ops/s # Warmup Iteration 18: 39057287.494 ops/s # Warmup Iteration 19: 38995112.909 ops/s # Warmup Iteration 20: 39066421.796 ops/s Iteration 1: 32127004.028 ops/s Iteration 2: 37499140.095 ops/s Iteration 3: 36539432.705 ops/s Iteration 4: 33907894.925 ops/s Iteration 5: 38054763.625 ops/s Iteration 6: 38733118.320 ops/s Iteration 7: 38622694.446 ops/s Iteration 8: 32751376.012 ops/s Iteration 9: 35019544.916 ops/s Iteration 10: 38858825.000 ops/s Iteration 11: 38910765.208 ops/s Iteration 12: 31043691.167 ops/s Iteration 13: 35840369.231 ops/s Iteration 14: 37385724.264 ops/s Iteration 15: 39023580.476 ops/s Iteration 16: 38741774.638 ops/s Iteration 17: 31542116.877 ops/s Iteration 18: 34719270.004 ops/s Iteration 19: 38908161.679 ops/s Iteration 20: 38963054.209 ops/s # Run progress: 70.00% complete, ETA 00:08:05 # Fork: 8 of 10 # Warmup Iteration 1: 30003148.195 ops/s # Warmup Iteration 2: 35462032.545 ops/s # Warmup Iteration 3: 33071571.604 ops/s # Warmup Iteration 4: 35387917.187 ops/s # Warmup Iteration 5: 34586963.687 ops/s # Warmup Iteration 6: 35448683.915 ops/s # Warmup Iteration 7: 32807613.797 ops/s # Warmup Iteration 8: 35915731.511 ops/s # Warmup Iteration 9: 33149556.434 ops/s # Warmup Iteration 10: 35195706.499 ops/s # Warmup Iteration 11: 33596358.520 ops/s # Warmup Iteration 12: 35353239.403 ops/s # Warmup Iteration 13: 33508839.669 ops/s # Warmup Iteration 14: 35930589.401 ops/s # Warmup Iteration 15: 35941526.705 ops/s # Warmup Iteration 16: 30714684.903 ops/s # Warmup Iteration 17: 32933652.946 ops/s # Warmup Iteration 18: 34942977.718 ops/s # Warmup Iteration 19: 35976691.542 ops/s # Warmup Iteration 20: 30498957.215 ops/s Iteration 1: 35631149.027 ops/s Iteration 2: 36122657.348 ops/s Iteration 3: 29257602.641 ops/s Iteration 4: 31964437.318 ops/s Iteration 5: 35123171.218 ops/s Iteration 6: 35988762.826 ops/s Iteration 7: 33333800.776 ops/s Iteration 8: 33839869.951 ops/s Iteration 9: 35877741.832 ops/s Iteration 10: 35977710.893 ops/s Iteration 11: 35928268.472 ops/s Iteration 12: 31871331.622 ops/s Iteration 13: 33520435.719 ops/s Iteration 14: 35271300.135 ops/s Iteration 15: 36042505.102 ops/s Iteration 16: 36026785.491 ops/s Iteration 17: 29606290.243 ops/s Iteration 18: 34555390.595 ops/s Iteration 19: 35040209.776 ops/s Iteration 20: 35526038.193 ops/s # Run progress: 80.00% complete, ETA 00:05:25 # Fork: 9 of 10 # Warmup Iteration 1: 30780286.932 ops/s # Warmup Iteration 2: 38554787.285 ops/s # Warmup Iteration 3: 35499729.607 ops/s # Warmup Iteration 4: 38123652.259 ops/s # Warmup Iteration 5: 38287045.843 ops/s # Warmup Iteration 6: 38909349.390 ops/s # Warmup Iteration 7: 34160614.025 ops/s # Warmup Iteration 8: 36714969.322 ops/s # Warmup Iteration 9: 35724081.900 ops/s # Warmup Iteration 10: 34344266.878 ops/s # Warmup Iteration 11: 36691467.403 ops/s # Warmup Iteration 12: 38395178.914 ops/s # Warmup Iteration 13: 37303043.548 ops/s # Warmup Iteration 14: 39117495.343 ops/s # Warmup Iteration 15: 38410409.515 ops/s # Warmup Iteration 16: 36557299.657 ops/s # Warmup Iteration 17: 39137443.252 ops/s # Warmup Iteration 18: 38967960.625 ops/s # Warmup Iteration 19: 34626900.596 ops/s # Warmup Iteration 20: 37683007.712 ops/s Iteration 1: 39077842.127 ops/s Iteration 2: 34347095.059 ops/s Iteration 3: 37355881.353 ops/s Iteration 4: 39049529.112 ops/s Iteration 5: 34227298.880 ops/s Iteration 6: 34912978.620 ops/s Iteration 7: 36002838.823 ops/s Iteration 8: 39311390.901 ops/s Iteration 9: 33107477.135 ops/s Iteration 10: 37288472.263 ops/s Iteration 11: 36867078.249 ops/s Iteration 12: 38982164.040 ops/s Iteration 13: 33384786.397 ops/s Iteration 14: 35745764.678 ops/s Iteration 15: 36567956.300 ops/s Iteration 16: 38434831.820 ops/s Iteration 17: 38847394.679 ops/s Iteration 18: 34190903.091 ops/s Iteration 19: 36003870.959 ops/s Iteration 20: 36590393.071 ops/s # Run progress: 90.00% complete, ETA 00:02:44 # Fork: 10 of 10 # Warmup Iteration 1: 28889703.379 ops/s # Warmup Iteration 2: 38603492.577 ops/s # Warmup Iteration 3: 34043794.352 ops/s # Warmup Iteration 4: 37985769.911 ops/s # Warmup Iteration 5: 37337263.671 ops/s # Warmup Iteration 6: 38294358.387 ops/s # Warmup Iteration 7: 33591758.046 ops/s # Warmup Iteration 8: 35677991.195 ops/s # Warmup Iteration 9: 35626708.813 ops/s # Warmup Iteration 10: 35001658.546 ops/s # Warmup Iteration 11: 34895453.027 ops/s # Warmup Iteration 12: 38305387.208 ops/s # Warmup Iteration 13: 31137358.688 ops/s # Warmup Iteration 14: 38642485.864 ops/s # Warmup Iteration 15: 38586100.872 ops/s # Warmup Iteration 16: 35486297.661 ops/s # Warmup Iteration 17: 38936456.857 ops/s # Warmup Iteration 18: 38776288.591 ops/s # Warmup Iteration 19: 38495102.028 ops/s # Warmup Iteration 20: 34583854.305 ops/s Iteration 1: 35593487.267 ops/s Iteration 2: 36719438.475 ops/s Iteration 3: 33895023.416 ops/s Iteration 4: 34812011.249 ops/s Iteration 5: 38906937.001 ops/s Iteration 6: 38182707.915 ops/s Iteration 7: 35294815.634 ops/s Iteration 8: 37426940.293 ops/s Iteration 9: 37385447.731 ops/s Iteration 10: 38628608.563 ops/s Iteration 11: 34323084.222 ops/s Iteration 12: 35285335.362 ops/s Iteration 13: 36995943.696 ops/s Iteration 14: 38944858.911 ops/s Iteration 15: 38516334.356 ops/s Iteration 16: 34389934.288 ops/s Iteration 17: 35448198.837 ops/s Iteration 18: 38375706.441 ops/s Iteration 19: 38841034.693 ops/s Iteration 20: 38600776.337 ops/s Result "org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put": 35604933.518 ?(99.9%) 654975.515 ops/s [Average] (min, avg, max) = (25558172.378, 35604933.518, 39524804.534), stdev = 2773207.341 CI (99.9%): [34949958.003, 36259909.033] (assumes normal distribution) # Run complete. Total time: 00:27:51 Benchmark Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 35604933.518 ? 654975.515 ops/s ... So a performance degradation of roughly 7%. Regards, Ben Walsh From: Paul Sandoz To: Ben Walsh Cc: core-libs-dev Date: 01/02/2018 22:50 Subject: Re: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup Hi Ben, I don?t think you require the fence in all the cases you have currently placed it e.g. here for example $memtype$ y = $toBits$(x); UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); + Reference.reachabilityFence(this); return this; since ?this? is being returned it will be kept live during the unsafe access. Would you mind providing a JMH benchmark and results for the performance with and without the fence for say a put and/or a get. I would like to understand the performance impact on HotSpot, this is one reason why we have yet to add such fences as it will likely impact performance. At the moment we are relying on the method not being inlined, which is an expedient technique to make it functional and keep a reference alive but not necessarily optimal for usages in DBB. For more details see: https://bugs.openjdk.java.net/browse/JDK-8169605 https://bugs.openjdk.java.net/browse/JDK-8149610 Thanks, Paul. On Feb 1, 2018, at 6:55 AM, Ben Walsh wrote: This contribution forms a partial solution to the problem detailed here - http://thevirtualmachinist.blogspot.ca/2011/07/subtle-issue-of-reachability.html . In this context, this contribution provides "markers" such that a suitably "aware" compiler can reduce the chances of such a problem occurring with each of the DirectBuffer objects and MappedByteBuffer object. The reachabilityFences may prevent crashes / exceptions due to cleaning up the backing memory before the user has finished using the pointer. Any compiler not suitably "aware" could be modified to make use of the "markers". I would like to pair with a sponsor to contribute this patch ... ------------------------------------------------------------------------------------------------------------------- diff -r d51e64840b4f src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -33,15 +33,21 @@ private $type$ get$Type$(long a) { $memtype$ x = UNSAFE.get$Memtype$Unaligned(null, a, bigEndian); - return $fromBits$(x); + $type$ y = $fromBits$(x); + Reference.reachabilityFence(this); + return y; } public $type$ get$Type$() { - return get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); + $type$ y = get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); + Reference.reachabilityFence(this); + return y; } public $type$ get$Type$(int i) { - return get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); + $type$ y = get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); + Reference.reachabilityFence(this); + return y; } #end[rw] @@ -50,6 +56,7 @@ #if[rw] $memtype$ y = $toBits$(x); UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); diff -r d51e64840b4f src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -28,6 +28,7 @@ package java.nio; import java.io.FileDescriptor; +import java.lang.ref.Reference; import jdk.internal.misc.VM; import jdk.internal.ref.Cleaner; import sun.nio.ch.DirectBuffer; @@ -312,6 +313,7 @@ public $Type$Buffer put($type$ x) { #if[rw] UNSAFE.put$Swaptype$(ix(nextPutIndex()), $swap$($toBits$(x))); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); @@ -321,6 +323,7 @@ public $Type$Buffer put(int i, $type$ x) { #if[rw] UNSAFE.put$Swaptype$(ix(checkIndex(i)), $swap$($toBits$(x))); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); @@ -347,6 +350,7 @@ if (srem > rem) throw new BufferOverflowException(); UNSAFE.copyMemory(sb.ix(spos), ix(pos), (long)srem << $LG_BYTES_PER_VALUE$); + Reference.reachabilityFence(this); sb.position(spos + srem); position(pos + srem); } else if (src.hb != null) { @@ -413,6 +417,7 @@ int rem = (pos <= lim ? lim - pos : 0); UNSAFE.copyMemory(ix(pos), ix(0), (long)rem << $LG_BYTES_PER_VALUE$); + Reference.reachabilityFence(this); position(rem); limit(capacity()); discardMark(); @@ -505,6 +510,7 @@ void _put(int i, byte b) { // package-private #if[rw] UNSAFE.putByte(address + i, b); + Reference.reachabilityFence(this); #else[rw] throw new ReadOnlyBufferException(); #end[rw] diff -r d51e64840b4f src/java.base/share/classes/java/nio/MappedByteBuffer.java --- a/src/java.base/share/classes/java/nio/MappedByteBuffer.java Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/MappedByteBuffer.java Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -26,6 +26,7 @@ package java.nio; import java.io.FileDescriptor; +import java.lang.ref.Reference; import jdk.internal.misc.Unsafe; @@ -164,6 +165,7 @@ // is computed as we go along to prevent the compiler from otherwise // considering the loop as dead code. Unsafe unsafe = Unsafe.getUnsafe(); + Reference.reachabilityFence(this); int ps = Bits.pageSize(); int count = Bits.pageCount(length); long a = mappingAddress(offset); ------------------------------------------------------------------------------------------------------------------- Regards, Ben Walsh Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From jason_mehrens at hotmail.com Mon Feb 5 17:09:20 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Mon, 5 Feb 2018 17:09:20 +0000 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> , Message-ID: Aleksei, Looks good. We should avoid this.initCause in readObject and prefer super.initCause so subclasses can't alter the deserialization behavior. While I can't think of a case off the top of my head, I would prefer that we trap and convert IllegalStateException into a InvalidClassException in readObject. That keeps the readObject contract intact in case something malicious is going on. That is just my preference though. Jason ________________________________________ From: Aleks Efimov Sent: Monday, February 5, 2018 10:51 AM To: Jason Mehrens; 'Roger Riggs' Cc: core-libs-dev Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception Hi Roger, Jason, I've tried to avoid doing the same stuff as I did for XPathException to minimize the changes to the SAXException class. But I was naive to think so: Tried to keep the exception field in place to avoid modifications required to keep serial form intact (similar as it was done in JDK-8009581 bug mentioned by Jason), but I missed the 'initCause' method (spotted by Roger, thank you!) that can make cause/exception values inconsistent. To avoid the API modification and for the sake of clarity I proceeded with removal of the 'exception' field. The new version of the fix: - Removes 'exception' field - Maintains the serial form of the SAXException class - Handles the difference in SAXException:exception and Throwable:cause types 'SAXException::getException', i.e. SAXException:exception is Exception typed and Throwable:cause is Throwable typed. - Avoids any changes to API and serial form of the class, but maintaining it similar to JDK-8009581) - There is a copy of SAXException class in java.base module that is required for j.u.Properties::loadFromXML, it has been update with the same changes. New regression test has been added to test: - Serial compatibility with legacy JDKs (similar to JDK-8009581) - usages of SAXException::initCause/getException Webrev with the fix and new regression: http://cr.openjdk.java.net/~aefimov/6857903/dev/01/ Testing shows no JCK/regression tests failures with the proposed fix. Best Regards, Aleksei On 12/21/2017 07:00 PM, Jason Mehrens wrote: > Aleksei, > > You have to override all of the constructors to always disable initCause. Actually a similar issue was covered in: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html > > Pretty much everything was hashed out in those threads. > > Here is the final webrev for that: > > http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html > > That should be a good cookbook for changes and tests required for this issue. Note that it gets rid of the Exception field and maintains serial compatibility. > Looking back on that change, I would have changed it so the InvalidClassException added the double cause as suppressed exceptions. > > Jason > > ________________________________________ > From: core-libs-dev on behalf of Aleks Efimov > Sent: Thursday, December 21, 2017 11:27 AM > To: core-libs-dev > Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception > > Hello, > > Please, help to review the fix for jaxp bug that fixes SAXException to > correctly set exception cause with 'setCause' method: > http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ > I've tried to keep the fix miminal with respect to serial form of this > API class, i.e. kept private 'exception' field. > > The new test case was added to IssueTracker56Test unit test. Testing > showed no related JCK/JTREG failures. > > With Best Regards, > Aleksei > > From Roger.Riggs at Oracle.com Mon Feb 5 18:41:25 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 5 Feb 2018 13:41:25 -0500 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: <87k1vs315x.fsf@mid.deneb.enyo.de> References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> <7812E4FB-51DE-41C1-A517-D5D81F939F8A@oracle.com> <4ddec36e-4ec7-26a2-ca38-3448983973b8@oracle.com> <87k1vs315x.fsf@mid.deneb.enyo.de> Message-ID: Hi Florian, Thanks for the reminder, I'll add it to 8196716 Refactor FileDescriptor FDCleanup Thanks, Roger On 2/4/2018 4:17 PM, Florian Weimer wrote: > * Roger Riggs: > >> Updated in place. >> ? http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ > Doesn't this leak the file descriptor of SocketCleanable.register > throws? From paul.sandoz at oracle.com Mon Feb 5 19:33:29 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 5 Feb 2018 11:33:29 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: <530A3E61-838C-47AD-A029-6AA1F55375DF@oracle.com> Hi Ben, Thanks for looking into this. > On Feb 5, 2018, at 8:52 AM, Ben Walsh wrote: > > Running with the following test under the JMH benchmark framework (never > used this before, so please do point out any issues with the test) ? > Your benchmark looks good, i would return the byte buffer thereby avoiding any risk of dead code elimination (generally a best practice even if not absolutely required). As a follow on you might wanna try measuring a loop e.g.: ByteBuffer bb = ... for (int i = 0; i < L; i++) { bb.put((byte)i) } return bb; where L is parameterized (see JMH?s @Param annotation), then you can easily vary from 1 upwards. Performance might be affected in loops if unrolling and/or vectorization is perturbed by the fence (i doubt the compiler will hoist the fence out of the loop given the JIT is forced to *not* inline it). To test for vectorization you could write a test method to get int values from the buffer and sum ?em up returning the sum. Alas a 7% hit on simple access is not good :-( we really need the JIT to track the argument passed to the method. Thanks, Paul. > > ------------------------------------------------------------------------------------------------------------------- > Result "org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put": > 33100911.857 ?(99.9%) 747461.951 ops/s [Average] > (min, avg, max) = (25373082.559, 33100911.857, 38885170.177), stdev = > 3164800.705 > CI (99.9%): [32353449.906, 33848373.808] (assumes normal distribution) > > > # Run complete. Total time: 00:27:27 > > Benchmark Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 33100911.857 ? > 747461.951 ops/s > > > > Result "org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put": > 35604933.518 ?(99.9%) 654975.515 ops/s [Average] > (min, avg, max) = (25558172.378, 35604933.518, 39524804.534), stdev = > 2773207.341 > CI (99.9%): [34949958.003, 36259909.033] (assumes normal distribution) > > > # Run complete. Total time: 00:27:51 > > Benchmark Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 35604933.518 ? > 654975.515 ops/s > > > ... So a performance degradation of roughly 7%. > > > Regards, > Ben Walsh > From mandy.chung at oracle.com Mon Feb 5 23:50:40 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 5 Feb 2018 15:50:40 -0800 Subject: Cannot add attribute into main attributes of a jar if there is no version In-Reply-To: <76598dd7-bc24-6ec2-ebc0-baaf402431db@paratix.ch> References: <79D7D305-6B0E-4298-8950-2986B54C6F52@oracle.com> <8F0CFC56-CCBA-4A3E-8B93-7299820E4F00@oracle.com> <5c562c18-4f9a-2809-a120-2194ab4c3d21@oracle.com> <76598dd7-bc24-6ec2-ebc0-baaf402431db@paratix.ch> Message-ID: On 2/4/18 1:38 PM, Philipp Kunz wrote: > duplicate of https://bugs.openjdk.java.net/browse/JDK-6910466? > Yes it is.? I closed JDK-8196371 as a dup. thanks Mandy From xueming.shen at oracle.com Tue Feb 6 02:00:45 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 05 Feb 2018 18:00:45 -0800 Subject: RFR JDK-8164278: java.util.Base64.EncOutputStream/DecInputStream is slower than corresponding version in javax.mail package Message-ID: <5A790C4D.1090309@oracle.com> Hi, Please help review the change for JDK-8164278. issue: https://bugs.openjdk.java.net/browse/JDK-8164278 webrev: http://cr.openjdk.java.net/~sherman/8164278/webrev jmh.src: http://cr.openjdk.java.net/~sherman/8164278/Base64BM.java jmh.result: http://cr.openjdk.java.net/~sherman/8164278/base64.bm Base64.Decoder.decode0: Adopted the "similar" optimization approach we took in Base64.Encoder.encode0() to add a "fast path" to decode a block of 4-byte units together (current implementation decodes one single byte per while/loop. The jmh benchmark result indicates a big speed boost (those decodeArray/Mime/Url results, from 30% to 2 times faster, depends on input size). Base64.Encoder.encode0() It appears encode0() was fully optimized in 1.8. Can't get it faster :-) Tried to use Unsafe.getLong/putLong instead of byte by byte access. But it appears the 8-byte "vectorization" does not bring us enough speed up, the performance is the same as the current one. See encode00() at http://cr.openjdk.java.net/~sherman/8164278/webrev.00 Base64.Encoder.wrap(OutputStream)/EncOutputStream.write(): If my memory serves me right, the current implementation was under the assumption that the underlying output stream probably is buffered (if invoker cares). It would be a redundant if EncOutputStream buffers bytes again. It appears this is a wrong assumption. It is much slower to write 4 bytes separately, compared to bundle them together in a byte[4] and write into underlying, even the underlying output stream is a ByteArrayOutputStream. Again, the proposed change is to add a fast path loop, as we do in encode0(), to decode/ write a block of 3-byte->4-byte unites. It appears this fast loop can help the compiler to optimize away some boundary checks, therefor is much faster. The jmh result Base64BM.encodeOS suggests the new implementation is almost 4 times faster and is almost the same as java.mail's stream encoder. Base64.Decoder.wrap(InputStream)/DecInputStream.read(): Same as the approach we take for decode0(), to add a fast path decode block of 4-byte unites together. The jmh result Base64BM.decodeOS (the name probably should be decodeIS, for InputStream, but anyway...) shows the proposed one is 4 times faster than the existing impl and double the java.mail (Base64BM.decodeOS_javamail) implementation. However, there is a side-effect of adding a buffering mechanism into DecInputStream. The current implementation read bytes from the underlying stream one by one, it never reads more bytes than it needs, which means it should/is supposed to just stop at the last byte that it needs to decode, when there is "=" present in the stream. With buffering, it's possible more bytes (after "=", which indicates "end of base64 stream") might be read/consumed in and buffered. A concern? if this is indeed a concern, the only alternative might be to add a separate method to support this "faster-buffered-decoder"? Thanks, Sherman From aleksej.efimov at oracle.com Tue Feb 6 02:46:56 2018 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Tue, 6 Feb 2018 02:46:56 +0000 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> Message-ID: <3d4a77bf-5797-e1b9-6a85-977a00fda3e3@oracle.com> Thank you for your comments, Jason! New webrev can be found at this location: http://cr.openjdk.java.net/~aefimov/6857903/dev/02/ The list of changes: - this.initCause() -> super.initCause() - 'readObject' has been modified to convert IllegalStateException to InvalidClassException. The test case that triggers IllegalStateException in SAXException::readObject has been added to 'testReadObjectIllegalStateException' test method. Best Regards, Aleksei On 02/05/2018 05:09 PM, Jason Mehrens wrote: > Aleksei, > > Looks good. We should avoid this.initCause in readObject and prefer super.initCause so subclasses can't alter the deserialization behavior. > While I can't think of a case off the top of my head, I would prefer that we trap and convert IllegalStateException into a InvalidClassException in readObject. > That keeps the readObject contract intact in case something malicious is going on. That is just my preference though. > > Jason > ________________________________________ > From: Aleks Efimov > Sent: Monday, February 5, 2018 10:51 AM > To: Jason Mehrens; 'Roger Riggs' > Cc: core-libs-dev > Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception > > Hi Roger, Jason, > > I've tried to avoid doing the same stuff as I did for XPathException to > minimize the changes to the SAXException class. But I was naive to think so: > Tried to keep the exception field in place to avoid modifications > required to keep serial form intact (similar as it was done in > JDK-8009581 bug mentioned by Jason), but I missed the 'initCause' method > (spotted by Roger, thank you!) that can make cause/exception values > inconsistent. To avoid the API modification and for the sake of clarity > I proceeded with removal of the 'exception' field. The new version of > the fix: > - Removes 'exception' field > - Maintains the serial form of the SAXException class > - Handles the difference in SAXException:exception and Throwable:cause > types 'SAXException::getException', i.e. SAXException:exception is > Exception typed and Throwable:cause is Throwable typed. > - Avoids any changes to API and serial form of the class, but > maintaining it similar to JDK-8009581) > - There is a copy of SAXException class in java.base module that is > required for j.u.Properties::loadFromXML, it has been update with the > same changes. > > New regression test has been added to test: > - Serial compatibility with legacy JDKs (similar to JDK-8009581) > - usages of SAXException::initCause/getException > > Webrev with the fix and new regression: > http://cr.openjdk.java.net/~aefimov/6857903/dev/01/ > > Testing shows no JCK/regression tests failures with the proposed fix. > > Best Regards, > Aleksei > > > On 12/21/2017 07:00 PM, Jason Mehrens wrote: >> Aleksei, >> >> You have to override all of the constructors to always disable initCause. Actually a similar issue was covered in: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html >> >> Pretty much everything was hashed out in those threads. >> >> Here is the final webrev for that: >> >> http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html >> >> That should be a good cookbook for changes and tests required for this issue. Note that it gets rid of the Exception field and maintains serial compatibility. >> Looking back on that change, I would have changed it so the InvalidClassException added the double cause as suppressed exceptions. >> >> Jason >> >> ________________________________________ >> From: core-libs-dev on behalf of Aleks Efimov >> Sent: Thursday, December 21, 2017 11:27 AM >> To: core-libs-dev >> Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception >> >> Hello, >> >> Please, help to review the fix for jaxp bug that fixes SAXException to >> correctly set exception cause with 'setCause' method: >> http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ >> I've tried to keep the fix miminal with respect to serial form of this >> API class, i.e. kept private 'exception' field. >> >> The new test case was added to IssueTracker56Test unit test. Testing >> showed no related JCK/JTREG failures. >> >> With Best Regards, >> Aleksei >> >> From amaembo at gmail.com Tue Feb 6 03:05:31 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Tue, 6 Feb 2018 10:05:31 +0700 Subject: Collections.addAll: remove outdated performance advice and add a note about atomicity In-Reply-To: <8fb4dd69-1cb8-56db-c9c6-b4748d4f1698@oracle.com> References: <8fb4dd69-1cb8-56db-c9c6-b4748d4f1698@oracle.com> Message-ID: Hello! Thank you for pointers to the previous discussions. Yes, I would suggest to consider a spec cleanup separately, because adding new Collection default method would certainly take much longer time to discuss. I agree that we can exclude the statement about atomicity if it causes doubts. Removal of performance advice and "identical" word would suffice. At least this will not make people drawing wrong conclusions anymore. Thanks, Tagir. On Fri, Feb 2, 2018 at 7:55 AM, Stuart Marks wrote: > Links to existing material in OpenJDK: > > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050326.html > > https://bugs.openjdk.java.net/browse/JDK-8193031 > > I agree with the removal of the performance advice. We should also remove > "identical"; my suggested replacement was "as if". > > We can even add some hedging about whether the operation is atomic for > certain collections (i.e., the synchronized ones). However, I'm not sure if > there is anything useful to say about atomicity of bulk updates. They're > only atomic for the synchronized collections, which are largely disused. > It's pointless to talk about atomicity for non-concurrent collections, and > the operations aren't atomic for things like CopyOnWriteArrayList and a Set > projection of ConcurrentHashMap. So I'm not sure discussion of atomicity is > useful or warranted here. > > (Also, adding a collection of elements one at a time to a > CopyOnWriteArrayList: *shudder*.) > > I think the most useful thing is to define a new array-reading default > method. Each implementation can then override it to use the best technique > for that implementation. (I had previously called this "addEach" but I'm > flexible on naming.) > > Adding a new default method is kind of far afield from where this started. > If the spec is bothersome, perhaps we could consider a spec cleanup change > separately from implementation changes and definition of a new default > method. > > s'marks > > > > > > On 1/30/18 7:07 PM, Martin Buchholz wrote: >> >> I tried to tackle this here: >> http://openjdk.markmail.org/thread/eet2zd6ig3pfpv5g >> and it's still on my TODO list but not likely to get to top spot soon. >> >> On Tue, Jan 30, 2018 at 7:00 PM, Tagir Valeev wrote: >> >>> Hello! >>> >>> I suggest a patch for java.util.Collections#addAll JavaDoc: >>> >>> --- Collections.java 2018-01-31 09:39:31.599107500 +0700 >>> +++ Collections.java.patched 2018-01-31 09:51:11.929059600 +0700 >>> @@ -5406,4 +5406,8 @@ >>> * The behavior of this convenience method is identical to that of >>> - * {@code c.addAll(Arrays.asList(elements))}, but this method is >>> likely >>> - * to run significantly faster under most implementations. >>> + * {@code c.addAll(Arrays.asList(elements))} except possible >>> + * difference in intermediate state visibility for concurrent or >>> + * synchronized collections. Calling this method does not guarantee >>> + * that the intermediate state (some of elements are added) is >>> invisible, >>> + * even if the collection itself provides such guarantee for its >>> + * {@link Collection#addAll(Collection)} method. >>> * >>> >>> First, currently it says that Collections#addAll is likely to run >>> significantly faster. However it's only marginally faster for >>> collections which delegate their addAll method to standard >>> AbstractCollection#addAll implementation. Also it could be much slower >>> for collections which have optimized addAll (like ArrayList, >>> CopyOnWriteArrayList, ConcurrentLinkedDeque, etc.). I don't know a >>> single example of collection where Collections#addAll is actually >>> significantly faster. Also it says that the behavior is identical, >>> while it's not. If, e.g. c is a collection returned from >>> synchronizedCollection, then intermediate state of >>> c.addAll(Arrays.asList(elements)) would not be visible under >>> synchronized(c) in another thread. On the other hand, replacing such >>> call with Collections.addAll(c, elements) (to make it "significantly >>> faster") will lift this guarantee: now you can see partially added >>> array. >>> >>> What do you think? Should I file an issue? >>> >>> With best regards, >>> Tagir Valeev. >>> > From peter.levart at gmail.com Tue Feb 6 09:15:58 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 6 Feb 2018 10:15:58 +0100 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: <604f59a2-a3f2-5c3f-067b-2002665b9441@gmail.com> References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> <604f59a2-a3f2-5c3f-067b-2002665b9441@gmail.com> Message-ID: Hi, I checked the behavior of Objects.requireNonNull(this) at appropriate place instead of Reference.reachabilityFence(this) and it does seem to work. For example in the following test (see method m()): import java.util.Objects; import java.util.concurrent.atomic.AtomicLong; public class ReachabilityTest { ??? private static final AtomicLong seq = new AtomicLong(); ??? private static final AtomicLong deallocatedHwm = new AtomicLong(); ??? private static long allocate() { ??????? return seq.incrementAndGet(); ??? } ??? private static void deallocate(long address) { ??????? deallocatedHwm.accumulateAndGet(address, Math::max); ??? } ??? private static void access(long address) { ??????? if (deallocatedHwm.get() == address) { ??????????? throw new IllegalStateException( ??????????????? "Address " + address + " has allready been deallocated"); ??????? } ??? } ??? private long address = allocate(); ??? @SuppressWarnings("deprecation") ??? @Override ??? protected void finalize() throws Throwable { ??????? deallocate(address); ??????? address = 0; ??? } ??? void m() { ??????? long a = address; ??????? if (a != 0) { ??????????? System.gc(); ??????????? try { Thread.sleep(1); } catch (InterruptedException e) {} ??????????? access(a); ??????????? // Reference.reachabilityFence(this); ??????????? Objects.requireNonNull(this); ??????? } ??? } ??? static void test() { ??????? new ReachabilityTest().m(); ??? } ??? public static void main(String[] args) { ??????? while (true) { ??????????? test(); ??????? } ??? } } ...Objects.requireNonNull does prevent otherwise reliable provoked failure, just like Reference.reachabilityFence. As to the speed of optimized-away Objects.requireNonNull() vs. Reference.reachabilityFence() I tried the following benchmark: package jdk.test; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.BenchmarkMode; import org.openjdk.jmh.annotations.Fork; import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Measurement; import org.openjdk.jmh.annotations.Mode; import org.openjdk.jmh.annotations.OutputTimeUnit; import org.openjdk.jmh.annotations.Param; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import org.openjdk.jmh.annotations.Warmup; import java.lang.ref.Reference; import java.util.Arrays; import java.util.Objects; import java.util.concurrent.TimeUnit; @BenchmarkMode(Mode.AverageTime) @Warmup(iterations = 5) @Measurement(iterations = 10) @OutputTimeUnit(TimeUnit.NANOSECONDS) @Fork(1) @State(Scope.Thread) public class ReachabilityBench { ??? static class Buf0 { ??????? final byte[] buf; ??????? Buf0(int len) { ??????????? buf = new byte[len]; ??????? } ??????? @SuppressWarnings("deprecation") ??????? @Override ??????? protected void finalize() throws Throwable { ??????????? Arrays.fill(buf, (byte) -1); ??????? } ??????? byte get(int i) { ??????????? byte[] b = buf; ??????????? // this might get finalized before accessing the b element ??????????? byte r = b[i]; ??????????? return r; ??????? } ??? } ??? static class Buf1 { ??????? final byte[] buf; ??????? Buf1(int len) { ??????????? buf = new byte[len]; ??????? } ??????? @SuppressWarnings("deprecation") ??????? @Override ??????? protected void finalize() throws Throwable { ??????????? Arrays.fill(buf, (byte) -1); ??????? } ??????? byte get(int i) { ??????????? byte[] b = buf; ??????????? byte r = b[i]; ??????????? Reference.reachabilityFence(this); ??????????? return r; ??????? } ??? } ??? static class Buf2 { ??????? final byte[] buf; ??????? Buf2(int len) { ??????????? buf = new byte[len]; ??????? } ??????? @SuppressWarnings("deprecation") ??????? @Override ??????? protected void finalize() throws Throwable { ??????????? Arrays.fill(buf, (byte) -1); ??????? } ??????? byte get(int i) { ??????????? byte[] b = buf; ??????????? byte r = b[i]; ??????????? Objects.requireNonNull(this); ??????????? return r; ??????? } ??? } ??? Buf0 buf0; ??? Buf1 buf1; ??? Buf2 buf2; ??? @Param({"64"}) ??? public int len; ??? @Setup(Level.Trial) ??? public void setup() { ??????? buf0 = new Buf0(len); ??????? buf1 = new Buf1(len); ??????? buf2 = new Buf2(len); ??? } ??? @Benchmark ??? public int sumBuf0() { ??????? int s = 0; ??????? for (int i = 0; i < len; i++) { ??????????? s += buf0.get(i); ??????? } ??????? return s; ??? } ??? @Benchmark ??? public int sumBuf1() { ??????? int s = 0; ??????? for (int i = 0; i < len; i++) { ??????????? s += buf1.get(i); ??????? } ??????? return s; ??? } ??? @Benchmark ??? public int sumBuf2() { ??????? int s = 0; ??????? for (int i = 0; i < len; i++) { ??????????? s += buf2.get(i); ??????? } ??????? return s; ??? } } The results are as follows: Benchmark????????????????? (len)? Mode? Cnt??? Score?? Error? Units ReachabilityBench.sumBuf0???? 64? avgt?? 10?? 20.530 ? 0.052? ns/op ReachabilityBench.sumBuf1???? 64? avgt?? 10? 184.402 ? 0.531? ns/op ReachabilityBench.sumBuf2???? 64? avgt?? 10?? 20.502 ? 0.027? ns/op Objects.requireNonNull() shows zero overhead here. I guess the main question is whether Objects.requireNonNull(this) behavior in the former test is a result of chance and current Hotspot behavior or is it somehow guaranteed by the spec. Regards, Peter On 02/03/2018 02:04 AM, Peter Levart wrote: > Hi, > > I have one question, maybe stupid. I'm wondering about one situation. > Suppose you have this Java code: > > void m() { > ??? // code before... > > ??? Objects.requireNonNull(this); > } > > Of course, the non-null check will never throw NPE. The check will > most likely even be optimized away by JIT. But is JIT allowed to > optimize it away so that the "code before" observes the effects of > 'this' being unreachable? Wouldn't that violate the semantics of the > program as it would 1st observe that some object is not reachable and > after that it would observer that 'this' still points to some object > which by definition must be it? > > If JIT is not allowed to optimize the null check so that the object > becomes unreachable, then Objects.requireNonNull could be used as a > replacement for Reference.reachabilityFence. It might even perform > better. > > What do you think? > > Regards, Peter > > > On 02/01/18 23:50, Paul Sandoz wrote: >> Hi Ben, >> >> I don?t think you require the fence in all the cases you have >> currently placed it e.g. here for example >> >> ???????? $memtype$ y = $toBits$(x); >> ???????? UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); >> +??????? Reference.reachabilityFence(this); >> ???????? return this; >> >> since ?this? is being returned it will be kept live during the unsafe >> access. >> >> Would you mind providing a JMH benchmark and results for the >> performance with and without the fence for say a put and/or a get. I >> would like to understand the performance impact on HotSpot, this is >> one reason why we have yet to add such fences as it will likely >> impact performance. >> >> At the moment we are relying on the method not being inlined, which >> is an expedient technique to make it functional and keep a reference >> alive but not necessarily optimal for usages in DBB. >> >> For more details see: >> >> ?? https://bugs.openjdk.java.net/browse/JDK-8169605 >> >> ?? https://bugs.openjdk.java.net/browse/JDK-8149610 >> >> >> Thanks, >> Paul. >> >>> On Feb 1, 2018, at 6:55 AM, Ben Walsh wrote: >>> >>> This contribution forms a partial solution to the problem detailed >>> here - >>> http://thevirtualmachinist.blogspot.ca/2011/07/subtle-issue-of-reachability.html >>> >>> . >>> >>> In this context, this contribution provides "markers" such that a >>> suitably >>> "aware" compiler can reduce the chances of such a problem occurring >>> with >>> each of the DirectBuffer objects and MappedByteBuffer >>> object. The reachabilityFences may prevent crashes / exceptions due to >>> cleaning up the backing memory before the user has finished using the >>> pointer. >>> >>> Any compiler not suitably "aware" could be modified to make use of the >>> "markers". >>> >>> >>> I would like to pair with a sponsor to contribute this patch ... >>> >>> ------------------------------------------------------------------------------------------------------------------- >>> >>> diff -r d51e64840b4f >>> src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template >>> --- >>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template >>> >>> Wed Jan 31 12:04:53 2018 +0800 >>> +++ >>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template >>> >>> Thu Feb 01 11:30:10 2018 +0000 >>> @@ -1,5 +1,5 @@ >>> /* >>> - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights >>> reserved. >>> + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights >>> reserved. >>> ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> ? * >>> ? * This code is free software; you can redistribute it and/or >>> modify it >>> @@ -33,15 +33,21 @@ >>> >>> ???? private $type$ get$Type$(long a) { >>> ???????? $memtype$ x = UNSAFE.get$Memtype$Unaligned(null, a, >>> bigEndian); >>> -??????? return $fromBits$(x); >>> +??????? $type$ y = $fromBits$(x); >>> +??????? Reference.reachabilityFence(this); >>> +??????? return y; >>> ???? } >>> >>> ???? public $type$ get$Type$() { >>> -??????? return get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); >>> +??????? $type$ y = get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); >>> +??????? Reference.reachabilityFence(this); >>> +??????? return y; >>> ???? } >>> >>> ???? public $type$ get$Type$(int i) { >>> -??????? return get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); >>> +??????? $type$ y = get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); >>> +??????? Reference.reachabilityFence(this); >>> +??????? return y; >>> ???? } >>> >>> #end[rw] >>> @@ -50,6 +56,7 @@ >>> #if[rw] >>> ???????? $memtype$ y = $toBits$(x); >>> ???????? UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); >>> +??????? Reference.reachabilityFence(this); >>> ???????? return this; >>> #else[rw] >>> ???????? throw new ReadOnlyBufferException(); >>> diff -r d51e64840b4f >>> src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> --- >>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> Wed Jan 31 12:04:53 2018 +0800 >>> +++ >>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> Thu Feb 01 11:30:10 2018 +0000 >>> @@ -1,5 +1,5 @@ >>> /* >>> - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights >>> reserved. >>> + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights >>> reserved. >>> ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> ? * >>> ? * This code is free software; you can redistribute it and/or >>> modify it >>> @@ -28,6 +28,7 @@ >>> package java.nio; >>> >>> import java.io.FileDescriptor; >>> +import java.lang.ref.Reference; >>> import jdk.internal.misc.VM; >>> import jdk.internal.ref.Cleaner; >>> import sun.nio.ch.DirectBuffer; >>> @@ -312,6 +313,7 @@ >>> ???? public $Type$Buffer put($type$ x) { >>> #if[rw] >>> ???????? UNSAFE.put$Swaptype$(ix(nextPutIndex()), $swap$($toBits$(x))); >>> +??????? Reference.reachabilityFence(this); >>> ???????? return this; >>> #else[rw] >>> ???????? throw new ReadOnlyBufferException(); >>> @@ -321,6 +323,7 @@ >>> ???? public $Type$Buffer put(int i, $type$ x) { >>> #if[rw] >>> ???????? UNSAFE.put$Swaptype$(ix(checkIndex(i)), $swap$($toBits$(x))); >>> +??????? Reference.reachabilityFence(this); >>> ???????? return this; >>> #else[rw] >>> ???????? throw new ReadOnlyBufferException(); >>> @@ -347,6 +350,7 @@ >>> ???????????? if (srem > rem) >>> ???????????????? throw new BufferOverflowException(); >>> ???????????? UNSAFE.copyMemory(sb.ix(spos), ix(pos), (long)srem << >>> $LG_BYTES_PER_VALUE$); >>> +??????????? Reference.reachabilityFence(this); >>> ???????????? sb.position(spos + srem); >>> ???????????? position(pos + srem); >>> ???????? } else if (src.hb != null) { >>> @@ -413,6 +417,7 @@ >>> ???????? int rem = (pos <= lim ? lim - pos : 0); >>> >>> ???????? UNSAFE.copyMemory(ix(pos), ix(0), (long)rem << >>> $LG_BYTES_PER_VALUE$); >>> +??????? Reference.reachabilityFence(this); >>> ???????? position(rem); >>> ???????? limit(capacity()); >>> ???????? discardMark(); >>> @@ -505,6 +510,7 @@ >>> ???? void _put(int i, byte b) {????????????????? // package-private >>> #if[rw] >>> ???????? UNSAFE.putByte(address + i, b); >>> +??????? Reference.reachabilityFence(this); >>> #else[rw] >>> ???????? throw new ReadOnlyBufferException(); >>> #end[rw] >>> diff -r d51e64840b4f >>> src/java.base/share/classes/java/nio/MappedByteBuffer.java >>> --- a/src/java.base/share/classes/java/nio/MappedByteBuffer.java Wed >>> Jan >>> 31 12:04:53 2018 +0800 >>> +++ b/src/java.base/share/classes/java/nio/MappedByteBuffer.java Thu >>> Feb >>> 01 11:30:10 2018 +0000 >>> @@ -1,5 +1,5 @@ >>> /* >>> - * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights >>> reserved. >>> + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights >>> reserved. >>> ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> ? * >>> ? * This code is free software; you can redistribute it and/or >>> modify it >>> @@ -26,6 +26,7 @@ >>> package java.nio; >>> >>> import java.io.FileDescriptor; >>> +import java.lang.ref.Reference; >>> import jdk.internal.misc.Unsafe; >>> >>> >>> @@ -164,6 +165,7 @@ >>> ???????? // is computed as we go along to prevent the compiler from >>> otherwise >>> ???????? // considering the loop as dead code. >>> ???????? Unsafe unsafe = Unsafe.getUnsafe(); >>> +??????? Reference.reachabilityFence(this); >>> ???????? int ps = Bits.pageSize(); >>> ???????? int count = Bits.pageCount(length); >>> ???????? long a = mappingAddress(offset); >>> ------------------------------------------------------------------------------------------------------------------- >>> >>> >>> >>> Regards, >>> Ben Walsh >>> >>> >>> Unless stated otherwise above: >>> IBM United Kingdom Limited - Registered in England and Wales with >>> number >>> 741598. >>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire >>> PO6 3AU >>> > > From jason_mehrens at hotmail.com Tue Feb 6 15:25:51 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Tue, 6 Feb 2018 15:25:51 +0000 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: <3d4a77bf-5797-e1b9-6a85-977a00fda3e3@oracle.com> References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> , <3d4a77bf-5797-e1b9-6a85-977a00fda3e3@oracle.com> Message-ID: Aleksei, I missed this before but, it looks like we are calling the public getException method in writeObject. We should create a private method to service both the public getException method and writeObject to guard against subclass tampering. Everything else looks good. Jason ________________________________________ From: Aleks Efimov Sent: Monday, February 5, 2018 8:46 PM To: Jason Mehrens Cc: core-libs-dev Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception Thank you for your comments, Jason! New webrev can be found at this location: http://cr.openjdk.java.net/~aefimov/6857903/dev/02/ The list of changes: - this.initCause() -> super.initCause() - 'readObject' has been modified to convert IllegalStateException to InvalidClassException. The test case that triggers IllegalStateException in SAXException::readObject has been added to 'testReadObjectIllegalStateException' test method. Best Regards, Aleksei On 02/05/2018 05:09 PM, Jason Mehrens wrote: > Aleksei, > > Looks good. We should avoid this.initCause in readObject and prefer super.initCause so subclasses can't alter the deserialization behavior. > While I can't think of a case off the top of my head, I would prefer that we trap and convert IllegalStateException into a InvalidClassException in readObject. > That keeps the readObject contract intact in case something malicious is going on. That is just my preference though. > > Jason > ________________________________________ > From: Aleks Efimov > Sent: Monday, February 5, 2018 10:51 AM > To: Jason Mehrens; 'Roger Riggs' > Cc: core-libs-dev > Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception > > Hi Roger, Jason, > > I've tried to avoid doing the same stuff as I did for XPathException to > minimize the changes to the SAXException class. But I was naive to think so: > Tried to keep the exception field in place to avoid modifications > required to keep serial form intact (similar as it was done in > JDK-8009581 bug mentioned by Jason), but I missed the 'initCause' method > (spotted by Roger, thank you!) that can make cause/exception values > inconsistent. To avoid the API modification and for the sake of clarity > I proceeded with removal of the 'exception' field. The new version of > the fix: > - Removes 'exception' field > - Maintains the serial form of the SAXException class > - Handles the difference in SAXException:exception and Throwable:cause > types 'SAXException::getException', i.e. SAXException:exception is > Exception typed and Throwable:cause is Throwable typed. > - Avoids any changes to API and serial form of the class, but > maintaining it similar to JDK-8009581) > - There is a copy of SAXException class in java.base module that is > required for j.u.Properties::loadFromXML, it has been update with the > same changes. > > New regression test has been added to test: > - Serial compatibility with legacy JDKs (similar to JDK-8009581) > - usages of SAXException::initCause/getException > > Webrev with the fix and new regression: > http://cr.openjdk.java.net/~aefimov/6857903/dev/01/ > > Testing shows no JCK/regression tests failures with the proposed fix. > > Best Regards, > Aleksei > > > On 12/21/2017 07:00 PM, Jason Mehrens wrote: >> Aleksei, >> >> You have to override all of the constructors to always disable initCause. Actually a similar issue was covered in: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html >> >> Pretty much everything was hashed out in those threads. >> >> Here is the final webrev for that: >> >> http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html >> >> That should be a good cookbook for changes and tests required for this issue. Note that it gets rid of the Exception field and maintains serial compatibility. >> Looking back on that change, I would have changed it so the InvalidClassException added the double cause as suppressed exceptions. >> >> Jason >> >> ________________________________________ >> From: core-libs-dev on behalf of Aleks Efimov >> Sent: Thursday, December 21, 2017 11:27 AM >> To: core-libs-dev >> Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception >> >> Hello, >> >> Please, help to review the fix for jaxp bug that fixes SAXException to >> correctly set exception cause with 'setCause' method: >> http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ >> I've tried to keep the fix miminal with respect to serial form of this >> API class, i.e. kept private 'exception' field. >> >> The new test case was added to IssueTracker56Test unit test. Testing >> showed no related JCK/JTREG failures. >> >> With Best Regards, >> Aleksei >> >> From Roger.Riggs at Oracle.com Tue Feb 6 15:31:54 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 6 Feb 2018 10:31:54 -0500 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: <3d4a77bf-5797-e1b9-6a85-977a00fda3e3@oracle.com> References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> <3d4a77bf-5797-e1b9-6a85-977a00fda3e3@oracle.com> Message-ID: <89e36dca-ace4-ec9e-539b-5667b70502e3@Oracle.com> Hi Aleksei, SAXException.java x 2, SAXExceptionInitCause: ? Please use explicit imports (not java.io.*). SAXEXception.java:145: I would have expected: if (cause instanceof Exception) {...} SAXExceptionInitCause.java: ? Great to see all the test cases. ? Generally, we recommend using try-with-resources but in this case it does not add much because ? exceptions can't occur when using BAOS. Thanks, Roger On 2/5/2018 9:46 PM, Aleks Efimov wrote: > Thank you for your comments, Jason! > New webrev can be found at this location: > http://cr.openjdk.java.net/~aefimov/6857903/dev/02/ > > The list of changes: > - this.initCause() -> super.initCause() > - 'readObject' has been modified to convert IllegalStateException to > InvalidClassException. The test case that triggers > IllegalStateException in SAXException::readObject has been added to > 'testReadObjectIllegalStateException' test method. > > Best Regards, > Aleksei > > On 02/05/2018 05:09 PM, Jason Mehrens wrote: >> Aleksei, >> >> Looks good.? We should avoid this.initCause in readObject and prefer >> super.initCause so subclasses can't alter the deserialization behavior. >> While I can't think of a case off the top of my head, I would prefer >> that we trap and convert IllegalStateException into a >> InvalidClassException in readObject. >> That keeps the readObject contract intact in case something malicious >> is going on.? That is just my preference though. >> >> Jason >> ________________________________________ >> From: Aleks Efimov >> Sent: Monday, February 5, 2018 10:51 AM >> To: Jason Mehrens; 'Roger Riggs' >> Cc: core-libs-dev >> Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does >> not correctly set Exception >> >> Hi Roger, Jason, >> >> I've tried to avoid doing the same stuff as I did for XPathException to >> minimize the changes to the SAXException class. But I was naive to >> think so: >> Tried to keep the exception field in place to avoid modifications >> required to keep serial form intact (similar as it was done in >> JDK-8009581 bug mentioned by Jason), but I missed the 'initCause' method >> (spotted by Roger, thank you!) that can make cause/exception values >> inconsistent. To avoid the API modification and for the sake of clarity >> I proceeded with removal of the 'exception' field. The new version of >> the fix: >> - Removes 'exception' field >> - Maintains the serial form of the SAXException class >> - Handles the difference in SAXException:exception and Throwable:cause >> types 'SAXException::getException', i.e. SAXException:exception is >> Exception typed and Throwable:cause is Throwable typed. >> - Avoids any changes to API and serial form of the class, but >> maintaining it similar to JDK-8009581) >> - There is a copy of SAXException class in java.base module that is >> required for j.u.Properties::loadFromXML, it has been update with the >> same changes. >> >> New regression test has been added to test: >> - Serial compatibility with legacy JDKs (similar to JDK-8009581) >> - usages of SAXException::initCause/getException >> >> Webrev with the fix and new regression: >> http://cr.openjdk.java.net/~aefimov/6857903/dev/01/ >> >> Testing shows no JCK/regression tests failures with the proposed fix. >> >> Best Regards, >> Aleksei >> >> >> On 12/21/2017 07:00 PM, Jason Mehrens wrote: >>> Aleksei, >>> >>> You have to override all of the constructors to always disable >>> initCause.? Actually a similar issue was covered in: >>> >>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html >>> >>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html >>> >>> >>> Pretty much everything was hashed out in those threads. >>> >>> Here is the final webrev for that: >>> >>> http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html >>> >>> >>> That should be a good cookbook for changes and tests required for >>> this issue.? Note that it gets rid of the Exception field and >>> maintains serial compatibility. >>> Looking back on that change, I would have changed it so the >>> InvalidClassException added the double cause as suppressed exceptions. >>> >>> Jason >>> >>> ________________________________________ >>> From: core-libs-dev on >>> behalf of Aleks Efimov >>> Sent: Thursday, December 21, 2017 11:27 AM >>> To: core-libs-dev >>> Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not >>> correctly set Exception >>> >>> Hello, >>> >>> Please, help to review the fix for jaxp bug that fixes SAXException to >>> correctly set exception cause with 'setCause' method: >>> http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ >>> I've tried to keep the fix miminal with respect to serial form of this >>> API class, i.e. kept private 'exception' field. >>> >>> The new test case was added to IssueTracker56Test unit test. Testing >>> showed no related JCK/JTREG failures. >>> >>> With Best Regards, >>> Aleksei >>> >>> > From Roger.Riggs at Oracle.com Tue Feb 6 16:28:56 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 6 Feb 2018 11:28:56 -0500 Subject: RFR JDK-8164278: java.util.Base64.EncOutputStream/DecInputStream is slower than corresponding version in javax.mail package In-Reply-To: <5A790C4D.1090309@oracle.com> References: <5A790C4D.1090309@oracle.com> Message-ID: <6521d6c9-ee09-e830-1535-ac290ccdb76d@Oracle.com> Hi Sherman, On 2/5/2018 9:00 PM, Xueming Shen wrote: > Hi, > > Please help review the change for? JDK-8164278. > > issue: https://bugs.openjdk.java.net/browse/JDK-8164278 > webrev: http://cr.openjdk.java.net/~sherman/8164278/webrev Are the reentrant locks necessary?? concurrent reads from streams are not usually synchronized so its the caller that need to synchronize. If locks are necessary, why no lock for the EncOutputStream buffer? 809: Can the buffer byte array be sized based on linemax?? The field declaration should ? be with the other fields at the top of the file. 848: checkNewline compares == with linemax; that works when each byte is counted separately ?It seems like it would safer if it was ">=". 957: can sbBuf be shared with lbuf? More on the input buffering question below. > > jmh.src: http://cr.openjdk.java.net/~sherman/8164278/Base64BM.java > jmh.result: http://cr.openjdk.java.net/~sherman/8164278/base64.bm > > Base64.Decoder.decode0: > ??? Adopted the "similar" optimization approach we took in > Base64.Encoder.encode0() > ??? to add a "fast path" to decode a block of 4-byte units together > (current implementation > ??? decodes one single byte per while/loop. The jmh benchmark result > indicates a big speed > ??? boost? (those decodeArray/Mime/Url results, from 30% to 2 times > faster, depends on > ??? input size). :) > > Base64.Encoder.encode0() > ??? It appears encode0() was fully optimized in 1.8. Can't get it > faster :-) Tried to use > ??? Unsafe.getLong/putLong instead of byte by byte access. But it > appears the 8-byte > ??? "vectorization" does not bring us enough speed up, the performance > is the same as the > ??? current one. See encode00() at > ??? http://cr.openjdk.java.net/~sherman/8164278/webrev.00 > > Base64.Encoder.wrap(OutputStream)/EncOutputStream.write(): > ??? If my memory serves me right, the current implementation was under > the assumption that > ??? the underlying output stream probably is buffered (if invoker > cares). It would be a redundant > ??? if EncOutputStream buffers bytes again. It appears this is a wrong > assumption. It is much > ??? slower to write 4 bytes separately, compared to bundle them > together in a byte[4] and write > ??? into underlying, even the underlying output stream is a > ByteArrayOutputStream. > ??? Again, the proposed change is to add a fast path loop, as we do in > encode0(), to decode/ > ??? write a block of 3-byte->4-byte unites. It appears this fast loop > can help the compiler to > ??? optimize away some boundary checks, therefor is much faster. > ??? The jmh result Base64BM.encodeOS suggests the new implementation > is almost 4 times faster > ??? and is almost the same as java.mail's stream encoder. > > Base64.Decoder.wrap(InputStream)/DecInputStream.read(): > ??? Same as the approach we take for decode0(), to add a fast path > decode block of 4-byte unites > ??? together. > ??? The jmh result Base64BM.decodeOS (the name probably should be > decodeIS, for InputStream, > ??? but anyway...) shows the proposed one is 4 times faster than the > existing impl and double > ??? the? java.mail (Base64BM.decodeOS_javamail) implementation. > > ??? However, there is a side-effect of adding a buffering mechanism > into DecInputStream. The > ??? current implementation read bytes from the underlying stream one > by one, it never reads > ??? more bytes than it needs, which means it should/is supposed to > just stop at the last byte > ??? that it needs to decode, when there is "=" present in the stream. > ??? With buffering, it's possible more bytes (after "=", which > indicates "end of base64 stream") might > ??? be read/consumed in and buffered.? A concern? if this is indeed a > concern, the only alternative > ??? might be to add a separate method to support this > "faster-buffered-decoder"? How much buffering is needed to speed it up?? Can the mark/reset functions of the underlying stream be used to backup the stream if it overshoots? If mark and reset are not supported then read 1 byte at a time. Regards, Roger > > Thanks, > Sherman > > > From xueming.shen at oracle.com Tue Feb 6 16:56:59 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 06 Feb 2018 08:56:59 -0800 Subject: RFR JDK-8164278: java.util.Base64.EncOutputStream/DecInputStream is slower than corresponding version in javax.mail package In-Reply-To: <6521d6c9-ee09-e830-1535-ac290ccdb76d@Oracle.com> References: <5A790C4D.1090309@oracle.com> <6521d6c9-ee09-e830-1535-ac290ccdb76d@Oracle.com> Message-ID: <5A79DE5B.70600@oracle.com> On 2/6/18, 8:28 AM, Roger Riggs wrote: > Hi Sherman, > > On 2/5/2018 9:00 PM, Xueming Shen wrote: >> Hi, >> >> Please help review the change for JDK-8164278. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8164278 >> webrev: http://cr.openjdk.java.net/~sherman/8164278/webrev > Are the reentrant locks necessary? concurrent reads from streams are > not usually > synchronized so its the caller that need to synchronize. > If locks are necessary, why no lock for the EncOutputStream buffer? > Thanks Roger, I don't really care the "correctnesss" of the de/coding via the in/output stream under concurrent access. As you said the caller should take the responsibility for that. In case of DecInputStream there is risk that the "pos" might be updated out of the buffer boundary and trigger exception, if under concurrent access. It's not the case for EncOutputStream. But yes, let me see if I can get away with this via local copy of the pos/cnt. > > 809: Can the buffer byte array be sized based on linemax? The field > declaration should > be with the other fields at the top of the file. sure possible with some calculation. > > > 848: checkNewline compares == with linemax; that works when each byte > is counted separately > It seems like it would safer if it was ">=". updated. > > 957: can sbBuf be shared with lbuf? probably not. lbuf is for the bytes from underlying input stream. sbBuf is to store/get the first byte of the decoded result. > >> However, there is a side-effect of adding a buffering mechanism >> into DecInputStream. The >> current implementation read bytes from the underlying stream one >> by one, it never reads >> more bytes than it needs, which means it should/is supposed to >> just stop at the last byte >> that it needs to decode, when there is "=" present in the stream. >> With buffering, it's possible more bytes (after "=", which >> indicates "end of base64 stream") might >> be read/consumed in and buffered. A concern? if this is indeed a >> concern, the only alternative >> might be to add a separate method to support this >> "faster-buffered-decoder"? > > How much buffering is needed to speed it up? Can the mark/reset > functions of the underlying > stream be used to backup the stream if it overshoots? > If mark and reset are not supported then read 1 byte at a time. > will take a look at the mark/reset possibility. maybe I will put DecInputStream optimization to a separate rfe. -Sherman From xueming.shen at oracle.com Tue Feb 6 17:23:29 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 06 Feb 2018 09:23:29 -0800 Subject: RFR JDK-8164278: java.util.Base64.EncOutputStream/DecInputStream is slower than corresponding version in javax.mail package In-Reply-To: <5A79DE5B.70600@oracle.com> References: <5A790C4D.1090309@oracle.com> <6521d6c9-ee09-e830-1535-ac290ccdb76d@Oracle.com> <5A79DE5B.70600@oracle.com> Message-ID: <5A79E491.8090602@oracle.com> On 2/6/18, 8:56 AM, Xueming Shen wrote: > >> >> 848: checkNewline compares == with linemax; that works when each byte >> is counted separately >> It seems like it would safer if it was ">=". > > updated. > I took it back. >= does not work here. -sherman From claes.redestad at oracle.com Tue Feb 6 17:51:22 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 6 Feb 2018 18:51:22 +0100 Subject: RFR: 8196869: Optimize Locale creation Message-ID: Hi, we can refactor sun.util.locale.BaseLocale+LocaleObjectCache to minimize the number of SoftReferences created in Locale:: and when looking up already defined BaseLocales inside the Locale constructor. http://cr.openjdk.java.net/~redestad/8196869/jdk.00/ This is mainly a tiny startup optimization, dropping executed bytecode during startup by a few thousand and reducing the minimum retained heap by a few Kb, but also speeds up microbenchmarks repeatedly calling the Locale constructor by ~1.25x. Tests stressing that dereferenced Locales are disposed of promptly remain happy. Thanks! /Claes From paul.sandoz at oracle.com Tue Feb 6 19:55:34 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 6 Feb 2018 11:55:34 -0800 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: References: Message-ID: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> Quick observation: 261 private BaseLocale getBaseLocale() { 262 return (holder == null) ? holderRef.get() : holder; 263 } This method can return null if the soft ref has been cleared. But you don?t check in equals: 270 if (obj instanceof Key && this.hash == ((Key)obj).hash) { 271 BaseLocale other = ((Key) obj).getBaseLocale(); 272 BaseLocale locale = this.getBaseLocale(); 273 if (LocaleUtils.caseIgnoreMatch(other.getLanguage(), locale.getLanguage()) Paul. > On Feb 6, 2018, at 9:51 AM, Claes Redestad wrote: > > Hi, > > we can refactor sun.util.locale.BaseLocale+LocaleObjectCache to minimize the number of SoftReferences created in Locale:: and when looking up already defined BaseLocales inside the Locale constructor. > > http://cr.openjdk.java.net/~redestad/8196869/jdk.00/ > > This is mainly a tiny startup optimization, dropping executed bytecode during startup by a few thousand and reducing the minimum retained heap by a few Kb, but also speeds up microbenchmarks repeatedly calling the Locale constructor by ~1.25x. Tests stressing that dereferenced Locales are disposed of promptly remain happy. > > Thanks! > > /Claes > From vladimir.x.ivanov at oracle.com Tue Feb 6 19:23:48 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 6 Feb 2018 22:23:48 +0300 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> <604f59a2-a3f2-5c3f-067b-2002665b9441@gmail.com> Message-ID: <4ef38d5d-d349-4cb6-e7b3-019f094cc314@oracle.com> > Objects.requireNonNull() shows zero overhead here. > > I guess the main question is whether Objects.requireNonNull(this) > behavior in the former test is a result of chance and current Hotspot > behavior or is it somehow guaranteed by the spec. I haven't looked into what actually happens in JIT-compilers on your benchmark, but I'm surprised it works at all. Explicit null check on the receiver is an easy target for elimination and should be effectively a no-op in generated code. (And that's what you observe with the benchmark!) Once the check is gone, nothing keeps receiver alive anymore (past the last usage). So, I'd say such behavior it's a matter of chance in your case and can't be relied on in general. And definitely not something guaranteed by JVMS. Best regards, Vladimir Ivanov From claes.redestad at oracle.com Wed Feb 7 10:26:56 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 7 Feb 2018 11:26:56 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> Message-ID: Hi Paul, On 2018-02-06 20:55, Paul Sandoz wrote: > Quick observation: > > 261 private BaseLocale getBaseLocale() { > 262 return (holder == null) ? holderRef.get() : holder; > 263 } > > This method can return null if the soft ref has been cleared. > > > But you don?t check in equals: > > 270 if (obj instanceof Key && this.hash == ((Key)obj).hash) { > 271 BaseLocale other = ((Key) obj).getBaseLocale(); > 272 BaseLocale locale = this.getBaseLocale(); > 273 if (LocaleUtils.caseIgnoreMatch(other.getLanguage(), locale.getLanguage()) good eye! It seems this wasn't caught by the existing regression tests since none of them recreate Locales in that are likely to have been reclaimed, but still likely to still be in the CHM (it's a race of sorts since they'll be removed when the ReferenceQueue processing happen). I added a regression test with the smallest and quickest reproducer I could come up with that provokes a NPE if we don't check null along with the fix to Key#equals: http://cr.openjdk.java.net/~redestad/8196869/jdk.01/ For the normalize(Key) case we can deduce that a !normalized Key will always have a strongly referenced BaseLocale and thus not need to deal with getBaseLocale() returning null. I clarified this in the code and added an assert (that would be triggered by the added test if it wasn't true). Thanks! /Claes From peter.levart at gmail.com Wed Feb 7 12:12:50 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 7 Feb 2018 13:12:50 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> Message-ID: <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> Hi Claes, I studied the code briefly and understand why BaseLocale.Key now has to hold a SoftReference to a BaseLocale object when the same object is also part of CacheEntry which is also a SoftReference. But I don't see a reason why pre-patch BaseLocale.Key had to hold SoftReference(s) to individual String attributes. Couldn't it simply hold strong references to individual String attributes instead? The LocaleObjectCache.cleanStaleEntryies() would remove cleared CacheEntry(s) together with corresponding Key(s) in that case too. So one SoftReference less, do you agree? I don't know if it is important for LocaleObjectCache.get() to always return a canonicalized instance per key so that this always holds: ??? (cache.get(k1) == cache.get(k2)) == k1.equals(k2) If it is important, then I noticed a pre-existing race that violates above invariant: ? 67???????????? CacheEntry newEntry = new CacheEntry<>(key, newVal, queue); ? 68 ? 69???????????? entry = map.putIfAbsent(key, newEntry); ? 70???????????? if (entry == null) { ? 71???????????????? value = newVal; ? 72???????????? } else { ? 73???????????????? value = entry.get(); ? 74???????????????? if (value == null) { ? 75???????????????????? map.put(key, newEntry); ? 76???????????????????? value = newVal; ? 77???????????????? } ? 78???????????? } ...which can simply be fixed: ??????????? CacheEntry newEntry = new CacheEntry<>(key, newVal, queue); ??????????? while (true) { ??????????????? entry = map.putIfAbsent(key, newEntry); ??????????????? if (entry == null) { ??????????????????? value = newVal; ??????????????????? break; ??????????????? } else { ??????????????????? value = entry.get(); ??????????????????? if (value == null) { ??????????????????????? if (map.replace(key, entry, newEntry)) { ??????????????????????????? value = newVal; ??????????????????????????? break; ??????????????????????? } ??????????????????? } ??????????????? } ??????????? } Regards, Peter On 02/07/2018 11:26 AM, Claes Redestad wrote: > Hi Paul, > > > On 2018-02-06 20:55, Paul Sandoz wrote: >> Quick observation: >> >> ? 261???????? private BaseLocale getBaseLocale() { >> ? 262???????????? return (holder == null) ? holderRef.get() : holder; >> ? 263???????? } >> >> This method can return null if the soft ref has been cleared. >> >> >> But you don?t check in equals: >> >> ? 270???????????? if (obj instanceof Key && this.hash == >> ((Key)obj).hash) { >> ? 271???????????????? BaseLocale other = ((Key) obj).getBaseLocale(); >> ? 272???????????????? BaseLocale locale = this.getBaseLocale(); >> ? 273???????????????? if >> (LocaleUtils.caseIgnoreMatch(other.getLanguage(), locale.getLanguage()) > > good eye! > > It seems this wasn't caught by the existing regression tests since > none of them > recreate Locales in that are likely to have been reclaimed, but still > likely to still > be in the CHM (it's a race of sorts since they'll be removed when the > ReferenceQueue > processing happen). > > I added a regression test with the smallest and quickest reproducer I > could come up > with that provokes a NPE if we don't check null along with the fix to > Key#equals: > > http://cr.openjdk.java.net/~redestad/8196869/jdk.01/ > > For the normalize(Key) case we can deduce that a !normalized Key will > always have > a strongly referenced BaseLocale and thus not need to deal with > getBaseLocale() > returning null. I clarified this in the code and added an assert (that > would be triggered > by the added test if it wasn't true). > > Thanks! > > /Claes From peter.levart at gmail.com Wed Feb 7 12:31:29 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 7 Feb 2018 13:31:29 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> Message-ID: Hi Claes, Maybe I was to quick with my clicking on Send button... If the Key simply held strong references to individual String attributes, LocaleObjectCache.cleanStaleEntries would also have to be modified to make sure it does not remove valid entries that happen to share equal Key(s) with cleared entries. So instead of this: ??? private void cleanStaleEntries() { ??????? CacheEntry entry; ??????? while ((entry = (CacheEntry)queue.poll()) != null) { ??????????? map.remove(entry.getKey()); ??????? } ??? } The method would have to be like this: ??? private void cleanStaleEntries() { ??????? CacheEntry entry; ??????? while ((entry = (CacheEntry)queue.poll()) != null) { ??????????? map.remove(entry.getKey(), entry); ??????? } ??? } (Notice the use of two-argument Map.remove() method in the modified variant). Regards, Peter P.S. I now understand the hypothetical need to have individual String attributes wrapped with SoftReference(s) in pre-patched Key. The code maybe relied on the fact that SoftReference(s) to individual String attributes were cleared together with CacheEntry(s). When they were cleared, such Keys suddenly only matched themselves (i.e. no other Key instance would be equal to them). But if Key's SoftReference(s) were not cleared before corresponding CacheEntry was cleared, cleanStaleEntries() running concurrently with get() could remove freshly inserted entries too. This would not be observed as wrong behavior though. Just sub-optimal performance. On 02/07/2018 01:12 PM, Peter Levart wrote: > Hi Claes, > > I studied the code briefly and understand why BaseLocale.Key now has > to hold a SoftReference to a BaseLocale object when the same object is > also part of CacheEntry which is also a SoftReference. But I don't see > a reason why pre-patch BaseLocale.Key had to hold SoftReference(s) to > individual String attributes. Couldn't it simply hold strong > references to individual String attributes instead? The > LocaleObjectCache.cleanStaleEntryies() would remove cleared > CacheEntry(s) together with corresponding Key(s) in that case too. So > one SoftReference less, do you agree? > > I don't know if it is important for LocaleObjectCache.get() to always > return a canonicalized instance per key so that this always holds: > > ??? (cache.get(k1) == cache.get(k2)) == k1.equals(k2) > > If it is important, then I noticed a pre-existing race that violates > above invariant: > > ? 67???????????? CacheEntry newEntry = new CacheEntry<>(key, > newVal, queue); > ? 68 > ? 69???????????? entry = map.putIfAbsent(key, newEntry); > ? 70???????????? if (entry == null) { > ? 71???????????????? value = newVal; > ? 72???????????? } else { > ? 73???????????????? value = entry.get(); > ? 74???????????????? if (value == null) { > ? 75???????????????????? map.put(key, newEntry); > ? 76???????????????????? value = newVal; > ? 77???????????????? } > ? 78???????????? } > > ...which can simply be fixed: > > ??????????? CacheEntry newEntry = new CacheEntry<>(key, newVal, > queue); > > ??????????? while (true) { > ??????????????? entry = map.putIfAbsent(key, newEntry); > ??????????????? if (entry == null) { > ??????????????????? value = newVal; > ??????????????????? break; > ??????????????? } else { > ??????????????????? value = entry.get(); > ??????????????????? if (value == null) { > ??????????????????????? if (map.replace(key, entry, newEntry)) { > ??????????????????????????? value = newVal; > ??????????????????????????? break; > ??????????????????????? } > ??????????????????? } > ??????????????? } > ??????????? } > > > Regards, Peter > > > On 02/07/2018 11:26 AM, Claes Redestad wrote: >> Hi Paul, >> >> >> On 2018-02-06 20:55, Paul Sandoz wrote: >>> Quick observation: >>> >>> ? 261???????? private BaseLocale getBaseLocale() { >>> ? 262???????????? return (holder == null) ? holderRef.get() : holder; >>> ? 263???????? } >>> >>> This method can return null if the soft ref has been cleared. >>> >>> >>> But you don?t check in equals: >>> >>> ? 270???????????? if (obj instanceof Key && this.hash == >>> ((Key)obj).hash) { >>> ? 271???????????????? BaseLocale other = ((Key) obj).getBaseLocale(); >>> ? 272???????????????? BaseLocale locale = this.getBaseLocale(); >>> ? 273???????????????? if >>> (LocaleUtils.caseIgnoreMatch(other.getLanguage(), locale.getLanguage()) >> >> good eye! >> >> It seems this wasn't caught by the existing regression tests since >> none of them >> recreate Locales in that are likely to have been reclaimed, but still >> likely to still >> be in the CHM (it's a race of sorts since they'll be removed when the >> ReferenceQueue >> processing happen). >> >> I added a regression test with the smallest and quickest reproducer I >> could come up >> with that provokes a NPE if we don't check null along with the fix to >> Key#equals: >> >> http://cr.openjdk.java.net/~redestad/8196869/jdk.01/ >> >> For the normalize(Key) case we can deduce that a !normalized Key will >> always have >> a strongly referenced BaseLocale and thus not need to deal with >> getBaseLocale() >> returning null. I clarified this in the code and added an assert >> (that would be triggered >> by the added test if it wasn't true). >> >> Thanks! >> >> /Claes > From claes.redestad at oracle.com Wed Feb 7 12:48:55 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 7 Feb 2018 13:48:55 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> Message-ID: <82ead3ee-63a8-854c-429e-98ef05ccddf1@oracle.com> Hi, On 2018-02-07 13:12, Peter Levart wrote: > Hi Claes, > > I studied the code briefly and understand why BaseLocale.Key now has > to hold a SoftReference to a BaseLocale object when the same object is > also part of CacheEntry which is also a SoftReference. But I don't see > a reason why pre-patch BaseLocale.Key had to hold SoftReference(s) to > individual String attributes. Couldn't it simply hold strong > references to individual String attributes instead? The > LocaleObjectCache.cleanStaleEntryies() would remove cleared > CacheEntry(s) together with corresponding Key(s) in that case too. So > one SoftReference less, do you agree? cleanStaleEntries is sufficient for making sure the Key gets cleared eventually, yes, but having part of the Key softly reachable expedites memory reclamation in some important cases. > > I don't know if it is important for LocaleObjectCache.get() to always > return a canonicalized instance per key so that this always holds: > > ??? (cache.get(k1) == cache.get(k2)) == k1.equals(k2) I believe LocaleObjectCache is intended as a best effort cache solution with soft memory semantics for performance, not a strict canonicalization facility. That said I think removing the race you pointed out here is probably a good thing to do. /Claes > > If it is important, then I noticed a pre-existing race that violates > above invariant: > > ? 67???????????? CacheEntry newEntry = new CacheEntry<>(key, > newVal, queue); > ? 68 > ? 69???????????? entry = map.putIfAbsent(key, newEntry); > ? 70???????????? if (entry == null) { > ? 71???????????????? value = newVal; > ? 72???????????? } else { > ? 73???????????????? value = entry.get(); > ? 74???????????????? if (value == null) { > ? 75???????????????????? map.put(key, newEntry); > ? 76???????????????????? value = newVal; > ? 77???????????????? } > ? 78???????????? } > > ...which can simply be fixed: > > ??????????? CacheEntry newEntry = new CacheEntry<>(key, newVal, > queue); > > ??????????? while (true) { > ??????????????? entry = map.putIfAbsent(key, newEntry); > ??????????????? if (entry == null) { > ??????????????????? value = newVal; > ??????????????????? break; > ??????????????? } else { > ??????????????????? value = entry.get(); > ??????????????????? if (value == null) { > ??????????????????????? if (map.replace(key, entry, newEntry)) { > ??????????????????????????? value = newVal; > ??????????????????????????? break; > ??????????????????????? } > ??????????????????? } > ??????????????? } > ??????????? } > > > Regards, Peter > > > On 02/07/2018 11:26 AM, Claes Redestad wrote: >> Hi Paul, >> >> >> On 2018-02-06 20:55, Paul Sandoz wrote: >>> Quick observation: >>> >>> ? 261???????? private BaseLocale getBaseLocale() { >>> ? 262???????????? return (holder == null) ? holderRef.get() : holder; >>> ? 263???????? } >>> >>> This method can return null if the soft ref has been cleared. >>> >>> >>> But you don?t check in equals: >>> >>> ? 270???????????? if (obj instanceof Key && this.hash == >>> ((Key)obj).hash) { >>> ? 271???????????????? BaseLocale other = ((Key) obj).getBaseLocale(); >>> ? 272???????????????? BaseLocale locale = this.getBaseLocale(); >>> ? 273???????????????? if >>> (LocaleUtils.caseIgnoreMatch(other.getLanguage(), locale.getLanguage()) >> >> good eye! >> >> It seems this wasn't caught by the existing regression tests since >> none of them >> recreate Locales in that are likely to have been reclaimed, but still >> likely to still >> be in the CHM (it's a race of sorts since they'll be removed when the >> ReferenceQueue >> processing happen). >> >> I added a regression test with the smallest and quickest reproducer I >> could come up >> with that provokes a NPE if we don't check null along with the fix to >> Key#equals: >> >> http://cr.openjdk.java.net/~redestad/8196869/jdk.01/ >> >> For the normalize(Key) case we can deduce that a !normalized Key will >> always have >> a strongly referenced BaseLocale and thus not need to deal with >> getBaseLocale() >> returning null. I clarified this in the code and added an assert >> (that would be triggered >> by the added test if it wasn't true). >> >> Thanks! >> >> /Claes > From peter.levart at gmail.com Wed Feb 7 12:55:23 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 7 Feb 2018 13:55:23 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: <82ead3ee-63a8-854c-429e-98ef05ccddf1@oracle.com> References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> <82ead3ee-63a8-854c-429e-98ef05ccddf1@oracle.com> Message-ID: <55e2d730-3773-ad52-64ef-5de8ae5f43db@gmail.com> Hi Claes, On 02/07/2018 01:48 PM, Claes Redestad wrote: > Hi, > > On 2018-02-07 13:12, Peter Levart wrote: >> Hi Claes, >> >> I studied the code briefly and understand why BaseLocale.Key now has >> to hold a SoftReference to a BaseLocale object when the same object >> is also part of CacheEntry which is also a SoftReference. But I don't >> see a reason why pre-patch BaseLocale.Key had to hold >> SoftReference(s) to individual String attributes. Couldn't it simply >> hold strong references to individual String attributes instead? The >> LocaleObjectCache.cleanStaleEntryies() would remove cleared >> CacheEntry(s) together with corresponding Key(s) in that case too. So >> one SoftReference less, do you agree? > > cleanStaleEntries is sufficient for making sure the Key gets cleared > eventually, yes, but having > part of the Key softly reachable expedites memory reclamation in some > important cases. When no cleanStaleEntries() is called for a long time, right? Would making CacheEntry extend jdk.internal.ref.SoftCleanable instead of SoftReference help here? You could remove the cleanStaleEntries method entirely and just remove the Map entry in SoftCleanable's performCleanup method. Regards, Peter > >> >> I don't know if it is important for LocaleObjectCache.get() to always >> return a canonicalized instance per key so that this always holds: >> >> ??? (cache.get(k1) == cache.get(k2)) == k1.equals(k2) > > I believe LocaleObjectCache is intended as a best effort cache > solution with soft memory > semantics for performance, not a strict canonicalization facility. > That said I think removing > the race you pointed out here is probably a good thing to do. > > /Claes > >> >> If it is important, then I noticed a pre-existing race that violates >> above invariant: >> >> ? 67???????????? CacheEntry newEntry = new CacheEntry<>(key, >> newVal, queue); >> ? 68 >> ? 69???????????? entry = map.putIfAbsent(key, newEntry); >> ? 70???????????? if (entry == null) { >> ? 71???????????????? value = newVal; >> ? 72???????????? } else { >> ? 73???????????????? value = entry.get(); >> ? 74???????????????? if (value == null) { >> ? 75???????????????????? map.put(key, newEntry); >> ? 76???????????????????? value = newVal; >> ? 77???????????????? } >> ? 78???????????? } >> >> ...which can simply be fixed: >> >> ??????????? CacheEntry newEntry = new CacheEntry<>(key, newVal, >> queue); >> >> ??????????? while (true) { >> ??????????????? entry = map.putIfAbsent(key, newEntry); >> ??????????????? if (entry == null) { >> ??????????????????? value = newVal; >> ??????????????????? break; >> ??????????????? } else { >> ??????????????????? value = entry.get(); >> ??????????????????? if (value == null) { >> ??????????????????????? if (map.replace(key, entry, newEntry)) { >> ??????????????????????????? value = newVal; >> ??????????????????????????? break; >> ??????????????????????? } >> ??????????????????? } >> ??????????????? } >> ??????????? } >> >> >> Regards, Peter >> >> >> On 02/07/2018 11:26 AM, Claes Redestad wrote: >>> Hi Paul, >>> >>> >>> On 2018-02-06 20:55, Paul Sandoz wrote: >>>> Quick observation: >>>> >>>> ? 261???????? private BaseLocale getBaseLocale() { >>>> ? 262???????????? return (holder == null) ? holderRef.get() : holder; >>>> ? 263???????? } >>>> >>>> This method can return null if the soft ref has been cleared. >>>> >>>> >>>> But you don?t check in equals: >>>> >>>> ? 270???????????? if (obj instanceof Key && this.hash == >>>> ((Key)obj).hash) { >>>> ? 271???????????????? BaseLocale other = ((Key) obj).getBaseLocale(); >>>> ? 272???????????????? BaseLocale locale = this.getBaseLocale(); >>>> ? 273???????????????? if >>>> (LocaleUtils.caseIgnoreMatch(other.getLanguage(), >>>> locale.getLanguage()) >>> >>> good eye! >>> >>> It seems this wasn't caught by the existing regression tests since >>> none of them >>> recreate Locales in that are likely to have been reclaimed, but >>> still likely to still >>> be in the CHM (it's a race of sorts since they'll be removed when >>> the ReferenceQueue >>> processing happen). >>> >>> I added a regression test with the smallest and quickest reproducer >>> I could come up >>> with that provokes a NPE if we don't check null along with the fix >>> to Key#equals: >>> >>> http://cr.openjdk.java.net/~redestad/8196869/jdk.01/ >>> >>> For the normalize(Key) case we can deduce that a !normalized Key >>> will always have >>> a strongly referenced BaseLocale and thus not need to deal with >>> getBaseLocale() >>> returning null. I clarified this in the code and added an assert >>> (that would be triggered >>> by the added test if it wasn't true). >>> >>> Thanks! >>> >>> /Claes >> > From Alan.Bateman at oracle.com Wed Feb 7 13:03:18 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 7 Feb 2018 13:03:18 +0000 Subject: RFR 8196298 Add null Reader and Writer In-Reply-To: References: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> Message-ID: <36bf7199-d852-825a-62a9-35c9b0e49158@oracle.com> On 03/02/2018 17:05, Patrick Reinhart wrote: > : > I reworked the tests and Writer implementation accordingly > > http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.01 > Just catching up on this. nullReader's javadoc suggests that mark(int) does not nothing but this seems to conflict with the Reader's javadoc where it is specified to throw IOE when mark is not supported. I'm a bit uneasy with len==0 check in the read method. I realize this is trying to mirror InputStream.read and maybe some Reader implementations but it's not specified behavior. I think we have to re-examine the Reader spec to clarify this to avoid creating more inconsistencies. Similarly in nullWriter() where it looks like the the len==0 checks is in unspecified territory. I think this will need clarifications to the Writer spec. One obvious inconsistency is to close the writer and call write("") and write("",? 0, 0). The first will fail as the writer is closed, the second does not fail. Another one is read(CharBuffer). Suppose CharBuffer cb has 0 bytes remaining; if I invoke nullReader().read(cb) then it looks like it will return 0 whereas the read(CharBuffer) method is specified to return -1. -Alan From ben_walsh at uk.ibm.com Wed Feb 7 14:18:20 2018 From: ben_walsh at uk.ibm.com (Ben Walsh) Date: Wed, 7 Feb 2018 14:18:20 +0000 Subject: [PATCH] ObjectInputStream Reading Performance Optimisation Message-ID: As per the guidance here - http://mail.openjdk.java.net/pipermail/valhalla-dev/2018-February/003772.html , I am resubmitting this patch to this mailing list ... The main advantage of this optimisation is that a suitably "aware" compiler can avoid stack walking by using a private method named redirectedReadObject that can be called from a call site that was calling readObject originally (in some cases). The conditions when this can be done are essentially when we know that the caller method's class is a "user defined class" because we have that caller method/class at compile time and can check if it was loaded by a user defined class loader. If so, the compiler emits code to call the private redirectedReadObject method (instead of readObject that was there in the byte codes) and it avoids doing a stack walk to discover the latest user class loader because it has an extra parameter that is a class loader that can be passed in, in cases when the compiler knows the answer at compile time. In such cases, the compiler generates code to pass in the latest user defined class loader known to redirectedReadObject. I would like to pair with a sponsor to contribute this patch ... ----------------------------------------------------------------------------------------------------------------------- diff -r 60c19c384333 src/java.base/share/classes/java/io/ClassCache.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/java.base/share/classes/java/io/ClassCache.java Fri Feb 02 11:52:41 2018 +0000 @@ -0,0 +1,403 @@ +/* + * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Oracle designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +package java.io; + +import java.lang.ref.ReferenceQueue; +import java.lang.ref.WeakReference; +import java.security.AccessController; +import java.security.PrivilegedAction; +import java.util.Objects; +import java.util.concurrent.ConcurrentHashMap; +/* ClassCache is Primarily responsible for Caching the results of the className lookups and hence to avoid + * multiple Lookup for same Class Instance. + * ClassCache provides a ConcurrentHash based ClassCache which is looked up prior to calling the class.forName + * Method resolveClass() from ObjectInputStream uses this Cache. + * + * Caching is done only when the actually used loader for a Class is one of the Sytem loaders (ie) App Class Loader, + * System Extension loader and BootStrap loader + * + */ +final class ClassCache { +/* Main Cache for storing the Class.forName results Here Key used would be CacheKey */ + private final ConcurrentHashMap cache = + new ConcurrentHashMap(); +/* Initiating Loader to CacheKey mapping in the Cache used by Reaper thread for removal on stale Loaders */ + private final ConcurrentHashMap loaderKeys = + new ConcurrentHashMap(); + /* Keeps a link of Actual System Loader used to Initiating Loader mapping */ + private final ConcurrentHashMap canonicalLoaderRefs = + new ConcurrentHashMap(); + /* Reference Queue registered for notification on stale Loaders */ + private final ReferenceQueue staleLoaderRefs = + new ReferenceQueue(); +/* + * Constructor Populates Canonical Loader Refs with System Loader Entries and initializes the reaper thread which + * monitors the ReferenceQueue for stale loaders + */ + + public ClassCache() { + ClassLoader loader = ClassLoader.getSystemClassLoader(); + while (loader != null) { + setCanonicalSystemLoaderRef(loader); + loader = loader.getParent(); + } + setCanonicalSystemLoaderRef(null); + AccessController.doPrivileged( + new CreateReaperAction(this, staleLoaderRefs)).start(); + } + /* + * sets Canonical Loader reference for the loader + */ + + private void setCanonicalSystemLoaderRef(ClassLoader loader) { + LoaderRef newKey = new LoaderRef(loader, staleLoaderRefs, true); + assert (canonicalLoaderRefs.put(newKey, newKey) == null); + } + + /* + * get Canonical Loader reference for the loader + */ + + + LoaderRef getCanonicalLoaderRef(Object loaderObj) { + LoaderRef newKey = new LoaderRef(loaderObj, staleLoaderRefs); + + LoaderRef key = canonicalLoaderRefs.get(newKey); + if (key == null) { + key = canonicalLoaderRefs.putIfAbsent(newKey, newKey); + if (key == null) { + return newKey; + } + } + + newKey.clear(); + return key; + } +/* + * Remove unused LoaderKey, and initiates corresponding CacheKey entry the main Cache + */ + void removeStaleRef(LoaderRef loaderRef) { + canonicalLoaderRefs.remove(loaderRef); + CacheKey key = loaderKeys.remove(loaderRef); + while (key != null) { + cache.remove(key); + key = key.next; + } + } + +/* + * Identifies if the loader used to load is one of the system loaders, + * if so updates the cache and the LoaderKey, and also a StaleLoaderReference for the initiating LoaderObject + * via LoaderRef Constructor + */ + void update(CacheKey key, Class result) { + Object resultLoaderObj = + LoaderRef.getLoaderObj(result.getClassLoader()); + if (getCanonicalLoaderRef(resultLoaderObj).isSystem == false) { + return; + } + + Object oldValue = cache.replace(key, result); + assert (oldValue instanceof FutureValue) : + ("Value replaced is of type '" + oldValue.getClass().getName() + + "', not of type '" + FutureValue.class.getName() + "'."); + + LoaderRef loaderRef = key.loaderRef; + if (loaderRef.isSystem == false) { + key.next = loaderKeys.get(loaderRef); + if (key.next == null) { + key.next = loaderKeys.putIfAbsent(loaderRef, key); + if (key.next == null) return; + } + while (!loaderKeys.replace(loaderRef, key.next, key)) { + key.next = loaderKeys.get(loaderRef); + } + } + } +/* + * Creates a New Entry in the Cache + */ + private Object createEntry(CacheKey key) { + FutureValue newValue = new FutureValue(key, this); //Does actual call to class.forName as required. + Object value = cache.putIfAbsent(key, newValue); + if (value == null) value = newValue; + return value; + } +/* + * This is the entry point in to the cache from ObjectInputStream. First Lookup is done based on the className and Loader + */ + public Class get(String className, ClassLoader loader) + throws ClassNotFoundException { + LookupKey key = new LookupKey(className, loader, this); + Object value = cache.get(key); + if (value == null) { + value = createEntry(key.createCacheKey()); + } + + if (value instanceof FutureValue) { + + return ((FutureValue)value).get(); + } + + return (Class)value; + } + + /* + * FutureValue implements Future Mechanics that is required for addressing contention issues in the HashMap + */ + + private static final class FutureValue { + private final CacheKey key; + private final LoaderRef loaderRef; + private final ClassCache cache; + private Class value = null; + + FutureValue(CacheKey key, ClassCache cache) { + this.key = key; + this.loaderRef = key.loaderRef; + this.cache = cache; + } + + /* + * tries to get the value from Cache, if not available tries to do get class.forName with active loader + * and replace the entry in the cache as required + */ + Class get() throws ClassNotFoundException { + synchronized (this) { + if (value != null) return value; + value = Class.forName(key.className, false, + loaderRef.getActiveLoader()); + } + if (value != null) { + cache.update(key, value); + } + + return value; + } + } + + private static final class CreateReaperAction + implements PrivilegedAction { + private final ClassCache cache; + private final ReferenceQueue queue; + + CreateReaperAction(ClassCache cache, ReferenceQueue queue) { + this.cache = cache; + this.queue = queue; + } + + public Thread run() { + return new Reaper(cache, queue); + } + } + + private static final class Reaper extends Thread { + private final WeakReference cacheRef; + private final ReferenceQueue queue; + + Reaper(ClassCache cache, ReferenceQueue queue) { + super("ClassCache Reaper"); + this.queue = queue; + cacheRef = new WeakReference(cache, queue); + setDaemon(true); + setContextClassLoader(null); + } +/* + * Blocks on remove() on queur reference and calls processStaleRef() when any loader is removed.(non-Javadoc) + * @see java.lang.Thread#run() + */ + public void run() { + Object staleRef = null; + do { + try { + staleRef = queue.remove(); + if (staleRef == cacheRef) break; + + processStaleRef((LoaderRef)staleRef); + } catch (InterruptedException e) { } + } while (true); + } + + private void processStaleRef(LoaderRef staleRef) { + ClassCache cache = cacheRef.get(); + if (cache == null) return; + + cache.removeStaleRef(staleRef); + } + } + + /* + * The use of the loaderRefs map is to allow efficient processing + * of one Weak Reference for each stale ClassLoader, rather than one WR for each entry in the cache. + * + * CacheKey as well as Lookup Key will be refering to this LoaderRef. + * + * Initiating Class Loaders needs to be referred for both lookup and Caching,so for performance reasons + * a LoaderRef is maintained which would be used by both LookupKey and CachingKey + * (ie) LookupKey will actually store the LoaderObj, but it will be canonically refered via a loaderRed + * by the caching Key + * LoaderKey has LoaderRef Objects as well and is used to Link the Initiating Loader with the actual cache Entries + * which is used to remove Stale reference entries. + */ + + private static class LoaderRef extends WeakReference { + private static final String NULL_LOADER = new String(""); + private final int hashcode; + public final boolean isSystem; + + static Object getLoaderObj(ClassLoader loader) { + return ((loader == null) ? NULL_LOADER : loader); + } + + LoaderRef(Object loaderObj, ReferenceQueue queue) { + this(false, Objects.requireNonNull(loaderObj), queue); + } + + LoaderRef(ClassLoader loader, ReferenceQueue queue, + boolean isSystem) { + this(isSystem, getLoaderObj(loader), queue); + } + + /* + * Creates a new weak reference that refers to the given object and is registered with the given queue. + */ + + private LoaderRef(boolean isSystem, Object loaderObj, + ReferenceQueue queue) { + super(loaderObj, queue); + String loaderClassName = ((loaderObj == NULL_LOADER) ? + NULL_LOADER : loaderObj.getClass().getName()); + hashcode = (loaderClassName.hashCode() + + System.identityHashCode(loaderObj)); + this.isSystem = isSystem; + } + + public final boolean equals(Object o) { + if (this == o) return true; + if (!(o instanceof LoaderRef)) return false; + + Object loader = get(); + return ((loader != null) && (loader == ((LoaderRef)o).get())); + } + + public final int hashCode() { + return hashcode; + } + + ClassLoader getActiveLoader() { + Object loaderObj = Objects.requireNonNull(get()); + return ((loaderObj == NULL_LOADER) ? null : (ClassLoader)loaderObj); + } + } + /* + * For better clarity and to avoid multiple lookups to the cache. Key is implemented to have + * one abstract key to final sub classes which serve specific purpose + * LookupKey - This is a short lived key, not part of any hashmap and stores the strong reference to + * loaderobject + * CachingKey - uses the same hash as LookupKey and has a means to be generated from LookupKey and has reference + * to the Loaderobj via a weakreference. + */ + + private static abstract class Key { + public final String className; + protected final int hashcode; + + protected Key(String className, int hashcode) { + this.className = className; + this.hashcode = hashcode; + } + + abstract Object getLoaderObj(); + + public final boolean equals(Object o) { + if (this == o) return true; + if (!(o instanceof Key)) return false; + + Key k = (Key)o; + Object loaderObj = getLoaderObj(); + return (className.equals(k.className) && + (loaderObj != null) && (loaderObj == k.getLoaderObj())); + } + + public final int hashCode() { + return hashcode; + } + } + /* + * Lookup Key hash code is framed using loadername hash + loader's system identity hashcode. This + * is same as the hashcode maintained in CacheKey + */ + + private static final class LookupKey extends Key { + private final Object loaderObj; + private final ClassCache cache; + + private static int hashCode(String className, ClassLoader loader) { + int hashcode = className.hashCode(); + if (loader != null) { + hashcode += (loader.getClass().getName().hashCode() + + System.identityHashCode(loader)); + } + return hashcode; + } + + public LookupKey(String className, ClassLoader loader, + ClassCache cache) { + super(Objects.requireNonNull(className), + hashCode(className, loader)); + loaderObj = LoaderRef.getLoaderObj(loader); + this.cache = cache; + } + + Object getLoaderObj() { + return loaderObj; + } + + CacheKey createCacheKey() { + return new CacheKey(className, hashcode, + cache.getCanonicalLoaderRef(loaderObj)); + } + } + + /* + * CacheKey is the actual key that is stored in the cache, and it stores the weakreference + * of the Initiating loader object via loaderRef + * + */ + private static final class CacheKey extends Key { + public final LoaderRef loaderRef; + public CacheKey next = null; + + CacheKey(String className, int hashcode, LoaderRef loaderRef) { + super(className, hashcode); + this.loaderRef = loaderRef; + } + + Object getLoaderObj() { + return loaderRef.get(); + } + } +} diff -r 60c19c384333 src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java Wed Jan 31 15:15:09 2018 -0800 +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java Fri Feb 02 11:52:41 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1996, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1996, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -308,6 +308,32 @@ private ObjectInputFilter serialFilter; /** + * cache LUDCL (Latest User Defined Class Loader) till completion of + * read* requests + */ + + /* ClassCache Entry for caching class.forName results upon enableClassCaching */ + private static final ClassCache classCache; + private static final boolean isClassCachingEnabled; + static { + isClassCachingEnabled = + AccessController.doPrivileged(new GetClassCachingSettingAction()); + classCache = (isClassCachingEnabled ? new ClassCache() : null); + } + + + /** if true LUDCL/forName results would be cached, true by default */ + private static final class GetClassCachingSettingAction + implements PrivilegedAction { + public Boolean run() { + String property = + System.getProperty("openjdk.enableClassCaching", "true"); + return property.equalsIgnoreCase("true"); + } + } + private ClassLoader cachedLudcl; + + /** * Creates an ObjectInputStream that reads from the specified InputStream. * A serialization stream header is read from the stream and verified. * This constructor will block until the corresponding ObjectOutputStream @@ -412,10 +438,64 @@ public final Object readObject() throws IOException, ClassNotFoundException { + return readObjectImpl(null); + } + + /** + * Whenever jit compiler encounters processing readObject() method + * it will replace the call with redirectedReadObject() method to improve + * the performance for custom serialisation. JIT provide the class loader through + * the caller parameter to avoid the stack walk through while calling + * latestUserDefinedLoader(). + * + * @throws ClassNotFoundException if the class of a serialized object + * could not be found. + * @throws IOException if an I/O error occurs. + * + */ + + private static Object redirectedReadObject(ObjectInputStream iStream, Class caller) + throws ClassNotFoundException, IOException + { + return iStream.readObjectImpl(caller); + } + + /** + * Actual implementation of readObject method which fetches classloader using + * latestUserDefinedLoader() method if caller is null. If caller is not null which means + * jit passes the class loader info and hence avoids calling latestUserDefinedLoader() + * method call to improve the performance for custom serialisation. + * + * @throws ClassNotFoundException if the class of a serialized object + * could not be found. + * @throws IOException if an I/O error occurs. + */ + private Object readObjectImpl(Class caller) + throws ClassNotFoundException, IOException + { + if (enableOverride) { return readObjectOverride(); } + ClassLoader oldCachedLudcl = null; + boolean setCached = false; + + if ((curContext == null) && (isClassCachingEnabled)) { + oldCachedLudcl = cachedLudcl; + + // If caller is not provided, follow the standard path to get the cachedLudcl. + // Otherwise use the class loader provided by JIT as the cachedLudcl. + + if (caller == null) { + cachedLudcl = latestUserDefinedLoader(); + } else { + cachedLudcl = caller.getClassLoader(); + } + + setCached = true; + } + // if nested read, passHandle contains handle of enclosing object int outerHandle = passHandle; try { @@ -432,6 +512,9 @@ return obj; } finally { passHandle = outerHandle; + if (setCached) { + cachedLudcl = oldCachedLudcl; + } if (closed && depth == 0) { clear(); } @@ -511,6 +594,16 @@ * @since 1.4 */ public Object readUnshared() throws IOException, ClassNotFoundException { + + ClassLoader oldCachedLudcl = null; + boolean setCached = false; + + if ((curContext == null) && (isClassCachingEnabled)) { + oldCachedLudcl = cachedLudcl; + cachedLudcl = latestUserDefinedLoader(); + setCached = true; + } + // if nested read, passHandle contains handle of enclosing object int outerHandle = passHandle; try { @@ -527,6 +620,9 @@ return obj; } finally { passHandle = outerHandle; + if (setCached) { + cachedLudcl = oldCachedLudcl; + } if (closed && depth == 0) { clear(); } @@ -682,7 +778,10 @@ { String name = desc.getName(); try { - return Class.forName(name, false, latestUserDefinedLoader()); + return ((classCache == null) ? + Class.forName(name, false, latestUserDefinedLoader()) : + classCache.get(name, cachedLudcl)); + } catch (ClassNotFoundException ex) { Class cl = primClasses.get(name); if (cl != null) { ----------------------------------------------------------------------------------------------------------------------- Regards, Ben Walsh Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From claes.redestad at oracle.com Wed Feb 7 14:23:36 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 7 Feb 2018 15:23:36 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: <55e2d730-3773-ad52-64ef-5de8ae5f43db@gmail.com> References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> <82ead3ee-63a8-854c-429e-98ef05ccddf1@oracle.com> <55e2d730-3773-ad52-64ef-5de8ae5f43db@gmail.com> Message-ID: Hi, On 2018-02-07 13:55, Peter Levart wrote: > > Would making CacheEntry extend jdk.internal.ref.SoftCleanable instead > of SoftReference help here? You could remove the cleanStaleEntries > method entirely and just remove the Map entry in SoftCleanable's > performCleanup method. possible, but that'd be a larger change than I'm comfortable with for now. As Locale is initialized on bootstrap, a Cleaner-based impl. would mean starting an innocuous thread unconditionally, which would defeat the intent to optimize the minimal time to bootstrap the JVM. If we could tease things apart even further so that a SoftCleanable and Cleaners are only set up when initializing any non-constant Locale then I think we should contemplate this as a follow up. Updated webrev: http://cr.openjdk.java.net/~redestad/8196869/jdk.02/ - use map.remove(entry.getKey(), entry) instead of map.remove(entry.getKey()) - for most Locales, Locale$LocaleKey.exts is null, so using the BaseLocale as key ? directly in Locale allows us to avoid loading Locale$LocaleKey except in exceptional ? circumstances. - use map.replace to safely update the entry when putIfAbsent returns an object ? but it points to a cleared value, so that (benign) races to create new Locale objects ? will canonicalize /Claes From peter.levart at gmail.com Wed Feb 7 15:01:40 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 7 Feb 2018 16:01:40 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> <82ead3ee-63a8-854c-429e-98ef05ccddf1@oracle.com> <55e2d730-3773-ad52-64ef-5de8ae5f43db@gmail.com> Message-ID: <11a70d82-9983-9bfe-06db-4a2ad6494ed2@gmail.com> Hi Claes, On 02/07/2018 03:23 PM, Claes Redestad wrote: > Updated webrev: > > http://cr.openjdk.java.net/~redestad/8196869/jdk.02/ > > - use map.remove(entry.getKey(), entry) instead of > map.remove(entry.getKey()) > - for most Locales, Locale$LocaleKey.exts is null, so using the > BaseLocale as key > ? directly in Locale allows us to avoid loading Locale$LocaleKey > except in exceptional > ? circumstances. > - use map.replace to safely update the entry when putIfAbsent returns > an object > ? but it points to a cleared value, so that (benign) races to create > new Locale objects > ? will canonicalize I don't know it it's only me, but I see an old version of LocaleObjectCache in webrev at above URL. Regards, Peter From claes.redestad at oracle.com Wed Feb 7 15:12:25 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 7 Feb 2018 16:12:25 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: <11a70d82-9983-9bfe-06db-4a2ad6494ed2@gmail.com> References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> <82ead3ee-63a8-854c-429e-98ef05ccddf1@oracle.com> <55e2d730-3773-ad52-64ef-5de8ae5f43db@gmail.com> <11a70d82-9983-9bfe-06db-4a2ad6494ed2@gmail.com> Message-ID: On 2018-02-07 16:01, Peter Levart wrote: > Hi Claes, > > On 02/07/2018 03:23 PM, Claes Redestad wrote: >> Updated webrev: >> >> http://cr.openjdk.java.net/~redestad/8196869/jdk.02/ >> >> - use map.remove(entry.getKey(), entry) instead of >> map.remove(entry.getKey()) >> - for most Locales, Locale$LocaleKey.exts is null, so using the >> BaseLocale as key >> ? directly in Locale allows us to avoid loading Locale$LocaleKey >> except in exceptional >> ? circumstances. >> - use map.replace to safely update the entry when putIfAbsent returns >> an object >> ? but it points to a cleared value, so that (benign) races to create >> new Locale objects >> ? will canonicalize > > I don't know it it's only me, but I see an old version of > LocaleObjectCache in webrev at above URL. Sorry about that - the intended webrev have now replace the old on in-place. /Claes From robin.westberg at oracle.com Wed Feb 7 15:18:05 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Wed, 7 Feb 2018 16:18:05 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason Message-ID: Hi all, Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 Best regards, Robin From vladimir.x.ivanov at oracle.com Wed Feb 7 15:22:35 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 7 Feb 2018 18:22:35 +0300 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: <4ef38d5d-d349-4cb6-e7b3-019f094cc314@oracle.com> References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> <604f59a2-a3f2-5c3f-067b-2002665b9441@gmail.com> <4ef38d5d-d349-4cb6-e7b3-019f094cc314@oracle.com> Message-ID: <1347b5ac-13c8-88a2-4e98-8b8735d410a0@oracle.com> Peter, > >> Objects.requireNonNull() shows zero overhead here. >> >> I guess the main question is whether Objects.requireNonNull(this) >> behavior in the former test is a result of chance and current Hotspot >> behavior or is it somehow guaranteed by the spec. > > I haven't looked into what actually happens in JIT-compilers on your > benchmark, but I'm surprised it works at all. So, here's why Objects.requireNonNull() keeps the receiver alive in your test case. JIT-compilers in HotSpot aggressively prune dead locals [1], but they do that based on method bytecode analysis [2] (and not on optimized IR). So, any usage of a local extends its live range, even if that usage is eliminated in generated code. It means an oop in that local will live past its last usage in generated code and all safepoints (in generated code) till last usage (on bytecode level) will enumerate the local it is held in. If there were GC-only safepoints supported, then JITs could still prune unused locals from oop maps, but HotSpot doesn't have them and all safepoints in generated code keep full JVM state, so it's always possible to deoptimize at any one of them (and then run into the code which is eliminated in generated code). If there are no safepoints till the end of the method, then nothing will keep the object alive. But there's no way for GC to collect it, since GCs rely on safepoints to mark thread stack. (That's why I mentioned GC-only safepoints earlier.) As a conclusion: removing @DontInline on Reference.reachabilityFence() should eliminate most of the overhead (no call anymore, additional spill may be needed) and still keep it working. It's not guaranteed by JVMS, but at least should work on HotSpot JVM (in its current state). So, nice discovery, Peter! :-) Want to file an RFE & fix it? Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/jdk/hs/file/45b6aae769cc/src/hotspot/share/opto/graphKit.cpp#l736 [2] http://hg.openjdk.java.net/jdk/hs/file/45b6aae769cc/src/hotspot/share/compiler/methodLiveness.cpp#l37 > Explicit null check on the receiver is an easy target for elimination > and should be effectively a no-op in generated code. (And that's what > you observe with the benchmark!) Once the check is gone, nothing keeps > receiver alive anymore (past the last usage). > > So, I'd say such behavior it's a matter of chance in your case and can't > be relied on in general. And definitely not something guaranteed by JVMS. > > Best regards, > Vladimir Ivanov From Alan.Bateman at oracle.com Wed Feb 7 15:30:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 7 Feb 2018 15:30:11 +0000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: Message-ID: <22b0955b-e1dc-a3b7-ab1b-8e4f0ec49670@oracle.com> On 07/02/2018 15:18, Robin Westberg wrote: > Hi all, > > Please review the following change that adds an event-based tracing > event that is generated when the VM shuts down. The intent is to > capture shutdowns that occur after the VM has been properly > initialized (as initialization problems would most likely mean that > the tracing framework hasn?t been properly started either). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 > Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ > > Testing:?hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 > Can you elaborate a bit on why this isn't in JVM_Halt? Is this partially to help with cases where the shutdown hooks or finalizers run at exit cause issues? -Alan From paul.sandoz at oracle.com Wed Feb 7 16:31:38 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 7 Feb 2018 08:31:38 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: <1347b5ac-13c8-88a2-4e98-8b8735d410a0@oracle.com> References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> <604f59a2-a3f2-5c3f-067b-2002665b9441@gmail.com> <4ef38d5d-d349-4cb6-e7b3-019f094cc314@oracle.com> <1347b5ac-13c8-88a2-4e98-8b8735d410a0@oracle.com> Message-ID: <9826D581-E1F9-47E9-BD18-A9CD7D9E5AFD@oracle.com> > On Feb 7, 2018, at 7:22 AM, Vladimir Ivanov wrote: > > Peter, > >>> Objects.requireNonNull() shows zero overhead here. >>> >>> I guess the main question is whether Objects.requireNonNull(this) behavior in the former test is a result of chance and current Hotspot behavior or is it somehow guaranteed by the spec. >> I haven't looked into what actually happens in JIT-compilers on your benchmark, but I'm surprised it works at all. > > So, here's why Objects.requireNonNull() keeps the receiver alive in your test case. > > JIT-compilers in HotSpot aggressively prune dead locals [1], but they do that based on method bytecode analysis [2] (and not on optimized IR). So, any usage of a local extends its live range, even if that usage is eliminated in generated code. It means an oop in that local will live past its last usage in generated code and all safepoints (in generated code) till last usage (on bytecode level) will enumerate the local it is held in. > > If there were GC-only safepoints supported, then JITs could still prune unused locals from oop maps, but HotSpot doesn't have them and all safepoints in generated code keep full JVM state, so it's always possible to deoptimize at any one of them (and then run into the code which is eliminated in generated code). > > If there are no safepoints till the end of the method, then nothing will keep the object alive. But there's no way for GC to collect it, since GCs rely on safepoints to mark thread stack. (That's why I mentioned GC-only safepoints earlier.) > > As a conclusion: removing @DontInline on Reference.reachabilityFence() should eliminate most of the overhead (no call anymore, additional spill may be needed) and still keep it working. It's not guaranteed by JVMS, but at least should work on HotSpot JVM (in its current state). > > So, nice discovery, Peter! :-) Yes, Peter, thank you, for pushing on this. We would of course need to be careful and revisit if any changes occur. Vladimir, just to be sure I presume your analysis applies to both C1 and C2? what about compilers such as Graal? Ben, i still think additional performance analysis is still valuable (such performance tests are also useful for another reason, consolidating unsafe accesses using the double addressing mode, thereby removing another difference between heap and direct buffers). Paul. > Want to file an RFE & fix it? > > Best regards, > Vladimir Ivanov > > [1] http://hg.openjdk.java.net/jdk/hs/file/45b6aae769cc/src/hotspot/share/opto/graphKit.cpp#l736 > > [2] http://hg.openjdk.java.net/jdk/hs/file/45b6aae769cc/src/hotspot/share/compiler/methodLiveness.cpp#l37 > >> Explicit null check on the receiver is an easy target for elimination and should be effectively a no-op in generated code. (And that's what you observe with the benchmark!) Once the check is gone, nothing keeps receiver alive anymore (past the last usage). >> So, I'd say such behavior it's a matter of chance in your case and can't be relied on in general. And definitely not something guaranteed by JVMS. >> Best regards, >> Vladimir Ivanov From vladimir.x.ivanov at oracle.com Wed Feb 7 16:49:40 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 7 Feb 2018 19:49:40 +0300 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: <9826D581-E1F9-47E9-BD18-A9CD7D9E5AFD@oracle.com> References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> <604f59a2-a3f2-5c3f-067b-2002665b9441@gmail.com> <4ef38d5d-d349-4cb6-e7b3-019f094cc314@oracle.com> <1347b5ac-13c8-88a2-4e98-8b8735d410a0@oracle.com> <9826D581-E1F9-47E9-BD18-A9CD7D9E5AFD@oracle.com> Message-ID: <820b59d6-3e12-0c74-88b2-c3d1d6362ae7@oracle.com> > Vladimir, just to be sure I presume your analysis applies to both C1 and C2? what about compilers such as Graal? I only looked at C1 & C2, but I'm sure it applies to Graal as well: GC interaction mechanism is the same for all JIT-compilers in HotSpot. Best regards, Vladimir Ivanov From lance.andersen at oracle.com Wed Feb 7 16:57:22 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 7 Feb 2018 11:57:22 -0500 Subject: RFR 8190378: Java EE and CORBA modules removal Message-ID: Hi all, I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ To make the open review easier, I have broken the changes into 5 webrevs: build changes are: http://cr.openjdk.java.net/~lancea/8190378/open_changes/build_webrev/ miscellaneous changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/misc_webrev/ module changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/modules_webrev/ rmic changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/rmic_webrev/ test changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/tests_webrev/ As part of the removal, the following issues have also been logged: Removal of the Java EE and CORBA tools from the documentation: https://bugs.openjdk.java.net/browse/JDK-8193906 Updating the RMIC man pages for the removal of the -iiop and -idl options: https://bugs.openjdk.java.net/browse/JDK-8196510 Hotspot tests may require further updating or just removed: https://bugs.openjdk.java.net/browse/JDK-8194310 jdeprescan will need updates due to the removal of the Java EE and CORBA modules: https://bugs.openjdk.java.net/browse/JDK-8194308 Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From erik.joelsson at oracle.com Wed Feb 7 17:11:24 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 7 Feb 2018 09:11:24 -0800 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: References: Message-ID: Build changes look good. /Erik On 2018-02-07 08:57, Lance Andersen wrote: > Hi all, > > I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. > The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 > > The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ > > To make the open review easier, I have broken the changes into 5 webrevs: > build changes are: http://cr.openjdk.java.net/~lancea/8190378/open_changes/build_webrev/ > miscellaneous changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/misc_webrev/ > module changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/modules_webrev/ > rmic changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/rmic_webrev/ > test changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/tests_webrev/ > > As part of the removal, the following issues have also been logged: > Removal of the Java EE and CORBA tools from the documentation: https://bugs.openjdk.java.net/browse/JDK-8193906 > Updating the RMIC man pages for the removal of the -iiop and -idl options: https://bugs.openjdk.java.net/browse/JDK-8196510 > Hotspot tests may require further updating or just removed: https://bugs.openjdk.java.net/browse/JDK-8194310 > jdeprescan will need updates due to the removal of the Java EE and CORBA modules: https://bugs.openjdk.java.net/browse/JDK-8194308 > > > > Best, > Lance > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From patrick at reini.net Wed Feb 7 17:38:06 2018 From: patrick at reini.net (Patrick Reinhart) Date: Wed, 7 Feb 2018 18:38:06 +0100 Subject: RFR 8196298 Add null Reader and Writer In-Reply-To: <36bf7199-d852-825a-62a9-35c9b0e49158@oracle.com> References: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> <36bf7199-d852-825a-62a9-35c9b0e49158@oracle.com> Message-ID: <9FD69071-578D-4BA3-AAEC-40BBCFA4B737@reini.net> > Am 07.02.2018 um 14:03 schrieb Alan Bateman : > > On 03/02/2018 17:05, Patrick Reinhart wrote: >> : >> I reworked the tests and Writer implementation accordingly >> >> http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.01 >> > Just catching up on this. > > nullReader?s javadoc suggests that mark(int) does not nothing but this seems to conflict with the Reader's javadoc where it is specified to throw IOE when mark is not supported. So I will change the API description accordingly there? > > I'm a bit uneasy with len==0 check in the read method. I realize this is trying to mirror InputStream.read and maybe some Reader implementations but it's not specified behavior. I think we have to re-examine the Reader spec to clarify this to avoid creating more inconsistencies. That?s exactly what I did, so for me it?s ok to look into the Reader spec also as I already modify that File. I personally thought that a null like reader will always be at the end of the stream and therefore return -1 and do no checking of how much space may be left on the consuming end. That seem to me more the natural way of thinking. > > Similarly in nullWriter() where it looks like the the len==0 checks is in unspecified territory. I think this will need clarifications to the Writer spec. One obvious inconsistency is to close the writer and call write("") and write("", 0, 0). The first will fail as the writer is closed, the second does not fail. Here I think the same holds true as I wrote for the Reader? > > Another one is read(CharBuffer). Suppose CharBuffer cb has 0 bytes remaining; if I invoke nullReader().read(cb) then it looks like it will return 0 whereas the read(CharBuffer) method is specified to return -1. > I see your point there, the implementation I did there was taken from the StringReader, where it will be the same len==0 check as you mentioned before? So it looks like we really need to look into those implementations? -Patrick From sean.coffey at oracle.com Wed Feb 7 18:03:50 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 7 Feb 2018 18:03:50 +0000 Subject: RFR : 8196854 :TestFlushableGZIPOutputStream failing with IndexOutOfBoundsException Message-ID: <6da57541-0fc1-6359-315c-6d06e15fa842@oracle.com> A jdk8u (and earlier) issue where some new/recent test code meant that IndexOutOfBoundsException could be thrown. https://bugs.openjdk.java.net/browse/JDK-8196854 http://cr.openjdk.java.net/~coffeys/webrev.8196854/webrev/ The buf array was never required and I've verified that the original 8189789 issue can still be reproduced with a buggy JDK. -- Regards, Sean. From goetz.lindenmaier at sap.com Wed Feb 7 20:20:53 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 Feb 2018 20:20:53 +0000 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: References: Message-ID: <7811bd0dc275498a8cc3417af961bdfc@sap.com> Hi Lance, Would you mind to add similar change as for test/jdk/tools/launcher/VersionCheck.java also for test/jdk/tools/launcher/HelpFlagsTest.java That test is pretty recent which is why you may have missed it. It will not fail though, there will just be left over stuff in it. Best regards, Goetz. -----Original Message----- From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Lance Andersen Sent: Wednesday, February 7, 2018 5:57 PM To: core-libs-dev ; build-dev Subject: RFR 8190378: Java EE and CORBA modules removal Hi all, I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ To make the open review easier, I have broken the changes into 5 webrevs: build changes are: http://cr.openjdk.java.net/~lancea/8190378/open_changes/build_webrev/ miscellaneous changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/misc_webrev/ module changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/modules_webrev/ rmic changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/rmic_webrev/ test changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/tests_webrev/ As part of the removal, the following issues have also been logged: Removal of the Java EE and CORBA tools from the documentation: https://bugs.openjdk.java.net/browse/JDK-8193906 Updating the RMIC man pages for the removal of the -iiop and -idl options: https://bugs.openjdk.java.net/browse/JDK-8196510 Hotspot tests may require further updating or just removed: https://bugs.openjdk.java.net/browse/JDK-8194310 jdeprescan will need updates due to the removal of the Java EE and CORBA modules: https://bugs.openjdk.java.net/browse/JDK-8194308 Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From erik.gahlin at oracle.com Wed Feb 7 20:47:29 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 7 Feb 2018 21:47:29 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: Message-ID: <5A7B65E1.5050003@oracle.com> Hi Robin, I can sponsor this. I wonder if we should change the name of the event to "Shutdown" instead? It will give us flexibility to add other shutdown related fields in the future. Could you change the label and field to "Status Code" and statusCode. Event fields should have headline-style capitalization and statusCode allows us to add other status related information in the future. Thanks Erik > Hi all, > > Please review the following change that adds an event-based tracing > event that is generated when the VM shuts down. The intent is to > capture shutdowns that occur after the VM has been properly > initialized (as initialization problems would most likely mean that > the tracing framework hasn?t been properly started either). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 > Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ > > Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 > > Best regards, > Robin From robin.westberg at oracle.com Wed Feb 7 20:52:13 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Wed, 7 Feb 2018 21:52:13 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <22b0955b-e1dc-a3b7-ab1b-8e4f0ec49670@oracle.com> References: <22b0955b-e1dc-a3b7-ab1b-8e4f0ec49670@oracle.com> Message-ID: <9F1116D1-F61C-47AA-8137-21BDCDA84ADC@oracle.com> Hi Alan, > On 7 Feb 2018, at 16:30, Alan Bateman wrote: > > On 07/02/2018 15:18, Robin Westberg wrote: >> Hi all, >> >> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >> > Can you elaborate a bit on why this isn't in JVM_Halt? Is this partially to help with cases where the shutdown hooks or finalizers run at exit cause issues? Sure, the main problem I had with that approach is actually that the tracing framework shuts down from a shutdown hook, so events generated in JVM_Halt will be lost. And I couldn?t think of any good way to capture the exit code and proper stack trace from inside that shutdown hook, but I could perhaps explore that option a bit further. Best regards, Robin From lance.andersen at oracle.com Wed Feb 7 21:07:37 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 7 Feb 2018 16:07:37 -0500 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: <7811bd0dc275498a8cc3417af961bdfc@sap.com> References: <7811bd0dc275498a8cc3417af961bdfc@sap.com> Message-ID: Hi Goetz, I updated the webrev with the suggested update http://cr.openjdk.java.net/~lancea/8190378/open_changes/tests_webrev/test/jdk/tools/launcher/HelpFlagsTest.java.sdiff.html Best Lance > On Feb 7, 2018, at 3:20 PM, Lindenmaier, Goetz wrote: > > Hi Lance, > > Would you mind to add similar change as for > test/jdk/tools/launcher/VersionCheck.java > also for > test/jdk/tools/launcher/HelpFlagsTest.java > > That test is pretty recent which is why you may have missed it. > It will not fail though, there will just be left over stuff in it. > > Best regards, > Goetz. > > > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Lance Andersen > Sent: Wednesday, February 7, 2018 5:57 PM > To: core-libs-dev ; build-dev > Subject: RFR 8190378: Java EE and CORBA modules removal > > Hi all, > > I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. > The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 > > The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ > > To make the open review easier, I have broken the changes into 5 webrevs: > build changes are: http://cr.openjdk.java.net/~lancea/8190378/open_changes/build_webrev/ > miscellaneous changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/misc_webrev/ > module changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/modules_webrev/ > rmic changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/rmic_webrev/ > test changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/tests_webrev/ > > As part of the removal, the following issues have also been logged: > Removal of the Java EE and CORBA tools from the documentation: https://bugs.openjdk.java.net/browse/JDK-8193906 > Updating the RMIC man pages for the removal of the -iiop and -idl options: https://bugs.openjdk.java.net/browse/JDK-8196510 > Hotspot tests may require further updating or just removed: https://bugs.openjdk.java.net/browse/JDK-8194310 > jdeprescan will need updates due to the removal of the Java EE and CORBA modules: https://bugs.openjdk.java.net/browse/JDK-8194308 > > > > Best, > Lance > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From mandy.chung at oracle.com Wed Feb 7 22:00:54 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 7 Feb 2018 14:00:54 -0800 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: References: Message-ID: <640e4b80-9083-809a-2a90-798f9d848d2d@oracle.com> Hi Lance, Great to see this JEP moving along. I reviewed all changes except test/langtools/tools/javac tests. Looks fine overall. Minor comments: src/java.base/share/lib/security/default.policy - no change in this file. test/jdk/tools/jmod/hashes/HashesTest.java test/jdk/tools/launcher/modules/addexports/AddExportsTest.java - I think we should replace this test case with a different upgradeable module. It's okay to remove this case in this patch and follow up separately with a new JBS issue. test/langtools/tools/jdeps/modules/patches/java/sql/NonNull.java - copyright start year needs update. Mandy On 2/7/18 8:57 AM, Lance Andersen wrote: > Hi all, > > I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. > The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 > > The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ > > To make the open review easier, I have broken the changes into 5 webrevs: > build changes are: http://cr.openjdk.java.net/~lancea/8190378/open_changes/build_webrev/ > miscellaneous changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/misc_webrev/ > module changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/modules_webrev/ > rmic changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/rmic_webrev/ > test changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/tests_webrev/ > > As part of the removal, the following issues have also been logged: > Removal of the Java EE and CORBA tools from the documentation: https://bugs.openjdk.java.net/browse/JDK-8193906 > Updating the RMIC man pages for the removal of the -iiop and -idl options: https://bugs.openjdk.java.net/browse/JDK-8196510 > Hotspot tests may require further updating or just removed: https://bugs.openjdk.java.net/browse/JDK-8194310 > jdeprescan will need updates due to the removal of the Java EE and CORBA modules: https://bugs.openjdk.java.net/browse/JDK-8194308 > > > > Best, > Lance > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From joe.darcy at oracle.com Wed Feb 7 22:12:37 2018 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 7 Feb 2018 14:12:37 -0800 Subject: JDK 11 RFR of JDK-8196995: java.lang.Character should not state UTF-16 encoding is used for strings Message-ID: Hello, Text in java.lang.Character states a UTF-16 character encoding is used for java.lang.String. While was true for many years, it is not necessarily true and not true in practice as of JDK 9 due to the improvements from JEP 254: Compact Strings. The statement about the encoding should be corrected. Please review the patch below which does this. (I've formatted the patch so that the change is text is made clear; I'll re-flow the paragraph before pushing. Thanks, -Joe diff -r 0b1138ce244f src/java.base/share/classes/java/lang/Character.java --- a/src/java.base/share/classes/java/lang/Character.java??? Tue Feb 06 10:17:31 2018 -0800 +++ b/src/java.base/share/classes/java/lang/Character.java??? Wed Feb 07 11:38:06 2018 -0800 @@ -75,7 +75,7 @@ ? * Characters whose code points are greater ? * than U+FFFF are called supplementary characters.? The Java ? * platform uses the UTF-16 representation in {@code char} arrays and - * in the {@code String} and {@code StringBuffer} classes. In + * may use it elsewhere. In ? * this representation, supplementary characters are represented as a pair ? * of {@code char} values, the first from the high-surrogates ? * range, (\uD800-\uDBFF), the second from the From lance.andersen at oracle.com Wed Feb 7 22:33:13 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 7 Feb 2018 17:33:13 -0500 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: <640e4b80-9083-809a-2a90-798f9d848d2d@oracle.com> References: <640e4b80-9083-809a-2a90-798f9d848d2d@oracle.com> Message-ID: <93C1985C-9E4B-4DCE-A306-E4A733007C11@oracle.com> Hi Mandy > On Feb 7, 2018, at 5:00 PM, mandy chung wrote: > > Hi Lance, > > Great to see this JEP moving along. I reviewed all changes except > test/langtools/tools/javac tests. > > Looks fine overall. Thank you for the review. > Minor comments: > > src/java.base/share/lib/security/default.policy > - no change in this file. Weird, not sure what happened, but it is now fixed. > > test/jdk/tools/jmod/hashes/HashesTest.java > test/jdk/tools/launcher/modules/addexports/AddExportsTest.java > - I think we should replace this test case with a different upgradeable module. > It's okay to remove this case in this patch and follow up separately with > a new JBS issue. I can do that. > > test/langtools/tools/jdeps/modules/patches/java/sql/NonNull.java > - copyright start year needs update. Updated. Best Lance > > Mandy > > On 2/7/18 8:57 AM, Lance Andersen wrote: >> Hi all, >> >> I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. >> The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 >> >> The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ >> >> To make the open review easier, I have broken the changes into 5 webrevs: >> build changes are: http://cr.openjdk.java.net/~lancea/8190378/open_changes/build_webrev/ >> miscellaneous changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/misc_webrev/ >> module changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/modules_webrev/ >> rmic changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/rmic_webrev/ >> test changes are at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/tests_webrev/ >> >> As part of the removal, the following issues have also been logged: >> Removal of the Java EE and CORBA tools from the documentation: https://bugs.openjdk.java.net/browse/JDK-8193906 >> Updating the RMIC man pages for the removal of the -iiop and -idl options: https://bugs.openjdk.java.net/browse/JDK-8196510 >> Hotspot tests may require further updating or just removed: https://bugs.openjdk.java.net/browse/JDK-8194310 >> jdeprescan will need updates due to the removal of the Java EE and CORBA modules: https://bugs.openjdk.java.net/browse/JDK-8194308 >> >> >> >> Best, >> Lance >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From claes.redestad at oracle.com Thu Feb 8 00:38:05 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 8 Feb 2018 01:38:05 +0100 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> <82ead3ee-63a8-854c-429e-98ef05ccddf1@oracle.com> <55e2d730-3773-ad52-64ef-5de8ae5f43db@gmail.com> Message-ID: Hi, On 2018-02-07 15:23, Claes Redestad wrote: > - use map.replace to safely update the entry when putIfAbsent returns > an object > ? but it points to a cleared value, so that (benign) races to create > new Locale objects > ? will canonicalize turns out the implementation attempted here with map.replace was problematic, causing certain tests to fail. Seems there are subtle issues here around establishing a stable equality relationship which may or may not be easy to resolve, so I reverted back these particular changes to LocaleObjectCache from this RFE. I also did some cleanup in BaseLocale based on offline feedback from Paul: http://cr.openjdk.java.net/~redestad/8196869/jdk.03/ Thanks! /Claes From aleksej.efimov at oracle.com Thu Feb 8 01:24:29 2018 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Thu, 8 Feb 2018 01:24:29 +0000 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: <89e36dca-ace4-ec9e-539b-5667b70502e3@Oracle.com> References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> <3d4a77bf-5797-e1b9-6a85-977a00fda3e3@oracle.com> <89e36dca-ace4-ec9e-539b-5667b70502e3@Oracle.com> Message-ID: <9f6c9bae-ccc1-e92d-ae5c-54b4ef88ff4a@oracle.com> Hi Jason, Roger, Thanks for your comments. The new webrev with suggested/recommended edits can be viewed at this location: http://cr.openjdk.java.net/~aefimov/6857903/dev/03/ Thanks, Aleksei PS: Also configured my IDE to use explicit imports =) On 02/06/2018 03:31 PM, Roger Riggs wrote: > Hi Aleksei, > > SAXException.java x 2, SAXExceptionInitCause: > ? Please use explicit imports (not java.io.*). > > SAXEXception.java:145: I would have expected: > if (cause instanceof Exception) {...} > > SAXExceptionInitCause.java: > ? Great to see all the test cases. > > ? Generally, we recommend using try-with-resources but in this case it > does not add much because > ? exceptions can't occur when using BAOS. > > Thanks, Roger > > On 2/5/2018 9:46 PM, Aleks Efimov wrote: >> Thank you for your comments, Jason! >> New webrev can be found at this location: >> http://cr.openjdk.java.net/~aefimov/6857903/dev/02/ >> >> The list of changes: >> - this.initCause() -> super.initCause() >> - 'readObject' has been modified to convert IllegalStateException to >> InvalidClassException. The test case that triggers >> IllegalStateException in SAXException::readObject has been added to >> 'testReadObjectIllegalStateException' test method. >> >> Best Regards, >> Aleksei >> >> On 02/05/2018 05:09 PM, Jason Mehrens wrote: >>> Aleksei, >>> >>> Looks good.? We should avoid this.initCause in readObject and prefer >>> super.initCause so subclasses can't alter the deserialization behavior. >>> While I can't think of a case off the top of my head, I would prefer >>> that we trap and convert IllegalStateException into a >>> InvalidClassException in readObject. >>> That keeps the readObject contract intact in case something >>> malicious is going on.? That is just my preference though. >>> >>> Jason >>> ________________________________________ >>> From: Aleks Efimov >>> Sent: Monday, February 5, 2018 10:51 AM >>> To: Jason Mehrens; 'Roger Riggs' >>> Cc: core-libs-dev >>> Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does >>> not correctly set Exception >>> >>> Hi Roger, Jason, >>> >>> I've tried to avoid doing the same stuff as I did for XPathException to >>> minimize the changes to the SAXException class. But I was naive to >>> think so: >>> Tried to keep the exception field in place to avoid modifications >>> required to keep serial form intact (similar as it was done in >>> JDK-8009581 bug mentioned by Jason), but I missed the 'initCause' >>> method >>> (spotted by Roger, thank you!) that can make cause/exception values >>> inconsistent. To avoid the API modification and for the sake of clarity >>> I proceeded with removal of the 'exception' field. The new version of >>> the fix: >>> - Removes 'exception' field >>> - Maintains the serial form of the SAXException class >>> - Handles the difference in SAXException:exception and Throwable:cause >>> types 'SAXException::getException', i.e. SAXException:exception is >>> Exception typed and Throwable:cause is Throwable typed. >>> - Avoids any changes to API and serial form of the class, but >>> maintaining it similar to JDK-8009581) >>> - There is a copy of SAXException class in java.base module that is >>> required for j.u.Properties::loadFromXML, it has been update with the >>> same changes. >>> >>> New regression test has been added to test: >>> - Serial compatibility with legacy JDKs (similar to JDK-8009581) >>> - usages of SAXException::initCause/getException >>> >>> Webrev with the fix and new regression: >>> http://cr.openjdk.java.net/~aefimov/6857903/dev/01/ >>> >>> Testing shows no JCK/regression tests failures with the proposed fix. >>> >>> Best Regards, >>> Aleksei >>> >>> >>> On 12/21/2017 07:00 PM, Jason Mehrens wrote: >>>> Aleksei, >>>> >>>> You have to override all of the constructors to always disable >>>> initCause.? Actually a similar issue was covered in: >>>> >>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html >>>> >>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html >>>> >>>> >>>> Pretty much everything was hashed out in those threads. >>>> >>>> Here is the final webrev for that: >>>> >>>> http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html >>>> >>>> >>>> That should be a good cookbook for changes and tests required for >>>> this issue.? Note that it gets rid of the Exception field and >>>> maintains serial compatibility. >>>> Looking back on that change, I would have changed it so the >>>> InvalidClassException added the double cause as suppressed exceptions. >>>> >>>> Jason >>>> >>>> ________________________________________ >>>> From: core-libs-dev on >>>> behalf of Aleks Efimov >>>> Sent: Thursday, December 21, 2017 11:27 AM >>>> To: core-libs-dev >>>> Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does >>>> not correctly set Exception >>>> >>>> Hello, >>>> >>>> Please, help to review the fix for jaxp bug that fixes SAXException to >>>> correctly set exception cause with 'setCause' method: >>>> http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ >>>> I've tried to keep the fix miminal with respect to serial form of this >>>> API class, i.e. kept private 'exception' field. >>>> >>>> The new test case was added to IssueTracker56Test unit test. Testing >>>> showed no related JCK/JTREG failures. >>>> >>>> With Best Regards, >>>> Aleksei >>>> >>>> >> > From paul.sandoz at oracle.com Thu Feb 8 02:06:15 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 7 Feb 2018 18:06:15 -0800 Subject: RFR: 8196869: Optimize Locale creation In-Reply-To: References: <895DA133-B0AC-4CB9-855B-656B42DF6CC2@oracle.com> <731b5af1-eae0-5f9e-901e-f9d5ae7a93a2@gmail.com> <82ead3ee-63a8-854c-429e-98ef05ccddf1@oracle.com> <55e2d730-3773-ad52-64ef-5de8ae5f43db@gmail.com> Message-ID: This is looking much better, definitely easier to understand. Paul. > On Feb 7, 2018, at 4:38 PM, Claes Redestad wrote: > > Hi, > > On 2018-02-07 15:23, Claes Redestad wrote: >> - use map.replace to safely update the entry when putIfAbsent returns an object >> but it points to a cleared value, so that (benign) races to create new Locale objects >> will canonicalize > > turns out the implementation attempted here with map.replace was problematic, > causing certain tests to fail. Seems there are subtle issues here around > establishing a stable equality relationship which may or may not be easy to > resolve, so I reverted back these particular changes to LocaleObjectCache from this > RFE. > > I also did some cleanup in BaseLocale based on offline feedback from Paul: > > http://cr.openjdk.java.net/~redestad/8196869/jdk.03/ > > Thanks! > > /Claes From martinrb at google.com Thu Feb 8 02:20:05 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 7 Feb 2018 18:20:05 -0800 Subject: JDK 11 RFR of JDK-8196995: java.lang.Character should not state UTF-16 encoding is used for strings In-Reply-To: References: Message-ID: I'm not sure about this. The LATIN-1 optimization is an implementation detail. The important thing is the API, and it remains char-oriented (although we've added codepoint APIs over the years). Strictly speaking, you can store any 16-bit integer sequences in char[], String, or StringBuffer - no encoding is enforced. What is really UTF-16 is the way that such sequences are expected to be interpret by *other* APIs, e.g. System.out.println (and methods like codePointAt) On Wed, Feb 7, 2018 at 2:12 PM, joe darcy wrote: > Hello, > > Text in java.lang.Character states a UTF-16 character encoding is used for > java.lang.String. While was true for many years, it is not necessarily true > and not true in practice as of JDK 9 due to the improvements from JEP 254: > Compact Strings. > > The statement about the encoding should be corrected. > > Please review the patch below which does this. (I've formatted the patch > so that the change is text is made clear; I'll re-flow the paragraph before > pushing. > > Thanks, > > -Joe > > diff -r 0b1138ce244f src/java.base/share/classes/java/lang/Character.java > --- a/src/java.base/share/classes/java/lang/Character.java Tue Feb 06 > 10:17:31 2018 -0800 > +++ b/src/java.base/share/classes/java/lang/Character.java Wed Feb 07 > 11:38:06 2018 -0800 > @@ -75,7 +75,7 @@ > * Characters whose code points are greater > * than U+FFFF are called supplementary characters. The Java > * platform uses the UTF-16 representation in {@code char} arrays and > - * in the {@code String} and {@code StringBuffer} classes. In > + * may use it elsewhere. In > * this representation, supplementary characters are represented as a pair > * of {@code char} values, the first from the high-surrogates > * range, (\uD800-\uDBFF), the second from the > > From david.holmes at oracle.com Thu Feb 8 03:28:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 8 Feb 2018 13:28:31 +1000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: Message-ID: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> Hi Robin, Adding in hotspot-runtime-dev as all the hotspot changes belong to runtime. I had an initial look through this. To be honest I don't like it. We seem to have to record little bits of information all over the place just so they can be reported by the shutdown event. It seems untidy. :( Can you rename _starting_thread to _main_thread please. The use of "starting" in thread.hpp/cpp is really a naming anomaly. The main thread is the thread that loads the JVM. And that would be consistent with set_exception_in_main_thread. Though why do we care if the main thread exited with an unhandled exception? And if we do care, why do we only care when the shutdown reason is ""No remaining non-daemon Java threads"? I don't like the need to add os::get_abort_exit_code. Do we really need it? What useful information does it convey? It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. Let's see what others think ... Thanks, David On 8/02/2018 1:18 AM, Robin Westberg wrote: > Hi all, > > Please review the following change that adds an event-based tracing > event that is generated when the VM shuts down. The intent is to capture > shutdowns that occur after the VM has been properly initialized (as > initialization problems would most likely mean that the tracing > framework hasn?t been properly started either). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 > Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ > Testing:?hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 > > Best regards, > Robin From xueming.shen at oracle.com Thu Feb 8 03:32:34 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 07 Feb 2018 19:32:34 -0800 Subject: RFR JDK-8164278: java.util.Base64.EncOutputStream/DecInputStream is slower than corresponding version in javax.mail package In-Reply-To: <5A790C4D.1090309@oracle.com> References: <5A790C4D.1090309@oracle.com> Message-ID: <5A7BC4D2.3050000@oracle.com> Hi Roger, Given the concern of the possible incompatible behavior change of over reading bytes from the underlying stream. I decided to give up last proposed changes for DecInputStream for now. With some "minor" cleanup and tuning I still have about 10%+ improvement with various input size sampling. Let's address the decoder stream in separate rfe later, if desired. The updated webrev and the jmh results are at http://cr.openjdk.java.net/~sherman/8164278/webrev http://cr.openjdk.java.net/~sherman/8164278/base64.bm Last version of webrev and corresponding jmh result can be found at http://cr.openjdk.java.net/~sherman/8164278/webrev.02 http://cr.openjdk.java.net/~sherman/8164278/base64.bm.old Thanks, Sherman On 2/5/18, 6:00 PM, Xueming Shen wrote: > Hi, > > Please help review the change for JDK-8164278. > > issue: https://bugs.openjdk.java.net/browse/JDK-8164278 > webrev: http://cr.openjdk.java.net/~sherman/8164278/webrev > > jmh.src: http://cr.openjdk.java.net/~sherman/8164278/Base64BM.java > jmh.result: http://cr.openjdk.java.net/~sherman/8164278/base64.bm > > Base64.Decoder.decode0: > Adopted the "similar" optimization approach we took in > Base64.Encoder.encode0() > to add a "fast path" to decode a block of 4-byte units together > (current implementation > decodes one single byte per while/loop. The jmh benchmark result > indicates a big speed > boost (those decodeArray/Mime/Url results, from 30% to 2 times > faster, depends on > input size). > > Base64.Encoder.encode0() > It appears encode0() was fully optimized in 1.8. Can't get it > faster :-) Tried to use > Unsafe.getLong/putLong instead of byte by byte access. But it > appears the 8-byte > "vectorization" does not bring us enough speed up, the performance > is the same as the > current one. See encode00() at > http://cr.openjdk.java.net/~sherman/8164278/webrev.00 > > Base64.Encoder.wrap(OutputStream)/EncOutputStream.write(): > If my memory serves me right, the current implementation was under > the assumption that > the underlying output stream probably is buffered (if invoker > cares). It would be a redundant > if EncOutputStream buffers bytes again. It appears this is a wrong > assumption. It is much > slower to write 4 bytes separately, compared to bundle them > together in a byte[4] and write > into underlying, even the underlying output stream is a > ByteArrayOutputStream. > Again, the proposed change is to add a fast path loop, as we do in > encode0(), to decode/ > write a block of 3-byte->4-byte unites. It appears this fast loop > can help the compiler to > optimize away some boundary checks, therefor is much faster. > The jmh result Base64BM.encodeOS suggests the new implementation > is almost 4 times faster > and is almost the same as java.mail's stream encoder. > > Base64.Decoder.wrap(InputStream)/DecInputStream.read(): > Same as the approach we take for decode0(), to add a fast path > decode block of 4-byte unites > together. > The jmh result Base64BM.decodeOS (the name probably should be > decodeIS, for InputStream, > but anyway...) shows the proposed one is 4 times faster than the > existing impl and double > the java.mail (Base64BM.decodeOS_javamail) implementation. > > However, there is a side-effect of adding a buffering mechanism > into DecInputStream. The > current implementation read bytes from the underlying stream one > by one, it never reads > more bytes than it needs, which means it should/is supposed to > just stop at the last byte > that it needs to decode, when there is "=" present in the stream. > With buffering, it's possible more bytes (after "=", which > indicates "end of base64 stream") might > be read/consumed in and buffered. A concern? if this is indeed a > concern, the only alternative > might be to add a separate method to support this > "faster-buffered-decoder"? > > Thanks, > Sherman > > > From david.holmes at oracle.com Thu Feb 8 03:32:24 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Feb 2018 19:32:24 -0800 (PST) Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: Moving to core-libs-dev. Code reviews don't take place on jdk-dev but on the component area mailing lists. Thanks, David On 8/02/2018 6:10 AM, yumin qi wrote: > Hi, > > Please review the fix (extra small) for: > bug 8194154: https://bugs.openjdk.java.net/browse/JDK-8194154 > webrev: http://cr.openjdk.java.net/~minqi/8194154/ > > Summary: canonicalize will check and fold double (or more) slashes, but > in function collapsible(char*) and splitNames(char*, char**), it failed to > process strings like "//". This results in the former does not give a > correct number of substrings and the later does not give a correct array of > substrings. The fix add a check if the character is '/' after a '/'. > > The test case added will crash without the fix. > > Thanks > Yumin > From yumin.qi at gmail.com Thu Feb 8 05:15:59 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 7 Feb 2018 21:15:59 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: David, Thanks! On Wed, Feb 7, 2018 at 7:32 PM, David Holmes wrote: > Moving to core-libs-dev. Code reviews don't take place on jdk-dev but on > the component area mailing lists. > > Thanks, > David > > > On 8/02/2018 6:10 AM, yumin qi wrote: > >> Hi, >> >> Please review the fix (extra small) for: >> bug 8194154: https://bugs.openjdk.java.net/browse/JDK-8194154 >> webrev: http://cr.openjdk.java.net/~minqi/8194154/ >> >> Summary: canonicalize will check and fold double (or more) slashes, but >> in function collapsible(char*) and splitNames(char*, char**), it failed to >> process strings like "//". This results in the former does not give a >> correct number of substrings and the later does not give a correct array >> of >> substrings. The fix add a check if the character is '/' after a '/'. >> >> The test case added will crash without the fix. >> >> Thanks >> Yumin >> >> From Alan.Bateman at oracle.com Thu Feb 8 08:04:02 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 8 Feb 2018 08:04:02 +0000 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: References: Message-ID: On 07/02/2018 16:57, Lance Andersen wrote: > Hi all, > > I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. > The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 > > The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ > 800 KLOC deleted, wonderful! The update to technology-summary.html page means its html title no longer matches the contents. We should probably change it to "JCP Technologies in JDK 11" for now. The removal of test cases from the tests in tools/launcher/modules removes most of the test coverage for the upgrade module path. We'll need to replace these sub-tests. Can you create an issue to track that? Everything else looks good and it's okay to track residual issues with other JIRA issues. I think the important thing is to get this monster patch into JDK builds soon so that libraries and the eco system can start to adjust. -Alan From Alan.Bateman at oracle.com Thu Feb 8 11:53:33 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 8 Feb 2018 11:53:33 +0000 Subject: JDK 11 RFR of JDK-8196995: java.lang.Character should not state UTF-16 encoding is used for strings In-Reply-To: References: Message-ID: <66d19661-b8f3-82b6-e4ea-a72ce43ff0cc@oracle.com> On 07/02/2018 22:12, joe darcy wrote: > Hello, > > Text in java.lang.Character states a UTF-16 character encoding is used > for java.lang.String. While was true for many years, it is not > necessarily true and not true in practice as of JDK 9 due to the > improvements from JEP 254: Compact Strings. > > The statement about the encoding should be corrected. > > Please review the patch below which does this. (I've formatted the > patch so that the change is text is made clear; I'll re-flow the > paragraph before pushing. I'm not sure that this is worth changing. You could replace "classes" with "API" and add a note to say that an implementation may use an more optimization representation but I don't think it's really needed. -Alan From lance.andersen at oracle.com Thu Feb 8 13:37:51 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 8 Feb 2018 08:37:51 -0500 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: References: Message-ID: <360C5181-3B9E-4339-B460-04A8385F2461@oracle.com> > On Feb 8, 2018, at 3:04 AM, Alan Bateman wrote: > > On 07/02/2018 16:57, Lance Andersen wrote: >> Hi all, >> >> I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. >> The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 >> >> The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ >> > 800 KLOC deleted, wonderful! > > The update to technology-summary.html page means its html title no longer matches the contents. We should probably change it to "JCP Technologies in JDK 11" for now. I updated the webrev. Thanks for catching that (btw we missed this for JDK 10) > > The removal of test cases from the tests in tools/launcher/modules removes most of the test coverage for the upgrade module path. We'll need to replace these sub-tests. Can you create an issue to track that? I can do that > > Everything else looks good and it's okay to track residual issues with other JIRA issues. I think the important thing is to get this monster patch into JDK builds soon so that libraries and the eco system can start to adjust. Thank you Alan for the review Best Lance > > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at Oracle.com Thu Feb 8 14:36:19 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 8 Feb 2018 09:36:19 -0500 Subject: RFR : 8196854 :TestFlushableGZIPOutputStream failing with IndexOutOfBoundsException In-Reply-To: <6da57541-0fc1-6359-315c-6d06e15fa842@oracle.com> References: <6da57541-0fc1-6359-315c-6d06e15fa842@oracle.com> Message-ID: <82957028-3057-3e7e-c071-8bdd27d808f8@Oracle.com> Hi Sean, Looks fine, Roger On 2/7/2018 1:03 PM, Se?n Coffey wrote: > A jdk8u (and earlier) issue where some new/recent test code meant that > IndexOutOfBoundsException could be thrown. > > https://bugs.openjdk.java.net/browse/JDK-8196854 > http://cr.openjdk.java.net/~coffeys/webrev.8196854/webrev/ > > The buf array was never required and I've verified that the original > 8189789 issue can still be reproduced with a buggy JDK. > From jason_mehrens at hotmail.com Thu Feb 8 15:13:45 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Thu, 8 Feb 2018 15:13:45 +0000 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: <9f6c9bae-ccc1-e92d-ae5c-54b4ef88ff4a@oracle.com> References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> <3d4a77bf-5797-e1b9-6a85-977a00fda3e3@oracle.com> <89e36dca-ace4-ec9e-539b-5667b70502e3@Oracle.com>, <9f6c9bae-ccc1-e92d-ae5c-54b4ef88ff4a@oracle.com> Message-ID: Aleksei, Looks good to me. Jason ________________________________________ From: Aleks Efimov Sent: Wednesday, February 7, 2018 7:24 PM To: Roger Riggs; Jason Mehrens Cc: core-libs-dev at openjdk.java.net Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception Hi Jason, Roger, Thanks for your comments. The new webrev with suggested/recommended edits can be viewed at this location: http://cr.openjdk.java.net/~aefimov/6857903/dev/03/ Thanks, Aleksei PS: Also configured my IDE to use explicit imports =) On 02/06/2018 03:31 PM, Roger Riggs wrote: > Hi Aleksei, > > SAXException.java x 2, SAXExceptionInitCause: > Please use explicit imports (not java.io.*). > > SAXEXception.java:145: I would have expected: > if (cause instanceof Exception) {...} > > SAXExceptionInitCause.java: > Great to see all the test cases. > > Generally, we recommend using try-with-resources but in this case it > does not add much because > exceptions can't occur when using BAOS. > > Thanks, Roger > > On 2/5/2018 9:46 PM, Aleks Efimov wrote: >> Thank you for your comments, Jason! >> New webrev can be found at this location: >> http://cr.openjdk.java.net/~aefimov/6857903/dev/02/ >> >> The list of changes: >> - this.initCause() -> super.initCause() >> - 'readObject' has been modified to convert IllegalStateException to >> InvalidClassException. The test case that triggers >> IllegalStateException in SAXException::readObject has been added to >> 'testReadObjectIllegalStateException' test method. >> >> Best Regards, >> Aleksei >> >> On 02/05/2018 05:09 PM, Jason Mehrens wrote: >>> Aleksei, >>> >>> Looks good. We should avoid this.initCause in readObject and prefer >>> super.initCause so subclasses can't alter the deserialization behavior. >>> While I can't think of a case off the top of my head, I would prefer >>> that we trap and convert IllegalStateException into a >>> InvalidClassException in readObject. >>> That keeps the readObject contract intact in case something >>> malicious is going on. That is just my preference though. >>> >>> Jason >>> ________________________________________ >>> From: Aleks Efimov >>> Sent: Monday, February 5, 2018 10:51 AM >>> To: Jason Mehrens; 'Roger Riggs' >>> Cc: core-libs-dev >>> Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does >>> not correctly set Exception >>> >>> Hi Roger, Jason, >>> >>> I've tried to avoid doing the same stuff as I did for XPathException to >>> minimize the changes to the SAXException class. But I was naive to >>> think so: >>> Tried to keep the exception field in place to avoid modifications >>> required to keep serial form intact (similar as it was done in >>> JDK-8009581 bug mentioned by Jason), but I missed the 'initCause' >>> method >>> (spotted by Roger, thank you!) that can make cause/exception values >>> inconsistent. To avoid the API modification and for the sake of clarity >>> I proceeded with removal of the 'exception' field. The new version of >>> the fix: >>> - Removes 'exception' field >>> - Maintains the serial form of the SAXException class >>> - Handles the difference in SAXException:exception and Throwable:cause >>> types 'SAXException::getException', i.e. SAXException:exception is >>> Exception typed and Throwable:cause is Throwable typed. >>> - Avoids any changes to API and serial form of the class, but >>> maintaining it similar to JDK-8009581) >>> - There is a copy of SAXException class in java.base module that is >>> required for j.u.Properties::loadFromXML, it has been update with the >>> same changes. >>> >>> New regression test has been added to test: >>> - Serial compatibility with legacy JDKs (similar to JDK-8009581) >>> - usages of SAXException::initCause/getException >>> >>> Webrev with the fix and new regression: >>> http://cr.openjdk.java.net/~aefimov/6857903/dev/01/ >>> >>> Testing shows no JCK/regression tests failures with the proposed fix. >>> >>> Best Regards, >>> Aleksei >>> >>> >>> On 12/21/2017 07:00 PM, Jason Mehrens wrote: >>>> Aleksei, >>>> >>>> You have to override all of the constructors to always disable >>>> initCause. Actually a similar issue was covered in: >>>> >>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html >>>> >>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html >>>> >>>> >>>> Pretty much everything was hashed out in those threads. >>>> >>>> Here is the final webrev for that: >>>> >>>> http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html >>>> >>>> >>>> That should be a good cookbook for changes and tests required for >>>> this issue. Note that it gets rid of the Exception field and >>>> maintains serial compatibility. >>>> Looking back on that change, I would have changed it so the >>>> InvalidClassException added the double cause as suppressed exceptions. >>>> >>>> Jason >>>> >>>> ________________________________________ >>>> From: core-libs-dev on >>>> behalf of Aleks Efimov >>>> Sent: Thursday, December 21, 2017 11:27 AM >>>> To: core-libs-dev >>>> Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does >>>> not correctly set Exception >>>> >>>> Hello, >>>> >>>> Please, help to review the fix for jaxp bug that fixes SAXException to >>>> correctly set exception cause with 'setCause' method: >>>> http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ >>>> I've tried to keep the fix miminal with respect to serial form of this >>>> API class, i.e. kept private 'exception' field. >>>> >>>> The new test case was added to IssueTracker56Test unit test. Testing >>>> showed no related JCK/JTREG failures. >>>> >>>> With Best Regards, >>>> Aleksei >>>> >>>> >> > From xueming.shen at oracle.com Thu Feb 8 15:33:34 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 8 Feb 2018 07:33:34 -0800 Subject: RFR : 8196854 :TestFlushableGZIPOutputStream failing with IndexOutOfBoundsException In-Reply-To: <6da57541-0fc1-6359-315c-6d06e15fa842@oracle.com> References: <6da57541-0fc1-6359-315c-6d06e15fa842@oracle.com> Message-ID: <2A60DABE-67DB-4321-95F6-86123BF1CCC8@oracle.com> +1 On Feb 7, 2018, at 10:03 AM, Se?n Coffey wrote: A jdk8u (and earlier) issue where some new/recent test code meant that IndexOutOfBoundsException could be thrown. https://bugs.openjdk.java.net/browse/JDK-8196854 http://cr.openjdk.java.net/~coffeys/webrev.8196854/webrev/ The buf array was never required and I've verified that the original 8189789 issue can still be reproduced with a buggy JDK. -- Regards, Sean. From Roger.Riggs at Oracle.com Thu Feb 8 15:43:17 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 8 Feb 2018 10:43:17 -0500 Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception In-Reply-To: References: <28ef46f9-dcb8-71da-9d56-d3e7c094251e@oracle.com> <3d4a77bf-5797-e1b9-6a85-977a00fda3e3@oracle.com> <89e36dca-ace4-ec9e-539b-5667b70502e3@Oracle.com> <9f6c9bae-ccc1-e92d-ae5c-54b4ef88ff4a@oracle.com> Message-ID: <354c3edc-cefb-fa2b-e1ea-0fff08a1037e@Oracle.com> Looks fine? +1 Thanks, Roger On 2/8/2018 10:13 AM, Jason Mehrens wrote: > Aleksei, > > Looks good to me. > > Jason > ________________________________________ > From: Aleks Efimov > Sent: Wednesday, February 7, 2018 7:24 PM > To: Roger Riggs; Jason Mehrens > Cc: core-libs-dev at openjdk.java.net > Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does not correctly set Exception > > Hi Jason, Roger, > > Thanks for your comments. > The new webrev with suggested/recommended edits can be viewed at this > location: > http://cr.openjdk.java.net/~aefimov/6857903/dev/03/ > > Thanks, > Aleksei > > PS: Also configured my IDE to use explicit imports =) > > On 02/06/2018 03:31 PM, Roger Riggs wrote: >> Hi Aleksei, >> >> SAXException.java x 2, SAXExceptionInitCause: >> Please use explicit imports (not java.io.*). >> >> SAXEXception.java:145: I would have expected: >> if (cause instanceof Exception) {...} >> >> SAXExceptionInitCause.java: >> Great to see all the test cases. >> >> Generally, we recommend using try-with-resources but in this case it >> does not add much because >> exceptions can't occur when using BAOS. >> >> Thanks, Roger >> >> On 2/5/2018 9:46 PM, Aleks Efimov wrote: >>> Thank you for your comments, Jason! >>> New webrev can be found at this location: >>> http://cr.openjdk.java.net/~aefimov/6857903/dev/02/ >>> >>> The list of changes: >>> - this.initCause() -> super.initCause() >>> - 'readObject' has been modified to convert IllegalStateException to >>> InvalidClassException. The test case that triggers >>> IllegalStateException in SAXException::readObject has been added to >>> 'testReadObjectIllegalStateException' test method. >>> >>> Best Regards, >>> Aleksei >>> >>> On 02/05/2018 05:09 PM, Jason Mehrens wrote: >>>> Aleksei, >>>> >>>> Looks good. We should avoid this.initCause in readObject and prefer >>>> super.initCause so subclasses can't alter the deserialization behavior. >>>> While I can't think of a case off the top of my head, I would prefer >>>> that we trap and convert IllegalStateException into a >>>> InvalidClassException in readObject. >>>> That keeps the readObject contract intact in case something >>>> malicious is going on. That is just my preference though. >>>> >>>> Jason >>>> ________________________________________ >>>> From: Aleks Efimov >>>> Sent: Monday, February 5, 2018 10:51 AM >>>> To: Jason Mehrens; 'Roger Riggs' >>>> Cc: core-libs-dev >>>> Subject: Re: RFR [11] (JAXP): 6857903: SAXException.initCause() does >>>> not correctly set Exception >>>> >>>> Hi Roger, Jason, >>>> >>>> I've tried to avoid doing the same stuff as I did for XPathException to >>>> minimize the changes to the SAXException class. But I was naive to >>>> think so: >>>> Tried to keep the exception field in place to avoid modifications >>>> required to keep serial form intact (similar as it was done in >>>> JDK-8009581 bug mentioned by Jason), but I missed the 'initCause' >>>> method >>>> (spotted by Roger, thank you!) that can make cause/exception values >>>> inconsistent. To avoid the API modification and for the sake of clarity >>>> I proceeded with removal of the 'exception' field. The new version of >>>> the fix: >>>> - Removes 'exception' field >>>> - Maintains the serial form of the SAXException class >>>> - Handles the difference in SAXException:exception and Throwable:cause >>>> types 'SAXException::getException', i.e. SAXException:exception is >>>> Exception typed and Throwable:cause is Throwable typed. >>>> - Avoids any changes to API and serial form of the class, but >>>> maintaining it similar to JDK-8009581) >>>> - There is a copy of SAXException class in java.base module that is >>>> required for j.u.Properties::loadFromXML, it has been update with the >>>> same changes. >>>> >>>> New regression test has been added to test: >>>> - Serial compatibility with legacy JDKs (similar to JDK-8009581) >>>> - usages of SAXException::initCause/getException >>>> >>>> Webrev with the fix and new regression: >>>> http://cr.openjdk.java.net/~aefimov/6857903/dev/01/ >>>> >>>> Testing shows no JCK/regression tests failures with the proposed fix. >>>> >>>> Best Regards, >>>> Aleksei >>>> >>>> >>>> On 12/21/2017 07:00 PM, Jason Mehrens wrote: >>>>> Aleksei, >>>>> >>>>> You have to override all of the constructors to always disable >>>>> initCause. Actually a similar issue was covered in: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-May/016908.html >>>>> >>>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017594.html >>>>> >>>>> >>>>> Pretty much everything was hashed out in those threads. >>>>> >>>>> Here is the final webrev for that: >>>>> >>>>> http://cr.openjdk.java.net/~dmeetry/8009581/webrev.5/jaxp/src/javax/xml/xpath/XPathException.java.udiff.html >>>>> >>>>> >>>>> That should be a good cookbook for changes and tests required for >>>>> this issue. Note that it gets rid of the Exception field and >>>>> maintains serial compatibility. >>>>> Looking back on that change, I would have changed it so the >>>>> InvalidClassException added the double cause as suppressed exceptions. >>>>> >>>>> Jason >>>>> >>>>> ________________________________________ >>>>> From: core-libs-dev on >>>>> behalf of Aleks Efimov >>>>> Sent: Thursday, December 21, 2017 11:27 AM >>>>> To: core-libs-dev >>>>> Subject: RFR [11] (JAXP): 6857903: SAXException.initCause() does >>>>> not correctly set Exception >>>>> >>>>> Hello, >>>>> >>>>> Please, help to review the fix for jaxp bug that fixes SAXException to >>>>> correctly set exception cause with 'setCause' method: >>>>> http://cr.openjdk.java.net/~aefimov/6857903/dev/00/ >>>>> I've tried to keep the fix miminal with respect to serial form of this >>>>> API class, i.e. kept private 'exception' field. >>>>> >>>>> The new test case was added to IssueTracker56Test unit test. Testing >>>>> showed no related JCK/JTREG failures. >>>>> >>>>> With Best Regards, >>>>> Aleksei >>>>> >>>>> From robin.westberg at oracle.com Thu Feb 8 15:50:50 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Thu, 8 Feb 2018 16:50:50 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> Message-ID: <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> Hi David, > On 8 Feb 2018, at 04:28, David Holmes wrote: > > Hi Robin, > > Adding in hotspot-runtime-dev as all the hotspot changes belong to runtime. Thanks, sorry about that.. > I had an initial look through this. > > To be honest I don't like it. We seem to have to record little bits of information all over the place just so they can be reported by the shutdown event. It seems untidy. :( > > Can you rename _starting_thread to _main_thread please. The use of "starting" in thread.hpp/cpp is really a naming anomaly. The main thread is the thread that loads the JVM. And that would be consistent with set_exception_in_main_thread. > > Though why do we care if the main thread exited with an unhandled exception? And if we do care, why do we only care when the shutdown reason is ""No remaining non-daemon Java threads"? > > I don't like the need to add os::get_abort_exit_code. Do we really need it? What useful information does it convey? Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. > It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). Best regards, Robin > Let's see what others think ... > > Thanks, > David > > On 8/02/2018 1:18 AM, Robin Westberg wrote: >> Hi all, >> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >> Best regards, >> Robin From ben_walsh at uk.ibm.com Thu Feb 8 16:22:32 2018 From: ben_walsh at uk.ibm.com (Ben Walsh) Date: Thu, 8 Feb 2018 16:22:32 +0000 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: Hi Paul, Following up with the requested loop and vectorization benchmarks ... (Do the vectorization benchmark results imply that the Hotspot compiler has been unable to perform the vectorization optimisation due to the presence of the reachabilityFence ?) ----------------------------------------------------------------------------------------------------------------------- Loop Benchmarking ---- ------------ package org.sample; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Param; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import java.nio.ByteBuffer; @State(Scope.Benchmark) public class ByteBufferBenchmark { @Param({"1", "10", "100", "1000", "10000"}) public int L; @State(Scope.Benchmark) public static class ByteBufferContainer { ByteBuffer bb; @Setup(Level.Invocation) public void initByteBuffer() { bb = ByteBuffer.allocateDirect(10000); } ByteBuffer getByteBuffer() { return bb; } } @Benchmark public ByteBuffer benchmark_byte_buffer_put(ByteBufferContainer bbC) { ByteBuffer bb = bbC.getByteBuffer(); for (int i = 0; i < L; i++) { bb.put((byte)i); } return bb; } } Without Changes Benchmark (L) Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 29303145.752 ? 635979.750 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 24260859.017 ? 528891.303 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 8512366.637 ? 136615.070 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 1323756.037 ? 21485.369 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 145965.305 ? 1301.469 ops/s With Changes Benchmark (L) Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 28893540.122 ? 754554.747 ops/s -1.398% ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 15317696.355 ? 231621.608 ops/s -36.863% ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 2546599.578 ? 32136.873 ops/s -70.084% ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 288832.514 ? 3854.522 ops/s -78.181% ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 29747.386 ? 214.831 ops/s -79.620% ----------------------------------------------------------------------------------------------------------------------- Vectorization Benchmarking ------------- ------------ package org.sample; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Param; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import java.nio.ByteBuffer; @State(Scope.Benchmark) public class ByteBufferBenchmark { @Param({"1", "10", "100", "1000", "10000"}) public int L; @State(Scope.Benchmark) public static class ByteBufferContainer { ByteBuffer bb; @Setup(Level.Invocation) public void initByteBuffer() { bb = ByteBuffer.allocateDirect(4 * 10000); for (int i = 0; i < 10000; i++) { bb.putInt(i); } } ByteBuffer getByteBuffer() { return bb; } } @Benchmark public int benchmark_byte_buffer_put(ByteBufferContainer bbC) { ByteBuffer bb = bbC.getByteBuffer(); bb.position(0); int sum = 0; for (int i = 0; i < L; i++) { sum += bb.getInt(); } return sum; } } Without Changes Benchmark (L) Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 29677205.748 ? 544721.142 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 18219951.454 ? 320724.793 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 7767650.826 ? 121798.910 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 1646075.010 ? 9804.499 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 183489.418 ? 1355.967 ops/s With Changes Benchmark (L) Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 15230086.695 ? 390174.190 ops/s -48.681% ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 8126310.728 ? 123661.342 ops/s -55.399% ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 1582699.233 ? 7278.744 ops/s -79.624% ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 179726.465 ? 802.333 ops/s -89.082% ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 18327.049 ? 9.506 ops/s -90.012% NB : For reference - for this and previous benchmarking results ... "Without Changes" and "With Changes" - java -version ... openjdk version "10-internal" 2018-03-20 OpenJDK Runtime Environment (build 10-internal+0-adhoc.walshbp.jdk) OpenJDK 64-Bit Server VM (build 10-internal+0-adhoc.walshbp.jdk, mixed mode) ----------------------------------------------------------------------------------------------------------------------- Regards, Ben Walsh From: Ben Walsh/UK/IBM To: Paul Sandoz Cc: core-libs-dev Date: 05/02/2018 16:47 Subject: Re: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup Running with the following test under the JMH benchmark framework (never used this before, so please do point out any issues with the test) ... ------------------------------------------------------------------------------------------------------------------- package org.sample; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import java.nio.ByteBuffer; public class ByteBufferBenchmark { @State(Scope.Benchmark) public static class ByteBufferContainer { ByteBuffer bb; @Setup(Level.Invocation) public void initByteBuffer() { bb = ByteBuffer.allocateDirect(1); } ByteBuffer getByteBuffer() { return bb; } } @Benchmark public void benchmark_byte_buffer_put(ByteBufferContainer bbC) { bbC.getByteBuffer().put((byte)42); } } ------------------------------------------------------------------------------------------------------------------- ... I get the following results with and without the changes ... [9walsbp at dolphin ~]$ ./build_with/jdk/bin/java -jar benchmarks.jar WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/home/9walsbp/benchmarks.jar) to field java.io.Console.cs WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release # JMH version: 1.20 # VM version: JDK 10-internal, VM 10-internal+0-adhoc.walshbp.jdk # VM invoker: /home/9walsbp/build_with/jdk/bin/java # VM options: # Warmup: 20 iterations, 1 s each # Measurement: 20 iterations, 1 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put # Run progress: 0.00% complete, ETA 00:06:40 # Fork: 1 of 10 # Warmup Iteration 1: 28572294.700 ops/s # Warmup Iteration 2: 33102015.934 ops/s # Warmup Iteration 3: 32883031.968 ops/s # Warmup Iteration 4: 32875302.586 ops/s # Warmup Iteration 5: 33708018.941 ops/s # Warmup Iteration 6: 34107603.885 ops/s # Warmup Iteration 7: 31615123.362 ops/s # Warmup Iteration 8: 34306874.876 ops/s # Warmup Iteration 9: 32615070.232 ops/s # Warmup Iteration 10: 34268641.323 ops/s # Warmup Iteration 11: 32209257.766 ops/s # Warmup Iteration 12: 34111408.999 ops/s # Warmup Iteration 13: 33527360.187 ops/s # Warmup Iteration 14: 34331726.136 ops/s # Warmup Iteration 15: 34325119.218 ops/s # Warmup Iteration 16: 31562168.904 ops/s # Warmup Iteration 17: 33227878.358 ops/s # Warmup Iteration 18: 34151154.273 ops/s # Warmup Iteration 19: 29264381.793 ops/s # Warmup Iteration 20: 30308058.560 ops/s Iteration 1: 33502237.659 ops/s Iteration 2: 33591469.068 ops/s Iteration 3: 29330487.674 ops/s Iteration 4: 31401627.173 ops/s Iteration 5: 32346391.980 ops/s Iteration 6: 34106932.868 ops/s Iteration 7: 29961218.721 ops/s Iteration 8: 31166503.731 ops/s Iteration 9: 34180826.531 ops/s Iteration 10: 34436432.202 ops/s Iteration 11: 28003272.167 ops/s Iteration 12: 30947688.218 ops/s Iteration 13: 33727404.696 ops/s Iteration 14: 33794727.241 ops/s Iteration 15: 34089932.945 ops/s Iteration 16: 30229615.627 ops/s Iteration 17: 32747950.250 ops/s Iteration 18: 34195332.494 ops/s Iteration 19: 34225966.793 ops/s Iteration 20: 30253949.366 ops/s # Run progress: 10.00% complete, ETA 00:27:45 # Fork: 2 of 10 # Warmup Iteration 1: 28189458.524 ops/s # Warmup Iteration 2: 33385943.637 ops/s # Warmup Iteration 3: 31474459.740 ops/s # Warmup Iteration 4: 32994555.969 ops/s # Warmup Iteration 5: 33580210.619 ops/s # Warmup Iteration 6: 34030937.323 ops/s # Warmup Iteration 7: 31891882.950 ops/s # Warmup Iteration 8: 34277935.152 ops/s # Warmup Iteration 9: 31636968.061 ops/s # Warmup Iteration 10: 34376216.983 ops/s # Warmup Iteration 11: 31790184.297 ops/s # Warmup Iteration 12: 34069806.835 ops/s # Warmup Iteration 13: 31737398.413 ops/s # Warmup Iteration 14: 34450863.223 ops/s # Warmup Iteration 15: 34568522.458 ops/s # Warmup Iteration 16: 30029081.882 ops/s # Warmup Iteration 17: 32092783.620 ops/s # Warmup Iteration 18: 33063366.197 ops/s # Warmup Iteration 19: 27694130.000 ops/s # Warmup Iteration 20: 31264525.792 ops/s Iteration 1: 33330874.247 ops/s Iteration 2: 34150690.011 ops/s Iteration 3: 28497705.592 ops/s Iteration 4: 31877639.021 ops/s Iteration 5: 33367324.280 ops/s Iteration 6: 34472176.807 ops/s Iteration 7: 29032738.714 ops/s Iteration 8: 31544864.097 ops/s Iteration 9: 33752028.095 ops/s Iteration 10: 34473732.353 ops/s Iteration 11: 29982974.569 ops/s Iteration 12: 30603336.272 ops/s Iteration 13: 32481117.249 ops/s Iteration 14: 33393570.191 ops/s Iteration 15: 34358979.713 ops/s Iteration 16: 27617909.839 ops/s Iteration 17: 30833802.394 ops/s Iteration 18: 32004461.408 ops/s Iteration 19: 34074910.495 ops/s Iteration 20: 34346053.320 ops/s # Run progress: 20.00% complete, ETA 00:24:26 # Fork: 3 of 10 # Warmup Iteration 1: 28503366.982 ops/s # Warmup Iteration 2: 33475519.627 ops/s # Warmup Iteration 3: 34588321.205 ops/s # Warmup Iteration 4: 35947256.417 ops/s # Warmup Iteration 5: 37638512.766 ops/s # Warmup Iteration 6: 38208797.422 ops/s # Warmup Iteration 7: 32824617.800 ops/s # Warmup Iteration 8: 35767442.077 ops/s # Warmup Iteration 9: 36548126.754 ops/s # Warmup Iteration 10: 38337143.976 ops/s # Warmup Iteration 11: 35379051.550 ops/s # Warmup Iteration 12: 38381933.847 ops/s # Warmup Iteration 13: 36543582.434 ops/s # Warmup Iteration 14: 38178008.316 ops/s # Warmup Iteration 15: 38213974.988 ops/s # Warmup Iteration 16: 32176964.179 ops/s # Warmup Iteration 17: 35487842.133 ops/s # Warmup Iteration 18: 38381350.345 ops/s # Warmup Iteration 19: 31504263.357 ops/s # Warmup Iteration 20: 37831665.793 ops/s Iteration 1: 37530637.288 ops/s Iteration 2: 38628685.180 ops/s Iteration 3: 35170239.750 ops/s Iteration 4: 32397535.287 ops/s Iteration 5: 38643366.565 ops/s Iteration 6: 35723555.410 ops/s Iteration 7: 37073023.139 ops/s Iteration 8: 38332509.859 ops/s Iteration 9: 36047299.714 ops/s Iteration 10: 38666580.432 ops/s Iteration 11: 38827527.980 ops/s Iteration 12: 38885170.177 ops/s Iteration 13: 38552899.367 ops/s Iteration 14: 38231144.659 ops/s Iteration 15: 26785494.819 ops/s Iteration 16: 35979289.874 ops/s Iteration 17: 36233311.409 ops/s Iteration 18: 37322531.784 ops/s Iteration 19: 30917505.079 ops/s Iteration 20: 36602131.217 ops/s # Run progress: 30.00% complete, ETA 00:19:56 # Fork: 4 of 10 # Warmup Iteration 1: 28201486.347 ops/s # Warmup Iteration 2: 33954913.216 ops/s # Warmup Iteration 3: 33614818.543 ops/s # Warmup Iteration 4: 36075462.255 ops/s # Warmup Iteration 5: 35868466.813 ops/s # Warmup Iteration 6: 37539151.611 ops/s # Warmup Iteration 7: 35791698.290 ops/s # Warmup Iteration 8: 37804228.841 ops/s # Warmup Iteration 9: 36209952.096 ops/s # Warmup Iteration 10: 37995654.523 ops/s # Warmup Iteration 11: 36279556.807 ops/s # Warmup Iteration 12: 37876471.909 ops/s # Warmup Iteration 13: 35472820.096 ops/s # Warmup Iteration 14: 38192862.745 ops/s # Warmup Iteration 15: 38214750.954 ops/s # Warmup Iteration 16: 33616490.113 ops/s # Warmup Iteration 17: 37147216.192 ops/s # Warmup Iteration 18: 37759898.719 ops/s # Warmup Iteration 19: 31762576.627 ops/s # Warmup Iteration 20: 35935371.693 ops/s Iteration 1: 37612237.054 ops/s Iteration 2: 31182849.622 ops/s Iteration 3: 35771979.960 ops/s Iteration 4: 37617025.341 ops/s Iteration 5: 31274273.020 ops/s Iteration 6: 36798699.672 ops/s Iteration 7: 37923256.148 ops/s Iteration 8: 38143822.112 ops/s Iteration 9: 29781444.210 ops/s Iteration 10: 34202436.254 ops/s Iteration 11: 36538073.378 ops/s Iteration 12: 37944072.150 ops/s Iteration 13: 30172512.287 ops/s Iteration 14: 34294490.289 ops/s Iteration 15: 37096818.141 ops/s Iteration 16: 37884541.579 ops/s Iteration 17: 31627891.962 ops/s Iteration 18: 34458196.904 ops/s Iteration 19: 37949147.371 ops/s Iteration 20: 38084720.812 ops/s # Run progress: 40.00% complete, ETA 00:17:39 # Fork: 5 of 10 # Warmup Iteration 1: 26430556.314 ops/s # Warmup Iteration 2: 28702874.225 ops/s # Warmup Iteration 3: 28012568.331 ops/s # Warmup Iteration 4: 30857741.012 ops/s # Warmup Iteration 5: 30671764.934 ops/s # Warmup Iteration 6: 31729981.637 ops/s # Warmup Iteration 7: 30171277.707 ops/s # Warmup Iteration 8: 29817290.663 ops/s # Warmup Iteration 9: 31275042.135 ops/s # Warmup Iteration 10: 32028867.062 ops/s # Warmup Iteration 11: 30066034.279 ops/s # Warmup Iteration 12: 30772389.223 ops/s # Warmup Iteration 13: 30066233.182 ops/s # Warmup Iteration 14: 32035909.235 ops/s # Warmup Iteration 15: 32192119.140 ops/s # Warmup Iteration 16: 30496387.577 ops/s # Warmup Iteration 17: 30957378.046 ops/s # Warmup Iteration 18: 31420314.443 ops/s # Warmup Iteration 19: 31981518.074 ops/s # Warmup Iteration 20: 26748312.384 ops/s Iteration 1: 28740606.592 ops/s Iteration 2: 30156532.719 ops/s Iteration 3: 31775304.195 ops/s Iteration 4: 26843600.302 ops/s Iteration 5: 30284709.884 ops/s Iteration 6: 32124615.948 ops/s Iteration 7: 32123075.530 ops/s Iteration 8: 26454319.079 ops/s Iteration 9: 31913402.742 ops/s Iteration 10: 32437412.737 ops/s Iteration 11: 32164716.512 ops/s Iteration 12: 25373082.559 ops/s Iteration 13: 30293078.502 ops/s Iteration 14: 31128369.793 ops/s Iteration 15: 31415561.253 ops/s Iteration 16: 32120636.468 ops/s Iteration 17: 25984394.598 ops/s Iteration 18: 30199573.863 ops/s Iteration 19: 32556187.120 ops/s Iteration 20: 32636129.190 ops/s # Run progress: 50.00% complete, ETA 00:14:30 # Fork: 6 of 10 # Warmup Iteration 1: 28150411.806 ops/s # Warmup Iteration 2: 28054601.733 ops/s # Warmup Iteration 3: 33178455.042 ops/s # Warmup Iteration 4: 33339230.141 ops/s # Warmup Iteration 5: 33332791.104 ops/s # Warmup Iteration 6: 33624536.190 ops/s # Warmup Iteration 7: 31299303.684 ops/s # Warmup Iteration 8: 33037261.267 ops/s # Warmup Iteration 9: 33356211.045 ops/s # Warmup Iteration 10: 33861930.201 ops/s # Warmup Iteration 11: 33270425.776 ops/s # Warmup Iteration 12: 33692415.570 ops/s # Warmup Iteration 13: 33297727.491 ops/s # Warmup Iteration 14: 33820015.956 ops/s # Warmup Iteration 15: 33808521.564 ops/s # Warmup Iteration 16: 31213861.694 ops/s # Warmup Iteration 17: 32149452.446 ops/s # Warmup Iteration 18: 33684868.956 ops/s # Warmup Iteration 19: 30155197.410 ops/s # Warmup Iteration 20: 33566808.050 ops/s Iteration 1: 33654501.647 ops/s Iteration 2: 29509826.671 ops/s Iteration 3: 29610518.398 ops/s Iteration 4: 32646220.404 ops/s Iteration 5: 33773773.299 ops/s Iteration 6: 27873055.802 ops/s Iteration 7: 30681259.230 ops/s Iteration 8: 32717634.647 ops/s Iteration 9: 33758469.696 ops/s Iteration 10: 27232349.464 ops/s Iteration 11: 30616039.535 ops/s Iteration 12: 31682354.340 ops/s Iteration 13: 32580910.390 ops/s Iteration 14: 33811388.262 ops/s Iteration 15: 27191817.186 ops/s Iteration 16: 29873209.932 ops/s Iteration 17: 32087557.824 ops/s Iteration 18: 32644007.761 ops/s Iteration 19: 33799135.934 ops/s Iteration 20: 26758101.989 ops/s # Run progress: 60.00% complete, ETA 00:11:43 # Fork: 7 of 10 # Warmup Iteration 1: 26592644.866 ops/s # Warmup Iteration 2: 33131524.362 ops/s # Warmup Iteration 3: 30127771.163 ops/s # Warmup Iteration 4: 31924144.092 ops/s # Warmup Iteration 5: 32001599.069 ops/s # Warmup Iteration 6: 33648812.429 ops/s # Warmup Iteration 7: 30874748.951 ops/s # Warmup Iteration 8: 33802650.338 ops/s # Warmup Iteration 9: 31054184.796 ops/s # Warmup Iteration 10: 33484369.763 ops/s # Warmup Iteration 11: 32834442.289 ops/s # Warmup Iteration 12: 34217782.915 ops/s # Warmup Iteration 13: 33036828.248 ops/s # Warmup Iteration 14: 34368855.165 ops/s # Warmup Iteration 15: 34415523.035 ops/s # Warmup Iteration 16: 31187588.079 ops/s # Warmup Iteration 17: 32594818.974 ops/s # Warmup Iteration 18: 34086802.016 ops/s # Warmup Iteration 19: 34340272.384 ops/s # Warmup Iteration 20: 28259357.798 ops/s Iteration 1: 31757196.070 ops/s Iteration 2: 33991747.455 ops/s Iteration 3: 34024018.852 ops/s Iteration 4: 33996716.542 ops/s Iteration 5: 29979159.027 ops/s Iteration 6: 34012243.975 ops/s Iteration 7: 33990813.479 ops/s Iteration 8: 34083707.765 ops/s Iteration 9: 34107629.820 ops/s Iteration 10: 27896785.113 ops/s Iteration 11: 30772172.806 ops/s Iteration 12: 33765139.270 ops/s Iteration 13: 34369680.171 ops/s Iteration 14: 34465158.746 ops/s Iteration 15: 28118201.187 ops/s Iteration 16: 30669149.170 ops/s Iteration 17: 32676396.546 ops/s Iteration 18: 34190840.882 ops/s Iteration 19: 34238377.372 ops/s Iteration 20: 27737131.008 ops/s # Run progress: 70.00% complete, ETA 00:08:38 # Fork: 8 of 10 # Warmup Iteration 1: 28630824.119 ops/s # Warmup Iteration 2: 33818762.541 ops/s # Warmup Iteration 3: 35038007.951 ops/s # Warmup Iteration 4: 36304928.826 ops/s # Warmup Iteration 5: 36354056.932 ops/s # Warmup Iteration 6: 37772679.687 ops/s # Warmup Iteration 7: 35227674.534 ops/s # Warmup Iteration 8: 38029506.671 ops/s # Warmup Iteration 9: 34585248.627 ops/s # Warmup Iteration 10: 37248219.973 ops/s # Warmup Iteration 11: 34593434.702 ops/s # Warmup Iteration 12: 35464181.287 ops/s # Warmup Iteration 13: 36696009.835 ops/s # Warmup Iteration 14: 38316197.354 ops/s # Warmup Iteration 15: 38266037.386 ops/s # Warmup Iteration 16: 37189662.171 ops/s # Warmup Iteration 17: 36523113.799 ops/s # Warmup Iteration 18: 37988022.876 ops/s # Warmup Iteration 19: 38051736.146 ops/s # Warmup Iteration 20: 31182088.185 ops/s Iteration 1: 37190441.731 ops/s Iteration 2: 37956882.826 ops/s Iteration 3: 38140733.603 ops/s Iteration 4: 37252516.227 ops/s Iteration 5: 31725393.553 ops/s Iteration 6: 36471814.199 ops/s Iteration 7: 38114054.355 ops/s Iteration 8: 32616409.003 ops/s Iteration 9: 37786024.918 ops/s Iteration 10: 37992346.638 ops/s Iteration 11: 38123237.062 ops/s Iteration 12: 38287723.965 ops/s Iteration 13: 38345880.203 ops/s Iteration 14: 31130022.209 ops/s Iteration 15: 37986966.666 ops/s Iteration 16: 36960266.429 ops/s Iteration 17: 38008103.179 ops/s Iteration 18: 38014088.784 ops/s Iteration 19: 38110627.648 ops/s Iteration 20: 38072572.537 ops/s # Run progress: 80.00% complete, ETA 00:05:39 # Fork: 9 of 10 # Warmup Iteration 1: 28298513.112 ops/s # Warmup Iteration 2: 32838632.035 ops/s # Warmup Iteration 3: 30181316.633 ops/s # Warmup Iteration 4: 32909381.568 ops/s # Warmup Iteration 5: 31944847.556 ops/s # Warmup Iteration 6: 33397277.722 ops/s # Warmup Iteration 7: 30750924.273 ops/s # Warmup Iteration 8: 33913172.836 ops/s # Warmup Iteration 9: 31744562.260 ops/s # Warmup Iteration 10: 33605243.253 ops/s # Warmup Iteration 11: 31510842.700 ops/s # Warmup Iteration 12: 33828656.623 ops/s # Warmup Iteration 13: 31525600.401 ops/s # Warmup Iteration 14: 34036366.024 ops/s # Warmup Iteration 15: 34031912.703 ops/s # Warmup Iteration 16: 29092574.272 ops/s # Warmup Iteration 17: 32539488.819 ops/s # Warmup Iteration 18: 32600231.822 ops/s # Warmup Iteration 19: 32928686.312 ops/s # Warmup Iteration 20: 33536586.546 ops/s Iteration 1: 30083475.241 ops/s Iteration 2: 32341395.220 ops/s Iteration 3: 33803703.325 ops/s Iteration 4: 33915545.687 ops/s Iteration 5: 30320419.165 ops/s Iteration 6: 33472727.881 ops/s Iteration 7: 33757353.957 ops/s Iteration 8: 28339704.610 ops/s Iteration 9: 33045545.359 ops/s Iteration 10: 33815271.934 ops/s Iteration 11: 33793968.698 ops/s Iteration 12: 33715714.353 ops/s Iteration 13: 33979632.042 ops/s Iteration 14: 29129416.976 ops/s Iteration 15: 32352103.774 ops/s Iteration 16: 32349089.070 ops/s Iteration 17: 32198286.479 ops/s Iteration 18: 29763083.673 ops/s Iteration 19: 30985401.866 ops/s Iteration 20: 30704515.609 ops/s # Run progress: 90.00% complete, ETA 00:02:45 # Fork: 10 of 10 # Warmup Iteration 1: 25845863.121 ops/s # Warmup Iteration 2: 33218974.229 ops/s # Warmup Iteration 3: 32576422.877 ops/s # Warmup Iteration 4: 33624698.333 ops/s # Warmup Iteration 5: 32881377.229 ops/s # Warmup Iteration 6: 33675931.296 ops/s # Warmup Iteration 7: 31678171.139 ops/s # Warmup Iteration 8: 31865457.116 ops/s # Warmup Iteration 9: 32035721.100 ops/s # Warmup Iteration 10: 33759626.899 ops/s # Warmup Iteration 11: 32020679.489 ops/s # Warmup Iteration 12: 33497805.239 ops/s # Warmup Iteration 13: 31818025.906 ops/s # Warmup Iteration 14: 34086770.330 ops/s # Warmup Iteration 15: 34168967.286 ops/s # Warmup Iteration 16: 28807710.513 ops/s # Warmup Iteration 17: 31152856.399 ops/s # Warmup Iteration 18: 33772983.915 ops/s # Warmup Iteration 19: 34020496.828 ops/s # Warmup Iteration 20: 28814678.720 ops/s Iteration 1: 31022028.693 ops/s Iteration 2: 33609915.911 ops/s Iteration 3: 32123372.543 ops/s Iteration 4: 27813625.332 ops/s Iteration 5: 31379592.611 ops/s Iteration 6: 32644747.339 ops/s Iteration 7: 33793411.987 ops/s Iteration 8: 30422277.460 ops/s Iteration 9: 32626659.385 ops/s Iteration 10: 33719480.304 ops/s Iteration 11: 33980011.600 ops/s Iteration 12: 28107750.520 ops/s Iteration 13: 31126083.092 ops/s Iteration 14: 32649007.164 ops/s Iteration 15: 33744248.267 ops/s Iteration 16: 34026461.866 ops/s Iteration 17: 27717690.095 ops/s Iteration 18: 31491088.409 ops/s Iteration 19: 32894429.409 ops/s Iteration 20: 33722538.163 ops/s Result "org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put": 33100911.857 ?(99.9%) 747461.951 ops/s [Average] (min, avg, max) = (25373082.559, 33100911.857, 38885170.177), stdev = 3164800.705 CI (99.9%): [32353449.906, 33848373.808] (assumes normal distribution) # Run complete. Total time: 00:27:27 Benchmark Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 33100911.857 ? 747461.951 ops/s [9walsbp at dolphin ~]$ ./build_without/jdk/bin/java -jar benchmarks.jar WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/home/9walsbp/benchmarks.jar) to field java.io.Console.cs WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release # JMH version: 1.20 # VM version: JDK 10-internal, VM 10-internal+0-adhoc.walshbp.jdk # VM invoker: /home/9walsbp/build_without/jdk/bin/java # VM options: # Warmup: 20 iterations, 1 s each # Measurement: 20 iterations, 1 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put # Run progress: 0.00% complete, ETA 00:06:40 # Fork: 1 of 10 # Warmup Iteration 1: 29113219.200 ops/s # Warmup Iteration 2: 38390854.347 ops/s # Warmup Iteration 3: 37576196.805 ops/s # Warmup Iteration 4: 38517882.972 ops/s # Warmup Iteration 5: 38838678.528 ops/s # Warmup Iteration 6: 38722019.753 ops/s # Warmup Iteration 7: 30279515.339 ops/s # Warmup Iteration 8: 37301591.985 ops/s # Warmup Iteration 9: 38937014.242 ops/s # Warmup Iteration 10: 38446665.693 ops/s # Warmup Iteration 11: 35988668.446 ops/s # Warmup Iteration 12: 37681609.419 ops/s # Warmup Iteration 13: 38993187.956 ops/s # Warmup Iteration 14: 37414379.929 ops/s # Warmup Iteration 15: 38753712.247 ops/s # Warmup Iteration 16: 38753551.189 ops/s # Warmup Iteration 17: 33767765.399 ops/s # Warmup Iteration 18: 37450537.087 ops/s # Warmup Iteration 19: 39118737.402 ops/s # Warmup Iteration 20: 39031101.838 ops/s Iteration 1: 32912027.168 ops/s Iteration 2: 37597695.079 ops/s Iteration 3: 37116802.899 ops/s Iteration 4: 37425452.741 ops/s Iteration 5: 31759009.838 ops/s Iteration 6: 37428294.033 ops/s Iteration 7: 38607838.742 ops/s Iteration 8: 34858025.193 ops/s Iteration 9: 36273653.347 ops/s Iteration 10: 38886985.073 ops/s Iteration 11: 38684265.699 ops/s Iteration 12: 30418949.886 ops/s Iteration 13: 36510488.388 ops/s Iteration 14: 37732064.835 ops/s Iteration 15: 37907332.810 ops/s Iteration 16: 38640267.003 ops/s Iteration 17: 31545863.662 ops/s Iteration 18: 36326351.514 ops/s Iteration 19: 37504107.281 ops/s Iteration 20: 36925861.540 ops/s # Run progress: 10.00% complete, ETA 00:23:50 # Fork: 2 of 10 # Warmup Iteration 1: 31027078.683 ops/s # Warmup Iteration 2: 32638372.801 ops/s # Warmup Iteration 3: 32343711.957 ops/s # Warmup Iteration 4: 33981414.940 ops/s # Warmup Iteration 5: 34883411.692 ops/s # Warmup Iteration 6: 35725521.918 ops/s # Warmup Iteration 7: 33357134.144 ops/s # Warmup Iteration 8: 33753068.420 ops/s # Warmup Iteration 9: 35403231.692 ops/s # Warmup Iteration 10: 35235976.314 ops/s # Warmup Iteration 11: 35321898.883 ops/s # Warmup Iteration 12: 34820537.854 ops/s # Warmup Iteration 13: 35920876.086 ops/s # Warmup Iteration 14: 35251249.967 ops/s # Warmup Iteration 15: 34446183.776 ops/s # Warmup Iteration 16: 33048031.622 ops/s # Warmup Iteration 17: 31531379.814 ops/s # Warmup Iteration 18: 33034792.981 ops/s # Warmup Iteration 19: 34738150.009 ops/s # Warmup Iteration 20: 30077393.793 ops/s Iteration 1: 31766138.031 ops/s Iteration 2: 32851639.920 ops/s Iteration 3: 33660705.695 ops/s Iteration 4: 30490177.412 ops/s Iteration 5: 33922213.740 ops/s Iteration 6: 35233366.402 ops/s Iteration 7: 35626761.045 ops/s Iteration 8: 30340609.672 ops/s Iteration 9: 34019673.519 ops/s Iteration 10: 36014726.921 ops/s Iteration 11: 35974210.612 ops/s Iteration 12: 30201358.306 ops/s Iteration 13: 31810853.163 ops/s Iteration 14: 33132733.085 ops/s Iteration 15: 34555927.009 ops/s Iteration 16: 35932316.359 ops/s Iteration 17: 29234047.193 ops/s Iteration 18: 33989744.306 ops/s Iteration 19: 33949712.258 ops/s Iteration 20: 35135848.150 ops/s # Run progress: 20.00% complete, ETA 00:21:55 # Fork: 3 of 10 # Warmup Iteration 1: 30980353.496 ops/s # Warmup Iteration 2: 32972554.880 ops/s # Warmup Iteration 3: 33057281.401 ops/s # Warmup Iteration 4: 34633426.034 ops/s # Warmup Iteration 5: 34517376.033 ops/s # Warmup Iteration 6: 35702523.091 ops/s # Warmup Iteration 7: 34365724.187 ops/s # Warmup Iteration 8: 36032644.388 ops/s # Warmup Iteration 9: 35136119.559 ops/s # Warmup Iteration 10: 36140189.976 ops/s # Warmup Iteration 11: 33882357.986 ops/s # Warmup Iteration 12: 35778782.987 ops/s # Warmup Iteration 13: 33902794.405 ops/s # Warmup Iteration 14: 36155738.838 ops/s # Warmup Iteration 15: 36206714.982 ops/s # Warmup Iteration 16: 32084735.481 ops/s # Warmup Iteration 17: 34564731.167 ops/s # Warmup Iteration 18: 35168148.516 ops/s # Warmup Iteration 19: 30421286.177 ops/s # Warmup Iteration 20: 31997040.084 ops/s Iteration 1: 35802127.743 ops/s Iteration 2: 30223432.040 ops/s Iteration 3: 33523343.257 ops/s Iteration 4: 35660309.083 ops/s Iteration 5: 35997090.209 ops/s Iteration 6: 31239191.252 ops/s Iteration 7: 36147747.729 ops/s Iteration 8: 36147969.988 ops/s Iteration 9: 36208488.301 ops/s Iteration 10: 29792833.489 ops/s Iteration 11: 34580451.949 ops/s Iteration 12: 36064305.066 ops/s Iteration 13: 36041893.088 ops/s Iteration 14: 35855446.086 ops/s Iteration 15: 31661873.490 ops/s Iteration 16: 35036383.811 ops/s Iteration 17: 35857209.919 ops/s Iteration 18: 36029402.370 ops/s Iteration 19: 35930595.692 ops/s Iteration 20: 31699203.984 ops/s # Run progress: 30.00% complete, ETA 00:19:36 # Fork: 4 of 10 # Warmup Iteration 1: 31296595.515 ops/s # Warmup Iteration 2: 37908633.020 ops/s # Warmup Iteration 3: 35250565.078 ops/s # Warmup Iteration 4: 37752285.871 ops/s # Warmup Iteration 5: 37914294.026 ops/s # Warmup Iteration 6: 38377635.946 ops/s # Warmup Iteration 7: 32733864.039 ops/s # Warmup Iteration 8: 38064547.309 ops/s # Warmup Iteration 9: 36544704.445 ops/s # Warmup Iteration 10: 33882227.479 ops/s # Warmup Iteration 11: 36836752.434 ops/s # Warmup Iteration 12: 38446392.157 ops/s # Warmup Iteration 13: 36364073.955 ops/s # Warmup Iteration 14: 38706702.104 ops/s # Warmup Iteration 15: 38652050.082 ops/s # Warmup Iteration 16: 34322809.290 ops/s # Warmup Iteration 17: 37700500.798 ops/s # Warmup Iteration 18: 38906944.697 ops/s # Warmup Iteration 19: 33195845.787 ops/s # Warmup Iteration 20: 35821942.775 ops/s Iteration 1: 39417175.657 ops/s Iteration 2: 39325998.402 ops/s Iteration 3: 39056076.106 ops/s Iteration 4: 32921279.546 ops/s Iteration 5: 39054221.339 ops/s Iteration 6: 36515839.874 ops/s Iteration 7: 32172960.935 ops/s Iteration 8: 37976219.446 ops/s Iteration 9: 38929526.685 ops/s Iteration 10: 38894878.237 ops/s Iteration 11: 31978400.534 ops/s Iteration 12: 36903626.167 ops/s Iteration 13: 39131870.806 ops/s Iteration 14: 39345626.043 ops/s Iteration 15: 39026205.926 ops/s Iteration 16: 36806664.950 ops/s Iteration 17: 38630575.119 ops/s Iteration 18: 37854064.857 ops/s Iteration 19: 39440319.418 ops/s Iteration 20: 39524804.534 ops/s # Run progress: 40.00% complete, ETA 00:16:54 # Fork: 5 of 10 # Warmup Iteration 1: 31110038.023 ops/s # Warmup Iteration 2: 34412040.058 ops/s # Warmup Iteration 3: 31235963.893 ops/s # Warmup Iteration 4: 30767353.395 ops/s # Warmup Iteration 5: 34439323.107 ops/s # Warmup Iteration 6: 36266948.529 ops/s # Warmup Iteration 7: 36055758.564 ops/s # Warmup Iteration 8: 34585869.673 ops/s # Warmup Iteration 9: 36567490.492 ops/s # Warmup Iteration 10: 35504554.003 ops/s # Warmup Iteration 11: 36179962.205 ops/s # Warmup Iteration 12: 32394762.264 ops/s # Warmup Iteration 13: 35653335.954 ops/s # Warmup Iteration 14: 35620586.786 ops/s # Warmup Iteration 15: 35863809.190 ops/s # Warmup Iteration 16: 36259837.483 ops/s # Warmup Iteration 17: 36302387.266 ops/s # Warmup Iteration 18: 33192738.788 ops/s # Warmup Iteration 19: 36523702.645 ops/s # Warmup Iteration 20: 36629186.036 ops/s Iteration 1: 36694296.125 ops/s Iteration 2: 36680567.770 ops/s Iteration 3: 36495812.469 ops/s Iteration 4: 32343259.943 ops/s Iteration 5: 33671386.030 ops/s Iteration 6: 35555005.646 ops/s Iteration 7: 35211358.656 ops/s Iteration 8: 36081526.871 ops/s Iteration 9: 36107058.585 ops/s Iteration 10: 32729883.053 ops/s Iteration 11: 35724907.378 ops/s Iteration 12: 36499143.866 ops/s Iteration 13: 36603461.237 ops/s Iteration 14: 36631899.844 ops/s Iteration 15: 36121361.065 ops/s Iteration 16: 33671874.478 ops/s Iteration 17: 36083864.464 ops/s Iteration 18: 36667481.469 ops/s Iteration 19: 36722951.627 ops/s Iteration 20: 36566197.534 ops/s # Run progress: 50.00% complete, ETA 00:13:22 # Fork: 6 of 10 # Warmup Iteration 1: 31029872.095 ops/s # Warmup Iteration 2: 37230506.713 ops/s # Warmup Iteration 3: 38809151.616 ops/s # Warmup Iteration 4: 38329927.142 ops/s # Warmup Iteration 5: 38143210.031 ops/s # Warmup Iteration 6: 38809118.052 ops/s # Warmup Iteration 7: 33967885.816 ops/s # Warmup Iteration 8: 38749239.488 ops/s # Warmup Iteration 9: 38579178.021 ops/s # Warmup Iteration 10: 36703103.973 ops/s # Warmup Iteration 11: 38954258.462 ops/s # Warmup Iteration 12: 39112773.031 ops/s # Warmup Iteration 13: 38741746.259 ops/s # Warmup Iteration 14: 39076601.751 ops/s # Warmup Iteration 15: 38986191.092 ops/s # Warmup Iteration 16: 37138531.823 ops/s # Warmup Iteration 17: 37857790.291 ops/s # Warmup Iteration 18: 38843053.603 ops/s # Warmup Iteration 19: 38787113.306 ops/s # Warmup Iteration 20: 35306823.192 ops/s Iteration 1: 37055062.962 ops/s Iteration 2: 38855425.772 ops/s Iteration 3: 35013411.175 ops/s Iteration 4: 35274085.229 ops/s Iteration 5: 38493671.894 ops/s Iteration 6: 36760520.168 ops/s Iteration 7: 37653602.024 ops/s Iteration 8: 37387676.588 ops/s Iteration 9: 38555022.030 ops/s Iteration 10: 28687868.812 ops/s Iteration 11: 27046401.908 ops/s Iteration 12: 29259494.625 ops/s Iteration 13: 26667525.465 ops/s Iteration 14: 25558172.378 ops/s Iteration 15: 35911579.186 ops/s Iteration 16: 36880307.369 ops/s Iteration 17: 38333722.473 ops/s Iteration 18: 34790538.914 ops/s Iteration 19: 36686520.151 ops/s Iteration 20: 37382730.439 ops/s # Run progress: 60.00% complete, ETA 00:10:48 # Fork: 7 of 10 # Warmup Iteration 1: 31057306.449 ops/s # Warmup Iteration 2: 35840428.213 ops/s # Warmup Iteration 3: 34143557.080 ops/s # Warmup Iteration 4: 36667927.955 ops/s # Warmup Iteration 5: 36452958.037 ops/s # Warmup Iteration 6: 38232808.327 ops/s # Warmup Iteration 7: 35551071.320 ops/s # Warmup Iteration 8: 38541440.865 ops/s # Warmup Iteration 9: 36172684.586 ops/s # Warmup Iteration 10: 38633066.168 ops/s # Warmup Iteration 11: 36349123.423 ops/s # Warmup Iteration 12: 38523683.054 ops/s # Warmup Iteration 13: 36124569.851 ops/s # Warmup Iteration 14: 38950543.533 ops/s # Warmup Iteration 15: 38869447.272 ops/s # Warmup Iteration 16: 31093331.577 ops/s # Warmup Iteration 17: 37392337.846 ops/s # Warmup Iteration 18: 39057287.494 ops/s # Warmup Iteration 19: 38995112.909 ops/s # Warmup Iteration 20: 39066421.796 ops/s Iteration 1: 32127004.028 ops/s Iteration 2: 37499140.095 ops/s Iteration 3: 36539432.705 ops/s Iteration 4: 33907894.925 ops/s Iteration 5: 38054763.625 ops/s Iteration 6: 38733118.320 ops/s Iteration 7: 38622694.446 ops/s Iteration 8: 32751376.012 ops/s Iteration 9: 35019544.916 ops/s Iteration 10: 38858825.000 ops/s Iteration 11: 38910765.208 ops/s Iteration 12: 31043691.167 ops/s Iteration 13: 35840369.231 ops/s Iteration 14: 37385724.264 ops/s Iteration 15: 39023580.476 ops/s Iteration 16: 38741774.638 ops/s Iteration 17: 31542116.877 ops/s Iteration 18: 34719270.004 ops/s Iteration 19: 38908161.679 ops/s Iteration 20: 38963054.209 ops/s # Run progress: 70.00% complete, ETA 00:08:05 # Fork: 8 of 10 # Warmup Iteration 1: 30003148.195 ops/s # Warmup Iteration 2: 35462032.545 ops/s # Warmup Iteration 3: 33071571.604 ops/s # Warmup Iteration 4: 35387917.187 ops/s # Warmup Iteration 5: 34586963.687 ops/s # Warmup Iteration 6: 35448683.915 ops/s # Warmup Iteration 7: 32807613.797 ops/s # Warmup Iteration 8: 35915731.511 ops/s # Warmup Iteration 9: 33149556.434 ops/s # Warmup Iteration 10: 35195706.499 ops/s # Warmup Iteration 11: 33596358.520 ops/s # Warmup Iteration 12: 35353239.403 ops/s # Warmup Iteration 13: 33508839.669 ops/s # Warmup Iteration 14: 35930589.401 ops/s # Warmup Iteration 15: 35941526.705 ops/s # Warmup Iteration 16: 30714684.903 ops/s # Warmup Iteration 17: 32933652.946 ops/s # Warmup Iteration 18: 34942977.718 ops/s # Warmup Iteration 19: 35976691.542 ops/s # Warmup Iteration 20: 30498957.215 ops/s Iteration 1: 35631149.027 ops/s Iteration 2: 36122657.348 ops/s Iteration 3: 29257602.641 ops/s Iteration 4: 31964437.318 ops/s Iteration 5: 35123171.218 ops/s Iteration 6: 35988762.826 ops/s Iteration 7: 33333800.776 ops/s Iteration 8: 33839869.951 ops/s Iteration 9: 35877741.832 ops/s Iteration 10: 35977710.893 ops/s Iteration 11: 35928268.472 ops/s Iteration 12: 31871331.622 ops/s Iteration 13: 33520435.719 ops/s Iteration 14: 35271300.135 ops/s Iteration 15: 36042505.102 ops/s Iteration 16: 36026785.491 ops/s Iteration 17: 29606290.243 ops/s Iteration 18: 34555390.595 ops/s Iteration 19: 35040209.776 ops/s Iteration 20: 35526038.193 ops/s # Run progress: 80.00% complete, ETA 00:05:25 # Fork: 9 of 10 # Warmup Iteration 1: 30780286.932 ops/s # Warmup Iteration 2: 38554787.285 ops/s # Warmup Iteration 3: 35499729.607 ops/s # Warmup Iteration 4: 38123652.259 ops/s # Warmup Iteration 5: 38287045.843 ops/s # Warmup Iteration 6: 38909349.390 ops/s # Warmup Iteration 7: 34160614.025 ops/s # Warmup Iteration 8: 36714969.322 ops/s # Warmup Iteration 9: 35724081.900 ops/s # Warmup Iteration 10: 34344266.878 ops/s # Warmup Iteration 11: 36691467.403 ops/s # Warmup Iteration 12: 38395178.914 ops/s # Warmup Iteration 13: 37303043.548 ops/s # Warmup Iteration 14: 39117495.343 ops/s # Warmup Iteration 15: 38410409.515 ops/s # Warmup Iteration 16: 36557299.657 ops/s # Warmup Iteration 17: 39137443.252 ops/s # Warmup Iteration 18: 38967960.625 ops/s # Warmup Iteration 19: 34626900.596 ops/s # Warmup Iteration 20: 37683007.712 ops/s Iteration 1: 39077842.127 ops/s Iteration 2: 34347095.059 ops/s Iteration 3: 37355881.353 ops/s Iteration 4: 39049529.112 ops/s Iteration 5: 34227298.880 ops/s Iteration 6: 34912978.620 ops/s Iteration 7: 36002838.823 ops/s Iteration 8: 39311390.901 ops/s Iteration 9: 33107477.135 ops/s Iteration 10: 37288472.263 ops/s Iteration 11: 36867078.249 ops/s Iteration 12: 38982164.040 ops/s Iteration 13: 33384786.397 ops/s Iteration 14: 35745764.678 ops/s Iteration 15: 36567956.300 ops/s Iteration 16: 38434831.820 ops/s Iteration 17: 38847394.679 ops/s Iteration 18: 34190903.091 ops/s Iteration 19: 36003870.959 ops/s Iteration 20: 36590393.071 ops/s # Run progress: 90.00% complete, ETA 00:02:44 # Fork: 10 of 10 # Warmup Iteration 1: 28889703.379 ops/s # Warmup Iteration 2: 38603492.577 ops/s # Warmup Iteration 3: 34043794.352 ops/s # Warmup Iteration 4: 37985769.911 ops/s # Warmup Iteration 5: 37337263.671 ops/s # Warmup Iteration 6: 38294358.387 ops/s # Warmup Iteration 7: 33591758.046 ops/s # Warmup Iteration 8: 35677991.195 ops/s # Warmup Iteration 9: 35626708.813 ops/s # Warmup Iteration 10: 35001658.546 ops/s # Warmup Iteration 11: 34895453.027 ops/s # Warmup Iteration 12: 38305387.208 ops/s # Warmup Iteration 13: 31137358.688 ops/s # Warmup Iteration 14: 38642485.864 ops/s # Warmup Iteration 15: 38586100.872 ops/s # Warmup Iteration 16: 35486297.661 ops/s # Warmup Iteration 17: 38936456.857 ops/s # Warmup Iteration 18: 38776288.591 ops/s # Warmup Iteration 19: 38495102.028 ops/s # Warmup Iteration 20: 34583854.305 ops/s Iteration 1: 35593487.267 ops/s Iteration 2: 36719438.475 ops/s Iteration 3: 33895023.416 ops/s Iteration 4: 34812011.249 ops/s Iteration 5: 38906937.001 ops/s Iteration 6: 38182707.915 ops/s Iteration 7: 35294815.634 ops/s Iteration 8: 37426940.293 ops/s Iteration 9: 37385447.731 ops/s Iteration 10: 38628608.563 ops/s Iteration 11: 34323084.222 ops/s Iteration 12: 35285335.362 ops/s Iteration 13: 36995943.696 ops/s Iteration 14: 38944858.911 ops/s Iteration 15: 38516334.356 ops/s Iteration 16: 34389934.288 ops/s Iteration 17: 35448198.837 ops/s Iteration 18: 38375706.441 ops/s Iteration 19: 38841034.693 ops/s Iteration 20: 38600776.337 ops/s Result "org.sample.ByteBufferBenchmark.benchmark_byte_buffer_put": 35604933.518 ?(99.9%) 654975.515 ops/s [Average] (min, avg, max) = (25558172.378, 35604933.518, 39524804.534), stdev = 2773207.341 CI (99.9%): [34949958.003, 36259909.033] (assumes normal distribution) # Run complete. Total time: 00:27:51 Benchmark Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 35604933.518 ? 654975.515 ops/s ... So a performance degradation of roughly 7%. Regards, Ben Walsh From: Paul Sandoz To: Ben Walsh Cc: core-libs-dev Date: 01/02/2018 22:50 Subject: Re: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup Hi Ben, I don?t think you require the fence in all the cases you have currently placed it e.g. here for example $memtype$ y = $toBits$(x); UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); + Reference.reachabilityFence(this); return this; since ?this? is being returned it will be kept live during the unsafe access. Would you mind providing a JMH benchmark and results for the performance with and without the fence for say a put and/or a get. I would like to understand the performance impact on HotSpot, this is one reason why we have yet to add such fences as it will likely impact performance. At the moment we are relying on the method not being inlined, which is an expedient technique to make it functional and keep a reference alive but not necessarily optimal for usages in DBB. For more details see: https://bugs.openjdk.java.net/browse/JDK-8169605 https://bugs.openjdk.java.net/browse/JDK-8149610 Thanks, Paul. On Feb 1, 2018, at 6:55 AM, Ben Walsh wrote: This contribution forms a partial solution to the problem detailed here - http://thevirtualmachinist.blogspot.ca/2011/07/subtle-issue-of-reachability.html . In this context, this contribution provides "markers" such that a suitably "aware" compiler can reduce the chances of such a problem occurring with each of the DirectBuffer objects and MappedByteBuffer object. The reachabilityFences may prevent crashes / exceptions due to cleaning up the backing memory before the user has finished using the pointer. Any compiler not suitably "aware" could be modified to make use of the "markers". I would like to pair with a sponsor to contribute this patch ... ------------------------------------------------------------------------------------------------------------------- diff -r d51e64840b4f src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer-bin.java.template Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -33,15 +33,21 @@ private $type$ get$Type$(long a) { $memtype$ x = UNSAFE.get$Memtype$Unaligned(null, a, bigEndian); - return $fromBits$(x); + $type$ y = $fromBits$(x); + Reference.reachabilityFence(this); + return y; } public $type$ get$Type$() { - return get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); + $type$ y = get$Type$(ix(nextGetIndex($BYTES_PER_VALUE$))); + Reference.reachabilityFence(this); + return y; } public $type$ get$Type$(int i) { - return get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); + $type$ y = get$Type$(ix(checkIndex(i, $BYTES_PER_VALUE$))); + Reference.reachabilityFence(this); + return y; } #end[rw] @@ -50,6 +56,7 @@ #if[rw] $memtype$ y = $toBits$(x); UNSAFE.put$Memtype$Unaligned(null, a, y, bigEndian); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); diff -r d51e64840b4f src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -28,6 +28,7 @@ package java.nio; import java.io.FileDescriptor; +import java.lang.ref.Reference; import jdk.internal.misc.VM; import jdk.internal.ref.Cleaner; import sun.nio.ch.DirectBuffer; @@ -312,6 +313,7 @@ public $Type$Buffer put($type$ x) { #if[rw] UNSAFE.put$Swaptype$(ix(nextPutIndex()), $swap$($toBits$(x))); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); @@ -321,6 +323,7 @@ public $Type$Buffer put(int i, $type$ x) { #if[rw] UNSAFE.put$Swaptype$(ix(checkIndex(i)), $swap$($toBits$(x))); + Reference.reachabilityFence(this); return this; #else[rw] throw new ReadOnlyBufferException(); @@ -347,6 +350,7 @@ if (srem > rem) throw new BufferOverflowException(); UNSAFE.copyMemory(sb.ix(spos), ix(pos), (long)srem << $LG_BYTES_PER_VALUE$); + Reference.reachabilityFence(this); sb.position(spos + srem); position(pos + srem); } else if (src.hb != null) { @@ -413,6 +417,7 @@ int rem = (pos <= lim ? lim - pos : 0); UNSAFE.copyMemory(ix(pos), ix(0), (long)rem << $LG_BYTES_PER_VALUE$); + Reference.reachabilityFence(this); position(rem); limit(capacity()); discardMark(); @@ -505,6 +510,7 @@ void _put(int i, byte b) { // package-private #if[rw] UNSAFE.putByte(address + i, b); + Reference.reachabilityFence(this); #else[rw] throw new ReadOnlyBufferException(); #end[rw] diff -r d51e64840b4f src/java.base/share/classes/java/nio/MappedByteBuffer.java --- a/src/java.base/share/classes/java/nio/MappedByteBuffer.java Wed Jan 31 12:04:53 2018 +0800 +++ b/src/java.base/share/classes/java/nio/MappedByteBuffer.java Thu Feb 01 11:30:10 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -26,6 +26,7 @@ package java.nio; import java.io.FileDescriptor; +import java.lang.ref.Reference; import jdk.internal.misc.Unsafe; @@ -164,6 +165,7 @@ // is computed as we go along to prevent the compiler from otherwise // considering the loop as dead code. Unsafe unsafe = Unsafe.getUnsafe(); + Reference.reachabilityFence(this); int ps = Bits.pageSize(); int count = Bits.pageCount(length); long a = mappingAddress(offset); ------------------------------------------------------------------------------------------------------------------- Regards, Ben Walsh Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Roger.Riggs at Oracle.com Thu Feb 8 16:26:30 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 8 Feb 2018 11:26:30 -0500 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> Message-ID: <71fa9742-c18a-abe2-3d5f-ccef7de9b0fd@Oracle.com> Hi Robin, Saving useful information about failures during shutdown is useful. Can the shutdown of JFR be put off? Is the use of shutdown hook still necessary? Thanks, Roger On 2/8/2018 10:50 AM, Robin Westberg wrote: > Hi David, > >> On 8 Feb 2018, at 04:28, David Holmes wrote: ... >> I had an initial look through this. >> >> To be honest I don't like it. We seem to have to record little bits of information all over the place just so they can be reported by the shutdown event. It seems untidy. :( >> >> Can you rename _starting_thread to _main_thread please. The use of "starting" in thread.hpp/cpp is really a naming anomaly. The main thread is the thread that loads the JVM. And that would be consistent with set_exception_in_main_thread. >> >> Though why do we care if the main thread exited with an unhandled exception? And if we do care, why do we only care when the shutdown reason is ""No remaining non-daemon Java threads"? >> >> I don't like the need to add os::get_abort_exit_code. Do we really need it? What useful information does it convey? > Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. > > Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. > >> It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. > In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). > > Best regards, > Robin > >> Let's see what others think ... >> >> Thanks, >> David >> >> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>> Hi all, >>> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>> Best regards, >>> Robin From paul.sandoz at oracle.com Thu Feb 8 16:54:15 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 8 Feb 2018 08:54:15 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: Hi Ben, Thanks. I anticipated a performance hit but not necessarily a 10x. Without looking at the generated code of the benchmark method it is hard to be sure [*], but i believe the fence is interfering with loop unrolling and/or vectorization, the comparative differences between byte and int may be related to vectorization (for byte there may be less or limited support for vectorization). How about we now try another experiment commenting out the @DontInline on the fence method and re-run the benchmarks. From Peter?s observations and Vladimir?s analysis we should be able to remove that, or even, contrary to what we initial expected when adding this feature, change to @ForceInline! Thanks, Paul. [*] If you are running on linux you can use the excellent JMH perfasm feature to dump the hot parts of HotSpots generated code. > On Feb 8, 2018, at 8:22 AM, Ben Walsh wrote: > > Hi Paul, > > Following up with the requested loop and vectorization benchmarks ... > > > (Do the vectorization benchmark results imply that the Hotspot compiler > has been unable to perform the vectorization optimisation due to the > presence of the reachabilityFence ?) > > > ----------------------------------------------------------------------------------------------------------------------- > > > Loop Benchmarking > ---- ------------ > > package org.sample; > > import org.openjdk.jmh.annotations.Benchmark; > import org.openjdk.jmh.annotations.Level; > import org.openjdk.jmh.annotations.Param; > import org.openjdk.jmh.annotations.Scope; > import org.openjdk.jmh.annotations.Setup; > import org.openjdk.jmh.annotations.State; > > import java.nio.ByteBuffer; > > @State(Scope.Benchmark) > public class ByteBufferBenchmark { > > @Param({"1", "10", "100", "1000", "10000"}) > public int L; > > @State(Scope.Benchmark) > public static class ByteBufferContainer { > > ByteBuffer bb; > > @Setup(Level.Invocation) > public void initByteBuffer() { > bb = ByteBuffer.allocateDirect(10000); > } > > ByteBuffer getByteBuffer() { > return bb; > } > } > > @Benchmark > public ByteBuffer benchmark_byte_buffer_put(ByteBufferContainer bbC) { > > ByteBuffer bb = bbC.getByteBuffer(); > > for (int i = 0; i < L; i++) { > bb.put((byte)i); > } > > return bb; > } > > } > > > Without Changes > > Benchmark (L) Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29303145.752 ? 635979.750 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 24260859.017 ? 528891.303 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 8512366.637 ? 136615.070 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1323756.037 ? 21485.369 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 145965.305 ? 1301.469 ops/s > > > With Changes > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 28893540.122 ? 754554.747 ops/s -1.398% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 15317696.355 ? 231621.608 ops/s -36.863% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 2546599.578 ? 32136.873 ops/s -70.084% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 288832.514 ? 3854.522 ops/s -78.181% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 29747.386 > ? 214.831 ops/s -79.620% > > > ----------------------------------------------------------------------------------------------------------------------- > > > Vectorization Benchmarking > ------------- ------------ > > package org.sample; > > import org.openjdk.jmh.annotations.Benchmark; > import org.openjdk.jmh.annotations.Level; > import org.openjdk.jmh.annotations.Param; > import org.openjdk.jmh.annotations.Scope; > import org.openjdk.jmh.annotations.Setup; > import org.openjdk.jmh.annotations.State; > > import java.nio.ByteBuffer; > > @State(Scope.Benchmark) > public class ByteBufferBenchmark { > > @Param({"1", "10", "100", "1000", "10000"}) > public int L; > > @State(Scope.Benchmark) > public static class ByteBufferContainer { > > ByteBuffer bb; > > @Setup(Level.Invocation) > public void initByteBuffer() { > bb = ByteBuffer.allocateDirect(4 * 10000); > > for (int i = 0; i < 10000; i++) { > bb.putInt(i); > } > } > > ByteBuffer getByteBuffer() { > return bb; > } > > } > > @Benchmark > public int benchmark_byte_buffer_put(ByteBufferContainer bbC) { > > ByteBuffer bb = bbC.getByteBuffer(); > > bb.position(0); > > int sum = 0; > > for (int i = 0; i < L; i++) { > sum += bb.getInt(); > } > > return sum; > > } > > } > > > Without Changes > > Benchmark (L) Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29677205.748 ? 544721.142 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 18219951.454 ? 320724.793 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 7767650.826 ? 121798.910 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1646075.010 ? 9804.499 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 183489.418 ? 1355.967 ops/s > > > With Changes > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 15230086.695 ? 390174.190 ops/s -48.681% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 8126310.728 ? 123661.342 ops/s -55.399% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 1582699.233 ? 7278.744 ops/s -79.624% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 179726.465 ? 802.333 ops/s -89.082% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 18327.049 > ? 9.506 ops/s -90.012% > > > > NB : For reference - for this and previous benchmarking results ... > > "Without Changes" and "With Changes" - java -version ... > > openjdk version "10-internal" 2018-03-20 > OpenJDK Runtime Environment (build 10-internal+0-adhoc.walshbp.jdk) > OpenJDK 64-Bit Server VM (build 10-internal+0-adhoc.walshbp.jdk, mixed > mode) > > > ----------------------------------------------------------------------------------------------------------------------- > > > Regards, > Ben Walsh > From yumin.qi at gmail.com Thu Feb 8 17:35:45 2018 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 8 Feb 2018 09:35:45 -0800 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> References: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> Message-ID: HI, Alan As in your email to RFR of 8194154 which now has a new fix in canonicalize_md.c, switch back to discuss solution here again. The current fix is not in java, instead, I put it in C function. In the function before the fix, it assumes no more "//" pattern in the string so failed to get correct substrings with the case as the test case. The fix should not cause other problem since it does double check if double or more slashes in sequential. The APIs indicates 'user.dir' etc system property should not be rewritten, but did not prevent modifying them in practise. Thanks Yumin On Tue, Jan 2, 2018 at 6:09 PM, Wenqian Pei wrote: > > Can you please review this patch? > > Thanks in advance! > > Wenqian Pei > ------------------------------------------------------------------????Alan > Bateman ?????2018?1?3?(???) 01:30???????(??) < > wenqian.peiwq at alibaba-inc.com>; core-libs-dev net>? ?????(??) ; ???(??) < > sanhong.lsh at alibaba-inc.com>? ??Re: RFR: 8194154 patch for crash at > File.getCanonicalPath() > On 25/12/2017 09:40, Wenqian Pei wrote: > > Hi: > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8194154 > > > > We found that if user defines -Duser.dir like "/ > home/a/b/c/", jvm will crash at File.getCanonicalPath() in > java process bootstrap (before invoking user's java code). The native > implematation of canonicalize_md.c:collapsible(char *names) > has problem in processing double '/', parameter 'names' > need normalized before JNI_CALL. > > > user.dir is a "read-only" property and should never be changed on the > command-line or in a running VM (it breaks APIs in other areas to have > user.dir be different to the actual working directory). Someday we need > to figure out how to enforce this. > > In the mean-time, the runtime shouldn't crash so I agree it > should be fixed. > > -Alan. > From alexandre.iline at oracle.com Thu Feb 8 18:40:35 2018 From: alexandre.iline at oracle.com (Alexandre (Shura) Iline) Date: Thu, 8 Feb 2018 10:40:35 -0800 Subject: RFR 8178342: Missing @modules in jdk/jdk/nio/zipfs Message-ID: Hi. Can you took a quick look on this fix. While adding missing @modules declaration in jdk/nio/zipfs/ZeroDate.java, I noticed that @modules declaration is not in the right location in other tests. Per the agreement it should come before any @run. Bug: https://bugs.openjdk.java.net/browse/JDK-8178342 Webrev: http://cr.openjdk.java.net/~shurailine/8178342/webrev.00/ Thank you. Shura From joe.darcy at oracle.com Thu Feb 8 18:59:35 2018 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 8 Feb 2018 10:59:35 -0800 Subject: JDK 11 RFR of JDK-8196995: java.lang.Character should not state UTF-16 encoding is used for strings In-Reply-To: <66d19661-b8f3-82b6-e4ea-a72ce43ff0cc@oracle.com> References: <66d19661-b8f3-82b6-e4ea-a72ce43ff0cc@oracle.com> Message-ID: <9dca6d72-4f4b-eb84-145e-9de145198b96@oracle.com> Hello, On 2/8/2018 3:53 AM, Alan Bateman wrote: > On 07/02/2018 22:12, joe darcy wrote: >> Hello, >> >> Text in java.lang.Character states a UTF-16 character encoding is >> used for java.lang.String. While was true for many years, it is not >> necessarily true and not true in practice as of JDK 9 due to the >> improvements from JEP 254: Compact Strings. >> >> The statement about the encoding should be corrected. >> >> Please review the patch below which does this. (I've formatted the >> patch so that the change is text is made clear; I'll re-flow the >> paragraph before pushing. > I'm not sure that this is worth changing. You could replace "classes" > with "API" and add a note to say that an implementation may use an > more optimization representation but I don't think it's really needed. > In response to this feedback and others, how about: ???? [...] The Java ? * platform uses the UTF-16 representation in {@code char} arrays and - * in the {@code String} and {@code StringBuffer} classes. In + * presents a UTF-16 model in the string-related API. IMO anyway, I think saying "uses a UTF-16 representation for String" is at best misleading with the current implementation since 8 != 16 for the compressed representation is used for all Latin-1 strings. Thanks, -Joe From xueming.shen at oracle.com Thu Feb 8 19:13:33 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 08 Feb 2018 11:13:33 -0800 Subject: JDK 11 RFR of JDK-8196995: java.lang.Character should not state UTF-16 encoding is used for strings In-Reply-To: <9dca6d72-4f4b-eb84-145e-9de145198b96@oracle.com> References: <66d19661-b8f3-82b6-e4ea-a72ce43ff0cc@oracle.com> <9dca6d72-4f4b-eb84-145e-9de145198b96@oracle.com> Message-ID: <5A7CA15D.6030406@oracle.com> On 2/8/18, 10:59 AM, joe darcy wrote: > Hello, > > On 2/8/2018 3:53 AM, Alan Bateman wrote: >> On 07/02/2018 22:12, joe darcy wrote: >>> Hello, >>> >>> Text in java.lang.Character states a UTF-16 character encoding is >>> used for java.lang.String. While was true for many years, it is not >>> necessarily true and not true in practice as of JDK 9 due to the >>> improvements from JEP 254: Compact Strings. >>> >>> The statement about the encoding should be corrected. >>> >>> Please review the patch below which does this. (I've formatted the >>> patch so that the change is text is made clear; I'll re-flow the >>> paragraph before pushing. >> I'm not sure that this is worth changing. You could replace "classes" >> with "API" and add a note to say that an implementation may use an >> more optimization representation but I don't think it's really needed. >> > > In response to this feedback and others, how about: > > [...] The Java > * platform uses the UTF-16 representation in {@code char} arrays and > - * in the {@code String} and {@code StringBuffer} classes. In > + * presents a UTF-16 model in the string-related API. > > IMO anyway, I think saying "uses a UTF-16 representation for String" > is at best misleading with the current implementation since 8 != 16 > for the compressed representation is used for all Latin-1 strings. > Well, encoding/charset is the concept of a mapping between a character and a corresponding code point value. We are still using the UTF16 encoding scheme to represent a character in jvm. How to represent/store that UTF16 code point value in String class is an implementation detail. A 16-bit for "char" and a 1-byte for "latin1" (still in Unicode charset) + 2 byte for the rest in String class. As I said in my previous email. The mention of 8859-1 in the JEP might cause the confusion. At early stage of the project we were really experimenting on using different "encoding", including utf8. But the project ended up with staying with UTF-16, with a "customized/compressed" storage mechanism to store the UTF16 codepoint value. -Sherman From mandy.chung at oracle.com Thu Feb 8 21:57:40 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 8 Feb 2018 13:57:40 -0800 Subject: RFR 8178342: Missing @modules in jdk/jdk/nio/zipfs In-Reply-To: References: Message-ID: <6692bb75-a248-d6b5-0758-afd767740060@oracle.com> +1 Mandy On 2/8/18 10:40 AM, Alexandre (Shura) Iline wrote: > Hi. > > Can you took a quick look on this fix. While adding missing @modules declaration in jdk/nio/zipfs/ZeroDate.java, I noticed that @modules declaration is not in the right location in other tests. Per the agreement it should come before any @run. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8178342 > Webrev: http://cr.openjdk.java.net/~shurailine/8178342/webrev.00/ > > Thank you. > > Shura > From martinrb at google.com Thu Feb 8 22:20:42 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 8 Feb 2018 14:20:42 -0800 Subject: JDK 11 RFR of JDK-8196995: java.lang.Character should not state UTF-16 encoding is used for strings In-Reply-To: <9dca6d72-4f4b-eb84-145e-9de145198b96@oracle.com> References: <66d19661-b8f3-82b6-e4ea-a72ce43ff0cc@oracle.com> <9dca6d72-4f4b-eb84-145e-9de145198b96@oracle.com> Message-ID: Instead of saying something specific about char[], String etc you can be more general and say something about the entire Java Platform. "The Java Platform generally processes human language text represented as sequences of {@code char} values in the UTF-16 encoding of Unicode" But doing this well would involve a greater overhaul of the class doc. Character will always be confusing because it is ambiguous about whether referring to a Unicode character or a UTF-16 code unit. On Thu, Feb 8, 2018 at 10:59 AM, joe darcy wrote: > Hello, > > On 2/8/2018 3:53 AM, Alan Bateman wrote: > >> On 07/02/2018 22:12, joe darcy wrote: >> >>> Hello, >>> >>> Text in java.lang.Character states a UTF-16 character encoding is used >>> for java.lang.String. While was true for many years, it is not necessarily >>> true and not true in practice as of JDK 9 due to the improvements from JEP >>> 254: Compact Strings. >>> >>> The statement about the encoding should be corrected. >>> >>> Please review the patch below which does this. (I've formatted the patch >>> so that the change is text is made clear; I'll re-flow the paragraph before >>> pushing. >>> >> I'm not sure that this is worth changing. You could replace "classes" >> with "API" and add a note to say that an implementation may use an more >> optimization representation but I don't think it's really needed. >> >> > In response to this feedback and others, how about: > > [...] The Java > * platform uses the UTF-16 representation in {@code char} arrays and > - * in the {@code String} and {@code StringBuffer} classes. In > + * presents a UTF-16 model in the string-related API. > > IMO anyway, I think saying "uses a UTF-16 representation for String" is at > best misleading with the current implementation since 8 != 16 for the > compressed representation is used for all Latin-1 strings. > > Thanks, > > -Joe > From joe.darcy at oracle.com Thu Feb 8 22:28:01 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Thu, 08 Feb 2018 14:28:01 -0800 Subject: JDK 11 RFR of JDK-8196995: java.lang.Character should not state UTF-16 encoding is used for strings In-Reply-To: <5A7CA15D.6030406@oracle.com> References: <66d19661-b8f3-82b6-e4ea-a72ce43ff0cc@oracle.com> <9dca6d72-4f4b-eb84-145e-9de145198b96@oracle.com> <5A7CA15D.6030406@oracle.com> Message-ID: <5A7CCEF1.5000404@oracle.com> Since other people who work more closely in the area than me don't seem to find the wording confusing or misleading, I'll just close out the bug. Thanks, -Joe On 2/8/2018 11:13 AM, Xueming Shen wrote: > On 2/8/18, 10:59 AM, joe darcy wrote: >> Hello, >> >> On 2/8/2018 3:53 AM, Alan Bateman wrote: >>> On 07/02/2018 22:12, joe darcy wrote: >>>> Hello, >>>> >>>> Text in java.lang.Character states a UTF-16 character encoding is >>>> used for java.lang.String. While was true for many years, it is not >>>> necessarily true and not true in practice as of JDK 9 due to the >>>> improvements from JEP 254: Compact Strings. >>>> >>>> The statement about the encoding should be corrected. >>>> >>>> Please review the patch below which does this. (I've formatted the >>>> patch so that the change is text is made clear; I'll re-flow the >>>> paragraph before pushing. >>> I'm not sure that this is worth changing. You could replace >>> "classes" with "API" and add a note to say that an implementation >>> may use an more optimization representation but I don't think it's >>> really needed. >>> >> >> In response to this feedback and others, how about: >> >> [...] The Java >> * platform uses the UTF-16 representation in {@code char} arrays and >> - * in the {@code String} and {@code StringBuffer} classes. In >> + * presents a UTF-16 model in the string-related API. >> >> IMO anyway, I think saying "uses a UTF-16 representation for String" >> is at best misleading with the current implementation since 8 != 16 >> for the compressed representation is used for all Latin-1 strings. >> > > Well, encoding/charset is the concept of a mapping between a character > and a corresponding > code point value. We are still using the UTF16 encoding scheme to > represent a character in > jvm. How to represent/store that UTF16 code point value in String > class is an implementation > detail. A 16-bit for "char" and a 1-byte for "latin1" (still in > Unicode charset) + 2 byte for the > rest in String class. > > As I said in my previous email. The mention of 8859-1 in the JEP might > cause the confusion. > At early stage of the project we were really experimenting on using > different "encoding", including > utf8. But the project ended up with staying with UTF-16, with a > "customized/compressed" storage > mechanism to store the UTF16 codepoint value. > > -Sherman > From huizhe.wang at oracle.com Fri Feb 9 00:47:49 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 8 Feb 2018 16:47:49 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> Message-ID: Hi all, The CSR for the enhancement is now approved. Thanks Joe! The webrev has been updated accordingly. Please let me know if you have any further comment on the implementation. JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ Thanks, Joe On 2/2/2018 12:46 PM, Joe Wang wrote: > Thanks Jason. Will update that accordingly. > > Best, > Joe > > On 2/2/2018 11:22 AM, Jason Mehrens wrote: >> Joe, >> >> The identity check in CS.compare makes sense.? However, it won't be >> null hostile if we call CS.compare(null, null) and that doesn't seem >> right. >> Usually when writing comparator classes I end up with: >> === >> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >> ???? return 0; >> } >> === >> >> Jason >> ________________________________________ >> From: core-libs-dev on >> behalf of Joe Wang >> Sent: Friday, February 2, 2018 1:01 PM >> To: core-libs-dev >> Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, >> StringBuilder, and StringBuffer >> >> Hi, >> >> Thanks all for comments and suggestions. I've updated the webrev. Please >> review. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> Thanks, >> Joe >> >> On 1/31/2018 9:31 PM, Joe Wang wrote: >>> Hi Tagir, >>> >>> Thanks for the comment. I will consider adding that to the javadoc >>> emphasizing that the comparison is performed from 0 to length() - 1 of >>> the two sequences. >>> >>> Best, >>> Joe >>> >>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>> Hello! >>>> >>>> An AbstractStringBuilder#compareTo implementation is wrong. You cannot >>>> simply compare the whole byte array. Here's the test-case: >>>> >>>> public class Test { >>>> ??? public static void main(String[] args) { >>>> ????? StringBuilder sb1 = new StringBuilder("test1"); >>>> ????? StringBuilder sb2 = new StringBuilder("test2"); >>>> ????? sb1.setLength(4); >>>> ????? sb2.setLength(4); >>>> ????? System.out.println(sb1.compareTo(sb2)); >>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>> ??? } >>>> } >>>> >>>> We truncated the stringbuilders making their content equal, so >>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >>>> the original content (before the truncation) as truncation, of course, >>>> does not zero the truncated bytes, neither does it reallocate the >>>> array (unless explicitly asked via trimToSize). >>>> >>>> With best regards, >>>> Tagir Valeev. >>>> >>>> >>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>> wrote: >>>>> Hi, >>>>> >>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>> StringBuffer. >>>>> >>>>> The Comparable implementations for StringBuilder/Buffer are similar >>>>> to that >>>>> of String, allowing comparison operations between two >>>>> StringBuilders/Buffers, e.g. >>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>> method >>>>> 'compare' is added instead of implementing the Comparable interface. >>>>> This >>>>> 'compare' method may take CharSequence implementations such as >>>>> String, >>>>> StringBuilder and StringBuffer, making it possible to perform >>>>> comparison >>>>> among them. The previous example for example is equivalent to >>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>> >>>>> Tests for java.base have been independent from each other. The new >>>>> tests are >>>>> therefore created to have no dependency on each other or sharing any >>>>> code. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>> >>>>> Thanks, >>>>> Joe > From martinrb at google.com Fri Feb 9 01:19:31 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 8 Feb 2018 17:19:31 -0800 Subject: RFR: jsr166 jdk integration 2018-02 Message-ID: http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html 8190324: ThreadPoolExecutor should not specify a dependency on finalization http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ThreadPoolExecutor-finalize/index.html https://bugs.openjdk.java.net/browse/JDK-8190324 Another set of uninteresting changes, some suggested by intellij: 8195590: Miscellaneous changes imported from jsr166 CVS 2018-02 http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html https://bugs.openjdk.java.net/browse/JDK-8195590 From david.holmes at oracle.com Fri Feb 9 03:39:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Feb 2018 13:39:09 +1000 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: On 9/02/2018 11:19 AM, Martin Buchholz wrote: > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > > 8190324: ThreadPoolExecutor should not specify a dependency on finalization > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ThreadPoolExecutor-finalize/index.html > https://bugs.openjdk.java.net/browse/JDK-8190324 Looks okay. > Another set of uninteresting changes, some suggested by intellij: > > 8195590: Miscellaneous changes imported from jsr166 CVS 2018-02 > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html > https://bugs.openjdk.java.net/browse/JDK-8195590 Hmmmm ... a summary of changes in the bug report would be nice. I don't find the addition of reachability fences uninteresting though. Why are they needed in those specific cases? Thanks, David From martinrb at google.com Fri Feb 9 04:07:02 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 8 Feb 2018 20:07:02 -0800 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: On Thu, Feb 8, 2018 at 7:39 PM, David Holmes wrote: > On 9/02/2018 11:19 AM, Martin Buchholz wrote: > >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integr >> ation/overview.html >> >> 8190324: ThreadPoolExecutor should not specify a dependency on >> finalization >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integr >> ation/ThreadPoolExecutor-finalize/index.html >> https://bugs.openjdk.java.net/browse/JDK-8190324 >> > > Looks okay. > > Another set of uninteresting changes, some suggested by intellij: >> >> 8195590: Miscellaneous changes imported from jsr166 CVS 2018-02 >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integr >> ation/miscellaneous/index.html >> https://bugs.openjdk.java.net/browse/JDK-8195590 >> > > Hmmmm ... a summary of changes in the bug report would be nice. > > Done. > I don't find the addition of reachability fences uninteresting though. Why > are they needed in those specific cases? > perhaps the standard should be for every method of a class with a finalizer to have reachabilityFence(this) at the end? Imagine newSingleThreadExecutor().execute(someTask); Is is unlikely but possible for the "instant garbage" executor to be finalized while super.execute is in progress, causing failure with RejectedExecutionException because the delegatee is already shutdown. From david.holmes at oracle.com Fri Feb 9 04:22:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Feb 2018 14:22:01 +1000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> Message-ID: <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> Hi Robin, On 9/02/2018 1:50 AM, Robin Westberg wrote: > Hi David, > >> On 8 Feb 2018, at 04:28, David Holmes wrote: >> >> Hi Robin, >> >> Adding in hotspot-runtime-dev as all the hotspot changes belong to runtime. > > Thanks, sorry about that.. > >> I had an initial look through this. >> >> To be honest I don't like it. We seem to have to record little bits of information all over the place just so they can be reported by the shutdown event. It seems untidy. :( >> >> Can you rename _starting_thread to _main_thread please. The use of "starting" in thread.hpp/cpp is really a naming anomaly. The main thread is the thread that loads the JVM. And that would be consistent with set_exception_in_main_thread. >> >> Though why do we care if the main thread exited with an unhandled exception? And if we do care, why do we only care when the shutdown reason is ""No remaining non-daemon Java threads"? >> >> I don't like the need to add os::get_abort_exit_code. Do we really need it? What useful information does it convey? > > Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. Yes and that particular example of launcher behaviour is an archaic throwback to C programming - where end of "main" means end of the program - IMHO. I don't think it is of interest - and certainly not worth jumping through so many hoops to make the info available. > Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. That sounds good to me. :) >> It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. > > In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). Yes that is useful. But I find the need to code things the way they are, unfortunate. But given current constraints, so be it. Thanks, David > Best regards, > Robin > >> Let's see what others think ... >> >> Thanks, >> David >> >> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>> Hi all, >>> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>> Best regards, >>> Robin > From ivan.gerasimov at oracle.com Fri Feb 9 04:32:30 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 8 Feb 2018 20:32:30 -0800 Subject: RFR 8197462 : Inconsistent exception messages for invalid capturing group names Message-ID: Hello! Capturing group name can be used in a regular expression in two contexts: When introducing a group (?...) or when referring it \k. If the name is invalid (i.e. does not start with a Latin letter, or contains wrong chars) then we may see different error messages, some of which look confusing. Here are examples of the messages produced by the current JDK: Unknown look-behind group near index 3 (?<>) ^ named capturing group is missing trailing '>' near index 4 \\k<> ^ Unknown look-behind group near index 4 (?<.>) ^ (named capturing group <.> does not exit near index 4 \\k<.> ^ named capturing group is missing trailing '>' near index 4 (?) ^ named capturing group is missing trailing '>' near index 4 \\k ^ In particular, this diversity is caused by that the internal Pattern.groupname() function lacks a check for the very first character of the name. So that when \k is parsed, the first char is always accepted, no matter what it was. Some cleanup was also done along the way. Would you please help review the fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8197462 WEBREV: http://cr.openjdk.java.net/~igerasim/8197462/00/webrev/ Thanks in advance! -- With kind regards, Ivan Gerasimov From david.holmes at oracle.com Fri Feb 9 04:39:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Feb 2018 14:39:12 +1000 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: Hi Martin, On 9/02/2018 2:07 PM, Martin Buchholz wrote: > > > On Thu, Feb 8, 2018 at 7:39 PM, David Holmes > wrote: > > On 9/02/2018 11:19 AM, Martin Buchholz wrote: > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > > > 8190324: ThreadPoolExecutor should not specify a dependency on > finalization > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ThreadPoolExecutor-finalize/index.html > > https://bugs.openjdk.java.net/browse/JDK-8190324 > > > > Looks okay. > > Another set of uninteresting changes, some suggested by intellij: > > 8195590: Miscellaneous changes imported from jsr166 CVS 2018-02 > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html > > https://bugs.openjdk.java.net/browse/JDK-8195590 > > > > Hmmmm ... a summary of changes in the bug report would be nice. > > > Done. > > I don't find the addition of reachability fences uninteresting > though. Why are they needed in those specific cases? > > > perhaps the standard should be for every method of a class with a > finalizer to have reachabilityFence(this) at the end? > > Imagine > > newSingleThreadExecutor().execute(someTask); > > Is is unlikely but possible for the "instant garbage" executor to be > finalized while super.execute is in progress, causing failure with > RejectedExecutionException because the delegatee is already shutdown. Wow! DelegatedExecutorService doesn't even have a finalize() method that does a shutdown. So we have to put the reachabilityFence in it to prevent the delegatee from becoming unreachable due to the delegator becoming unreachable! This whole notion that "this" can be unreachable whilst a method on it is still executing is just broken - it's an unusable programming model. Any code that does "new Foo().bar()" could fail unless Foo.bar() has a reachability fence in it. :( Sheesh! Cheers, David From martinrb at google.com Fri Feb 9 05:35:55 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 8 Feb 2018 21:35:55 -0800 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: On Thu, Feb 8, 2018 at 8:39 PM, David Holmes wrote: > > Wow! DelegatedExecutorService doesn't even have a finalize() method that > does a shutdown. So we have to put the reachabilityFence in it to prevent > the delegatee from becoming unreachable due to the delegator becoming > unreachable! > > This whole notion that "this" can be unreachable whilst a method on it is > still executing is just broken - it's an unusable programming model. Any > code that does "new Foo().bar()" could fail unless Foo.bar() has a > reachability fence in it. :( Sheesh! > I've argued similarly even in the past month that "this" should always remain reachable while a method on "this" is executing, in the presence of a finalize method. (It would also be nice if java language scope mapped to reachability range, but that information is already lost in the translation to bytecode.) But apparently it is quite a burden on any implementation to ensure reachability, especially after inlining. We've never had a report of this problem occurring in practice. From david.holmes at oracle.com Fri Feb 9 06:39:48 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Feb 2018 16:39:48 +1000 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: On 9/02/2018 3:35 PM, Martin Buchholz wrote: > On Thu, Feb 8, 2018 at 8:39 PM, David Holmes > wrote: > Wow! DelegatedExecutorService doesn't even have a finalize() method > that does a shutdown. So we have to put the reachabilityFence in it > to prevent the delegatee from becoming unreachable due to the > delegator becoming unreachable! > > This whole notion that "this" can be unreachable whilst a method on > it is still executing is just broken - it's an unusable programming > model. Any code that does "new Foo().bar()" could fail unless > Foo.bar() has a reachability fence in it. :( Sheesh! > > I've argued similarly even in the past month that "this" should always > remain reachable while a method on "this" is executing, in the presence > of a finalize method.? (It would also be nice if java language scope > mapped to reachability range, but that information is already lost in > the translation to bytecode.)? But apparently it is quite a burden on > any implementation to ensure reachability, especially after inlining. > > We've never had a report of this problem occurring in practice. Well at least it's only an issue if there does exist a finalize() method that could interfere with the executing method. I have to assume that somehow the VM is clever enough that if 'this' is in a register and we will later access a field at this+offset, that the GC will not free the memory used by 'this'. Somewhat ironic we add the reachabilityFence in the same changeset that removes the action of the finalize() method in TPE. (But of course the ExecutorService passed to the delegating service may still have one.) David From martinrb at google.com Fri Feb 9 07:01:31 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 8 Feb 2018 23:01:31 -0800 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: On Thu, Feb 8, 2018 at 10:39 PM, David Holmes wrote: > > Well at least it's only an issue if there does exist a finalize() method > that could interfere with the executing method. I have to assume that > somehow the VM is clever enough that if 'this' is in a register and we will > later access a field at this+offset, that the GC will not free the memory > used by 'this'. > I expect the opposite, that the VM may see an opportunity to prefetch that other field. Maybe it needed to read an adjacent field, and was able to read both fields with a single instruction. Maybe escape analysis allowed it to infer that the object could never be accessed by another thread that might write to the field. Even more extreme - in the case of newSingleThreadExecutor().execute(someTask); the VM can do enough analysis to conclude the executor will never be used after invoking super.execute, so the VM can "optimize" by constructing the executor, immediately finalizing it, and only then calling the body of execute. From robin.westberg at oracle.com Fri Feb 9 15:41:12 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 9 Feb 2018 16:41:12 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <71fa9742-c18a-abe2-3d5f-ccef7de9b0fd@Oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <71fa9742-c18a-abe2-3d5f-ccef7de9b0fd@Oracle.com> Message-ID: Hi Roger, > On 8 Feb 2018, at 17:26, Roger Riggs wrote: > > Hi Robin, > > Saving useful information about failures during shutdown is useful. > > Can the shutdown of JFR be put off? Is the use of shutdown hook still necessary? The part of JFR that manages recordings is written in Java, so the only other alternative I can see would be to introduce a hook at the very beginning of JVM_Halt. That one could call back into Java again and create the JFR shutdown hook thread from there instead. Not sure if there may be issues with the VM continuing to run a while longer after JVM_Halt is invoked, but I can?t see any obvious problem. It is an interesting idea, such a mechanism would allow capturing events generated from other shutdown hooks or finalizers as well. Something like that would probably be a lot easier to try out after JFR is open source though. Best regards, Robin > > Thanks, Roger > > > On 2/8/2018 10:50 AM, Robin Westberg wrote: >> Hi David, >> >>> On 8 Feb 2018, at 04:28, David Holmes wrote: > ... >>> I had an initial look through this. >>> >>> To be honest I don't like it. We seem to have to record little bits of information all over the place just so they can be reported by the shutdown event. It seems untidy. :( >>> >>> Can you rename _starting_thread to _main_thread please. The use of "starting" in thread.hpp/cpp is really a naming anomaly. The main thread is the thread that loads the JVM. And that would be consistent with set_exception_in_main_thread. >>> >>> Though why do we care if the main thread exited with an unhandled exception? And if we do care, why do we only care when the shutdown reason is ""No remaining non-daemon Java threads"? >>> >>> I don't like the need to add os::get_abort_exit_code. Do we really need it? What useful information does it convey? >> Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. >> >> Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. >> >>> It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. >> In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). >> >> Best regards, >> Robin >> >>> Let's see what others think ... >>> >>> Thanks, >>> David >>> >>> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>>> Hi all, >>>> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>>> Best regards, >>>> Robin > From robin.westberg at oracle.com Fri Feb 9 15:41:20 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 9 Feb 2018 16:41:20 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <5A7B65E1.5050003@oracle.com> References: <5A7B65E1.5050003@oracle.com> Message-ID: Hi Erik, > On 7 Feb 2018, at 21:47, Erik Gahlin wrote: > > Hi Robin, > > I can sponsor this. > > I wonder if we should change the name of the event to "Shutdown" instead? It will give us flexibility to add other shutdown related fields in the future. Makes sense, I?ll rename it. > Could you change the label and field to "Status Code" and statusCode. Event fields should have headline-style capitalization and statusCode allows us to add other status related information in the future. Removed the status field now, I?ll keep it in mind if it gets added back in the future. Best regards, Robin > > Thanks > Erik > >> Hi all, >> >> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >> >> Best regards, >> Robin > From robin.westberg at oracle.com Fri Feb 9 15:41:27 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 9 Feb 2018 16:41:27 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> Message-ID: <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> Hi David, > On 9 Feb 2018, at 05:22, David Holmes wrote: > > Hi Robin, > >> Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. > > Yes and that particular example of launcher behaviour is an archaic throwback to C programming - where end of "main" means end of the program - IMHO. I don't think it is of interest - and certainly not worth jumping through so many hoops to make the info available. > >> Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. > > That sounds good to me. :) > >>> It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. >> In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). > > Yes that is useful. But I find the need to code things the way they are, unfortunate. But given current constraints, so be it. Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ Best regards, Robin > > Thanks, > David > >> Best regards, >> Robin >>> Let's see what others think ... >>> >>> Thanks, >>> David >>> >>> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>>> Hi all, >>>> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>>> Best regards, >>>> Robin From martinrb at google.com Fri Feb 9 17:15:32 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 9 Feb 2018 09:15:32 -0800 Subject: Weird timezone issues with Timestamp.valueOf(String s) In-Reply-To: <004c01d3a1b7$e1efe9c0$a5cfbd40$@gmail.com> References: <004c01d3a1b7$e1efe9c0$a5cfbd40$@gmail.com> Message-ID: [redirect to core-libs-dev] On Fri, Feb 9, 2018 at 7:08 AM, wrote: > Not that I encourage using date/time classes from the packages of either > java.util or java.sql, but sometimes it happens under the hood. > > > > We found a weird issue with timestamps. > > In our (PostgreSQL) database we have a column of SQL type TIMESTAMP - no > timezone. > > > > When JPA fills our entity field with a java.sql.Timestamp value, it is > given > an inherent timezone of the system it's running on; in our case it was CET > (UTC +1). > > Now this timezone isn't immediately obvious, because if you print it to > system out, it seems to have the correct time. However when you convert it > with .toInstant() the timezone rears its ugly head. > > > > I digged deeper and found Timestamp.valueOf(String s), it does a lot of > magic, but in the end it calls its own deprecated constructor new > Timestamp(int year, int month, int date, int hour, int minute, int second, > int nano). > > That constructor calls the deprecated constructor of java.util.Date(int > year, int month, int date, int hour, int minute, int second) and that is > the one doing something with the current timezone on the system. > > The method Timestamp.valueOf(String s) itself however, is not deprecated > and > I find this odd. > > More odd is that Timestamp.valueOf(LocalDateTime dateTime) has a > @SuppresWarnings("deprecation") annotation. > > > > Thus I found that Timestamp.valueOf("2017-10-04 00:00:00") on a system > running in CET timezone yields a different result than one running in UTC > timezone. > > Here is some example code where I put the output of the println's in > comments. > > > > > TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("Europe/Amsterdam"))); > > Timestamp fromStringCet = Timestamp.valueOf("2017-10-04 00:00:00"); > > System.out.println(fromStringCet); // 2017-10-04 00:00:00.0 > > System.out.println(fromStringCet.toInstant()); // 2017-10-03T22:00:00Z > > > > TimeZone.setDefault(TimeZone.getTimeZone(ZoneOffset.UTC)); > > Timestamp fromStringUtc = Timestamp.valueOf("2017-10-04 00:00:00"); > > System.out.println(fromStringUtc); // 2017-10-04 00:00:00.0 > > System.out.println(fromStringUtc.toInstant()); // 2017-10-04T00:00:00Z > > > > System.out.println(fromStringCet.equals(fromStringUtc)); // false > > > > LocalDateTime localDateTime = LocalDateTime.of(2017, 10, 4, 0, 0, 0); > > > > > TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("Europe/Amsterdam"))); > > Timestamp fromLocalDateTimeCet = Timestamp.valueOf(localDateTime); > > System.out.println(fromLocalDateTimeCet.toInstant()); // > 2017-10-03T22:00:00Z > > > > TimeZone.setDefault(TimeZone.getTimeZone(ZoneOffset.UTC)); > > Timestamp fromLocalDateTimeUtc = Timestamp.valueOf(localDateTime); > > System.out.println(fromLocalDateTimeUtc.toInstant()); // 2017-10-04 > 00:00:00.0 > > > > System.out.println(fromLocalDateTimeCet.equals(fromLocalDateTimeUtc)); > // false > > > > So what to do? > > > > Some options are: > > * Make Timestamp.valueOf(String s) deprecated? > * Always use UTC when doing the implicit conversion > > > > Thoughts? > > > > Dave Franken > > > > From Alan.Bateman at oracle.com Fri Feb 9 18:01:35 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 9 Feb 2018 18:01:35 +0000 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: References: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> Message-ID: On 08/02/2018 17:35, yumin qi wrote: > HI, Alan > > ? As in your email to RFR of 8194154 which now has a new fix in > canonicalize_md.c, switch back to discuss solution here again. > ? The current fix is not in java, instead, I put it in C function. In > the function before the fix, it assumes no more "//" pattern in the > string so failed to get correct substrings with the case as the test case. > ? The fix should not cause other problem since it does double check if > double or more slashes in sequential. > ? The APIs indicates 'user.dir' etc system property should not be > rewritten, but did not prevent modifying them in practise. > Sure, but it's really hairy for anything to be depend on that.? We can improve the reliability by changing java.io.UnixFileSystem to read the system property at most once. Second/subsequent usages should only need to do a permission check (for the security manager case). Going further then we need to get to the point where the APIs never read a modified property, this goes for java.home and several others too. I'll study the patch you have but I think we also need to create issues to get us to the point where changing this system property in a running VM doesn't impact running code. -Alan From hboehm at google.com Fri Feb 9 19:19:32 2018 From: hboehm at google.com (Hans Boehm) Date: Fri, 9 Feb 2018 11:19:32 -0800 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: On Thu, Feb 8, 2018 at 9:35 PM, Martin Buchholz wrote: > On Thu, Feb 8, 2018 at 8:39 PM, David Holmes > wrote: > > > > > Wow! DelegatedExecutorService doesn't even have a finalize() method that > > does a shutdown. So we have to put the reachabilityFence in it to prevent > > the delegatee from becoming unreachable due to the delegator becoming > > unreachable! > > > > This whole notion that "this" can be unreachable whilst a method on it is > > still executing is just broken - it's an unusable programming model. Any > > code that does "new Foo().bar()" could fail unless Foo.bar() has a > > reachability fence in it. :( Sheesh! > > > > I've argued similarly even in the past month that "this" should always > remain reachable while a method on "this" is executing, in the presence of > a finalize method. (It would also be nice if java language scope mapped to > reachability range, but that information is already lost in the translation > to bytecode.) But apparently it is quite a burden on any implementation to > ensure reachability, especially after inlining. > > We've never had a report of this problem occurring in practice. > The downside of treating "this" specially is that it makes it even harder to explain the other cases, e.g. when the object being finalized prematurely was an explicit parameter, or perhaps even the result of a factory method. The presence of a finalize() method seems to be less and less of an indication of whether this treatment is needed. It's harder to determine that objects are being monitored by a Cleaner or PhantomReference. But the issues are exactly the same. In my opinion, we're basically stuck with two ways to fix this: 1) Disallow visible dead reference elimination altogether. 2) Require the programmer to specify when it's problematic. I think (1) is very intrusive, and probably impractical at this point, since it requires preservation of reliable scope information throughout the tool chain. ReachabilityFence() is an unexpectedly clumsy way to do (2), since the same information needs to be repeated for every method. My sense is that this does happen in practice, but indeed extremely rarely. One of my former HP colleagues pointed out to me, shortly after my original JavaOne talk on this topic, that this explained a problem they had been trying to track down. I haven't heard of other instances. I suspect they are generally written off as "alpha particles" or "gc bugs" or the like, especially since, in the native code case, they are likely to show up later as heap corruption, and are often essentially untraceable. From paul.sandoz at oracle.com Fri Feb 9 19:50:50 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 9 Feb 2018 11:50:50 -0800 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: <37BCA4AD-D9C9-4372-B59E-7F417750A071@oracle.com> +1 Hans is right that reachabilityFence can be clumsy but it?s the best we have right now. Paul. > On Feb 8, 2018, at 5:19 PM, Martin Buchholz wrote: > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > > 8190324: ThreadPoolExecutor should not specify a dependency on finalization > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ThreadPoolExecutor-finalize/index.html > https://bugs.openjdk.java.net/browse/JDK-8190324 > > Another set of uninteresting changes, some suggested by intellij: > > 8195590: Miscellaneous changes imported from jsr166 CVS 2018-02 > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html > https://bugs.openjdk.java.net/browse/JDK-8195590 > From Roger.Riggs at Oracle.com Fri Feb 9 21:38:40 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 9 Feb 2018 16:38:40 -0500 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> Message-ID: <9856406f-ba57-fd14-1287-39d185bc74d3@Oracle.com> Hi Joe, Looking good, but a few comments: AbstractStringBuilder: 111? the coder() method should be private and since there is only a few uses ? the function could be inlined. StringBuffer:192: extra leading space before "}" Thanks, Roger On 2/8/2018 7:47 PM, Joe Wang wrote: > Hi all, > > The CSR for the enhancement is now approved. Thanks Joe! > > The webrev has been updated accordingly. Please let me know if you > have any further comment on the implementation. > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Thanks, > Joe > > On 2/2/2018 12:46 PM, Joe Wang wrote: >> Thanks Jason. Will update that accordingly. >> >> Best, >> Joe >> >> On 2/2/2018 11:22 AM, Jason Mehrens wrote: >>> Joe, >>> >>> The identity check in CS.compare makes sense.? However, it won't be >>> null hostile if we call CS.compare(null, null) and that doesn't seem >>> right. >>> Usually when writing comparator classes I end up with: >>> === >>> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >>> ???? return 0; >>> } >>> === >>> >>> Jason >>> ________________________________________ >>> From: core-libs-dev on >>> behalf of Joe Wang >>> Sent: Friday, February 2, 2018 1:01 PM >>> To: core-libs-dev >>> Subject: Re: RFR (JDK11) 8137326: Methods for comparing >>> CharSequence, StringBuilder, and StringBuffer >>> >>> Hi, >>> >>> Thanks all for comments and suggestions. I've updated the webrev. >>> Please >>> review. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> Thanks, >>> Joe >>> >>> On 1/31/2018 9:31 PM, Joe Wang wrote: >>>> Hi Tagir, >>>> >>>> Thanks for the comment. I will consider adding that to the javadoc >>>> emphasizing that the comparison is performed from 0 to length() - 1 of >>>> the two sequences. >>>> >>>> Best, >>>> Joe >>>> >>>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>>> Hello! >>>>> >>>>> An AbstractStringBuilder#compareTo implementation is wrong. You >>>>> cannot >>>>> simply compare the whole byte array. Here's the test-case: >>>>> >>>>> public class Test { >>>>> ??? public static void main(String[] args) { >>>>> ????? StringBuilder sb1 = new StringBuilder("test1"); >>>>> ????? StringBuilder sb2 = new StringBuilder("test2"); >>>>> ????? sb1.setLength(4); >>>>> ????? sb2.setLength(4); >>>>> ????? System.out.println(sb1.compareTo(sb2)); >>>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>>> ??? } >>>>> } >>>>> >>>>> We truncated the stringbuilders making their content equal, so >>>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >>>>> the original content (before the truncation) as truncation, of >>>>> course, >>>>> does not zero the truncated bytes, neither does it reallocate the >>>>> array (unless explicitly asked via trimToSize). >>>>> >>>>> With best regards, >>>>> Tagir Valeev. >>>>> >>>>> >>>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>>> StringBuffer. >>>>>> >>>>>> The Comparable implementations for StringBuilder/Buffer are similar >>>>>> to that >>>>>> of String, allowing comparison operations between two >>>>>> StringBuilders/Buffers, e.g. >>>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>>> method >>>>>> 'compare' is added instead of implementing the Comparable interface. >>>>>> This >>>>>> 'compare' method may take CharSequence implementations such as >>>>>> String, >>>>>> StringBuilder and StringBuffer, making it possible to perform >>>>>> comparison >>>>>> among them. The previous example for example is equivalent to >>>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>>> >>>>>> Tests for java.base have been independent from each other. The new >>>>>> tests are >>>>>> therefore created to have no dependency on each other or sharing any >>>>>> code. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>> >>>>>> Thanks, >>>>>> Joe >> > From jason_mehrens at hotmail.com Fri Feb 9 21:46:40 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Fri, 9 Feb 2018 21:46:40 +0000 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com>, Message-ID: Joe, In CharSequence, does it make sense to cache the result of the length calculation? As in change: ==== for (int i = 0; i < Math.min(cs1.length(), cs2.length()); i++) { ==== for (int i = 0, len = Math.min(cs1.length(), cs2.length()); i < len; i++) { ==== Jason ________________________________________ From: core-libs-dev on behalf of Joe Wang Sent: Thursday, February 8, 2018 6:47 PM To: core-libs-dev Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer Hi all, The CSR for the enhancement is now approved. Thanks Joe! The webrev has been updated accordingly. Please let me know if you have any further comment on the implementation. JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ Thanks, Joe On 2/2/2018 12:46 PM, Joe Wang wrote: > Thanks Jason. Will update that accordingly. > > Best, > Joe > > On 2/2/2018 11:22 AM, Jason Mehrens wrote: >> Joe, >> >> The identity check in CS.compare makes sense. However, it won't be >> null hostile if we call CS.compare(null, null) and that doesn't seem >> right. >> Usually when writing comparator classes I end up with: >> === >> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >> return 0; >> } >> === >> >> Jason >> ________________________________________ >> From: core-libs-dev on >> behalf of Joe Wang >> Sent: Friday, February 2, 2018 1:01 PM >> To: core-libs-dev >> Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, >> StringBuilder, and StringBuffer >> >> Hi, >> >> Thanks all for comments and suggestions. I've updated the webrev. Please >> review. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> Thanks, >> Joe >> >> On 1/31/2018 9:31 PM, Joe Wang wrote: >>> Hi Tagir, >>> >>> Thanks for the comment. I will consider adding that to the javadoc >>> emphasizing that the comparison is performed from 0 to length() - 1 of >>> the two sequences. >>> >>> Best, >>> Joe >>> >>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>> Hello! >>>> >>>> An AbstractStringBuilder#compareTo implementation is wrong. You cannot >>>> simply compare the whole byte array. Here's the test-case: >>>> >>>> public class Test { >>>> public static void main(String[] args) { >>>> StringBuilder sb1 = new StringBuilder("test1"); >>>> StringBuilder sb2 = new StringBuilder("test2"); >>>> sb1.setLength(4); >>>> sb2.setLength(4); >>>> System.out.println(sb1.compareTo(sb2)); >>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>> } >>>> } >>>> >>>> We truncated the stringbuilders making their content equal, so >>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >>>> the original content (before the truncation) as truncation, of course, >>>> does not zero the truncated bytes, neither does it reallocate the >>>> array (unless explicitly asked via trimToSize). >>>> >>>> With best regards, >>>> Tagir Valeev. >>>> >>>> >>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>> wrote: >>>>> Hi, >>>>> >>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>> StringBuffer. >>>>> >>>>> The Comparable implementations for StringBuilder/Buffer are similar >>>>> to that >>>>> of String, allowing comparison operations between two >>>>> StringBuilders/Buffers, e.g. >>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>> method >>>>> 'compare' is added instead of implementing the Comparable interface. >>>>> This >>>>> 'compare' method may take CharSequence implementations such as >>>>> String, >>>>> StringBuilder and StringBuffer, making it possible to perform >>>>> comparison >>>>> among them. The previous example for example is equivalent to >>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>> >>>>> Tests for java.base have been independent from each other. The new >>>>> tests are >>>>> therefore created to have no dependency on each other or sharing any >>>>> code. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>> >>>>> Thanks, >>>>> Joe > From basinilya at gmail.com Sat Feb 10 21:15:18 2018 From: basinilya at gmail.com (Basin Ilya) Date: Sun, 11 Feb 2018 00:15:18 +0300 Subject: FileOutputStream.available() and pipe EOF Message-ID: <127e9711-345b-8baa-4578-1c44a04910b8@gmail.com> Hi list. My question relates to streams returned by java.lang.Process.getInputStream() On Linux calling available() after the other side of the pipe was closed will throw: java.io.IOException: Stream Closed This is very handy, because you can distinguish EOF and a pause in transmission. On Windows same Oracle JDK version 1.8.0_161 behaves in a traditional manner and available() returns 0 in both cases. Will this ever change? From forax at univ-mlv.fr Sat Feb 10 21:27:37 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 10 Feb 2018 22:27:37 +0100 (CET) Subject: FileOutputStream.available() and pipe EOF In-Reply-To: <127e9711-345b-8baa-4578-1c44a04910b8@gmail.com> References: <127e9711-345b-8baa-4578-1c44a04910b8@gmail.com> Message-ID: <1207461253.2760977.1518298057675.JavaMail.zimbra@u-pem.fr> Hi Basin, or instead of relying on an undocumented behaviour, you can use any overloads of read(), if it returns -1, it's the ends of the stream. cheers, R?mi ----- Mail original ----- > De: "Basin Ilya" > ?: "core-libs-dev" > Envoy?: Samedi 10 F?vrier 2018 22:15:18 > Objet: FileOutputStream.available() and pipe EOF > Hi list. > > My question relates to streams returned by > java.lang.Process.getInputStream() > > On Linux calling available() after the other side of the pipe was closed > will throw: > > java.io.IOException: Stream Closed > > This is very handy, because you can distinguish EOF and a pause in > transmission. > > On Windows same Oracle JDK version 1.8.0_161 behaves in a traditional > manner and available() returns 0 in both cases. Will this ever change? From basinilya at gmail.com Sat Feb 10 21:35:25 2018 From: basinilya at gmail.com (Basin Ilya) Date: Sun, 11 Feb 2018 00:35:25 +0300 Subject: FileOutputStream.available() and pipe EOF In-Reply-To: <1207461253.2760977.1518298057675.JavaMail.zimbra@u-pem.fr> References: <127e9711-345b-8baa-4578-1c44a04910b8@gmail.com> <1207461253.2760977.1518298057675.JavaMail.zimbra@u-pem.fr> Message-ID: <8d9a525f-b980-9efd-6b6d-29e9d20c8f27@gmail.com> Unfortunately, read() is 1) uninterruptilbe 2) Unlike sockets, close() or even Thread.stop() won't cancel a read pipe operation on Windows 11.02.2018 0:27, Remi Forax ?????: > Hi Basin, > or instead of relying on an undocumented behaviour, you can use any overloads of read(), if it returns -1, it's the ends of the stream. > > cheers, > R?mi > > ----- Mail original ----- >> De: "Basin Ilya" >> ?: "core-libs-dev" >> Envoy?: Samedi 10 F?vrier 2018 22:15:18 >> Objet: FileOutputStream.available() and pipe EOF > >> Hi list. >> >> My question relates to streams returned by >> java.lang.Process.getInputStream() >> >> On Linux calling available() after the other side of the pipe was closed >> will throw: >> >> java.io.IOException: Stream Closed >> >> This is very handy, because you can distinguish EOF and a pause in >> transmission. >> >> On Windows same Oracle JDK version 1.8.0_161 behaves in a traditional >> manner and available() returns 0 in both cases. Will this ever change? From martinrb at google.com Sat Feb 10 23:17:15 2018 From: martinrb at google.com (Martin Buchholz) Date: Sat, 10 Feb 2018 15:17:15 -0800 Subject: RFR: jsr166 jdk integration 2018-02 In-Reply-To: References: Message-ID: Hans, thanks for another great lesson! On Fri, Feb 9, 2018 at 11:19 AM, Hans Boehm wrote: > > The downside of treating "this" specially is that it makes it even harder > to explain the other cases, e.g. when the object being finalized > prematurely was an explicit parameter, or perhaps even the result of a > factory method. > Java programmer intuition says the "this" object receiver is special. E.g. there's no way to explicitly null out "this" in a method body to be nice to the GC, as you can with a method argument. Think how crazy "this = null;" looks! But that's what the VM is doing on your behalf. Anyways, back in the real world this change got submitted with reachabilityFence. From david.holmes at oracle.com Mon Feb 12 06:54:40 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Feb 2018 16:54:40 +1000 Subject: RFR 8184289: Obsolete -XX:+UnsyncloadClass and -XX:+MustCallLoadClassInternal options In-Reply-To: <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> References: <87cef54c-8de9-1cd0-27c8-cabc6d2aa13d@oracle.com> <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> Message-ID: <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> Hi Harold, Adding core-libs-dev so they are aware of the ClassLoader change. On 10/02/2018 5:44 AM, harold seigel wrote: > Hi David, > > Thanks for reviewing this. > > Please see updated webrev: > http://cr.openjdk.java.net/~hseigel/bug_8184289.2/webrev/index.html This all seems fine to me. I agree there is question mark over the SystemDictionary code now only used for the null/boot loader: 848 if ((class_loader.is_null())) { 849 if (k == NULL && HAS_PENDING_EXCEPTION 850 && PENDING_EXCEPTION->is_a(SystemDictionary::LinkageError_klass())) { 851 MutexLocker mu(SystemDictionary_lock, THREAD); 852 InstanceKlass* check = find_class(d_hash, name, dictionary); 853 if (check != NULL) { 854 // Klass is already loaded, so just use it 855 k = check; 856 CLEAR_PENDING_EXCEPTION; 857 guarantee((!class_loader.is_null()), "dup definition for bootstrap loader?"); 858 } 859 } This seems to be a negative test, that we should in fact never get in this situation when dealing with the boot loader. But in that case we don't need lines 855 and 856 anymore and the code would just collapse to: 848 if ((class_loader.is_null())) { 849 if (k == NULL && HAS_PENDING_EXCEPTION 850 && PENDING_EXCEPTION->is_a(SystemDictionary::LinkageError_klass())) { 851 MutexLocker mu(SystemDictionary_lock, THREAD); 852 guarantee(find_class(d_hash, name, dictionary) == NULL, 853 "dup definition for bootstrap loader?"); 854 } 855 } and in that case I'd probably rather see this as an assertion than a guarantee, and the whole block can be debug-only. Thanks, David ----- > And, please see in-line comments. > > > On 2/8/2018 5:42 PM, David Holmes wrote: >> Hi Harold, >> >> First I'm pretty sure this one can't be pushed until the version bump >> arrives in jdk/hs :) > I hope the version bump arrives soon. >> >> On 9/02/2018 6:53 AM, harold seigel wrote: >>> Hi, >>> >>> Please review this JDK-11 change to obsolete the UnsyncloadClass and >>> MustCallLoadClassInternal options.? With this change, these options >>> are still accepted on the command line but have no affect other than >>> to generate these warning messages: >>> >>> ??? Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option >>> ??? UnsyncloadClass; support was removed in 11.0 >>> >>> ??? Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option >>> ??? MustCallLoadClassInternal; support was removed in 11.0 >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8184289/webrev/index.html >> >> Overall looks good. Tricky to untangle things in the SD especially! >> >> src/hotspot/share/classfile/systemDictionary.cpp >> >> Looking at this change, and the comment: >> >> -?????? // For UnsyncloadClass only >> ??????? // If they got a linkageError, check if a parallel class load >> succeeded. >> ??????? // If it did, then for bytecode resolution the specification >> requires >> ??????? // that we return the same result we did for the other thread, >> i.e. the >> ??????? // successfully loaded InstanceKlass >> ??????? // Should not get here for classloaders that support parallelism >> ??????? // with the new cleaner mechanism, even with >> AllowParallelDefineClass >> ??????? // Bootstrap goes through here to allow for an extra guarantee >> check >> !?????? if (UnsyncloadClass || (class_loader.is_null())) { >> >> It's not clear why all the "UnsyncLoadClass only" stuff is also being >> done for the "Bootstrap" (? bootloader?) case. But in any case the >> comment block doesn't read correctly now as this is all, and only, >> about the bootstrap case. I'd suggest: >> >> -?????? // For UnsyncloadClass only >> +?????? // For bootloader only: >> ??????? // If they got a linkageError, check if a parallel class load >> succeeded. >> ??????? // If it did, then for bytecode resolution the specification >> requires >> ??????? // that we return the same result we did for the other thread, >> i.e. the >> ??????? // successfully loaded InstanceKlass >> ??????? // Should not get here for classloaders that support parallelism >> ??????? // with the new cleaner mechanism, even with >> AllowParallelDefineClass >> -?????? // Bootstrap goes through here to allow for an extra guarantee >> check > Done. >> >> >> Question: is ClassLoader.loadClassInternal now obsolete as well? > Yes.? Thanks for pointing that out.? The new webrev contains significant > changes needed to remove loadClassInternal. >> >> --- >> >> src/hotspot/share/runtime/arguments.cpp >> >> Is there some reason to leave the expiration version unset? Do you >> think the obsoletion warning may be needed for a couple of releases ?? > I figured whoever expires the option can put in the version. >> >> --- >> >> ?test/hotspot/jtreg/runtime/CommandLine/ObsoleteFlagErrorMessage.java >> >> You don't need to add anything here. This is not a test of all >> obsolete flags, it is just testing some specific handling of obsolete >> flags. > I reverted this change. > > Thanks! Harold >> >> Thanks, >> David >> ----- >> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8184289 >>> >>> The change was tested with Mach5 tiers 1-5 and the NSK parallel class >>> loading tests.? Also, JDK's containing the change were built on all >>> Mach5 platforms. >>> >>> Thanks, Harold >>> > From david.holmes at oracle.com Mon Feb 12 07:07:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Feb 2018 17:07:03 +1000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> Message-ID: <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> Hi Robin, On 10/02/2018 1:41 AM, Robin Westberg wrote: > Hi David, > >> On 9 Feb 2018, at 05:22, David Holmes > > wrote: >> >> Hi Robin, >> >>> Right, almost all the runtime changes are made in order to try to >>> figure out what the process exit code from the launcher will >>> eventually be. For example, the launcher returns 1 if the main thread >>> exited with an unhandled exception, but 0 otherwise. But I actually >>> agree that this information is probably only of marginal use (it >>> could always be captured from wherever Java is launched if someone >>> really wants it), so it is perhaps not worth the trouble. >> >> Yes and that particular example of launcher behaviour is an archaic >> throwback to C programming - where end of "main" means end of the >> program - IMHO. I don't think it is of interest - and certainly not >> worth jumping through so many hoops to make the info available. >> >>> Discussed it a bit with Erik Gahlin, and perhaps the best option here >>> is to simply remove the status code field from the event, that would >>> simplify things and make the code you mention above go away. >> >> That sounds good to me. :) >> >>>> It is unfortunate that you need to add beforeHalt to deal with the >>>> event mechanism itself being turned off (?) by a shutdown hook. That >>>> would seem to potentially lose a lot of event information given >>>> hooks can run in arbitrary order and execute arbitrary Java code. >>>> And essentially you end up recording the initial reason shutdown >>>> started, though potentially it could end up terminating for a >>>> different reason. >>> In this case I think it actually conveys useful information if you >>> are trying to diagnose an unexpected shutdown. It should be useful to >>> find the initial request of an orderly shutdown, even if something >>> else happens during the shutdown sequence like a finalizer calling >>> exit (in which case you could possibly end up with two shutdown >>> events, but both may contain interesting information). >> >> Yes that is useful. But I find the need to code things the way they >> are, unfortunate. But given current constraints, so be it. > > Okay, I?ve removed the code related to the status field, certainly makes > the change a bit less intrusive. > > Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ > Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ This looks much cleaner/neater to me - thanks. Tool Nit: the git-webrev tool you use has an annoying quirk compared to our own webrev tool - it first reports the number of files changed even when it is reporting on an individual file! I'm used to seeing a first number that represents number of lines changes - and the "1" in this case throws me as I like to quickly look at the size of the change for each file when deciding what order to look at them. :( Thanks, David > Best regards, > Robin > >> >> Thanks, >> David >> >>> Best regards, >>> Robin >>>> Let's see what others think ... >>>> >>>> Thanks, >>>> David >>>> >>>> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>>>> Hi all, >>>>> Please review the following change that adds an event-based tracing >>>>> event that is generated when the VM shuts down. The intent is to >>>>> capture shutdowns that occur after the VM has been properly >>>>> initialized (as initialization problems would most likely mean that >>>>> the tracing framework hasn?t been properly started either). >>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>>>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>>>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>>>> Best regards, >>>>> Robin > From Alan.Bateman at oracle.com Mon Feb 12 07:52:08 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 12 Feb 2018 07:52:08 +0000 Subject: RFR 8184289: Obsolete -XX:+UnsyncloadClass and -XX:+MustCallLoadClassInternal options In-Reply-To: <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> References: <87cef54c-8de9-1cd0-27c8-cabc6d2aa13d@oracle.com> <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> Message-ID: On 12/02/2018 06:54, David Holmes wrote: > Hi Harold, > > Adding core-libs-dev so they are aware of the ClassLoader change. Thanks, that part is okay and good to see this going away. -Alan From Alan.Bateman at oracle.com Mon Feb 12 08:02:42 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 12 Feb 2018 08:02:42 +0000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> Message-ID: On 12/02/2018 07:07, David Holmes wrote: > >> Okay, I?ve removed the code related to the status field, certainly >> makes the change a bit less intrusive. >> >> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ > > This looks much cleaner/neater to me - thanks. > The updates to Runtime/Shutdown seems okay. -Alan From martijnverburg at gmail.com Mon Feb 12 10:06:11 2018 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 12 Feb 2018 10:06:11 +0000 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: <360C5181-3B9E-4339-B460-04A8385F2461@oracle.com> References: <360C5181-3B9E-4339-B460-04A8385F2461@oracle.com> Message-ID: One could almost shed a tear - of joy! This is exactly the use case for the module system that the developer community at large will understand. Thanks for this change and a leaner meaner JDK. Cheers, Martijn On 8 February 2018 at 13:37, Lance Andersen wrote: > > > On Feb 8, 2018, at 3:04 AM, Alan Bateman > wrote: > > > > On 07/02/2018 16:57, Lance Andersen wrote: > >> Hi all, > >> > >> I think we are at a point where we are ready to start reviewing the > changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, > has been targeted to JDK 11. > >> The CSR for removing the modules has been approved: > https://bugs.openjdk.java.net/browse/JDK-8193757 < > https://bugs.openjdk.java.net/browse/JDK-8193757> > >> > >> The open webrev can be found at: http://cr.openjdk.java.net/~ > lancea/8190378/open_changes/ lancea/8190378/open_changes/> > >> > > 800 KLOC deleted, wonderful! > > > > The update to technology-summary.html page means its html title no > longer matches the contents. We should probably change it to "JCP > Technologies in JDK 11" for now. > > I updated the webrev. Thanks for catching that (btw we missed this for JDK > 10) > > > > The removal of test cases from the tests in tools/launcher/modules > removes most of the test coverage for the upgrade module path. We'll need > to replace these sub-tests. Can you create an issue to track that? > > I can do that > > > > Everything else looks good and it's okay to track residual issues with > other JIRA issues. I think the important thing is to get this monster patch > into JDK builds soon so that libraries and the eco system can start to > adjust. > > Thank you Alan for the review > > Best > Lance > > > > -Alan > > > < > http://oracle.com/us/design/oracle-email-sig-198324.gif> > Lance Andersen| > Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From Alan.Bateman at oracle.com Mon Feb 12 11:09:49 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 12 Feb 2018 11:09:49 +0000 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: References: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> Message-ID: <94168540-0cf6-8f9f-41bd-3f64ba41187f@oracle.com> On 09/02/2018 18:01, Alan Bateman wrote: > : > > I'll study the patch you have but I think we also need to create > issues to get us to the point where changing this system property in a > running VM doesn't impact running code. Looking at it again, I think we should change java.io.UnixFileSystem (and the Windows equivalent) to cache the value of user.dir to avoid difficult to diagnose issues with bad code changing the value of this property in a running VM. This should reduce the issue down to cases where user.dir is changed on the command line (never supported either of course) to a value that is not "/" but has trailing or duplicate slashes. When reduced down then the alternatives are to change the native canonicalize method as you have done or alternatively do it once at UnixFileSystem initialization time so that canonicalize does not have to deal with this case. The former would require changing the description of the function (it currently reads "The input path is assumed to contain no duplicate slashes"), the latter avoids any changes to the native implementation. -Alan diff -r 0937e5f799df src/java.base/unix/classes/java/io/UnixFileSystem.java --- a/src/java.base/unix/classes/java/io/UnixFileSystem.java??? Sat Feb 10 07:06:16 2018 -0500 +++ b/src/java.base/unix/classes/java/io/UnixFileSystem.java??? Mon Feb 12 10:49:40 2018 +0000 @@ -34,12 +34,14 @@ ???? private final char slash; ???? private final char colon; ???? private final String javaHome; +??? private final String userDir; ???? public UnixFileSystem() { ???????? Properties props = GetPropertyAction.privilegedGetProperties(); ???????? slash = props.getProperty("file.separator").charAt(0); ???????? colon = props.getProperty("path.separator").charAt(0); ???????? javaHome = props.getProperty("java.home"); +??????? userDir = props.getProperty("user.dir"); ???? } @@ -128,7 +130,11 @@ ???? public String resolve(File f) { ???????? if (isAbsolute(f)) return f.getPath(); -??????? return resolve(System.getProperty("user.dir"), f.getPath()); +??????? SecurityManager sm = System.getSecurityManager(); +??????? if (sm != null) { +??????????? sm.checkPropertyAccess("user.dir"); +??????? } +??????? return resolve(userDir, f.getPath()); ???? } ???? // Caches for canonicalization results to improve startup performance. From srinivas.dama at oracle.com Mon Feb 12 12:01:17 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 12 Feb 2018 04:01:17 -0800 (PST) Subject: RFR: 8196959: NullPointerException in discovery003.java Message-ID: Hi, Please review http://cr.openjdk.java.net/~sdama/8196959/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196959 This is small modification of fix for https://bugs.openjdk.java.net/browse/JDK-8011697. Fix is to handle the corner case where the engineName is null. Regards, Srinivas From sundararajan.athijegannathan at oracle.com Mon Feb 12 13:09:12 2018 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 12 Feb 2018 18:39:12 +0530 Subject: RFR: 8196959: NullPointerException in discovery003.java In-Reply-To: References: Message-ID: <5A8191F8.1040006@oracle.com> Looks good. -Sundar On 12/02/18, 5:31 PM, Srinivas Dama wrote: > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196959/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196959 > > This is small modification of fix for https://bugs.openjdk.java.net/browse/JDK-8011697. > Fix is to handle the corner case where the engineName is null. > > > Regards, > Srinivas From coleen.phillimore at oracle.com Mon Feb 12 13:53:13 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 12 Feb 2018 08:53:13 -0500 Subject: RFR 8184289: Obsolete -XX:+UnsyncloadClass and -XX:+MustCallLoadClassInternal options In-Reply-To: <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> References: <87cef54c-8de9-1cd0-27c8-cabc6d2aa13d@oracle.com> <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> Message-ID: <9789ec33-2578-9022-dcff-c03c213b4f78@oracle.com> On 2/12/18 1:54 AM, David Holmes wrote: > Hi Harold, > > Adding core-libs-dev so they are aware of the ClassLoader change. > > On 10/02/2018 5:44 AM, harold seigel wrote: >> Hi David, >> >> Thanks for reviewing this. >> >> Please see updated webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8184289.2/webrev/index.html > > This all seems fine to me. > > I agree there is question mark over the SystemDictionary code now only > used for the null/boot loader: > > ?848?????? if ((class_loader.is_null())) { > ?849???????? if (k == NULL && HAS_PENDING_EXCEPTION > ?850?????????? && > PENDING_EXCEPTION->is_a(SystemDictionary::LinkageError_klass())) { > ?851?????????? MutexLocker mu(SystemDictionary_lock, THREAD); > ?852?????????? InstanceKlass* check = find_class(d_hash, name, > dictionary); > ?853?????????? if (check != NULL) { > ?854???????????? // Klass is already loaded, so just use it > ?855???????????? k = check; > ?856???????????? CLEAR_PENDING_EXCEPTION; > ?857???????????? guarantee((!class_loader.is_null()), "dup definition > for bootstrap loader?"); > ?858?????????? } > ?859???????? } > > This seems to be a negative test, that we should in fact never get in > this situation when dealing with the boot loader. But in that case we > don't need lines 855 and 856 anymore and the code would just collapse to: > > ?848?????? if ((class_loader.is_null())) { > ?849???????? if (k == NULL && HAS_PENDING_EXCEPTION > ?850?????????? && > PENDING_EXCEPTION->is_a(SystemDictionary::LinkageError_klass())) { > ?851?????????? MutexLocker mu(SystemDictionary_lock, THREAD); > ?852?????????? guarantee(find_class(d_hash, name, dictionary) == NULL, > ?853???????????????????? "dup definition for bootstrap loader?"); > ?854???????? } > ?855?????? } > > and in that case I'd probably rather see this as an assertion than a > guarantee, and the whole block can be debug-only. (just to you two).? If this is decided that it's an assert only, my vote would be to remove it to simplify the logic here, which is the main benefit of removing these options.? It's too special-casey as it is. Coleen > > Thanks, > David > ----- > >> And, please see in-line comments. >> >> >> On 2/8/2018 5:42 PM, David Holmes wrote: >>> Hi Harold, >>> >>> First I'm pretty sure this one can't be pushed until the version >>> bump arrives in jdk/hs :) >> I hope the version bump arrives soon. >>> >>> On 9/02/2018 6:53 AM, harold seigel wrote: >>>> Hi, >>>> >>>> Please review this JDK-11 change to obsolete the UnsyncloadClass >>>> and MustCallLoadClassInternal options.? With this change, these >>>> options are still accepted on the command line but have no affect >>>> other than to generate these warning messages: >>>> >>>> ??? Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option >>>> ??? UnsyncloadClass; support was removed in 11.0 >>>> >>>> ??? Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option >>>> ??? MustCallLoadClassInternal; support was removed in 11.0 >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8184289/webrev/index.html >>> >>> Overall looks good. Tricky to untangle things in the SD especially! >>> >>> src/hotspot/share/classfile/systemDictionary.cpp >>> >>> Looking at this change, and the comment: >>> >>> -?????? // For UnsyncloadClass only >>> ??????? // If they got a linkageError, check if a parallel class >>> load succeeded. >>> ??????? // If it did, then for bytecode resolution the specification >>> requires >>> ??????? // that we return the same result we did for the other >>> thread, i.e. the >>> ??????? // successfully loaded InstanceKlass >>> ??????? // Should not get here for classloaders that support >>> parallelism >>> ??????? // with the new cleaner mechanism, even with >>> AllowParallelDefineClass >>> ??????? // Bootstrap goes through here to allow for an extra >>> guarantee check >>> !?????? if (UnsyncloadClass || (class_loader.is_null())) { >>> >>> It's not clear why all the "UnsyncLoadClass only" stuff is also >>> being done for the "Bootstrap" (? bootloader?) case. But in any case >>> the comment block doesn't read correctly now as this is all, and >>> only, about the bootstrap case. I'd suggest: >>> >>> -?????? // For UnsyncloadClass only >>> +?????? // For bootloader only: >>> ??????? // If they got a linkageError, check if a parallel class >>> load succeeded. >>> ??????? // If it did, then for bytecode resolution the specification >>> requires >>> ??????? // that we return the same result we did for the other >>> thread, i.e. the >>> ??????? // successfully loaded InstanceKlass >>> ??????? // Should not get here for classloaders that support >>> parallelism >>> ??????? // with the new cleaner mechanism, even with >>> AllowParallelDefineClass >>> -?????? // Bootstrap goes through here to allow for an extra >>> guarantee check >> Done. >>> >>> >>> Question: is ClassLoader.loadClassInternal now obsolete as well? >> Yes.? Thanks for pointing that out.? The new webrev contains >> significant changes needed to remove loadClassInternal. >>> >>> --- >>> >>> src/hotspot/share/runtime/arguments.cpp >>> >>> Is there some reason to leave the expiration version unset? Do you >>> think the obsoletion warning may be needed for a couple of releases ?? >> I figured whoever expires the option can put in the version. >>> >>> --- >>> >>> ?test/hotspot/jtreg/runtime/CommandLine/ObsoleteFlagErrorMessage.java >>> >>> You don't need to add anything here. This is not a test of all >>> obsolete flags, it is just testing some specific handling of >>> obsolete flags. >> I reverted this change. >> >> Thanks! Harold >>> >>> Thanks, >>> David >>> ----- >>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8184289 >>>> >>>> The change was tested with Mach5 tiers 1-5 and the NSK parallel >>>> class loading tests.? Also, JDK's containing the change were built >>>> on all Mach5 platforms. >>>> >>>> Thanks, Harold >>>> >> From harold.seigel at oracle.com Mon Feb 12 16:13:05 2018 From: harold.seigel at oracle.com (harold seigel) Date: Mon, 12 Feb 2018 11:13:05 -0500 Subject: RFR 8184289: Obsolete -XX:+UnsyncloadClass and -XX:+MustCallLoadClassInternal options In-Reply-To: References: <87cef54c-8de9-1cd0-27c8-cabc6d2aa13d@oracle.com> <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> Message-ID: Hi Alan, Thanks for looking at this. Harold On 2/12/2018 2:52 AM, Alan Bateman wrote: > On 12/02/2018 06:54, David Holmes wrote: >> Hi Harold, >> >> Adding core-libs-dev so they are aware of the ClassLoader change. > Thanks, that part is okay and good to see this going away. > > -Alan From huizhe.wang at oracle.com Mon Feb 12 18:25:30 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 12 Feb 2018 10:25:30 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <9856406f-ba57-fd14-1287-39d185bc74d3@Oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <9856406f-ba57-fd14-1287-39d185bc74d3@Oracle.com> Message-ID: <5A81DC1A.4010101@oracle.com> On 2/9/18, 1:38 PM, Roger Riggs wrote: > Hi Joe, > > Looking good, but a few comments: > > AbstractStringBuilder: 111 the coder() method should be private and > since there is only a few uses > the function could be inlined. Indeed, the coder() method was added along with the new method. The coder field was referenced directly in all existing uses. I've removed the coder() method, and instead refer to the coder field directly. > > StringBuffer:192: extra leading space before "}" Fixed. Thanks, Joe > > Thanks, Roger > > On 2/8/2018 7:47 PM, Joe Wang wrote: >> Hi all, >> >> The CSR for the enhancement is now approved. Thanks Joe! >> >> The webrev has been updated accordingly. Please let me know if you >> have any further comment on the implementation. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> Thanks, >> Joe >> >> On 2/2/2018 12:46 PM, Joe Wang wrote: >>> Thanks Jason. Will update that accordingly. >>> >>> Best, >>> Joe >>> >>> On 2/2/2018 11:22 AM, Jason Mehrens wrote: >>>> Joe, >>>> >>>> The identity check in CS.compare makes sense. However, it won't be >>>> null hostile if we call CS.compare(null, null) and that doesn't >>>> seem right. >>>> Usually when writing comparator classes I end up with: >>>> === >>>> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >>>> return 0; >>>> } >>>> === >>>> >>>> Jason >>>> ________________________________________ >>>> From: core-libs-dev on >>>> behalf of Joe Wang >>>> Sent: Friday, February 2, 2018 1:01 PM >>>> To: core-libs-dev >>>> Subject: Re: RFR (JDK11) 8137326: Methods for comparing >>>> CharSequence, StringBuilder, and StringBuffer >>>> >>>> Hi, >>>> >>>> Thanks all for comments and suggestions. I've updated the webrev. >>>> Please >>>> review. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>> >>>> Thanks, >>>> Joe >>>> >>>> On 1/31/2018 9:31 PM, Joe Wang wrote: >>>>> Hi Tagir, >>>>> >>>>> Thanks for the comment. I will consider adding that to the javadoc >>>>> emphasizing that the comparison is performed from 0 to length() - >>>>> 1 of >>>>> the two sequences. >>>>> >>>>> Best, >>>>> Joe >>>>> >>>>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>>>> Hello! >>>>>> >>>>>> An AbstractStringBuilder#compareTo implementation is wrong. You >>>>>> cannot >>>>>> simply compare the whole byte array. Here's the test-case: >>>>>> >>>>>> public class Test { >>>>>> public static void main(String[] args) { >>>>>> StringBuilder sb1 = new StringBuilder("test1"); >>>>>> StringBuilder sb2 = new StringBuilder("test2"); >>>>>> sb1.setLength(4); >>>>>> sb2.setLength(4); >>>>>> System.out.println(sb1.compareTo(sb2)); >>>>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>>>> } >>>>>> } >>>>>> >>>>>> We truncated the stringbuilders making their content equal, so >>>>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo >>>>>> compares >>>>>> the original content (before the truncation) as truncation, of >>>>>> course, >>>>>> does not zero the truncated bytes, neither does it reallocate the >>>>>> array (unless explicitly asked via trimToSize). >>>>>> >>>>>> With best regards, >>>>>> Tagir Valeev. >>>>>> >>>>>> >>>>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>>>> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>>>> StringBuffer. >>>>>>> >>>>>>> The Comparable implementations for StringBuilder/Buffer are similar >>>>>>> to that >>>>>>> of String, allowing comparison operations between two >>>>>>> StringBuilders/Buffers, e.g. >>>>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>>>> method >>>>>>> 'compare' is added instead of implementing the Comparable >>>>>>> interface. >>>>>>> This >>>>>>> 'compare' method may take CharSequence implementations such as >>>>>>> String, >>>>>>> StringBuilder and StringBuffer, making it possible to perform >>>>>>> comparison >>>>>>> among them. The previous example for example is equivalent to >>>>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>>>> >>>>>>> Tests for java.base have been independent from each other. The new >>>>>>> tests are >>>>>>> therefore created to have no dependency on each other or sharing >>>>>>> any >>>>>>> code. >>>>>>> >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>>> >>>>>>> Thanks, >>>>>>> Joe >>> >> > From huizhe.wang at oracle.com Mon Feb 12 18:25:50 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 12 Feb 2018 10:25:50 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com>, Message-ID: <5A81DC2E.9080806@oracle.com> Done. Thanks Jason! Joe On 2/9/18, 1:46 PM, Jason Mehrens wrote: > Joe, > In CharSequence, does it make sense to cache the result of the length calculation? > As in change: > ==== > for (int i = 0; i< Math.min(cs1.length(), cs2.length()); i++) { > ==== > for (int i = 0, len = Math.min(cs1.length(), cs2.length()); i< len; i++) { > ==== > > Jason > ________________________________________ > From: core-libs-dev on behalf of Joe Wang > Sent: Thursday, February 8, 2018 6:47 PM > To: core-libs-dev > Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer > > Hi all, > > The CSR for the enhancement is now approved. Thanks Joe! > > The webrev has been updated accordingly. Please let me know if you have > any further comment on the implementation. > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Thanks, > Joe > > On 2/2/2018 12:46 PM, Joe Wang wrote: >> Thanks Jason. Will update that accordingly. >> >> Best, >> Joe >> >> On 2/2/2018 11:22 AM, Jason Mehrens wrote: >>> Joe, >>> >>> The identity check in CS.compare makes sense. However, it won't be >>> null hostile if we call CS.compare(null, null) and that doesn't seem >>> right. >>> Usually when writing comparator classes I end up with: >>> === >>> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >>> return 0; >>> } >>> === >>> >>> Jason >>> ________________________________________ >>> From: core-libs-dev on >>> behalf of Joe Wang >>> Sent: Friday, February 2, 2018 1:01 PM >>> To: core-libs-dev >>> Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, >>> StringBuilder, and StringBuffer >>> >>> Hi, >>> >>> Thanks all for comments and suggestions. I've updated the webrev. Please >>> review. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> Thanks, >>> Joe >>> >>> On 1/31/2018 9:31 PM, Joe Wang wrote: >>>> Hi Tagir, >>>> >>>> Thanks for the comment. I will consider adding that to the javadoc >>>> emphasizing that the comparison is performed from 0 to length() - 1 of >>>> the two sequences. >>>> >>>> Best, >>>> Joe >>>> >>>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>>> Hello! >>>>> >>>>> An AbstractStringBuilder#compareTo implementation is wrong. You cannot >>>>> simply compare the whole byte array. Here's the test-case: >>>>> >>>>> public class Test { >>>>> public static void main(String[] args) { >>>>> StringBuilder sb1 = new StringBuilder("test1"); >>>>> StringBuilder sb2 = new StringBuilder("test2"); >>>>> sb1.setLength(4); >>>>> sb2.setLength(4); >>>>> System.out.println(sb1.compareTo(sb2)); >>>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>>> } >>>>> } >>>>> >>>>> We truncated the stringbuilders making their content equal, so >>>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >>>>> the original content (before the truncation) as truncation, of course, >>>>> does not zero the truncated bytes, neither does it reallocate the >>>>> array (unless explicitly asked via trimToSize). >>>>> >>>>> With best regards, >>>>> Tagir Valeev. >>>>> >>>>> >>>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>>> StringBuffer. >>>>>> >>>>>> The Comparable implementations for StringBuilder/Buffer are similar >>>>>> to that >>>>>> of String, allowing comparison operations between two >>>>>> StringBuilders/Buffers, e.g. >>>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>>> method >>>>>> 'compare' is added instead of implementing the Comparable interface. >>>>>> This >>>>>> 'compare' method may take CharSequence implementations such as >>>>>> String, >>>>>> StringBuilder and StringBuffer, making it possible to perform >>>>>> comparison >>>>>> among them. The previous example for example is equivalent to >>>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>>> >>>>>> Tests for java.base have been independent from each other. The new >>>>>> tests are >>>>>> therefore created to have no dependency on each other or sharing any >>>>>> code. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>> >>>>>> Thanks, >>>>>> Joe From jason_mehrens at hotmail.com Mon Feb 12 18:35:35 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Mon, 12 Feb 2018 18:35:35 +0000 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A81DC2E.9080806@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com>, , <5A81DC2E.9080806@oracle.com> Message-ID: Looks good. Jason ________________________________________ From: Joe Wang Sent: Monday, February 12, 2018 12:25 PM To: Jason Mehrens Cc: core-libs-dev Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer Done. Thanks Jason! Joe On 2/9/18, 1:46 PM, Jason Mehrens wrote: > Joe, > In CharSequence, does it make sense to cache the result of the length calculation? > As in change: > ==== > for (int i = 0; i< Math.min(cs1.length(), cs2.length()); i++) { > ==== > for (int i = 0, len = Math.min(cs1.length(), cs2.length()); i< len; i++) { > ==== > > Jason > ________________________________________ > From: core-libs-dev on behalf of Joe Wang > Sent: Thursday, February 8, 2018 6:47 PM > To: core-libs-dev > Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer > > Hi all, > > The CSR for the enhancement is now approved. Thanks Joe! > > The webrev has been updated accordingly. Please let me know if you have > any further comment on the implementation. > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Thanks, > Joe > > On 2/2/2018 12:46 PM, Joe Wang wrote: >> Thanks Jason. Will update that accordingly. >> >> Best, >> Joe >> >> On 2/2/2018 11:22 AM, Jason Mehrens wrote: >>> Joe, >>> >>> The identity check in CS.compare makes sense. However, it won't be >>> null hostile if we call CS.compare(null, null) and that doesn't seem >>> right. >>> Usually when writing comparator classes I end up with: >>> === >>> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >>> return 0; >>> } >>> === >>> >>> Jason >>> ________________________________________ >>> From: core-libs-dev on >>> behalf of Joe Wang >>> Sent: Friday, February 2, 2018 1:01 PM >>> To: core-libs-dev >>> Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, >>> StringBuilder, and StringBuffer >>> >>> Hi, >>> >>> Thanks all for comments and suggestions. I've updated the webrev. Please >>> review. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> Thanks, >>> Joe >>> >>> On 1/31/2018 9:31 PM, Joe Wang wrote: >>>> Hi Tagir, >>>> >>>> Thanks for the comment. I will consider adding that to the javadoc >>>> emphasizing that the comparison is performed from 0 to length() - 1 of >>>> the two sequences. >>>> >>>> Best, >>>> Joe >>>> >>>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>>> Hello! >>>>> >>>>> An AbstractStringBuilder#compareTo implementation is wrong. You cannot >>>>> simply compare the whole byte array. Here's the test-case: >>>>> >>>>> public class Test { >>>>> public static void main(String[] args) { >>>>> StringBuilder sb1 = new StringBuilder("test1"); >>>>> StringBuilder sb2 = new StringBuilder("test2"); >>>>> sb1.setLength(4); >>>>> sb2.setLength(4); >>>>> System.out.println(sb1.compareTo(sb2)); >>>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>>> } >>>>> } >>>>> >>>>> We truncated the stringbuilders making their content equal, so >>>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >>>>> the original content (before the truncation) as truncation, of course, >>>>> does not zero the truncated bytes, neither does it reallocate the >>>>> array (unless explicitly asked via trimToSize). >>>>> >>>>> With best regards, >>>>> Tagir Valeev. >>>>> >>>>> >>>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>>> StringBuffer. >>>>>> >>>>>> The Comparable implementations for StringBuilder/Buffer are similar >>>>>> to that >>>>>> of String, allowing comparison operations between two >>>>>> StringBuilders/Buffers, e.g. >>>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>>> method >>>>>> 'compare' is added instead of implementing the Comparable interface. >>>>>> This >>>>>> 'compare' method may take CharSequence implementations such as >>>>>> String, >>>>>> StringBuilder and StringBuffer, making it possible to perform >>>>>> comparison >>>>>> among them. The previous example for example is equivalent to >>>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>>> >>>>>> Tests for java.base have been independent from each other. The new >>>>>> tests are >>>>>> therefore created to have no dependency on each other or sharing any >>>>>> code. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>> >>>>>> Thanks, >>>>>> Joe From karen.kinnear at oracle.com Mon Feb 12 19:39:33 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Mon, 12 Feb 2018 14:39:33 -0500 Subject: RFR 8184289: Obsolete -XX:+UnsyncloadClass and -XX:+MustCallLoadClassInternal options In-Reply-To: References: <87cef54c-8de9-1cd0-27c8-cabc6d2aa13d@oracle.com> <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> Message-ID: <391F0F9C-9676-47E1-AE0C-B7CC1153267B@oracle.com> Harold, Thanks for doing this. I think you told me that 1) the version change has made it in 2) you also put 12 as an expiration date 3) you are running the ParallelClassLoading tests with the remaining two flags (you?ve already run them without any flags): AllowParallelDefineClass = true and AlwaysLockClassLoader=true In terms of the guarantee in question // For UnsyncloadClass only 848 // If they got a linkageError, check if a parallel class load succeeded. 849 // If it did, then for bytecode resolution the specification requires 850 // that we return the same result we did for the other thread, i.e. the 851 // successfully loaded InstanceKlass 852 // Should not get here for classloaders that support parallelism 853 // with the new cleaner mechanism, even with AllowParallelDefineClass 854 // Bootstrap goes through here to allow for an extra guarantee check 855 if (UnsyncloadClass || (class_loader.is_null())) { 856 if (k == NULL && HAS_PENDING_EXCEPTION 857 && PENDING_EXCEPTION->is_a(SystemDictionary::LinkageError_klass())) { 858 MutexLocker mu(SystemDictionary_lock, THREAD); 859 InstanceKlass* check = find_class(d_hash, name, dictionary); 860 if (check != NULL) { 861 // Klass is already loaded, so just use it 862 k = check; 863 CLEAR_PENDING_EXCEPTION; 864 guarantee((!class_loader.is_null()), "dup definition for bootstrap loader?"); 865 } 866 } 867 } 1) I agree you can remove the entire section - the guarantee was there for future proofing in case we ever allowed parallel class loading of the same class for the null loader and to make sure I didn?t have any logic holes. - I would not put an assertion for the first half of the condition - I would remove completely - the code currently prevents parallel class loading of the same class for the null loader at: resolve_instance_class_or_null see line 785 ? while (!class_has_been_loaded && old probe && old probe->instance_load_in_progress()) { // case x: bootstrap class loader: prevent futile class loading, // wait on first requestor if (class_loader.is_null()) { SystemDictionary_lock->wait(); This logic means that there is a registered INSTANCE_LOAD on this placeholder entry. Other minor comments (sorry if you already got these and I missed them in earlier emails) - all in SystemDictionary.cpp 1. line 72 comment ?Five cases:? -> ?Four cases:? So you removed case 3 and renumbered, so old references to case 4 -> case 3 ,and old references to case 5 become case 4: So - line 786, ?Case 4? -> ?case 3? thanks, Karen > On Feb 12, 2018, at 11:13 AM, harold seigel wrote: > > Hi Alan, > > Thanks for looking at this. > > Harold > > On 2/12/2018 2:52 AM, Alan Bateman wrote: >> On 12/02/2018 06:54, David Holmes wrote: >>> Hi Harold, >>> >>> Adding core-libs-dev so they are aware of the ClassLoader change. >> Thanks, that part is okay and good to see this going away. >> >> -Alan > From vitalyd at gmail.com Mon Feb 12 20:36:59 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 12 Feb 2018 15:36:59 -0500 Subject: IllegalAccessException trying to load a ResourceBundle in 9u181 Message-ID: Hi all, I'm not sure if core-libs is the right mailing list for jigsaw/modules questions these days (rather than jigsaw-dev), so please feel free to forward this there if it's the more appropriate list. I have the following code carried over from java 8 (actually much earlier than that, but that's beside the point): final Resource rb = ResourceBundle.getBundle("sun.security.util.AuthResources"); In 9u181, this fails with: Exception in thread "main" java.util.MissingResourceException: Can't find bundle for base name sun.security.util.AuthResources, locale en_US at java.base/java.util.ResourceBundle. throwMissingResourceException(ResourceBundle.java:2045) at java.base/java.util.ResourceBundle.getBundleImpl( ResourceBundle.java:1679) at java.base/java.util.ResourceBundle.getBundleImpl( ResourceBundle.java:1583) at java.base/java.util.ResourceBundle.getBundleImpl( ResourceBundle.java:1546) at java.base/java.util.ResourceBundle.getBundle( ResourceBundle.java:838) Caused by: java.lang.IllegalAccessException: unnamed module can't load sun.security.util.AuthResources in module java.base at java.base/java.util.ResourceBundle$Control. newBundle(ResourceBundle.java:3167) at java.base/java.util.ResourceBundle.loadBundle( ResourceBundle.java:1984) at java.base/java.util.ResourceBundle.findBundle( ResourceBundle.java:1766) at java.base/java.util.ResourceBundle.findBundle( ResourceBundle.java:1718) at java.base/java.util.ResourceBundle.findBundle( ResourceBundle.java:1718) at java.base/java.util.ResourceBundle.getBundleImpl( ResourceBundle.java:1652) ... 4 more Not surprising given encapsulation. So I tried running with "--add-exports java.base/sun.security.util=ALL-UNNAMED" but that still fails, which *is* surprising based on my understanding. I also tried (for good measure) with a similar add-opens but no luck there either. I feel like I must be missing something obvious here, or doing something silly. What's the right way to allow the getBundle call to succeed here? Thanks From harold.seigel at oracle.com Mon Feb 12 20:50:55 2018 From: harold.seigel at oracle.com (harold seigel) Date: Mon, 12 Feb 2018 15:50:55 -0500 Subject: RFR 8184289: Obsolete -XX:+UnsyncloadClass and -XX:+MustCallLoadClassInternal options In-Reply-To: <391F0F9C-9676-47E1-AE0C-B7CC1153267B@oracle.com> References: <87cef54c-8de9-1cd0-27c8-cabc6d2aa13d@oracle.com> <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> <391F0F9C-9676-47E1-AE0C-B7CC1153267B@oracle.com> Message-ID: <6683ecf8-8a93-a043-106e-98f9ec2db37a@oracle.com> Hi Karen, Thanks for looking at this! I re-ran the ParallelClassLoading tests with no options, with -XX:+AllowParallelDefineClass, and with -XX:+AlwaysLockClassLoader, and all the tests passed.? I also determined that the few Mach5 regression test failures that I encountered were unrelated to my change. Please see this latest webrev that adds 12 as an expiration date, removes the 'guarantee' section, and fixes the comment at line 786 of systemDictionary.cpp. http://cr.openjdk.java.net/~hseigel/bug_8184289.3/webrev/index.html Thanks, Harold On 2/12/2018 2:39 PM, Karen Kinnear wrote: > Harold, > > Thanks for doing this. > > I think you told me that > 1) the version change has made it in > 2) you also put 12 as an expiration date > 3) you are running the ParallelClassLoading tests with the remaining > two flags (you?ve already > run them without any flags): > ? AllowParallelDefineClass = true and AlwaysLockClassLoader=true > > In terms of the guarantee in question > // For UnsyncloadClass only > 848 // If they got a linkageError, check if a parallel class load succeeded. > 849 // If it did, then for bytecode resolution the specification requires > 850 // that we return the same result we did for the other thread, i.e. the > 851 // successfully loaded InstanceKlass > 852 // Should not get here for classloaders that support parallelism > 853 // with the new cleaner mechanism, even with AllowParallelDefineClass > 854 // Bootstrap goes through here to allow for an extra guarantee check > 855 if (UnsyncloadClass || (class_loader.is_null())) { > 856 if (k == NULL && HAS_PENDING_EXCEPTION > 857 && PENDING_EXCEPTION->is_a(SystemDictionary::LinkageError_klass())) { > 858 MutexLocker mu(SystemDictionary_lock, THREAD); > 859 InstanceKlass* check = find_class(d_hash, name, dictionary); > 860 if (check != NULL) { > 861 // Klass is already loaded, so just use it > 862 k = check; > 863 CLEAR_PENDING_EXCEPTION; > 864 guarantee((!class_loader.is_null()), "dup definition for bootstrap loader?"); > 865 } > 866 } > 867 } > > ? 1) I agree you can remove the entire section > ? ? ? - the guarantee was there for future proofing in case we ever > allowed parallel class loading of the > ? ? ? ? same class for the null loader and to make sure I didn?t have > any logic holes. > ? ? ? - I would not put an assertion for the first half of the > condition - I would remove completely > ? ? ? - the code currently prevents parallel class loading of the same > class for the null loader at: > resolve_instance_class_or_null see line 785 ? > ? while (!class_has_been_loaded && old probe && old > probe->instance_load_in_progress()) { > ? ? ?// ?case x: bootstrap class loader: prevent futile class loading, > ? ? ?// wait on first requestor > ? ? ?if (class_loader.is_null()) { > ? ? ? ? SystemDictionary_lock->wait(); > > This logic means that there is a registered INSTANCE_LOAD on this > placeholder entry. > > > Other minor comments (sorry if you already got these and I missed them > in earlier emails) > - all in SystemDictionary.cpp > > 1. line 72 comment ?Five cases:? -> ?Four cases:? > So you removed case 3 and renumbered, so old references to case 4 -> > case 3 ,and old references > to case 5 become case 4: > > So - line 786, ?Case 4? -> ?case 3? > > thanks, > Karen > > >> On Feb 12, 2018, at 11:13 AM, harold seigel > > wrote: >> >> Hi Alan, >> >> Thanks for looking at this. >> >> Harold >> >> On 2/12/2018 2:52 AM, Alan Bateman wrote: >>> On 12/02/2018 06:54, David Holmes wrote: >>>> Hi Harold, >>>> >>>> Adding core-libs-dev so they are aware of the ClassLoader change. >>> Thanks, that part is okay and good to see this going away. >>> >>> -Alan >> > From karen.kinnear at oracle.com Mon Feb 12 20:55:53 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Mon, 12 Feb 2018 15:55:53 -0500 Subject: RFR 8184289: Obsolete -XX:+UnsyncloadClass and -XX:+MustCallLoadClassInternal options In-Reply-To: <6683ecf8-8a93-a043-106e-98f9ec2db37a@oracle.com> References: <87cef54c-8de9-1cd0-27c8-cabc6d2aa13d@oracle.com> <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> <391F0F9C-9676-47E1-AE0C-B7CC1153267B@oracle.com> <6683ecf8-8a93-a043-106e-98f9ec2db37a@oracle.com> Message-ID: <3640D0F2-3ADC-4F79-A983-49C7571972DA@oracle.com> Harold, Looks good. Many thanks! Karen > On Feb 12, 2018, at 3:50 PM, harold seigel wrote: > > Hi Karen, > > Thanks for looking at this! > > I re-ran the ParallelClassLoading tests with no options, with -XX:+AllowParallelDefineClass, and with -XX:+AlwaysLockClassLoader, and all the tests passed. I also determined that the few Mach5 regression test failures that I encountered were unrelated to my change. > Please see this latest webrev that adds 12 as an expiration date, removes the 'guarantee' section, and fixes the comment at line 786 of systemDictionary.cpp. > http://cr.openjdk.java.net/~hseigel/bug_8184289.3/webrev/index.html > Thanks, Harold > > On 2/12/2018 2:39 PM, Karen Kinnear wrote: >> Harold, >> >> Thanks for doing this. >> >> I think you told me that >> 1) the version change has made it in >> 2) you also put 12 as an expiration date >> 3) you are running the ParallelClassLoading tests with the remaining two flags (you?ve already >> run them without any flags): >> AllowParallelDefineClass = true and AlwaysLockClassLoader=true >> >> In terms of the guarantee in question >> // For UnsyncloadClass only >> 848 // If they got a linkageError, check if a parallel class load succeeded. >> 849 // If it did, then for bytecode resolution the specification requires >> 850 // that we return the same result we did for the other thread, i.e. the >> 851 // successfully loaded InstanceKlass >> 852 // Should not get here for classloaders that support parallelism >> 853 // with the new cleaner mechanism, even with AllowParallelDefineClass >> 854 // Bootstrap goes through here to allow for an extra guarantee check >> 855 if (UnsyncloadClass || (class_loader.is_null())) { >> 856 if (k == NULL && HAS_PENDING_EXCEPTION >> 857 && PENDING_EXCEPTION->is_a(SystemDictionary::LinkageError_klass())) { >> 858 MutexLocker mu(SystemDictionary_lock, THREAD); >> 859 InstanceKlass* check = find_class(d_hash, name, dictionary); >> 860 if (check != NULL) { >> 861 // Klass is already loaded, so just use it >> 862 k = check; >> 863 CLEAR_PENDING_EXCEPTION; >> 864 guarantee((!class_loader.is_null()), "dup definition for bootstrap loader?"); >> 865 } >> 866 } >> 867 } >> >> 1) I agree you can remove the entire section >> - the guarantee was there for future proofing in case we ever allowed parallel class loading of the >> same class for the null loader and to make sure I didn?t have any logic holes. >> - I would not put an assertion for the first half of the condition - I would remove completely >> - the code currently prevents parallel class loading of the same class for the null loader at: >> resolve_instance_class_or_null see line 785 ? >> while (!class_has_been_loaded && old probe && old probe->instance_load_in_progress()) { >> // case x: bootstrap class loader: prevent futile class loading, >> // wait on first requestor >> if (class_loader.is_null()) { >> SystemDictionary_lock->wait(); >> >> This logic means that there is a registered INSTANCE_LOAD on this placeholder entry. >> >> >> Other minor comments (sorry if you already got these and I missed them in earlier emails) >> - all in SystemDictionary.cpp >> >> 1. line 72 comment ?Five cases:? -> ?Four cases:? >> So you removed case 3 and renumbered, so old references to case 4 -> case 3 ,and old references >> to case 5 become case 4: >> >> So - line 786, ?Case 4? -> ?case 3? >> >> thanks, >> Karen >> >> >>> On Feb 12, 2018, at 11:13 AM, harold seigel > wrote: >>> >>> Hi Alan, >>> >>> Thanks for looking at this. >>> >>> Harold >>> >>> On 2/12/2018 2:52 AM, Alan Bateman wrote: >>>> On 12/02/2018 06:54, David Holmes wrote: >>>>> Hi Harold, >>>>> >>>>> Adding core-libs-dev so they are aware of the ClassLoader change. >>>> Thanks, that part is okay and good to see this going away. >>>> >>>> -Alan >>> >> > From Roger.Riggs at Oracle.com Mon Feb 12 22:36:14 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 12 Feb 2018 17:36:14 -0500 Subject: RFR: 8195976: Add JNDI test javax/naming/dns/AttributeTests/GetAny.java In-Reply-To: References: <9CAEA585-EB5C-4F55-88D3-124E589778EB@oracle.com> <441789eb-a0ed-5eee-1d74-af5a888e9e82@oracle.com> Message-ID: Hi Chris, Looks fine. These are only positive tests, are there negative tests planned? Thanks, Roger On 1/23/2018 5:14 AM, Chris Yin wrote: > Thank you Alan, I just moved it to com/sun/jndi/dns/ as you suggested and removed unused "@modules jdk.naming.dns/com.sun.jndi.dns?, updated webrev as below, thanks > > http://cr.openjdk.java.net/~xiaofeya/8195976/webrev.00/ > > Regards, > Chris > >> On 23 Jan 2018, at 3:53 PM, Alan Bateman wrote: >> >> >> >> On 23/01/2018 07:01, Chris Yin wrote: >>> Please review the added JNDI test javax/naming/dns/AttributeTests/GetAny.java, thanks >>> >>> >> You may want to move it to com/sun/jndi/dns so that it's with the other tests for the DNS provider (as there is no javax.naming.dns API). Also I suspect you don't need "@modules jdk.naming.dns/com.sun.jndi.dns" as it doesn't appear to make direct use of the classes in the implementation. >> >> -Alan From Roger.Riggs at Oracle.com Mon Feb 12 22:37:53 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 12 Feb 2018 17:37:53 -0500 Subject: RFR: 8196770: Add JNDI test com/sun/jndi/ldap/blits/AddTests/AddNewEntry.java In-Reply-To: <7C7F5163-0680-4796-BAA7-AA8D6EEB3635@oracle.com> References: <7C7F5163-0680-4796-BAA7-AA8D6EEB3635@oracle.com> Message-ID: <846a77a6-0272-5010-184e-bb41742dde73@Oracle.com> Hi Chris, I don't know DNS details well enough to validate the test assertions. Otherwise, the code is fine. Regards, Roger On 2/5/2018 3:15 AM, Chris Yin wrote: > Please review the added JNDI test com/sun/jndi/ldap/blits/AddTests/AddNewEntry.java, thanks > > It?s to verify capability to add a new entry to the directory using the ADD operation. This test will use LDAPServer to playback dump ldap message to simulate test environment. > > bug: https://bugs.openjdk.java.net/browse/JDK-8196770 > webrev: http://cr.openjdk.java.net/~xiaofeya/8196770/webrev.00/ > > Regards > Chris From david.holmes at oracle.com Tue Feb 13 00:41:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Feb 2018 10:41:19 +1000 Subject: RFR 8184289: Obsolete -XX:+UnsyncloadClass and -XX:+MustCallLoadClassInternal options In-Reply-To: <6683ecf8-8a93-a043-106e-98f9ec2db37a@oracle.com> References: <87cef54c-8de9-1cd0-27c8-cabc6d2aa13d@oracle.com> <069da93d-c595-5f3f-9597-9d02780a0b0d@oracle.com> <8e2285c7-809b-c229-f1bc-cd45d441bc15@oracle.com> <391F0F9C-9676-47E1-AE0C-B7CC1153267B@oracle.com> <6683ecf8-8a93-a043-106e-98f9ec2db37a@oracle.com> Message-ID: <92760d99-4314-cbb8-7bce-36a4599cf0d3@oracle.com> Looks good! Thanks, David On 13/02/2018 6:50 AM, harold seigel wrote: > Hi Karen, > > Thanks for looking at this! > > I re-ran the ParallelClassLoading tests with no options, with > -XX:+AllowParallelDefineClass, and with -XX:+AlwaysLockClassLoader, and > all the tests passed.? I also determined that the few Mach5 regression > test failures that I encountered were unrelated to my change. > > Please see this latest webrev that adds 12 as an expiration date, > removes the 'guarantee' section, and fixes the comment at line 786 of > systemDictionary.cpp. > > http://cr.openjdk.java.net/~hseigel/bug_8184289.3/webrev/index.html > > Thanks, Harold > > On 2/12/2018 2:39 PM, Karen Kinnear wrote: >> Harold, >> >> Thanks for doing this. >> >> I think you told me that >> 1) the version change has made it in >> 2) you also put 12 as an expiration date >> 3) you are running the ParallelClassLoading tests with the remaining >> two flags (you?ve already >> run them without any flags): >> ? AllowParallelDefineClass = true and AlwaysLockClassLoader=true >> >> In terms of the guarantee in question >> // For UnsyncloadClass only >> 848 // If they got a linkageError, check if a parallel class load succeeded. >> 849 // If it did, then for bytecode resolution the specification requires >> 850 // that we return the same result we did for the other thread, i.e. the >> 851 // successfully loaded InstanceKlass >> 852 // Should not get here for classloaders that support parallelism >> 853 // with the new cleaner mechanism, even with AllowParallelDefineClass >> 854 // Bootstrap goes through here to allow for an extra guarantee check >> 855 if (UnsyncloadClass || (class_loader.is_null())) { >> 856 if (k == NULL && HAS_PENDING_EXCEPTION >> 857 && PENDING_EXCEPTION->is_a(SystemDictionary::LinkageError_klass())) { >> 858 MutexLocker mu(SystemDictionary_lock, THREAD); >> 859 InstanceKlass* check = find_class(d_hash, name, dictionary); >> 860 if (check != NULL) { >> 861 // Klass is already loaded, so just use it >> 862 k = check; >> 863 CLEAR_PENDING_EXCEPTION; >> 864 guarantee((!class_loader.is_null()), "dup definition for bootstrap loader?"); >> 865 } >> 866 } >> 867 } >> >> ? 1) I agree you can remove the entire section >> ? ? ? - the guarantee was there for future proofing in case we ever >> allowed parallel class loading of the >> ? ? ? ? same class for the null loader and to make sure I didn?t have >> any logic holes. >> ? ? ? - I would not put an assertion for the first half of the >> condition - I would remove completely >> ? ? ? - the code currently prevents parallel class loading of the same >> class for the null loader at: >> resolve_instance_class_or_null see line 785 ? >> ? while (!class_has_been_loaded && old probe && old >> probe->instance_load_in_progress()) { >> ? ? ?// ?case x: bootstrap class loader: prevent futile class loading, >> ? ? ?// wait on first requestor >> ? ? ?if (class_loader.is_null()) { >> ? ? ? ? SystemDictionary_lock->wait(); >> >> This logic means that there is a registered INSTANCE_LOAD on this >> placeholder entry. >> >> >> Other minor comments (sorry if you already got these and I missed them >> in earlier emails) >> - all in SystemDictionary.cpp >> >> 1. line 72 comment ?Five cases:? -> ?Four cases:? >> So you removed case 3 and renumbered, so old references to case 4 -> >> case 3 ,and old references >> to case 5 become case 4: >> >> So - line 786, ?Case 4? -> ?case 3? >> >> thanks, >> Karen >> >> >>> On Feb 12, 2018, at 11:13 AM, harold seigel >> > wrote: >>> >>> Hi Alan, >>> >>> Thanks for looking at this. >>> >>> Harold >>> >>> On 2/12/2018 2:52 AM, Alan Bateman wrote: >>>> On 12/02/2018 06:54, David Holmes wrote: >>>>> Hi Harold, >>>>> >>>>> Adding core-libs-dev so they are aware of the ClassLoader change. >>>> Thanks, that part is okay and good to see this going away. >>>> >>>> -Alan >>> >> > From mandy.chung at oracle.com Tue Feb 13 00:44:41 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 12 Feb 2018 16:44:41 -0800 Subject: IllegalAccessException trying to load a ResourceBundle in 9u181 In-Reply-To: References: Message-ID: On 2/12/18 12:36 PM, Vitaly Davidovich wrote: > Hi all, > > I'm not sure if core-libs is the right mailing list for jigsaw/modules > questions these days (rather than jigsaw-dev), so please feel free to > forward this there if it's the more appropriate list. cc'ing? jigsaw-dev > I have the following code carried over from java 8 (actually much earlier > than that, but that's beside the point): > final Resource rb = > ResourceBundle.getBundle("sun.security.util.AuthResources"); Resource bundle follows the same encapsulation rule as described in Module::getResourceAsStream [1] except that a resource bundle is considered as a resource regardless of its format. ResourceBundle.getBundle(String baseName) finds the specified resource bundle from the caller's module.? It will first search the resource bundle local in the caller's module (via Module::getResourceAsStream) and then using the caller's class loader to search for the resource (via ClassLoader::getResourceAsStream).?? Since the caller is unnamed module in your case, for it to access "sun.security.util.AuthResources", java.base/sun.security.util has to be open unconditionally and there is no CLI option to do that. If you call ResourceBundle.getBundle("sun.security.util.AuthResources", Object.class.getModule()) specifying the module of the resource bundle, then you can break the encapsulation by `--add-opens java.base/sun.security.util=ALL-UNNAMED` I'm a bit surprised that you depend on JDK internal resource bundle.? Can you help us understand why you use it? Mandy [1] https://download.java.net/java/jdk10/docs/api/java/util/ResourceBundle.html#getBundle(java.lang.String,java.util.Locale,java.lang.Module) From david.holmes at oracle.com Tue Feb 13 01:14:20 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Feb 2018 11:14:20 +1000 Subject: IllegalAccessException trying to load a ResourceBundle in 9u181 In-Reply-To: References: Message-ID: <1712873d-be93-a765-4726-7936bcc7a7ba@oracle.com> Hi Vitaly, See this thread: http://mail.openjdk.java.net/pipermail/jigsaw-dev/2018-February/013584.html David On 13/02/2018 6:36 AM, Vitaly Davidovich wrote: > Hi all, > > I'm not sure if core-libs is the right mailing list for jigsaw/modules > questions these days (rather than jigsaw-dev), so please feel free to > forward this there if it's the more appropriate list. > > I have the following code carried over from java 8 (actually much earlier > than that, but that's beside the point): > final Resource rb = > ResourceBundle.getBundle("sun.security.util.AuthResources"); > > In 9u181, this fails with: > > Exception in thread "main" java.util.MissingResourceException: Can't find > bundle for base name sun.security.util.AuthResources, locale en_US > > at java.base/java.util.ResourceBundle. > throwMissingResourceException(ResourceBundle.java:2045) > > at java.base/java.util.ResourceBundle.getBundleImpl( > ResourceBundle.java:1679) > > at java.base/java.util.ResourceBundle.getBundleImpl( > ResourceBundle.java:1583) > > at java.base/java.util.ResourceBundle.getBundleImpl( > ResourceBundle.java:1546) > > at java.base/java.util.ResourceBundle.getBundle( > ResourceBundle.java:838) > > > > Caused by: java.lang.IllegalAccessException: unnamed module can't load > sun.security.util.AuthResources in module java.base > > at java.base/java.util.ResourceBundle$Control. > newBundle(ResourceBundle.java:3167) > > at java.base/java.util.ResourceBundle.loadBundle( > ResourceBundle.java:1984) > > at java.base/java.util.ResourceBundle.findBundle( > ResourceBundle.java:1766) > > at java.base/java.util.ResourceBundle.findBundle( > ResourceBundle.java:1718) > > at java.base/java.util.ResourceBundle.findBundle( > ResourceBundle.java:1718) > > at java.base/java.util.ResourceBundle.getBundleImpl( > ResourceBundle.java:1652) > > ... 4 more > > > Not surprising given encapsulation. So I tried running with "--add-exports > java.base/sun.security.util=ALL-UNNAMED" but that still fails, which *is* > surprising based on my understanding. I also tried (for good measure) with > a similar add-opens but no luck there either. > > > I feel like I must be missing something obvious here, or doing something > silly. What's the right way to allow the getBundle call to succeed here? > > > Thanks > From xu.y.yin at oracle.com Tue Feb 13 01:29:36 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 13 Feb 2018 09:29:36 +0800 Subject: RFR: 8195976: Add JNDI test javax/naming/dns/AttributeTests/GetAny.java In-Reply-To: References: <9CAEA585-EB5C-4F55-88D3-124E589778EB@oracle.com> <441789eb-a0ed-5eee-1d74-af5a888e9e82@oracle.com> Message-ID: <5E0CC182-4CCF-4D9A-B242-E405CAA09E82@oracle.com> Hi, Roger Many thanks for your review. Yes, there are also negative tests planned Thanks & Regards, Chris > On 13 Feb 2018, at 6:36 AM, Roger Riggs wrote: > > Hi Chris, > > Looks fine. > > These are only positive tests, are there negative tests planned? > > Thanks, Roger > > On 1/23/2018 5:14 AM, Chris Yin wrote: >> Thank you Alan, I just moved it to com/sun/jndi/dns/ as you suggested and removed unused "@modules jdk.naming.dns/com.sun.jndi.dns?, updated webrev as below, thanks >> >> http://cr.openjdk.java.net/~xiaofeya/8195976/webrev.00/ >> >> Regards, >> Chris >> >>> On 23 Jan 2018, at 3:53 PM, Alan Bateman wrote: >>> >>> >>> >>> On 23/01/2018 07:01, Chris Yin wrote: >>>> Please review the added JNDI test javax/naming/dns/AttributeTests/GetAny.java, thanks >>>> >>>> >>> You may want to move it to com/sun/jndi/dns so that it's with the other tests for the DNS provider (as there is no javax.naming.dns API). Also I suspect you don't need "@modules jdk.naming.dns/com.sun.jndi.dns" as it doesn't appear to make direct use of the classes in the implementation. >>> >>> -Alan > From xu.y.yin at oracle.com Tue Feb 13 01:33:14 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 13 Feb 2018 09:33:14 +0800 Subject: RFR: 8196770: Add JNDI test com/sun/jndi/ldap/blits/AddTests/AddNewEntry.java In-Reply-To: <846a77a6-0272-5010-184e-bb41742dde73@Oracle.com> References: <7C7F5163-0680-4796-BAA7-AA8D6EEB3635@oracle.com> <846a77a6-0272-5010-184e-bb41742dde73@Oracle.com> Message-ID: Thank you, Roger Regards, Chris > On 13 Feb 2018, at 6:37 AM, Roger Riggs wrote: > > Hi Chris, > > I don't know DNS details well enough to validate the test assertions. > Otherwise, the code is fine. > > Regards, Roger > > > On 2/5/2018 3:15 AM, Chris Yin wrote: >> Please review the added JNDI test com/sun/jndi/ldap/blits/AddTests/AddNewEntry.java, thanks >> >> It?s to verify capability to add a new entry to the directory using the ADD operation. This test will use LDAPServer to playback dump ldap message to simulate test environment. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8196770 >> webrev: http://cr.openjdk.java.net/~xiaofeya/8196770/webrev.00/ >> >> Regards >> Chris > From xueming.shen at oracle.com Tue Feb 13 04:09:08 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 12 Feb 2018 20:09:08 -0800 Subject: RFR 8197462 : Inconsistent exception messages for invalid capturing group names In-Reply-To: References: Message-ID: <5A8264E4.9030803@oracle.com> Hi Ivan, The code handles group name was added later. So "historically" those cases trigger "unknow look-behind group" when the first character after "<" is not "=" or "?". With the addition of the group name support, it's actually hard to say which one is more accurate, incorrect group name or incorrect "looks behind". Sure with a tailing ">" it might be more desired to lean to group name. It's definitely a bug not to check whether or not the first char is alpha for \\k<. I'm fine with the proposed change. Thanks, Sherman On 2/8/18, 8:32 PM, Ivan Gerasimov wrote: > Hello! > > Capturing group name can be used in a regular expression in two > contexts: When introducing a group (?...) or when referring it > \k. > If the name is invalid (i.e. does not start with a Latin letter, or > contains wrong chars) then we may see different error messages, some > of which look confusing. > > Here are examples of the messages produced by the current JDK: > Unknown look-behind group near index 3 > (?<>) > ^ > named capturing group is missing trailing '>' near index 4 > \\k<> > ^ > Unknown look-behind group near index 4 > (?<.>) > ^ > (named capturing group <.> does not exit near index 4 > \\k<.> > ^ > named capturing group is missing trailing '>' near index 4 > (?) > ^ > named capturing group is missing trailing '>' near index 4 > \\k > ^ > > In particular, this diversity is caused by that the internal > Pattern.groupname() function lacks a check for the very first > character of the name. > So that when \k is parsed, the first char is always accepted, no > matter what it was. > > Some cleanup was also done along the way. > > Would you please help review the fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8197462 > WEBREV: http://cr.openjdk.java.net/~igerasim/8197462/00/webrev/ > > Thanks in advance! > From xueming.shen at oracle.com Tue Feb 13 06:24:17 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 12 Feb 2018 22:24:17 -0800 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer Message-ID: <5A828491.8020301@oracle.com> Hi, Please help review the proposal to add following constructors and methods in String class to take ByteBuffer as the input and output data buffer. public String(ByteBuffer bytes, Charset cs); public String(ByteBuffer bytes, String csname); public int getBytes(byte[] dst, int offset, Charset cs); public int getBytes(byte[] dst, int offset, String csname); public int getBytes(ByteBuffer bytes, Charset cs); public int getBytes(ByteBuffer bytes, Charset csn); This is based on the change proposed by Richard in jdk9 time frame [1]. But since we were fully occupied by compact string support work back then, this one did not make into the top jdk9 priority list. issue: https://bugs.openjdk.java.net/browse/JDK-8021560 webrev: http://cr.openjdk.java.net/~sherman/8021560/webrev The implementation in StringCoding class is a little "duplicated" for various types of encoding/decoding variants. There is definitely room to further consolidate. But I would assume it might be better to keep the new code path separated from the existing "fully tuned" methods, for now, to avoid possible performance regression for existing String methods. Thanks, Sherman [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029672.html From scolebourne at joda.org Tue Feb 13 08:13:35 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 13 Feb 2018 08:13:35 +0000 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: <5A828491.8020301@oracle.com> References: <5A828491.8020301@oracle.com> Message-ID: Is it necessary to add the "String csname" variants at this point in Java's lifetime? Don't most people use instances of Charset now? Stephen On 13 February 2018 at 06:24, Xueming Shen wrote: > Hi, > > Please help review the proposal to add following constructors and methods in > String > class to take ByteBuffer as the input and output data buffer. > > public String(ByteBuffer bytes, Charset cs); > public String(ByteBuffer bytes, String csname); > public int getBytes(byte[] dst, int offset, Charset cs); > public int getBytes(byte[] dst, int offset, String csname); > public int getBytes(ByteBuffer bytes, Charset cs); > public int getBytes(ByteBuffer bytes, Charset csn); > > This is based on the change proposed by Richard in jdk9 time frame [1]. But > since > we were fully occupied by compact string support work back then, this one > did > not make into the top jdk9 priority list. > > issue: https://bugs.openjdk.java.net/browse/JDK-8021560 > webrev: http://cr.openjdk.java.net/~sherman/8021560/webrev > > The implementation in StringCoding class is a little "duplicated" for > various types > of encoding/decoding variants. There is definitely room to further > consolidate. But > I would assume it might be better to keep the new code path separated from > the > existing "fully tuned" methods, for now, to avoid possible performance > regression > for existing String methods. > > Thanks, > Sherman > > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029672.html From Alan.Bateman at oracle.com Tue Feb 13 08:41:01 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Feb 2018 08:41:01 +0000 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: <5A828491.8020301@oracle.com> References: <5A828491.8020301@oracle.com> Message-ID: <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> On 13/02/2018 06:24, Xueming Shen wrote: > Hi, > > Please help review the proposal to add following constructors and > methods in String > class to take ByteBuffer as the input and output data buffer. > > public String(ByteBuffer bytes, Charset cs); > public String(ByteBuffer bytes, String csname); These constructors looks good (for the parameter names then I assume you meant "src" rather than "bytes" here). > public int getBytes(byte[] dst, int offset, Charset cs); > public int getBytes(byte[] dst, int offset, String csname); > public int getBytes(ByteBuffer bytes, Charset cs); > public int getBytes(ByteBuffer bytes, Charset csn); These four methods encode as many characters as possible into the destination byte[] or buffer but don't give any indication that the destination didn't have enough space to encode the entire string. I thus worry they could be a hazard and result in buggy code. If there is insufficient space then the user of the API doesn't know how many characters were encoded so it's not easy to substring and call getBytes again to encode the remaining characters. There is also the issue of how to size the destination. What would you think about having them fail when there is insufficient space? If they do fail then there is a side effect that they will have written to the destination so that would need to be documented too. -Alan From claes.redestad at oracle.com Tue Feb 13 10:29:17 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 13 Feb 2018 11:29:17 +0100 Subject: [11] RFR: 8197829: Optimize CLDRCalendarDataProviderImpl::retrieveInteger Message-ID: <9df825f3-680f-61df-644e-74e5a8247451@oracle.com> Hi, please review this improvement to the retrieveInteger method in CLDRCalendarDataProviderImpl. Bug: https://bugs.openjdk.java.net/browse/JDK-8197829 Webrev: http://cr.openjdk.java.net/~redestad/8197829/jdk.00/ This deals with a tiny startup regression that appeared in 10+36 on some applications and systems[1] Thanks! /Claes [1] Likely due to https://bugs.openjdk.java.net/browse/JDK-8190918 From vitalyd at gmail.com Tue Feb 13 13:03:15 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 13 Feb 2018 08:03:15 -0500 Subject: IllegalAccessException trying to load a ResourceBundle in 9u181 In-Reply-To: References: Message-ID: Hi Mandy, Thanks very much for your response. Some comments inline ... On Mon, Feb 12, 2018 at 7:44 PM, mandy chung wrote: > > > On 2/12/18 12:36 PM, Vitaly Davidovich wrote: > > Hi all, > > I'm not sure if core-libs is the right mailing list for jigsaw/modules > questions these days (rather than jigsaw-dev), so please feel free to > forward this there if it's the more appropriate list. > > > cc'ing jigsaw-dev > > I have the following code carried over from java 8 (actually much earlier > than that, but that's beside the point): > final Resource rb = > ResourceBundle.getBundle("sun.security.util.AuthResources"); > > > Resource bundle follows the same encapsulation rule as described in > Module::getResourceAsStream [1] except that a resource bundle is considered > as a resource regardless of its format. > Good to know - thanks. > > ResourceBundle.getBundle(String baseName) finds the specified resource > bundle from the caller's module. It will first search the resource bundle > local in the caller's module (via Module::getResourceAsStream) and then > using the caller's class loader to search for the resource (via > ClassLoader::getResourceAsStream). Since the caller is unnamed module > in your case, for it to access "sun.security.util.AuthResources", > java.base/sun.security.util has to be open unconditionally and there is no > CLI option to do that. > Understood. > > If you call ResourceBundle.getBundle("sun.security.util.AuthResources", > Object.class.getModule()) specifying the module of the resource bundle, > then you can break the encapsulation by `--add-opens > java.base/sun.security.util=ALL-UNNAMED` > So I tried calling getBundle(..., Object.class.getModule()), and that worked - great. Interestingly, I didn't have to add-opens (or add-export for that matter). Is that expected? > > I'm a bit surprised that you depend on JDK internal resource bundle. Can > you help us understand why you use it? > Sigh. Yes, this is an ugly wart in the system, with lots of historical baggage. The (really) short of it is we have a custom javax.security.auth.spi.LoginModule implementation, which loads this ResourceBundle (into a static final field). This is something that's being addressed in parallel, but I'm trying to make some progress with migrating the codebase to Java 9 in the meantime. I should also mention that the real problem I'm troubleshooting is a NoClassDefFoundError when this custom LoginModule is loaded. After some class load/resolution tracing, it appears to me that the impl is found but fails to initialize properly due to the ResourceBundle static field initialization. That's what started me down a path of trying to minimize the code and seeing what the issue is. It's also worth mentioning that I'm trying to run the real code on the 9u181 JVM but with code compiled on Java 8 (we have a bunch of compile-time issues to resolve so wanted to see if we can at least make progress on running on a 9 JVM). I would like to try patching this custom login module to use the new getBundle() overload that takes a Module, but I guess I'll need to compile it with a java 9 javac. So yeah, it's ugly unfortunately. Anyway, thanks for the help - I will try a few more things on my end. If you have any suggestions to deal with this frankenstein setup I described above, I'm all ears. > > Mandy > [1] https://download.java.net/java/jdk10/docs/api/java/util/ > ResourceBundle.html#getBundle(java.lang.String,java.util. > Locale,java.lang.Module) > From vitalyd at gmail.com Tue Feb 13 13:03:33 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 13 Feb 2018 08:03:33 -0500 Subject: IllegalAccessException trying to load a ResourceBundle in 9u181 In-Reply-To: <1712873d-be93-a765-4726-7936bcc7a7ba@oracle.com> References: <1712873d-be93-a765-4726-7936bcc7a7ba@oracle.com> Message-ID: On Mon, Feb 12, 2018 at 8:14 PM, David Holmes wrote: > Hi Vitaly, > > See this thread: > > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2018-Febru > ary/013584.html Thanks David! > > > David > > > On 13/02/2018 6:36 AM, Vitaly Davidovich wrote: > >> Hi all, >> >> I'm not sure if core-libs is the right mailing list for jigsaw/modules >> questions these days (rather than jigsaw-dev), so please feel free to >> forward this there if it's the more appropriate list. >> >> I have the following code carried over from java 8 (actually much earlier >> than that, but that's beside the point): >> final Resource rb = >> ResourceBundle.getBundle("sun.security.util.AuthResources"); >> >> In 9u181, this fails with: >> >> Exception in thread "main" java.util.MissingResourceException: Can't find >> bundle for base name sun.security.util.AuthResources, locale en_US >> >> at java.base/java.util.ResourceBundle. >> throwMissingResourceException(ResourceBundle.java:2045) >> >> at java.base/java.util.ResourceBundle.getBundleImpl( >> ResourceBundle.java:1679) >> >> at java.base/java.util.ResourceBundle.getBundleImpl( >> ResourceBundle.java:1583) >> >> at java.base/java.util.ResourceBundle.getBundleImpl( >> ResourceBundle.java:1546) >> >> at java.base/java.util.ResourceBundle.getBundle( >> ResourceBundle.java:838) >> >> >> >> Caused by: java.lang.IllegalAccessException: unnamed module can't load >> sun.security.util.AuthResources in module java.base >> >> at java.base/java.util.ResourceBundle$Control. >> newBundle(ResourceBundle.java:3167) >> >> at java.base/java.util.ResourceBundle.loadBundle( >> ResourceBundle.java:1984) >> >> at java.base/java.util.ResourceBundle.findBundle( >> ResourceBundle.java:1766) >> >> at java.base/java.util.ResourceBundle.findBundle( >> ResourceBundle.java:1718) >> >> at java.base/java.util.ResourceBundle.findBundle( >> ResourceBundle.java:1718) >> >> at java.base/java.util.ResourceBundle.getBundleImpl( >> ResourceBundle.java:1652) >> >> ... 4 more >> >> >> Not surprising given encapsulation. So I tried running with >> "--add-exports >> java.base/sun.security.util=ALL-UNNAMED" but that still fails, which *is* >> surprising based on my understanding. I also tried (for good measure) >> with >> a similar add-opens but no luck there either. >> >> >> I feel like I must be missing something obvious here, or doing something >> silly. What's the right way to allow the getBundle call to succeed here? >> >> >> Thanks >> >> From david.holmes at oracle.com Tue Feb 13 13:05:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Feb 2018 23:05:35 +1000 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: <360C5181-3B9E-4339-B460-04A8385F2461@oracle.com> References: <360C5181-3B9E-4339-B460-04A8385F2461@oracle.com> Message-ID: <6ae59d8a-578c-fe1e-b92c-deab72a109af@oracle.com> Lance, In Docs.gmk you seem to have missed this: 445 ################################################################################ 446 # Setup generation of the Java SE API documentation (javadoc + modulegraph) 447 448 # The Java SE module scope is just java.se.ee and its transitive indirect 449 # exports. 450 JAVASE_MODULES := java.se.ee 451 452 $(eval $(call SetupApiDocsGeneration, JAVASE_API, \ 453 MODULES := $(JAVASE_MODULES), \ 454 SHORT_NAME := $(JAVASE_SHORT_NAME), \ 455 LONG_NAME := $(JAVASE_LONG_NAME), \ 456 TARGET_DIR := $(IMAGES_OUTPUTDIR)/javase-docs/api, \ 457 )) which now results in a doc build hanging due to the non-existent java.se.ee module. It leads to execution of: grep -e @moduleGraph with no module-info.java argument, which causes grep to hang trying to read from stdin. At least that is what my analysis indicates. I simply set: JAVASE_MODULES := to work around it. David ----- On 8/02/2018 11:37 PM, Lance Andersen wrote: > >> On Feb 8, 2018, at 3:04 AM, Alan Bateman wrote: >> >> On 07/02/2018 16:57, Lance Andersen wrote: >>> Hi all, >>> >>> I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. >>> The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 >>> >>> The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ >>> >> 800 KLOC deleted, wonderful! >> >> The update to technology-summary.html page means its html title no longer matches the contents. We should probably change it to "JCP Technologies in JDK 11" for now. > > I updated the webrev. Thanks for catching that (btw we missed this for JDK 10) >> >> The removal of test cases from the tests in tools/launcher/modules removes most of the test coverage for the upgrade module path. We'll need to replace these sub-tests. Can you create an issue to track that? > > I can do that >> >> Everything else looks good and it's okay to track residual issues with other JIRA issues. I think the important thing is to get this monster patch into JDK builds soon so that libraries and the eco system can start to adjust. > > Thank you Alan for the review > > Best > Lance >> >> -Alan > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From Alan.Bateman at oracle.com Tue Feb 13 13:57:08 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Feb 2018 13:57:08 +0000 Subject: IllegalAccessException trying to load a ResourceBundle in 9u181 In-Reply-To: References: Message-ID: <46c81f70-6cb5-bad1-ccd3-c98201324c5b@oracle.com> On 13/02/2018 13:03, Vitaly Davidovich wrote: > : > So I tried calling getBundle(..., Object.class.getModule()), and that > worked - great. Interestingly, I didn't have to add-opens (or add-export > for that matter). Is that expected? > It works because the sun.security.util.AuthResources resource bundle for the default locale is in java.base. You'll get MissingResourceException once java.base is fully encapsulated (testing with --illegal-access=deny to try it). -Alan From vitalyd at gmail.com Tue Feb 13 14:05:35 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 13 Feb 2018 09:05:35 -0500 Subject: IllegalAccessException trying to load a ResourceBundle in 9u181 In-Reply-To: <46c81f70-6cb5-bad1-ccd3-c98201324c5b@oracle.com> References: <46c81f70-6cb5-bad1-ccd3-c98201324c5b@oracle.com> Message-ID: On Tue, Feb 13, 2018 at 8:57 AM, Alan Bateman wrote: > > > On 13/02/2018 13:03, Vitaly Davidovich wrote: > >> : >> So I tried calling getBundle(..., Object.class.getModule()), and that >> worked - great. Interestingly, I didn't have to add-opens (or add-export >> for that matter). Is that expected? >> >> It works because the sun.security.util.AuthResources resource bundle for > the default locale is in java.base. You'll get MissingResourceException > once java.base is fully encapsulated (testing with --illegal-access=deny to > try it). Right, thanks Alan. While we're on the subject, when do you expect java.base to be fully encapsulated? > > > -Alan > From Alan.Bateman at oracle.com Tue Feb 13 14:10:28 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Feb 2018 14:10:28 +0000 Subject: IllegalAccessException trying to load a ResourceBundle in 9u181 In-Reply-To: References: <46c81f70-6cb5-bad1-ccd3-c98201324c5b@oracle.com> Message-ID: <62424ae0-7d1e-ffd6-44f5-567a20fba788@oracle.com> On 13/02/2018 14:05, Vitaly Davidovich wrote: > > Right, thanks Alan.? While we're on the subject, when do you expect > java.base to be fully encapsulated? > TBD. From david.lloyd at redhat.com Tue Feb 13 14:37:44 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 13 Feb 2018 08:37:44 -0600 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> Message-ID: On Tue, Feb 13, 2018 at 2:41 AM, Alan Bateman wrote: > On 13/02/2018 06:24, Xueming Shen wrote: >> >> Hi, >> >> Please help review the proposal to add following constructors and methods >> in String >> class to take ByteBuffer as the input and output data buffer. >> >> public String(ByteBuffer bytes, Charset cs); >> public String(ByteBuffer bytes, String csname); > > These constructors looks good (for the parameter names then I assume you > meant "src" rather than "bytes" here). > >> public int getBytes(byte[] dst, int offset, Charset cs); >> public int getBytes(byte[] dst, int offset, String csname); >> public int getBytes(ByteBuffer bytes, Charset cs); >> public int getBytes(ByteBuffer bytes, Charset csn); > > These four methods encode as many characters as possible into the > destination byte[] or buffer but don't give any indication that the > destination didn't have enough space to encode the entire string. I thus > worry they could be a hazard and result in buggy code. If there is > insufficient space then the user of the API doesn't know how many characters > were encoded so it's not easy to substring and call getBytes again to encode > the remaining characters. There is also the issue of how to size the > destination. What would you think about having them fail when there is > insufficient space? If they do fail then there is a side effect that they > will have written to the destination so that would need to be documented > too. The ones that output to a ByteBuffer have more flexibility in that the buffer position can be moved according to the number of bytes written, but the method _could_ return the number of _chars_ actually written. But this is not particularly useful without variants which accept an offset into the string, unless it can be shown that s.substring(coffs).getBytes(xxx) is reasonably efficient. It might be better to shuffle this around a little and instead have a Charset[Encoder].getBytes(int codePoint, byte[] b, int offs, int len)/.getBytes(int codePoint, ByteBuffer buf) kind of thing which returns the number of bytes or e.g. -1 or -count if there isn't enough space in the target. Then it would be less onerous for users to write simple for-each-codepoint loops which encode as far as is reasonable but no farther without too many error-handling gymnastics. -- - DML From robin.westberg at oracle.com Tue Feb 13 14:59:06 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Tue, 13 Feb 2018 15:59:06 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> Message-ID: <7B06DA71-AB33-480B-B176-A3163A78495B@oracle.com> Hi David, > On 12 Feb 2018, at 08:07, David Holmes wrote: > > Hi Robin, > > On 10/02/2018 1:41 AM, Robin Westberg wrote: >> Hi David, >>> On 9 Feb 2018, at 05:22, David Holmes >> wrote: >>> >>> Hi Robin, >>> >>>> Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. >>> >>> Yes and that particular example of launcher behaviour is an archaic throwback to C programming - where end of "main" means end of the program - IMHO. I don't think it is of interest - and certainly not worth jumping through so many hoops to make the info available. >>> >>>> Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. >>> >>> That sounds good to me. :) >>> >>>>> It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. >>>> In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). >>> >>> Yes that is useful. But I find the need to code things the way they are, unfortunate. But given current constraints, so be it. >> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ > > This looks much cleaner/neater to me - thanks. Thanks for reviewing! > Tool Nit: the git-webrev tool you use has an annoying quirk compared to our own webrev tool - it first reports the number of files changed even when it is reporting on an individual file! I'm used to seeing a first number that represents number of lines changes - and the "1" in this case throws me as I like to quickly look at the size of the change for each file when deciding what order to look at them. :( Good point, I?ll make sure the git-webrev tool is updated to more closely match the output of the standard webrev tool. Best regards, Robin > > Thanks, > David > >> Best regards, >> Robin >>> >>> Thanks, >>> David >>> >>>> Best regards, >>>> Robin >>>>> Let's see what others think ... >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>>>>> Hi all, >>>>>> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>>>>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>>>>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>>>>> Best regards, >>>>>> Robin From robin.westberg at oracle.com Tue Feb 13 14:59:12 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Tue, 13 Feb 2018 15:59:12 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> Message-ID: <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> Hi Alan, > On 12 Feb 2018, at 09:02, Alan Bateman wrote: > > > > On 12/02/2018 07:07, David Holmes wrote: >> >>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>> >>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >> >> This looks much cleaner/neater to me - thanks. >> > The updates to Runtime/Shutdown seems okay. Thanks for reviewing! Best regards, Robin > > -Alan From naoto.sato at oracle.com Tue Feb 13 16:51:58 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 13 Feb 2018 08:51:58 -0800 Subject: [11] RFR: 8197829: Optimize CLDRCalendarDataProviderImpl::retrieveInteger In-Reply-To: <9df825f3-680f-61df-644e-74e5a8247451@oracle.com> References: <9df825f3-680f-61df-644e-74e5a8247451@oracle.com> Message-ID: <9584d80a-5a4d-9c6d-ded1-50336eba46b3@oracle.com> Hi Claes, Looks good to me. Please add noreg-perf tag to the issue. Naoto On 2/13/18 2:29 AM, Claes Redestad wrote: > Hi, > > please review this improvement to the retrieveInteger method in > CLDRCalendarDataProviderImpl. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8197829 > Webrev: http://cr.openjdk.java.net/~redestad/8197829/jdk.00/ > > This deals with a tiny startup regression that appeared in 10+36 on some > applications and systems[1] > > Thanks! > > /Claes > > [1] Likely due to https://bugs.openjdk.java.net/browse/JDK-8190918 From claes.redestad at oracle.com Tue Feb 13 16:59:22 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 13 Feb 2018 17:59:22 +0100 Subject: [11] RFR: 8197829: Optimize CLDRCalendarDataProviderImpl::retrieveInteger In-Reply-To: <9584d80a-5a4d-9c6d-ded1-50336eba46b3@oracle.com> References: <9df825f3-680f-61df-644e-74e5a8247451@oracle.com> <9584d80a-5a4d-9c6d-ded1-50336eba46b3@oracle.com> Message-ID: <8a8a4435-d9f7-b5b9-48ad-5571aee72def@oracle.com> On 2018-02-13 17:51, Naoto Sato wrote: > Hi Claes, > > Looks good to me. Thanks, Naoto! > Please add noreg-perf tag to the issue. Done. /Claes From xueming.shen at oracle.com Tue Feb 13 17:00:07 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 13 Feb 2018 09:00:07 -0800 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> Message-ID: <5A831997.4030005@oracle.com> On 2/13/18, 12:41 AM, Alan Bateman wrote: > >> public int getBytes(byte[] dst, int offset, Charset cs); >> public int getBytes(byte[] dst, int offset, String csname); >> public int getBytes(ByteBuffer bytes, Charset cs); >> public int getBytes(ByteBuffer bytes, Charset csn); > These four methods encode as many characters as possible into the > destination byte[] or buffer but don't give any indication that the > destination didn't have enough space to encode the entire string. I > thus worry they could be a hazard and result in buggy code. If there > is insufficient space then the user of the API doesn't know how many > characters were encoded so it's not easy to substring and call > getBytes again to encode the remaining characters. There is also the > issue of how to size the destination. What would you think about > having them fail when there is insufficient space? If they do fail > then there is a side effect that they will have written to the > destination so that would need to be documented too. > > Well It's the old "how to control/manage the en/decoding state" issue we have to deal with for this kind of apis. I think we faced the similar issue when designing bae64 de/ encoder. A possible solution is to return negative value to indicate the buffer/byte[] provided is not big enough and its value is the length needed to de/encode the whole String. So you can pass in a zero-remaining ByteBuffer or byte[0] to ask for the size needed, or we can go even further, take a NULL, like the old C str method to return the size needed, though I don't think we want to go that far :-) That said one of the motivations of these APIs is to easy the use of doing de/encoding between ByteBuffer and String, the extra step to handle the "needed buffer size" appears to be against the original goal. But the api can always say "please provide big-enough buffer, otherwise the method returns the size needed in negative value and the passed in buffer might be partially filled ..." -Sherman From lance.andersen at oracle.com Tue Feb 13 17:54:03 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 13 Feb 2018 12:54:03 -0500 Subject: RFR 8190378: Java EE and CORBA modules removal In-Reply-To: <6ae59d8a-578c-fe1e-b92c-deab72a109af@oracle.com> References: <360C5181-3B9E-4339-B460-04A8385F2461@oracle.com> <6ae59d8a-578c-fe1e-b92c-deab72a109af@oracle.com> Message-ID: <60DB93E9-3DBC-4850-B42D-8177C2D1551E@oracle.com> Hi David, I thought I did make that change. Let me go back and double check and if not, will push. My build worked fine but maybe I missed this in a merge. Best Lance > On Feb 13, 2018, at 8:05 AM, David Holmes wrote: > > Lance, > > In Docs.gmk you seem to have missed this: > > 445 ################################################################################ > 446 # Setup generation of the Java SE API documentation (javadoc + modulegraph) > 447 > 448 # The Java SE module scope is just java.se.ee and its transitive indirect > 449 # exports. > 450 JAVASE_MODULES := java.se.ee > 451 > 452 $(eval $(call SetupApiDocsGeneration, JAVASE_API, \ > 453 MODULES := $(JAVASE_MODULES), \ > 454 SHORT_NAME := $(JAVASE_SHORT_NAME), \ > 455 LONG_NAME := $(JAVASE_LONG_NAME), \ > 456 TARGET_DIR := $(IMAGES_OUTPUTDIR)/javase-docs/api, \ > 457 )) > > which now results in a doc build hanging due to the non-existent java.se.ee module. It leads to execution of: > > grep -e @moduleGraph > > with no module-info.java argument, which causes grep to hang trying to read from stdin. > > At least that is what my analysis indicates. I simply set: > > JAVASE_MODULES := > > to work around it. > > David > ----- > > On 8/02/2018 11:37 PM, Lance Andersen wrote: >>> On Feb 8, 2018, at 3:04 AM, Alan Bateman wrote: >>> >>> On 07/02/2018 16:57, Lance Andersen wrote: >>>> Hi all, >>>> >>>> I think we are at a point where we are ready to start reviewing the changes to remove the Java EE and CORBA modules as JEP 320, JDK-8189188, has been targeted to JDK 11. >>>> The CSR for removing the modules has been approved: https://bugs.openjdk.java.net/browse/JDK-8193757 >>>> >>>> The open webrev can be found at: http://cr.openjdk.java.net/~lancea/8190378/open_changes/ >>>> >>> 800 KLOC deleted, wonderful! >>> >>> The update to technology-summary.html page means its html title no longer matches the contents. We should probably change it to "JCP Technologies in JDK 11" for now. >> I updated the webrev. Thanks for catching that (btw we missed this for JDK 10) >>> >>> The removal of test cases from the tests in tools/launcher/modules removes most of the test coverage for the upgrade module path. We'll need to replace these sub-tests. Can you create an issue to track that? >> I can do that >>> >>> Everything else looks good and it's okay to track residual issues with other JIRA issues. I think the important thing is to get this monster patch into JDK builds soon so that libraries and the eco system can start to adjust. >> Thank you Alan for the review >> Best >> Lance >>> >>> -Alan >> > >> > > >> >Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From ecki at zusammenkunft.net Tue Feb 13 21:55:29 2018 From: ecki at zusammenkunft.net (Bernd) Date: Tue, 13 Feb 2018 22:55:29 +0100 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> Message-ID: Hello, 2018-01-25 17:41 GMT+01:00 Se?n Coffey : > > Classes nearer to those below were touched via JDK-8186080: Transform XML > interfaces > http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 > http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 > > This may be connected with some tools trying to redefine the classes > perhaps. Needs more investigating. Perhaps the XMLSchemaLoader changes are > a factor ? > I have ben able to extract a minimal reproducer. It is not related to XMLUnit, only to powermock. If it instruments com.sun but not javax.xml (and other combinations) then it fails. For details see the readme in this maven project: https://github.com/ecki/reproduce-schemaex I also found a way to make it work with both versions, so its no longer an issue for me, but there is definitely some changes (which might also be triggered in AppServers or OSGi containers with partially reconfigured implementations. Not sure if you want to investigate deeper). Gruss Bernd Here is the stacktrace anyway: >> >> com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: Schema >> factory class >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl does >> not >> extend from SchemaDVFactory. >> at >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. >> getInstance(SchemaDVFactory.java:75) >> at >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. >> getInstance(SchemaDVFactory.java:57) >> at >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >> reset(XMLSchemaLoader.java:1024) >> at >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >> loadGrammar(XMLSchemaLoader.java:556) >> at >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >> loadGrammar(XMLSchemaLoader.java:535) >> at >> com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchema >> Factory.newSchema(XMLSchemaFactory.java:254) >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. >> java:638) >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. >> java:654) >> at >> com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchema >> Validator(BuilderTestHelper.java:57) >> at >> com.seeburger.api.test.helpers.BuilderTestHelper.validateAnd >> Compare(BuilderTestHelper.java:73) >> at >> com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuild >> erTest.java:196) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >> ssorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >> thodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod >> (PowerMockJUnit44RunnerDelegateImpl.java:310) >> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie. >> java:89) >> at >> org.junit.internal.runners.MethodRoadie.runBeforesThenTestTh >> enAfters(MethodRoadie.java:97) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(P >> owerMockJUnit44RunnerDelegateImpl.java:294) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestIn >> Super(PowerMockJUnit47RunnerDelegateImpl.java:127) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(P >> owerMockJUnit47RunnerDelegateImpl.java:82) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThe >> nTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) >> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie >> .java:87) >> at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl.invokeTestMethod(PowerMockJUni >> t44RunnerDelegateImpl.java:207) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:146) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) >> at >> org.junit.internal.runners.ClassRoadie.runUnprotected(ClassR >> oadie.java:34) >> at >> org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) >> at >> org.powermock.modules.junit4.common.internal.impl.JUnit4Test >> SuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) >> at >> org.powermock.modules.junit4.common.internal.impl.AbstractCo >> mmonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) >> at >> org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner.java:59) >> at >> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. >> run(JUnit4TestReference.java:86) >> at >> org.eclipse.jdt.internal.junit.runner.TestExecution.run( >> TestExecution.java:38) >> at >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >> sts(RemoteTestRunner.java:539) >> at >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >> sts(RemoteTestRunner.java:761) >> at >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( >> RemoteTestRunner.java:461) >> at >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( >> RemoteTestRunner.java:207) >> >> on the classpath jaxb-impl-2.2.5.jar but the specific packages are only >> loaded from rt.jar and redefined. I asume the later is done by Powermock. >> >> Line 611: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from >> C:\Program >> Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >> Line 616: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >> Line 617: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >> Line 618: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from >> __JVM_DefineClass__] >> Line 619: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from >> __JVM_DefineClass__] >> Line 620: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from >> __JVM_DefineClass__] >> >> Is that something you are concerned? >> >> Gruss >> Bernd >> > From yumin.qi at gmail.com Tue Feb 13 23:45:40 2018 From: yumin.qi at gmail.com (yumin qi) Date: Tue, 13 Feb 2018 15:45:40 -0800 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: <94168540-0cf6-8f9f-41bd-3f64ba41187f@oracle.com> References: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> <94168540-0cf6-8f9f-41bd-3f64ba41187f@oracle.com> Message-ID: Alan, In fact, if property "user.dir" is cached, the problem will go away without any changes to native or UnixFileSystem.java, since it won't canonicalize the string supplied from user. The output will be getProperty("user.dir") + "/" + file. (The result is not as expected, user can not change the property, means user has to work around the problem to reach their targeted directory to do things like loading classes) Any change to "user.dir" in java program does not have any effect. For Windows: --- a/src/java.base/windows/classes/java/io/WinNTFileSystem.java Fri Feb 02 10:32:59 2018 -0800 +++ b/src/java.base/windows/classes/java/io/WinNTFileSystem.java Tue Feb 13 23:40:21 2018 +0000 @@ -43,12 +43,14 @@ private final char slash; private final char altSlash; private final char semicolon; + private final String userDir; public WinNTFileSystem() { Properties props = GetPropertyAction.privilegedGetProperties(); slash = props.getProperty("file.separator").charAt(0); semicolon = props.getProperty("path.separator").charAt(0); altSlash = (this.slash == '\\') ? '/' : '\\'; + userDir = props.getProperty("user.dir"); } private boolean isSlash(char c) { @@ -347,7 +349,7 @@ private String getUserPath() { /* For both compatibility and security, we must look this up every time */ - return normalize(System.getProperty("user.dir")); + return normalize(userDir); } private String getDrive(String path) { Does it need normalize(userDir) in getUserPath()? I think we need here. I will update the webrev and send for another round of review: the change in native will be reverted, and made changes to UnixFileSystem.java. Thanks Yumin On Mon, Feb 12, 2018 at 3:09 AM, Alan Bateman wrote: > On 09/02/2018 18:01, Alan Bateman wrote: > >> : >> >> I'll study the patch you have but I think we also need to create issues >> to get us to the point where changing this system property in a running VM >> doesn't impact running code. >> > Looking at it again, I think we should change java.io.UnixFileSystem (and > the Windows equivalent) to cache the value of user.dir to avoid difficult > to diagnose issues with bad code changing the value of this property in a > running VM. This should reduce the issue down to cases where user.dir is > changed on the command line (never supported either of course) to a value > that is not "/" but has trailing or duplicate slashes. > > When reduced down then the alternatives are to change the native > canonicalize method as you have done or alternatively do it once at > UnixFileSystem initialization time so that canonicalize does not have to > deal with this case. The former would require changing the description of > the function (it currently reads "The input path is assumed to contain no > duplicate slashes"), the latter avoids any changes to the native > implementation. > > -Alan > > > diff -r 0937e5f799df src/java.base/unix/classes/jav > a/io/UnixFileSystem.java > --- a/src/java.base/unix/classes/java/io/UnixFileSystem.java Sat Feb > 10 07:06:16 2018 -0500 > +++ b/src/java.base/unix/classes/java/io/UnixFileSystem.java Mon Feb > 12 10:49:40 2018 +0000 > @@ -34,12 +34,14 @@ > private final char slash; > private final char colon; > private final String javaHome; > + private final String userDir; > > public UnixFileSystem() { > Properties props = GetPropertyAction.privilegedGetProperties(); > slash = props.getProperty("file.separator").charAt(0); > colon = props.getProperty("path.separator").charAt(0); > javaHome = props.getProperty("java.home"); > + userDir = props.getProperty("user.dir"); > } > > > @@ -128,7 +130,11 @@ > > public String resolve(File f) { > if (isAbsolute(f)) return f.getPath(); > - return resolve(System.getProperty("user.dir"), f.getPath()); > + SecurityManager sm = System.getSecurityManager(); > + if (sm != null) { > + sm.checkPropertyAccess("user.dir"); > + } > + return resolve(userDir, f.getPath()); > } > > // Caches for canonicalization results to improve startup performance. > > From yumin.qi at gmail.com Wed Feb 14 01:23:07 2018 From: yumin.qi at gmail.com (yumin qi) Date: Tue, 13 Feb 2018 17:23:07 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: Hi, I have update the webrev: http://cr.openjdk.java.net/~minqi/8194154/webrev1/ In this version, as suggested by Alan(thanks!), property of "user.dir" is cached and behave like it is 'read only'. The change made to *inux as well as windows. Since property of "user.dir" is cached, any changes via property setting for it has no effect. Thanks Yumin On Wed, Feb 7, 2018 at 10:54 PM, Alan Bateman wrote: > On 07/02/2018 20:10, yumin qi wrote: > >> Hi, >> >> Please review the fix (extra small) for: >> bug 8194154: https://bugs.openjdk.java.net/browse/JDK-8194154 >> webrev: http://cr.openjdk.java.net/~minqi/8194154/ >> >> Summary: canonicalize will check and fold double (or more) slashes, but >> in function collapsible(char*) and splitNames(char*, char**), it failed to >> process strings like "//". This results in the former does not give a >> correct number of substrings and the later does not give a correct array >> of >> substrings. The fix add a check if the character is '/' after a '/'. >> >> The JDK has never supported setting user.dir in this way (it breaks > several things). There is a thread discussing this on core-libs-dev so best > to bring it to that mailing list to discuss again. > > -Alan > From martinrb at google.com Wed Feb 14 01:30:10 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 13 Feb 2018 17:30:10 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer Message-ID: Unlike most fixes to data races, this one should benefit performance. http://cr.openjdk.java.net/~martin/webrevs/jdk/Finalizer-data-race/ https://bugs.openjdk.java.net/browse/JDK-8197812 From martinrb at google.com Wed Feb 14 01:33:39 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 13 Feb 2018 17:33:39 -0800 Subject: RFR: 8197893: Mistaken type check in CheckedEntrySet.toArray Message-ID: http://cr.openjdk.java.net/~martin/webrevs/jdk/CheckedEntrySet-toArray/ https://bugs.openjdk.java.net/browse/JDK-8197893 From brian.burkhalter at oracle.com Wed Feb 14 01:52:54 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 13 Feb 2018 17:52:54 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: <62023DC7-4804-4849-805D-51A65AA23CF7@oracle.com> Hi, As for the test alone you could add the @requires annotation [1] @requires (os.family == "linux") | (os.family == "mac") | (os.family == "solaris") | (os.family == "aix") instead of programmatically exiting if the platform is not Windows. (I am assuming that this should be run on all Unix variants, not just Linux.). Thanks, Brian [1] http://openjdk.java.net/jtreg/tag-spec.html#requires_names On Feb 13, 2018, at 5:23 PM, yumin qi wrote: > I have update the webrev: > http://cr.openjdk.java.net/~minqi/8194154/webrev1/ From yumin.qi at gmail.com Wed Feb 14 04:06:59 2018 From: yumin.qi at gmail.com (yumin qi) Date: Tue, 13 Feb 2018 20:06:59 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: <62023DC7-4804-4849-805D-51A65AA23CF7@oracle.com> References: <62023DC7-4804-4849-805D-51A65AA23CF7@oracle.com> Message-ID: HI, Brian Thanks, updated on same link. I don't have platforms other than linux so could not test on those platforms. BTW, tried jtreg, it seems the version has not modified to run with current jdk repo. Thanks Yumin On Tue, Feb 13, 2018 at 5:52 PM, Brian Burkhalter < brian.burkhalter at oracle.com> wrote: > Hi, > > As for the test alone you could add the @requires annotation [1] > > @requires (os.family == "linux") | (os.family == "mac") | (os.family == > "solaris") | (os.family == "aix") > > instead of programmatically exiting if the platform is not Windows. (I am > assuming that this should be run on all Unix variants, not just Linux.). > > Thanks, > > Brian > > [1] http://openjdk.java.net/jtreg/tag-spec.html#requires_names > > On Feb 13, 2018, at 5:23 PM, yumin qi wrote: > > I have update the webrev: > http://cr.openjdk.java.net/~minqi/8194154/webrev1/ > > > From peter.levart at gmail.com Wed Feb 14 06:58:07 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 14 Feb 2018 07:58:07 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: References: Message-ID: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> Hi Martin, On 02/14/18 02:30, Martin Buchholz wrote: > Unlike most fixes to data races, this one should benefit performance. > > http://cr.openjdk.java.net/~martin/webrevs/jdk/Finalizer-data-race/ > https://bugs.openjdk.java.net/browse/JDK-8197812 Nice catch (for java-tsan tool)! Although I think this race is benign. The code that synchronizes on 'this' is the following: ? 46???? private boolean hasBeenFinalized() { ? 47???????? return (next == this); ? 48???? } And the only place where 'next' is set to 'this' is in remove() and remove is called from synchronized (this) block too. So hasBeenFinalized() may see any or no writes of 'next' performed out of synchronized (this), but they are never writes of value 'this', so the outcome is always 'false' in this case. The only write to 'next' with value of 'this' happens while synchronized (this) and synchronized (lock) at the same time so this write synchronizes with the read of 'next' in hasBeenFinalized() as well as with all other writes of 'next'. It may be that the intent was to refrain from using the shared 'lock' lock for the 2nd and subsequent calls to runFinalizer() and only use the more fine-grained 'this' lock in this case? If someone was able to call runFinalizer() on the same instance in a loop he could prevent or slow-down normal processing of other Finalizer(s) if the shared 'lock' was always employed. What do you think? Regards, Peter From peter.levart at gmail.com Wed Feb 14 08:19:55 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 14 Feb 2018 09:19:55 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> Message-ID: <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> Hi Martin, On 02/14/2018 07:58 AM, Peter Levart wrote: > It may be that the intent was to refrain from using the shared 'lock' > lock for the 2nd and subsequent calls to runFinalizer() and only use > the more fine-grained 'this' lock in this case? > > If someone was able to call runFinalizer() on the same instance in a > loop he could prevent or slow-down normal processing of other > Finalizer(s) if the shared 'lock' was always employed. What do you think? I checked all uses of runFinalizer() and I don't think it can be called more than twice for the same Finalizer instance (for example if there is a race between runAllFinalizers() and processing of Finalizers taken from the ReferenceQueue). So your patch is a nice simplification. Regards, Peter From peter.levart at gmail.com Wed Feb 14 08:47:38 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 14 Feb 2018 09:47:38 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> Message-ID: <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> Hi Again, While studying the code of Finalizer I found a strange discrepancy between handling of 'unfinalized' field during processing of Finalizer(s) taken from ReferenceQueue and taken from the 'unfinalized' pointer itself (as in runAllFinalizers()). The remove() method is as follows: ??? private void remove() { ??????? synchronized (lock) { ??????????? if (unfinalized == this) { ??????????????? if (this.next != null) { ??????????????????? unfinalized = this.next; ??????????????? } else { ??????????????????? unfinalized = this.prev; ??????????????? } ??????????? } ??????????? if (this.next != null) { ??????????????? this.next.prev = this.prev; ??????????? } ??????????? if (this.prev != null) { ??????????????? this.prev.next = this.next; ??????????? } ??????????? this.next = this;?? /* Indicates that this has been finalized */ ??????????? this.prev = this; ??????? } ??? } When Finalizer(s) are taken from ReferenceQueue, they are processed in arbitrary order, so once in a while it can happen that the Finalizer at the head of the list (pointed to by 'unfinalized') is processed this way. The body of the 1st if statement in above method is executed in such situation: ??????????? if (unfinalized == this) { ??????????????? if (this.next != null) { ??????????????????? unfinalized = this.next; ??????????????? } else { ??????????????????? unfinalized = this.prev; ??????????????? } ??????????? } But I can't figure out a situation when this.next would be null while this.prev would be non-null. In other words, when 'unfinalized' would not point to the head of the list. For comparison, when Finalizer(s) are processed from the 'unfinalized' pointer directly (as in runAllFinalizers()), the following is executed: ??? ??? ??? ??? for (;;) { ??????????????????? Finalizer f; ??????????????????? synchronized (lock) { ??????????????????????? f = unfinalized; ??????????????????????? if (f == null) break; ??????????????????????? unfinalized = f.next; ??????????????????? } ??????????????????? f.runFinalizer(jla); ??????????????? } ... so it is assumed that the 'unfinalized' always points to the head. If this was not true, not all Finalizer(s) would be processed. So what I'm asking is whether this simplified if statement in remove() would be equivalent: ??????????? if (unfinalized == this) { ??????????????? unfinalized = this.next; ??????????? } Or am I missing some situation? Regards, Peter On 02/14/2018 09:19 AM, Peter Levart wrote: > Hi Martin, > > On 02/14/2018 07:58 AM, Peter Levart wrote: >> It may be that the intent was to refrain from using the shared 'lock' >> lock for the 2nd and subsequent calls to runFinalizer() and only use >> the more fine-grained 'this' lock in this case? >> >> If someone was able to call runFinalizer() on the same instance in a >> loop he could prevent or slow-down normal processing of other >> Finalizer(s) if the shared 'lock' was always employed. What do you >> think? > > I checked all uses of runFinalizer() and I don't think it can be > called more than twice for the same Finalizer instance (for example if > there is a race between runAllFinalizers() and processing of > Finalizers taken from the ReferenceQueue). So your patch is a nice > simplification. > > Regards, Peter > From peter.levart at gmail.com Wed Feb 14 09:24:35 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 14 Feb 2018 10:24:35 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> Message-ID: <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> Hi, I may have found a situation described here... On 02/14/2018 09:47 AM, Peter Levart wrote: > > When Finalizer(s) are taken from ReferenceQueue, they are processed in > arbitrary order, so once in a while it can happen that the Finalizer > at the head of the list (pointed to by 'unfinalized') is processed > this way. The body of the 1st if statement in above method is executed > in such situation: > > ??????????? if (unfinalized == this) { > ??????????????? if (this.next != null) { > ??????????????????? unfinalized = this.next; > ??????????????? } else { > ??????????????????? unfinalized = this.prev; > ??????????????? } > ??????????? } > > But I can't figure out a situation when this.next would be null while > this.prev would be non-null. In other words, when 'unfinalized' would > not point to the head of the list. This is the situation: unfinalized ??? | ??? v Finalizer[1].next --> Finalizer[2].next --> null Finalizer[2].prev --> Finalizer[1].prev --> null runAllFinalizers() is executed and the 1st pass of the for loop takes the Finalizer[1] from the 'unfinalized' pointer and moves the pointer to Finalizer[2]: unfinalized ---------------- ?????????????????????????? | ?????????????????????????? v Finalizer[1].next --> Finalizer[2].next --> null Finalizer[2].prev --> Finalizer[1].prev --> null ...now finalizer thread kicks-in and takes the Finalizer[2] from the ReferenceQueue and calls remove(). The 1st if statement in remove() would evaluate the condition to true, as 'unfinalized' now points to Finalizer[2], so the body of 1st if would make unfinalized point back to Finalizer[1]: unfinalized ??? | ??? v Finalizer[1].next --> Finalizer[2].next --> null Finalizer[2].prev --> Finalizer[1].prev --> null ...and the remaining remove() code would unlink the Finalizer[2] from the list, resulting in this state of the list: unfinalized ??? | ??? v Finalizer[1].next --> null Finalizer[1].prev --> null This might be seen as a "correct" state, because Finalizer[1] has not been removed from the list yet although it is in the process of being removed because the next thing runAllFinalizers() loop would do is it would call runFinalizer() on it. So the simplified handling in remove(): ??? if (unfinalized == this) { ??? ??? unfinalized = this.next; ??? } ...would result in correct behavior too. Even more so because if the 2nd call to runAllFinalizers() was being executed concurrently (I don't know if this is possible though), the Finalizer[1] from above situation would be taken by the concurrent call again and its runFinalizer() executed. runFinalizer() is idempotent so no harm done here, but... Regards, Peter From peter.levart at gmail.com Wed Feb 14 09:39:09 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 14 Feb 2018 10:39:09 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> Message-ID: <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> I could even claim that simplifying the if statement in remove() to: ??? if (unfinalized == this) { ??? ??? unfinalized = this.next; ??? } makes checking for hasBeenFinalized() in runFinalizer() redundant as it would not be possible for runFinalizer() to be called more than once for each Finalizer instance because: - ReferenceQueue never returns the same Reference instance twice or more times. - 'unfinalized' will never point back to the same Finalizer instance for the 2nd time, because it always "travels" in the forward direction (unfinalized = unfinalized.next). Regards, Peter (I rest now). On 02/14/2018 10:24 AM, Peter Levart wrote: > Hi, > > I may have found a situation described here... > > On 02/14/2018 09:47 AM, Peter Levart wrote: >> >> When Finalizer(s) are taken from ReferenceQueue, they are processed >> in arbitrary order, so once in a while it can happen that the >> Finalizer at the head of the list (pointed to by 'unfinalized') is >> processed this way. The body of the 1st if statement in above method >> is executed in such situation: >> >> ??????????? if (unfinalized == this) { >> ??????????????? if (this.next != null) { >> ??????????????????? unfinalized = this.next; >> ??????????????? } else { >> ??????????????????? unfinalized = this.prev; >> ??????????????? } >> ??????????? } >> >> But I can't figure out a situation when this.next would be null while >> this.prev would be non-null. In other words, when 'unfinalized' would >> not point to the head of the list. > > This is the situation: > > unfinalized > ??? | > ??? v > Finalizer[1].next --> Finalizer[2].next --> null > Finalizer[2].prev --> Finalizer[1].prev --> null > > runAllFinalizers() is executed and the 1st pass of the for loop takes > the Finalizer[1] from the 'unfinalized' pointer and moves the pointer > to Finalizer[2]: > > unfinalized ---------------- > ?????????????????????????? | > ?????????????????????????? v > Finalizer[1].next --> Finalizer[2].next --> null > Finalizer[2].prev --> Finalizer[1].prev --> null > > ...now finalizer thread kicks-in and takes the Finalizer[2] from the > ReferenceQueue and calls remove(). The 1st if statement in remove() > would evaluate the condition to true, as 'unfinalized' now points to > Finalizer[2], so the body of 1st if would make unfinalized point back > to Finalizer[1]: > > unfinalized > ??? | > ??? v > Finalizer[1].next --> Finalizer[2].next --> null > Finalizer[2].prev --> Finalizer[1].prev --> null > > ...and the remaining remove() code would unlink the Finalizer[2] from > the list, resulting in this state of the list: > > unfinalized > ??? | > ??? v > Finalizer[1].next --> null > Finalizer[1].prev --> null > > This might be seen as a "correct" state, because Finalizer[1] has not > been removed from the list yet although it is in the process of being > removed because the next thing runAllFinalizers() loop would do is it > would call runFinalizer() on it. So the simplified handling in remove(): > > ??? if (unfinalized == this) { > ??? ??? unfinalized = this.next; > ??? } > > ...would result in correct behavior too. Even more so because if the > 2nd call to runAllFinalizers() was being executed concurrently (I > don't know if this is possible though), the Finalizer[1] from above > situation would be taken by the concurrent call again and its > runFinalizer() executed. runFinalizer() is idempotent so no harm done > here, but... > > Regards, Peter > From peter.levart at gmail.com Wed Feb 14 09:58:57 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 14 Feb 2018 10:58:57 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> Message-ID: <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> I take back this claim. Of course the the following race is possible: - Thread1: calls runAllFinalizers and takes a Finalizer from 'unprocessed' list. - Thread2: takee the same Finalizer instance from ReferenceQueue and calls runFinalizer() - Thread1: calls runFinalizer() with the same instance for the 2nd time now. Regards, Peter On 02/14/2018 10:39 AM, Peter Levart wrote: > I could even claim that simplifying the if statement in remove() to: > > ??? if (unfinalized == this) { > ??? ??? unfinalized = this.next; > ??? } > > makes checking for hasBeenFinalized() in runFinalizer() redundant as > it would not be possible for runFinalizer() to be called more than > once for each Finalizer instance because: > - ReferenceQueue never returns the same Reference instance twice or > more times. > - 'unfinalized' will never point back to the same Finalizer instance > for the 2nd time, because it always "travels" in the forward direction > (unfinalized = unfinalized.next). > > Regards, Peter > > (I rest now). > > On 02/14/2018 10:24 AM, Peter Levart wrote: >> Hi, >> >> I may have found a situation described here... >> >> On 02/14/2018 09:47 AM, Peter Levart wrote: >>> >>> When Finalizer(s) are taken from ReferenceQueue, they are processed >>> in arbitrary order, so once in a while it can happen that the >>> Finalizer at the head of the list (pointed to by 'unfinalized') is >>> processed this way. The body of the 1st if statement in above method >>> is executed in such situation: >>> >>> ??????????? if (unfinalized == this) { >>> ??????????????? if (this.next != null) { >>> ??????????????????? unfinalized = this.next; >>> ??????????????? } else { >>> ??????????????????? unfinalized = this.prev; >>> ??????????????? } >>> ??????????? } >>> >>> But I can't figure out a situation when this.next would be null >>> while this.prev would be non-null. In other words, when >>> 'unfinalized' would not point to the head of the list. >> >> This is the situation: >> >> unfinalized >> ??? | >> ??? v >> Finalizer[1].next --> Finalizer[2].next --> null >> Finalizer[2].prev --> Finalizer[1].prev --> null >> >> runAllFinalizers() is executed and the 1st pass of the for loop takes >> the Finalizer[1] from the 'unfinalized' pointer and moves the pointer >> to Finalizer[2]: >> >> unfinalized ---------------- >> ?????????????????????????? | >> ?????????????????????????? v >> Finalizer[1].next --> Finalizer[2].next --> null >> Finalizer[2].prev --> Finalizer[1].prev --> null >> >> ...now finalizer thread kicks-in and takes the Finalizer[2] from the >> ReferenceQueue and calls remove(). The 1st if statement in remove() >> would evaluate the condition to true, as 'unfinalized' now points to >> Finalizer[2], so the body of 1st if would make unfinalized point back >> to Finalizer[1]: >> >> unfinalized >> ??? | >> ??? v >> Finalizer[1].next --> Finalizer[2].next --> null >> Finalizer[2].prev --> Finalizer[1].prev --> null >> >> ...and the remaining remove() code would unlink the Finalizer[2] from >> the list, resulting in this state of the list: >> >> unfinalized >> ??? | >> ??? v >> Finalizer[1].next --> null >> Finalizer[1].prev --> null >> >> This might be seen as a "correct" state, because Finalizer[1] has not >> been removed from the list yet although it is in the process of being >> removed because the next thing runAllFinalizers() loop would do is it >> would call runFinalizer() on it. So the simplified handling in remove(): >> >> ??? if (unfinalized == this) { >> ??? ??? unfinalized = this.next; >> ??? } >> >> ...would result in correct behavior too. Even more so because if the >> 2nd call to runAllFinalizers() was being executed concurrently (I >> don't know if this is possible though), the Finalizer[1] from above >> situation would be taken by the concurrent call again and its >> runFinalizer() executed. runFinalizer() is idempotent so no harm done >> here, but... >> >> Regards, Peter >> > From volker.simonis at gmail.com Wed Feb 14 10:20:22 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 14 Feb 2018 11:20:22 +0100 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string Message-ID: Hi, can I please get a review for the following tiny fix: http://cr.openjdk.java.net/~simonis/webrevs/2018/8197927/ https://bugs.openjdk.java.net/browse/JDK-8197927 The new Java 10 specification makes the 'java.vendor.version' property mandatory [1] but the current implementations doesn't allow to set it to an empty string. Currently, if we don't configure the build with --with-vendor-version=XXX the 'java.vendor.version' property won't be set at all, which violates the spec. Setting it to an empty string will be ignored and has the same behavior like not setting it at all. This also makes default OpenJDK builds (which haven't been configured with --with-vendor-version non-compliant). The fix is trivial: unconditionally set the 'java.vendor.version' property to the value of VENDOR_VERSION_STRING in VersionProps.java.template, even if VENDOR_VERSION_STRING is the empty string (which is the default if --with-vendor-version wasn't given at config time). Thank you and best regards, Volker [1] http://cr.openjdk.java.net/~iris/se/10/pfd/java-se-10-pfd-spec/apidiffs/java/lang/System-report.html#method:getProperties() From peter.levart at gmail.com Wed Feb 14 10:49:00 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 14 Feb 2018 11:49:00 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> Message-ID: <36d9bc2c-48b7-7b6f-5650-e9df31fa8be6@gmail.com> Hi Martin, On 02/14/2018 10:58 AM, Peter Levart wrote: > > I take back this claim. Of course the the following race is possible: > > - Thread1: calls runAllFinalizers and takes a Finalizer from > 'unprocessed' list. > - Thread2: takee the same Finalizer instance from ReferenceQueue and > calls runFinalizer() > - Thread1: calls runFinalizer() with the same instance for the 2nd > time now. ... but this could be "fixed" if the taking of next Finalizer from 'unfinalized' list and removing it from the same list was a single atomic operation. What do you say of the following further simplification: http://cr.openjdk.java.net/~plevart/jdk-dev/8197812_Data_race_in_Finalizer/webrev.01/ Regards, Peter > > Regards, Peter > > On 02/14/2018 10:39 AM, Peter Levart wrote: >> I could even claim that simplifying the if statement in remove() to: >> >> ??? if (unfinalized == this) { >> ??? ??? unfinalized = this.next; >> ??? } >> >> makes checking for hasBeenFinalized() in runFinalizer() redundant as >> it would not be possible for runFinalizer() to be called more than >> once for each Finalizer instance because: >> - ReferenceQueue never returns the same Reference instance twice or >> more times. >> - 'unfinalized' will never point back to the same Finalizer instance >> for the 2nd time, because it always "travels" in the forward >> direction (unfinalized = unfinalized.next). >> >> Regards, Peter >> >> (I rest now). From thomas.stuefe at gmail.com Wed Feb 14 10:50:23 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 14 Feb 2018 11:50:23 +0100 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string In-Reply-To: References: Message-ID: Fix is fine, trivial and I do not think there is any risk attached to it. I am not in any position to comment whether this is P1. Copyright year needs adjusting. Kind Regards, Thomas On Wed, Feb 14, 2018 at 11:20 AM, Volker Simonis wrote: > Hi, > > can I please get a review for the following tiny fix: > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8197927/ > https://bugs.openjdk.java.net/browse/JDK-8197927 > > The new Java 10 specification makes the 'java.vendor.version' property > mandatory [1] but the current implementations doesn't allow to set it > to an empty string. > > Currently, if we don't configure the build with > --with-vendor-version=XXX the 'java.vendor.version' property won't be > set at all, which violates the spec. Setting it to an empty string > will be ignored and has the same behavior like not setting it at all. > This also makes default OpenJDK builds (which haven't been configured > with --with-vendor-version non-compliant). > > The fix is trivial: unconditionally set the 'java.vendor.version' > property to the value of VENDOR_VERSION_STRING in > VersionProps.java.template, even if VENDOR_VERSION_STRING is the empty > string (which is the default if --with-vendor-version wasn't given at > config time). > > Thank you and best regards, > Volker > > > [1] http://cr.openjdk.java.net/~iris/se/10/pfd/java-se-10-pfd- > spec/apidiffs/java/lang/System-report.html#method:getProperties() > From peter.levart at gmail.com Wed Feb 14 11:01:43 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 14 Feb 2018 12:01:43 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <36d9bc2c-48b7-7b6f-5650-e9df31fa8be6@gmail.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <36d9bc2c-48b7-7b6f-5650-e9df31fa8be6@gmail.com> Message-ID: <901bfa19-cdbb-9b82-a4ae-149e452fbb91@gmail.com> On 02/14/2018 11:49 AM, Peter Levart wrote: > Hi Martin, > > On 02/14/2018 10:58 AM, Peter Levart wrote: >> >> I take back this claim. Of course the the following race is possible: >> >> - Thread1: calls runAllFinalizers and takes a Finalizer from >> 'unprocessed' list. >> - Thread2: takee the same Finalizer instance from ReferenceQueue and >> calls runFinalizer() >> - Thread1: calls runFinalizer() with the same instance for the 2nd >> time now. > > ... but this could be "fixed" if the taking of next Finalizer from > 'unfinalized' list and removing it from the same list was a single > atomic operation. What do you say of the following further > simplification: > > http://cr.openjdk.java.net/~plevart/jdk-dev/8197812_Data_race_in_Finalizer/webrev.01/ > Even that has a flaw. Running the next unfinalized Finalizer from runAllFinalizers() does not prevent the same Finalizer to be returned from the ReferenceQueue. So the check must remain in place. Sorry for this long monologue. I truly rest now. Regards, Peter From volker.simonis at gmail.com Wed Feb 14 11:03:04 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 14 Feb 2018 12:03:04 +0100 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string In-Reply-To: References: Message-ID: Thanks Thomas. The sole reason for making it P1 is that it is currently not possible to build a Java 10 conforming version of OpenJDK with an empty vendor version. The specification doesn't mandate a non-empty vendor version and I don't think OpenJDK should mandate on either. Regards, Volker On Wed, Feb 14, 2018 at 11:50 AM, Thomas St?fe wrote: > Fix is fine, trivial and I do not think there is any risk attached to it. I > am not in any position to comment whether this is P1. Copyright year needs > adjusting. > > Kind Regards, Thomas > > On Wed, Feb 14, 2018 at 11:20 AM, Volker Simonis > wrote: >> >> Hi, >> >> can I please get a review for the following tiny fix: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2018/8197927/ >> https://bugs.openjdk.java.net/browse/JDK-8197927 >> >> The new Java 10 specification makes the 'java.vendor.version' property >> mandatory [1] but the current implementations doesn't allow to set it >> to an empty string. >> >> Currently, if we don't configure the build with >> --with-vendor-version=XXX the 'java.vendor.version' property won't be >> set at all, which violates the spec. Setting it to an empty string >> will be ignored and has the same behavior like not setting it at all. >> This also makes default OpenJDK builds (which haven't been configured >> with --with-vendor-version non-compliant). >> >> The fix is trivial: unconditionally set the 'java.vendor.version' >> property to the value of VENDOR_VERSION_STRING in >> VersionProps.java.template, even if VENDOR_VERSION_STRING is the empty >> string (which is the default if --with-vendor-version wasn't given at >> config time). >> >> Thank you and best regards, >> Volker >> >> >> [1] >> http://cr.openjdk.java.net/~iris/se/10/pfd/java-se-10-pfd-spec/apidiffs/java/lang/System-report.html#method:getProperties() > > From Alan.Bateman at oracle.com Wed Feb 14 12:42:41 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Feb 2018 12:42:41 +0000 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: On 14/02/2018 01:23, yumin qi wrote: > Hi, > > ? I have update the webrev: > http://cr.openjdk.java.net/~minqi/8194154/webrev1/ > > > ? In this version, as suggested by Alan(thanks!), property of > "user.dir" is cached and behave like it is 'read only'. The change > made to *inux as well as windows. Since property of "user.dir" is > cached, any changes via property setting for it has no effect. > This looks much better but you've removed a permission check from the Windows getUserPath implementation. That code the same permission check as the resolve method in the Unix implementation. I think the test needs works. The simplest would be to call getCanonicalFile before changing the system property, then call it after and check that you an equal result. Also no need to hack the Properties object, you can use System.setProperty instead. -Alan From david.holmes at oracle.com Wed Feb 14 12:43:38 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Feb 2018 22:43:38 +1000 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: Message-ID: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Adding in core-libs-dev as there's nothing related to hotspot directly here. David On 14/02/2018 9:32 PM, Adam Farley8 wrote: > Hi All, > > Currently, diagnostic core files generated from OpenJDK seem to lump all > of the > native memory usages together, making it near-impossible for someone to > figure > out *what* is using all that memory in the event of a memory leak. > > The OpenJ9 VM has a feature which allows it to track the allocation of > native > memory for Direct Byte Buffers (DBBs), and to supply that information into > the > cores when they are generated. This makes it a *lot* easier to find out > what is using > all that native memory, making memory leak resolution less like some dark > art, and > more like logical debugging. > > To use this feature, there is a native method referenced in Unsafe.java. > To open > up this feature so that any VM can make use of it, the java code below > sets the > stage for it. This change starts letting people call DBB-specific methods > when > allocating native memory, and getting into the habit of using it. > > Thoughts? > > Best Regards > > Adam Farley > > P.S. Code: > > diff --git > a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > @@ -85,7 +85,7 @@ > // Paranoia > return; > } > - UNSAFE.freeMemory(address); > + UNSAFE.freeDBBMemory(address); > address = 0; > Bits.unreserveMemory(size, capacity); > } > @@ -118,7 +118,7 @@ > > long base = 0; > try { > - base = UNSAFE.allocateMemory(size); > + base = UNSAFE.allocateDBBMemory(size); > } catch (OutOfMemoryError x) { > Bits.unreserveMemory(size, cap); > throw x; > diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > @@ -632,6 +632,26 @@ > } > > /** > + * Allocates a new block of native memory for DirectByteBuffers, of > the > + * given size in bytes. The contents of the memory are > uninitialized; > + * they will generally be garbage. The resulting native pointer will > + * never be zero, and will be aligned for all value types. Dispose > of > + * this memory by calling {@link #freeDBBMemory} or resize it with > + * {@link #reallocateDBBMemory}. > + * > + * @throws RuntimeException if the size is negative or too large > + * for the native size_t type > + * > + * @throws OutOfMemoryError if the allocation is refused by the > system > + * > + * @see #getByte(long) > + * @see #putByte(long, byte) > + */ > + public long allocateDBBMemory(long bytes) { > + return allocateMemory(bytes); > + } > + > + /** > * Resizes a new block of native memory, to the given size in bytes. > The > * contents of the new block past the size of the old block are > * uninitialized; they will generally be garbage. The resulting > native > @@ -687,6 +707,27 @@ > } > > /** > + * Resizes a new block of native memory for DirectByteBuffers, to the > + * given size in bytes. The contents of the new block past the size > of > + * the old block are uninitialized; they will generally be garbage. > The > + * resulting native pointer will be zero if and only if the requested > size > + * is zero. The resulting native pointer will be aligned for all > value > + * types. Dispose of this memory by calling {@link #freeDBBMemory}, > or > + * resize it with {@link #reallocateDBBMemory}. The address passed > to > + * this method may be null, in which case an allocation will be > performed. > + * > + * @throws RuntimeException if the size is negative or too large > + * for the native size_t type > + * > + * @throws OutOfMemoryError if the allocation is refused by the > system > + * > + * @see #allocateDBBMemory > + */ > + public long reallocateDBBMemory(long address, long bytes) { > + return reallocateMemory(address, bytes); > + } > + > + /** > * Sets all bytes in a given block of memory to a fixed value > * (usually zero). > * > @@ -918,6 +959,17 @@ > checkPointer(null, address); > } > > + /** > + * Disposes of a block of native memory, as obtained from {@link > + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address > passed > + * to this method may be null, in which case no action is taken. > + * > + * @see #allocateDBBMemory > + */ > + public void freeDBBMemory(long address) { > + freeMemory(address); > + } > + > /// random queries > > /** > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > From david.holmes at oracle.com Wed Feb 14 12:53:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Feb 2018 22:53:42 +1000 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> References: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Message-ID: On 14/02/2018 10:43 PM, David Holmes wrote: > Adding in core-libs-dev as there's nothing related to hotspot directly > here. Correction, this is of course leading to a proposed change in hotspot to implement the new Unsafe methods and perform the native memory tracking. Of course we already have NMT so the obvious question is how this will fit in with NMT? David > David > > On 14/02/2018 9:32 PM, Adam Farley8 wrote: >> Hi All, >> >> Currently, diagnostic core files generated from OpenJDK seem to lump all >> of the >> native memory usages together, making it near-impossible for someone to >> figure >> out *what* is using all that memory in the event of a memory leak. >> >> The OpenJ9 VM has a feature which allows it to track the allocation of >> native >> memory for Direct Byte Buffers (DBBs), and to supply that information >> into >> the >> cores when they are generated. This makes it a *lot* easier to find out >> what is using >> all that native memory, making memory leak resolution less like some dark >> art, and >> more like logical debugging. >> >> To use this feature, there is a native method referenced in Unsafe.java. >> To open >> up this feature so that any VM can make use of it, the java code below >> sets the >> stage for it. This change starts letting people call DBB-specific methods >> when >> allocating native memory, and getting into the habit of using it. >> >> Thoughts? >> >> Best Regards >> >> Adam Farley >> >> P.S. Code: >> >> diff --git >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> @@ -85,7 +85,7 @@ >> ????????????????? // Paranoia >> ????????????????? return; >> ????????????? } >> -??????????? UNSAFE.freeMemory(address); >> +??????????? UNSAFE.freeDBBMemory(address); >> ????????????? address = 0; >> ????????????? Bits.unreserveMemory(size, capacity); >> ????????? } >> @@ -118,7 +118,7 @@ >> ????????? long base = 0; >> ????????? try { >> -??????????? base = UNSAFE.allocateMemory(size); >> +??????????? base = UNSAFE.allocateDBBMemory(size); >> ????????? } catch (OutOfMemoryError x) { >> ????????????? Bits.unreserveMemory(size, cap); >> ????????????? throw x; >> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> @@ -632,6 +632,26 @@ >> ????? } >> ????? /** >> +???? * Allocates a new block of native memory for DirectByteBuffers, of >> the >> +???? * given size in bytes.? The contents of the memory are >> uninitialized; >> +???? * they will generally be garbage.? The resulting native pointer >> will >> +???? * never be zero, and will be aligned for all value types.? Dispose >> of >> +???? * this memory by calling {@link #freeDBBMemory} or resize it with >> +???? * {@link #reallocateDBBMemory}. >> +???? * >> +???? * @throws RuntimeException if the size is negative or too large >> +???? *????????????????????????? for the native size_t type >> +???? * >> +???? * @throws OutOfMemoryError if the allocation is refused by the >> system >> +???? * >> +???? * @see #getByte(long) >> +???? * @see #putByte(long, byte) >> +???? */ >> +??? public long allocateDBBMemory(long bytes) { >> +??????? return allocateMemory(bytes); >> +??? } >> + >> +??? /** >> ?????? * Resizes a new block of native memory, to the given size in >> bytes. >> The >> ?????? * contents of the new block past the size of the old block are >> ?????? * uninitialized; they will generally be garbage.? The resulting >> native >> @@ -687,6 +707,27 @@ >> ????? } >> ????? /** >> +???? * Resizes a new block of native memory for DirectByteBuffers, to >> the >> +???? * given size in bytes.? The contents of the new block past the size >> of >> +???? * the old block are uninitialized; they will generally be garbage. >> The >> +???? * resulting native pointer will be zero if and only if the >> requested >> size >> +???? * is zero.? The resulting native pointer will be aligned for all >> value >> +???? * types.? Dispose of this memory by calling {@link #freeDBBMemory}, >> or >> +???? * resize it with {@link #reallocateDBBMemory}.? The address passed >> to >> +???? * this method may be null, in which case an allocation will be >> performed. >> +???? * >> +???? * @throws RuntimeException if the size is negative or too large >> +???? *????????????????????????? for the native size_t type >> +???? * >> +???? * @throws OutOfMemoryError if the allocation is refused by the >> system >> +???? * >> +???? * @see #allocateDBBMemory >> +???? */ >> +??? public long reallocateDBBMemory(long address, long bytes) { >> +??????? return reallocateMemory(address, bytes); >> +??? } >> + >> +??? /** >> ?????? * Sets all bytes in a given block of memory to a fixed value >> ?????? * (usually zero). >> ?????? * >> @@ -918,6 +959,17 @@ >> ????????? checkPointer(null, address); >> ????? } >> +??? /** >> +???? * Disposes of a block of native memory, as obtained from {@link >> +???? * #allocateDBBMemory} or {@link #reallocateDBBMemory}.? The address >> passed >> +???? * to this method may be null, in which case no action is taken. >> +???? * >> +???? * @see #allocateDBBMemory >> +???? */ >> +??? public void freeDBBMemory(long address) { >> +??????? freeMemory(address); >> +??? } >> + >> ????? /// random queries >> ????? /** >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 >> 3AU >> From adam.farley at uk.ibm.com Wed Feb 14 13:04:44 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 14 Feb 2018 13:04:44 +0000 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> References: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Message-ID: > Adding in core-libs-dev as there's nothing related to hotspot directly here. > > David Thought it was best to pass this through hotspot lists first, as full completion of the native side of things will probably require hotspot changes. You're quite right though, I should have cc'd hotspot *and* core libs. - Adam > > On 14/02/2018 9:32 PM, Adam Farley8 wrote: >> Hi All, >> >> Currently, diagnostic core files generated from OpenJDK seem to lump all >> of the >> native memory usages together, making it near-impossible for someone to >> figure >> out *what* is using all that memory in the event of a memory leak. >> >> The OpenJ9 VM has a feature which allows it to track the allocation of >> native >> memory for Direct Byte Buffers (DBBs), and to supply that information into >> the >> cores when they are generated. This makes it a *lot* easier to find out >> what is using >> all that native memory, making memory leak resolution less like some dark >> art, and >> more like logical debugging. >> >> To use this feature, there is a native method referenced in Unsafe.java. >> To open >> up this feature so that any VM can make use of it, the java code below >> sets the >> stage for it. This change starts letting people call DBB-specific methods >> when >> allocating native memory, and getting into the habit of using it. >> >> Thoughts? >> >> Best Regards >> >> Adam Farley >> >> P.S. Code: >> >> diff --git >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> @@ -85,7 +85,7 @@ >> // Paranoia >> return; >> } >> - UNSAFE.freeMemory(address); >> + UNSAFE.freeDBBMemory(address); >> address = 0; >> Bits.unreserveMemory(size, capacity); >> } >> @@ -118,7 +118,7 @@ >> >> long base = 0; >> try { >> - base = UNSAFE.allocateMemory(size); >> + base = UNSAFE.allocateDBBMemory(size); >> } catch (OutOfMemoryError x) { >> Bits.unreserveMemory(size, cap); >> throw x; >> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> @@ -632,6 +632,26 @@ >> } >> >> /** >> + * Allocates a new block of native memory for DirectByteBuffers, of >> the >> + * given size in bytes. The contents of the memory are >> uninitialized; >> + * they will generally be garbage. The resulting native pointer will >> + * never be zero, and will be aligned for all value types. Dispose >> of >> + * this memory by calling {@link #freeDBBMemory} or resize it with >> + * {@link #reallocateDBBMemory}. >> + * >> + * @throws RuntimeException if the size is negative or too large >> + * for the native size_t type >> + * >> + * @throws OutOfMemoryError if the allocation is refused by the >> system >> + * >> + * @see #getByte(long) >> + * @see #putByte(long, byte) >> + */ >> + public long allocateDBBMemory(long bytes) { >> + return allocateMemory(bytes); >> + } >> + >> + /** >> * Resizes a new block of native memory, to the given size in bytes. >> The >> * contents of the new block past the size of the old block are >> * uninitialized; they will generally be garbage. The resulting >> native >> @@ -687,6 +707,27 @@ >> } >> >> /** >> + * Resizes a new block of native memory for DirectByteBuffers, to the >> + * given size in bytes. The contents of the new block past the size >> of >> + * the old block are uninitialized; they will generally be garbage. >> The >> + * resulting native pointer will be zero if and only if the requested >> size >> + * is zero. The resulting native pointer will be aligned for all >> value >> + * types. Dispose of this memory by calling {@link #freeDBBMemory}, >> or >> + * resize it with {@link #reallocateDBBMemory}. The address passed >> to >> + * this method may be null, in which case an allocation will be >> performed. >> + * >> + * @throws RuntimeException if the size is negative or too large >> + * for the native size_t type >> + * >> + * @throws OutOfMemoryError if the allocation is refused by the >> system >> + * >> + * @see #allocateDBBMemory >> + */ >> + public long reallocateDBBMemory(long address, long bytes) { >> + return reallocateMemory(address, bytes); >> + } >> + >> + /** >> * Sets all bytes in a given block of memory to a fixed value >> * (usually zero). >> * >> @@ -918,6 +959,17 @@ >> checkPointer(null, address); >> } >> >> + /** >> + * Disposes of a block of native memory, as obtained from {@link >> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address >> passed >> + * to this method may be null, in which case no action is taken. >> + * >> + * @see #allocateDBBMemory >> + */ >> + public void freeDBBMemory(long address) { >> + freeMemory(address); >> + } >> + >> /// random queries >> >> /** >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >> > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From thomas.stuefe at gmail.com Wed Feb 14 13:16:14 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 14 Feb 2018 14:16:14 +0100 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Message-ID: On Wed, Feb 14, 2018 at 1:53 PM, David Holmes wrote: > On 14/02/2018 10:43 PM, David Holmes wrote: > >> Adding in core-libs-dev as there's nothing related to hotspot directly >> here. >> > > Correction, this is of course leading to a proposed change in hotspot to > implement the new Unsafe methods and perform the native memory tracking. Of > course we already have NMT so the obvious question is how this will fit in > with NMT? > > I thought Unsafe.allocateMemory is served by hotspot os::malloc(), is it not? So, allocations should show up in NMT with "Unsafe_AllocateMemory0". ..Thomas > David > > > David >> >> On 14/02/2018 9:32 PM, Adam Farley8 wrote: >> >>> Hi All, >>> >>> Currently, diagnostic core files generated from OpenJDK seem to lump all >>> of the >>> native memory usages together, making it near-impossible for someone to >>> figure >>> out *what* is using all that memory in the event of a memory leak. >>> >>> The OpenJ9 VM has a feature which allows it to track the allocation of >>> native >>> memory for Direct Byte Buffers (DBBs), and to supply that information >>> into >>> the >>> cores when they are generated. This makes it a *lot* easier to find out >>> what is using >>> all that native memory, making memory leak resolution less like some dark >>> art, and >>> more like logical debugging. >>> >>> To use this feature, there is a native method referenced in Unsafe.java. >>> To open >>> up this feature so that any VM can make use of it, the java code below >>> sets the >>> stage for it. This change starts letting people call DBB-specific methods >>> when >>> allocating native memory, and getting into the habit of using it. >>> >>> Thoughts? >>> >>> Best Regards >>> >>> Adam Farley >>> >>> P.S. Code: >>> >>> diff --git >>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> @@ -85,7 +85,7 @@ >>> // Paranoia >>> return; >>> } >>> - UNSAFE.freeMemory(address); >>> + UNSAFE.freeDBBMemory(address); >>> address = 0; >>> Bits.unreserveMemory(size, capacity); >>> } >>> @@ -118,7 +118,7 @@ >>> long base = 0; >>> try { >>> - base = UNSAFE.allocateMemory(size); >>> + base = UNSAFE.allocateDBBMemory(size); >>> } catch (OutOfMemoryError x) { >>> Bits.unreserveMemory(size, cap); >>> throw x; >>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>> @@ -632,6 +632,26 @@ >>> } >>> /** >>> + * Allocates a new block of native memory for DirectByteBuffers, of >>> the >>> + * given size in bytes. The contents of the memory are >>> uninitialized; >>> + * they will generally be garbage. The resulting native pointer >>> will >>> + * never be zero, and will be aligned for all value types. Dispose >>> of >>> + * this memory by calling {@link #freeDBBMemory} or resize it with >>> + * {@link #reallocateDBBMemory}. >>> + * >>> + * @throws RuntimeException if the size is negative or too large >>> + * for the native size_t type >>> + * >>> + * @throws OutOfMemoryError if the allocation is refused by the >>> system >>> + * >>> + * @see #getByte(long) >>> + * @see #putByte(long, byte) >>> + */ >>> + public long allocateDBBMemory(long bytes) { >>> + return allocateMemory(bytes); >>> + } >>> + >>> + /** >>> * Resizes a new block of native memory, to the given size in >>> bytes. >>> The >>> * contents of the new block past the size of the old block are >>> * uninitialized; they will generally be garbage. The resulting >>> native >>> @@ -687,6 +707,27 @@ >>> } >>> /** >>> + * Resizes a new block of native memory for DirectByteBuffers, to >>> the >>> + * given size in bytes. The contents of the new block past the size >>> of >>> + * the old block are uninitialized; they will generally be garbage. >>> The >>> + * resulting native pointer will be zero if and only if the >>> requested >>> size >>> + * is zero. The resulting native pointer will be aligned for all >>> value >>> + * types. Dispose of this memory by calling {@link #freeDBBMemory}, >>> or >>> + * resize it with {@link #reallocateDBBMemory}. The address passed >>> to >>> + * this method may be null, in which case an allocation will be >>> performed. >>> + * >>> + * @throws RuntimeException if the size is negative or too large >>> + * for the native size_t type >>> + * >>> + * @throws OutOfMemoryError if the allocation is refused by the >>> system >>> + * >>> + * @see #allocateDBBMemory >>> + */ >>> + public long reallocateDBBMemory(long address, long bytes) { >>> + return reallocateMemory(address, bytes); >>> + } >>> + >>> + /** >>> * Sets all bytes in a given block of memory to a fixed value >>> * (usually zero). >>> * >>> @@ -918,6 +959,17 @@ >>> checkPointer(null, address); >>> } >>> + /** >>> + * Disposes of a block of native memory, as obtained from {@link >>> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address >>> passed >>> + * to this method may be null, in which case no action is taken. >>> + * >>> + * @see #allocateDBBMemory >>> + */ >>> + public void freeDBBMemory(long address) { >>> + freeMemory(address); >>> + } >>> + >>> /// random queries >>> /** >>> >>> Unless stated otherwise above: >>> IBM United Kingdom Limited - Registered in England and Wales with number >>> 741598. >>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 >>> 3AU >>> >>> From adam.farley at uk.ibm.com Wed Feb 14 13:32:46 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 14 Feb 2018 13:32:46 +0000 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Message-ID: >> Adding in core-libs-dev as there's nothing related to hotspot directly >> here. > > Correction, this is of course leading to a proposed change in hotspot to > implement the new Unsafe methods and perform the native memory tracking. Hah, I wrote the same thing in a parallel reply. Jinx. :) - Adam > Of course we already have NMT so the obvious question is how this will > fit in with NMT? > > David It will add granularity to Native Memory Tracking, allowing people to tell, at a glance, how much of the allocated native memory has been used for Direct Byte Buffers. This makes native memory OOM debugging easier. Think of it as an NMT upgrade. Here's an example of what the output should look like: https://developer.ibm.com/answers/questions/288697/why-does-nativememinfo-in-javacore-show-incorrect.html?sort=oldest - Adam > >> David >> >> On 14/02/2018 9:32 PM, Adam Farley8 wrote: >>> Hi All, >>> >>> Currently, diagnostic core files generated from OpenJDK seem to lump all >>> of the >>> native memory usages together, making it near-impossible for someone to >>> figure >>> out *what* is using all that memory in the event of a memory leak. >>> >>> The OpenJ9 VM has a feature which allows it to track the allocation of >>> native >>> memory for Direct Byte Buffers (DBBs), and to supply that information >>> into >>> the >>> cores when they are generated. This makes it a *lot* easier to find out >>> what is using >>> all that native memory, making memory leak resolution less like some dark >>> art, and >>> more like logical debugging. >>> >>> To use this feature, there is a native method referenced in Unsafe.java. >>> To open >>> up this feature so that any VM can make use of it, the java code below >>> sets the >>> stage for it. This change starts letting people call DBB-specific methods >>> when >>> allocating native memory, and getting into the habit of using it. >>> >>> Thoughts? >>> >>> Best Regards >>> >>> Adam Farley >>> >>> P.S. Code: >>> >>> diff --git >>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>> @@ -85,7 +85,7 @@ >>> // Paranoia >>> return; >>> } >>> - UNSAFE.freeMemory(address); >>> + UNSAFE.freeDBBMemory(address); >>> address = 0; >>> Bits.unreserveMemory(size, capacity); >>> } >>> @@ -118,7 +118,7 @@ >>> long base = 0; >>> try { >>> - base = UNSAFE.allocateMemory(size); >>> + base = UNSAFE.allocateDBBMemory(size); >>> } catch (OutOfMemoryError x) { >>> Bits.unreserveMemory(size, cap); >>> throw x; >>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>> @@ -632,6 +632,26 @@ >>> } >>> /** >>> + * Allocates a new block of native memory for DirectByteBuffers, of >>> the >>> + * given size in bytes. The contents of the memory are >>> uninitialized; >>> + * they will generally be garbage. The resulting native pointer >>> will >>> + * never be zero, and will be aligned for all value types. Dispose >>> of >>> + * this memory by calling {@link #freeDBBMemory} or resize it with >>> + * {@link #reallocateDBBMemory}. >>> + * >>> + * @throws RuntimeException if the size is negative or too large >>> + * for the native size_t type >>> + * >>> + * @throws OutOfMemoryError if the allocation is refused by the >>> system >>> + * >>> + * @see #getByte(long) >>> + * @see #putByte(long, byte) >>> + */ >>> + public long allocateDBBMemory(long bytes) { >>> + return allocateMemory(bytes); >>> + } >>> + >>> + /** >>> * Resizes a new block of native memory, to the given size in >>> bytes. >>> The >>> * contents of the new block past the size of the old block are >>> * uninitialized; they will generally be garbage. The resulting >>> native >>> @@ -687,6 +707,27 @@ >>> } >>> /** >>> + * Resizes a new block of native memory for DirectByteBuffers, to >>> the >>> + * given size in bytes. The contents of the new block past the size >>> of >>> + * the old block are uninitialized; they will generally be garbage. >>> The >>> + * resulting native pointer will be zero if and only if the >>> requested >>> size >>> + * is zero. The resulting native pointer will be aligned for all >>> value >>> + * types. Dispose of this memory by calling {@link #freeDBBMemory}, >>> or >>> + * resize it with {@link #reallocateDBBMemory}. The address passed >>> to >>> + * this method may be null, in which case an allocation will be >>> performed. >>> + * >>> + * @throws RuntimeException if the size is negative or too large >>> + * for the native size_t type >>> + * >>> + * @throws OutOfMemoryError if the allocation is refused by the >>> system >>> + * >>> + * @see #allocateDBBMemory >>> + */ >>> + public long reallocateDBBMemory(long address, long bytes) { >>> + return reallocateMemory(address, bytes); >>> + } >>> + >>> + /** >>> * Sets all bytes in a given block of memory to a fixed value >>> * (usually zero). >>> * >>> @@ -918,6 +959,17 @@ >>> checkPointer(null, address); >>> } >>> + /** >>> + * Disposes of a block of native memory, as obtained from {@link >>> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address >>> passed >>> + * to this method may be null, in which case no action is taken. >>> + * >>> + * @see #allocateDBBMemory >>> + */ >>> + public void freeDBBMemory(long address) { >>> + freeMemory(address); >>> + } >>> + >>> /// random queries >>> /** >>> >>> Unless stated otherwise above: >>> IBM United Kingdom Limited - Registered in England and Wales with number >>> 741598. >>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 >>> 3AU >>> Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From thomas.stuefe at gmail.com Wed Feb 14 14:12:18 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 14 Feb 2018 15:12:18 +0100 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Message-ID: On Wed, Feb 14, 2018 at 2:32 PM, Adam Farley8 wrote: > >> Adding in core-libs-dev as there's nothing related to hotspot directly > >> here. > > > > Correction, this is of course leading to a proposed change in hotspot to > > > implement the new Unsafe methods and perform the native memory tracking. > > > Hah, I wrote the same thing in a parallel reply. Jinx. :) > > - Adam > > > Of course we already have NMT so the obvious question is how this will > > fit in with NMT? > > > > David > > It will add granularity to Native Memory Tracking, allowing people > to tell, at a glance, how much of the allocated native memory has been > used for Direct Byte Buffers. This makes native memory OOM > debugging easier. > > Think of it as an NMT upgrade. > > Here's an example of what the output should look like: > > https://developer.ibm.com/answers/questions/288697/why- > does-nativememinfo-in-javacore-show-incorrect.html?sort=oldest > > - Adam > > I think NMT walks the stack, so we should get allocation points grouped by call stacks. Provided we have symbols loaded for the native library using Unsafe.allocateMemory(), this should give us too a fine granularity. But I have not yet tested this in practice. Maybe Zhengyu knows more. ..Thomas > > > >> David > >> > >> On 14/02/2018 9:32 PM, Adam Farley8 wrote: > >>> Hi All, > >>> > >>> Currently, diagnostic core files generated from OpenJDK seem to lump > all > >>> of the > >>> native memory usages together, making it near-impossible for someone > to > >>> figure > >>> out *what* is using all that memory in the event of a memory leak. > >>> > >>> The OpenJ9 VM has a feature which allows it to track the allocation of > >>> native > >>> memory for Direct Byte Buffers (DBBs), and to supply that information > >>> into > >>> the > >>> cores when they are generated. This makes it a *lot* easier to find > out > >>> what is using > >>> all that native memory, making memory leak resolution less like some > dark > >>> art, and > >>> more like logical debugging. > >>> > >>> To use this feature, there is a native method referenced in > Unsafe.java. > >>> To open > >>> up this feature so that any VM can make use of it, the java code below > >>> sets the > >>> stage for it. This change starts letting people call DBB-specific > methods > >>> when > >>> allocating native memory, and getting into the habit of using it. > >>> > >>> Thoughts? > >>> > >>> Best Regards > >>> > >>> Adam Farley > >>> > >>> P.S. Code: > >>> > >>> diff --git > >>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >>> --- > a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >>> +++ > b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >>> @@ -85,7 +85,7 @@ > >>> // Paranoia > >>> return; > >>> } > >>> - UNSAFE.freeMemory(address); > >>> + UNSAFE.freeDBBMemory(address); > >>> address = 0; > >>> Bits.unreserveMemory(size, capacity); > >>> } > >>> @@ -118,7 +118,7 @@ > >>> long base = 0; > >>> try { > >>> - base = UNSAFE.allocateMemory(size); > >>> + base = UNSAFE.allocateDBBMemory(size); > >>> } catch (OutOfMemoryError x) { > >>> Bits.unreserveMemory(size, cap); > >>> throw x; > >>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >>> @@ -632,6 +632,26 @@ > >>> } > >>> /** > >>> + * Allocates a new block of native memory for DirectByteBuffers, > of > >>> the > >>> + * given size in bytes. The contents of the memory are > >>> uninitialized; > >>> + * they will generally be garbage. The resulting native pointer > >>> will > >>> + * never be zero, and will be aligned for all value types. > Dispose > >>> of > >>> + * this memory by calling {@link #freeDBBMemory} or resize it > with > >>> + * {@link #reallocateDBBMemory}. > >>> + * > >>> + * @throws RuntimeException if the size is negative or too large > >>> + * for the native size_t type > >>> + * > >>> + * @throws OutOfMemoryError if the allocation is refused by the > >>> system > >>> + * > >>> + * @see #getByte(long) > >>> + * @see #putByte(long, byte) > >>> + */ > >>> + public long allocateDBBMemory(long bytes) { > >>> + return allocateMemory(bytes); > >>> + } > >>> + > >>> + /** > >>> * Resizes a new block of native memory, to the given size in > >>> bytes. > >>> The > >>> * contents of the new block past the size of the old block are > >>> * uninitialized; they will generally be garbage. The resulting > >>> native > >>> @@ -687,6 +707,27 @@ > >>> } > >>> /** > >>> + * Resizes a new block of native memory for DirectByteBuffers, to > > >>> the > >>> + * given size in bytes. The contents of the new block past the > size > >>> of > >>> + * the old block are uninitialized; they will generally be > garbage. > >>> The > >>> + * resulting native pointer will be zero if and only if the > >>> requested > >>> size > >>> + * is zero. The resulting native pointer will be aligned for all > >>> value > >>> + * types. Dispose of this memory by calling {@link > #freeDBBMemory}, > >>> or > >>> + * resize it with {@link #reallocateDBBMemory}. The address > passed > >>> to > >>> + * this method may be null, in which case an allocation will be > >>> performed. > >>> + * > >>> + * @throws RuntimeException if the size is negative or too large > >>> + * for the native size_t type > >>> + * > >>> + * @throws OutOfMemoryError if the allocation is refused by the > >>> system > >>> + * > >>> + * @see #allocateDBBMemory > >>> + */ > >>> + public long reallocateDBBMemory(long address, long bytes) { > >>> + return reallocateMemory(address, bytes); > >>> + } > >>> + > >>> + /** > >>> * Sets all bytes in a given block of memory to a fixed value > >>> * (usually zero). > >>> * > >>> @@ -918,6 +959,17 @@ > >>> checkPointer(null, address); > >>> } > >>> + /** > >>> + * Disposes of a block of native memory, as obtained from {@link > >>> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The > address > >>> passed > >>> + * to this method may be null, in which case no action is taken. > >>> + * > >>> + * @see #allocateDBBMemory > >>> + */ > >>> + public void freeDBBMemory(long address) { > >>> + freeMemory(address); > >>> + } > >>> + > >>> /// random queries > >>> /** > >>> > >>> Unless stated otherwise above: > >>> IBM United Kingdom Limited - Registered in England and Wales with > number > >>> 741598. > >>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > > >>> 3AU > >>> > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > From adam.farley at uk.ibm.com Wed Feb 14 14:13:00 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 14 Feb 2018 14:13:00 +0000 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java Message-ID: Hi All, -- Short version -- Could a committer please take the fix for JDK-8190187 (full code included in the bug) and: 1) Complete the CSR process for the new JNI Return code. 2) Commit the changes that contain (a) the new return code, and (b) the non-Hotspot code that handles the new code. Backporting would be appreciated, but is optional. Also optional: Commit the jdwp code that serves as an example for how this code should be used. CSR again, unfortunately. -- Long Version -- We have a bug in OpenJDK where if you pass an info-only option (like -agentlib:jdwp=help) in through the JNI interface, it can exit your code with RC 0. I think this is a bug because if you planned to do anything after starting a Java VM, you have to do it in an exit hook. If an info-only option is used, exit(0) should not happen. Instead, the user's code should be told the VM cannot be used. Bug Link: https://bugs.openjdk.java.net/browse/JDK-8190187 I have proposed the creation of a new JNI return code indicating a "silent exit", where the vm encountered no error, but that it is not in a fit state to be used. See the link for an implementation of this, along with a handy test. In short, if you agree that this is a problem, we need a champion to: 1) Complete the CSR process for the new JNI Return code. 2) Commit the changes that contain (a) the new return code, and (b) the non-Hotspot code that handles the new code. Optionally, if you want to commit the code containing an example of the fix, you will need to: 3) Commit the Hotspot code that channels the new return code. 4) Complete the CSR process for the jdwp behaviour change. 5) Commit the jdwp code. Note that all of this code has *already been written*. This should be an easy commit once the CSR is done with. Best Regards Adam Farley Best Regards Adam Farley Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Alan.Bateman at oracle.com Wed Feb 14 14:45:31 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Feb 2018 14:45:31 +0000 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java In-Reply-To: References: Message-ID: On 14/02/2018 14:13, Adam Farley8 wrote: > Hi All, > > -- Short version -- > > Could a committer please take the fix for JDK-8190187 (full code included > in the bug) and: > > 1) Complete the CSR process for the new JNI Return code. > 2) Commit the changes that contain (a) the new return code, and (b) the > non-Hotspot code that handles the new code. The patches attached to the JIRA issue are missing the changes to the JVM TI spec (jvmti.xml). There is also text to be written for the JNI spec if this proposal goes ahead. I don't agree that the launcher should be looking for "-agentlib:jdwp=help" in the command line as it's just one of many ways that the debugger agent might be started (e.g. -Xrunjdwp:, _JAVA_TOOLS_OPTIONS, ...). > Backporting would be appreciated, but is optional. I don't think these changes are appropriate to backport as they include specification changes/ -Alan From mark.reinhold at oracle.com Wed Feb 14 15:26:29 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 14 Feb 2018 07:26:29 -0800 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string In-Reply-To: References: Message-ID: <20180214072629.629736428@eggemoggin.niobe.net> 2018/2/14 2:20:22 -0800, volker.simonis at gmail.com: > can I please get a review for the following tiny fix: > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8197927/ > https://bugs.openjdk.java.net/browse/JDK-8197927 > > The new Java 10 specification makes the 'java.vendor.version' property > mandatory [1] but the current implementations doesn't allow to set it > to an empty string. This is a bug in the specification, not the implementation. As I just wrote in a comment on 8197927: JEP 322 expresses the intended behavior: If `--with-vendor-version` is not specified at build time then the `java.vendor.version` property has no value. (That's different from the empty string, which is a value.) The bug is in the specification, which should be revised so as not to require that this property have a value. Fixing the spec is not critical for 10; I suggest downgrading 8197927 to P2 and targeting 11. - Mark From zgu at redhat.com Wed Feb 14 13:53:46 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 14 Feb 2018 08:53:46 -0500 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Message-ID: On 02/14/2018 08:16 AM, Thomas St?fe wrote: > On Wed, Feb 14, 2018 at 1:53 PM, David Holmes > wrote: > >> On 14/02/2018 10:43 PM, David Holmes wrote: >> >>> Adding in core-libs-dev as there's nothing related to hotspot directly >>> here. >>> >> >> Correction, this is of course leading to a proposed change in hotspot to >> implement the new Unsafe methods and perform the native memory tracking. Of >> course we already have NMT so the obvious question is how this will fit in >> with NMT? >> >> > I thought Unsafe.allocateMemory is served by hotspot os::malloc(), is it > not? So, allocations should show up in NMT with "Unsafe_AllocateMemory0". Could use another category? to make it earlier to identify. Thanks, -Zhengyu > > ..Thomas > > > >> David >> >> >> David >>> >>> On 14/02/2018 9:32 PM, Adam Farley8 wrote: >>> >>>> Hi All, >>>> >>>> Currently, diagnostic core files generated from OpenJDK seem to lump all >>>> of the >>>> native memory usages together, making it near-impossible for someone to >>>> figure >>>> out *what* is using all that memory in the event of a memory leak. >>>> >>>> The OpenJ9 VM has a feature which allows it to track the allocation of >>>> native >>>> memory for Direct Byte Buffers (DBBs), and to supply that information >>>> into >>>> the >>>> cores when they are generated. This makes it a *lot* easier to find out >>>> what is using >>>> all that native memory, making memory leak resolution less like some dark >>>> art, and >>>> more like logical debugging. >>>> >>>> To use this feature, there is a native method referenced in Unsafe.java. >>>> To open >>>> up this feature so that any VM can make use of it, the java code below >>>> sets the >>>> stage for it. This change starts letting people call DBB-specific methods >>>> when >>>> allocating native memory, and getting into the habit of using it. >>>> >>>> Thoughts? >>>> >>>> Best Regards >>>> >>>> Adam Farley >>>> >>>> P.S. Code: >>>> >>>> diff --git >>>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>>> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>>> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>>> @@ -85,7 +85,7 @@ >>>> // Paranoia >>>> return; >>>> } >>>> - UNSAFE.freeMemory(address); >>>> + UNSAFE.freeDBBMemory(address); >>>> address = 0; >>>> Bits.unreserveMemory(size, capacity); >>>> } >>>> @@ -118,7 +118,7 @@ >>>> long base = 0; >>>> try { >>>> - base = UNSAFE.allocateMemory(size); >>>> + base = UNSAFE.allocateDBBMemory(size); >>>> } catch (OutOfMemoryError x) { >>>> Bits.unreserveMemory(size, cap); >>>> throw x; >>>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>>> @@ -632,6 +632,26 @@ >>>> } >>>> /** >>>> + * Allocates a new block of native memory for DirectByteBuffers, of >>>> the >>>> + * given size in bytes. The contents of the memory are >>>> uninitialized; >>>> + * they will generally be garbage. The resulting native pointer >>>> will >>>> + * never be zero, and will be aligned for all value types. Dispose >>>> of >>>> + * this memory by calling {@link #freeDBBMemory} or resize it with >>>> + * {@link #reallocateDBBMemory}. >>>> + * >>>> + * @throws RuntimeException if the size is negative or too large >>>> + * for the native size_t type >>>> + * >>>> + * @throws OutOfMemoryError if the allocation is refused by the >>>> system >>>> + * >>>> + * @see #getByte(long) >>>> + * @see #putByte(long, byte) >>>> + */ >>>> + public long allocateDBBMemory(long bytes) { >>>> + return allocateMemory(bytes); >>>> + } >>>> + >>>> + /** >>>> * Resizes a new block of native memory, to the given size in >>>> bytes. >>>> The >>>> * contents of the new block past the size of the old block are >>>> * uninitialized; they will generally be garbage. The resulting >>>> native >>>> @@ -687,6 +707,27 @@ >>>> } >>>> /** >>>> + * Resizes a new block of native memory for DirectByteBuffers, to >>>> the >>>> + * given size in bytes. The contents of the new block past the size >>>> of >>>> + * the old block are uninitialized; they will generally be garbage. >>>> The >>>> + * resulting native pointer will be zero if and only if the >>>> requested >>>> size >>>> + * is zero. The resulting native pointer will be aligned for all >>>> value >>>> + * types. Dispose of this memory by calling {@link #freeDBBMemory}, >>>> or >>>> + * resize it with {@link #reallocateDBBMemory}. The address passed >>>> to >>>> + * this method may be null, in which case an allocation will be >>>> performed. >>>> + * >>>> + * @throws RuntimeException if the size is negative or too large >>>> + * for the native size_t type >>>> + * >>>> + * @throws OutOfMemoryError if the allocation is refused by the >>>> system >>>> + * >>>> + * @see #allocateDBBMemory >>>> + */ >>>> + public long reallocateDBBMemory(long address, long bytes) { >>>> + return reallocateMemory(address, bytes); >>>> + } >>>> + >>>> + /** >>>> * Sets all bytes in a given block of memory to a fixed value >>>> * (usually zero). >>>> * >>>> @@ -918,6 +959,17 @@ >>>> checkPointer(null, address); >>>> } >>>> + /** >>>> + * Disposes of a block of native memory, as obtained from {@link >>>> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address >>>> passed >>>> + * to this method may be null, in which case no action is taken. >>>> + * >>>> + * @see #allocateDBBMemory >>>> + */ >>>> + public void freeDBBMemory(long address) { >>>> + freeMemory(address); >>>> + } >>>> + >>>> /// random queries >>>> /** >>>> >>>> Unless stated otherwise above: >>>> IBM United Kingdom Limited - Registered in England and Wales with number >>>> 741598. >>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 >>>> 3AU >>>> >>>> From zgu at redhat.com Wed Feb 14 14:51:31 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 14 Feb 2018 09:51:31 -0500 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Message-ID: <012e8977-3cd6-ceef-cd67-a8ae55427264@redhat.com> >> Think of it as an NMT upgrade. >> >> Here's an example of what the output should look like: >> >> https://developer.ibm.com/answers/questions/288697/why- >> does-nativememinfo-in-javacore-show-incorrect.html?sort=oldest >> >> - Adam >> >> > I think NMT walks the stack, so we should get allocation points grouped by > call stacks. Provided we have symbols loaded for the native library using > Unsafe.allocateMemory(), this should give us too a fine granularity. But I > have not yet tested this in practice. Maybe Zhengyu knows more. Quick test shows this call site: [0x00007f8558b26243] Unsafe_AllocateMemory0+0x93 [0x00007f8537b085cb] (malloc=2KB type=Internal #1) I will take a look why there is a frame not decoded. Thanks, -Zhengyu > > ..Thomas > > >>> >>>> David >>>> >>>> On 14/02/2018 9:32 PM, Adam Farley8 wrote: >>>>> Hi All, >>>>> >>>>> Currently, diagnostic core files generated from OpenJDK seem to lump >> all >>>>> of the >>>>> native memory usages together, making it near-impossible for someone >> to >>>>> figure >>>>> out *what* is using all that memory in the event of a memory leak. >>>>> >>>>> The OpenJ9 VM has a feature which allows it to track the allocation of >>>>> native >>>>> memory for Direct Byte Buffers (DBBs), and to supply that information >>>>> into >>>>> the >>>>> cores when they are generated. This makes it a *lot* easier to find >> out >>>>> what is using >>>>> all that native memory, making memory leak resolution less like some >> dark >>>>> art, and >>>>> more like logical debugging. >>>>> >>>>> To use this feature, there is a native method referenced in >> Unsafe.java. >>>>> To open >>>>> up this feature so that any VM can make use of it, the java code below >>>>> sets the >>>>> stage for it. This change starts letting people call DBB-specific >> methods >>>>> when >>>>> allocating native memory, and getting into the habit of using it. >>>>> >>>>> Thoughts? >>>>> >>>>> Best Regards >>>>> >>>>> Adam Farley >>>>> >>>>> P.S. Code: >>>>> >>>>> diff --git >>>>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>>>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>>>> --- >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>>>> +++ >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >>>>> @@ -85,7 +85,7 @@ >>>>> // Paranoia >>>>> return; >>>>> } >>>>> - UNSAFE.freeMemory(address); >>>>> + UNSAFE.freeDBBMemory(address); >>>>> address = 0; >>>>> Bits.unreserveMemory(size, capacity); >>>>> } >>>>> @@ -118,7 +118,7 @@ >>>>> long base = 0; >>>>> try { >>>>> - base = UNSAFE.allocateMemory(size); >>>>> + base = UNSAFE.allocateDBBMemory(size); >>>>> } catch (OutOfMemoryError x) { >>>>> Bits.unreserveMemory(size, cap); >>>>> throw x; >>>>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>>>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>>>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>>>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >>>>> @@ -632,6 +632,26 @@ >>>>> } >>>>> /** >>>>> + * Allocates a new block of native memory for DirectByteBuffers, >> of >>>>> the >>>>> + * given size in bytes. The contents of the memory are >>>>> uninitialized; >>>>> + * they will generally be garbage. The resulting native pointer >>>>> will >>>>> + * never be zero, and will be aligned for all value types. >> Dispose >>>>> of >>>>> + * this memory by calling {@link #freeDBBMemory} or resize it >> with >>>>> + * {@link #reallocateDBBMemory}. >>>>> + * >>>>> + * @throws RuntimeException if the size is negative or too large >>>>> + * for the native size_t type >>>>> + * >>>>> + * @throws OutOfMemoryError if the allocation is refused by the >>>>> system >>>>> + * >>>>> + * @see #getByte(long) >>>>> + * @see #putByte(long, byte) >>>>> + */ >>>>> + public long allocateDBBMemory(long bytes) { >>>>> + return allocateMemory(bytes); >>>>> + } >>>>> + >>>>> + /** >>>>> * Resizes a new block of native memory, to the given size in >>>>> bytes. >>>>> The >>>>> * contents of the new block past the size of the old block are >>>>> * uninitialized; they will generally be garbage. The resulting >>>>> native >>>>> @@ -687,6 +707,27 @@ >>>>> } >>>>> /** >>>>> + * Resizes a new block of native memory for DirectByteBuffers, to >> >>>>> the >>>>> + * given size in bytes. The contents of the new block past the >> size >>>>> of >>>>> + * the old block are uninitialized; they will generally be >> garbage. >>>>> The >>>>> + * resulting native pointer will be zero if and only if the >>>>> requested >>>>> size >>>>> + * is zero. The resulting native pointer will be aligned for all >>>>> value >>>>> + * types. Dispose of this memory by calling {@link >> #freeDBBMemory}, >>>>> or >>>>> + * resize it with {@link #reallocateDBBMemory}. The address >> passed >>>>> to >>>>> + * this method may be null, in which case an allocation will be >>>>> performed. >>>>> + * >>>>> + * @throws RuntimeException if the size is negative or too large >>>>> + * for the native size_t type >>>>> + * >>>>> + * @throws OutOfMemoryError if the allocation is refused by the >>>>> system >>>>> + * >>>>> + * @see #allocateDBBMemory >>>>> + */ >>>>> + public long reallocateDBBMemory(long address, long bytes) { >>>>> + return reallocateMemory(address, bytes); >>>>> + } >>>>> + >>>>> + /** >>>>> * Sets all bytes in a given block of memory to a fixed value >>>>> * (usually zero). >>>>> * >>>>> @@ -918,6 +959,17 @@ >>>>> checkPointer(null, address); >>>>> } >>>>> + /** >>>>> + * Disposes of a block of native memory, as obtained from {@link >>>>> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The >> address >>>>> passed >>>>> + * to this method may be null, in which case no action is taken. >>>>> + * >>>>> + * @see #allocateDBBMemory >>>>> + */ >>>>> + public void freeDBBMemory(long address) { >>>>> + freeMemory(address); >>>>> + } >>>>> + >>>>> /// random queries >>>>> /** >>>>> >>>>> Unless stated otherwise above: >>>>> IBM United Kingdom Limited - Registered in England and Wales with >> number >>>>> 741598. >>>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 >> >>>>> 3AU >>>>> >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >> From ecki at zusammenkunft.net Wed Feb 14 15:55:18 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Wed, 14 Feb 2018 16:55:18 +0100 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native MemoryUsage for Direct Byte Buffers In-Reply-To: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> References: <4d7c4629-7866-13f8-1777-2ff610784dfd@oracle.com> Message-ID: <5a845be7.01c0df0a.35f1a.f0ec@mx.google.com> Maybe instead adding a ?allocation request type? argment to allocate Memory? (and wrap it with a typed allocator in the buffer interfacessomewhere?) the ?DBB? part Looks especially cryptic. We have similiar concepts for NMT in the native Code. Besides I mentioned a while back that the JMX part of the memory Accounting could be improved as well. Those two could probably be unified. This is where I see most Troubleshooting activities struggle. Gruss Bernd -- http://bernd.eckenfels.net Von: David Holmes Gesendet: Mittwoch, 14. Februar 2018 16:44 An: Adam Farley8; hotspot-dev at openjdk.java.net; core-libs-dev Libs Betreff: Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native MemoryUsage for Direct Byte Buffers Adding in core-libs-dev as there's nothing related to hotspot directly here. David On 14/02/2018 9:32 PM, Adam Farley8 wrote: > Hi All, > > Currently, diagnostic core files generated from OpenJDK seem to lump all > of the > native memory usages together, making it near-impossible for someone to > figure > out *what* is using all that memory in the event of a memory leak. > > The OpenJ9 VM has a feature which allows it to track the allocation of > native > memory for Direct Byte Buffers (DBBs), and to supply that information into > the > cores when they are generated. This makes it a *lot* easier to find out > what is using > all that native memory, making memory leak resolution less like some dark > art, and > more like logical debugging. > > To use this feature, there is a native method referenced in Unsafe.java. > To open > up this feature so that any VM can make use of it, the java code below > sets the > stage for it. This change starts letting people call DBB-specific methods > when > allocating native memory, and getting into the habit of using it. > > Thoughts? > > Best Regards > > Adam Farley > > P.S. Code: > > diff --git > a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > @@ -85,7 +85,7 @@ > // Paranoia > return; > } > - UNSAFE.freeMemory(address); > + UNSAFE.freeDBBMemory(address); > address = 0; > Bits.unreserveMemory(size, capacity); > } > @@ -118,7 +118,7 @@ > > long base = 0; > try { > - base = UNSAFE.allocateMemory(size); > + base = UNSAFE.allocateDBBMemory(size); > } catch (OutOfMemoryError x) { > Bits.unreserveMemory(size, cap); > throw x; > diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > @@ -632,6 +632,26 @@ > } > > /** > + * Allocates a new block of native memory for DirectByteBuffers, of > the > + * given size in bytes. The contents of the memory are > uninitialized; > + * they will generally be garbage. The resulting native pointer will > + * never be zero, and will be aligned for all value types. Dispose > of > + * this memory by calling {@link #freeDBBMemory} or resize it with > + * {@link #reallocateDBBMemory}. > + * > + * @throws RuntimeException if the size is negative or too large > + * for the native size_t type > + * > + * @throws OutOfMemoryError if the allocation is refused by the > system > + * > + * @see #getByte(long) > + * @see #putByte(long, byte) > + */ > + public long allocateDBBMemory(long bytes) { > + return allocateMemory(bytes); > + } > + > + /** > * Resizes a new block of native memory, to the given size in bytes. > The > * contents of the new block past the size of the old block are > * uninitialized; they will generally be garbage. The resulting > native > @@ -687,6 +707,27 @@ > } > > /** > + * Resizes a new block of native memory for DirectByteBuffers, to the > + * given size in bytes. The contents of the new block past the size > of > + * the old block are uninitialized; they will generally be garbage. > The > + * resulting native pointer will be zero if and only if the requested > size > + * is zero. The resulting native pointer will be aligned for all > value > + * types. Dispose of this memory by calling {@link #freeDBBMemory}, > or > + * resize it with {@link #reallocateDBBMemory}. The address passed > to > + * this method may be null, in which case an allocation will be > performed. > + * > + * @throws RuntimeException if the size is negative or too large > + * for the native size_t type > + * > + * @throws OutOfMemoryError if the allocation is refused by the > system > + * > + * @see #allocateDBBMemory > + */ > + public long reallocateDBBMemory(long address, long bytes) { > + return reallocateMemory(address, bytes); > + } > + > + /** > * Sets all bytes in a given block of memory to a fixed value > * (usually zero). > * > @@ -918,6 +959,17 @@ > checkPointer(null, address); > } > > + /** > + * Disposes of a block of native memory, as obtained from {@link > + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address > passed > + * to this method may be null, in which case no action is taken. > + * > + * @see #allocateDBBMemory > + */ > + public void freeDBBMemory(long address) { > + freeMemory(address); > + } > + > /// random queries > > /** > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > From volker.simonis at gmail.com Wed Feb 14 16:04:15 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 14 Feb 2018 17:04:15 +0100 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string In-Reply-To: <20180214072629.629736428@eggemoggin.niobe.net> References: <20180214072629.629736428@eggemoggin.niobe.net> Message-ID: On Wed, Feb 14, 2018 at 4:26 PM, wrote: > 2018/2/14 2:20:22 -0800, volker.simonis at gmail.com: >> can I please get a review for the following tiny fix: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2018/8197927/ >> https://bugs.openjdk.java.net/browse/JDK-8197927 >> >> The new Java 10 specification makes the 'java.vendor.version' property >> mandatory [1] but the current implementations doesn't allow to set it >> to an empty string. > > This is a bug in the specification, not the implementation. As I just > wrote in a comment on 8197927: > > JEP 322 expresses the intended behavior: If `--with-vendor-version` is > not specified at build time then the `java.vendor.version` property has > no value. (That's different from the empty string, which is a value.) > The bug is in the specification, which should be revised so as not to > require that this property have a value. Fixing the spec is not > critical for 10; I suggest downgrading 8197927 to P2 and targeting 11. > What you mean by "not require the 'java.vendor.version' property to have a value"? We can not set a system property to NULL because System.setProperty() will throw a NPE in that case. So either the specification requires 'java.vendor.version' as mandatory (as it is now) in which case it does need to have a value different from NULL. Or we change the specification such that 'java.vendor.version' isn't a mandatory property any more. But it is not possible to have 'java.vendor.version' as mandatory property with "no value" as envisioned by the JEP. Do you propose to change the specification such that 'java.vendor.version' isn't a mandatory property any more? That would be fine for me and I agree that we could target this bug to 11 if that's what you propose. We could actually list "java.vendor.version" under "Implementation Note:" which contains some additional, non-mandatory properties. > - Mark From paul.sandoz at oracle.com Wed Feb 14 17:53:26 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 14 Feb 2018 09:53:26 -0800 Subject: RFR: 8197893: Mistaken type check in CheckedEntrySet.toArray In-Reply-To: References: Message-ID: <398FF5E4-51CF-49A6-89AA-538A28B3D503@oracle.com> > On Feb 13, 2018, at 5:33 PM, Martin Buchholz wrote: > > http://cr.openjdk.java.net/~martin/webrevs/jdk/CheckedEntrySet-toArray/ > https://bugs.openjdk.java.net/browse/JDK-8197893 +1 Paul. From Roger.Riggs at Oracle.com Wed Feb 14 18:08:12 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 14 Feb 2018 13:08:12 -0500 Subject: RFR JDK-8164278: java.util.Base64.EncOutputStream/DecInputStream is slower than corresponding version in javax.mail package In-Reply-To: <5A7BC4D2.3050000@oracle.com> References: <5A790C4D.1090309@oracle.com> <5A7BC4D2.3050000@oracle.com> Message-ID: Hi Sherman, I found updates in http://cr.openjdk.java.net/~sherman/8164278/webrev.04 That looks fine. Typo:? line 1008: "neve" Roger On 2/7/2018 10:32 PM, Xueming Shen wrote: > Hi Roger, > > Given the concern of the possible incompatible behavior change of over > reading bytes > from the underlying stream. I decided to give up last proposed changes > for DecInputStream > for now. With some "minor" cleanup and tuning I still have about 10%+ > improvement with > various input size sampling. Let's address the decoder stream in > separate rfe later, if desired. > > The updated webrev and the jmh results are at > > http://cr.openjdk.java.net/~sherman/8164278/webrev > http://cr.openjdk.java.net/~sherman/8164278/base64.bm > > Last version of webrev and corresponding jmh result can be found at > http://cr.openjdk.java.net/~sherman/8164278/webrev.02 > http://cr.openjdk.java.net/~sherman/8164278/base64.bm.old > > Thanks, > Sherman > > On 2/5/18, 6:00 PM, Xueming Shen wrote: >> Hi, >> >> Please help review the change for? JDK-8164278. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8164278 >> webrev: http://cr.openjdk.java.net/~sherman/8164278/webrev >> >> jmh.src: http://cr.openjdk.java.net/~sherman/8164278/Base64BM.java >> jmh.result: http://cr.openjdk.java.net/~sherman/8164278/base64.bm >> >> Base64.Decoder.decode0: >> ??? Adopted the "similar" optimization approach we took in >> Base64.Encoder.encode0() >> ??? to add a "fast path" to decode a block of 4-byte units together >> (current implementation >> ??? decodes one single byte per while/loop. The jmh benchmark result >> indicates a big speed >> ??? boost? (those decodeArray/Mime/Url results, from 30% to 2 times >> faster, depends on >> ??? input size). >> >> Base64.Encoder.encode0() >> ??? It appears encode0() was fully optimized in 1.8. Can't get it >> faster :-) Tried to use >> ??? Unsafe.getLong/putLong instead of byte by byte access. But it >> appears the 8-byte >> ??? "vectorization" does not bring us enough speed up, the >> performance is the same as the >> ??? current one. See encode00() at >> ??? http://cr.openjdk.java.net/~sherman/8164278/webrev.00 >> >> Base64.Encoder.wrap(OutputStream)/EncOutputStream.write(): >> ??? If my memory serves me right, the current implementation was >> under the assumption that >> ??? the underlying output stream probably is buffered (if invoker >> cares). It would be a redundant >> ??? if EncOutputStream buffers bytes again. It appears this is a >> wrong assumption. It is much >> ??? slower to write 4 bytes separately, compared to bundle them >> together in a byte[4] and write >> ??? into underlying, even the underlying output stream is a >> ByteArrayOutputStream. >> ??? Again, the proposed change is to add a fast path loop, as we do >> in encode0(), to decode/ >> ??? write a block of 3-byte->4-byte unites. It appears this fast loop >> can help the compiler to >> ??? optimize away some boundary checks, therefor is much faster. >> ??? The jmh result Base64BM.encodeOS suggests the new implementation >> is almost 4 times faster >> ??? and is almost the same as java.mail's stream encoder. >> >> Base64.Decoder.wrap(InputStream)/DecInputStream.read(): >> ??? Same as the approach we take for decode0(), to add a fast path >> decode block of 4-byte unites >> ??? together. >> ??? The jmh result Base64BM.decodeOS (the name probably should be >> decodeIS, for InputStream, >> ??? but anyway...) shows the proposed one is 4 times faster than the >> existing impl and double >> ??? the? java.mail (Base64BM.decodeOS_javamail) implementation. >> >> ??? However, there is a side-effect of adding a buffering mechanism >> into DecInputStream. The >> ??? current implementation read bytes from the underlying stream one >> by one, it never reads >> ??? more bytes than it needs, which means it should/is supposed to >> just stop at the last byte >> ??? that it needs to decode, when there is "=" present in the stream. >> ??? With buffering, it's possible more bytes (after "=", which >> indicates "end of base64 stream") might >> ??? be read/consumed in and buffered.? A concern? if this is indeed a >> concern, the only alternative >> ??? might be to add a separate method to support this >> "faster-buffered-decoder"? >> >> Thanks, >> Sherman >> >> >> > From yumin.qi at gmail.com Wed Feb 14 18:48:41 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 14 Feb 2018 10:48:41 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: Hi, Alan Thanks. Updated on same link http://cr.openjdk.java.net/~minqi/8194154/webrev1/ as your recommendation. Yumin On Wed, Feb 14, 2018 at 4:42 AM, Alan Bateman wrote: > On 14/02/2018 01:23, yumin qi wrote: > >> Hi, >> >> I have update the webrev: >> http://cr.openjdk.java.net/~minqi/8194154/webrev1/ < >> http://cr.openjdk.java.net/%7Eminqi/8194154/webrev1/> >> >> In this version, as suggested by Alan(thanks!), property of "user.dir" >> is cached and behave like it is 'read only'. The change made to *inux as >> well as windows. Since property of "user.dir" is cached, any changes via >> property setting for it has no effect. >> >> This looks much better but you've removed a permission check from the > Windows getUserPath implementation. That code the same permission check as > the resolve method in the Unix implementation. > > I think the test needs works. The simplest would be to call > getCanonicalFile before changing the system property, then call it after > and check that you an equal result. Also no need to hack the Properties > object, you can use System.setProperty instead. > > -Alan > > > From Roger.Riggs at Oracle.com Wed Feb 14 19:13:41 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 14 Feb 2018 14:13:41 -0500 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: References: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> <94168540-0cf6-8f9f-41bd-3f64ba41187f@oracle.com> Message-ID: <84b5c500-35be-fff2-787f-aad5eb613ffc@Oracle.com> Hi, In the test, the message can be improved; though the exception should never occur it will be misleading if it does. Instead: ??? "Changing property user.dir should have no effect on the getCanonicalPath" It is cleaner to throw AssertionError for the test failure. Throwing FileSystemException makes it look like it came from inside the file system implementation. Regards, Roger On 2/13/2018 6:45 PM, yumin qi wrote: > Alan, > > In fact, if property "user.dir" is cached, the problem will go away > without any changes to native or UnixFileSystem.java, since it won't > canonicalize the string supplied from user. The output will be > getProperty("user.dir") + "/" + file. (The result is not as expected, > user can not change the property, means user has to work around the problem > to reach their targeted directory to do things like loading classes) > > Any change to "user.dir" in java program does not have any effect. > For Windows: > > --- a/src/java.base/windows/classes/java/io/WinNTFileSystem.java Fri Feb 02 > 10:32:59 2018 -0800 > +++ b/src/java.base/windows/classes/java/io/WinNTFileSystem.java Tue Feb 13 > 23:40:21 2018 +0000 > @@ -43,12 +43,14 @@ > private final char slash; > private final char altSlash; > private final char semicolon; > + private final String userDir; > > public WinNTFileSystem() { > Properties props = GetPropertyAction.privilegedGetProperties(); > slash = props.getProperty("file.separator").charAt(0); > semicolon = props.getProperty("path.separator").charAt(0); > altSlash = (this.slash == '\\') ? '/' : '\\'; > + userDir = props.getProperty("user.dir"); > } > > private boolean isSlash(char c) { > @@ -347,7 +349,7 @@ > private String getUserPath() { > /* For both compatibility and security, > we must look this up every time */ > - return normalize(System.getProperty("user.dir")); > + return normalize(userDir); > } > > private String getDrive(String path) { > > Does it need normalize(userDir) in getUserPath()? I think we need here. > > I will update the webrev and send for another round of review: the change > in native will be reverted, and made changes to UnixFileSystem.java. > > > Thanks > Yumin > > > On Mon, Feb 12, 2018 at 3:09 AM, Alan Bateman > wrote: > >> On 09/02/2018 18:01, Alan Bateman wrote: >> >>> : >>> >>> I'll study the patch you have but I think we also need to create issues >>> to get us to the point where changing this system property in a running VM >>> doesn't impact running code. >>> >> Looking at it again, I think we should change java.io.UnixFileSystem (and >> the Windows equivalent) to cache the value of user.dir to avoid difficult >> to diagnose issues with bad code changing the value of this property in a >> running VM. This should reduce the issue down to cases where user.dir is >> changed on the command line (never supported either of course) to a value >> that is not "/" but has trailing or duplicate slashes. >> >> When reduced down then the alternatives are to change the native >> canonicalize method as you have done or alternatively do it once at >> UnixFileSystem initialization time so that canonicalize does not have to >> deal with this case. The former would require changing the description of >> the function (it currently reads "The input path is assumed to contain no >> duplicate slashes"), the latter avoids any changes to the native >> implementation. >> >> -Alan >> >> >> diff -r 0937e5f799df src/java.base/unix/classes/jav >> a/io/UnixFileSystem.java >> --- a/src/java.base/unix/classes/java/io/UnixFileSystem.java Sat Feb >> 10 07:06:16 2018 -0500 >> +++ b/src/java.base/unix/classes/java/io/UnixFileSystem.java Mon Feb >> 12 10:49:40 2018 +0000 >> @@ -34,12 +34,14 @@ >> private final char slash; >> private final char colon; >> private final String javaHome; >> + private final String userDir; >> >> public UnixFileSystem() { >> Properties props = GetPropertyAction.privilegedGetProperties(); >> slash = props.getProperty("file.separator").charAt(0); >> colon = props.getProperty("path.separator").charAt(0); >> javaHome = props.getProperty("java.home"); >> + userDir = props.getProperty("user.dir"); >> } >> >> >> @@ -128,7 +130,11 @@ >> >> public String resolve(File f) { >> if (isAbsolute(f)) return f.getPath(); >> - return resolve(System.getProperty("user.dir"), f.getPath()); >> + SecurityManager sm = System.getSecurityManager(); >> + if (sm != null) { >> + sm.checkPropertyAccess("user.dir"); >> + } >> + return resolve(userDir, f.getPath()); >> } >> >> // Caches for canonicalization results to improve startup performance. >> >> From brian.burkhalter at oracle.com Wed Feb 14 19:17:03 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 14 Feb 2018 11:17:03 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: Hello, In the test you have /* @test requires (os.family == "linux") | (os.family == "mac") | (os.family == "solaris") | (os.family == "aix") @bug 8194154 @summary Test parsing path with double slashes on unix like platforms. */ but I think you need to have the @requires annotation on a separate line for example as: /* @test * @requires (os.family == "linux") | (os.family == "mac?) * | (os.family == "solaris") | (os.family == "aix") * @bug 8194154 * @summary Test parsing path with double slashes on unix like platforms. */ Thanks, Brian On Feb 14, 2018, at 10:48 AM, yumin qi wrote: > Thanks. Updated on same link > http://cr.openjdk.java.net/~minqi/8194154/webrev1/ > as your recommendation. From mandy.chung at oracle.com Wed Feb 14 21:15:18 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 14 Feb 2018 13:15:18 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> Message-ID: <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> On 2/14/18 1:58 AM, Peter Levart wrote: > > I take back this claim. Of course the the following race is possible: > > - Thread1: calls runAllFinalizers and takes a Finalizer from > 'unprocessed' list. > - Thread2: takee the same Finalizer instance from ReferenceQueue and > calls runFinalizer() > - Thread1: calls runFinalizer() with the same instance for the 2nd > time now. runAllFinalizers is invoked during shutdown when System.runFinalizersOnExit has been called. I have been wanting to remove System::runFinalizersOnExit [1] which is the culprit of causing this complicated handling.?? Probably time to remove it in 11? Mandy [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-January/031041.html From sangheon.kim at oracle.com Wed Feb 14 21:45:04 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 14 Feb 2018 13:45:04 -0800 Subject: RFR(xs): 8193909: Obsolete(remove) Co-operative Memory Management (CMM) Message-ID: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> Hi all, Could I have some reviews for CMM removal? This is closed CR but some public codes also need small modifications. This CR is for removing stuff related to an Oracle JDK module/package. Changes are just removing CMM from lists or in a test to skip the testing logic. CR: https://bugs.openjdk.java.net/browse/JDK-8193909 Webrev: http://cr.openjdk.java.net/~sangheki/8193909/webrev.0 Testing: hs-tier1~5, jdk1~3, open/test/jdk:jdk_core Thanks, Sangheon From martinrb at google.com Wed Feb 14 21:54:04 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 14 Feb 2018 13:54:04 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> Message-ID: Indeed I had suppressed all memory of runFinalizersOnExit. (Sorry about that.) I support removing it in jdk11. Mandy, would you like to file the CSR? On Wed, Feb 14, 2018 at 1:15 PM, mandy chung wrote: > > > On 2/14/18 1:58 AM, Peter Levart wrote: > > > I take back this claim. Of course the the following race is possible: > > - Thread1: calls runAllFinalizers and takes a Finalizer from 'unprocessed' > list. > - Thread2: takee the same Finalizer instance from ReferenceQueue and calls > runFinalizer() > - Thread1: calls runFinalizer() with the same instance for the 2nd time > now. > > > runAllFinalizers is invoked during shutdown when > System.runFinalizersOnExit has been called. > > I have been wanting to remove System::runFinalizersOnExit [1] which is the > culprit of causing this complicated handling. Probably time to remove it > in 11? > > Mandy > > [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2015- > January/031041.html > From mandy.chung at oracle.com Wed Feb 14 21:59:23 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 14 Feb 2018 13:59:23 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> Message-ID: On 2/14/18 1:54 PM, Martin Buchholz wrote: > Indeed I had suppressed all memory of runFinalizersOnExit. (Sorry > about that.) > I support removing it in jdk11. Great. > Mandy, would you like to file the CSR? Yes I plan to do that for: ??? https://bugs.openjdk.java.net/browse/JDK-4240589 Mandy > > On Wed, Feb 14, 2018 at 1:15 PM, mandy chung > wrote: > > > > On 2/14/18 1:58 AM, Peter Levart wrote: >> >> I take back this claim. Of course the the following race is >> possible: >> >> - Thread1: calls runAllFinalizers and takes a Finalizer from >> 'unprocessed' list. >> - Thread2: takee the same Finalizer instance from ReferenceQueue >> and calls runFinalizer() >> - Thread1: calls runFinalizer() with the same instance for the >> 2nd time now. > > runAllFinalizers is invoked during shutdown when > System.runFinalizersOnExit has been called. > > I have been wanting to remove System::runFinalizersOnExit [1] > which is the culprit of causing this complicated handling.?? > Probably time to remove it in 11? > > Mandy > > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-January/031041.html > > > From yumin.qi at gmail.com Wed Feb 14 22:36:41 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 14 Feb 2018 14:36:41 -0800 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: <84b5c500-35be-fff2-787f-aad5eb613ffc@Oracle.com> References: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> <94168540-0cf6-8f9f-41bd-3f64ba41187f@oracle.com> <84b5c500-35be-fff2-787f-aad5eb613ffc@Oracle.com> Message-ID: HI, Roger and Brian Updated again in same link. I could not run jtreg so please have a look if the options are good for @run Thanks Yumin On Wed, Feb 14, 2018 at 11:13 AM, Roger Riggs wrote: > Hi, > > In the test, the message can be improved; though the exception should > never occur > it will be misleading if it does. > Instead: > "Changing property user.dir should have no effect on the > getCanonicalPath" > > It is cleaner to throw AssertionError for the test failure. > Throwing FileSystemException makes it look like it came from inside the > file system implementation. > > Regards, Roger > > > On 2/13/2018 6:45 PM, yumin qi wrote: > >> Alan, >> >> In fact, if property "user.dir" is cached, the problem will go away >> without any changes to native or UnixFileSystem.java, since it won't >> canonicalize the string supplied from user. The output will be >> getProperty("user.dir") + "/" + file. (The result is not as expected, >> user can not change the property, means user has to work around the >> problem >> to reach their targeted directory to do things like loading classes) >> >> Any change to "user.dir" in java program does not have any effect. >> For Windows: >> >> --- a/src/java.base/windows/classes/java/io/WinNTFileSystem.java Fri Feb >> 02 >> 10:32:59 2018 -0800 >> +++ b/src/java.base/windows/classes/java/io/WinNTFileSystem.java Tue Feb >> 13 >> 23:40:21 2018 +0000 >> @@ -43,12 +43,14 @@ >> private final char slash; >> private final char altSlash; >> private final char semicolon; >> + private final String userDir; >> >> public WinNTFileSystem() { >> Properties props = GetPropertyAction.privilegedGetProperties(); >> slash = props.getProperty("file.separator").charAt(0); >> semicolon = props.getProperty("path.separator").charAt(0); >> altSlash = (this.slash == '\\') ? '/' : '\\'; >> + userDir = props.getProperty("user.dir"); >> } >> >> private boolean isSlash(char c) { >> @@ -347,7 +349,7 @@ >> private String getUserPath() { >> /* For both compatibility and security, >> we must look this up every time */ >> - return normalize(System.getProperty("user.dir")); >> + return normalize(userDir); >> } >> >> private String getDrive(String path) { >> >> Does it need normalize(userDir) in getUserPath()? I think we need here. >> >> I will update the webrev and send for another round of review: the >> change >> in native will be reverted, and made changes to UnixFileSystem.java. >> >> >> Thanks >> Yumin >> >> >> On Mon, Feb 12, 2018 at 3:09 AM, Alan Bateman >> wrote: >> >> On 09/02/2018 18:01, Alan Bateman wrote: >>> >>> : >>>> >>>> I'll study the patch you have but I think we also need to create issues >>>> to get us to the point where changing this system property in a running >>>> VM >>>> doesn't impact running code. >>>> >>>> Looking at it again, I think we should change java.io.UnixFileSystem >>> (and >>> the Windows equivalent) to cache the value of user.dir to avoid difficult >>> to diagnose issues with bad code changing the value of this property in a >>> running VM. This should reduce the issue down to cases where user.dir is >>> changed on the command line (never supported either of course) to a value >>> that is not "/" but has trailing or duplicate slashes. >>> >>> When reduced down then the alternatives are to change the native >>> canonicalize method as you have done or alternatively do it once at >>> UnixFileSystem initialization time so that canonicalize does not have to >>> deal with this case. The former would require changing the description of >>> the function (it currently reads "The input path is assumed to contain no >>> duplicate slashes"), the latter avoids any changes to the native >>> implementation. >>> >>> -Alan >>> >>> >>> diff -r 0937e5f799df src/java.base/unix/classes/jav >>> a/io/UnixFileSystem.java >>> --- a/src/java.base/unix/classes/java/io/UnixFileSystem.java Sat Feb >>> 10 07:06:16 2018 -0500 >>> +++ b/src/java.base/unix/classes/java/io/UnixFileSystem.java Mon Feb >>> 12 10:49:40 2018 +0000 >>> @@ -34,12 +34,14 @@ >>> private final char slash; >>> private final char colon; >>> private final String javaHome; >>> + private final String userDir; >>> >>> public UnixFileSystem() { >>> Properties props = GetPropertyAction.privilegedGe >>> tProperties(); >>> slash = props.getProperty("file.separator").charAt(0); >>> colon = props.getProperty("path.separator").charAt(0); >>> javaHome = props.getProperty("java.home"); >>> + userDir = props.getProperty("user.dir"); >>> } >>> >>> >>> @@ -128,7 +130,11 @@ >>> >>> public String resolve(File f) { >>> if (isAbsolute(f)) return f.getPath(); >>> - return resolve(System.getProperty("user.dir"), f.getPath()); >>> + SecurityManager sm = System.getSecurityManager(); >>> + if (sm != null) { >>> + sm.checkPropertyAccess("user.dir"); >>> + } >>> + return resolve(userDir, f.getPath()); >>> } >>> >>> // Caches for canonicalization results to improve startup >>> performance. >>> >>> >>> > From david.holmes at oracle.com Wed Feb 14 22:40:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Feb 2018 08:40:02 +1000 Subject: RFR(xs): 8193909: Obsolete(remove) Co-operative Memory Management (CMM) In-Reply-To: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> References: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> Message-ID: That all seems trivially fine. Thanks, David On 15/02/2018 7:45 AM, sangheon.kim wrote: > Hi all, > > Could I have some reviews for CMM removal? > This is closed CR but some public codes also need small modifications. > This CR is for removing stuff related to an Oracle JDK module/package. > Changes are just removing CMM from lists or in a test to skip the > testing logic. > > CR: https://bugs.openjdk.java.net/browse/JDK-8193909 > Webrev: http://cr.openjdk.java.net/~sangheki/8193909/webrev.0 > Testing: hs-tier1~5, jdk1~3, open/test/jdk:jdk_core > > Thanks, > Sangheon > From brian.burkhalter at oracle.com Wed Feb 14 22:50:28 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 14 Feb 2018 14:50:28 -0800 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: References: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> <94168540-0cf6-8f9f-41bd-3f64ba41187f@oracle.com> <84b5c500-35be-fff2-787f-aad5eb613ffc@Oracle.com> Message-ID: Yumin, As for the test options you would need to make this change: --- a/test/jdk/java/io/File/ParseCanonicalPath.java +++ b/test/jdk/java/io/File/ParseCanonicalPath.java @@ -25,7 +25,7 @@ (os.family == "solaris") | (os.family == "aix") @bug 8194154 @summary Test parsing path with double slashes on unix like platforms. - @run -ea main ParseCanonicalPath + @run main/othervm -ea ParseCanonicalPath */ A better alternative might be to throw a RuntimeException instead of using assert. Thanks, Brian On Feb 14, 2018, at 2:36 PM, yumin qi wrote: > Updated again in same link. > I could not run jtreg so please have a look if the options are good for @run From jesper.wilhelmsson at oracle.com Wed Feb 14 22:53:21 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Wed, 14 Feb 2018 23:53:21 +0100 Subject: RFR(xs): 8193909: Obsolete(remove) Co-operative Memory Management (CMM) In-Reply-To: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> References: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> Message-ID: <71FFFDAB-B096-48F1-BC18-765F5D6EE734@oracle.com> Looks good! /Jesper > On 14 Feb 2018, at 22:45, sangheon.kim wrote: > > Hi all, > > Could I have some reviews for CMM removal? > This is closed CR but some public codes also need small modifications. This CR is for removing stuff related to an Oracle JDK module/package. > Changes are just removing CMM from lists or in a test to skip the testing logic. > > CR: https://bugs.openjdk.java.net/browse/JDK-8193909 > Webrev: http://cr.openjdk.java.net/~sangheki/8193909/webrev.0 > Testing: hs-tier1~5, jdk1~3, open/test/jdk:jdk_core > > Thanks, > Sangheon > From sangheon.kim at oracle.com Wed Feb 14 23:02:38 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 14 Feb 2018 15:02:38 -0800 Subject: RFR(xs): 8193909: Obsolete(remove) Co-operative Memory Management (CMM) In-Reply-To: <71FFFDAB-B096-48F1-BC18-765F5D6EE734@oracle.com> References: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> <71FFFDAB-B096-48F1-BC18-765F5D6EE734@oracle.com> Message-ID: <91fd2bf6-e4ac-e4fb-7c34-8332b78a3e27@oracle.com> Hi David and Jesper, Thank you for the review. Sangheon On 02/14/2018 02:53 PM, jesper.wilhelmsson at oracle.com wrote: > Looks good! > /Jesper > >> On 14 Feb 2018, at 22:45, sangheon.kim wrote: >> >> Hi all, >> >> Could I have some reviews for CMM removal? >> This is closed CR but some public codes also need small modifications. This CR is for removing stuff related to an Oracle JDK module/package. >> Changes are just removing CMM from lists or in a test to skip the testing logic. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8193909 >> Webrev: http://cr.openjdk.java.net/~sangheki/8193909/webrev.0 >> Testing: hs-tier1~5, jdk1~3, open/test/jdk:jdk_core >> >> Thanks, >> Sangheon >> From mandy.chung at oracle.com Wed Feb 14 23:45:50 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 14 Feb 2018 15:45:50 -0800 Subject: RFR(xs): 8193909: Obsolete(remove) Co-operative Memory Management (CMM) In-Reply-To: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> References: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> Message-ID: <07c86a73-1695-9080-57df-80ef15261221@oracle.com> +1 Mandy On 2/14/18 1:45 PM, sangheon.kim wrote: > Hi all, > > Could I have some reviews for CMM removal? > This is closed CR but some public codes also need small modifications. > This CR is for removing stuff related to an Oracle JDK module/package. > Changes are just removing CMM from lists or in a test to skip the > testing logic. > > CR: https://bugs.openjdk.java.net/browse/JDK-8193909 > Webrev: http://cr.openjdk.java.net/~sangheki/8193909/webrev.0 > Testing: hs-tier1~5, jdk1~3, open/test/jdk:jdk_core > > Thanks, > Sangheon > From yumin.qi at gmail.com Wed Feb 14 23:52:30 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 14 Feb 2018 15:52:30 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: Brian, Updated using RuntimeException, which will not need -ea so removed. http://cr.openjdk.java.net/~minqi/8194154/webrev1/ Thanks Yumin On Wed, Feb 14, 2018 at 11:17 AM, Brian Burkhalter < brian.burkhalter at oracle.com> wrote: > Hello, > > In the test you have > > /* @test requires (os.family == "linux") | (os.family == "mac") | (os.family == "solaris") | (os.family == "aix") > @bug 8194154 > @summary Test parsing path with double slashes on unix like platforms. > */ > > but I think you need to have the @requires annotation on a separate line > for example as: > > /* @test > * @requires (os.family == "linux") | (os.family == "mac?) > * | (os.family == "solaris") | (os.family == "aix") > * @bug 8194154 > * @summary Test parsing path with double slashes on unix like platforms. > */ > > Thanks, > > Brian > > On Feb 14, 2018, at 10:48 AM, yumin qi wrote: > > Thanks. Updated on same link > http://cr.openjdk.java.net/~minqi/8194154/webrev1/ > as your recommendation. > > > From sangheon.kim at oracle.com Wed Feb 14 23:58:21 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 14 Feb 2018 15:58:21 -0800 Subject: RFR(xs): 8193909: Obsolete(remove) Co-operative Memory Management (CMM) In-Reply-To: <07c86a73-1695-9080-57df-80ef15261221@oracle.com> References: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> <07c86a73-1695-9080-57df-80ef15261221@oracle.com> Message-ID: Hi Mandy, Thank you for the review! Sangheon On 02/14/2018 03:45 PM, mandy chung wrote: > +1 > > Mandy > > On 2/14/18 1:45 PM, sangheon.kim wrote: >> Hi all, >> >> Could I have some reviews for CMM removal? >> This is closed CR but some public codes also need small >> modifications. This CR is for removing stuff related to an Oracle JDK >> module/package. >> Changes are just removing CMM from lists or in a test to skip the >> testing logic. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8193909 >> Webrev: http://cr.openjdk.java.net/~sangheki/8193909/webrev.0 >> Testing: hs-tier1~5, jdk1~3, open/test/jdk:jdk_core >> >> Thanks, >> Sangheon >> > From stuart.marks at oracle.com Thu Feb 15 01:09:35 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 14 Feb 2018 17:09:35 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> Message-ID: <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> Hi Joe, Overall, looks good. Are there any tests of AbstractStringBuilder.compareTo() that exercise comparisons of the Latin1 vs UTF16 coders? A couple people have raised the issue of the SB's implementing Comparable but not overriding equals(). This is unusual but well-defined. I do think it deserves the "inconsistent with equals" treatment. Something like the following should be added to both SB's: ========== @apiNote {@code StringBuilder} implements {@code Comparable} but does not override {@link Object#equals equals}. Thus, the natural ordering of {@code StringBuilder} is inconsistent with equals. Care should be exercised if {@code StringBuilder} objects are used as keys in a {@code SortedMap} or elements in a {@code SortedSet}. See {@link Comparable}, {@link java.util.SortedMap SortedMap}, or {@link java.util.SortedSet SortedSet} for more information. ========== Respectively for StringBuffer. (Adjust markup to taste.) Thanks, s'marks On 2/8/18 4:47 PM, Joe Wang wrote: > Hi all, > > The CSR for the enhancement is now approved. Thanks Joe! > > The webrev has been updated accordingly. Please let me know if you have any > further comment on the implementation. > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Thanks, > Joe > > On 2/2/2018 12:46 PM, Joe Wang wrote: >> Thanks Jason. Will update that accordingly. >> >> Best, >> Joe >> >> On 2/2/2018 11:22 AM, Jason Mehrens wrote: >>> Joe, >>> >>> The identity check in CS.compare makes sense.? However, it won't be null >>> hostile if we call CS.compare(null, null) and that doesn't seem right. >>> Usually when writing comparator classes I end up with: >>> === >>> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >>> ???? return 0; >>> } >>> === >>> >>> Jason >>> ________________________________________ >>> From: core-libs-dev on behalf of Joe >>> Wang >>> Sent: Friday, February 2, 2018 1:01 PM >>> To: core-libs-dev >>> Subject: Re: RFR (JDK11) 8137326: Methods for comparing CharSequence, >>> StringBuilder, and StringBuffer >>> >>> Hi, >>> >>> Thanks all for comments and suggestions. I've updated the webrev. Please >>> review. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> Thanks, >>> Joe >>> >>> On 1/31/2018 9:31 PM, Joe Wang wrote: >>>> Hi Tagir, >>>> >>>> Thanks for the comment. I will consider adding that to the javadoc >>>> emphasizing that the comparison is performed from 0 to length() - 1 of >>>> the two sequences. >>>> >>>> Best, >>>> Joe >>>> >>>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>>> Hello! >>>>> >>>>> An AbstractStringBuilder#compareTo implementation is wrong. You cannot >>>>> simply compare the whole byte array. Here's the test-case: >>>>> >>>>> public class Test { >>>>> ??? public static void main(String[] args) { >>>>> ????? StringBuilder sb1 = new StringBuilder("test1"); >>>>> ????? StringBuilder sb2 = new StringBuilder("test2"); >>>>> ????? sb1.setLength(4); >>>>> ????? sb2.setLength(4); >>>>> ????? System.out.println(sb1.compareTo(sb2)); >>>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>>> ??? } >>>>> } >>>>> >>>>> We truncated the stringbuilders making their content equal, so >>>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares >>>>> the original content (before the truncation) as truncation, of course, >>>>> does not zero the truncated bytes, neither does it reallocate the >>>>> array (unless explicitly asked via trimToSize). >>>>> >>>>> With best regards, >>>>> Tagir Valeev. >>>>> >>>>> >>>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>>> StringBuffer. >>>>>> >>>>>> The Comparable implementations for StringBuilder/Buffer are similar >>>>>> to that >>>>>> of String, allowing comparison operations between two >>>>>> StringBuilders/Buffers, e.g. >>>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>>> method >>>>>> 'compare' is added instead of implementing the Comparable interface. >>>>>> This >>>>>> 'compare' method may take CharSequence implementations such as String, >>>>>> StringBuilder and StringBuffer, making it possible to perform >>>>>> comparison >>>>>> among them. The previous example for example is equivalent to >>>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>>> >>>>>> Tests for java.base have been independent from each other. The new >>>>>> tests are >>>>>> therefore created to have no dependency on each other or sharing any >>>>>> code. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>> >>>>>> Thanks, >>>>>> Joe >> > From david.holmes at oracle.com Thu Feb 15 01:59:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Feb 2018 11:59:14 +1000 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java In-Reply-To: References: Message-ID: <8e5891d5-65ac-baf8-3cdc-883a8dab4433@oracle.com> On 15/02/2018 12:13 AM, Adam Farley8 wrote: > Hi All, > > -- Short version -- > > Could a committer please take the fix for JDK-8190187 (full code included > in the bug) and: > > 1) Complete the CSR process for the new JNI Return code. > 2) Commit the changes that contain (a) the new return code, and (b) the > non-Hotspot code that handles the new code. As I stated here: http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-October/049597.html "I can file a bug for this, but it will take some work to see how to fit this into the existing specifications and file CSR requests for those changes. This won't make 18.3 (or whatever it gets called). You will need a "champion" to help flesh this out in full and work with you on those CSR requests. I can't volunteer to do that at this time." Nobody has offered to champion this. Sorry. David ----- > Backporting would be appreciated, but is optional. > > Also optional: Commit the jdwp code that serves as an example for how this > code should be used. CSR again, unfortunately. > > > -- Long Version -- > > We have a bug in OpenJDK where if you pass an info-only option (like > -agentlib:jdwp=help) in through the JNI interface, it can exit your code > with RC 0. > > I think this is a bug because if you planned to do anything after starting > a Java VM, you have to do it in an exit hook. > > If an info-only option is used, exit(0) should not happen. Instead, the > user's code should be told the VM cannot be used. > > Bug Link: https://bugs.openjdk.java.net/browse/JDK-8190187 > > I have proposed the creation of a new JNI return code indicating a "silent > exit", where the vm encountered no error, but that it is not in a fit > state to be used. > > See the link for an implementation of this, along with a handy test. > > In short, if you agree that this is a problem, we need a champion to: > > 1) Complete the CSR process for the new JNI Return code. > 2) Commit the changes that contain (a) the new return code, and (b) the > non-Hotspot code that handles the new code. > > Optionally, if you want to commit the code containing an example of the > fix, you will need to: > > 3) Commit the Hotspot code that channels the new return code. > 4) Complete the CSR process for the jdwp behaviour change. > 5) Commit the jdwp code. > > Note that all of this code has *already been written*. This should be an > easy commit once the CSR is done with. > > Best Regards > > Adam Farley > > > > Best Regards > > Adam Farley > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > From martinrb at google.com Thu Feb 15 05:47:24 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 14 Feb 2018 21:47:24 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> Message-ID: Embarrassed by my failure to take runFinalizersOnExit into account, I tried to redo the patch to be actually correct. http://cr.openjdk.java.net/~martin/webrevs/jdk/Finalizer-data-race/ even though we may succeed in making runFinalizersOnExit go away, and I'm not planning to submit any time soon. The runFinalizersOnExit case needs different mechanics for removing elements from "unfinalized". The simple invariant is that you need to run the finalizer whenever an element is unlinked from unfinalized. From peter.levart at gmail.com Thu Feb 15 07:47:14 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 15 Feb 2018 08:47:14 +0100 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> Message-ID: Hi Martin, On 02/15/2018 06:47 AM, Martin Buchholz wrote: > Embarrassed by my failure to take runFinalizersOnExit into account, I tried > to redo the patch to be actually correct. I think that your patch was correct the first time too, but in view of future removing the need for "pollFirst" from unfinalized list, the revised patch separates this logic entirely into the to-be-removed method so it is preferable. > http://cr.openjdk.java.net/~martin/webrevs/jdk/Finalizer-data-race/ > even though we may succeed in making runFinalizersOnExit go away, and I'm > not planning to submit any time soon. > The runFinalizersOnExit case needs different mechanics for removing > elements from "unfinalized". > The simple invariant is that you need to run the finalizer whenever an > element is unlinked from unfinalized. > ...and the check for "already been finalized" needs to be performed only in deregisterAndRunFinalizer as you did in new patch. Once runAllFinalizers is removed, this check and marking code could be removed too. Although not strictly necessary for correctness, for the time being, it would be nice to be consistent in marking the Finalizer object "already finalized" in both places. Either set both next and prev to this or next to this and prev to null. Regards, Peter From xueming.shen at oracle.com Thu Feb 15 07:49:29 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 14 Feb 2018 23:49:29 -0800 Subject: RFR: JDK-8197988, mach5 T2 test javax/net/ssl/interop/ClientHelloChromeInterOp.java failed after JDK-8164278 Message-ID: <5A853B89.9080302@oracle.com> Hi, Please help review the change for JDK-819798 issue: https://bugs.openjdk.java.net/browse/JDK-8197988 webrev: http://cr.openjdk.java.net/~sherman/8197988/webrev The change for JDK-8164278 caused a decoding regression. A Tier2 test case, javax/net/ssl/interop/ClientHelloChromeInterOp.java, that uses mime-base64-decoder failed. The "condition" that triggers the decoding to go the fast path is incorrect in the code pushed for JDK-819798. The intent here is to only start/try the fast path when we are at the beginning of a 4-byte legal base64 unit. The correct/appropriate check is to see if shiftto == 18, which means the decoder is still at the beginning of 4-byte unit. ( bits == 0 is a false indicator, as bits might include decoded byte value of 0). The fix is a relatively easy/one line change. Thanks, Sherman From Alan.Bateman at oracle.com Thu Feb 15 08:15:16 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Feb 2018 08:15:16 +0000 Subject: RFR: JDK-8197988, mach5 T2 test javax/net/ssl/interop/ClientHelloChromeInterOp.java failed after JDK-8164278 In-Reply-To: <5A853B89.9080302@oracle.com> References: <5A853B89.9080302@oracle.com> Message-ID: <5a9a058c-9113-363a-cd28-118214a9b7bd@oracle.com> On 15/02/2018 07:49, Xueming Shen wrote: > Hi, > > Please help review the change for? JDK-819798 > > issue: https://bugs.openjdk.java.net/browse/JDK-8197988 > webrev: http://cr.openjdk.java.net/~sherman/8197988/webrev > > The change for JDK-8164278 caused a decoding regression. A Tier2 > test case,? javax/net/ssl/interop/ClientHelloChromeInterOp.java, that > uses mime-base64-decoder failed. > > The "condition" that triggers the decoding to go the fast path is > incorrect in > the code pushed for JDK-819798. > > The intent here is to only start/try the fast path when we are at the > beginning > of a 4-byte legal base64 unit. The correct/appropriate check is to see if > shiftto == 18, which means the decoder is still at the beginning of > 4-byte > unit. ( bits == 0 is a false indicator, as bits might include decoded > byte value of > 0). The fix is a relatively easy/one line change. This looks okay. Should we create another issue to add more tests in this area to avoid the SSL code finding issues in this area again. -Alan From xueming.shen at oracle.com Thu Feb 15 08:22:36 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 15 Feb 2018 00:22:36 -0800 Subject: RFR: JDK-8197988, mach5 T2 test javax/net/ssl/interop/ClientHelloChromeInterOp.java failed after JDK-8164278 In-Reply-To: <5a9a058c-9113-363a-cd28-118214a9b7bd@oracle.com> References: <5A853B89.9080302@oracle.com> <5a9a058c-9113-363a-cd28-118214a9b7bd@oracle.com> Message-ID: <5A85434C.3080907@oracle.com> On 2/15/18, 12:15 AM, Alan Bateman wrote: > On 15/02/2018 07:49, Xueming Shen wrote: >> Hi, >> >> Please help review the change for JDK-819798 >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8197988 >> webrev: http://cr.openjdk.java.net/~sherman/8197988/webrev >> >> The change for JDK-8164278 caused a decoding regression. A Tier2 >> test case, javax/net/ssl/interop/ClientHelloChromeInterOp.java, that >> uses mime-base64-decoder failed. >> >> The "condition" that triggers the decoding to go the fast path is >> incorrect in >> the code pushed for JDK-819798. >> >> The intent here is to only start/try the fast path when we are at the >> beginning >> of a 4-byte legal base64 unit. The correct/appropriate check is to >> see if >> shiftto == 18, which means the decoder is still at the beginning of >> 4-byte >> unit. ( bits == 0 is a false indicator, as bits might include decoded >> byte value of >> 0). The fix is a relatively easy/one line change. > This looks okay. Should we create another issue to add more tests in > this area to avoid the SSL code finding issues in this area again. > Yes, I will do that tomorrow. Just need to get the mach5 going first tonight. thanks, Sherman From Alan.Bateman at oracle.com Thu Feb 15 08:23:07 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Feb 2018 08:23:07 +0000 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: Message-ID: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> On 14/02/2018 23:52, yumin qi wrote: > Brian, > > ? Updated using RuntimeException, which will not need -ea so removed. > http://cr.openjdk.java.net/~minqi/8194154/webrev1/ > > Can you explain why resolve has been changed to call normalize parent+child? I can't help thinking there is something else going on if this change is needed. I see the mails about the test so I'll skip that except to say that it should not need "require". The test should be able to run on all platforms. -Alan From adam.farley at uk.ibm.com Thu Feb 15 09:37:20 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Thu, 15 Feb 2018 09:37:20 +0000 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java In-Reply-To: <8e5891d5-65ac-baf8-3cdc-883a8dab4433@oracle.com> References: <8e5891d5-65ac-baf8-3cdc-883a8dab4433@oracle.com> Message-ID: On 15/02/2018 12:13 AM, Adam Farley8 wrote: >> Hi All, >> >> -- Short version -- >> >> Could a committer please take the fix for JDK-8190187 (full code included >> in the bug) and: >> >> 1) Complete the CSR process for the new JNI Return code. >> 2) Commit the changes that contain (a) the new return code, and (b) the >> non-Hotspot code that handles the new code. > > As I stated here: > > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_core-2Dlibs-2Ddev_2017-2DOctober_049597.html&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=P5m8KWUXJf-CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=OUKJytaKUZmaM4zCoF2GSmIrYLPEE4cSP1LkBgAIduo&s=DIuL0lNnE0xXAqlADCHi8uv2u6kcKcOWbob0WHHArSU&e= > > "I can file a bug for this, but it will take some work to see how to fit > this into the existing specifications and file CSR requests for those > changes. This won't make 18.3 (or whatever it gets called). You will > need a "champion" to help flesh this out in full and work with you on > those CSR requests. I can't volunteer to do that at this time." > > Nobody has offered to champion this. Sorry. > > David > ----- Yup, I remember you saying that David. I plan to work on the issues you raised in your previous email, and continue to post messages in these lists until someone who has free cycles spots my emails and agrees to commit the code. It'll happen sooner or later. :) - Adam > >> Backporting would be appreciated, but is optional. >> >> Also optional: Commit the jdwp code that serves as an example for how this >> code should be used. CSR again, unfortunately. >> >> >> -- Long Version -- >> >> We have a bug in OpenJDK where if you pass an info-only option (like >> -agentlib:jdwp=help) in through the JNI interface, it can exit your code >> with RC 0. >> >> I think this is a bug because if you planned to do anything after starting >> a Java VM, you have to do it in an exit hook. >> >> If an info-only option is used, exit(0) should not happen. Instead, the >> user's code should be told the VM cannot be used. >> >> Bug Link: https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8190187&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=P5m8KWUXJf-CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=OUKJytaKUZmaM4zCoF2GSmIrYLPEE4cSP1LkBgAIduo&s=P_S35LF3YxaxKQWY7-svCzm_WvvcUU0nSx8QP9dH9O0&e= >> >> I have proposed the creation of a new JNI return code indicating a "silent >> exit", where the vm encountered no error, but that it is not in a fit >> state to be used. >> >> See the link for an implementation of this, along with a handy test. >> >> In short, if you agree that this is a problem, we need a champion to: >> >> 1) Complete the CSR process for the new JNI Return code. >> 2) Commit the changes that contain (a) the new return code, and (b) the >> non-Hotspot code that handles the new code. >> >> Optionally, if you want to commit the code containing an example of the >> fix, you will need to: >> >> 3) Commit the Hotspot code that channels the new return code. >> 4) Complete the CSR process for the jdwp behaviour change. >> 5) Commit the jdwp code. >> >> Note that all of this code has *already been written*. This should be an >> easy commit once the CSR is done with. >> >> Best Regards >> >> Adam Farley >> >> >> >> Best Regards >> >> Adam Farley >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >> > > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From thomas.schatzl at oracle.com Thu Feb 15 11:32:48 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 15 Feb 2018 12:32:48 +0100 Subject: RFR(xs): 8193909: Obsolete(remove) Co-operative Memory Management (CMM) In-Reply-To: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> References: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> Message-ID: <1518694368.4282.26.camel@oracle.com> Hi, On Wed, 2018-02-14 at 13:45 -0800, sangheon.kim wrote: > Hi all, > > Could I have some reviews for CMM removal? > This is closed CR but some public codes also need small > modifications. > This CR is for removing stuff related to an Oracle JDK > module/package. > Changes are just removing CMM from lists or in a test to skip the > testing logic. > > CR: https://bugs.openjdk.java.net/browse/JDK-8193909 > Webrev: http://cr.openjdk.java.net/~sangheki/8193909/webrev.0 > Testing: hs-tier1~5, jdk1~3, open/test/jdk:jdk_core > looks good. Thomas From Alan.Bateman at oracle.com Thu Feb 15 11:42:21 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Feb 2018 11:42:21 +0000 Subject: 8193818 : Remove unused single_step field from java.lang.Thread In-Reply-To: References: Message-ID: On 19/12/2017 11:06, Alan Bateman wrote: > I've been going through the fields in java.lang.Thread and I'm > wondering if this field can be removed: > > ??? /* Whether or not to single_step this thread. */ > ??? private boolean???? single_step; > > This field was used in the original Classic VM (pre-OpenJDK history). > It doesn't appear to be used in the HotSpot VM. > > Does anyone know of any reason to keep it? Are there other VMs using > it by any chance? No one screamed so I'd like to go ahead and remove this field. I've created JDK-8193818 to track it, the change (below) is trivial. -Alan --- a/src/java.base/share/classes/java/lang/Thread.java +++ b/src/java.base/share/classes/java/lang/Thread.java @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 1994, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1994, 2018, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -150,9 +150,6 @@ ???? private Thread???????? threadQ; ???? private long?????????? eetop; -??? /* Whether or not to single_step this thread. */ -??? private boolean???? single_step; - ???? /* Whether or not the thread is a daemon thread. */ ???? private boolean???? daemon = false; From lance.andersen at oracle.com Thu Feb 15 11:48:14 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 15 Feb 2018 06:48:14 -0500 Subject: 8193818 : Remove unused single_step field from java.lang.Thread In-Reply-To: References: Message-ID: +1 > On Feb 15, 2018, at 6:42 AM, Alan Bateman wrote: > > On 19/12/2017 11:06, Alan Bateman wrote: >> I've been going through the fields in java.lang.Thread and I'm wondering if this field can be removed: >> >> /* Whether or not to single_step this thread. */ >> private boolean single_step; >> >> This field was used in the original Classic VM (pre-OpenJDK history). It doesn't appear to be used in the HotSpot VM. >> >> Does anyone know of any reason to keep it? Are there other VMs using it by any chance? > No one screamed so I'd like to go ahead and remove this field. I've created JDK-8193818 to track it, the change (below) is trivial. > > -Alan > > > --- a/src/java.base/share/classes/java/lang/Thread.java > +++ b/src/java.base/share/classes/java/lang/Thread.java > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 1994, 2017, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 1994, 2018, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -150,9 +150,6 @@ > private Thread threadQ; > private long eetop; > > - /* Whether or not to single_step this thread. */ > - private boolean single_step; > - > /* Whether or not the thread is a daemon thread. */ > private boolean daemon = false; Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From david.holmes at oracle.com Thu Feb 15 12:17:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Feb 2018 22:17:51 +1000 Subject: 8193818 : Remove unused single_step field from java.lang.Thread In-Reply-To: References: Message-ID: <89ff2a93-cd45-e69b-5908-c9d7994cdeb3@oracle.com> On 15/02/2018 9:42 PM, Alan Bateman wrote: > On 19/12/2017 11:06, Alan Bateman wrote: >> I've been going through the fields in java.lang.Thread and I'm >> wondering if this field can be removed: >> >> ??? /* Whether or not to single_step this thread. */ >> ??? private boolean???? single_step; >> >> This field was used in the original Classic VM (pre-OpenJDK history). >> It doesn't appear to be used in the HotSpot VM. >> >> Does anyone know of any reason to keep it? Are there other VMs using >> it by any chance? > No one screamed so I'd like to go ahead and remove this field. I've > created JDK-8193818 to track it, the change (below) is trivial. Looks good to me :) Thanks, David > -Alan > > > --- a/src/java.base/share/classes/java/lang/Thread.java > +++ b/src/java.base/share/classes/java/lang/Thread.java > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 1994, 2017, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 1994, 2018, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -150,9 +150,6 @@ > ???? private Thread???????? threadQ; > ???? private long?????????? eetop; > > -??? /* Whether or not to single_step this thread. */ > -??? private boolean???? single_step; > - > ???? /* Whether or not the thread is a daemon thread. */ > ???? private boolean???? daemon = false; From james.laskey at oracle.com Thu Feb 15 13:02:06 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 15 Feb 2018 09:02:06 -0400 Subject: RFR: 8196959: NullPointerException in discovery003.java In-Reply-To: References: Message-ID: <70EFBEDA-B215-4044-85FC-A2C34176FB89@oracle.com> +1 > On Feb 14, 2018, at 1:24 AM, Srinivas Dama wrote: > > Jim, > > May I have second review for this thread in core-libs-dev group. > > Regards, > Srinivas > > ----- Forwarded Message ----- > From: sundararajan.athijegannathan at oracle.com > To: srinivas.dama at oracle.com > Cc: core-libs-dev at openjdk.java.net > Sent: Monday, February 12, 2018 6:25:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: Re: RFR: 8196959: NullPointerException in discovery003.java > > Looks good. > > -Sundar > > On 12/02/18, 5:31 PM, Srinivas Dama wrote: >> Hi, >> >> Please review http://cr.openjdk.java.net/~sdama/8196959/webrev.00/ >> for https://bugs.openjdk.java.net/browse/JDK-8196959 >> >> This is small modification of fix for https://bugs.openjdk.java.net/browse/JDK-8011697. >> Fix is to handle the corner case where the engineName is null. >> >> >> Regards, >> Srinivas From robin.westberg at oracle.com Thu Feb 15 13:35:23 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Thu, 15 Feb 2018 14:35:23 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> Message-ID: <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Hi again, Here?s the (hopefully) final version: Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ Best regards, Robin > On 14 Feb 2018, at 16:15, Robin Westberg wrote: > > Hi Roger, > >> On 13 Feb 2018, at 16:17, Roger Riggs wrote: >> >> Hi Robin, >> >> It looks like the status argument to BeforeHalt is discarded in JVM_BeforeHalt >> and is not inserted into the event. >> That suggests it should be removed all the way back to Shutdown.beforeHalt. > > You are right, my thinking was that the interface wouldn?t need to be changed if we decided to revisit the event in the future and add the status code. But that is perhaps unlikely, so can certainly remove the argument for now. > > Best regards, > Robin > >> >> Roger >> >> >> >> On 2/13/2018 9:59 AM, Robin Westberg wrote: >>> Hi Alan, >>> >>>> On 12 Feb 2018, at 09:02, Alan Bateman wrote: >>>> >>>> >>>> >>>> On 12/02/2018 07:07, David Holmes wrote: >>>>>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>>>>> >>>>>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>>> This looks much cleaner/neater to me - thanks. >>>>> >>>> The updates to Runtime/Shutdown seems okay. >>> Thanks for reviewing! >>> >>> Best regards, >>> Robin >>> >>>> -Alan >> > From Alan.Bateman at oracle.com Thu Feb 15 14:00:59 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Feb 2018 14:00:59 +0000 Subject: 8193819: Error message when module does not have a ModuleMainClass attribute is confusing Message-ID: There's a typo in the launcher resource used for the error message when `java -m` fails because the module doesn't have a main class. The name of the attribute is "ModuleMainClass", not "MainClass". Trivial change. -Alan diff --git a/src/java.base/share/classes/sun/launcher/resources/launcher.properties b/src/java.base/share/classes/sun/launcher/resources/launcher.properties --- a/src/java.base/share/classes/sun/launcher/resources/launcher.properties +++ b/src/java.base/share/classes/sun/launcher/resources/launcher.properties @@ -220,7 +220,7 @@ ???? Error: The JavaFX launchApplication method has the wrong signature, it\n\ ???? must be declared static and return a value of type void ?java.launcher.module.error1=\ -??? module {0} does not have a MainClass attribute, use -m / +??? module {0} does not have a ModuleMainClass attribute, use -m / ?java.launcher.module.error2=\ ???? Error: Could not find or load main class {0} in module {1} ?java.launcher.module.error3=\ From sundararajan.athijegannathan at oracle.com Thu Feb 15 14:14:23 2018 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 15 Feb 2018 19:44:23 +0530 Subject: 8193819: Error message when module does not have a ModuleMainClass attribute is confusing In-Reply-To: References: Message-ID: <5A8595BF.8030000@oracle.com> +1 -Sundar On 15/02/18, 7:30 PM, Alan Bateman wrote: > There's a typo in the launcher resource used for the error message > when `java -m` fails because the module doesn't have a main class. The > name of the attribute is "ModuleMainClass", not "MainClass". Trivial > change. > > -Alan > > > diff --git > a/src/java.base/share/classes/sun/launcher/resources/launcher.properties > b/src/java.base/share/classes/sun/launcher/resources/launcher.properties > --- > a/src/java.base/share/classes/sun/launcher/resources/launcher.properties > +++ > b/src/java.base/share/classes/sun/launcher/resources/launcher.properties > @@ -220,7 +220,7 @@ > Error: The JavaFX launchApplication method has the wrong > signature, it\n\ > must be declared static and return a value of type void > java.launcher.module.error1=\ > - module {0} does not have a MainClass attribute, use -m > / > + module {0} does not have a ModuleMainClass attribute, use -m > / > java.launcher.module.error2=\ > Error: Could not find or load main class {0} in module {1} > java.launcher.module.error3=\ From sangheon.kim at oracle.com Thu Feb 15 15:00:17 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Thu, 15 Feb 2018 07:00:17 -0800 Subject: RFR(xs): 8193909: Obsolete(remove) Co-operative Memory Management (CMM) In-Reply-To: <1518694368.4282.26.camel@oracle.com> References: <1554d7d4-df1f-5254-9014-f4f1d0c2aac6@oracle.com> <1518694368.4282.26.camel@oracle.com> Message-ID: <10e5b37c-fdcc-3666-5934-c301b4d18fa0@oracle.com> Hi Thomas, On 02/15/2018 03:32 AM, Thomas Schatzl wrote: > Hi, > > On Wed, 2018-02-14 at 13:45 -0800, sangheon.kim wrote: >> Hi all, >> >> Could I have some reviews for CMM removal? >> This is closed CR but some public codes also need small >> modifications. >> This CR is for removing stuff related to an Oracle JDK >> module/package. >> Changes are just removing CMM from lists or in a test to skip the >> testing logic. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8193909 >> Webrev: http://cr.openjdk.java.net/~sangheki/8193909/webrev.0 >> Testing: hs-tier1~5, jdk1~3, open/test/jdk:jdk_core >> > looks good. Thank you for your review! Sangheon > > Thomas From Roger.Riggs at Oracle.com Thu Feb 15 16:00:57 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 15 Feb 2018 11:00:57 -0500 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Message-ID: <07ea69ac-7b62-992f-5275-97c990906a2b@Oracle.com> Hi Robin, Looks fine to me. (How is this tested?, Normal, exceptional, etc.) Thanks, Roger On 2/15/2018 8:35 AM, Robin Westberg wrote: > Hi again, > > Here?s the (hopefully) final version: > > Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ > > Incremental: > http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ > > > Best regards, > Robin > >> On 14 Feb 2018, at 16:15, Robin Westberg > > wrote: >> >> Hi Roger, >> >>> On 13 Feb 2018, at 16:17, Roger Riggs >> > wrote: >>> >>> Hi Robin, >>> >>> It looks like the status argument to BeforeHalt is discarded in >>> JVM_BeforeHalt >>> and is not inserted into the event. >>> That suggests it should be removed all the way back to >>> Shutdown.beforeHalt. >> >> You are right, my thinking was that the interface wouldn?t need to be >> changed if we decided to revisit the event in the future and add the >> status code. But that is perhaps unlikely, so can certainly remove >> the argument for now. >> >> Best regards, >> Robin >> >>> >>> Roger >>> >>> >>> >>> On 2/13/2018 9:59 AM, Robin Westberg wrote: >>>> Hi Alan, >>>> >>>>> On 12 Feb 2018, at 09:02, Alan Bateman >>>> > wrote: >>>>> >>>>> >>>>> >>>>> On 12/02/2018 07:07, David Holmes wrote: >>>>>>> Okay, I?ve removed the code related to the status field, >>>>>>> certainly makes the change a bit less intrusive. >>>>>>> >>>>>>> Updated webrev: >>>>>>> http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>>>> >>>>>>> Incremental: >>>>>>> http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>>>>> >>>>>> This looks much cleaner/neater to me - thanks. >>>>>> >>>>> The updates to Runtime/Shutdown seems okay. >>>> Thanks for reviewing! >>>> >>>> Best regards, >>>> Robin >>>> >>>>> -Alan >>> >> > From adam.farley at uk.ibm.com Thu Feb 15 17:02:39 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Thu, 15 Feb 2018 17:02:39 +0000 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java In-Reply-To: References: Message-ID: On 14/02/2018 14:13, Adam Farley8 wrote: >> Hi All, >> >> -- Short version -- >> >> Could a committer please take the fix for JDK-8190187 (full code included >> in the bug) and: >> >> 1) Complete the CSR process for the new JNI Return code. >> 2) Commit the changes that contain (a) the new return code, and (b) the >> non-Hotspot code that handles the new code. > The patches attached to the JIRA issue are missing the changes to the > JVM TI spec (jvmti.xml). I'm not seeing the JNI return codes in that file. Are you after one of those dated updates near the bottom? e.g. ``` Added JNI_SILENT_EXIT return code for early exits without errors. java.c's ParseArguments function now sets pret value to 2 for a NULL pwhat. This allows us to clearly identify when no class was set, but no other error has occurred. This is undone in java.c's ContinueInNewThread method, so the surface behaviour does not change. ``` > There is also text to be written for the JNI spec if this proposal goes > ahead. I assume you mean the "RETURNS" section of the JNI_CreateJavaVM bit on the invocation.html web page. Something like this? ``` RETURNS: Returns JNI_OK on success; returns a suitable JNI error code (a negative number) on failure. The sole exception is a silent exit, which returns JNI error code JNI_SILENT_EXIT. This indicates that the VM cannot be used, but that this is the intended behaviour for the arguments used. E.g. -Xlog:help (which prints help output and then destroys the VM) ``` > > I don't agree that the launcher should be looking for > "-agentlib:jdwp=help" in the command line as it's just one of many ways > that the debugger agent might be started (e.g. -Xrunjdwp:, > _JAVA_TOOLS_OPTIONS, ...). We can avoid that by finding a way around this line in ContinueInNewThread (java.c): ``` return (ret != 0) ? ret : rslt; ``` I have devised a means to do this, as outlined in the jvmti.xml change above. I put the changes into a recent clone of jdk/jdk, and attached the hg diff, along with an improved test. If you have Author or Committer privileges, could you add the files to the bug please? > >> Backporting would be appreciated, but is optional. > I don't think these changes are appropriate to backport as they include > specification changes/ > >-Alan This is fair. - Adam Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From martinrb at google.com Thu Feb 15 17:05:22 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 15 Feb 2018 09:05:22 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> Message-ID: On Wed, Feb 14, 2018 at 11:47 PM, Peter Levart wrote: > > Although not strictly necessary for correctness, for the time being, it > would be nice to be consistent in marking the Finalizer object "already > finalized" in both places. Either set both next and prev to this or next to > this and prev to null. > > OK, now only next is ever self-linked. this.prev = null; this.next = this; // mark as finalized From bob.vandette at oracle.com Thu Feb 15 17:07:38 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 15 Feb 2018 12:07:38 -0500 Subject: JEP [DRAFT]: Container aware Java Message-ID: <9B0DF35E-0019-46DE-A6C1-AE4DE2416844@oracle.com> I?d like to re-propose the following JEP that will enhance the Java runtime to be more container aware. This will add an Internal Java API that will provide container specific statistics. Some of the initial goals of the previous JEP proposal has been integrated into JDK 10 under an RFE (JDK-8146115). This JEP is now focused on providing a Java API that exports Container runtime configuration and metrics. Since the scope of this JEP have changed, I?m re-submitting it for comment and endorsement. JEP Issue: https://bugs.openjdk.java.net/browse/JDK-8182070 Here?s a Text dump of the JEP contents for your convenience: Summary ------- Container aware Java runtime Goals ----- Provide an internal API that can be used to extract container specific configuration and runtime statistics. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux cgroups technology so that we will be able to easily support other container technologies running on Linux in addition to Docker. Non-Goals --------- It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. Success Metrics --------------- Success will be measured by the improvement in information that will be available to tools which visualize resource usage of containers that are running Java processes. Motivation ---------- Container technology is becoming more and more prevalent in Cloud based applications. The Cloud Serverless application programming model motivates developers to split large monolithic applications into 100s of smaller pieces each running in thier own container. This move increases the importance of the observability of each running container process. Adding the proposed set of APIs will allow more details related to each container process to be made available to external tools thereby improving the observability. Description ----------- This enhancement will be made up of the following work items: A. Detecting if Java is running in a container. The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. A new API will be made available for this purpose. B. Exposing container resource limits, configuration and runtime statistics. There are several configuration options and limits that can be imposed upon a running container. Not all of these are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that the process has been allocated but there are other options that we might want to base runtime decisions on. In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. The goal is to provide this information in addition to the configuration data. I propose adding a new jdk.internal.Platform class that will allow access to this information. Here are some of the types of configuration and consumption statistics that would be made available: isContainerized Memory Limit Total Memory Limit Soft Memory Limit Max Memory Usage Current Memory Usage Maximum Kernel Memory CPU Shares CPU Period CPU Quota Number of CPUs CPU Sets CPU Set Memory Nodes CPU Usage CPU Usage Per CPU Block I/O Weight Block I/O Device Weight Device I/O Read Rate Device I/O Write Rate OOM Kill Enabled OOM Score Adjustment Memory Swappiness Shared Memory Size Alternatives ------------ There are a few existing tools available to extract some of the same container statistics. These tools could be used instead. The benefit of providing a core Java internal API is that this information can be expose by current Java serviceability tools such as JMX and JFR along side other JVM specific information. Testing ------- Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. Risks and Assumptions --------------------- Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. Other alternative container technologies based on hypervisors are being developed that could replace the use of cgroups for container isloation. Dependencies ----------- None at this time. From james.laskey at oracle.com Thu Feb 15 17:20:42 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 15 Feb 2018 13:20:42 -0400 Subject: RFR: 8197594 - String and character repeat Message-ID: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> This is a pre-CSR code review [1] for String repeat methods (Enhancement). The proposal is to introduce four new methods; 1. public String repeat(final int count) 2. public static String repeat(final char ch, final int count) 3. public static String repeat(final int codepoint, final int count) 4. public static String repeat(final CharSequence seq, final int count) For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). 3 and 4 convert to String before calling 1. Performance runs with jmh (results as comment in [2]) show that these methods are significantly faster that StringBuilder equivalents. - fewer memory allocations - fewer char to byte array conversions - faster pyramid replication vs O(N) copying I left StringBuilder out of scope. It falls under the category of Appendables#append with repeat. A much bigger project. All comments welcome. Especially around the need for convenience methods, the JavaDoc content and expanding the tests. ? Jim [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 From Randy.Crihfield at Oracle.com Thu Feb 15 17:25:01 2018 From: Randy.Crihfield at Oracle.com (Randy Crihfield) Date: Thu, 15 Feb 2018 12:25:01 -0500 Subject: RFR: 8193660 Check SOURCE line in "release" file for closedjdk Message-ID: <5A85C26D.7010800@Oracle.com> I need to revise the resource file test created for JDK-8192837 The new bug is https://bugs.openjdk.java.net/browse/JDK-8193660 The webrev is located at http://cr.openjdk.java.net/~shurailine/8193660/webrev.00/ Any comments/suggestions are welcome, also I will need a sponsor for it at the end? Randy From yumin.qi at gmail.com Thu Feb 15 17:25:00 2018 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 15 Feb 2018 09:25:00 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> Message-ID: Alan, The real reason is if we do not change on resolve, the string passed into native canonicalize will be like "//./a/" which is not expected in native part. In native part, we resume that no more "//" in the path string (already normalized in java). We can fix this by either: 1) in java part, like the webrev here or 2) in native canonicalize_md.c as webrev0. If the test run on Windows, should it run in unix like shell on Windows? The separator "/" is not for Windows. Thanks Yumin On Thu, Feb 15, 2018 at 12:23 AM, Alan Bateman wrote: > On 14/02/2018 23:52, yumin qi wrote: > >> Brian, >> >> Updated using RuntimeException, which will not need -ea so removed. >> http://cr.openjdk.java.net/~minqi/8194154/webrev1/ < >> http://cr.openjdk.java.net/%7Eminqi/8194154/webrev1/> >> >> Can you explain why resolve has been changed to call normalize > parent+child? I can't help thinking there is something else going on if > this change is needed. > > I see the mails about the test so I'll skip that except to say that it > should not need "require". The test should be able to run on all platforms. > > -Alan > > From brian.goetz at oracle.com Thu Feb 15 17:31:04 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 15 Feb 2018 09:31:04 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> Message-ID: I suggest merging 1 and 4 by making it an instance method on CS with a default in CS and an override on string and string builder? Sent from my MacBook Wheel > On Feb 15, 2018, at 9:20 AM, Jim Laskey wrote: > > This is a pre-CSR code review [1] for String repeat methods (Enhancement). > > The proposal is to introduce four new methods; > > 1. public String repeat(final int count) > 2. public static String repeat(final char ch, final int count) > 3. public static String repeat(final int codepoint, final int count) > 4. public static String repeat(final CharSequence seq, final int count) > > For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. > In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). > 3 and 4 convert to String before calling 1. > > Performance runs with jmh (results as comment in [2]) show that these > methods are significantly faster that StringBuilder equivalents. > - fewer memory allocations > - fewer char to byte array conversions > - faster pyramid replication vs O(N) copying > > I left StringBuilder out of scope. It falls under the category of > Appendables#append with repeat. A much bigger project. > > All comments welcome. Especially around the need for convenience > methods, the JavaDoc content and expanding the tests. > > ? Jim > > [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 > [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 > From james.laskey at oracle.com Thu Feb 15 17:34:19 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 15 Feb 2018 13:34:19 -0400 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> Message-ID: Very reasonable approach. > On Feb 15, 2018, at 1:31 PM, Brian Goetz wrote: > > I suggest merging 1 and 4 by making it an instance method on CS with a default in CS and an override on string and string builder? > > Sent from my MacBook Wheel > >> On Feb 15, 2018, at 9:20 AM, Jim Laskey wrote: >> >> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >> >> The proposal is to introduce four new methods; >> >> 1. public String repeat(final int count) >> 2. public static String repeat(final char ch, final int count) >> 3. public static String repeat(final int codepoint, final int count) >> 4. public static String repeat(final CharSequence seq, final int count) >> >> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >> 3 and 4 convert to String before calling 1. >> >> Performance runs with jmh (results as comment in [2]) show that these >> methods are significantly faster that StringBuilder equivalents. >> - fewer memory allocations >> - fewer char to byte array conversions >> - faster pyramid replication vs O(N) copying >> >> I left StringBuilder out of scope. It falls under the category of >> Appendables#append with repeat. A much bigger project. >> >> All comments welcome. Especially around the need for convenience >> methods, the JavaDoc content and expanding the tests. >> >> ? Jim >> >> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >> > From Alan.Bateman at oracle.com Thu Feb 15 17:41:23 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Feb 2018 17:41:23 +0000 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> Message-ID: <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> On 15/02/2018 17:25, yumin qi wrote: > Alan, > > ? The real reason is if we do not change on resolve, the string passed > into native canonicalize will be like "//./a/" which is not expected > in native part. In native part, we resume that no more "//" in the > path string (already normalized in java). > If that is the case then the test doesn't match the issue we have been discussing on this thread. Would it be possible to create a test case for the issue that you are actually trying to address? -Alan From volker.simonis at gmail.com Thu Feb 15 17:52:33 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 15 Feb 2018 18:52:33 +0100 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: <9B0DF35E-0019-46DE-A6C1-AE4DE2416844@oracle.com> References: <9B0DF35E-0019-46DE-A6C1-AE4DE2416844@oracle.com> Message-ID: Sounds cool! Is this JEP only about providing the corresponding API or also about using it internally (within HotSpot and class library) to better adopt to environment the JVM is running in? Either way, looking forward to see (and test) the first implementation bits! Regards, Volker On Thu, Feb 15, 2018 at 6:07 PM, Bob Vandette wrote: > I?d like to re-propose the following JEP that will enhance the Java runtime to be more container aware. > This will add an Internal Java API that will provide container specific statistics. Some of the initial goals > of the previous JEP proposal has been integrated into JDK 10 under an RFE (JDK-8146115). > This JEP is now focused on providing a Java API that exports Container runtime configuration and metrics. > > Since the scope of this JEP have changed, I?m re-submitting it for comment and endorsement. > > > JEP Issue: > > https://bugs.openjdk.java.net/browse/JDK-8182070 > > Here?s a Text dump of the JEP contents for your convenience: > > Summary > ------- > > Container aware Java runtime > > Goals > ----- > > Provide an internal API that can be used to extract container specific configuration and runtime statistics. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux cgroups technology so that we will be able to easily support other container technologies running on Linux in addition to Docker. > > Non-Goals > --------- > > It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. > > Success Metrics > --------------- > > Success will be measured by the improvement in information that will be available to tools which visualize resource usage of containers that are running Java processes. > > Motivation > ---------- > > Container technology is becoming more and more prevalent in Cloud based applications. The Cloud Serverless application programming model motivates developers to split large monolithic applications into 100s of smaller pieces each running in thier own container. This move increases the importance of the observability of each running container process. Adding the proposed set of APIs will allow more details related to each container process to be made available to external tools thereby improving the observability. > > Description > ----------- > > This enhancement will be made up of the following work items: > > A. Detecting if Java is running in a container. > > The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. A new API will be made available for this purpose. > > B. Exposing container resource limits, configuration and runtime statistics. > > There are several configuration options and limits that can be imposed upon a running container. Not all of these > are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that the process has been allocated but there are other options that we might want to base runtime decisions on. > > In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. The goal is to provide this information in addition to the configuration data. > > I propose adding a new jdk.internal.Platform class that will allow access to this information. > > Here are some of the types of configuration and consumption statistics that would be made available: > > isContainerized > Memory Limit > Total Memory Limit > Soft Memory Limit > Max Memory Usage > Current Memory Usage > Maximum Kernel Memory > CPU Shares > CPU Period > CPU Quota > Number of CPUs > CPU Sets > CPU Set Memory Nodes > CPU Usage > CPU Usage Per CPU > Block I/O Weight > Block I/O Device Weight > Device I/O Read Rate > Device I/O Write Rate > OOM Kill Enabled > OOM Score Adjustment > Memory Swappiness > Shared Memory Size > > Alternatives > ------------ > > There are a few existing tools available to extract some of the same container statistics. These tools could be used instead. The benefit of providing a core Java internal API is that this information can be expose by current Java serviceability tools such as JMX and JFR along side other JVM specific information. > > Testing > ------- > > Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. > > Risks and Assumptions > --------------------- > > Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. > > Other alternative container technologies based on hypervisors are being developed that could replace the use of cgroups for container isloation. > > Dependencies > ----------- > > None at this time. > From yumin.qi at gmail.com Thu Feb 15 18:00:28 2018 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 15 Feb 2018 10:00:28 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> Message-ID: OK, let's work on a suitable test case. Thanks Yumin On Thu, Feb 15, 2018 at 9:41 AM, Alan Bateman wrote: > On 15/02/2018 17:25, yumin qi wrote: > >> Alan, >> >> The real reason is if we do not change on resolve, the string passed >> into native canonicalize will be like "//./a/" which is not expected in >> native part. In native part, we resume that no more "//" in the path string >> (already normalized in java). >> >> If that is the case then the test doesn't match the issue we have been > discussing on this thread. Would it be possible to create a test case for > the issue that you are actually trying to address? > > -Alan > From bob.vandette at oracle.com Thu Feb 15 18:04:38 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 15 Feb 2018 13:04:38 -0500 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: <9B0DF35E-0019-46DE-A6C1-AE4DE2416844@oracle.com> Message-ID: > On Feb 15, 2018, at 12:52 PM, Volker Simonis wrote: > > Sounds cool! > > Is this JEP only about providing the corresponding API or also about > using it internally (within HotSpot and class library) to better adopt > to environment the JVM is running in? Thanks Volker. I integrated JDK-8146115 into JDK 10 which allows Hotspot to adapt to the container it?s running in. The configuration that is examined includes memory limits, cpu count including cpu shares and quotas. This JEP?s main focus is to provide an internal API that can be used by JDK tools such as JFR and JMX to export container statistics and configuration data. This JEP does not include the JFR and JMX implementations. They will hopefully follow shortly after. Bob. > > Either way, looking forward to see (and test) the first implementation bits! > > Regards, > Volker > > > On Thu, Feb 15, 2018 at 6:07 PM, Bob Vandette wrote: >> I?d like to re-propose the following JEP that will enhance the Java runtime to be more container aware. >> This will add an Internal Java API that will provide container specific statistics. Some of the initial goals >> of the previous JEP proposal has been integrated into JDK 10 under an RFE (JDK-8146115). >> This JEP is now focused on providing a Java API that exports Container runtime configuration and metrics. >> >> Since the scope of this JEP have changed, I?m re-submitting it for comment and endorsement. >> >> >> JEP Issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8182070 >> >> Here?s a Text dump of the JEP contents for your convenience: >> >> Summary >> ------- >> >> Container aware Java runtime >> >> Goals >> ----- >> >> Provide an internal API that can be used to extract container specific configuration and runtime statistics. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux cgroups technology so that we will be able to easily support other container technologies running on Linux in addition to Docker. >> >> Non-Goals >> --------- >> >> It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. >> >> Success Metrics >> --------------- >> >> Success will be measured by the improvement in information that will be available to tools which visualize resource usage of containers that are running Java processes. >> >> Motivation >> ---------- >> >> Container technology is becoming more and more prevalent in Cloud based applications. The Cloud Serverless application programming model motivates developers to split large monolithic applications into 100s of smaller pieces each running in thier own container. This move increases the importance of the observability of each running container process. Adding the proposed set of APIs will allow more details related to each container process to be made available to external tools thereby improving the observability. >> >> Description >> ----------- >> >> This enhancement will be made up of the following work items: >> >> A. Detecting if Java is running in a container. >> >> The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. A new API will be made available for this purpose. >> >> B. Exposing container resource limits, configuration and runtime statistics. >> >> There are several configuration options and limits that can be imposed upon a running container. Not all of these >> are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that the process has been allocated but there are other options that we might want to base runtime decisions on. >> >> In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. The goal is to provide this information in addition to the configuration data. >> >> I propose adding a new jdk.internal.Platform class that will allow access to this information. >> >> Here are some of the types of configuration and consumption statistics that would be made available: >> >> isContainerized >> Memory Limit >> Total Memory Limit >> Soft Memory Limit >> Max Memory Usage >> Current Memory Usage >> Maximum Kernel Memory >> CPU Shares >> CPU Period >> CPU Quota >> Number of CPUs >> CPU Sets >> CPU Set Memory Nodes >> CPU Usage >> CPU Usage Per CPU >> Block I/O Weight >> Block I/O Device Weight >> Device I/O Read Rate >> Device I/O Write Rate >> OOM Kill Enabled >> OOM Score Adjustment >> Memory Swappiness >> Shared Memory Size >> >> Alternatives >> ------------ >> >> There are a few existing tools available to extract some of the same container statistics. These tools could be used instead. The benefit of providing a core Java internal API is that this information can be expose by current Java serviceability tools such as JMX and JFR along side other JVM specific information. >> >> Testing >> ------- >> >> Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. >> >> Risks and Assumptions >> --------------------- >> >> Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. >> >> Other alternative container technologies based on hypervisors are being developed that could replace the use of cgroups for container isloation. >> >> Dependencies >> ----------- >> >> None at this time. >> From kedar.mhaswade at gmail.com Thu Feb 15 18:30:16 2018 From: kedar.mhaswade at gmail.com (kedar mhaswade) Date: Thu, 15 Feb 2018 10:30:16 -0800 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: <9B0DF35E-0019-46DE-A6C1-AE4DE2416844@oracle.com> Message-ID: This appears useful. Will Runtime.getRuntime().availableProcessors() return the processors available to the container after this integration, or will a new API be provided? I have seen thread pools being misconfigured (e.g. it returns the number of processors available on the host which is far more than that of processors allotted by cgroups to the container) because of the confusion (or because of just using the non-container-aware code). Regards, Kedar On Thu, Feb 15, 2018 at 10:04 AM, Bob Vandette wrote: > > > On Feb 15, 2018, at 12:52 PM, Volker Simonis > wrote: > > > > Sounds cool! > > > > Is this JEP only about providing the corresponding API or also about > > using it internally (within HotSpot and class library) to better adopt > > to environment the JVM is running in? > > Thanks Volker. > > I integrated JDK-8146115 into JDK 10 which allows Hotspot to adapt to the > container it?s running in. The configuration that is examined includes > memory limits, > cpu count including cpu shares and quotas. > > This JEP?s main focus is to provide an internal API that can be used by > JDK tools > such as JFR and JMX to export container statistics and configuration > data. This JEP > does not include the JFR and JMX implementations. They will hopefully > follow shortly after. > > Bob. > > > > > Either way, looking forward to see (and test) the first implementation > bits! > > > > Regards, > > Volker > > > > > > On Thu, Feb 15, 2018 at 6:07 PM, Bob Vandette > wrote: > >> I?d like to re-propose the following JEP that will enhance the Java > runtime to be more container aware. > >> This will add an Internal Java API that will provide container specific > statistics. Some of the initial goals > >> of the previous JEP proposal has been integrated into JDK 10 under an > RFE (JDK-8146115). > >> This JEP is now focused on providing a Java API that exports Container > runtime configuration and metrics. > >> > >> Since the scope of this JEP have changed, I?m re-submitting it for > comment and endorsement. > >> > >> > >> JEP Issue: > >> > >> https://bugs.openjdk.java.net/browse/JDK-8182070 > >> > >> Here?s a Text dump of the JEP contents for your convenience: > >> > >> Summary > >> ------- > >> > >> Container aware Java runtime > >> > >> Goals > >> ----- > >> > >> Provide an internal API that can be used to extract container specific > configuration and runtime statistics. This JEP will only support Docker on > Linux-x64 although the design should be flexible enough to allow support > for other platforms and container technologies. The initial focus will be > on Linux cgroups technology so that we will be able to easily support other > container technologies running on Linux in addition to Docker. > >> > >> Non-Goals > >> --------- > >> > >> It is not a goal of this JEP to support any platform other than Docker > container technology running on Linux x64. > >> > >> Success Metrics > >> --------------- > >> > >> Success will be measured by the improvement in information that will be > available to tools which visualize resource usage of containers that are > running Java processes. > >> > >> Motivation > >> ---------- > >> > >> Container technology is becoming more and more prevalent in Cloud based > applications. The Cloud Serverless application programming model motivates > developers to split large monolithic applications into 100s of smaller > pieces each running in thier own container. This move increases the > importance of the observability of each running container process. Adding > the proposed set of APIs will allow more details related to each container > process to be made available to external tools thereby improving the > observability. > >> > >> Description > >> ----------- > >> > >> This enhancement will be made up of the following work items: > >> > >> A. Detecting if Java is running in a container. > >> > >> The Java runtime, as well as any tests that we might write for this > feature, will need to be able to detect that the current Java process is > running in a container. A new API will be made available for this purpose. > >> > >> B. Exposing container resource limits, configuration and runtime > statistics. > >> > >> There are several configuration options and limits that can be imposed > upon a running container. Not all of these > >> are important to a running Java process. We clearly want to be able to > detect how many CPUs have been allocated to our process along with the > maximum amount of memory that the process has been allocated but there are > other options that we might want to base runtime decisions on. > >> > >> In addition, since Container typically impose limits on system > resources, they also provide the ability to easily access the amount of > consumption of these resources. The goal is to provide this information in > addition to the configuration data. > >> > >> I propose adding a new jdk.internal.Platform class that will allow > access to this information. > >> > >> Here are some of the types of configuration and consumption statistics > that would be made available: > >> > >> isContainerized > >> Memory Limit > >> Total Memory Limit > >> Soft Memory Limit > >> Max Memory Usage > >> Current Memory Usage > >> Maximum Kernel Memory > >> CPU Shares > >> CPU Period > >> CPU Quota > >> Number of CPUs > >> CPU Sets > >> CPU Set Memory Nodes > >> CPU Usage > >> CPU Usage Per CPU > >> Block I/O Weight > >> Block I/O Device Weight > >> Device I/O Read Rate > >> Device I/O Write Rate > >> OOM Kill Enabled > >> OOM Score Adjustment > >> Memory Swappiness > >> Shared Memory Size > >> > >> Alternatives > >> ------------ > >> > >> There are a few existing tools available to extract some of the same > container statistics. These tools could be used instead. The benefit of > providing a core Java internal API is that this information can be expose > by current Java serviceability tools such as JMX and JFR along side other > JVM specific information. > >> > >> Testing > >> ------- > >> > >> Docker/container specific tests should be added in order to validate > the functionality being provided with this JEP. > >> > >> Risks and Assumptions > >> --------------------- > >> > >> Docker is currently based on cgroups v1. Cgroups v2 is also available > but is incomplete and not yet supported by Docker. It's possible that v2 > could replace v1 in an incompatible way rendering this work unusable until > it is upgraded. > >> > >> Other alternative container technologies based on hypervisors are being > developed that could replace the use of cgroups for container isloation. > >> > >> Dependencies > >> ----------- > >> > >> None at this time. > >> > > From bob.vandette at oracle.com Thu Feb 15 18:36:44 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 15 Feb 2018 13:36:44 -0500 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: <9B0DF35E-0019-46DE-A6C1-AE4DE2416844@oracle.com> Message-ID: <277A93E7-BDE4-4391-B97F-22D5ADB968BB@oracle.com> > On Feb 15, 2018, at 1:30 PM, kedar mhaswade wrote: > > This appears useful. > > Will Runtime.getRuntime().availableProcessors() return the processors available to the container after this integration, or will a new API be provided? I have seen thread pools being misconfigured (e.g. it returns the number of processors available on the host which is far more than that of processors allotted by cgroups to the container) because of the confusion (or because of just using the non-container-aware code). I fixed your above problem in JDK 10 (https://bugs.openjdk.java.net/browse/JDK-8146115 ). This work exports much more detail about the container to monitoring tools. Bob. > > Regards, > Kedar > > On Thu, Feb 15, 2018 at 10:04 AM, Bob Vandette > wrote: > > > On Feb 15, 2018, at 12:52 PM, Volker Simonis > wrote: > > > > Sounds cool! > > > > Is this JEP only about providing the corresponding API or also about > > using it internally (within HotSpot and class library) to better adopt > > to environment the JVM is running in? > > Thanks Volker. > > I integrated JDK-8146115 into JDK 10 which allows Hotspot to adapt to the > container it?s running in. The configuration that is examined includes memory limits, > cpu count including cpu shares and quotas. > > This JEP?s main focus is to provide an internal API that can be used by JDK tools > such as JFR and JMX to export container statistics and configuration data. This JEP > does not include the JFR and JMX implementations. They will hopefully follow shortly after. > > Bob. > > > > > Either way, looking forward to see (and test) the first implementation bits! > > > > Regards, > > Volker > > > > > > On Thu, Feb 15, 2018 at 6:07 PM, Bob Vandette > wrote: > >> I?d like to re-propose the following JEP that will enhance the Java runtime to be more container aware. > >> This will add an Internal Java API that will provide container specific statistics. Some of the initial goals > >> of the previous JEP proposal has been integrated into JDK 10 under an RFE (JDK-8146115). > >> This JEP is now focused on providing a Java API that exports Container runtime configuration and metrics. > >> > >> Since the scope of this JEP have changed, I?m re-submitting it for comment and endorsement. > >> > >> > >> JEP Issue: > >> > >> https://bugs.openjdk.java.net/browse/JDK-8182070 > >> > >> Here?s a Text dump of the JEP contents for your convenience: > >> > >> Summary > >> ------- > >> > >> Container aware Java runtime > >> > >> Goals > >> ----- > >> > >> Provide an internal API that can be used to extract container specific configuration and runtime statistics. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux cgroups technology so that we will be able to easily support other container technologies running on Linux in addition to Docker. > >> > >> Non-Goals > >> --------- > >> > >> It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. > >> > >> Success Metrics > >> --------------- > >> > >> Success will be measured by the improvement in information that will be available to tools which visualize resource usage of containers that are running Java processes. > >> > >> Motivation > >> ---------- > >> > >> Container technology is becoming more and more prevalent in Cloud based applications. The Cloud Serverless application programming model motivates developers to split large monolithic applications into 100s of smaller pieces each running in thier own container. This move increases the importance of the observability of each running container process. Adding the proposed set of APIs will allow more details related to each container process to be made available to external tools thereby improving the observability. > >> > >> Description > >> ----------- > >> > >> This enhancement will be made up of the following work items: > >> > >> A. Detecting if Java is running in a container. > >> > >> The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. A new API will be made available for this purpose. > >> > >> B. Exposing container resource limits, configuration and runtime statistics. > >> > >> There are several configuration options and limits that can be imposed upon a running container. Not all of these > >> are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that the process has been allocated but there are other options that we might want to base runtime decisions on. > >> > >> In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. The goal is to provide this information in addition to the configuration data. > >> > >> I propose adding a new jdk.internal.Platform class that will allow access to this information. > >> > >> Here are some of the types of configuration and consumption statistics that would be made available: > >> > >> isContainerized > >> Memory Limit > >> Total Memory Limit > >> Soft Memory Limit > >> Max Memory Usage > >> Current Memory Usage > >> Maximum Kernel Memory > >> CPU Shares > >> CPU Period > >> CPU Quota > >> Number of CPUs > >> CPU Sets > >> CPU Set Memory Nodes > >> CPU Usage > >> CPU Usage Per CPU > >> Block I/O Weight > >> Block I/O Device Weight > >> Device I/O Read Rate > >> Device I/O Write Rate > >> OOM Kill Enabled > >> OOM Score Adjustment > >> Memory Swappiness > >> Shared Memory Size > >> > >> Alternatives > >> ------------ > >> > >> There are a few existing tools available to extract some of the same container statistics. These tools could be used instead. The benefit of providing a core Java internal API is that this information can be expose by current Java serviceability tools such as JMX and JFR along side other JVM specific information. > >> > >> Testing > >> ------- > >> > >> Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. > >> > >> Risks and Assumptions > >> --------------------- > >> > >> Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. > >> > >> Other alternative container technologies based on hypervisors are being developed that could replace the use of cgroups for container isloation. > >> > >> Dependencies > >> ----------- > >> > >> None at this time. > >> > > From yumin.qi at gmail.com Thu Feb 15 18:41:30 2018 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 15 Feb 2018 10:41:30 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> Message-ID: There are two problems here, so become complex. 1) crash on parsing "//", which included in file path, on linux. This is fixed in UnixFileSystem.java in resolve function. 2) user.dir should be read only. This is fixed in both UnixFileSystem.java and WinNTFileSystem.java. The test case covers two of them. Should we handle them in two separate bugs? Yumin On Thu, Feb 15, 2018 at 10:00 AM, yumin qi wrote: > OK, let's work on a suitable test case. > > Thanks > Yumin > > On Thu, Feb 15, 2018 at 9:41 AM, Alan Bateman > wrote: > >> On 15/02/2018 17:25, yumin qi wrote: >> >>> Alan, >>> >>> The real reason is if we do not change on resolve, the string passed >>> into native canonicalize will be like "//./a/" which is not expected in >>> native part. In native part, we resume that no more "//" in the path string >>> (already normalized in java). >>> >>> If that is the case then the test doesn't match the issue we have been >> discussing on this thread. Would it be possible to create a test case for >> the issue that you are actually trying to address? >> >> -Alan >> > > From mandy.chung at oracle.com Thu Feb 15 19:10:34 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Feb 2018 11:10:34 -0800 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Message-ID: Hi Robin, Do you want a shutdown event for every call to Runtime::exit regardless where it is in the shutdown sequence?? or do you expect to get an event of the first thread calling JVM_Halt?? or the first thread starting the shutdown sequence but it may not be the thread halting? Mandy On 2/15/18 5:35 AM, Robin Westberg wrote: > Hi again, > > Here?s the (hopefully) final version: > > Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ > > Incremental: > http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ > > > Best regards, > Robin From ivan.gerasimov at oracle.com Thu Feb 15 19:36:44 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 15 Feb 2018 11:36:44 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> Message-ID: <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Hello! The link with the webrev returned 404, but I could find it at this location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ A few minor comments: 1) This check: 2992 final long limit = (long)count * 2L; 2993 if ((long)Integer.MAX_VALUE < limit) { can be possibly simplified as if (count > Integer.MAX_VALUE - count) { 2) Should String repeat(final int codepoint, final int count) be optimized for codepoints that can be represented with a single char? E.g. like this: public static String repeat(final int codepoint, final int count) { return Character.isBmpCodePoint(codepoint)) ? repeat((char) codepoint, count) : (new String(Character.toChars(codepoint))).repeat(count); } 3) Using long arithmetic can possibly be avoided in the common path of repeat(final int count): E.g. like this: if (count < 0) { throw new IllegalArgumentException("count is negative, " + count); } else if (count == 1) { return this; } else if (count == 0) { return ""; } final int len = value.length; if (Integer.MAX_VALUE / count < len) { throw new IllegalArgumentException( "Resulting string exceeds maximum string length: " + ((long)len * (long)count)); } final int limit = count * len; With kind regards, Ivan On 2/15/18 9:20 AM, Jim Laskey wrote: > This is a pre-CSR code review [1] for String repeat methods (Enhancement). > > The proposal is to introduce four new methods; > > 1. public String repeat(final int count) > 2. public static String repeat(final char ch, final int count) > 3. public static String repeat(final int codepoint, final int count) > 4. public static String repeat(final CharSequence seq, final int count) > > For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. > In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). > 3 and 4 convert to String before calling 1. > > Performance runs with jmh (results as comment in [2]) show that these > methods are significantly faster that StringBuilder equivalents. > - fewer memory allocations > - fewer char to byte array conversions > - faster pyramid replication vs O(N) copying > > I left StringBuilder out of scope. It falls under the category of > Appendables#append with repeat. A much bigger project. > > All comments welcome. Especially around the need for convenience > methods, the JavaDoc content and expanding the tests. > > ? Jim > > [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 > [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 > > -- With kind regards, Ivan Gerasimov From james.laskey at oracle.com Thu Feb 15 19:44:24 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 15 Feb 2018 15:44:24 -0400 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: > On Feb 15, 2018, at 3:36 PM, Ivan Gerasimov wrote: > > Hello! > > The link with the webrev returned 404, but I could find it at this location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ > > A few minor comments: > > 1) > > This check: > > 2992 final long limit = (long)count * 2L; > 2993 if ((long)Integer.MAX_VALUE < limit) { > > can be possibly simplified as > if (count > Integer.MAX_VALUE - count) { Good. > > 2) > Should String repeat(final int codepoint, final int count) be optimized for codepoints that can be represented with a single char? > > E.g. like this: > > public static String repeat(final int codepoint, final int count) { > return Character.isBmpCodePoint(codepoint)) > ? repeat((char) codepoint, count) > : (new String(Character.toChars(codepoint))).repeat(count); > } Yes, avoid array allocation. > > 3) > Using long arithmetic can possibly be avoided in the common path of repeat(final int count): > > E.g. like this: > > if (count < 0) { > throw new IllegalArgumentException("count is negative, " + count); > } else if (count == 1) { > return this; > } else if (count == 0) { > return ""; > } > final int len = value.length; > if (Integer.MAX_VALUE / count < len) { > throw new IllegalArgumentException( > "Resulting string exceeds maximum string length: " + ((long)len * (long)count)); > } > final int limit = count * len; Good. Thank you. > > With kind regards, > Ivan > > On 2/15/18 9:20 AM, Jim Laskey wrote: >> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >> >> The proposal is to introduce four new methods; >> >> 1. public String repeat(final int count) >> 2. public static String repeat(final char ch, final int count) >> 3. public static String repeat(final int codepoint, final int count) >> 4. public static String repeat(final CharSequence seq, final int count) >> >> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >> 3 and 4 convert to String before calling 1. >> >> Performance runs with jmh (results as comment in [2]) show that these >> methods are significantly faster that StringBuilder equivalents. >> - fewer memory allocations >> - fewer char to byte array conversions >> - faster pyramid replication vs O(N) copying >> >> I left StringBuilder out of scope. It falls under the category of >> Appendables#append with repeat. A much bigger project. >> >> All comments welcome. Especially around the need for convenience >> methods, the JavaDoc content and expanding the tests. >> >> ? Jim >> >> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >> >> > > -- > With kind regards, > Ivan Gerasimov > From forax at univ-mlv.fr Thu Feb 15 20:04:00 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 15 Feb 2018 21:04:00 +0100 (CET) Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> Message-ID: <1035679069.1995730.1518725040338.JavaMail.zimbra@u-pem.fr> I'm not sure we need 4, it's just a convenient method that may be slower than if the user code calls toString() (because of profile pollution), so i'm not sure i pull it's own weight. And about adding a default method into CharSequence, the default method should return a CharSequence or String ? and what about the other implementations, AbstractStringBuilder and CharBuffer at least ? R?mi ----- Mail original ----- > De: "Jim Laskey" > ?: "Brian Goetz" > Cc: "core-libs-dev" > Envoy?: Jeudi 15 F?vrier 2018 18:34:19 > Objet: Re: RFR: 8197594 - String and character repeat > Very reasonable approach. > > >> On Feb 15, 2018, at 1:31 PM, Brian Goetz wrote: >> >> I suggest merging 1 and 4 by making it an instance method on CS with a default >> in CS and an override on string and string builder? >> >> Sent from my MacBook Wheel >> >>> On Feb 15, 2018, at 9:20 AM, Jim Laskey wrote: >>> >>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>> >>> The proposal is to introduce four new methods; >>> >>> 1. public String repeat(final int count) >>> 2. public static String repeat(final char ch, final int count) >>> 3. public static String repeat(final int codepoint, final int count) >>> 4. public static String repeat(final CharSequence seq, final int count) >>> >>> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >>> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >>> 3 and 4 convert to String before calling 1. >>> >>> Performance runs with jmh (results as comment in [2]) show that these >>> methods are significantly faster that StringBuilder equivalents. >>> - fewer memory allocations >>> - fewer char to byte array conversions >>> - faster pyramid replication vs O(N) copying >>> >>> I left StringBuilder out of scope. It falls under the category of >>> Appendables#append with repeat. A much bigger project. >>> >>> All comments welcome. Especially around the need for convenience >>> methods, the JavaDoc content and expanding the tests. >>> >>> ? Jim >>> >>> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >>> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >>> From Alan.Bateman at oracle.com Thu Feb 15 20:06:43 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Feb 2018 20:06:43 +0000 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> Message-ID: On 15/02/2018 18:41, yumin qi wrote: > There are two problems here, so become complex. > 1) crash on parsing "//", which included in file path, on linux. This > is fixed in UnixFileSystem.java in resolve function. > 2) user.dir should be read only. This is fixed in both > UnixFileSystem.java and WinNTFileSystem.java. > > The test case covers two of them. > Should we handle them in two separate bugs? > Assuming #2 is fixed and user.dir cannot be changed, do you still have #1? The current test just ensures that bad code changing user.dir in a running VM doesn't change getCanonicalPath. If there is more to #1 then I think it will need further tests. -Alan From james.laskey at oracle.com Thu Feb 15 20:16:48 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 15 Feb 2018 16:16:48 -0400 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <1035679069.1995730.1518725040338.JavaMail.zimbra@u-pem.fr> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <1035679069.1995730.1518725040338.JavaMail.zimbra@u-pem.fr> Message-ID: > On Feb 15, 2018, at 4:04 PM, Remi Forax wrote: > > I'm not sure we need 4, it's just a convenient method that may be slower than if the user code calls toString() (because of profile pollution), > so i'm not sure i pull it's own weight. > > And about adding a default method into CharSequence, the default method should return a CharSequence or String ? If you mean each class returns an instance of its class, I think that overlaps Appendable.. > and what about the other implementations, AbstractStringBuilder and CharBuffer at least ? This falls into the Appendable.append( T t, int count) realm mentioned originally. Long term this could be a goal, and maybe defaulting CharSequence#repeat returning a string would be shortsighted. But, I think having instance String#repeat returning a CharSequence would limit its use (methods expecting strings.) ? Jim > > R?mi > > ----- Mail original ----- >> De: "Jim Laskey" >> ?: "Brian Goetz" >> Cc: "core-libs-dev" >> Envoy?: Jeudi 15 F?vrier 2018 18:34:19 >> Objet: Re: RFR: 8197594 - String and character repeat > >> Very reasonable approach. >> >> >>> On Feb 15, 2018, at 1:31 PM, Brian Goetz wrote: >>> >>> I suggest merging 1 and 4 by making it an instance method on CS with a default >>> in CS and an override on string and string builder? >>> >>> Sent from my MacBook Wheel >>> >>>> On Feb 15, 2018, at 9:20 AM, Jim Laskey wrote: >>>> >>>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>>> >>>> The proposal is to introduce four new methods; >>>> >>>> 1. public String repeat(final int count) >>>> 2. public static String repeat(final char ch, final int count) >>>> 3. public static String repeat(final int codepoint, final int count) >>>> 4. public static String repeat(final CharSequence seq, final int count) >>>> >>>> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >>>> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >>>> 3 and 4 convert to String before calling 1. >>>> >>>> Performance runs with jmh (results as comment in [2]) show that these >>>> methods are significantly faster that StringBuilder equivalents. >>>> - fewer memory allocations >>>> - fewer char to byte array conversions >>>> - faster pyramid replication vs O(N) copying >>>> >>>> I left StringBuilder out of scope. It falls under the category of >>>> Appendables#append with repeat. A much bigger project. >>>> >>>> All comments welcome. Especially around the need for convenience >>>> methods, the JavaDoc content and expanding the tests. >>>> >>>> ? Jim >>>> >>>> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >>>> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >>>> From forax at univ-mlv.fr Thu Feb 15 20:24:13 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 15 Feb 2018 21:24:13 +0100 (CET) Subject: RFR: 8197594 - String and character repeat In-Reply-To: <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: <1503189141.1997950.1518726253020.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Ivan Gerasimov" > ?: "Jim Laskey" , "core-libs-dev" > Envoy?: Jeudi 15 F?vrier 2018 20:36:44 > Objet: Re: RFR: 8197594 - String and character repeat > Hello! > > The link with the webrev returned 404, but I could find it at this > location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ Jim, in static String repeat(char ch, int count) you can remove the 'else's because they are after a throw or a return as you have done in the other methods. in static String repeat(int codepoint, int count), the parenthesis just after the return seems unnecessary, in String repeat(int count) at the end instead of testing isLatin1() return new String(bytes, coder); should be equivalent. in replicate you can return 'result' after the call to Arrays.fill, to signal to readers that this is an optimization and that the primary code is below, it will also avoid to shift the primary code to the right. and nitpicking about the style, can you remove those pesky final, it's the Nashorn style but not the JDK one :) cheers, R?mi > On 2/15/18 9:20 AM, Jim Laskey wrote: >> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >> >> The proposal is to introduce four new methods; >> >> 1. public String repeat(final int count) >> 2. public static String repeat(final char ch, final int count) >> 3. public static String repeat(final int codepoint, final int count) >> 4. public static String repeat(final CharSequence seq, final int count) >> >> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >> 3 and 4 convert to String before calling 1. >> >> Performance runs with jmh (results as comment in [2]) show that these >> methods are significantly faster that StringBuilder equivalents. >> - fewer memory allocations >> - fewer char to byte array conversions >> - faster pyramid replication vs O(N) copying >> >> I left StringBuilder out of scope. It falls under the category of >> Appendables#append with repeat. A much bigger project. >> >> All comments welcome. Especially around the need for convenience >> methods, the JavaDoc content and expanding the tests. >> >> ? Jim >> >> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >> >> > > -- > With kind regards, > Ivan Gerasimov From james.laskey at oracle.com Thu Feb 15 20:28:22 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 15 Feb 2018 16:28:22 -0400 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <1503189141.1997950.1518726253020.JavaMail.zimbra@u-pem.fr> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> <1503189141.1997950.1518726253020.JavaMail.zimbra@u-pem.fr> Message-ID: Thank you Remi. Re final: Attila got into my head. > On Feb 15, 2018, at 4:24 PM, Remi Forax wrote: > > ----- Mail original ----- >> De: "Ivan Gerasimov" >> ?: "Jim Laskey" , "core-libs-dev" >> Envoy?: Jeudi 15 F?vrier 2018 20:36:44 >> Objet: Re: RFR: 8197594 - String and character repeat > >> Hello! >> >> The link with the webrev returned 404, but I could find it at this >> location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ > > Jim, > in static String repeat(char ch, int count) > you can remove the 'else's because they are after a throw or a return as you have done in the other methods. > > in static String repeat(int codepoint, int count), > the parenthesis just after the return seems unnecessary, > > in String repeat(int count) > at the end instead of testing isLatin1() > return new String(bytes, coder); > should be equivalent. > > in replicate > you can return 'result' after the call to Arrays.fill, to signal to readers that this is an optimization and that the primary code is below, it will also avoid to shift the primary code to the right. > > and nitpicking about the style, can you remove those pesky final, it's the Nashorn style but not the JDK one :) > > cheers, > R?mi > > >> On 2/15/18 9:20 AM, Jim Laskey wrote: >>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>> >>> The proposal is to introduce four new methods; >>> >>> 1. public String repeat(final int count) >>> 2. public static String repeat(final char ch, final int count) >>> 3. public static String repeat(final int codepoint, final int count) >>> 4. public static String repeat(final CharSequence seq, final int count) >>> >>> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >>> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >>> 3 and 4 convert to String before calling 1. >>> >>> Performance runs with jmh (results as comment in [2]) show that these >>> methods are significantly faster that StringBuilder equivalents. >>> - fewer memory allocations >>> - fewer char to byte array conversions >>> - faster pyramid replication vs O(N) copying >>> >>> I left StringBuilder out of scope. It falls under the category of >>> Appendables#append with repeat. A much bigger project. >>> >>> All comments welcome. Especially around the need for convenience >>> methods, the JavaDoc content and expanding the tests. >>> >>> ? Jim >>> >>> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >>> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >>> >>> >> >> -- >> With kind regards, >> Ivan Gerasimov From yumin.qi at gmail.com Thu Feb 15 20:28:57 2018 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 15 Feb 2018 12:28:57 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> Message-ID: Hi, Alan The real reason, for why File.getCanonicalPath() will fail after "user.dir" property changed is, in File.java: public String getCanonicalPath() throws IOException { if (isInvalid()) { throw new IOException("Invalid file path"); } return fs.canonicalize(fs.resolve(this)); } It passed this File to FileSystem.resolve(File): public String resolve(File f) { if (isAbsolute(f)) return f.getPath(); return resolve(System.getProperty("user.dir"), f.getPath()); } Note, here it get property of "user.dir" and passed the string directly into resolve(String, String). Checked all the usage of this function in all other places, they all normalize the string first then pass it to resolve. Like: File.java: public File(String parent, String child) { if (child == null) { throw new NullPointerException(); } if (parent != null) { if (parent.equals("")) { this.path = fs.resolve(fs.getDefaultParent(), fs.normalize(child)); } else { this.path = fs.resolve(fs.normalize(parent), fs.normalize(child)); } } else { this.path = fs.normalize(child); } this.prefixLength = fs.prefixLength(this.path); } Since the property string contains non-normalized characters, it crashed in native canonicalize. I believe user.dir from the system is normalized, so it is OK but after it is changed like "/home/a/b/c/", it crashed. Now with using cached "user.dir", the problem is gone. The ultimate guaranteed solution may be: public UnixFileSystem() { Properties props = GetPropertyAction.privilegedGetProperties(); slash = props.getProperty("file.separator").charAt(0); colon = props.getProperty("path.separator").charAt(0); javaHome = props.getProperty("java.home"); + userDir = normalize(props.getProperty("user.dir")); // is it necessary? } public String resolve(File f) { if (isAbsolute(f)) return f.getPath();- return resolve(System.getProperty("user.dir"), f.getPath());+ SecurityManager sm = System.getSecurityManager();+ if (sm != null) {+ sm.checkPropertyAccess("user.dir");+ }+ return resolve(userDir, f.getPath()); } So the changes in resolve should be removed. Since the bug is talking about the crash, the real reason is user.dir should not be changed, how about changing description to 8194154: System property user.dir should not be changed. The test case renamed to: UserDirChangedTest.java ? Thanks Yumin On Thu, Feb 15, 2018 at 10:41 AM, yumin qi wrote: > There are two problems here, so become complex. > 1) crash on parsing "//", which included in file path, on linux. This is > fixed in UnixFileSystem.java in resolve function. > 2) user.dir should be read only. This is fixed in both UnixFileSystem.java > and WinNTFileSystem.java. > > The test case covers two of them. > Should we handle them in two separate bugs? > > Yumin > > > On Thu, Feb 15, 2018 at 10:00 AM, yumin qi wrote: > >> OK, let's work on a suitable test case. >> >> Thanks >> Yumin >> >> On Thu, Feb 15, 2018 at 9:41 AM, Alan Bateman >> wrote: >> >>> On 15/02/2018 17:25, yumin qi wrote: >>> >>>> Alan, >>>> >>>> The real reason is if we do not change on resolve, the string passed >>>> into native canonicalize will be like "//./a/" which is not expected in >>>> native part. In native part, we resume that no more "//" in the path string >>>> (already normalized in java). >>>> >>>> If that is the case then the test doesn't match the issue we have been >>> discussing on this thread. Would it be possible to create a test case for >>> the issue that you are actually trying to address? >>> >>> -Alan >>> >> >> > From forax at univ-mlv.fr Thu Feb 15 20:30:31 2018 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 15 Feb 2018 21:30:31 +0100 (CET) Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <1035679069.1995730.1518725040338.JavaMail.zimbra@u-pem.fr> Message-ID: <539362961.1998375.1518726631102.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Jim Laskey" > ?: "Remi Forax" > Cc: "Brian Goetz" , "core-libs-dev" > Envoy?: Jeudi 15 F?vrier 2018 21:16:48 > Objet: Re: RFR: 8197594 - String and character repeat >> On Feb 15, 2018, at 4:04 PM, Remi Forax wrote: >> >> I'm not sure we need 4, it's just a convenient method that may be slower than if >> the user code calls toString() (because of profile pollution), >> so i'm not sure i pull it's own weight. >> >> And about adding a default method into CharSequence, the default method should >> return a CharSequence or String ? > > If you mean each class returns an instance of its class, I think that overlaps > Appendable.. yes, very true > >> and what about the other implementations, AbstractStringBuilder and CharBuffer >> at least ? > > This falls into the Appendable.append( T t, int count) realm mentioned > originally. > > Long term this could be a goal, and maybe defaulting CharSequence#repeat > returning a string would be shortsighted. > > But, I think having instance String#repeat returning a CharSequence would limit > its use (methods expecting strings.) yes, i think we should play safe here and only add repeat to String. > > ? Jim R?mi > >> >> R?mi >> >> ----- Mail original ----- >>> De: "Jim Laskey" >>> ?: "Brian Goetz" >>> Cc: "core-libs-dev" >>> Envoy?: Jeudi 15 F?vrier 2018 18:34:19 >>> Objet: Re: RFR: 8197594 - String and character repeat >> >>> Very reasonable approach. >>> >>> >>>> On Feb 15, 2018, at 1:31 PM, Brian Goetz wrote: >>>> >>>> I suggest merging 1 and 4 by making it an instance method on CS with a default >>>> in CS and an override on string and string builder? >>>> >>>> Sent from my MacBook Wheel >>>> >>>>> On Feb 15, 2018, at 9:20 AM, Jim Laskey wrote: >>>>> >>>>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>>>> >>>>> The proposal is to introduce four new methods; >>>>> >>>>> 1. public String repeat(final int count) >>>>> 2. public static String repeat(final char ch, final int count) >>>>> 3. public static String repeat(final int codepoint, final int count) >>>>> 4. public static String repeat(final CharSequence seq, final int count) >>>>> >>>>> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >>>>> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >>>>> 3 and 4 convert to String before calling 1. >>>>> >>>>> Performance runs with jmh (results as comment in [2]) show that these >>>>> methods are significantly faster that StringBuilder equivalents. >>>>> - fewer memory allocations >>>>> - fewer char to byte array conversions >>>>> - faster pyramid replication vs O(N) copying >>>>> >>>>> I left StringBuilder out of scope. It falls under the category of >>>>> Appendables#append with repeat. A much bigger project. >>>>> >>>>> All comments welcome. Especially around the need for convenience >>>>> methods, the JavaDoc content and expanding the tests. >>>>> >>>>> ? Jim >>>>> >>>>> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >>>>> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 From Roger.Riggs at Oracle.com Thu Feb 15 20:38:02 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 15 Feb 2018 15:38:02 -0500 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <1035679069.1995730.1518725040338.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Jim, Its cleaner to do the API (CSR) review before and without the implementation. It helps keep the issues separate. Don't make statements about count == Integer.MAX_VALUE / 2. There is no point, unless it should throw IAE. On 2/15/2018 3:16 PM, Jim Laskey wrote: >> On Feb 15, 2018, at 4:04 PM, Remi Forax wrote: >> >> I'm not sure we need 4, it's just a convenient method that may be slower than if the user code calls toString() (because of profile pollution), >> so i'm not sure i pull it's own weight. >> >> And about adding a default method into CharSequence, the default method should return a CharSequence or String ? > If you mean each class returns an instance of its class, I think that overlaps Appendable.. Beware thinking about default methods on interfaces; the dragons will get you. Recently, this was discussed in the context of Reader.transferTo. >> and what about the other implementations, AbstractStringBuilder and CharBuffer at least ? > This falls into the Appendable.append( T t, int count) realm mentioned originally. > > Long term this could be a goal, and maybe defaulting CharSequence#repeat returning a string would be shortsighted. > > But, I think having instance String#repeat returning a CharSequence would limit its use (methods expecting strings.) There is no point in returning CharSequence, a String is a CharSequence and can be used anywhere a CharSequence can. $.02, Roger > > ? Jim > >> R?mi >> >> ----- Mail original ----- >>> De: "Jim Laskey" >>> ?: "Brian Goetz" >>> Cc: "core-libs-dev" >>> Envoy?: Jeudi 15 F?vrier 2018 18:34:19 >>> Objet: Re: RFR: 8197594 - String and character repeat >>> Very reasonable approach. >>> >>> >>>> On Feb 15, 2018, at 1:31 PM, Brian Goetz wrote: >>>> >>>> I suggest merging 1 and 4 by making it an instance method on CS with a default >>>> in CS and an override on string and string builder? >>>> >>>> Sent from my MacBook Wheel >>>> >>>>> On Feb 15, 2018, at 9:20 AM, Jim Laskey wrote: >>>>> >>>>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>>>> >>>>> The proposal is to introduce four new methods; >>>>> >>>>> 1. public String repeat(final int count) >>>>> 2. public static String repeat(final char ch, final int count) >>>>> 3. public static String repeat(final int codepoint, final int count) >>>>> 4. public static String repeat(final CharSequence seq, final int count) >>>>> >>>>> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >>>>> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >>>>> 3 and 4 convert to String before calling 1. >>>>> >>>>> Performance runs with jmh (results as comment in [2]) show that these >>>>> methods are significantly faster that StringBuilder equivalents. >>>>> - fewer memory allocations >>>>> - fewer char to byte array conversions >>>>> - faster pyramid replication vs O(N) copying >>>>> >>>>> I left StringBuilder out of scope. It falls under the category of >>>>> Appendables#append with repeat. A much bigger project. >>>>> >>>>> All comments welcome. Especially around the need for convenience >>>>> methods, the JavaDoc content and expanding the tests. >>>>> >>>>> ? Jim >>>>> >>>>> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >>>>> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >>>>> From james.laskey at oracle.com Thu Feb 15 20:41:27 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 15 Feb 2018 16:41:27 -0400 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <1035679069.1995730.1518725040338.JavaMail.zimbra@u-pem.fr> Message-ID: Thank you Roger. > On Feb 15, 2018, at 4:38 PM, Roger Riggs wrote: > > Hi Jim, > > Its cleaner to do the API (CSR) review before and without the implementation. > It helps keep the issues separate. This was on advice from a member of the core libs team. He can speak up if he wants. > > Don't make statements about count == Integer.MAX_VALUE / 2. > There is no point, unless it should throw IAE. > > On 2/15/2018 3:16 PM, Jim Laskey wrote: >>> On Feb 15, 2018, at 4:04 PM, Remi Forax wrote: >>> >>> I'm not sure we need 4, it's just a convenient method that may be slower than if the user code calls toString() (because of profile pollution), >>> so i'm not sure i pull it's own weight. >>> >>> And about adding a default method into CharSequence, the default method should return a CharSequence or String ? >> If you mean each class returns an instance of its class, I think that overlaps Appendable.. > Beware thinking about default methods on interfaces; the dragons will get you. > Recently, this was discussed in the context of Reader.transferTo. >>> and what about the other implementations, AbstractStringBuilder and CharBuffer at least ? >> This falls into the Appendable.append( T t, int count) realm mentioned originally. >> >> Long term this could be a goal, and maybe defaulting CharSequence#repeat returning a string would be shortsighted. >> >> But, I think having instance String#repeat returning a CharSequence would limit its use (methods expecting strings.) > There is no point in returning CharSequence, a String is a CharSequence and can be used anywhere > a CharSequence can. > > $.02, Roger > >> >> ? Jim >> >>> R?mi >>> >>> ----- Mail original ----- >>>> De: "Jim Laskey" >>>> ?: "Brian Goetz" >>>> Cc: "core-libs-dev" >>>> Envoy?: Jeudi 15 F?vrier 2018 18:34:19 >>>> Objet: Re: RFR: 8197594 - String and character repeat >>>> Very reasonable approach. >>>> >>>> >>>>> On Feb 15, 2018, at 1:31 PM, Brian Goetz wrote: >>>>> >>>>> I suggest merging 1 and 4 by making it an instance method on CS with a default >>>>> in CS and an override on string and string builder? >>>>> >>>>> Sent from my MacBook Wheel >>>>> >>>>>> On Feb 15, 2018, at 9:20 AM, Jim Laskey wrote: >>>>>> >>>>>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>>>>> >>>>>> The proposal is to introduce four new methods; >>>>>> >>>>>> 1. public String repeat(final int count) >>>>>> 2. public static String repeat(final char ch, final int count) >>>>>> 3. public static String repeat(final int codepoint, final int count) >>>>>> 4. public static String repeat(final CharSequence seq, final int count) >>>>>> >>>>>> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >>>>>> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >>>>>> 3 and 4 convert to String before calling 1. >>>>>> >>>>>> Performance runs with jmh (results as comment in [2]) show that these >>>>>> methods are significantly faster that StringBuilder equivalents. >>>>>> - fewer memory allocations >>>>>> - fewer char to byte array conversions >>>>>> - faster pyramid replication vs O(N) copying >>>>>> >>>>>> I left StringBuilder out of scope. It falls under the category of >>>>>> Appendables#append with repeat. A much bigger project. >>>>>> >>>>>> All comments welcome. Especially around the need for convenience >>>>>> methods, the JavaDoc content and expanding the tests. >>>>>> >>>>>> ? Jim >>>>>> >>>>>> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >>>>>> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >>>>>> > From lowasser at google.com Thu Feb 15 20:52:11 2018 From: lowasser at google.com (Louis Wasserman) Date: Thu, 15 Feb 2018 20:52:11 +0000 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: I don't think there's a case for demand to merit having a repeat(CharSequence, int) at all. I did an analysis of usages of Guava's Strings.repeat on Google's codebase. Users might be rolling their own implementations, too, but this should be a very good proxy for demand. StringRepeat_SingleConstantChar = 4.475 K // strings with .length() == 1 StringRepeat_SingleConstantCodePoint = 28 // strings with .codePointCount(...) == 1 StringRepeat_MultiCodePointConstant = 1.156 K // constant strings neither of the above StringRepeat_CharSequenceToString = 2 // Strings.repeat(CharSequence.toString(), n) StringRepeat_NoneOfTheAbove = 248 Notably, it seems like basically nobody needs to repeat a CharSequence -- definitely not enough demand to merit the awkwardness of e.g. Rope.repeat(n) inheriting a repeat returning a String. Based on this data, I'd recommend providing one and only one method of this type: String.repeat(int). There's no real advantage to a static repeat(char, int) method when the overwhelming majority of these are constants: e.g. compare SomeUtilClass.repeat('*', n) versus "*".repeat(n). Character.toString(c).repeat(n) isn't a bad workaround if you don't have a constant char. There also isn't much demand for dealing with the code point case specially; the String.repeat(int) method seems like it'd handle that just fine. On Thu, Feb 15, 2018 at 11:44 AM Jim Laskey wrote: > > > > On Feb 15, 2018, at 3:36 PM, Ivan Gerasimov > wrote: > > > > Hello! > > > > The link with the webrev returned 404, but I could find it at this > location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ > > > > A few minor comments: > > > > 1) > > > > This check: > > > > 2992 final long limit = (long)count * 2L; > > 2993 if ((long)Integer.MAX_VALUE < limit) { > > > > can be possibly simplified as > > if (count > Integer.MAX_VALUE - count) { > > Good. > > > > > 2) > > Should String repeat(final int codepoint, final int count) be optimized > for codepoints that can be represented with a single char? > > > > E.g. like this: > > > > public static String repeat(final int codepoint, final int count) { > > return Character.isBmpCodePoint(codepoint)) > > ? repeat((char) codepoint, count) > > : (new String(Character.toChars(codepoint))).repeat(count); > > } > > Yes, avoid array allocation. > > > > > 3) > > Using long arithmetic can possibly be avoided in the common path of > repeat(final int count): > > > > E.g. like this: > > > > if (count < 0) { > > throw new IllegalArgumentException("count is negative, " + > count); > > } else if (count == 1) { > > return this; > > } else if (count == 0) { > > return ""; > > } > > final int len = value.length; > > if (Integer.MAX_VALUE / count < len) { > > throw new IllegalArgumentException( > > "Resulting string exceeds maximum string length: " + > ((long)len * (long)count)); > > } > > final int limit = count * len; > > Good. > > Thank you. > > > > > With kind regards, > > Ivan > > > > On 2/15/18 9:20 AM, Jim Laskey wrote: > >> This is a pre-CSR code review [1] for String repeat methods > (Enhancement). > >> > >> The proposal is to introduce four new methods; > >> > >> 1. public String repeat(final int count) > >> 2. public static String repeat(final char ch, final int count) > >> 3. public static String repeat(final int codepoint, final int count) > >> 4. public static String repeat(final CharSequence seq, final int count) > >> > >> For the sake of transparency, only 1 is necessary, 2-4 are convenience > methods. > >> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, > 10). > >> 3 and 4 convert to String before calling 1. > >> > >> Performance runs with jmh (results as comment in [2]) show that these > >> methods are significantly faster that StringBuilder equivalents. > >> - fewer memory allocations > >> - fewer char to byte array conversions > >> - faster pyramid replication vs O(N) copying > >> > >> I left StringBuilder out of scope. It falls under the category of > >> Appendables#append with repeat. A much bigger project. > >> > >> All comments welcome. Especially around the need for convenience > >> methods, the JavaDoc content and expanding the tests. > >> > >> ? Jim > >> > >> [1] webrev: > http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 > >> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 > >> > >> > > > > -- > > With kind regards, > > Ivan Gerasimov > > > > From james.laskey at oracle.com Thu Feb 15 20:55:38 2018 From: james.laskey at oracle.com (James Laskey) Date: Thu, 15 Feb 2018 16:55:38 -0400 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: <4F3D6A8C-F0B2-4B1E-AADA-B473DA77AB23@oracle.com> Good to have the data. Thank you Louis. Sent from my iPhone > On Feb 15, 2018, at 4:52 PM, Louis Wasserman wrote: > > I don't think there's a case for demand to merit having a repeat(CharSequence, int) at all. > > I did an analysis of usages of Guava's Strings.repeat on Google's codebase. Users might be rolling their own implementations, too, but this should be a very good proxy for demand. > > StringRepeat_SingleConstantChar = 4.475 K // strings with .length() == 1 > StringRepeat_SingleConstantCodePoint = 28 // strings with .codePointCount(...) == 1 > StringRepeat_MultiCodePointConstant = 1.156 K // constant strings neither of the above > StringRepeat_CharSequenceToString = 2 // Strings.repeat(CharSequence.toString(), n) > StringRepeat_NoneOfTheAbove = 248 > > Notably, it seems like basically nobody needs to repeat a CharSequence -- definitely not enough demand to merit the awkwardness of e.g. Rope.repeat(n) inheriting a repeat returning a String. > > Based on this data, I'd recommend providing one and only one method of this type: String.repeat(int). There's no real advantage to a static repeat(char, int) method when the overwhelming majority of these are constants: e.g. compare SomeUtilClass.repeat('*', n) versus "*".repeat(n). Character.toString(c).repeat(n) isn't a bad workaround if you don't have a constant char. There also isn't much demand for dealing with the code point case specially; the String.repeat(int) method seems like it'd handle that just fine. > >> On Thu, Feb 15, 2018 at 11:44 AM Jim Laskey wrote: >> >> >> > On Feb 15, 2018, at 3:36 PM, Ivan Gerasimov wrote: >> > >> > Hello! >> > >> > The link with the webrev returned 404, but I could find it at this location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ >> > >> > A few minor comments: >> > >> > 1) >> > >> > This check: >> > >> > 2992 final long limit = (long)count * 2L; >> > 2993 if ((long)Integer.MAX_VALUE < limit) { >> > >> > can be possibly simplified as >> > if (count > Integer.MAX_VALUE - count) { >> >> Good. >> >> > >> > 2) >> > Should String repeat(final int codepoint, final int count) be optimized for codepoints that can be represented with a single char? >> > >> > E.g. like this: >> > >> > public static String repeat(final int codepoint, final int count) { >> > return Character.isBmpCodePoint(codepoint)) >> > ? repeat((char) codepoint, count) >> > : (new String(Character.toChars(codepoint))).repeat(count); >> > } >> >> Yes, avoid array allocation. >> >> > >> > 3) >> > Using long arithmetic can possibly be avoided in the common path of repeat(final int count): >> > >> > E.g. like this: >> > >> > if (count < 0) { >> > throw new IllegalArgumentException("count is negative, " + count); >> > } else if (count == 1) { >> > return this; >> > } else if (count == 0) { >> > return ""; >> > } >> > final int len = value.length; >> > if (Integer.MAX_VALUE / count < len) { >> > throw new IllegalArgumentException( >> > "Resulting string exceeds maximum string length: " + ((long)len * (long)count)); >> > } >> > final int limit = count * len; >> >> Good. >> >> Thank you. >> >> > >> > With kind regards, >> > Ivan >> > >> > On 2/15/18 9:20 AM, Jim Laskey wrote: >> >> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >> >> >> >> The proposal is to introduce four new methods; >> >> >> >> 1. public String repeat(final int count) >> >> 2. public static String repeat(final char ch, final int count) >> >> 3. public static String repeat(final int codepoint, final int count) >> >> 4. public static String repeat(final CharSequence seq, final int count) >> >> >> >> For the sake of transparency, only 1 is necessary, 2-4 are convenience methods. >> >> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, 10). >> >> 3 and 4 convert to String before calling 1. >> >> >> >> Performance runs with jmh (results as comment in [2]) show that these >> >> methods are significantly faster that StringBuilder equivalents. >> >> - fewer memory allocations >> >> - fewer char to byte array conversions >> >> - faster pyramid replication vs O(N) copying >> >> >> >> I left StringBuilder out of scope. It falls under the category of >> >> Appendables#append with repeat. A much bigger project. >> >> >> >> All comments welcome. Especially around the need for convenience >> >> methods, the JavaDoc content and expanding the tests. >> >> >> >> ? Jim >> >> >> >> [1] webrev: http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >> >> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >> >> >> >> >> > >> > -- >> > With kind regards, >> > Ivan Gerasimov >> > >> From brian.goetz at oracle.com Thu Feb 15 21:15:07 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 15 Feb 2018 13:15:07 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: <3CDC5403-AD32-4893-A2B5-D36BC85538C9@oracle.com> This matches my intuition too. Sent from my MacBook Wheel > On Feb 15, 2018, at 12:52 PM, Louis Wasserman wrote: > > I don't think there's a case for demand to merit having a > repeat(CharSequence, int) at all. > > I did an analysis of usages of Guava's Strings.repeat on Google's > codebase. Users might be rolling their own implementations, too, but this > should be a very good proxy for demand. > > StringRepeat_SingleConstantChar = 4.475 K // strings with .length() == 1 > StringRepeat_SingleConstantCodePoint = 28 // strings with > .codePointCount(...) == 1 > StringRepeat_MultiCodePointConstant = 1.156 K // constant strings neither > of the above > StringRepeat_CharSequenceToString = 2 // > Strings.repeat(CharSequence.toString(), n) > StringRepeat_NoneOfTheAbove = 248 > > Notably, it seems like basically nobody needs to repeat a CharSequence -- > definitely not enough demand to merit the awkwardness of e.g. > Rope.repeat(n) inheriting a repeat returning a String. > > Based on this data, I'd recommend providing one and only one method of this > type: String.repeat(int). There's no real advantage to a static > repeat(char, int) method when the overwhelming majority of these are > constants: e.g. compare SomeUtilClass.repeat('*', n) versus "*".repeat(n). > Character.toString(c).repeat(n) isn't a bad workaround if you don't have a > constant char. There also isn't much demand for dealing with the code > point case specially; the String.repeat(int) method seems like it'd handle > that just fine. > >> On Thu, Feb 15, 2018 at 11:44 AM Jim Laskey wrote: >> >> >> >>> On Feb 15, 2018, at 3:36 PM, Ivan Gerasimov >> wrote: >>> >>> Hello! >>> >>> The link with the webrev returned 404, but I could find it at this >> location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ >>> >>> A few minor comments: >>> >>> 1) >>> >>> This check: >>> >>> 2992 final long limit = (long)count * 2L; >>> 2993 if ((long)Integer.MAX_VALUE < limit) { >>> >>> can be possibly simplified as >>> if (count > Integer.MAX_VALUE - count) { >> >> Good. >> >>> >>> 2) >>> Should String repeat(final int codepoint, final int count) be optimized >> for codepoints that can be represented with a single char? >>> >>> E.g. like this: >>> >>> public static String repeat(final int codepoint, final int count) { >>> return Character.isBmpCodePoint(codepoint)) >>> ? repeat((char) codepoint, count) >>> : (new String(Character.toChars(codepoint))).repeat(count); >>> } >> >> Yes, avoid array allocation. >> >>> >>> 3) >>> Using long arithmetic can possibly be avoided in the common path of >> repeat(final int count): >>> >>> E.g. like this: >>> >>> if (count < 0) { >>> throw new IllegalArgumentException("count is negative, " + >> count); >>> } else if (count == 1) { >>> return this; >>> } else if (count == 0) { >>> return ""; >>> } >>> final int len = value.length; >>> if (Integer.MAX_VALUE / count < len) { >>> throw new IllegalArgumentException( >>> "Resulting string exceeds maximum string length: " + >> ((long)len * (long)count)); >>> } >>> final int limit = count * len; >> >> Good. >> >> Thank you. >> >>> >>> With kind regards, >>> Ivan >>> >>>> On 2/15/18 9:20 AM, Jim Laskey wrote: >>>> This is a pre-CSR code review [1] for String repeat methods >> (Enhancement). >>>> >>>> The proposal is to introduce four new methods; >>>> >>>> 1. public String repeat(final int count) >>>> 2. public static String repeat(final char ch, final int count) >>>> 3. public static String repeat(final int codepoint, final int count) >>>> 4. public static String repeat(final CharSequence seq, final int count) >>>> >>>> For the sake of transparency, only 1 is necessary, 2-4 are convenience >> methods. >>>> In the case of 2, ?*?.repeat(10) performs as well as String.repeat(?*?, >> 10). >>>> 3 and 4 convert to String before calling 1. >>>> >>>> Performance runs with jmh (results as comment in [2]) show that these >>>> methods are significantly faster that StringBuilder equivalents. >>>> - fewer memory allocations >>>> - fewer char to byte array conversions >>>> - faster pyramid replication vs O(N) copying >>>> >>>> I left StringBuilder out of scope. It falls under the category of >>>> Appendables#append with repeat. A much bigger project. >>>> >>>> All comments welcome. Especially around the need for convenience >>>> methods, the JavaDoc content and expanding the tests. >>>> >>>> ? Jim >>>> >>>> [1] webrev: >> http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >>>> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >>>> >>>> >>> >>> -- >>> With kind regards, >>> Ivan Gerasimov >>> >> >> From stuart.marks at oracle.com Thu Feb 15 21:55:03 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 15 Feb 2018 13:55:03 -0800 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: <5A831997.4030005@oracle.com> References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> Message-ID: > public String(ByteBuffer bytes, Charset cs); > public String(ByteBuffer bytes, String csname); I think these constructors make good sense. They avoid an extra copy to an intermediate byte[]. One issue (also mentioned by Stephen Colebourne) is whether we need the csname overload. Arguably it's not needed if we have the Charset overload. And the csname overload throws UnsupportedEncodingException, which is checked. But the csname overload is apparently faster, since the decoder can be cached, and it's unclear when this can be remedied for the Charset case.... I could go either way on this one. ** I'd also suggest adding a CharBuffer constructor: public String(CharBuffer cbuf) This would be semantically equivalent to public String(char[] value, int offset, int count) except using the chars from the CharBuffer between the buffer's position and its limit. ** Regarding the getBytes() overloads: > public int getBytes(byte[] dst, int offset, Charset cs); > public int getBytes(byte[] dst, int offset, String csname); > public int getBytes(ByteBuffer bytes, Charset cs); > public int getBytes(ByteBuffer bytes, Charset csn); > On 2/13/18, 12:41 AM, Alan Bateman wrote: >> These four methods encode as many characters as possible into the destination >> byte[] or buffer but don't give any indication that the destination didn't >> have enough space to encode the entire string. I thus worry they could be a >> hazard and result in buggy code. If there is insufficient space then the user >> of the API doesn't know how many characters were encoded so it's not easy to >> substring and call getBytes again to encode the remaining characters. There is >> also the issue of how to size the destination. What would you think about >> having them fail when there is insufficient space? If they do fail then there >> is a side effect that they will have written to the destination so that would >> need to be documented too. I share Alan's concern here. If the intent is to reuse a byte[] or a ByteBuffer, then there needs to be an effective way to handle the case where the provided array/buffer doesn't have enough room to receive the decoded string. A variety of ways of dealing with this have been mentioned, such as throwing an exception; returning negative value to indicate failure, possibly also encoding the number of bytes written; or even allocating a fresh array or buffer of the proper size and returning that instead. The caller would have to check the return value and take care to handle all the cases properly. This is likely to be fairly error-prone. This also raises the question in my mind of what these getBytes() methods are intended for. On the one hand, they might be useful for the caller to manage its own memory allocation and reuse of arrays/buffers. If so, then it's necessary for intermediate results from partial processing to be handled properly. If the destination fills up, there needs to be a way to report how much of the input was consumed, so that a subsequent operation can pick up where the previous one left off. (This was one of David Lloyd's points.) If there's sufficient room in the destination, there needs to be a way to report this and how much space remains in the destination. One could contemplate adding all this information to the API. This eventually leads to CharsetEncoder.encode(CharBuffer in, ByteBuffer out, boolean endOfInput) which has all the necessary partial progress state in the buffers. On the other hand, maybe the intent of these APIs is for convenience. I'd observe that String already has this method: public byte[] getBytes(Charset) which returns the decoded bytes in a newly allocated array of the proper size. This is pretty convenient. It doesn't let the caller reuse a destination array or buffer... but that's what brings in all the partial result edge cases. Bottom line is that I'm not entirely sure of the use of these new getBytes() overloads. Maybe I've missed a use case where these work; if so, maybe somebody can describe it. s'marks From joe.darcy at oracle.com Thu Feb 15 21:59:17 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Thu, 15 Feb 2018 13:59:17 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <1035679069.1995730.1518725040338.JavaMail.zimbra@u-pem.fr> Message-ID: <5A8602B5.3050207@oracle.com> On 2/15/2018 12:38 PM, Roger Riggs wrote: > Hi Jim, > > Its cleaner to do the API (CSR) review before and without the > implementation. > It helps keep the issues separate. > > Don't make statements about count == Integer.MAX_VALUE / 2. > There is no point, unless it should throw IAE. > My general recommendation if the code review and CSR review are to be serialized is to estimate which one is more likely to generate feedback that might modify the proposal and run though that process first. Since there has already been feedback leading to updates of this proposal, code review first in this case seems like a reasonable choice :-) Developers at their discretion can run CSR and code review in parallel, but feedback may come from either process. Proposals still under development can also go through the two-phase CSR process to get some initial feedback before an intended-to-be-final version is produced. HTH, -Joe From mandy.chung at oracle.com Thu Feb 15 23:06:54 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Feb 2018 15:06:54 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit Message-ID: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> Runtime.runFinalizersOnExit has been deprecated since 1.2 (1998) and deprecated for removal in JDK 9. We analyzed the maven central artifacts few years ago that show very few uses of it. I also survey a recent version of most of the maven artifacts that references runFinalizatsOnExit no longer references it. I propose to remove Runtime.runFinalizersOnExit and System.runFinalizersOnExit methods in JDK 11. CSR: https://bugs.openjdk.java.net/browse/JDK-8198250 Webrev: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.00/ Thanks Mandy [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-January/031041.html From mandy.chung at oracle.com Thu Feb 15 23:20:21 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Feb 2018 15:20:21 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> Message-ID: <59507b7d-4797-21aa-b084-1687583b78fe@oracle.com> Hi Martin, On 2/14/18 9:47 PM, Martin Buchholz wrote: > Embarrassed by my failure to take runFinalizersOnExit into account, I > tried to redo the patch to be actually correct. > http://cr.openjdk.java.net/~martin/webrevs/jdk/Finalizer-data-race/ > > even though we may succeed in making runFinalizersOnExit go away, and > I'm not planning to submit any time soon. > The runFinalizersOnExit case needs different mechanics for removing > elements from "unfinalized". > The simple invariant is that you need to run the finalizer whenever an > element is unlinked from unfinalized. > I have posted RFR to remove runFinalizersOnExit.?? I think your patch would become cleaner as you may keep the remove method or inline in runFinalizer method as your original patch has. Do you want to wait until JDK-8198249?? or proceed this patch without waiting? Mandy [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-February/051518.html From xueming.shen at oracle.com Thu Feb 15 23:25:54 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 15 Feb 2018 15:25:54 -0800 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> Message-ID: <5A861702.7090102@oracle.com> On 2/15/18, 1:55 PM, Stuart Marks wrote: > >> On 2/13/18, 12:41 AM, Alan Bateman wrote: >>> These four methods encode as many characters as possible into the >>> destination byte[] or buffer but don't give any indication that the >>> destination didn't have enough space to encode the entire string. I >>> thus worry they could be a hazard and result in buggy code. If there >>> is insufficient space then the user of the API doesn't know how many >>> characters were encoded so it's not easy to substring and call >>> getBytes again to encode the remaining characters. There is also the >>> issue of how to size the destination. What would you think about >>> having them fail when there is insufficient space? If they do fail >>> then there is a side effect that they will have written to the >>> destination so that would need to be documented too. > > I share Alan's concern here. > > If the intent is to reuse a byte[] or a ByteBuffer, then there needs > to be an effective way to handle the case where the provided > array/buffer doesn't have enough room to receive the decoded string. A > variety of ways of dealing with this have been mentioned, such as > throwing an exception; returning negative value to indicate failure, To add the ref for the discussion. Here is one of the alternatives being discussed, to return the "-length", negative value of the space needed to encode the whole String, to hint the caller to pass in an appropriately sized buffer (only the String APIs are updated). This appears to be a better than the proposed "partial fill"/encode as many as possible approach. But it will still force the caller to take care of the return value and manage the possible buffer size issue himself, which triggers the question that whether or not the CharsetDecoder is a better place for such kind of use scenario. http://cr.openjdk.java.net/~sherman/8021560/webrev.v2 -Sherman From patrick at reini.net Thu Feb 15 23:33:54 2018 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 16 Feb 2018 00:33:54 +0100 Subject: RFR 8196298 Add null Reader and Writer In-Reply-To: <9FD69071-578D-4BA3-AAEC-40BBCFA4B737@reini.net> References: <507D4003-9BF4-4B2F-8BCF-736398D75590@reini.net> <36bf7199-d852-825a-62a9-35c9b0e49158@oracle.com> <9FD69071-578D-4BA3-AAEC-40BBCFA4B737@reini.net> Message-ID: <51229245-979a-fc60-4f37-1975f86faf4a@reini.net> Hi everyone responded so far. How should we proceed now? -Patrick Am 07.02.2018 um 18:38 schrieb Patrick Reinhart: >> Am 07.02.2018 um 14:03 schrieb Alan Bateman : >> >> On 03/02/2018 17:05, Patrick Reinhart wrote: >>> : >>> I reworked the tests and Writer implementation accordingly >>> >>> http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.01 >>> >> Just catching up on this. >> >> nullReader?s javadoc suggests that mark(int) does not nothing but this seems to conflict with the Reader's javadoc where it is specified to throw IOE when mark is not supported. > So I will change the API description accordingly there? > >> I'm a bit uneasy with len==0 check in the read method. I realize this is trying to mirror InputStream.read and maybe some Reader implementations but it's not specified behavior. I think we have to re-examine the Reader spec to clarify this to avoid creating more inconsistencies. > That?s exactly what I did, so for me it?s ok to look into the Reader spec also as I already modify that File. I personally thought that a null like reader will always be at the end of the stream and therefore return -1 and do no checking of how much space may be left on the consuming end. That seem to me more the natural way of thinking. > >> Similarly in nullWriter() where it looks like the the len==0 checks is in unspecified territory. I think this will need clarifications to the Writer spec. One obvious inconsistency is to close the writer and call write("") and write("", 0, 0). The first will fail as the writer is closed, the second does not fail. > Here I think the same holds true as I wrote for the Reader? > >> Another one is read(CharBuffer). Suppose CharBuffer cb has 0 bytes remaining; if I invoke nullReader().read(cb) then it looks like it will return 0 whereas the read(CharBuffer) method is specified to return -1. >> > I see your point there, the implementation I did there was taken from the StringReader, where it will be the same len==0 check as you mentioned before? So it looks like we really need to look into those implementations? > > -Patrick > > From mandy.chung at oracle.com Fri Feb 16 01:07:10 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Feb 2018 17:07:10 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> Message-ID: <10fbe2cf-3a16-a0a3-deec-34566fdffa5d@oracle.com> On 2/15/18 4:18 PM, Stuart Marks wrote: > > > On 2/15/18 3:06 PM, mandy chung wrote: >> Runtime.runFinalizersOnExit has been deprecated since 1.2 (1998) >> and deprecated for removal in JDK 9.? We analyzed the maven central >> artifacts few years ago that show very few uses of it.? I also >> survey a recent version of most of the maven artifacts that references >> runFinalizatsOnExit no longer references it.? I propose to remove >> Runtime.runFinalizersOnExit and System.runFinalizersOnExit methods >> in JDK 11. >> >> CSR: https://bugs.openjdk.java.net/browse/JDK-8198250 > > I've added myself to the reviewers list of this CSR. > > Note that you've listed the Compatibility Risk as Medium, but the > notes are in support of a low risk level. Given this analysis, I'd be > comfortable with Compatibility Risk being set to Low if you want to do > that. > I should make it clear.? I keep it medium because the data doesn't represent all libraries. >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.00/ > > Finalizer.java: > >> ?111??????? This method is used by both runFinalization. > > Delete "both". > I will fix that. > Thanks for updating the docs and comments in jdeprscan. > > Overall looks good. Someone else should look at the native code, > though, since I'm not too familiar with it. > Thanks Mandy From david.holmes at oracle.com Fri Feb 16 02:11:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 12:11:50 +1000 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> Message-ID: <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> Hi Mandy, Good to see this go. A few minor comments. First I've added comments on the CSR as some of the doc changes don't read correctly. src/hotspot/share/runtime/thread.cpp This comment doesn't read correctly: ! // won't be run. Note that if a shutdown hook was registered // was called, the Shutdown class would have already been loaded ! // (Runtime.addShutdownHook will load it). delete "was called, " --- src/java.base/share/classes/java/lang/Shutdown.java This was a bit confusing. :) I wasn't at all sure you needed the COMPLETED state (which is really a HOOKS_HAVE_BEEN_STARTED state). But I see it allows for a second exit( Runtime.runFinalizersOnExit has been deprecated since 1.2 (1998) > and deprecated for removal in JDK 9.? We analyzed the maven central > artifacts few years ago that show very few uses of it.? I also > survey a recent version of most of the maven artifacts that references > runFinalizatsOnExit no longer references it.? I propose to remove > Runtime.runFinalizersOnExit and System.runFinalizersOnExit methods > in JDK 11. > > > CSR: https://bugs.openjdk.java.net/browse/JDK-8198250 > > Webrev: > ? http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.00/ > > Thanks > Mandy > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-January/031041.html > > From david.holmes at oracle.com Fri Feb 16 02:33:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 12:33:03 +1000 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: <9B0DF35E-0019-46DE-A6C1-AE4DE2416844@oracle.com> Message-ID: <939916f0-3b99-7167-f495-abb29a61fb62@oracle.com> On 16/02/2018 4:30 AM, kedar mhaswade wrote: > This appears useful. > > Will Runtime.getRuntime().availableProcessors() return the processors > available to the container after this integration, or will a new API be > provided? I have seen thread pools being misconfigured (e.g. it returns the > number of processors available on the host which is far more than that of > processors allotted by cgroups to the container) because of the confusion > (or because of just using the non-container-aware code). Depending on what you mean by "allotted" the VM has reported the actual set of CPU's available to the VM process (as can be configured by cpu_sets) since JDK 9. In JDK 10 quotas/shares are also used to approximate a notion of "available processors". David > Regards, > Kedar > > On Thu, Feb 15, 2018 at 10:04 AM, Bob Vandette > wrote: > >> >>> On Feb 15, 2018, at 12:52 PM, Volker Simonis >> wrote: >>> >>> Sounds cool! >>> >>> Is this JEP only about providing the corresponding API or also about >>> using it internally (within HotSpot and class library) to better adopt >>> to environment the JVM is running in? >> >> Thanks Volker. >> >> I integrated JDK-8146115 into JDK 10 which allows Hotspot to adapt to the >> container it?s running in. The configuration that is examined includes >> memory limits, >> cpu count including cpu shares and quotas. >> >> This JEP?s main focus is to provide an internal API that can be used by >> JDK tools >> such as JFR and JMX to export container statistics and configuration >> data. This JEP >> does not include the JFR and JMX implementations. They will hopefully >> follow shortly after. >> >> Bob. >> >>> >>> Either way, looking forward to see (and test) the first implementation >> bits! >>> >>> Regards, >>> Volker >>> >>> >>> On Thu, Feb 15, 2018 at 6:07 PM, Bob Vandette >> wrote: >>>> I?d like to re-propose the following JEP that will enhance the Java >> runtime to be more container aware. >>>> This will add an Internal Java API that will provide container specific >> statistics. Some of the initial goals >>>> of the previous JEP proposal has been integrated into JDK 10 under an >> RFE (JDK-8146115). >>>> This JEP is now focused on providing a Java API that exports Container >> runtime configuration and metrics. >>>> >>>> Since the scope of this JEP have changed, I?m re-submitting it for >> comment and endorsement. >>>> >>>> >>>> JEP Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8182070 >>>> >>>> Here?s a Text dump of the JEP contents for your convenience: >>>> >>>> Summary >>>> ------- >>>> >>>> Container aware Java runtime >>>> >>>> Goals >>>> ----- >>>> >>>> Provide an internal API that can be used to extract container specific >> configuration and runtime statistics. This JEP will only support Docker on >> Linux-x64 although the design should be flexible enough to allow support >> for other platforms and container technologies. The initial focus will be >> on Linux cgroups technology so that we will be able to easily support other >> container technologies running on Linux in addition to Docker. >>>> >>>> Non-Goals >>>> --------- >>>> >>>> It is not a goal of this JEP to support any platform other than Docker >> container technology running on Linux x64. >>>> >>>> Success Metrics >>>> --------------- >>>> >>>> Success will be measured by the improvement in information that will be >> available to tools which visualize resource usage of containers that are >> running Java processes. >>>> >>>> Motivation >>>> ---------- >>>> >>>> Container technology is becoming more and more prevalent in Cloud based >> applications. The Cloud Serverless application programming model motivates >> developers to split large monolithic applications into 100s of smaller >> pieces each running in thier own container. This move increases the >> importance of the observability of each running container process. Adding >> the proposed set of APIs will allow more details related to each container >> process to be made available to external tools thereby improving the >> observability. >>>> >>>> Description >>>> ----------- >>>> >>>> This enhancement will be made up of the following work items: >>>> >>>> A. Detecting if Java is running in a container. >>>> >>>> The Java runtime, as well as any tests that we might write for this >> feature, will need to be able to detect that the current Java process is >> running in a container. A new API will be made available for this purpose. >>>> >>>> B. Exposing container resource limits, configuration and runtime >> statistics. >>>> >>>> There are several configuration options and limits that can be imposed >> upon a running container. Not all of these >>>> are important to a running Java process. We clearly want to be able to >> detect how many CPUs have been allocated to our process along with the >> maximum amount of memory that the process has been allocated but there are >> other options that we might want to base runtime decisions on. >>>> >>>> In addition, since Container typically impose limits on system >> resources, they also provide the ability to easily access the amount of >> consumption of these resources. The goal is to provide this information in >> addition to the configuration data. >>>> >>>> I propose adding a new jdk.internal.Platform class that will allow >> access to this information. >>>> >>>> Here are some of the types of configuration and consumption statistics >> that would be made available: >>>> >>>> isContainerized >>>> Memory Limit >>>> Total Memory Limit >>>> Soft Memory Limit >>>> Max Memory Usage >>>> Current Memory Usage >>>> Maximum Kernel Memory >>>> CPU Shares >>>> CPU Period >>>> CPU Quota >>>> Number of CPUs >>>> CPU Sets >>>> CPU Set Memory Nodes >>>> CPU Usage >>>> CPU Usage Per CPU >>>> Block I/O Weight >>>> Block I/O Device Weight >>>> Device I/O Read Rate >>>> Device I/O Write Rate >>>> OOM Kill Enabled >>>> OOM Score Adjustment >>>> Memory Swappiness >>>> Shared Memory Size >>>> >>>> Alternatives >>>> ------------ >>>> >>>> There are a few existing tools available to extract some of the same >> container statistics. These tools could be used instead. The benefit of >> providing a core Java internal API is that this information can be expose >> by current Java serviceability tools such as JMX and JFR along side other >> JVM specific information. >>>> >>>> Testing >>>> ------- >>>> >>>> Docker/container specific tests should be added in order to validate >> the functionality being provided with this JEP. >>>> >>>> Risks and Assumptions >>>> --------------------- >>>> >>>> Docker is currently based on cgroups v1. Cgroups v2 is also available >> but is incomplete and not yet supported by Docker. It's possible that v2 >> could replace v1 in an incompatible way rendering this work unusable until >> it is upgraded. >>>> >>>> Other alternative container technologies based on hypervisors are being >> developed that could replace the use of cgroups for container isloation. >>>> >>>> Dependencies >>>> ----------- >>>> >>>> None at this time. >>>> >> >> From david.holmes at oracle.com Fri Feb 16 02:44:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 12:44:13 +1000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Message-ID: Looks fine to me. Thanks, David ----- On 15/02/2018 11:35 PM, Robin Westberg wrote: > Hi again, > > Here?s the (hopefully) final version: > > Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ > Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ > > Best regards, > Robin > >> On 14 Feb 2018, at 16:15, Robin Westberg wrote: >> >> Hi Roger, >> >>> On 13 Feb 2018, at 16:17, Roger Riggs wrote: >>> >>> Hi Robin, >>> >>> It looks like the status argument to BeforeHalt is discarded in JVM_BeforeHalt >>> and is not inserted into the event. >>> That suggests it should be removed all the way back to Shutdown.beforeHalt. >> >> You are right, my thinking was that the interface wouldn?t need to be changed if we decided to revisit the event in the future and add the status code. But that is perhaps unlikely, so can certainly remove the argument for now. >> >> Best regards, >> Robin >> >>> >>> Roger >>> >>> >>> >>> On 2/13/2018 9:59 AM, Robin Westberg wrote: >>>> Hi Alan, >>>> >>>>> On 12 Feb 2018, at 09:02, Alan Bateman wrote: >>>>> >>>>> >>>>> >>>>> On 12/02/2018 07:07, David Holmes wrote: >>>>>>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>>>>>> >>>>>>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>>>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>>>> This looks much cleaner/neater to me - thanks. >>>>>> >>>>> The updates to Runtime/Shutdown seems okay. >>>> Thanks for reviewing! >>>> >>>> Best regards, >>>> Robin >>>> >>>>> -Alan >>> >> > From martinrb at google.com Fri Feb 16 03:13:08 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 15 Feb 2018 19:13:08 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: <59507b7d-4797-21aa-b084-1687583b78fe@oracle.com> References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> <59507b7d-4797-21aa-b084-1687583b78fe@oracle.com> Message-ID: On Thu, Feb 15, 2018 at 3:20 PM, mandy chung wrote: > > I have posted RFR to remove runFinalizersOnExit. I think your patch > would become cleaner as you may keep the remove method or inline in > runFinalizer method as your original patch has. > > Do you want to wait until JDK-8198249? or proceed this patch without > waiting? > I'd like to commit this patch now, in part because we're actually backporting this as a local patch to make race detection easier. Regarding remove(), I think it's actually cleaner inline, but if it's kept a separate method, then make it return a boolean, as with Collection remove methods. From stuart.marks at oracle.com Fri Feb 16 00:18:40 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 15 Feb 2018 16:18:40 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> Message-ID: On 2/15/18 3:06 PM, mandy chung wrote: > Runtime.runFinalizersOnExit has been deprecated since 1.2 (1998) > and deprecated for removal in JDK 9.? We analyzed the maven central > artifacts few years ago that show very few uses of it.? I also > survey a recent version of most of the maven artifacts that references > runFinalizatsOnExit no longer references it.? I propose to remove > Runtime.runFinalizersOnExit and System.runFinalizersOnExit methods > in JDK 11. > > CSR: https://bugs.openjdk.java.net/browse/JDK-8198250 I've added myself to the reviewers list of this CSR. Note that you've listed the Compatibility Risk as Medium, but the notes are in support of a low risk level. Given this analysis, I'd be comfortable with Compatibility Risk being set to Low if you want to do that. > Webrev: > ? http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.00/ Finalizer.java: > 111 This method is used by both runFinalization. Delete "both". Thanks for updating the docs and comments in jdeprscan. Overall looks good. Someone else should look at the native code, though, since I'm not too familiar with it. s'marks From mandy.chung at oracle.com Fri Feb 16 03:20:26 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Feb 2018 19:20:26 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> Message-ID: On 2/15/18 6:11 PM, David Holmes wrote: > Hi Mandy, > > Good to see this go. A few minor comments. > > First I've added comments on the CSR as some of the doc changes don't > read correctly. > Thanks for catching it.? What do you think about this revision: *

All registered {@link #addShutdownHook shutdown hooks}, if any, * are started in some unspecified order and allowed to run concurrently * until they finish. Once this is done the virtual machine * {@link #halt halts}. * *

If this method is invoked after the virtual machine has started * shutdown hooks then if shutdown hooks have already been run and * the status is nonzero then this method halts the virtual machine * with the given status code; otherwise, this method blocks indefinitely * (i.e. if shutdown hooks are being run or if shutdown hooks have already * been run and the status is zero). I'll update the CSR once we agree on one version. > src/hotspot/share/runtime/thread.cpp > > This comment doesn't read correctly: > > !???? // won't be run.? Note that if a shutdown hook was registered > ????? // was called, the Shutdown class would have already been loaded > !???? // (Runtime.addShutdownHook will load it). > > delete "was called, " > Deleted. > --- > > src/java.base/share/classes/java/lang/Shutdown.java > > This was a bit confusing. :) I wasn't at all sure you needed the > COMPLETED state (which is really a HOOKS_HAVE_BEEN_STARTED state). But > I see it allows for a second exit( preference to the initial exit() call (whether non-zero or not). That > seems to maintain existing behaviour. Yes this maintains the existing behavior. > > --- > > src/java.base/share/classes/java/lang/ref/Finalizer.java > > ????? private void remove() { > ????????? synchronized (lock) { > ????????????? if (unfinalized == this) { > -???????????????? if (this.next != null) { > ????????????????????? unfinalized = this.next; > -???????????????? } else { > -???????????????????? unfinalized = this.prev; > -???????????????? } > > This seems unrelated to this change. Is this the optimization Martin > proposed? > The only case when this.next == null and this.prev != null? could happen when runAllFinalizers were involved.? This case is no longer needed. > ! This method is used by both runFinalization. > ???????? The former method invokes all pending finalizers, while the > latter > ???????? invokes all uninvoked finalizers if on-exit finalization has > been > ???????? enabled. > > As Stuart said remove "both" in the modified line. But the following > line also needs changing or deleting. As does the one after that: > > ?116??????? These two methods could have been implemented by > offloading their work > > as there are no longer two methods. > Good catch.? Updated. > Otherwise this all seems okay. I was surprised none of this really > impacted the VM. :) > This is all done in the library side.? The VM did have the run_finalization_at_exit function but never used. thanks Mandy From martinrb at google.com Fri Feb 16 03:29:52 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 15 Feb 2018 19:29:52 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> Message-ID: This also changes the handling of (undeprecated) method runFinalization and that's obviously related but could be a separate changeset. We'll need to merge with my own pending change to Finalizer.java. From mandy.chung at oracle.com Fri Feb 16 03:31:47 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Feb 2018 19:31:47 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> <59507b7d-4797-21aa-b084-1687583b78fe@oracle.com> Message-ID: On 2/15/18 7:13 PM, Martin Buchholz wrote: > > > On Thu, Feb 15, 2018 at 3:20 PM, mandy chung > wrote: > > > I have posted RFR to remove runFinalizersOnExit.?? I think your > patch would become cleaner as you may keep the remove method or > inline in runFinalizer method as your original patch has. > > Do you want to wait until JDK-8198249?? or proceed this patch > without waiting? > > > I'd like to commit this patch now, in part because we're actually > backporting this as a local patch to make race detection easier. > > Regarding remove(), I think it's actually cleaner inline, but if it's > kept a separate method, then make it return a boolean, as with > Collection remove methods. > http://cr.openjdk.java.net/~martin/webrevs/jdk/Finalizer-data-race/ I reviewed this version.? It looks okay. With the removal of runFinalizersOnExit, I will inline the deregistration part as part of runFinalizers after I rebase with your change.? Hope that's okay with you. Mandy From martinrb at google.com Fri Feb 16 03:45:17 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 15 Feb 2018 19:45:17 -0800 Subject: RFR: 8197812: (ref) Data race in Finalizer In-Reply-To: References: <35bd3df4-b119-1469-16cb-fe706b7b553b@gmail.com> <8efcb115-d8fa-cac4-64c8-2ff2343c32fe@gmail.com> <4be7eba7-7188-bea2-a33d-9eebfe6ddd39@gmail.com> <421e83f2-b8a4-7add-819a-de13a005906c@gmail.com> <36ccadd4-f3b6-922e-c861-4763b4e90382@gmail.com> <226e5184-abff-df6a-c8d9-43b7e2710ac4@gmail.com> <2af68d5a-afb6-e5a3-93a5-df14ca0305d8@oracle.com> <59507b7d-4797-21aa-b084-1687583b78fe@oracle.com> Message-ID: Thanks, Mandy, changeset pushed. We are in agreement. On Thu, Feb 15, 2018 at 7:31 PM, mandy chung wrote: > > > On 2/15/18 7:13 PM, Martin Buchholz wrote: > > > > On Thu, Feb 15, 2018 at 3:20 PM, mandy chung > wrote: > >> >> I have posted RFR to remove runFinalizersOnExit. I think your patch >> would become cleaner as you may keep the remove method or inline in >> runFinalizer method as your original patch has. >> >> Do you want to wait until JDK-8198249? or proceed this patch without >> waiting? >> > > I'd like to commit this patch now, in part because we're actually > backporting this as a local patch to make race detection easier. > > Regarding remove(), I think it's actually cleaner inline, but if it's kept > a separate method, then make it return a boolean, as with Collection remove > methods. > > > http://cr.openjdk.java.net/~martin/webrevs/jdk/Finalizer-data-race/ > > I reviewed this version. It looks okay. > > With the removal of runFinalizersOnExit, I will inline the deregistration > part as part of runFinalizers after I rebase with your change. Hope that's > okay with you. > > Mandy > From mandy.chung at oracle.com Fri Feb 16 04:09:18 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Feb 2018 20:09:18 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> Message-ID: On 2/15/18 7:29 PM, Martin Buchholz wrote: > This also changes the handling of (undeprecated) method > runFinalization and that's obviously related but could be a separate > changeset. > I agree I should separate the runFinalization change.? I leave it in this webrev for now.? I will send out the final webrevs before I push (I am feeling under the weather a bit now). > We'll need to merge with my own pending change to Finalizer.java. Merged.? New version: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.02/ Mandy From david.holmes at oracle.com Fri Feb 16 05:14:33 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 15:14:33 +1000 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> Message-ID: <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> Hi Mandy, On 16/02/2018 1:20 PM, mandy chung wrote: > On 2/15/18 6:11 PM, David Holmes wrote: >> Hi Mandy, >> >> Good to see this go. A few minor comments. >> >> First I've added comments on the CSR as some of the doc changes don't >> read correctly. >> > > Thanks for catching it.? What do you think about this revision: > > *

All registered {@link #addShutdownHook shutdown hooks}, if any, > * are started in some unspecified order and allowed to run concurrently > * until they finish. Once this is done the virtual machine > * {@link #halt halts}. This sounds good. > *

If this method is invoked after the virtual machine has started > * shutdown hooks then if shutdown hooks have already been run and > * the status is nonzero then this method halts the virtual machine > * with the given status code; otherwise, this method blocks indefinitely > * (i.e. if shutdown hooks are being run or if shutdown hooks have already > * been run and the status is zero). This is still confusing to me (and I don't claim the original is not also confusing!). Do we actually care/distinguish between "hooks have been started" and "hooks have already been run" - which implies all user-defined hooks have finished? I think the only thing that is significant is whether hooks have been started, and then what the status code is. So I'd suggest simply: *

If this method is invoked after the virtual machine has started * shutdown hooks and the status is nonzero then this method halts the * virtual machine with the given status code. Otherwise, this method * blocks indefinitely. ? > I'll update the CSR once we agree on one version. > >> src/hotspot/share/runtime/thread.cpp >> >> This comment doesn't read correctly: >> >> !???? // won't be run.? Note that if a shutdown hook was registered >> ????? // was called, the Shutdown class would have already been loaded >> !???? // (Runtime.addShutdownHook will load it). >> >> delete "was called, " >> > > Deleted. > >> --- >> >> src/java.base/share/classes/java/lang/Shutdown.java >> >> This was a bit confusing. :) I wasn't at all sure you needed the >> COMPLETED state (which is really a HOOKS_HAVE_BEEN_STARTED state). But >> I see it allows for a second exit(> preference to the initial exit() call (whether non-zero or not). That >> seems to maintain existing behaviour. > > Yes this maintains the existing behavior. The more I look at this the more I think COMPLETED is not the right term here. >> >> --- >> >> src/java.base/share/classes/java/lang/ref/Finalizer.java >> >> ????? private void remove() { >> ????????? synchronized (lock) { >> ????????????? if (unfinalized == this) { >> -???????????????? if (this.next != null) { >> ????????????????????? unfinalized = this.next; >> -???????????????? } else { >> -???????????????????? unfinalized = this.prev; >> -???????????????? } >> >> This seems unrelated to this change. Is this the optimization Martin >> proposed? >> > > The only case when this.next == null and this.prev != null? could happen > when runAllFinalizers were involved.? This case is no longer needed. Okay - this is far from obvious though. >> ! This method is used by both runFinalization. >> ???????? The former method invokes all pending finalizers, while the >> latter >> ???????? invokes all uninvoked finalizers if on-exit finalization has >> been >> ???????? enabled. >> >> As Stuart said remove "both" in the modified line. But the following >> line also needs changing or deleting. As does the one after that: >> >> ?116??????? These two methods could have been implemented by >> offloading their work >> >> as there are no longer two methods. >> > > Good catch.? Updated. 102 * It could have been implemented by offloading their work to the s/their/the/ 105 * invokers of these methods from a stalled or deadlocked finalizer thread. s/these methods/that method/ All other updates seem okay. I did have one further thought - in Runtime does this change: public void runFinalization() { ! SharedSecrets.getJavaLangRefAccess().runFinalization(); } affect the classloading/initialization order at all? Thanks, David > >> Otherwise this all seems okay. I was surprised none of this really >> impacted the VM. :) >> > > This is all done in the library side.? The VM did have the > run_finalization_at_exit function but never used. > > thanks > Mandy > From scolebourne at joda.org Fri Feb 16 10:59:22 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 16 Feb 2018 10:59:22 +0000 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: On 15 February 2018 at 20:52, Louis Wasserman wrote: > Based on this data, I'd recommend providing one and only one method of this > type: String.repeat(int). Only adding the one instance method seems like the best plan in this case. Stephen From Alan.Bateman at oracle.com Fri Feb 16 11:46:48 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 16 Feb 2018 11:46:48 +0000 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> Message-ID: <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> On 15/02/2018 17:20, Jim Laskey wrote: > This is a pre-CSR code review [1] for String repeat methods (Enhancement). > > The proposal is to introduce four new methods; > > 1. public String repeat(final int count) > 2. public static String repeat(final char ch, final int count) > 3. public static String repeat(final int codepoint, final int count) > 4. public static String repeat(final CharSequence seq, final int count) > Just catching up on this thread and it's hard to see where the bidding is currently at. Are you planning to send an updated proposal, a list of methods is fine, even if it's just one, is okay (implementation can follow later). -Alan From Alan.Bateman at oracle.com Fri Feb 16 11:49:54 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 16 Feb 2018 11:49:54 +0000 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <5A8602B5.3050207@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <1035679069.1995730.1518725040338.JavaMail.zimbra@u-pem.fr> <5A8602B5.3050207@oracle.com> Message-ID: <1a20cd54-227a-22bb-8894-b45d4dc25667@oracle.com> On 15/02/2018 21:59, Joseph D. Darcy wrote: > : > > My general recommendation if the code review and CSR review are to be > serialized is to estimate which one is more likely to generate > feedback that might modify the proposal and run though that process > first. Since there has already been feedback leading to updates of > this proposal, code review first in this case seems like a reasonable > choice :-) I agree, esp. for API additions where it is highly likely there will be several iterations as the API is discussed. It's pointless finalizing a CSR without getting wider feedback first. -Alan. From nishit.jain at oracle.com Fri Feb 16 11:50:04 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Fri, 16 Feb 2018 17:20:04 +0530 Subject: [11] RFR JDK-8190904: Incorrect currency instance returned by java.util.Currency.getInstance() Message-ID: <84420a48-d666-c42e-e071-da37726e3e2a@oracle.com> Hi, Please review the fix for JDK-8190904. Bug: https://bugs.openjdk.java.net/browse/JDK-8190904 Webrev: http://cr.openjdk.java.net/~nishjain/8190904/webrev.07/ CSR: https://bugs.openjdk.java.net/browse/JDK-8196835 Issue: The currency superseding feature gives the possibility of allowing two similar Currency instances for a currency code, which is against the principle that "There can never more than one Currency instance for any given currency". Fix: Modify the superseding feature to not allow multiple entries with same currency code but different numeric and/or minor unit. These are ignored as inconsistent entries. Also, If there is any ISO 4217 currency data with same currency code as the currency entry given in the properties file, then the existing Currency data should be updated with the given currency values. Regards, Nishit Jain From Alan.Bateman at oracle.com Fri Feb 16 11:52:13 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 16 Feb 2018 11:52:13 +0000 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> Message-ID: On 15/02/2018 20:28, yumin qi wrote: > : > ? ? Since the property string contains non-normalized characters, it > crashed in native canonicalize. > ? ? I believe user.dir from the system is normalized, so it is OK but > after it is changed like "/home/a/b/c/", it crashed. > > ? ? Now with using cached "user.dir", the problem is gone. > ? ? : > > ? ? So the changes in resolve should be removed. > > ? ? Since the bug is talking about the crash, the real reason is > user.dir should not be changed, how about changing description to > ? ? 8194154: System property user.dir should not be changed. > > ? ? ?The test case renamed to: UserDirChangedTest.java? ?? > Thanks for the confirming. Yes, changing the bug description and test should be good and if you can post an updated webrev then I assume we can wrap this one up quickly. -Alan From Alan.Bateman at oracle.com Fri Feb 16 12:12:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 16 Feb 2018 12:12:11 +0000 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> Message-ID: <35de3053-cde9-8aa6-949a-62b4331899e0@oracle.com> On 16/02/2018 04:09, mandy chung wrote: > > Merged.? New version: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.02/ Good to see this change. One comment on Runtime.exec is that the 3rd paragraph of the updated spec is difficult to read (too many "ifs" and "thens"). What would you think about starting it with: "If this method is invoked after the virtual machine has started the shutdown hooks, the shutdown hooks have already run, and the status is nonzero, then this method halts the virtual machine with the given status code". -Alan From david.holmes at oracle.com Fri Feb 16 12:54:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 22:54:43 +1000 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <35de3053-cde9-8aa6-949a-62b4331899e0@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <35de3053-cde9-8aa6-949a-62b4331899e0@oracle.com> Message-ID: <31f716ef-cb0f-48b3-5a9e-5026cfe4af1e@oracle.com> On 16/02/2018 10:12 PM, Alan Bateman wrote: > On 16/02/2018 04:09, mandy chung wrote: >> >> Merged.? New version: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.02/ > Good to see this change. > > One comment on Runtime.exec is that the 3rd paragraph of the updated > spec is difficult to read (too many "ifs" and "thens"). What would you > think about starting it with: "If this method is invoked after the > virtual machine has started the shutdown hooks, the shutdown hooks have > already run, and the status is nonzero, then this method halts the > virtual machine with the given status code". I already suggested something even simpler :) If the hooks have already run then they must have been started so the first part is redundant. But I don't think we care about whether they have finished running, only that they have started. David > -Alan From james.laskey at oracle.com Fri Feb 16 13:10:34 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 16 Feb 2018 09:10:34 -0400 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> Message-ID: We?re going with the one instance method (Louis clinched it.) with recommended enhancements and not touching CharSequence. Working it up now. ? Jim > On Feb 16, 2018, at 7:46 AM, Alan Bateman wrote: > > On 15/02/2018 17:20, Jim Laskey wrote: >> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >> >> The proposal is to introduce four new methods; >> >> 1. public String repeat(final int count) >> 2. public static String repeat(final char ch, final int count) >> 3. public static String repeat(final int codepoint, final int count) >> 4. public static String repeat(final CharSequence seq, final int count) >> > Just catching up on this thread and it's hard to see where the bidding is currently at. Are you planning to send an updated proposal, a list of methods is fine, even if it's just one, is okay (implementation can follow later). > > -Alan > > > > > > > > > > From Ulf.Zibis at CoSoCo.de Fri Feb 16 13:58:27 2018 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Fri, 16 Feb 2018 14:58:27 +0100 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: <5A861702.7090102@oracle.com> References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> <5A861702.7090102@oracle.com> Message-ID: Hi, what is the difference of: - class StringCoding - class StringCoder - class StringCoding.StringDecoder - class StringCoding.StringEncoder Why we need so much variants of redundant code? I think it would be useful to have some explanation in the javadoc. Missing indentation in StringCoder.java : 322 if (enc == null) { 323 enc = cs.newEncoder() 324 .onMalformedInput(CodingErrorAction.REPLACE) 325 .onUnmappableCharacter(CodingErrorAction.REPLACE); 326 } -Ulf Am 16.02.2018 um 00:25 schrieb Xueming Shen: > > http://cr.openjdk.java.net/~sherman/8021560/webrev.v2 > > -Sherman > > > From peter.levart at gmail.com Fri Feb 16 14:52:06 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 16 Feb 2018 15:52:06 +0100 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> Message-ID: <203b784f-b173-faea-dd32-8472b11f3631@gmail.com> Hi Mandy, On 02/16/2018 05:09 AM, mandy chung wrote: > > > On 2/15/18 7:29 PM, Martin Buchholz wrote: >> This also changes the handling of (undeprecated) method >> runFinalization and that's obviously related but could be a separate >> changeset. >> > > I agree I should separate the runFinalization change.? I leave it in > this webrev for now.? I will send out the final webrevs before I push > (I am feeling under the weather a bit now). > >> We'll need to merge with my own pending change to Finalizer.java. > > Merged.? New version: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.02/ > > Mandy I think that the following now never applies (in Finalizer) since ReferenceQueue never returns the same Reference object more than once: ? 71???????????? if (this.next == this)????? // already finalized ? 72???????????????? return; You could change it into an assert to be comfortable. What do you think? Regards, Peter From xueming.shen at oracle.com Fri Feb 16 15:28:53 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 16 Feb 2018 07:28:53 -0800 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> <5A861702.7090102@oracle.com> Message-ID: <5A86F8B5.5070006@oracle.com> On 2/16/18, 5:58 AM, Ulf Zibis wrote: > Hi, > > what is the difference of: > - class StringCoding > - class StringCoder > - class StringCoding.StringDecoder > - class StringCoding.StringEncoder > > Why we need so much variants of redundant code? I think it would be > useful to have some explanation in the javadoc. > Ulf, Ignore StringCoder for now. It's a experimenting version to replace StringCoding, to reduce "some" redundant code. -Sherman From robin.westberg at oracle.com Fri Feb 16 16:00:19 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 16 Feb 2018 17:00:19 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <07ea69ac-7b62-992f-5275-97c990906a2b@Oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> <07ea69ac-7b62-992f-5275-97c990906a2b@Oracle.com> Message-ID: <447A6370-7C4B-46D9-8CAA-40BAE3454819@oracle.com> Hi Roger, > On 15 Feb 2018, at 17:00, Roger Riggs wrote: > > Hi Robin, > > Looks fine to me. Thanks for reviewing! > (How is this tested?, Normal, exceptional, etc.) The tests are part of the (currently closed) JFR tests. Best regards, Robin > > Thanks, Roger > > > On 2/15/2018 8:35 AM, Robin Westberg wrote: >> Hi again, >> >> Here?s the (hopefully) final version: >> >> Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ >> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ >> >> Best regards, >> Robin >> >>> On 14 Feb 2018, at 16:15, Robin Westberg > wrote: >>> >>> Hi Roger, >>> >>>> On 13 Feb 2018, at 16:17, Roger Riggs > wrote: >>>> >>>> Hi Robin, >>>> >>>> It looks like the status argument to BeforeHalt is discarded in JVM_BeforeHalt >>>> and is not inserted into the event. >>>> That suggests it should be removed all the way back to Shutdown.beforeHalt. >>> >>> You are right, my thinking was that the interface wouldn?t need to be changed if we decided to revisit the event in the future and add the status code. But that is perhaps unlikely, so can certainly remove the argument for now. >>> >>> Best regards, >>> Robin >>> >>>> >>>> Roger >>>> >>>> >>>> >>>> On 2/13/2018 9:59 AM, Robin Westberg wrote: >>>>> Hi Alan, >>>>> >>>>>> On 12 Feb 2018, at 09:02, Alan Bateman > wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 12/02/2018 07:07, David Holmes wrote: >>>>>>>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>>>>>>> >>>>>>>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>>>>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>>>>> This looks much cleaner/neater to me - thanks. >>>>>>> >>>>>> The updates to Runtime/Shutdown seems okay. >>>>> Thanks for reviewing! >>>>> >>>>> Best regards, >>>>> Robin >>>>> >>>>>> -Alan >>>> >>> >> > From robin.westberg at oracle.com Fri Feb 16 16:00:27 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 16 Feb 2018 17:00:27 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Message-ID: Hi Mandy, > On 15 Feb 2018, at 20:10, mandy chung wrote: > > Hi Robin, > > Do you want a shutdown event for every call to Runtime::exit regardless where it is in the shutdown sequence? or do you expect to get an event of the first thread calling JVM_Halt? or the first thread starting the shutdown sequence but it may not be the thread halting? I was aiming for an event from the first thread starting the shutdown sequence, as I think that?s probably the one you are looking for if you are diagnosing an unexpected shutdown. Any event generated after that point may not make it into the recording as the tracing framework shuts down from a shutdown hook. If finalizers on exit go away, is it still possible that the thread starting the shutdown sequence isn?t the one eventually calling JVM_Halt (in the absence of calls to Runtime.halt)? Best regards, Robin > > Mandy > > On 2/15/18 5:35 AM, Robin Westberg wrote: >> Hi again, >> >> Here?s the (hopefully) final version: >> >> Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ >> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ >> >> Best regards, >> Robin > From naoto.sato at oracle.com Fri Feb 16 16:51:53 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 16 Feb 2018 08:51:53 -0800 Subject: [11] RFR JDK-8190904: Incorrect currency instance returned by java.util.Currency.getInstance() In-Reply-To: <84420a48-d666-c42e-e071-da37726e3e2a@oracle.com> References: <84420a48-d666-c42e-e071-da37726e3e2a@oracle.com> Message-ID: <7a6087a3-746d-d5a5-f630-1160d09f1e92@oracle.com> Looks good to me. Naoto On 2/16/18 3:50 AM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8190904. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8190904 > Webrev: http://cr.openjdk.java.net/~nishjain/8190904/webrev.07/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8196835 > > Issue: The currency superseding feature gives the possibility of > allowing two similar Currency instances for a currency code, which is > against the principle that "There can never more than one Currency > instance for any given currency". > > Fix: Modify the superseding feature to not allow multiple entries with > same currency code but different numeric and/or minor unit. These are > ignored as inconsistent entries. Also, If there is any ISO 4217 currency > data with same currency code as the currency entry given in the > properties file, then the existing Currency data should be updated with > the given currency values. > > > Regards, > Nishit Jain From Alan.Bateman at oracle.com Fri Feb 16 14:14:44 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 16 Feb 2018 14:14:44 +0000 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> Message-ID: On 15/02/2018 21:55, Stuart Marks wrote: > : > > I'd also suggest adding a CharBuffer constructor: > > ??? public String(CharBuffer cbuf) > > This would be semantically equivalent to > > ??? public String(char[] value, int offset, int count) > > except using the chars from the CharBuffer between the buffer's > position and its limit. CharBuffer::toString already does this so adding this constructor may not be as useful as it initially looks. -Alan From aleksej.efimov at oracle.com Fri Feb 16 17:53:52 2018 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Fri, 16 Feb 2018 17:53:52 +0000 Subject: [11] RFR: (JAXP) 8038043: Xerces Update: XInclude update Message-ID: Hi, Please, help to review the update of XInclude related classes from the Apache Xerces 2.11.0 source. JBS: ??? https://bugs.openjdk.java.net/browse/JDK-8038043 The webrev: ??? http://cr.openjdk.java.net/~aefimov/8038043/11/00/ New regression test has been added to check the updated reporting of invalid bytes encountered during the parsing and inclusion of XML documents. New test and other regression tests shows no failures. With Best Regards, Aleksei From mark.reinhold at oracle.com Fri Feb 16 18:02:02 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 16 Feb 2018 10:02:02 -0800 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string In-Reply-To: References: <20180214072629.629736428@eggemoggin.niobe.net> Message-ID: <20180216100202.266728211@eggemoggin.niobe.net> 2018/2/14 8:04:15 -0800, volker.simonis at gmail.com: > On Wed, Feb 14, 2018 at 4:26 PM, mark.reinhold at oracle.com wrote: >> This is a bug in the specification, not the implementation. As I just >> wrote in a comment on 8197927: >> >> JEP 322 expresses the intended behavior: If `--with-vendor-version` is >> not specified at build time then the `java.vendor.version` property has >> no value. (That's different from the empty string, which is a value.) >> The bug is in the specification, which should be revised so as not to >> require that this property have a value. Fixing the spec is not >> critical for 10; I suggest downgrading 8197927 to P2 and targeting 11. > > What you mean by "not require the 'java.vendor.version' property to > have a value"? We can not set a system property to NULL because > System.setProperty() will throw a NPE in that case. So either the > specification requires 'java.vendor.version' as mandatory (as it is > now) in which case it does need to have a value different from NULL. > Or we change the specification such that 'java.vendor.version' isn't a > mandatory property any more. But it is not possible to have > 'java.vendor.version' as mandatory property with "no value" as > envisioned by the JEP. Of course it's possible. The specification need merely say that `java.vendor.version` is a standard system property that may, or may not, have a value. (Or, if you like, whose value may be `null`.) > Do you propose to change the specification such that > 'java.vendor.version' isn't a mandatory property any more? That would > be fine for me and I agree that we could target this bug to 11 if > that's what you propose. We could actually list "java.vendor.version" > under "Implementation Note:" which contains some additional, > non-mandatory properties. It'd be a standard property, not an implementation-specific property. Feel free to reassign 8197927 to me and I'll fix it in 11, since it was my oversight in the first place. - Mark From mandy.chung at oracle.com Fri Feb 16 18:24:24 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 16 Feb 2018 10:24:24 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <31f716ef-cb0f-48b3-5a9e-5026cfe4af1e@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <35de3053-cde9-8aa6-949a-62b4331899e0@oracle.com> <31f716ef-cb0f-48b3-5a9e-5026cfe4af1e@oracle.com> Message-ID: <8ad2b85c-5682-67f1-b7b8-661d2b5253df@oracle.com> On 2/16/18 4:54 AM, David Holmes wrote: > On 16/02/2018 10:12 PM, Alan Bateman wrote: >> On 16/02/2018 04:09, mandy chung wrote: >>> >>> Merged.? New version: >>> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.02/ >> Good to see this change. >> >> One comment on Runtime.exec is that the 3rd paragraph of the updated >> spec is difficult to read (too many "ifs" and "thens"). What would >> you think about starting it with: "If this method is invoked after >> the virtual machine has started the shutdown hooks, the shutdown >> hooks have already run, and the status is nonzero, then this method >> halts the virtual machine with the given status code". > > I already suggested something even simpler :) If the hooks have > already run then they must have been started so the first part is > redundant. But I don't think we care about whether they have finished > running, only that they have started. > I like the simplified version that David suggests.? The phrase about "already run" is redundant there.? I will go with this version and finalize the CSR. ? *

If this method is invoked after the virtual machine has started ? * shutdown hooks and the status is nonzero then this method halts the ? * virtual machine with the given status code. Otherwise, this method ? * blocks indefinitely. I'll look into the Shutdown implementation and address David's other comments separately. Thanks. Mandy From mandy.chung at oracle.com Fri Feb 16 18:26:26 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 16 Feb 2018 10:26:26 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <203b784f-b173-faea-dd32-8472b11f3631@gmail.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <203b784f-b173-faea-dd32-8472b11f3631@gmail.com> Message-ID: <486f1338-c8c0-5522-a88d-7ebede0ae62a@oracle.com> On 2/16/18 6:52 AM, Peter Levart wrote: > I think that the following now never applies (in Finalizer) since > ReferenceQueue never returns the same Reference object more than once: > > ? 71???????????? if (this.next == this)????? // already finalized > ? 72???????????????? return; > > You could change it into an assert to be comfortable. What do you think? Finalizer is loaded very early when VM starts and we can't use assert there (yes it's one of the bootstrapping issues in using new language features that we'd like to resolve.? We cleaned up several bootstrapping issues in JDK 9). I'll leave it as is. thanks Mandy From volker.simonis at gmail.com Fri Feb 16 18:59:57 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 16 Feb 2018 19:59:57 +0100 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string In-Reply-To: <20180216100202.266728211@eggemoggin.niobe.net> References: <20180214072629.629736428@eggemoggin.niobe.net> <20180216100202.266728211@eggemoggin.niobe.net> Message-ID: On Fri, Feb 16, 2018 at 7:02 PM, wrote: > 2018/2/14 8:04:15 -0800, volker.simonis at gmail.com: > > On Wed, Feb 14, 2018 at 4:26 PM, mark.reinhold at oracle.com wrote: > >> This is a bug in the specification, not the implementation. As I just > >> wrote in a comment on 8197927: > >> > >> JEP 322 expresses the intended behavior: If `--with-vendor-version` is > >> not specified at build time then the `java.vendor.version` property has > >> no value. (That's different from the empty string, which is a value.) > >> The bug is in the specification, which should be revised so as not to > >> require that this property have a value. Fixing the spec is not > >> critical for 10; I suggest downgrading 8197927 to P2 and targeting 11. > > > > What you mean by "not require the 'java.vendor.version' property to > > have a value"? We can not set a system property to NULL because > > System.setProperty() will throw a NPE in that case. So either the > > specification requires 'java.vendor.version' as mandatory (as it is > > now) in which case it does need to have a value different from NULL. > > Or we change the specification such that 'java.vendor.version' isn't a > > mandatory property any more. But it is not possible to have > > 'java.vendor.version' as mandatory property with "no value" as > > envisioned by the JEP. > > Of course it's possible. The specification need merely say that > `java.vendor.version` is a standard system property that may, or may > not, have a value. (Or, if you like, whose value may be `null`.) > > Sorry but I still don't get it. Do you agree that you can't assign NULL to a system property because you'll get a NPE? You could of course not assign it at all (as it is done now) in which case System.getProperty("java.vendor.version") would return NULL. But that means "it is not defined" which is different from "is has no value". You can still call System.getProperties().containsKey?("java.vendor.version") and it would return false which violates that specification because it mandates that a property with the "java.vendor.version" exists. Or am I missing something? > Do you propose to change the specification such that > > 'java.vendor.version' isn't a mandatory property any more? That would > > be fine for me and I agree that we could target this bug to 11 if > > that's what you propose. We could actually list "java.vendor.version" > > under "Implementation Note:" which contains some additional, > > non-mandatory properties. > > It'd be a standard property, not an implementation-specific property. > > Feel free to reassign 8197927 to me and I'll fix it in 11, since it was > my oversight in the first place. > > - Mark > From martinrb at google.com Fri Feb 16 19:11:31 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 16 Feb 2018 11:11:31 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <486f1338-c8c0-5522-a88d-7ebede0ae62a@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <203b784f-b173-faea-dd32-8472b11f3631@gmail.com> <486f1338-c8c0-5522-a88d-7ebede0ae62a@oracle.com> Message-ID: On Fri, Feb 16, 2018 at 10:26 AM, mandy chung wrote: > > > On 2/16/18 6:52 AM, Peter Levart wrote: > > I think that the following now never applies (in Finalizer) since > ReferenceQueue never returns the same Reference object more than once: > > 71 if (this.next == this) // already finalized > 72 return; > > You could change it into an assert to be comfortable. What do you think? > > > Finalizer is loaded very early when VM starts and we can't use assert > there (yes it's one of the bootstrapping issues in using new language > features that we'd like to resolve. We cleaned up several bootstrapping > issues in JDK 9). > > I'll leave it as is. > The small optimization to avoid checking for "already finalized" could be done in a follow-up. Before working on Finalizer.java I was not really aware of the global "unfinalized" data structure with a non-scalable lock. This could probably be made scalable with a large amount of effort, in the style of LongAdder. But probably not going to happen. From mark.reinhold at oracle.com Fri Feb 16 19:17:59 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 16 Feb 2018 11:17:59 -0800 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string In-Reply-To: References: <20180214072629.629736428@eggemoggin.niobe.net> <20180216100202.266728211@eggemoggin.niobe.net> Message-ID: <20180216111759.459999137@eggemoggin.niobe.net> 2018/2/16 10:59:57 -0800, volker.simonis at gmail.com: > On Fri, Feb 16, 2018 at 7:02 PM, mark.reinhold at oracle.com wrote: >> Of course it's possible. The specification need merely say that >> `java.vendor.version` is a standard system property that may, or may >> not, have a value. (Or, if you like, whose value may be `null`.) > > Sorry but I still don't get it. Do you agree that you can't assign NULL to > a system property because you'll get a NPE? I agree that `System.setProperty("java.vendor.version", null)` will throw an NPE, but I don't see how this fact is relevant. > You could of course not assign it at all (as it is done now) in which case > System.getProperty("java.vendor.version") would return NULL. But that means > "it is not defined" which is different from "is has no value". Let's not get caught up in fine philosophical distinctions between "not defined", "has no value", and `null`. Either `System::getProperty` returns a non-`null` value, or it doesn't. That's all that matters. > You can > still call System.getProperties().containsKey?("java.vendor.version") and > it would return false which violates that specification because it mandates > that a property with the "java.vendor.version" exists. The current specification mandates this. That's precisely the bug here. We can revise it so that it doesn't. - Mark From naoto.sato at oracle.com Fri Feb 16 19:22:24 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 16 Feb 2018 11:22:24 -0800 Subject: [11] RFR: JDK-8198228: Spec clarification: j.u.Locale.getDisplayName() Message-ID: Hello, Please review this small doc-only fix for this issue: https://bugs.openjdk.java.net/browse/JDK-8198228 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8198228/webrev.00/ It is just to clarify the field separator in locale display names. Corresponding CSR (8198236) is already approved. Naoto From mandy.chung at oracle.com Fri Feb 16 19:26:53 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 16 Feb 2018 11:26:53 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <203b784f-b173-faea-dd32-8472b11f3631@gmail.com> <486f1338-c8c0-5522-a88d-7ebede0ae62a@oracle.com> Message-ID: <3ad12e0a-140c-74f0-b67b-27a6f43633ea@oracle.com> On 2/16/18 11:11 AM, Martin Buchholz wrote: > > > The small optimization to avoid checking for "already finalized" could > be done in a follow-up. > > Before working on Finalizer.java I was not really aware of the global > "unfinalized" data structure with a non-scalable lock.? This could > probably be made scalable with a large amount of effort, in the style > of LongAdder. But probably not going to happen. You said what I will say - I don't see this will happen :) Mandy From paul.sandoz at oracle.com Fri Feb 16 19:47:05 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 16 Feb 2018 11:47:05 -0800 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: Message-ID: <39D8F43A-06BD-483B-8901-6F4444A8235F@oracle.com> Hi Adam, From reading the thread i cannot tell if this is part of a wider solution including some yet to be proposed HotSpot changes. As is i would be resistant to adding such standalone internal wrapper methods to Unsafe that have no apparent benefit within the OpenJDK itself since it's a maintenance burden. Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come from a DBB by looking at the call stack frame above the unsafe call? Thanks, Paul. > On Feb 14, 2018, at 3:32 AM, Adam Farley8 wrote: > > Hi All, > > Currently, diagnostic core files generated from OpenJDK seem to lump all > of the > native memory usages together, making it near-impossible for someone to > figure > out *what* is using all that memory in the event of a memory leak. > > The OpenJ9 VM has a feature which allows it to track the allocation of > native > memory for Direct Byte Buffers (DBBs), and to supply that information into > the > cores when they are generated. This makes it a *lot* easier to find out > what is using > all that native memory, making memory leak resolution less like some dark > art, and > more like logical debugging. > > To use this feature, there is a native method referenced in Unsafe.java. > To open > up this feature so that any VM can make use of it, the java code below > sets the > stage for it. This change starts letting people call DBB-specific methods > when > allocating native memory, and getting into the habit of using it. > > Thoughts? > > Best Regards > > Adam Farley > > P.S. Code: > > diff --git > a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > @@ -85,7 +85,7 @@ > // Paranoia > return; > } > - UNSAFE.freeMemory(address); > + UNSAFE.freeDBBMemory(address); > address = 0; > Bits.unreserveMemory(size, capacity); > } > @@ -118,7 +118,7 @@ > > long base = 0; > try { > - base = UNSAFE.allocateMemory(size); > + base = UNSAFE.allocateDBBMemory(size); > } catch (OutOfMemoryError x) { > Bits.unreserveMemory(size, cap); > throw x; > diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > @@ -632,6 +632,26 @@ > } > > /** > + * Allocates a new block of native memory for DirectByteBuffers, of > the > + * given size in bytes. The contents of the memory are > uninitialized; > + * they will generally be garbage. The resulting native pointer will > + * never be zero, and will be aligned for all value types. Dispose > of > + * this memory by calling {@link #freeDBBMemory} or resize it with > + * {@link #reallocateDBBMemory}. > + * > + * @throws RuntimeException if the size is negative or too large > + * for the native size_t type > + * > + * @throws OutOfMemoryError if the allocation is refused by the > system > + * > + * @see #getByte(long) > + * @see #putByte(long, byte) > + */ > + public long allocateDBBMemory(long bytes) { > + return allocateMemory(bytes); > + } > + > + /** > * Resizes a new block of native memory, to the given size in bytes. > The > * contents of the new block past the size of the old block are > * uninitialized; they will generally be garbage. The resulting > native > @@ -687,6 +707,27 @@ > } > > /** > + * Resizes a new block of native memory for DirectByteBuffers, to the > + * given size in bytes. The contents of the new block past the size > of > + * the old block are uninitialized; they will generally be garbage. > The > + * resulting native pointer will be zero if and only if the requested > size > + * is zero. The resulting native pointer will be aligned for all > value > + * types. Dispose of this memory by calling {@link #freeDBBMemory}, > or > + * resize it with {@link #reallocateDBBMemory}. The address passed > to > + * this method may be null, in which case an allocation will be > performed. > + * > + * @throws RuntimeException if the size is negative or too large > + * for the native size_t type > + * > + * @throws OutOfMemoryError if the allocation is refused by the > system > + * > + * @see #allocateDBBMemory > + */ > + public long reallocateDBBMemory(long address, long bytes) { > + return reallocateMemory(address, bytes); > + } > + > + /** > * Sets all bytes in a given block of memory to a fixed value > * (usually zero). > * > @@ -918,6 +959,17 @@ > checkPointer(null, address); > } > > + /** > + * Disposes of a block of native memory, as obtained from {@link > + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address > passed > + * to this method may be null, in which case no action is taken. > + * > + * @see #allocateDBBMemory > + */ > + public void freeDBBMemory(long address) { > + freeMemory(address); > + } > + > /// random queries > > /** > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From yumin.qi at gmail.com Fri Feb 16 20:35:00 2018 From: yumin.qi at gmail.com (yumin qi) Date: Fri, 16 Feb 2018 12:35:00 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> Message-ID: On Fri, Feb 16, 2018 at 3:52 AM, Alan Bateman wrote: > On 15/02/2018 20:28, yumin qi wrote: > >> : >> Since the property string contains non-normalized characters, it >> crashed in native canonicalize. >> I believe user.dir from the system is normalized, so it is OK but >> after it is changed like "/home/a/b/c/", it crashed. >> >> Now with using cached "user.dir", the problem is gone. >> : >> >> So the changes in resolve should be removed. >> >> Since the bug is talking about the crash, the real reason is user.dir >> should not be changed, how about changing description to >> 8194154: System property user.dir should not be changed. >> >> The test case renamed to: UserDirChangedTest.java ? >> >> Thanks for the confirming. Yes, changing the bug description and test > should be good and if you can post an updated webrev then I assume we can > wrap this one up quickly. > > Updated bug, and update webrev at same link: http://cr.openjdk.java.net/~minqi/8194154/webrev1/ Thanks Yumin From Roger.Riggs at Oracle.com Fri Feb 16 21:50:54 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 16 Feb 2018 16:50:54 -0500 Subject: [11] RFR: JDK-8198228: Spec clarification: j.u.Locale.getDisplayName() In-Reply-To: References: Message-ID: <8e9354bd-0190-4a8d-6c07-6b8f00937633@Oracle.com> Looks fine. Roger On 2/16/2018 2:22 PM, naoto.sato at oracle.com wrote: > Hello, > > Please review this small doc-only fix for this issue: > > https://bugs.openjdk.java.net/browse/JDK-8198228 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8198228/webrev.00/ > > It is just to clarify the field separator in locale display names. > Corresponding CSR (8198236) is already approved. > > Naoto From stuart.marks at oracle.com Fri Feb 16 21:56:28 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 16 Feb 2018 13:56:28 -0800 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> Message-ID: On 2/16/18 6:14 AM, Alan Bateman wrote: > On 15/02/2018 21:55, Stuart Marks wrote: >> I'd also suggest adding a CharBuffer constructor: >> >> ??? public String(CharBuffer cbuf) >> >> This would be semantically equivalent to >> >> ??? public String(char[] value, int offset, int count) >> >> except using the chars from the CharBuffer between the buffer's position and >> its limit. > CharBuffer::toString already does this so adding this constructor may not be as > useful as it initially looks. Seems like an argument that this function is in the wrong location. (Not joking.) I think CharBuffer.toString() is actually quite obscure. I note that most of the Buffer classes have toString() methods that report the *status* of the buffer, e.g. java.nio.HeapByteBuffer[pos=0 lim=1000 cap=1000] Compared to other Buffers' toString() methods, CharBuffer is the outlier: its toString() produces only the contents of the CharBuffer without any meta-information. This is rather surprising. s'marks From david.lloyd at redhat.com Fri Feb 16 22:13:25 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 16 Feb 2018 16:13:25 -0600 Subject: [JDK-6341887] Patch: java.util.zip: Add ByteBuffer methods to Inflater/Deflater Message-ID: It would be convenient to be able to inflate/deflate a direct or heap byte buffer without having to copy it through an array first. For my Friday mini-project this week, I've decided to take a crack at this. The attached patch is the result. Would anyone be interested in reviewing and maybe sponsoring this change? It would be really great to get it in to JDK 11. The patch includes a modification to FlaterTest to run through permutations of using direct and heap byte buffers. Though I couldn't get jtreg working, I did compile and run the test by hand and it seems to pass; the build also works fine with the new code. Extra thanks to Martin Balao for pointing me towards mapfile-vers when I couldn't figure out why I was getting UnsatisfiedLinkError. That one was not obvious. -- - DML -------------- next part -------------- commit becd36e852e55a29a4685577453944552c817b66 Author: David M. Lloyd Date: Fri Feb 16 11:00:10 2018 -0600 [JDK-6341887] Update Inflater/Deflator to handle ByteBuffer diff --git a/make/mapfiles/libzip/mapfile-vers b/make/mapfiles/libzip/mapfile-vers index d711d8e17f4..11ccc2d6ecb 100644 --- a/make/mapfiles/libzip/mapfile-vers +++ b/make/mapfiles/libzip/mapfile-vers @@ -33,20 +33,28 @@ SUNWprivate_1.1 { Java_java_util_zip_CRC32_update; Java_java_util_zip_CRC32_updateBytes0; Java_java_util_zip_CRC32_updateByteBuffer0; - Java_java_util_zip_Deflater_deflateBytes; + Java_java_util_zip_Deflater_deflateBytesBytes; + Java_java_util_zip_Deflater_deflateBytesBuffer; + Java_java_util_zip_Deflater_deflateBufferBytes; + Java_java_util_zip_Deflater_deflateBufferBuffer; Java_java_util_zip_Deflater_end; Java_java_util_zip_Deflater_getAdler; Java_java_util_zip_Deflater_init; Java_java_util_zip_Deflater_initIDs; Java_java_util_zip_Deflater_reset; Java_java_util_zip_Deflater_setDictionary; + Java_java_util_zip_Deflater_setDictionaryBuffer; Java_java_util_zip_Inflater_end; Java_java_util_zip_Inflater_getAdler; - Java_java_util_zip_Inflater_inflateBytes; + Java_java_util_zip_Inflater_inflateBytesBytes; + Java_java_util_zip_Inflater_inflateBytesBuffer; + Java_java_util_zip_Inflater_inflateBufferBytes; + Java_java_util_zip_Inflater_inflateBufferBuffer; Java_java_util_zip_Inflater_init; Java_java_util_zip_Inflater_initIDs; Java_java_util_zip_Inflater_reset; Java_java_util_zip_Inflater_setDictionary; + Java_java_util_zip_Inflater_setDictionaryBuffer; ZIP_Close; ZIP_CRC32; ZIP_FreeEntry; diff --git a/src/java.base/share/classes/java/util/zip/Deflater.java b/src/java.base/share/classes/java/util/zip/Deflater.java index c75dd4a33f0..5ededeb56ca 100644 --- a/src/java.base/share/classes/java/util/zip/Deflater.java +++ b/src/java.base/share/classes/java/util/zip/Deflater.java @@ -26,6 +26,9 @@ package java.util.zip; import java.lang.ref.Cleaner.Cleanable; +import java.nio.ByteBuffer; +import java.util.Objects; + import jdk.internal.ref.CleanerFactory; /** @@ -92,8 +95,7 @@ import jdk.internal.ref.CleanerFactory; public class Deflater { private final DeflaterZStreamRef zsRef; - private byte[] buf = new byte[0]; - private int off, len; + private ByteBuffer input; private int level, strategy; private boolean setParams; private boolean finish, finished; @@ -216,17 +218,11 @@ public class Deflater { * @see Deflater#needsInput */ public void setInput(byte[] b, int off, int len) { - if (b== null) { - throw new NullPointerException(); - } + Objects.requireNonNull(b); if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } - synchronized (zsRef) { - this.buf = b; - this.off = off; - this.len = len; - } + setInput(ByteBuffer.wrap(b, off, len)); } /** @@ -236,7 +232,22 @@ public class Deflater { * @see Deflater#needsInput */ public void setInput(byte[] b) { - setInput(b, 0, b.length); + // freeload the NPE off of wrap() + setInput(ByteBuffer.wrap(b)); + } + + /** + * Sets input data for compression. This should be called whenever + * needsInput() returns true indicating that more input data is required. + * @param byteBuffer the input data bytes + * @see Deflater#needsInput + * @since 11 + */ + public void setInput(ByteBuffer byteBuffer) { + Objects.requireNonNull(byteBuffer); + synchronized (zsRef) { + this.input = byteBuffer; + } } /** @@ -252,16 +263,11 @@ public class Deflater { * @see Inflater#getAdler */ public void setDictionary(byte[] b, int off, int len) { - if (b == null) { - throw new NullPointerException(); - } + Objects.requireNonNull(b); if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } - synchronized (zsRef) { - ensureOpen(); - setDictionary(zsRef.address(), b, off, len); - } + setDictionary(ByteBuffer.wrap(b, off, len)); } /** @@ -275,9 +281,36 @@ public class Deflater { * @see Inflater#getAdler */ public void setDictionary(byte[] b) { - setDictionary(b, 0, b.length); + Objects.requireNonNull(b); + setDictionary(ByteBuffer.wrap(b)); } + /** + * Sets preset dictionary for compression. A preset dictionary is used + * when the history buffer can be predetermined. When the data is later + * uncompressed with Inflater.inflate(), Inflater.getAdler() can be called + * in order to get the Adler-32 value of the dictionary required for + * decompression. + * @param byteBuffer the dictionary data bytes + * @see Inflater#inflate + * @see Inflater#getAdler + */ + public void setDictionary(ByteBuffer byteBuffer) { + Objects.requireNonNull(byteBuffer); + synchronized (zsRef) { + ensureOpen(); + final int limit = byteBuffer.limit(); + final int position = byteBuffer.position(); + if (byteBuffer.isDirect()) { + setDictionaryBuffer(zsRef.address(), byteBuffer, position, Math.max(limit - position, 0)); + } else { + final byte[] array = ZipUtils.getBufferArray(byteBuffer); + final int offset = ZipUtils.getBufferOffset(byteBuffer); + setDictionary(zsRef.address(), array, offset + position, Math.max(limit - position, 0)); + } + byteBuffer.position(limit); + } + } /** * Sets the compression strategy to the specified value. * @@ -338,7 +371,7 @@ public class Deflater { */ public boolean needsInput() { synchronized (zsRef) { - return len <= 0; + return ! input.hasRemaining(); } } @@ -404,6 +437,26 @@ public class Deflater { return deflate(b, 0, b.length, NO_FLUSH); } + /** + * Compresses the input data and fills specified buffer with compressed + * data. Returns actual number of bytes of compressed data. A return value + * of 0 indicates that {@link #needsInput() needsInput} should be called + * in order to determine if more input data is required. + * + *

This method uses {@link #NO_FLUSH} as its compression flush mode. + * An invocation of this method of the form {@code deflater.deflate(b)} + * yields the same result as the invocation of + * {@code deflater.deflate(b, 0, b.length, Deflater.NO_FLUSH)}. + * + * @param output the buffer for the compressed data + * @return the actual number of bytes of compressed data written to the + * output buffer + * @since 11 + */ + public int deflate(ByteBuffer output) { + return deflate(output, NO_FLUSH); + } + /** * Compresses the input data and fills the specified buffer with compressed * data. Returns actual number of bytes of data compressed. @@ -452,21 +505,109 @@ public class Deflater { * @since 1.7 */ public int deflate(byte[] b, int off, int len, int flush) { - if (b == null) { - throw new NullPointerException(); - } + Objects.requireNonNull(b); if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } + return deflate(ByteBuffer.wrap(b, off, len), flush); + } + + /** + * Compresses the input data and fills the specified buffer with compressed + * data. Returns actual number of bytes of data compressed. + * + *

Compression flush mode is one of the following three modes: + * + *

    + *
  • {@link #NO_FLUSH}: allows the deflater to decide how much data + * to accumulate, before producing output, in order to achieve the best + * compression (should be used in normal use scenario). A return value + * of 0 in this flush mode indicates that {@link #needsInput()} should + * be called in order to determine if more input data is required. + * + *
  • {@link #SYNC_FLUSH}: all pending output in the deflater is flushed, + * to the specified output buffer, so that an inflater that works on + * compressed data can get all input data available so far (In particular + * the {@link #needsInput()} returns {@code true} after this invocation + * if enough output space is provided). Flushing with {@link #SYNC_FLUSH} + * may degrade compression for some compression algorithms and so it + * should be used only when necessary. + * + *
  • {@link #FULL_FLUSH}: all pending output is flushed out as with + * {@link #SYNC_FLUSH}. The compression state is reset so that the inflater + * that works on the compressed output data can restart from this point + * if previous compressed data has been damaged or if random access is + * desired. Using {@link #FULL_FLUSH} too often can seriously degrade + * compression. + *
+ * + *

In the case of {@link #FULL_FLUSH} or {@link #SYNC_FLUSH}, if + * the return value is {@code len}, the space available in output + * buffer {@code b}, this method should be invoked again with the same + * {@code flush} parameter and more output space. Make sure that + * {@code len} is greater than 6 to avoid flush marker (5 bytes) being + * repeatedly output to the output buffer every time this method is + * invoked. + * + * @param output the buffer for the compressed data + * @param flush the compression flush mode + * @return the actual number of bytes of compressed data written to + * the output buffer + * + * @throws IllegalArgumentException if the flush mode is invalid + * @since 11 + */ + public int deflate(ByteBuffer output, int flush) { + Objects.requireNonNull(output); synchronized (zsRef) { ensureOpen(); if (flush == NO_FLUSH || flush == SYNC_FLUSH || flush == FULL_FLUSH) { - int thisLen = this.len; - int n = deflateBytes(zsRef.address(), b, off, len, flush); - bytesWritten += n; - bytesRead += (thisLen - this.len); - return n; + + final ByteBuffer input = this.input; + long result; + final int inputPos = input.position(); + final int outputPos = output.position(); + final int inputRem = Math.max(input.limit() - inputPos, 0); + final int outputRem = Math.max(output.limit() - outputPos, 0); + if (input.isDirect()) { + if (output.isDirect()) { + result = deflateBufferBuffer(zsRef.address(), + input, inputPos, inputRem, + output, outputPos, outputRem, + flush); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = deflateBufferBytes(zsRef.address(), + input, inputPos, inputRem, + outputArray, outputOffset + outputPos, outputRem, + flush); + } + } else { + final byte[] inputArray = ZipUtils.getBufferArray(input); + final int inputOffset = ZipUtils.getBufferOffset(input); + if (output.isDirect()) { + result = deflateBytesBuffer(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + output, outputPos, outputRem, + flush); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = deflateBytesBytes(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + outputArray, outputOffset + outputPos, outputRem, + flush); + } + } + int read = (int) (result & 0x7fff_ffffL); + int written = (int) (result >>> 31); + input.position(inputPos + read); + output.position(outputPos + written); + bytesWritten += written; + bytesRead += read; + return written; } throw new IllegalArgumentException(); } @@ -545,7 +686,7 @@ public class Deflater { reset(zsRef.address()); finish = false; finished = false; - off = len = 0; + input = ZipUtils.defaultBuf; bytesRead = bytesWritten = 0; } } @@ -560,7 +701,7 @@ public class Deflater { public void end() { synchronized (zsRef) { zsRef.clean(); - buf = null; + input = ZipUtils.defaultBuf; } } @@ -587,9 +728,22 @@ public class Deflater { private static native void initIDs(); private static native long init(int level, int strategy, boolean nowrap); - private static native void setDictionary(long addr, byte[] b, int off, int len); - private native int deflateBytes(long addr, byte[] b, int off, int len, - int flush); + private static native void setDictionary(long addr, byte[] b, int off, + int len); + private static native void setDictionaryBuffer(long addr, ByteBuffer buffer, int off, + int len); + private native long deflateBytesBytes(long addr, + byte[] inputArray, int inputOff, int inputLen, + byte[] outputArray, int outputOff, int outputLen, int flush); + private native long deflateBytesBuffer(long addr, + byte[] inputArray, int inputOff, int inputLen, + ByteBuffer outputBuffer, int outputPos, int outputLen, int flush); + private native long deflateBufferBytes(long addr, + ByteBuffer inputBuffer, int inputPos, int inputLen, + byte[] outputArray, int outputOff, int outputLen, int flush); + private native long deflateBufferBuffer(long addr, + ByteBuffer inputBuffer, int inputPos, int inputLen, + ByteBuffer outputBuffer, int outputPos, int outputLen, int flush); private static native int getAdler(long addr); private static native void reset(long addr); private static native void end(long addr); diff --git a/src/java.base/share/classes/java/util/zip/Inflater.java b/src/java.base/share/classes/java/util/zip/Inflater.java index 9c6d8aa3d83..43455602c01 100644 --- a/src/java.base/share/classes/java/util/zip/Inflater.java +++ b/src/java.base/share/classes/java/util/zip/Inflater.java @@ -26,6 +26,10 @@ package java.util.zip; import java.lang.ref.Cleaner.Cleanable; +import java.nio.ByteBuffer; +import java.nio.ReadOnlyBufferException; +import java.util.Objects; + import jdk.internal.ref.CleanerFactory; /** @@ -92,15 +96,12 @@ import jdk.internal.ref.CleanerFactory; public class Inflater { private final InflaterZStreamRef zsRef; - private byte[] buf = defaultBuf; - private int off, len; + private ByteBuffer input = ZipUtils.defaultBuf; private boolean finished; private boolean needDict; private long bytesRead; private long bytesWritten; - private static final byte[] defaultBuf = new byte[0]; - static { ZipUtils.loadLibrary(); initIDs(); @@ -138,17 +139,11 @@ public class Inflater { * @see Inflater#needsInput */ public void setInput(byte[] b, int off, int len) { - if (b == null) { - throw new NullPointerException(); - } + Objects.requireNonNull(b); if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } - synchronized (zsRef) { - this.buf = b; - this.off = off; - this.len = len; - } + setInput(ByteBuffer.wrap(b, off, len)); } /** @@ -159,7 +154,23 @@ public class Inflater { * @see Inflater#needsInput */ public void setInput(byte[] b) { - setInput(b, 0, b.length); + // freeload the NPE off of wrap() + setInput(ByteBuffer.wrap(b)); + } + + /** + * Sets input data for decompression. Should be called whenever + * needsInput() returns true indicating that more input data is + * required. + * @param byteBuffer the input data bytes + * @see Inflater#needsInput + * @since 11 + */ + public void setInput(ByteBuffer byteBuffer) { + Objects.requireNonNull(byteBuffer); + synchronized (zsRef) { + this.input = byteBuffer; + } } /** @@ -174,17 +185,11 @@ public class Inflater { * @see Inflater#getAdler */ public void setDictionary(byte[] b, int off, int len) { - if (b == null) { - throw new NullPointerException(); - } + Objects.requireNonNull(b); if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } - synchronized (zsRef) { - ensureOpen(); - setDictionary(zsRef.address(), b, off, len); - needDict = false; - } + setDictionary(ByteBuffer.wrap(b, off, len)); } /** @@ -197,7 +202,36 @@ public class Inflater { * @see Inflater#getAdler */ public void setDictionary(byte[] b) { - setDictionary(b, 0, b.length); + Objects.requireNonNull(b); + setDictionary(ByteBuffer.wrap(b)); + } + + /** + * Sets the preset dictionary to the given array of bytes. Should be + * called when inflate() returns 0 and needsDictionary() returns true + * indicating that a preset dictionary is required. The method getAdler() + * can be used to get the Adler-32 value of the dictionary needed. + * @param byteBuffer the dictionary data bytes + * @see Inflater#needsDictionary + * @see Inflater#getAdler + * @since 11 + */ + public void setDictionary(ByteBuffer byteBuffer) { + Objects.requireNonNull(byteBuffer); + synchronized (zsRef) { + ensureOpen(); + final int limit = byteBuffer.limit(); + final int position = byteBuffer.position(); + if (byteBuffer.isDirect()) { + setDictionaryBuffer(zsRef.address(), byteBuffer, position, Math.max(limit - position, 0)); + } else { + final byte[] array = ZipUtils.getBufferArray(byteBuffer); + final int offset = ZipUtils.getBufferOffset(byteBuffer); + setDictionary(zsRef.address(), array, offset + position, Math.max(limit - position, 0)); + } + byteBuffer.position(limit); + needDict = false; + } } /** @@ -208,7 +242,7 @@ public class Inflater { */ public int getRemaining() { synchronized (zsRef) { - return len; + return input.remaining(); } } @@ -220,7 +254,7 @@ public class Inflater { */ public boolean needsInput() { synchronized (zsRef) { - return len <= 0; + return ! input.hasRemaining(); } } @@ -265,20 +299,11 @@ public class Inflater { public int inflate(byte[] b, int off, int len) throws DataFormatException { - if (b == null) { - throw new NullPointerException(); - } + Objects.requireNonNull(b); if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } - synchronized (zsRef) { - ensureOpen(); - int thisLen = this.len; - int n = inflateBytes(zsRef.address(), b, off, len); - bytesWritten += n; - bytesRead += (thisLen - this.len); - return n; - } + return inflate(ByteBuffer.wrap(b, off, len)); } /** @@ -295,7 +320,73 @@ public class Inflater { * @see Inflater#needsDictionary */ public int inflate(byte[] b) throws DataFormatException { - return inflate(b, 0, b.length); + Objects.requireNonNull(b); + return inflate(ByteBuffer.wrap(b)); + } + + /** + * Uncompresses bytes into specified buffer. Returns actual number + * of bytes uncompressed. A return value of 0 indicates that + * needsInput() or needsDictionary() should be called in order to + * determine if more input data or a preset dictionary is required. + * In the latter case, getAdler() can be used to get the Adler-32 + * value of the dictionary required. + * @param output the buffer for the uncompressed data + * @return the actual number of uncompressed bytes + * @throws DataFormatException if the compressed data format is invalid + * @throws ReadOnlyBufferException if the given output buffer is read-only + * @see Inflater#needsInput + * @see Inflater#needsDictionary + * @since 11 + */ + public int inflate(ByteBuffer output) throws DataFormatException { + Objects.requireNonNull(output); + if (output.isReadOnly()) { + throw new ReadOnlyBufferException(); + } + synchronized (zsRef) { + ensureOpen(); + final ByteBuffer input = this.input; + long result; + final int inputPos = input.position(); + final int outputPos = output.position(); + final int inputRem = Math.max(input.limit() - inputPos, 0); + final int outputRem = Math.max(output.limit() - outputPos, 0); + if (input.isDirect()) { + if (output.isDirect()) { + result = inflateBufferBuffer(zsRef.address(), + input, inputPos, inputRem, + output, outputPos, outputRem); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = inflateBufferBytes(zsRef.address(), + input, inputPos, inputRem, + outputArray, outputOffset + outputPos, outputRem); + } + } else { + final byte[] inputArray = ZipUtils.getBufferArray(input); + final int inputOffset = ZipUtils.getBufferOffset(input); + if (output.isDirect()) { + result = inflateBytesBuffer(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + output, outputPos, outputRem); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = inflateBytesBytes(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + outputArray, outputOffset + outputPos, outputRem); + } + } + int read = (int) (result & 0x7fff_ffffL); + int written = (int) (result >>> 31); + input.position(inputPos + read); + output.position(outputPos + written); + bytesWritten += written; + bytesRead += read; + return written; + } } /** @@ -368,10 +459,9 @@ public class Inflater { synchronized (zsRef) { ensureOpen(); reset(zsRef.address()); - buf = defaultBuf; + input = ZipUtils.defaultBuf; finished = false; needDict = false; - off = len = 0; bytesRead = bytesWritten = 0; } } @@ -386,7 +476,7 @@ public class Inflater { public void end() { synchronized (zsRef) { zsRef.clean(); - buf = null; + input = null; } } @@ -426,8 +516,20 @@ public class Inflater { private static native long init(boolean nowrap); private static native void setDictionary(long addr, byte[] b, int off, int len); - private native int inflateBytes(long addr, byte[] b, int off, int len) - throws DataFormatException; + private static native void setDictionaryBuffer(long addr, ByteBuffer buffer, int off, + int len); + private native long inflateBytesBytes(long addr, + byte[] inputArray, int inputOff, int inputLen, + byte[] outputArray, int outputOff, int outputLen) throws DataFormatException; + private native long inflateBytesBuffer(long addr, + byte[] inputArray, int inputOff, int inputLen, + ByteBuffer outputBuffer, int outputPos, int outputLen) throws DataFormatException; + private native long inflateBufferBytes(long addr, + ByteBuffer inputBuffer, int inputPos, int inputLen, + byte[] outputArray, int outputOff, int outputLen) throws DataFormatException; + private native long inflateBufferBuffer(long addr, + ByteBuffer inputBuffer, int inputPos, int inputLen, + ByteBuffer outputBuffer, int outputPos, int outputLen) throws DataFormatException; private static native int getAdler(long addr); private static native void reset(long addr); private static native void end(long addr); diff --git a/src/java.base/share/classes/java/util/zip/ZipUtils.java b/src/java.base/share/classes/java/util/zip/ZipUtils.java index 45c5d8dbb67..c2a8fb238ce 100644 --- a/src/java.base/share/classes/java/util/zip/ZipUtils.java +++ b/src/java.base/share/classes/java/util/zip/ZipUtils.java @@ -25,6 +25,7 @@ package java.util.zip; +import java.nio.ByteBuffer; import java.nio.file.attribute.FileTime; import java.security.AccessController; import java.security.PrivilegedAction; @@ -37,6 +38,8 @@ import java.util.concurrent.TimeUnit; import static java.util.zip.ZipConstants.ENDHDR; +import jdk.internal.misc.Unsafe; + class ZipUtils { // used to adjust values between Windows and java epoch @@ -45,6 +48,8 @@ class ZipUtils { // used to indicate the corresponding windows time is not available public static final long WINDOWS_TIME_NOT_AVAILABLE = Long.MIN_VALUE; + static final ByteBuffer defaultBuf = ByteBuffer.allocateDirect(0); + /** * Converts Windows time (in microseconds, UTC/GMT) time to FileTime. */ @@ -281,4 +286,17 @@ class ZipUtils { AccessController.doPrivileged(pa); } } + + private static final Unsafe unsafe = Unsafe.getUnsafe(); + + private static final long byteBufferArrayOffset = unsafe.objectFieldOffset(ByteBuffer.class, "hb"); + private static final long byteBufferOffsetOffset = unsafe.objectFieldOffset(ByteBuffer.class, "offset"); + + static byte[] getBufferArray(ByteBuffer byteBuffer) { + return (byte[]) unsafe.getObject(byteBuffer, byteBufferArrayOffset); + } + + static int getBufferOffset(ByteBuffer byteBuffer) { + return unsafe.getInt(byteBuffer, byteBufferOffsetOffset); + } } diff --git a/src/java.base/share/native/libzip/Deflater.c b/src/java.base/share/native/libzip/Deflater.c index b666a16145a..d6d78a1b586 100644 --- a/src/java.base/share/native/libzip/Deflater.c +++ b/src/java.base/share/native/libzip/Deflater.c @@ -58,12 +58,6 @@ Java_java_util_zip_Deflater_initIDs(JNIEnv *env, jclass cls) CHECK_NULL(finishID); finishedID = (*env)->GetFieldID(env, cls, "finished", "Z"); CHECK_NULL(finishedID); - bufID = (*env)->GetFieldID(env, cls, "buf", "[B"); - CHECK_NULL(bufID); - offID = (*env)->GetFieldID(env, cls, "off", "I"); - CHECK_NULL(offID); - lenID = (*env)->GetFieldID(env, cls, "len", "I"); - CHECK_NULL(lenID); } JNIEXPORT jlong JNICALL @@ -104,22 +98,16 @@ Java_java_util_zip_Deflater_init(JNIEnv *env, jclass cls, jint level, } } -JNIEXPORT void JNICALL -Java_java_util_zip_Deflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, - jarray b, jint off, jint len) +static void doSetDictionary(JNIEnv *env, jlong addr, jbyte *buf, jint off, + jint len) { - Bytef *buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - int res; - if (buf == 0) {/* out of memory */ - return; - } - res = deflateSetDictionary((z_stream *)jlong_to_ptr(addr), buf + off, len); - (*env)->ReleasePrimitiveArrayCritical(env, b, buf, 0); + int res = deflateSetDictionary(jlong_to_ptr(addr), (Bytef *) (buf + off), len); switch (res) { case Z_OK: break; case Z_STREAM_ERROR: - JNU_ThrowIllegalArgumentException(env, 0); + case Z_DATA_ERROR: + JNU_ThrowIllegalArgumentException(env, ((z_stream *)jlong_to_ptr(addr))->msg); break; default: JNU_ThrowInternalError(env, ((z_stream *)jlong_to_ptr(addr))->msg); @@ -127,94 +115,188 @@ Java_java_util_zip_Deflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, } } -JNIEXPORT jint JNICALL -Java_java_util_zip_Deflater_deflateBytes(JNIEnv *env, jobject this, jlong addr, - jarray b, jint off, jint len, jint flush) +JNIEXPORT void JNICALL +Java_java_util_zip_Deflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, + jbyteArray b, jint off, jint len) +{ + jbyte *buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); + if (buf == NULL) /* out of memory */ + return; + doSetDictionary(env, addr, buf, off, len); + (*env)->ReleasePrimitiveArrayCritical(env, b, buf, 0); +} + +JNIEXPORT void JNICALL +Java_java_util_zip_Deflater_setDictionaryBuffer(JNIEnv *env, jclass cls, jlong addr, + jobject buffer, jint off, jint len) +{ + jbyte *buf = (*env)->GetDirectBufferAddress(env, buffer); + if (buf == NULL) { /* ??? */ + JNU_ThrowInternalError(env, "should not reach here"); + return; + } + doSetDictionary(env, addr, buf, off, len); +} + +static jlong doDeflate(JNIEnv *env, jobject this, jlong addr, + jbyte *input, jint inputOff, jint inputLen, + jbyte *output, jint outputOff, jint outputLen, + jint flush) { z_stream *strm = jlong_to_ptr(addr); + jint inputUsed = 0, outputUsed = 0; + + strm->next_in = (Bytef *) (input + inputOff); + strm->next_out = (Bytef *) (output + outputOff); + strm->avail_in = inputLen; + strm->avail_out = outputLen; - jarray this_buf = (*env)->GetObjectField(env, this, bufID); - jint this_off = (*env)->GetIntField(env, this, offID); - jint this_len = (*env)->GetIntField(env, this, lenID); - jbyte *in_buf; - jbyte *out_buf; - int res; if ((*env)->GetBooleanField(env, this, setParamsID)) { int level = (*env)->GetIntField(env, this, levelID); int strategy = (*env)->GetIntField(env, this, strategyID); - in_buf = (*env)->GetPrimitiveArrayCritical(env, this_buf, 0); - if (in_buf == NULL) { - // Throw OOME only when length is not zero - if (this_len != 0 && (*env)->ExceptionOccurred(env) == NULL) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - out_buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - if (out_buf == NULL) { - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); - if (len != 0 && (*env)->ExceptionOccurred(env) == NULL) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - - strm->next_in = (Bytef *) (in_buf + this_off); - strm->next_out = (Bytef *) (out_buf + off); - strm->avail_in = this_len; - strm->avail_out = len; - res = deflateParams(strm, level, strategy); - (*env)->ReleasePrimitiveArrayCritical(env, b, out_buf, 0); - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); + int res = deflateParams(strm, level, strategy); switch (res) { case Z_OK: (*env)->SetBooleanField(env, this, setParamsID, JNI_FALSE); + // fall through case Z_BUF_ERROR: - this_off += this_len - strm->avail_in; - (*env)->SetIntField(env, this, offID, this_off); - (*env)->SetIntField(env, this, lenID, strm->avail_in); - return (jint) (len - strm->avail_out); + inputUsed = inputLen - strm->avail_in; + outputUsed = outputLen - strm->avail_out; + break; default: JNU_ThrowInternalError(env, strm->msg); return 0; } } else { jboolean finish = (*env)->GetBooleanField(env, this, finishID); - in_buf = (*env)->GetPrimitiveArrayCritical(env, this_buf, 0); - if (in_buf == NULL) { - if (this_len != 0) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - out_buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - if (out_buf == NULL) { - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); - if (len != 0) - JNU_ThrowOutOfMemoryError(env, 0); - - return 0; - } - - strm->next_in = (Bytef *) (in_buf + this_off); - strm->next_out = (Bytef *) (out_buf + off); - strm->avail_in = this_len; - strm->avail_out = len; - res = deflate(strm, finish ? Z_FINISH : flush); - (*env)->ReleasePrimitiveArrayCritical(env, b, out_buf, 0); - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); + int res = deflate(strm, finish ? Z_FINISH : flush); switch (res) { case Z_STREAM_END: (*env)->SetBooleanField(env, this, finishedID, JNI_TRUE); /* fall through */ case Z_OK: case Z_BUF_ERROR: - this_off += this_len - strm->avail_in; - (*env)->SetIntField(env, this, offID, this_off); - (*env)->SetIntField(env, this, lenID, strm->avail_in); - return len - strm->avail_out; + inputUsed = inputLen - strm->avail_in; + outputUsed = outputLen - strm->avail_out; + break; default: JNU_ThrowInternalError(env, strm->msg); return 0; } } + return ((jlong)inputUsed) | (((jlong)outputUsed) << 31); +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Deflater_deflateBytesBytes(JNIEnv *env, jobject this, jlong addr, + jbyteArray inputArray, jint inputOff, jint inputLen, + jbyteArray outputArray, jint outputOff, jint outputLen, + jint flush) +{ + jbyte *input = (*env)->GetPrimitiveArrayCritical(env, inputArray, 0); + if (input == NULL) { + if (inputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + jbyte *output = (*env)->GetPrimitiveArrayCritical(env, outputArray, 0); + if (output == NULL) { + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + if (outputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + + jlong retVal = doDeflate(env, this, addr, + input, inputOff, inputLen, + output, outputOff, outputLen, + flush); + + (*env)->ReleasePrimitiveArrayCritical(env, outputArray, output, 0); + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + + return retVal; +} + + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Deflater_deflateBytesBuffer(JNIEnv *env, jobject this, jlong addr, + jbyteArray inputArray, jint inputOff, jint inputLen, + jobject outputBuffer, jint outputOff, jint outputLen, + jint flush) +{ + jbyte *input = (*env)->GetPrimitiveArrayCritical(env, inputArray, 0); + if (input == NULL) { + if (inputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + jbyte *output = (*env)->GetDirectBufferAddress(env, outputBuffer); + if (output == NULL) { + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + JNU_ThrowInternalError(env, "should not reach here"); + return 0L; + } + + jlong retVal = doDeflate(env, this, addr, + input, inputOff, inputLen, + output, outputOff, outputLen, + flush); + + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Deflater_deflateBufferBytes(JNIEnv *env, jobject this, jlong addr, + jobject inputBuffer, jint inputOff, jint inputLen, + jbyteArray outputArray, jint outputOff, jint outputLen, + jint flush) +{ + jbyte *input = (*env)->GetDirectBufferAddress(env, inputBuffer); + if (input == NULL) { + JNU_ThrowInternalError(env, "should not reach here"); + return 0L; + } + jbyte *output = (*env)->GetPrimitiveArrayCritical(env, outputArray, 0); + if (output == NULL) { + if (outputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + + jlong retVal = doDeflate(env, this, addr, + input, inputOff, inputLen, + output, outputOff, outputLen, + flush); + + (*env)->ReleasePrimitiveArrayCritical(env, outputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Deflater_deflateBufferBuffer(JNIEnv *env, jobject this, jlong addr, + jobject inputBuffer, jint inputOff, jint inputLen, + jobject outputBuffer, jint outputOff, jint outputLen, + jint flush) +{ + jbyte *input = (*env)->GetDirectBufferAddress(env, inputBuffer); + if (input == NULL) { + JNU_ThrowInternalError(env, "should not reach here"); + return 0L; + } + jbyte *output = (*env)->GetDirectBufferAddress(env, outputBuffer); + if (output == NULL) { + JNU_ThrowInternalError(env, "should not reach here"); + return 0L; + } + + return doDeflate(env, this, addr, + input, inputOff, inputLen, + output, outputOff, outputLen, + flush); } JNIEXPORT jint JNICALL diff --git a/src/java.base/share/native/libzip/Inflater.c b/src/java.base/share/native/libzip/Inflater.c index 2e21d084b39..5e79603cda7 100644 --- a/src/java.base/share/native/libzip/Inflater.c +++ b/src/java.base/share/native/libzip/Inflater.c @@ -44,7 +44,6 @@ static jfieldID needDictID; static jfieldID finishedID; -static jfieldID bufID, offID, lenID; JNIEXPORT void JNICALL Java_java_util_zip_Inflater_initIDs(JNIEnv *env, jclass cls) @@ -53,12 +52,6 @@ Java_java_util_zip_Inflater_initIDs(JNIEnv *env, jclass cls) CHECK_NULL(needDictID); finishedID = (*env)->GetFieldID(env, cls, "finished", "Z"); CHECK_NULL(finishedID); - bufID = (*env)->GetFieldID(env, cls, "buf", "[B"); - CHECK_NULL(bufID); - offID = (*env)->GetFieldID(env, cls, "off", "I"); - CHECK_NULL(offID); - lenID = (*env)->GetFieldID(env, cls, "len", "I"); - CHECK_NULL(lenID); } JNIEXPORT jlong JNICALL @@ -94,16 +87,10 @@ Java_java_util_zip_Inflater_init(JNIEnv *env, jclass cls, jboolean nowrap) } } -JNIEXPORT void JNICALL -Java_java_util_zip_Inflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, - jarray b, jint off, jint len) +static void doSetDictionary(JNIEnv *env, jlong addr, jbyte *buf, jint off, + jint len) { - Bytef *buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - int res; - if (buf == 0) /* out of memory */ - return; - res = inflateSetDictionary(jlong_to_ptr(addr), buf + off, len); - (*env)->ReleasePrimitiveArrayCritical(env, b, buf, 0); + int res = inflateSetDictionary(jlong_to_ptr(addr), (Bytef *) (buf + off), len); switch (res) { case Z_OK: break; @@ -117,68 +104,172 @@ Java_java_util_zip_Inflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, } } -JNIEXPORT jint JNICALL -Java_java_util_zip_Inflater_inflateBytes(JNIEnv *env, jobject this, jlong addr, - jarray b, jint off, jint len) +JNIEXPORT void JNICALL +Java_java_util_zip_Inflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, + jbyteArray b, jint off, jint len) +{ + jbyte *buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); + if (buf == NULL) /* out of memory */ + return; + doSetDictionary(env, addr, buf, off, len); + (*env)->ReleasePrimitiveArrayCritical(env, b, buf, 0); +} + +JNIEXPORT void JNICALL +Java_java_util_zip_Inflater_setDictionaryBuffer(JNIEnv *env, jclass cls, jlong addr, + jobject buffer, jint off, jint len) +{ + jbyte *buf = (*env)->GetDirectBufferAddress(env, buffer); + if (buf == NULL) { /* ??? */ + JNU_ThrowInternalError(env, "should not reach here"); + return; + } + doSetDictionary(env, addr, buf, off, len); +} + +static jlong doInflate(JNIEnv *env, jobject this, jlong addr, + jbyte *input, jint inputOff, jint inputLen, + jbyte *output, jint outputOff, jint outputLen) { z_stream *strm = jlong_to_ptr(addr); - jarray this_buf = (jarray)(*env)->GetObjectField(env, this, bufID); - jint this_off = (*env)->GetIntField(env, this, offID); - jint this_len = (*env)->GetIntField(env, this, lenID); + jint inputUsed = 0, outputUsed = 0; - jbyte *in_buf; - jbyte *out_buf; - int ret; + strm->next_in = (Bytef *) (input + inputOff); + strm->next_out = (Bytef *) (output + outputOff); + strm->avail_in = inputLen; + strm->avail_out = outputLen; - in_buf = (*env)->GetPrimitiveArrayCritical(env, this_buf, 0); - if (in_buf == NULL) { - if (this_len != 0 && (*env)->ExceptionOccurred(env) == NULL) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - out_buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - if (out_buf == NULL) { - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); - if (len != 0 && (*env)->ExceptionOccurred(env) == NULL) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - strm->next_in = (Bytef *) (in_buf + this_off); - strm->next_out = (Bytef *) (out_buf + off); - strm->avail_in = this_len; - strm->avail_out = len; - ret = inflate(strm, Z_PARTIAL_FLUSH); - (*env)->ReleasePrimitiveArrayCritical(env, b, out_buf, 0); - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); + int ret = inflate(strm, Z_PARTIAL_FLUSH); switch (ret) { case Z_STREAM_END: (*env)->SetBooleanField(env, this, finishedID, JNI_TRUE); /* fall through */ case Z_OK: - this_off += this_len - strm->avail_in; - (*env)->SetIntField(env, this, offID, this_off); - (*env)->SetIntField(env, this, lenID, strm->avail_in); - return (jint) (len - strm->avail_out); + inputUsed = inputLen - strm->avail_in; + outputUsed = outputLen - strm->avail_out; + break; case Z_NEED_DICT: (*env)->SetBooleanField(env, this, needDictID, JNI_TRUE); /* Might have consumed some input here! */ - this_off += this_len - strm->avail_in; - (*env)->SetIntField(env, this, offID, this_off); - (*env)->SetIntField(env, this, lenID, strm->avail_in); - return 0; + inputUsed = inputLen - strm->avail_in; + break; case Z_BUF_ERROR: - return 0; + break; case Z_DATA_ERROR: ThrowDataFormatException(env, strm->msg); - return 0; + break; case Z_MEM_ERROR: JNU_ThrowOutOfMemoryError(env, 0); - return 0; + break; default: JNU_ThrowInternalError(env, strm->msg); - return 0; + break; + } + return ((jlong)inputUsed) | (((jlong)outputUsed) << 31); +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Inflater_inflateBytesBytes(JNIEnv *env, jobject this, jlong addr, + jbyteArray inputArray, jint inputOff, jint inputLen, + jbyteArray outputArray, jint outputOff, jint outputLen) +{ + jbyte *input = (*env)->GetPrimitiveArrayCritical(env, inputArray, 0); + if (input == NULL) { + if (inputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + jbyte *output = (*env)->GetPrimitiveArrayCritical(env, outputArray, 0); + if (output == NULL) { + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + if (outputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; } + + jlong retVal = doInflate(env, this, addr, + input, inputOff, inputLen, + output, outputOff, outputLen); + + (*env)->ReleasePrimitiveArrayCritical(env, outputArray, output, 0); + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Inflater_inflateBytesBuffer(JNIEnv *env, jobject this, jlong addr, + jbyteArray inputArray, jint inputOff, jint inputLen, + jobject outputBuffer, jint outputOff, jint outputLen) +{ + jbyte *input = (*env)->GetPrimitiveArrayCritical(env, inputArray, 0); + if (input == NULL) { + if (inputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + jbyte *output = (*env)->GetDirectBufferAddress(env, outputBuffer); + if (output == NULL) { + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + JNU_ThrowInternalError(env, "should not reach here"); + return 0L; + } + + jlong retVal = doInflate(env, this, addr, + input, inputOff, inputLen, + output, outputOff, outputLen); + + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Inflater_inflateBufferBytes(JNIEnv *env, jobject this, jlong addr, + jobject inputBuffer, jint inputOff, jint inputLen, + jbyteArray outputArray, jint outputOff, jint outputLen) +{ + jbyte *input = (*env)->GetDirectBufferAddress(env, inputBuffer); + if (input == NULL) { + JNU_ThrowInternalError(env, "should not reach here"); + return 0L; + } + jbyte *output = (*env)->GetPrimitiveArrayCritical(env, outputArray, 0); + if (output == NULL) { + if (outputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + + jlong retVal = doInflate(env, this, addr, + input, inputOff, inputLen, + output, outputOff, outputLen); + + (*env)->ReleasePrimitiveArrayCritical(env, outputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Inflater_inflateBufferBuffer(JNIEnv *env, jobject this, jlong addr, + jobject inputBuffer, jint inputOff, jint inputLen, + jobject outputBuffer, jint outputOff, jint outputLen) +{ + jbyte *input = (*env)->GetDirectBufferAddress(env, inputBuffer); + if (input == NULL) { + JNU_ThrowInternalError(env, "should not reach here"); + return 0L; + } + jbyte *output = (*env)->GetDirectBufferAddress(env, outputBuffer); + if (output == NULL) { + JNU_ThrowInternalError(env, "should not reach here"); + return 0L; + } + + return doInflate(env, this, addr, + input, inputOff, inputLen, + output, outputOff, outputLen); } JNIEXPORT jint JNICALL diff --git a/test/jdk/java/util/zip/FlaterTest.java b/test/jdk/java/util/zip/FlaterTest.java index 7245440d033..b2e01c3e7cb 100644 --- a/test/jdk/java/util/zip/FlaterTest.java +++ b/test/jdk/java/util/zip/FlaterTest.java @@ -41,7 +41,7 @@ import java.util.zip.*; */ public class FlaterTest extends Thread { private static final int DATA_LEN = 1024 * 128; - private static byte[] data; + private static ByteBuffer data; // If true, print extra info. private static final boolean debug = false; @@ -51,15 +51,13 @@ public class FlaterTest extends Thread { Collections.synchronizedSet(new HashSet()); /** Fill in {@code data} with random values. */ - static void createData() { - ByteBuffer bb = ByteBuffer.allocate(8); - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - for (int i = 0; i < DATA_LEN; i++) { - bb.putDouble(0, Math.random()); - baos.write(bb.array(), 0, 8); + static void createData(boolean direct) { + ByteBuffer bb = direct ? ByteBuffer.allocateDirect(DATA_LEN * 8) : ByteBuffer.allocate(DATA_LEN * 8); + for (int i = 0; i < DATA_LEN * 8; i += 8) { + bb.putDouble(i, Math.random()); } - data = baos.toByteArray(); - if (debug) System.out.println("data length is " + data.length); + data = bb; + if (debug) System.out.println("data length is " + data.capacity()); } /** @return the length of the deflated {@code data}. */ @@ -68,7 +66,7 @@ public class FlaterTest extends Thread { Deflater deflater = new Deflater(); deflater.setInput(data); deflater.finish(); - byte[] out = new byte[data.length]; + byte[] out = new byte[data.capacity()]; rc = deflater.deflate(out); deflater.end(); if (debug) System.out.println("deflatedLength is " + rc); @@ -78,34 +76,40 @@ public class FlaterTest extends Thread { /** Compares given bytes with those in {@code data}. * @throws Exception if given bytes don't match {@code data}. */ - private static void validate(byte[] buf, int offset, int len) throws Exception { + private static void validate(ByteBuffer buf, int offset, int len) throws Exception { for (int i = 0; i < len; i++ ) { - if (buf[i] != data[offset+i]) { + if (buf.get(i) != data.get(offset+i)) { throw new Exception("mismatch at " + (offset + i)); } } } public static void realMain(String[] args) throws Throwable { - createData(); int numThreads = args.length > 0 ? Integer.parseInt(args[0]) : 5; - new FlaterTest().go(numThreads); + createData(false); + new FlaterTest().go(numThreads, false); + new FlaterTest().go(numThreads, true); + createData(true); + new FlaterTest().go(numThreads, false); + new FlaterTest().go(numThreads, true); } - private synchronized void go(int numThreads) throws Throwable { + private synchronized void go(int numThreads, boolean direct) throws Throwable { int deflatedLength = getDeflatedLength(); long time = System.currentTimeMillis(); for (int i = 0; i < numThreads; i++) { - Flater f = new Flater(deflatedLength); + Flater f = new Flater(deflatedLength, direct); flaters.add(f); f.start(); } - while (flaters.size() != 0) { - try { - Thread.currentThread().sleep(10); - } catch (InterruptedException ex) { - unexpected(ex); + synchronized (flaters) { + while (flaters.size() != 0) { + try { + flaters.wait(); + } catch (InterruptedException ex) { + unexpected(ex); + } } } time = System.currentTimeMillis() - time; @@ -116,32 +120,41 @@ public class FlaterTest extends Thread { /** Deflates and inflates data. */ static class Flater extends Thread { private final int deflatedLength; + private final boolean direct; - private Flater(int length) { + private Flater(int length, final boolean direct) { this.deflatedLength = length; + this.direct = direct; } /** Deflates and inflates {@code data}. */ public void run() { if (debug) System.out.println(getName() + " starting run()"); try { - byte[] deflated = DeflateData(deflatedLength); + ByteBuffer deflated = DeflateData(deflatedLength); InflateData(deflated); } catch (Throwable t) { t.printStackTrace(); fail(getName() + " failed"); } finally { - flaters.remove(this); + synchronized (flaters) { + flaters.remove(this); + if (flaters.isEmpty()) { + flaters.notifyAll(); + } + } } } /** Returns a copy of {@code data} in deflated form. */ - private byte[] DeflateData(int length) throws Throwable { + private ByteBuffer DeflateData(int length) throws Throwable { Deflater deflater = new Deflater(); + data.clear(); deflater.setInput(data); deflater.finish(); - byte[] out = new byte[length]; + ByteBuffer out = direct ? ByteBuffer.allocateDirect(length) : ByteBuffer.allocate(length); deflater.deflate(out); + out.flip(); return out; } @@ -149,14 +162,15 @@ public class FlaterTest extends Thread { * inflation. * @throws Exception if inflated bytes don't match {@code data}. */ - private void InflateData(byte[] bytes) throws Throwable { + private void InflateData(ByteBuffer bytes) throws Throwable { Inflater inflater = new Inflater(); - inflater.setInput(bytes, 0, bytes.length); + inflater.setInput(bytes); int len = 1024 * 8; int offset = 0; + ByteBuffer buf = direct ? ByteBuffer.allocateDirect(len) : ByteBuffer.allocate(len); while (inflater.getRemaining() > 0) { - byte[] buf = new byte[len]; - int inflated = inflater.inflate(buf, 0, len); + buf.clear(); + int inflated = inflater.inflate(buf); validate(buf, offset, inflated); offset += inflated; } From david.lloyd at redhat.com Fri Feb 16 22:16:38 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 16 Feb 2018 16:16:38 -0600 Subject: [JDK-6341887] Patch: java.util.zip: Add ByteBuffer methods to Inflater/Deflater In-Reply-To: References: Message-ID: Also available in more readable form at: https://github.com/dmlloyd/openjdk/commit/becd36e852e55a29a4685577453944552c817b66 On Fri, Feb 16, 2018 at 4:13 PM, David Lloyd wrote: > It would be convenient to be able to inflate/deflate a direct or heap > byte buffer without having to copy it through an array first. For my > Friday mini-project this week, I've decided to take a crack at this. > The attached patch is the result. Would anyone be interested in > reviewing and maybe sponsoring this change? It would be really great > to get it in to JDK 11. > > The patch includes a modification to FlaterTest to run through > permutations of using direct and heap byte buffers. Though I couldn't > get jtreg working, I did compile and run the test by hand and it seems > to pass; the build also works fine with the new code. > > Extra thanks to Martin Balao for pointing me towards mapfile-vers when > I couldn't figure out why I was getting UnsatisfiedLinkError. That > one was not obvious. > -- > - DML -- - DML From Roger.Riggs at Oracle.com Fri Feb 16 22:28:02 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 16 Feb 2018 17:28:02 -0500 Subject: RFR 8189330 Cleanup FileDescriptor implementation Message-ID: Please review a cleanup and refactoring of FileDescriptor.java. The Unix and Windows versions are merged to ease maintenance remove replicated code The FileCleanable class is extracted and SocketCleanable is updated. Webrev: ? http://cr.openjdk.java.net/~rriggs/webrev-fd-cleanup-8189330/ Issue: ? https://bugs.openjdk.java.net/browse/JDK-8189330 Thanks, Roger From richard.warburton at gmail.com Fri Feb 16 22:55:06 2018 From: richard.warburton at gmail.com (Richard Warburton) Date: Fri, 16 Feb 2018 22:55:06 +0000 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: <5A861702.7090102@oracle.com> References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> <5A861702.7090102@oracle.com> Message-ID: Hi gents, Thanks Sherman for taking up the bat on this one - most appreciated. These four methods encode as many characters as possible into the >>>> destination byte[] or buffer but don't give any indication that the >>>> destination didn't have enough space to encode the entire string. I thus >>>> worry they could be a hazard and result in buggy code. If there is >>>> insufficient space then the user of the API doesn't know how many >>>> characters were encoded so it's not easy to substring and call getBytes >>>> again to encode the remaining characters. There is also the issue of how to >>>> size the destination. What would you think about having them fail when >>>> there is insufficient space? If they do fail then there is a side effect >>>> that they will have written to the destination so that would need to be >>>> documented too. >>>> >>> >> I share Alan's concern here. >> >> If the intent is to reuse a byte[] or a ByteBuffer, then there needs to >> be an effective way to handle the case where the provided array/buffer >> doesn't have enough room to receive the decoded string. A variety of ways >> of dealing with this have been mentioned, such as throwing an exception; >> returning negative value to indicate failure, >> > > To add the ref for the discussion. Here is one of the alternatives being > discussed, > to return the "-length", negative value of the space needed to encode the > whole > String, to hint the caller to pass in an appropriately sized buffer (only > the String > APIs are updated). This appears to be a better than the proposed "partial > fill"/encode > as many as possible approach. But it will still force the caller to take > care of the > return value and manage the possible buffer size issue himself, which > triggers the > question that whether or not the CharsetDecoder is a better place for such > kind of > use scenario. > > http://cr.openjdk.java.net/~sherman/8021560/webrev.v2 > I think some of the context here around application level memory management of the byte buffers is missing. The getBytes() methods were supposed to be useful in scenarios where you have a String and you want to write the byte encoded version of it down some kind of connection. So you're going to take the byte[] or ByteBuffer and hand it off somewhere - either to a streaming protocol (eg: TCP), a framed message based protocol (eg: UDP, Aeron, etc.) or perhaps to a file. A common pattern for dealing with this kind of buffer is to avoid trying to allocate a new ByteBuffer for every message and to encode onto the existing buffers before writing them into something else, for example a NIO channel. The review is completely correct that the API's user needs to know when the ByteBuffer isn't large enough to deal with the encoding, but I think there are two strategies for expected API usage here if you run out of space. 1. Find out how much space you need, grow your buffer size and encode onto the bigger buffer. So this means that in the failure case the user ideally gets to know how big a buffer you need. I think this still works in terms of mitigating per message buffer allocation as in practice it means that you only allocate a larger buffer when a String is encoded that is longer than any previous String that you've seen before. It isn't strictly necessary to know how big a buffer is needed btw - as long as failure is indicated an API user could employ a strategy like double the buffer size and retry. I think that's suboptimal to say the least, however, and knowing how big a buffer needs to be is desirable. 2. Just write the bytes that you've encoded down the stream and retry with an offset incremented by the number of characters written. This requires that the getBytes() method encodes in terms of whole characters, rather than running out of space when encoding say a character that takes up multiple bytes encoded and also takes a "source offset" parameter - say the number of characters into the String that you are? This would work perfectly well in a streaming protocol. If your buffer size is N, you encode max N characters and write them down your Channel in a retry loop. Anyone dealing with async NIO is probably familiar with the concept of having a retry loop. It may also work perfectly well in a framed message based protocol. In practice any network protocol that has fixed-size framed messages and deals with arbitrary size encodings has to have a way to fragment longer-length blobs of data into its fixed size messages. I think either strategy for dealing with failure is valid, the problem is that if the API uses the return value to indicate failure, which I think is a good idea in a low-level performance oriented API then its difficult to offer both choices to the user. (1) needs the failure return code to be the number of bytes required for encoding. (2) needs the failure return code to indicate how far into the String you are in order to retry. I suspect given this tradeoff that Sherman's suggestion of using a -length (required number of bytes) return value is a good idea and just assuming API users only attempt (1) as a solution to the too-small-buffer failure. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From richard.warburton at gmail.com Fri Feb 16 23:04:10 2018 From: richard.warburton at gmail.com (Richard Warburton) Date: Fri, 16 Feb 2018 23:04:10 +0000 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> Message-ID: Hi gents, public String(ByteBuffer bytes, Charset cs); >> public String(ByteBuffer bytes, String csname); >> > > I think these constructors make good sense. They avoid an extra copy to an > intermediate byte[]. > > One issue (also mentioned by Stephen Colebourne) is whether we need the > csname overload. Arguably it's not needed if we have the Charset overload. > And the csname overload throws UnsupportedEncodingException, which is > checked. But the csname overload is apparently faster, since the decoder > can be cached, and it's unclear when this can be remedied for the Charset > case.... > > I could go either way on this one. > Yes - the original motivation for the csname overload was the ability to cache the decoder and also for consistency with other String encoding methods that normally have both the String name and also the Charset. If memory serves me correctly it was concluded that it was possible to cache the decoder iff the decoder could be trusted to not hold a reference to the formerly char[], now byte[], that sits within the String. Otherwise a rogue decoder could create a mutable string by holding a reference to its internal state for some malicious purpose. JDK charsets aren't rogue so it would be possible to check whether a charset was a JDK one and perform the caching, should that be a desirable + worthwhile optimization. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From xueming.shen at oracle.com Fri Feb 16 23:19:33 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 16 Feb 2018 15:19:33 -0800 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> Message-ID: <5A876705.7060303@oracle.com> On 02/16/2018 03:04 PM, Richard Warburton wrote: > Hi gents, > > public String(ByteBuffer bytes, Charset cs); > public String(ByteBuffer bytes, String csname); > > > I think these constructors make good sense. They avoid an extra copy to an intermediate byte[]. > > One issue (also mentioned by Stephen Colebourne) is whether we need the csname overload. Arguably it's not needed if we have the Charset overload. And the csname overload throws UnsupportedEncodingException, which is checked. But the csname overload is apparently faster, since the decoder can be cached, and it's unclear when this can be remedied for the Charset case.... > > I could go either way on this one. > > > Yes - the original motivation for the csname overload was the ability to cache the decoder and also for consistency with other String encoding methods that normally have both the String name and also the Charset. If memory serves me correctly it was concluded that it was possible to cache the decoder iff the decoder could be trusted to not hold a reference to the formerly char[], now byte[], that sits within the String. Otherwise a rogue decoder could create a mutable string by holding a reference to its internal state for some malicious purpose. JDK charsets aren't rogue so it would be possible to check whether a charset was a JDK one and perform the caching, should that be a desirable + worthwhile optimization. > > Yes, we don't cache external cs now (and yes, we still have to make some defensive copy her and there :-)) The StringCoder.java included in the v2 webrev is trying to cache the de/encoder from jdk's own charset. So if thing goes well as expected (rounds of jmh measurement) there should be no performance difference between csn and cs. In that case, should we drop the public String(ByteBuffer, String) constructor, as Stephen suggested? Though the caller then needs to get the charset somewhere, Charset.forName() for example, themselves. http://cr.openjdk.java.net/~sherman/8021560/StringCoder.java -sherman From stuart.marks at oracle.com Sat Feb 17 01:13:36 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 16 Feb 2018 17:13:36 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> Message-ID: Let me put in an argument for handling code points: > 3. public static String repeat(final int codepoint, final int count) Most of the String and Character API handles code points on an equal footing with chars. I think this is important, as over time Unicode is continuing to add supplementary characters -- those that can't be represented in a Java char value. Examples abound of how such characters are mishandled. Therefore, I believe Java APIs should have full support for code points. This is a small thing, and some might consider it a rare case -- how often does one need to repeat something like an emoji? The issue however isn't that particular use case. Instead what's required is the ability to handle *any Unicode character* uniformly, regardless of whether or not it's a supplementary character. The way to do that is to deal with code points, so any Java API that deals with character data must also handle code points. If we were to add just one method: > 1. public String repeat(final int count) the workaround is to take the character, turn it into a string, and call the repeat() method on it. For a 'char' value, this isn't too bad, but I'd argue it isn't pretty either: Character.toString(charVal).repeat(n) But this only handles BMP characters, not supplementary characters. Unfortunately, there's no direct way to turn a code point into a string -- you have to turn it into a byte array first! Thus, to get a string from a code point and repeat it, you have to do this: new String(Character.toChars(codepoint)).repeat(count) This is enough indirection that it's hard to discover, and I suspect that most people won't put in the effort to do this correctly, resulting in more code that mishandles supplementary characters. Thus, I think we need to add API #3 that performs the repeat function on code points. (Hm, the lack of Character.toString(codepoint) is covered by JDK-4993841, which is closed. I think I'll reopen it.) > 2. public static String repeat(final char ch, final int count) I can see that this API is not as important as one that handles code points, and it seems to be less frequently used according to Louis W's analysis. But if you have char data you want to repeat, not having this seems like an omission; it seems backwards to have to create a string from the char, only for repeat() to extract that char from that String in order to repeat it. Thus I've vote for inclusion of this method as well. s'marks On 2/16/18 5:10 AM, Jim Laskey wrote: > We?re going with the one instance method (Louis clinched it.) with recommended enhancements and not touching CharSequence. > > Working it up now. > > ? Jim > >> On Feb 16, 2018, at 7:46 AM, Alan Bateman wrote: >> >> On 15/02/2018 17:20, Jim Laskey wrote: >>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>> >>> The proposal is to introduce four new methods; >>> >>> 1. public String repeat(final int count) >>> 2. public static String repeat(final char ch, final int count) >>> 3. public static String repeat(final int codepoint, final int count) >>> 4. public static String repeat(final CharSequence seq, final int count) >>> >> Just catching up on this thread and it's hard to see where the bidding is currently at. Are you planning to send an updated proposal, a list of methods is fine, even if it's just one, is okay (implementation can follow later). >> >> -Alan From brian.goetz at oracle.com Sat Feb 17 02:02:28 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 16 Feb 2018 18:02:28 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> Message-ID: Disagree. On #3, most of the time the char being repeated is already a literal. So just make it a string. On #2, better to aim for string.ofCodePoint(int) and compose w repeat. Down to one method again :) Sent from my MacBook Wheel > On Feb 16, 2018, at 5:13 PM, Stuart Marks wrote: > > Let me put in an argument for handling code points: > >> 3. public static String repeat(final int codepoint, final int count) > > Most of the String and Character API handles code points on an equal footing with chars. I think this is important, as over time Unicode is continuing to add supplementary characters -- those that can't be represented in a Java char value. Examples abound of how such characters are mishandled. Therefore, I believe Java APIs should have full support for code points. > > This is a small thing, and some might consider it a rare case -- how often does one need to repeat something like an emoji? The issue however isn't that particular use case. Instead what's required is the ability to handle *any Unicode character* uniformly, regardless of whether or not it's a supplementary character. The way to do that is to deal with code points, so any Java API that deals with character data must also handle code points. > > If we were to add just one method: > >> 1. public String repeat(final int count) > > the workaround is to take the character, turn it into a string, and call the repeat() method on it. For a 'char' value, this isn't too bad, but I'd argue it isn't pretty either: > > Character.toString(charVal).repeat(n) > > But this only handles BMP characters, not supplementary characters. Unfortunately, there's no direct way to turn a code point into a string -- you have to turn it into a byte array first! Thus, to get a string from a code point and repeat it, you have to do this: > > new String(Character.toChars(codepoint)).repeat(count) > > This is enough indirection that it's hard to discover, and I suspect that most people won't put in the effort to do this correctly, resulting in more code that mishandles supplementary characters. > > Thus, I think we need to add API #3 that performs the repeat function on code points. > > (Hm, the lack of Character.toString(codepoint) is covered by JDK-4993841, which is closed. I think I'll reopen it.) > >> 2. public static String repeat(final char ch, final int count) > > I can see that this API is not as important as one that handles code points, and it seems to be less frequently used according to Louis W's analysis. But if you have char data you want to repeat, not having this seems like an omission; it seems backwards to have to create a string from the char, only for repeat() to extract that char from that String in order to repeat it. Thus I've vote for inclusion of this method as well. > > s'marks > > >> On 2/16/18 5:10 AM, Jim Laskey wrote: >> We?re going with the one instance method (Louis clinched it.) with recommended enhancements and not touching CharSequence. >> Working it up now. >> ? Jim >>> On Feb 16, 2018, at 7:46 AM, Alan Bateman wrote: >>> >>> On 15/02/2018 17:20, Jim Laskey wrote: >>>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>>> >>>> The proposal is to introduce four new methods; >>>> >>>> 1. public String repeat(final int count) >>>> 2. public static String repeat(final char ch, final int count) >>>> 3. public static String repeat(final int codepoint, final int count) >>>> 4. public static String repeat(final CharSequence seq, final int count) >>>> >>> Just catching up on this thread and it's hard to see where the bidding is currently at. Are you planning to send an updated proposal, a list of methods is fine, even if it's just one, is okay (implementation can follow later). >>> >>> -Alan > From xueming.shen at oracle.com Sat Feb 17 02:12:17 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 16 Feb 2018 18:12:17 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> Message-ID: <5A878F81.6050309@oracle.com> On 2/16/18, 5:13 PM, Stuart Marks wrote: > Let me put in an argument for handling code points: > >> 3. public static String repeat(final int codepoint, final int count) > > Most of the String and Character API handles code points on an equal > footing with chars. I think this is important, as over time Unicode is > continuing to add supplementary characters -- those that can't be > represented in a Java char value. Examples abound of how such > characters are mishandled. Therefore, I believe Java APIs should have > full support for code points. > > This is a small thing, and some might consider it a rare case -- how > often does one need to repeat something like an emoji? The issue > however isn't that particular use case. Instead what's required is the > ability to handle *any Unicode character* uniformly, regardless of > whether or not it's a supplementary character. The way to do that is > to deal with code points, so any Java API that deals with character > data must also handle code points. > > If we were to add just one method: > >> 1. public String repeat(final int count) > > the workaround is to take the character, turn it into a string, and > call the repeat() method on it. For a 'char' value, this isn't too > bad, but I'd argue it isn't pretty either: > > Character.toString(charVal).repeat(n) How about public static repeat(int count, char... chars)? String.repeat(100, '*'); String.repeat(100, 'x', 'y'); and it should not be too bad and kinda consistent to have String.repeat(n, Character.toChars(0x12345)); -sherman From brian.goetz at oracle.com Sat Feb 17 02:30:10 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 16 Feb 2018 18:30:10 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <5A878F81.6050309@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5A878F81.6050309@oracle.com> Message-ID: I really can?t see the value of more than one method. If we need other forms they should be for constructing strings not repeating strings. Sent from my MacBook Wheel > On Feb 16, 2018, at 6:12 PM, Xueming Shen wrote: > >> On 2/16/18, 5:13 PM, Stuart Marks wrote: >> Let me put in an argument for handling code points: >> >>> 3. public static String repeat(final int codepoint, final int count) >> >> Most of the String and Character API handles code points on an equal footing with chars. I think this is important, as over time Unicode is continuing to add supplementary characters -- those that can't be represented in a Java char value. Examples abound of how such characters are mishandled. Therefore, I believe Java APIs should have full support for code points. >> >> This is a small thing, and some might consider it a rare case -- how often does one need to repeat something like an emoji? The issue however isn't that particular use case. Instead what's required is the ability to handle *any Unicode character* uniformly, regardless of whether or not it's a supplementary character. The way to do that is to deal with code points, so any Java API that deals with character data must also handle code points. >> >> If we were to add just one method: >> >>> 1. public String repeat(final int count) >> >> the workaround is to take the character, turn it into a string, and call the repeat() method on it. For a 'char' value, this isn't too bad, but I'd argue it isn't pretty either: >> >> Character.toString(charVal).repeat(n) > > How about > > public static repeat(int count, char... chars)? > > String.repeat(100, '*'); > String.repeat(100, 'x', 'y'); > > and it should not be too bad and kinda consistent to have > > String.repeat(n, Character.toChars(0x12345)); > > -sherman From Alan.Bateman at oracle.com Sat Feb 17 07:02:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 17 Feb 2018 07:02:38 +0000 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> Message-ID: On 16/02/2018 21:56, Stuart Marks wrote: > : > > Seems like an argument that this function is in the wrong location. > > (Not joking.) > > I think CharBuffer.toString() is actually quite obscure. I note that > most of the Buffer classes have toString() methods that report the > *status* of the buffer, e.g. > > ??? java.nio.HeapByteBuffer[pos=0 lim=1000 cap=1000] > > Compared to other Buffers' toString() methods, CharBuffer is the > outlier: its toString() produces only the contents of the CharBuffer > without any meta-information. This is rather surprising. toString() is specified by CharSequence so the CharBuffer implementation has to return a String containing the characters in the buffer. -Alan From Alan.Bateman at oracle.com Sat Feb 17 07:45:56 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 17 Feb 2018 07:45:56 +0000 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> Message-ID: <69e8a6d9-3657-36b0-23c2-37db2775c633@oracle.com> On 16/02/2018 20:35, yumin qi wrote: > : > > > Updated bug,? and update webrev at same link: > http://cr.openjdk.java.net/~minqi/8194154/webrev1/ > > I think this version is good to go. -Alan From Alan.Bateman at oracle.com Sat Feb 17 09:05:54 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 17 Feb 2018 09:05:54 +0000 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> <5A861702.7090102@oracle.com> Message-ID: On 16/02/2018 22:55, Richard Warburton wrote: > : > I think some of the context here around application level memory management > of the byte buffers is missing. The getBytes() methods were supposed to be > useful in scenarios where you have a String and you want to write the byte > encoded version of it down some kind of connection. So you're going to take > the byte[] or ByteBuffer and hand it off somewhere - either to a streaming > protocol (eg: TCP), a framed message based protocol (eg: UDP, Aeron, etc.) > or perhaps to a file. A common pattern for dealing with this kind of buffer > is to avoid trying to allocate a new ByteBuffer for every message and to > encode onto the existing buffers before writing them into something else, > for example a NIO channel. > > The review is completely correct that the API's user needs to know when the > ByteBuffer isn't large enough to deal with the encoding, but I think there > are two strategies for expected API usage here if you run out of space. > > 1. Find out how much space you need, grow your buffer size and encode onto > the bigger buffer. So this means that in the failure case the user ideally > gets to know how big a buffer you need. I think this still works in terms > of mitigating per message buffer allocation as in practice it means that > you only allocate a larger buffer when a String is encoded that is longer > than any previous String that you've seen before. It isn't strictly > necessary to know how big a buffer is needed btw - as long as failure is > indicated an API user could employ a strategy like double the buffer size > and retry. I think that's suboptimal to say the least, however, and knowing > how big a buffer needs to be is desirable. > > 2. Just write the bytes that you've encoded down the stream and retry with > an offset incremented by the number of characters written. This requires > that the getBytes() method encodes in terms of whole characters, rather > than running out of space when encoding say a character that takes up > multiple bytes encoded and also takes a "source offset" parameter - say the > number of characters into the String that you are? This would work > perfectly well in a streaming protocol. If your buffer size is N, you > encode max N characters and write them down your Channel in a retry loop. > Anyone dealing with async NIO is probably familiar with the concept of > having a retry loop. It may also work perfectly well in a framed message > based protocol. In practice any network protocol that has fixed-size framed > messages and deals with arbitrary size encodings has to have a way to > fragment longer-length blobs of data into its fixed size messages. > > I think either strategy for dealing with failure is valid, the problem is > that if the API uses the return value to indicate failure, which I think is > a good idea in a low-level performance oriented API then its difficult to > offer both choices to the user. (1) needs the failure return code to be the > number of bytes required for encoding. (2) needs the failure return code to > indicate how far into the String you are in order to retry. I suspect given > this tradeoff that Sherman's suggestion of using a -length (required number > of bytes) return value is a good idea and just assuming API users only > attempt (1) as a solution to the too-small-buffer failure. > Just to add that the existing low-level / advanced API for this is CharsetEncoder. The CoderResult from an encode and the buffer positions means you know when there is overflow, the number of characters encoded, and how many bytes were added to the buffer. It also gives fine control on how encoding errors should be handled and you cache a CharsetEncoder to avoid some of the performance anomalies that come up in the Charset vs. charset name discussions. This is not an API that most developers will ever use directly but if the use-case is advanced cases (libraries or frameworks doing their own memory management as you mention above) then it might be an alternative to look at to avoid adding advanced use-case APIs to String. I don't think an encode(String, ByteBuffer) would look out of place although it would need a way to return the characters encoded count as part of the result. -Alan. From forax at univ-mlv.fr Sat Feb 17 09:50:05 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 17 Feb 2018 10:50:05 +0100 (CET) Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5A878F81.6050309@oracle.com> Message-ID: <1525051839.2467091.1518861005186.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Brian Goetz" > ?: "Xueming Shen" > Cc: "core-libs-dev" > Envoy?: Samedi 17 F?vrier 2018 03:30:10 > Objet: Re: RFR: 8197594 - String and character repeat > I really can?t see the value of more than one method. If we need other forms > they should be for constructing strings not repeating strings. > > Sent from my MacBook Wheel I fully agree. R?mi > >> On Feb 16, 2018, at 6:12 PM, Xueming Shen wrote: >> >>> On 2/16/18, 5:13 PM, Stuart Marks wrote: >>> Let me put in an argument for handling code points: >>> >>>> 3. public static String repeat(final int codepoint, final int count) >>> >>> Most of the String and Character API handles code points on an equal footing >>> with chars. I think this is important, as over time Unicode is continuing to >>> add supplementary characters -- those that can't be represented in a Java char >>> value. Examples abound of how such characters are mishandled. Therefore, I >>> believe Java APIs should have full support for code points. >>> >>> This is a small thing, and some might consider it a rare case -- how often does >>> one need to repeat something like an emoji? The issue however isn't that >>> particular use case. Instead what's required is the ability to handle *any >>> Unicode character* uniformly, regardless of whether or not it's a supplementary >>> character. The way to do that is to deal with code points, so any Java API that >>> deals with character data must also handle code points. >>> >>> If we were to add just one method: >>> >>>> 1. public String repeat(final int count) >>> >>> the workaround is to take the character, turn it into a string, and call the >>> repeat() method on it. For a 'char' value, this isn't too bad, but I'd argue it >>> isn't pretty either: >>> >>> Character.toString(charVal).repeat(n) >> >> How about >> >> public static repeat(int count, char... chars)? >> >> String.repeat(100, '*'); >> String.repeat(100, 'x', 'y'); >> >> and it should not be too bad and kinda consistent to have >> >> String.repeat(n, Character.toChars(0x12345)); >> > > -sherman From rcmuir at gmail.com Sat Feb 17 13:45:45 2018 From: rcmuir at gmail.com (Robert Muir) Date: Sat, 17 Feb 2018 08:45:45 -0500 Subject: RFR: JDK-8021560,(str) String constructors that take ByteBuffer In-Reply-To: References: <5A828491.8020301@oracle.com> <9a1fdaa7-de53-7180-2e25-0bdf19a2b2ca@oracle.com> <5A831997.4030005@oracle.com> <5A861702.7090102@oracle.com> Message-ID: On Sat, Feb 17, 2018 at 4:05 AM, Alan Bateman wrote: > Just to add that the existing low-level / advanced API for this is > CharsetEncoder. The CoderResult from an encode and the buffer positions > means you know when there is overflow, the number of characters encoded, and > how many bytes were added to the buffer. It also gives fine control on how > encoding errors should be handled and you cache a CharsetEncoder to avoid > some of the performance anomalies that come up in the Charset vs. charset > name discussions. > This is not an API that most developers will ever use > directly but if the use-case is advanced cases (libraries or frameworks > Really? How else are you supposed to convert bytes <-> characters reliably in java without using CharsetEncoder/Decoder? Its the only way to get an exception when something is wrong, instead of silently masking errors with replacement characters (the JDK seems to think its doing people favors there, its not). From martinrb at google.com Sat Feb 17 17:40:12 2018 From: martinrb at google.com (Martin Buchholz) Date: Sat, 17 Feb 2018 09:40:12 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <1525051839.2467091.1518861005186.JavaMail.zimbra@u-pem.fr> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5A878F81.6050309@oracle.com> <1525051839.2467091.1518861005186.JavaMail.zimbra@u-pem.fr> Message-ID: Let me join the chorus of agreement with Brian here. The most popular use case will forever be ASCII line of non-letter symbols. Another way to think about codepoints is as conversion between UTF-32 and UTF-16, or as general support for text in UTF-32 format, which is not supported well within the JDK (and we probably should not do too much work in this direction). On Sat, Feb 17, 2018 at 1:50 AM, Remi Forax wrote: > ----- Mail original ----- > > De: "Brian Goetz" > > > I really can?t see the value of more than one method. If we need other > forms > > they should be for constructing strings not repeating strings. > > > > Sent from my MacBook Wheel > > I fully agree. > > R?mi From brian.goetz at oracle.com Sat Feb 17 17:53:32 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 17 Feb 2018 09:53:32 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5A878F81.6050309@oracle.com> <1525051839.2467091.1518861005186.JavaMail.zimbra@u-pem.fr> Message-ID: > On Feb 17, 2018, at 9:40 AM, Martin Buchholz wrote: > > Let me join the chorus of agreement with Brian here. > > The most popular use case will forever be ASCII line of non-letter symbols. We may also wish to have a repeating version on StringBuilder, though: sb.append(INDENT_CHARS, indentLevel) (or a default method on Appendable with the same effect.) By way of background ? my primary motivation for these sorts of methods is to take things that require execution as _statements_ (loops, if-then, etc) and turn them into _expressions_, not primarily because they are more compact, but because they are then _composible_. Repeating a string today requires a loop (yes, I know you can do it with a stream expression), which means it can?t be done inline as a method parameter, requiring you to potentially unroll a deeply nested expression just to create a statement context to do this bit of paperwork. From xueming.shen at oracle.com Sat Feb 17 21:18:29 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Sat, 17 Feb 2018 13:18:29 -0800 Subject: [JDK-6341887] Patch: java.util.zip: Add ByteBuffer methods to Inflater/Deflater In-Reply-To: References: Message-ID: <5A889C25.60501@oracle.com> On 2/16/18, 2:13 PM, David Lloyd wrote: > It would be convenient to be able to inflate/deflate a direct or heap > byte buffer without having to copy it through an array first. For my > Friday mini-project this week, I've decided to take a crack at this. > The attached patch is the result. Would anyone be interested in > reviewing and maybe sponsoring this change? It would be really great > to get it in to JDK 11. > > The patch includes a modification to FlaterTest to run through > permutations of using direct and heap byte buffers. Though I couldn't > get jtreg working, I did compile and run the test by hand and it seems > to pass; the build also works fine with the new code. > > Extra thanks to Martin Balao for pointing me towards mapfile-vers when > I couldn't figure out why I was getting UnsatisfiedLinkError. That > one was not obvious. Hi David, Thanks for taking this one :-) some comments here. (1) I would assume you might have to do more for ByteBuffer, something like if (bf.isDirect()) { // GetDirectBufferAddress ... } else if (bf.hasArray()) { byte[] array = bf.getArray(); do(bf.getArray(), offset + pos, pos - limit); ... } else { // probably still have to copy the bytes out of ByteBuffer ... } btw, any reason go unsafe to get the byte[]? instead of ByteBuffer.getArray()? (2) It might be a concerned that you have to warp the input byte[] every time the inflate/deflate(byte[]...) is called. (Yes, I'm constantly hearing people complain that you have to wrap the byte[] to ByteBuffer to use CharsetEn/ Decoder, or even the implementation detail inside StringCoder), and it might be taken as a "regression". So it might be desired to wire the "bf.hasArray()) path inside de/encode(ByteArray) back to de/encode(byte[], int, int), to avoid the "unnecessary" ByteBuffer wrap. (3) Same "wrap concern" argument might go to the setInput(bye[] ...) as well. I'm not sure if it is worth keeping both byte[]/int/int and ByteBuffer as the "input" field of In/Deflater. (4) assume we keep the "wrap" approach. It appears ByteBuffer.wrap() does check the byte[]/off/len and throw an IndexOutOfBoundsException. So it might be better to take advantage of that check. (5) Deflater.input need to be initialized to a non-null value. Btw ZipUtil.defaultBuf needs to be "direct"? (6) It might be desired to have some jmh measure to make sure byte[] case does not have regression. Thanks, Sherman From stuart.marks at oracle.com Sun Feb 18 05:10:56 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Sat, 17 Feb 2018 21:10:56 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> Message-ID: <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> Fair enough. I'll be less unhappy if there is a way to convert from a code point to a String, as requested by JDK-4993841. This will reduce new String(Character.toChars(codepoint)).repeat(count) to Character.toString(codepoint).repeat(count) But this is still fairly roundabout. Since most cases are constants, the advice is to use a string literal instead of a char literal. This works for BMP characters, e.g. "-".repeat(10) or "\u2501".repeat(15). But if I want a non-BMP character as a string literal, I have encode it into a surrogate pair myself. For example, a string literal containing the character U+1D11A MUSICAL SYMBOL FIVE-LINE STAFF would be "\uD834\uDD1A". Ugh! Or, I could just call a function and live with it not being a constant. It would be nice if there were an escape sequence that allowed any Unicode code point, including supplementary characters, to be put to n a string literal. s'marks > On Feb 16, 2018, at 18:02, Brian Goetz wrote: > > Disagree. > > On #3, most of the time the char being repeated is already a literal. So just make it a string. > > On #2, better to aim for string.ofCodePoint(int) and compose w repeat. > > Down to one method again :) > > Sent from my MacBook Wheel > >> On Feb 16, 2018, at 5:13 PM, Stuart Marks wrote: >> >> Let me put in an argument for handling code points: >> >>> 3. public static String repeat(final int codepoint, final int count) >> >> Most of the String and Character API handles code points on an equal footing with chars. I think this is important, as over time Unicode is continuing to add supplementary characters -- those that can't be represented in a Java char value. Examples abound of how such characters are mishandled. Therefore, I believe Java APIs should have full support for code points. >> >> This is a small thing, and some might consider it a rare case -- how often does one need to repeat something like an emoji? The issue however isn't that particular use case. Instead what's required is the ability to handle *any Unicode character* uniformly, regardless of whether or not it's a supplementary character. The way to do that is to deal with code points, so any Java API that deals with character data must also handle code points. >> >> If we were to add just one method: >> >>> 1. public String repeat(final int count) >> >> the workaround is to take the character, turn it into a string, and call the repeat() method on it. For a 'char' value, this isn't too bad, but I'd argue it isn't pretty either: >> >> Character.toString(charVal).repeat(n) >> >> But this only handles BMP characters, not supplementary characters. Unfortunately, there's no direct way to turn a code point into a string -- you have to turn it into a byte array first! Thus, to get a string from a code point and repeat it, you have to do this: >> >> new String(Character.toChars(codepoint)).repeat(count) >> >> This is enough indirection that it's hard to discover, and I suspect that most people won't put in the effort to do this correctly, resulting in more code that mishandles supplementary characters. >> >> Thus, I think we need to add API #3 that performs the repeat function on code points. >> >> (Hm, the lack of Character.toString(codepoint) is covered by JDK-4993841, which is closed. I think I'll reopen it.) >> >>> 2. public static String repeat(final char ch, final int count) >> >> I can see that this API is not as important as one that handles code points, and it seems to be less frequently used according to Louis W's analysis. But if you have char data you want to repeat, not having this seems like an omission; it seems backwards to have to create a string from the char, only for repeat() to extract that char from that String in order to repeat it. Thus I've vote for inclusion of this method as well. >> >> s'marks >> >> >>> On 2/16/18 5:10 AM, Jim Laskey wrote: >>> We?re going with the one instance method (Louis clinched it.) with recommended enhancements and not touching CharSequence. >>> Working it up now. >>> ? Jim >>>> On Feb 16, 2018, at 7:46 AM, Alan Bateman wrote: >>>> >>>> On 15/02/2018 17:20, Jim Laskey wrote: >>>>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>>>> >>>>> The proposal is to introduce four new methods; >>>>> >>>>> 1. public String repeat(final int count) >>>>> 2. public static String repeat(final char ch, final int count) >>>>> 3. public static String repeat(final int codepoint, final int count) >>>>> 4. public static String repeat(final CharSequence seq, final int count) >>>>> >>>> Just catching up on this thread and it's hard to see where the bidding is currently at. Are you planning to send an updated proposal, a list of methods is fine, even if it's just one, is okay (implementation can follow later). >>>> >>>> -Alan >> > From Alan.Bateman at oracle.com Sun Feb 18 09:33:23 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 18 Feb 2018 09:33:23 +0000 Subject: [JDK-6341887] Patch: java.util.zip: Add ByteBuffer methods to Inflater/Deflater In-Reply-To: References: Message-ID: <4da286ec-6514-7682-ba8e-53a0646353c2@oracle.com> On 16/02/2018 22:13, David Lloyd wrote: > It would be convenient to be able to inflate/deflate a direct or heap > byte buffer without having to copy it through an array first. For my > Friday mini-project this week, I've decided to take a crack at this. > The attached patch is the result. Would anyone be interested in > reviewing and maybe sponsoring this change? It would be really great > to get it in to JDK 11. > > The patch includes a modification to FlaterTest to run through > permutations of using direct and heap byte buffers. Though I couldn't > get jtreg working, I did compile and run the test by hand and it seems > to pass; the build also works fine with the new code. > > Extra thanks to Martin Balao for pointing me towards mapfile-vers when > I couldn't figure out why I was getting UnsatisfiedLinkError. That > one was not obvious. Thanks for bringing this one up again, I think the last time that it was discussed here was in 2012 when Martin Kirst proposed a patch to add these methods. If you go through the archives then you'll see there were several issues with that proposal before the discussion petered out. I see Sherman is looking at implementation so I'll stick with the javadoc for now. At some point it will need a security review to ensure that there no possibility of tricking the implementation to access memory outside of the input/output. That is, we have to assume the user of setInput(ByteBuffer) is evil and will change the position/limit during deflate operations. I see the patch computes clamps the remainder before accessing memory, it will just need a closer look to make sure there are no issues. The patch will also need adjustments to make it consistent with the existing code but that can come later. On the APIs then the inflate and deflate methods look okay, I'll less sure that? setDcitionary(ByteBuffer) is really needed. Are you adding for setDcitionary(ByteBuffer) for consistency? The javadoc doesn't currently specify how it works with the buffer, e.g. inflate(ByteBuffer) doesn't specify that adds it bytes to the buffer starting at its position, it doesn't say if the position is adjusted. The javadoc will also need to set expectations on behavior when DataFormatException is thrown, is the position guaranteed to be unchanged or is it unspecified? -Alan From james.laskey at oracle.com Sun Feb 18 09:37:34 2018 From: james.laskey at oracle.com (James Laskey) Date: Sun, 18 Feb 2018 05:37:34 -0400 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> Message-ID: <8C10050B-DA2B-4F4C-B0E7-FBFA5683390C@oracle.com> Didn?t I hear someone mentioning ?\U1D11A? at some point? Sent from my iPhone > On Feb 18, 2018, at 1:10 AM, Stuart Marks wrote: > > Fair enough. I'll be less unhappy if there is a way to convert from a code point to a String, as requested by JDK-4993841. This will reduce > > new String(Character.toChars(codepoint)).repeat(count) > > to > > Character.toString(codepoint).repeat(count) > > But this is still fairly roundabout. Since most cases are constants, the advice is to use a string literal instead of a char literal. This works for BMP characters, e.g. "-".repeat(10) or "\u2501".repeat(15). But if I want a non-BMP character as a string literal, I have encode it into a surrogate pair myself. For example, a string literal containing the character U+1D11A MUSICAL SYMBOL FIVE-LINE STAFF would be "\uD834\uDD1A". Ugh! Or, I could just call a function and live with it not being a constant. It would be nice if there were an escape sequence that allowed any Unicode code point, including supplementary characters, to be put to n a string literal. > > s'marks > >> On Feb 16, 2018, at 18:02, Brian Goetz wrote: >> >> Disagree. >> >> On #3, most of the time the char being repeated is already a literal. So just make it a string. >> >> On #2, better to aim for string.ofCodePoint(int) and compose w repeat. >> >> Down to one method again :) >> >> Sent from my MacBook Wheel >> >>> On Feb 16, 2018, at 5:13 PM, Stuart Marks wrote: >>> >>> Let me put in an argument for handling code points: >>> >>>> 3. public static String repeat(final int codepoint, final int count) >>> >>> Most of the String and Character API handles code points on an equal footing with chars. I think this is important, as over time Unicode is continuing to add supplementary characters -- those that can't be represented in a Java char value. Examples abound of how such characters are mishandled. Therefore, I believe Java APIs should have full support for code points. >>> >>> This is a small thing, and some might consider it a rare case -- how often does one need to repeat something like an emoji? The issue however isn't that particular use case. Instead what's required is the ability to handle *any Unicode character* uniformly, regardless of whether or not it's a supplementary character. The way to do that is to deal with code points, so any Java API that deals with character data must also handle code points. >>> >>> If we were to add just one method: >>> >>>> 1. public String repeat(final int count) >>> >>> the workaround is to take the character, turn it into a string, and call the repeat() method on it. For a 'char' value, this isn't too bad, but I'd argue it isn't pretty either: >>> >>> Character.toString(charVal).repeat(n) >>> >>> But this only handles BMP characters, not supplementary characters. Unfortunately, there's no direct way to turn a code point into a string -- you have to turn it into a byte array first! Thus, to get a string from a code point and repeat it, you have to do this: >>> >>> new String(Character.toChars(codepoint)).repeat(count) >>> >>> This is enough indirection that it's hard to discover, and I suspect that most people won't put in the effort to do this correctly, resulting in more code that mishandles supplementary characters. >>> >>> Thus, I think we need to add API #3 that performs the repeat function on code points. >>> >>> (Hm, the lack of Character.toString(codepoint) is covered by JDK-4993841, which is closed. I think I'll reopen it.) >>> >>>> 2. public static String repeat(final char ch, final int count) >>> >>> I can see that this API is not as important as one that handles code points, and it seems to be less frequently used according to Louis W's analysis. But if you have char data you want to repeat, not having this seems like an omission; it seems backwards to have to create a string from the char, only for repeat() to extract that char from that String in order to repeat it. Thus I've vote for inclusion of this method as well. >>> >>> s'marks >>> >>> >>>> On 2/16/18 5:10 AM, Jim Laskey wrote: >>>> We?re going with the one instance method (Louis clinched it.) with recommended enhancements and not touching CharSequence. >>>> Working it up now. >>>> ? Jim >>>>> On Feb 16, 2018, at 7:46 AM, Alan Bateman wrote: >>>>> >>>>> On 15/02/2018 17:20, Jim Laskey wrote: >>>>>> This is a pre-CSR code review [1] for String repeat methods (Enhancement). >>>>>> >>>>>> The proposal is to introduce four new methods; >>>>>> >>>>>> 1. public String repeat(final int count) >>>>>> 2. public static String repeat(final char ch, final int count) >>>>>> 3. public static String repeat(final int codepoint, final int count) >>>>>> 4. public static String repeat(final CharSequence seq, final int count) >>>>>> >>>>> Just catching up on this thread and it's hard to see where the bidding is currently at. Are you planning to send an updated proposal, a list of methods is fine, even if it's just one, is okay (implementation can follow later). >>>>> >>>>> -Alan >>> >> > From amaembo at gmail.com Sun Feb 18 10:17:08 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Sun, 18 Feb 2018 17:17:08 +0700 Subject: Thread.interrupted() spec is confusing Message-ID: Hello! A Thread.interrupted() static method (not to be confused with Thread.isInterrupted() instance method) spec states: * *

A thread interruption ignored because a thread was not alive * at the time of the interrupt will be reflected by this method * returning false. The Thread.interrupted() always applies to the current thread. I don't understand how it's possible that a current thread is not alive. To me this note is redundant and should be removed. Am I missing something? With best regards, Tagir Valeev. From Ulf.Zibis at CoSoCo.de Sun Feb 18 14:02:26 2018 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Sun, 18 Feb 2018 15:02:26 +0100 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> Message-ID: <183393ea-90d4-6821-bfd7-e71f8d5b58d3@CoSoCo.de> Am 18.02.2018 um 06:10 schrieb Stuart Marks: > Fair enough. I'll be less unhappy if there is a way to convert from a code point to a String, as requested by JDK-4993841. This will reduce > > new String(Character.toChars(codepoint)).repeat(count) > > to > > Character.toString(codepoint).repeat(count) Shorter and maybe more logical to get a String by a String constructor, instead overloading toString() of Character: ??? String(codepoint).repeat(count) -Ulf From Alan.Bateman at oracle.com Sun Feb 18 18:16:30 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 18 Feb 2018 18:16:30 +0000 Subject: Thread.interrupted() spec is confusing In-Reply-To: References: Message-ID: On 18/02/2018 10:17, Tagir Valeev wrote: > Hello! > > A Thread.interrupted() static method (not to be confused with > Thread.isInterrupted() instance method) spec states: > > * > *

A thread interruption ignored because a thread was not alive > > * at the time of the interrupt will be reflected by this method > > * returning false. > > > The Thread.interrupted() always applies to the current thread. I don't > understand how it's possible that a current thread is not alive. To me this > note is redundant and should be removed. Am I missing something? > I think the wording could be improved but this about invoking Thread.interrupt before the thread is started. JDK-4082705 [1] has more on this. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-4082705 From martinrb at google.com Sun Feb 18 19:19:47 2018 From: martinrb at google.com (Martin Buchholz) Date: Sun, 18 Feb 2018 11:19:47 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: <8C10050B-DA2B-4F4C-B0E7-FBFA5683390C@oracle.com> References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> <8C10050B-DA2B-4F4C-B0E7-FBFA5683390C@oracle.com> Message-ID: On Sun, Feb 18, 2018 at 1:37 AM, James Laskey wrote: > Didn?t I hear someone mentioning ?\U1D11A? at some point? > Unicode codepoint escapes are still Not a Thing, apparently. Yes, it's a language feature, but an easy one to implement. Except for the usual bikeshedding about syntax - include the '+'? - how many digits to consume after the escape? How much do we trust Unicode to never ever grow beyond 5 hex digits? From xueming.shen at oracle.com Sun Feb 18 19:27:33 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Sun, 18 Feb 2018 11:27:33 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> <8C10050B-DA2B-4F4C-B0E7-FBFA5683390C@oracle.com> Message-ID: <5A89D3A5.5030808@oracle.com> On 2/18/18, 11:19 AM, Martin Buchholz wrote: > On Sun, Feb 18, 2018 at 1:37 AM, James Laskey > wrote: > >> Didn?t I hear someone mentioning ?\U1D11A? at some point? >> > Unicode codepoint escapes are still Not a Thing, apparently. > Yes, it's a language feature, but an easy one to implement. > Except for the usual bikeshedding about syntax > - include the '+'? > - how many digits to consume after the escape? How much do we trust > Unicode to never ever grow beyond 5 hex digits? |now we have some "similar" already in regex \x|{h...h} :-) It's hard to say "never" given we are adding so many emoji in Unicode but it will take a while to fill those unsigned space. / / From martinrb at google.com Sun Feb 18 19:50:54 2018 From: martinrb at google.com (Martin Buchholz) Date: Sun, 18 Feb 2018 11:50:54 -0800 Subject: Thread.interrupted() spec is confusing In-Reply-To: References: Message-ID: One can understand the desire to keep the spec wording between Thread.interrupted and Thread.isInterrupted consistent, but we can probably improve by @linkplain-ifying "alive" and adding the word "yet". On Sun, Feb 18, 2018 at 10:16 AM, Alan Bateman wrote: > On 18/02/2018 10:17, Tagir Valeev wrote: > >> Hello! >> >> A Thread.interrupted() static method (not to be confused with >> Thread.isInterrupted() instance method) spec states: >> >> * > va.base/share/classes/java/lang/Thread.java#l1030> >> *

A thread interruption ignored because a thread was not alive >> > va.base/share/classes/java/lang/Thread.java#l1031> >> * at the time of the interrupt will be reflected by this method >> > va.base/share/classes/java/lang/Thread.java#l1032> >> * returning false. >> >> >> The Thread.interrupted() always applies to the current thread. I don't >> understand how it's possible that a current thread is not alive. To me >> this >> note is redundant and should be removed. Am I missing something? >> >> I think the wording could be improved but this about invoking > Thread.interrupt before the thread is started. JDK-4082705 [1] has more on > this. > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-4082705 > From yumin.qi at gmail.com Mon Feb 19 05:09:31 2018 From: yumin.qi at gmail.com (yumin qi) Date: Sun, 18 Feb 2018 21:09:31 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: <69e8a6d9-3657-36b0-23c2-37db2775c633@oracle.com> References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> <69e8a6d9-3657-36b0-23c2-37db2775c633@oracle.com> Message-ID: Thanks! I need a sponsor for pushing it to jdk. Can you or someone else help to push it? Yumin On Fri, Feb 16, 2018 at 11:45 PM, Alan Bateman wrote: > On 16/02/2018 20:35, yumin qi wrote: > > : > >> >> Updated bug, and update webrev at same link: > http://cr.openjdk.java.net/~minqi/8194154/webrev1/ > > I think this version is good to go. > > -Alan > From volker.simonis at gmail.com Mon Feb 19 09:01:08 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 19 Feb 2018 10:01:08 +0100 Subject: [1] RFR(XXS): 8197927: Can't set mandatory 'java.vendor.version' property to empty string In-Reply-To: <20180216111759.459999137@eggemoggin.niobe.net> References: <20180214072629.629736428@eggemoggin.niobe.net> <20180216100202.266728211@eggemoggin.niobe.net> <20180216111759.459999137@eggemoggin.niobe.net> Message-ID: On Fri, Feb 16, 2018 at 8:17 PM, wrote: > 2018/2/16 10:59:57 -0800, volker.simonis at gmail.com: > > On Fri, Feb 16, 2018 at 7:02 PM, mark.reinhold at oracle.com wrote: > >> Of course it's possible. The specification need merely say that > >> `java.vendor.version` is a standard system property that may, or may > >> not, have a value. (Or, if you like, whose value may be `null`.) > > > > Sorry but I still don't get it. Do you agree that you can't assign NULL > to > > a system property because you'll get a NPE? > > I agree that `System.setProperty("java.vendor.version", null)` will > throw an NPE, but I don't see how this fact is relevant. > > > You could of course not assign it at all (as it is done now) in which > case > > System.getProperty("java.vendor.version") would return NULL. But that > means > > "it is not defined" which is different from "is has no value". > > Let's not get caught up in fine philosophical distinctions between "not > defined", "has no value", and `null`. Either `System::getProperty` > returns a non-`null` value, or it doesn't. That's all that matters. > > > You can > > still call System.getProperties().containsKey?("java.vendor.version") > and > > it would return false which violates that specification because it > mandates > > that a property with the "java.vendor.version" exists. > > The current specification mandates this. That's precisely the bug here. > We can revise it so that it doesn't. > > Re-targeted to 11 with priority P2 and reassigned to you. Thanks, Volker > - Mark > From Alan.Bateman at oracle.com Mon Feb 19 10:28:44 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 19 Feb 2018 10:28:44 +0000 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> <69e8a6d9-3657-36b0-23c2-37db2775c633@oracle.com> Message-ID: <3ca965ec-c472-1522-e89d-3c2092b7ed3e@oracle.com> On 19/02/2018 05:09, yumin qi wrote: > Thanks! > > I need a sponsor for pushing it to jdk. Can you or someone else help > to push it? minqi is a jdk committer. If this is you then you should be able to push it yourself. -Alan. From basinilya at gmail.com Mon Feb 19 12:43:43 2018 From: basinilya at gmail.com (Basin Ilya) Date: Mon, 19 Feb 2018 15:43:43 +0300 Subject: FileOutputStream.available() and pipe EOF In-Reply-To: <8d9a525f-b980-9efd-6b6d-29e9d20c8f27@gmail.com> References: <127e9711-345b-8baa-4578-1c44a04910b8@gmail.com> <1207461253.2760977.1518298057675.JavaMail.zimbra@u-pem.fr> <8d9a525f-b980-9efd-6b6d-29e9d20c8f27@gmail.com> Message-ID: <064bb16a-57e5-d52d-7d0c-8a45563cb506@gmail.com> >> or instead of relying on an undocumented behaviour Fine. Isn't the goal of Java to behave equally on various OSes? If there's a difference, we should adopt the best variant AND document it. And in this case it is to throw an exception on EOF. I don't know about Solaris, but on Windows it's a matter of what to do when PeekNamedPipe() returns EOF. We can even add a JVM option to restore the old behavior, like it was with String.split(). On 11.02.2018 0:35, Basin Ilya wrote: > Unfortunately, read() is > 1) uninterruptilbe > 2) Unlike sockets, close() or even Thread.stop() won't cancel a read > pipe operation on Windows > > > 11.02.2018 0:27, Remi Forax ?????: >> Hi Basin, >> or instead of relying on an undocumented behaviour, you can use any overloads of read(), if it returns -1, it's the ends of the stream. >> >> cheers, >> R?mi >> >> ----- Mail original ----- >>> De: "Basin Ilya" >>> ?: "core-libs-dev" >>> Envoy?: Samedi 10 F?vrier 2018 22:15:18 >>> Objet: FileOutputStream.available() and pipe EOF >> >>> Hi list. >>> >>> My question relates to streams returned by >>> java.lang.Process.getInputStream() >>> >>> On Linux calling available() after the other side of the pipe was closed >>> will throw: >>> >>> java.io.IOException: Stream Closed >>> >>> This is very handy, because you can distinguish EOF and a pause in >>> transmission. >>> >>> On Windows same Oracle JDK version 1.8.0_161 behaves in a traditional >>> manner and available() returns 0 in both cases. Will this ever change? From adam.farley at uk.ibm.com Mon Feb 19 13:08:01 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Mon, 19 Feb 2018 13:08:01 +0000 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: <39D8F43A-06BD-483B-8901-6F4444A8235F@oracle.com> References: <39D8F43A-06BD-483B-8901-6F4444A8235F@oracle.com> Message-ID: Hi Paul, > Hi Adam, > > From reading the thread i cannot tell if this is part of a wider solution including some yet to be proposed HotSpot changes. The wider solution would need to include some Hotspot changes, yes. I'm proposing raising a bug, committing the code we have here to "set the stage", and then we can invest more time&energy later if the concept goes down well and the community agrees to pursue the full solution. As an aside, I tried submitting a big code set (including hotspot changes) months ago, and I'm *still* struggling to find someone to commit the thing, so I figured I'd try a more gradual, staged approach this time. > > As is i would be resistant to adding such standalone internal wrapper methods to Unsafe that have no apparent benefit within the OpenJDK itself since it's a maintenance burden. I'm hoping the fact that the methods are a single line (sans comments, descriptors and curly braces) will minimise this burden. > > Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come from a DBB by looking at the call stack frame above the unsafe call? > > Thanks, > Paul. Yes that is possible, though I would advise against this because: A) Checking the call stack is expensive, and doing this every time we allocate native memory is an easy way to slow down a program, or rack up mips. and B) deciding which code path we're using based on the stack means the DBB class+method (and anything the parsing code mistakes for that class+method) can only ever allocate native memory for DBBs. What do you think? Best Regards Adam Farley > >> On Feb 14, 2018, at 3:32 AM, Adam Farley8 wrote: >> >> Hi All, >> >> Currently, diagnostic core files generated from OpenJDK seem to lump all >> of the >> native memory usages together, making it near-impossible for someone to >> figure >> out *what* is using all that memory in the event of a memory leak. >> >> The OpenJ9 VM has a feature which allows it to track the allocation of >> native >> memory for Direct Byte Buffers (DBBs), and to supply that information into >> the >> cores when they are generated. This makes it a *lot* easier to find out >> what is using >> all that native memory, making memory leak resolution less like some dark >> art, and >> more like logical debugging. >> >> To use this feature, there is a native method referenced in Unsafe.java. >> To open >> up this feature so that any VM can make use of it, the java code below >> sets the >> stage for it. This change starts letting people call DBB-specific methods >> when >> allocating native memory, and getting into the habit of using it. >> >> Thoughts? >> >> Best Regards >> >> Adam Farley >> >> P.S. Code: >> >> diff --git >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> @@ -85,7 +85,7 @@ >> // Paranoia >> return; >> } >> - UNSAFE.freeMemory(address); >> + UNSAFE.freeDBBMemory(address); >> address = 0; >> Bits.unreserveMemory(size, capacity); >> } >> @@ -118,7 +118,7 @@ >> >> long base = 0; >> try { >> - base = UNSAFE.allocateMemory(size); >> + base = UNSAFE.allocateDBBMemory(size); >> } catch (OutOfMemoryError x) { >> Bits.unreserveMemory(size, cap); >> throw x; >> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> @@ -632,6 +632,26 @@ >> } >> >> /** >> + * Allocates a new block of native memory for DirectByteBuffers, of >> the >> + * given size in bytes. The contents of the memory are >> uninitialized; >> + * they will generally be garbage. The resulting native pointer will >> + * never be zero, and will be aligned for all value types. Dispose >> of >> + * this memory by calling {@link #freeDBBMemory} or resize it with >> + * {@link #reallocateDBBMemory}. >> + * >> + * @throws RuntimeException if the size is negative or too large >> + * for the native size_t type >> + * >> + * @throws OutOfMemoryError if the allocation is refused by the >> system >> + * >> + * @see #getByte(long) >> + * @see #putByte(long, byte) >> + */ >> + public long allocateDBBMemory(long bytes) { >> + return allocateMemory(bytes); >> + } >> + >> + /** >> * Resizes a new block of native memory, to the given size in bytes. >> The >> * contents of the new block past the size of the old block are >> * uninitialized; they will generally be garbage. The resulting >> native >> @@ -687,6 +707,27 @@ >> } >> >> /** >> + * Resizes a new block of native memory for DirectByteBuffers, to the >> + * given size in bytes. The contents of the new block past the size >> of >> + * the old block are uninitialized; they will generally be garbage. >> The >> + * resulting native pointer will be zero if and only if the requested >> size >> + * is zero. The resulting native pointer will be aligned for all >> value >> + * types. Dispose of this memory by calling {@link #freeDBBMemory}, >> or >> + * resize it with {@link #reallocateDBBMemory}. The address passed >> to >> + * this method may be null, in which case an allocation will be >> performed. >> + * >> + * @throws RuntimeException if the size is negative or too large >> + * for the native size_t type >> + * >> + * @throws OutOfMemoryError if the allocation is refused by the >> system >> + * >> + * @see #allocateDBBMemory >> + */ >> + public long reallocateDBBMemory(long address, long bytes) { >> + return reallocateMemory(address, bytes); >> + } >> + >> + /** >> * Sets all bytes in a given block of memory to a fixed value >> * (usually zero). >> * >> @@ -918,6 +959,17 @@ >> checkPointer(null, address); >> } >> >> + /** >> + * Disposes of a block of native memory, as obtained from {@link >> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address >> passed >> + * to this method may be null, in which case no action is taken. >> + * >> + * @see #allocateDBBMemory >> + */ >> + public void freeDBBMemory(long address) { >> + freeMemory(address); >> + } >> + >> /// random queries >> >> /** >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From david.lloyd at redhat.com Mon Feb 19 15:49:01 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Mon, 19 Feb 2018 09:49:01 -0600 Subject: [JDK-6341887] Patch: java.util.zip: Add ByteBuffer methods to Inflater/Deflater In-Reply-To: <5A889C25.60501@oracle.com> References: <5A889C25.60501@oracle.com> Message-ID: On Sat, Feb 17, 2018 at 3:18 PM, Xueming Shen wrote: > Hi David, > > Thanks for taking this one :-) some comments here. Thanks for the review! > (1) I would assume you might have to do more for ByteBuffer, something like > [...] > btw, any reason go unsafe to get the byte[]? instead of > ByteBuffer.getArray()? Yes: input should always be settable from a read-only heap buffer without a performance penalty (i.e. copying the contents). This btw is why there is also an explicit check for read-only later on. > (2) It might be a concerned that you have to warp the input byte[] every time > the inflate/deflate(byte[]...) is called. (Yes, I'm constantly hearing people > complain that you have to wrap the byte[] to ByteBuffer to use CharsetEn/ > Decoder, or even the implementation detail inside StringCoder), and it might > be taken as a "regression". So it might be desired to wire the "bf.hasArray()) > path inside de/encode(ByteArray) back to de/encode(byte[], int, int), to avoid > the "unnecessary" ByteBuffer wrap. I can do some testing to see if there is an impact. I am working on the basis that the wrap may be optimized away (as it is in certain similar cases), but Inflater/Deflater is not final (nor are the inflate/deflate methods) that might not be true in practice. > (3) Same "wrap concern" argument might go to the setInput(bye[] ...) as well. > I'm not sure if it is worth keeping both byte[]/int/int and ByteBuffer as the > "input" field of In/Deflater. Since input is stored on the object, the above theoretical optimization is much less likely. Again I can do some testing; it might be a good idea to have separate byte[]/int/int and ByteBuffer in the end, though the added expense of checking for and updating the ByteBuffer position after each call in addition to the byte[]/int/int might nullify the benefit. Testing is required... > (4) assume we keep the "wrap" approach. It appears ByteBuffer.wrap() does > check the byte[]/off/len and throw an IndexOutOfBoundsException. So it might > be better to take advantage of that check. I wanted to however it throws the wrong exception type; I could catch and rethrow I guess though if you think that is better (assuming we keep the "wrap" approach as you say). > (5) Deflater.input need to be initialized to a non-null value. > Btw ZipUtil.defaultBuf needs to be "direct"? It doesn't need to be, but the direct buffer code path is possibly somewhat "friendlier" to GC since there's no GetPrimitiveArrayCritical call. > (6) It might be desired to have some jmh measure to make sure byte[] case > does not have regression. I'll work on this when I get a chance (maybe not until Friday though). One other thought I had this weekend was that I could possibly improve things by getting the direct buffer address on the Java side, and pass it in to JNI to avoid the calls to GetDirectBufferAddress. Another thought was to eliminate the remaining SetBooleanField calls by using the remaining two bits in the doInflate/doDeflate methods' return values. I'm not sure how expensive these calls are though. I could also replace the JNI field reads with more method parameters if this is a valuable thing to do. -- - DML From martinrb at google.com Mon Feb 19 15:55:36 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 19 Feb 2018 07:55:36 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> <8C10050B-DA2B-4F4C-B0E7-FBFA5683390C@oracle.com> Message-ID: . On Sun, Feb 18, 2018 at 11:19 AM, Martin Buchholz wrote: > > - how many digits to consume after the escape? How much do we trust > Unicode to never ever grow beyond 5 hex digits? > Oops, I already got it wrong - it's already at 6 hex digits because there are 17 planes, not 16. MAX_CODE_POINT is U+10FFFF. Yes, we need a variable width syntax like regex \x{h...h} And java regex also supports \N{name} The character with Unicode character name 'name' so we could do the same for the java language. Although it would be a little weird to have every Unicode update make some previously invalid source files valid. We could also say "It's 2018 and UTF-8 has won" and simply use UTF-8 in source files directly. No Unicode escapes needed. From david.lloyd at redhat.com Mon Feb 19 15:56:20 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Mon, 19 Feb 2018 09:56:20 -0600 Subject: [JDK-6341887] Patch: java.util.zip: Add ByteBuffer methods to Inflater/Deflater In-Reply-To: <4da286ec-6514-7682-ba8e-53a0646353c2@oracle.com> References: <4da286ec-6514-7682-ba8e-53a0646353c2@oracle.com> Message-ID: On Sun, Feb 18, 2018 at 3:33 AM, Alan Bateman wrote: > Thanks for bringing this one up again Thanks for taking the time to review. > I see Sherman is looking at implementation so I'll stick with the javadoc > for now. At some point it will need a security review to ensure that there > no possibility of tricking the implementation to access memory outside of > the input/output. That is, we have to assume the user of > setInput(ByteBuffer) is evil and will change the position/limit during > deflate operations. I see the patch computes clamps the remainder before > accessing memory, it will just need a closer look to make sure there are no > issues. The patch will also need adjustments to make it consistent with the > existing code but that can come later. I did write the code with this in mind: that the address should always be within the bounds of the buffer (be it heap or direct) at the time of the call, and that the offset into the buffer should not be beyond its end (or beginning). So the effect could be a thrown exception but escaping the buffer _should_ be impossible (by intent anyway; more eyes are always better for noticing mistakes of course). > On the APIs then the inflate and deflate methods look okay, I'll less sure > that setDcitionary(ByteBuffer) is really needed. Are you adding for > setDcitionary(ByteBuffer) for consistency? Yes; it was easy enough to add it so I did. > The javadoc doesn't currently specify how it works with the buffer, e.g. > inflate(ByteBuffer) doesn't specify that adds it bytes to the buffer > starting at its position, it doesn't say if the position is adjusted. The > javadoc will also need to set expectations on behavior when > DataFormatException is thrown, is the position guaranteed to be unchanged or > is it unspecified? I intended for it to work similarly to the old code. But I'll go through the zlib docs and just make sure all the assumptions are still correct, and the next version will have more concise docs in this regard and also with regards to the disposition of the buffer in each case. -- - DML From ben_walsh at uk.ibm.com Mon Feb 19 16:37:14 2018 From: ben_walsh at uk.ibm.com (Ben Walsh) Date: Mon, 19 Feb 2018 16:37:14 +0000 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: As requested, here are the results with modifications to the annotations on Reference.reachabilityFence. Much more promising ... * Benchmark 1 * Test Code : package org.sample; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import java.nio.ByteBuffer; public class ByteBufferBenchmark { @State(Scope.Benchmark) public static class ByteBufferContainer { ByteBuffer bb; @Setup(Level.Invocation) public void initByteBuffer() { bb = ByteBuffer.allocateDirect(1); } ByteBuffer getByteBuffer() { return bb; } } @Benchmark public void benchmark_byte_buffer_put(ByteBufferContainer bbC) { bbC.getByteBuffer().put((byte)42); } } Results : - Unmodified Build - Benchmark Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 35604933.518 ? 654975.515 ops/s - Build With Reference.reachabilityFences Added - Benchmark Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 33100911.857 ? 747461.951 ops/s -7.033% - Build With Reference.reachabilityFences Added And DontInline Replaced With ForceInline - Benchmark Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 34836320.294 ? 640188.408 ops/s -2.159% - Build With Reference.reachabilityFences Added And DontInline Removed - Benchmark Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 34740015.332 ? 556578.542 ops/s -2.429% * Benchmark 2 * Test Code : package org.sample; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Param; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import java.nio.ByteBuffer; @State(Scope.Benchmark) public class ByteBufferBenchmark { @Param({"1", "10", "100", "1000", "10000"}) public int L; @State(Scope.Benchmark) public static class ByteBufferContainer { ByteBuffer bb; @Setup(Level.Invocation) public void initByteBuffer() { bb = ByteBuffer.allocateDirect(10000); } ByteBuffer getByteBuffer() { return bb; } } @Benchmark public ByteBuffer benchmark_byte_buffer_put(ByteBufferContainer bbC) { ByteBuffer bb = bbC.getByteBuffer(); for (int i = 0; i < L; i++) { bb.put((byte)i); } return bb; } } Results : - Unmodified Build - Benchmark (L) Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 29303145.752 ? 635979.750 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 24260859.017 ? 528891.303 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 8512366.637 ? 136615.070 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 1323756.037 ? 21485.369 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 145965.305 ? 1301.469 ops/s - Build With Reference.reachabilityFences Added - Benchmark (L) Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 28893540.122 ? 754554.747 ops/s -1.398% ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 15317696.355 ? 231621.608 ops/s -36.863% ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 2546599.578 ? 32136.873 ops/s -70.084% ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 288832.514 ? 3854.522 ops/s -78.181% ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 29747.386 ? 214.831 ops/s -79.620% - Build With Reference.reachabilityFences Added And DontInline Replaced With ForceInline - Benchmark (L) Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 29372326.859 ? 525988.179 ops/s +0.236% ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 24326735.480 ? 484358.862 ops/s +0.272% ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 8492692.912 ? 120924.878 ops/s -0.231% ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 1332131.417 ? 14981.587 ops/s +0.633% ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 144990.569 ? 1518.877 ops/s -0.668% - Build With Reference.reachabilityFences Added And DontInline Removed - Benchmark (L) Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 29842696.017 ? 462902.634 ops/s +1.841% ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 24842729.069 ? 436174.452 ops/s +2.398% ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 8518393.953 ? 129254.536 ops/s +0.071% ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 1344772.370 ? 15916.867 ops/s +1.588% ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 145087.256 ? 1277.491 ops/s -0.602% * Benchmark 3 * Test Code : package org.sample; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Param; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import java.nio.ByteBuffer; @State(Scope.Benchmark) public class ByteBufferBenchmark { @Param({"1", "10", "100", "1000", "10000"}) public int L; @State(Scope.Benchmark) public static class ByteBufferContainer { ByteBuffer bb; @Setup(Level.Invocation) public void initByteBuffer() { bb = ByteBuffer.allocateDirect(4 * 10000); for (int i = 0; i < 10000; i++) { bb.putInt(i); } } ByteBuffer getByteBuffer() { return bb; } } @Benchmark public int benchmark_byte_buffer_put(ByteBufferContainer bbC) { ByteBuffer bb = bbC.getByteBuffer(); bb.position(0); int sum = 0; for (int i = 0; i < L; i++) { sum += bb.getInt(); } return sum; } } Results : - Unmodified Build - Benchmark (L) Mode Cnt Score Error Units ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 29677205.748 ? 544721.142 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 18219951.454 ? 320724.793 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 7767650.826 ? 121798.910 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 1646075.010 ? 9804.499 ops/s ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 183489.418 ? 1355.967 ops/s - Build With Reference.reachabilityFences Added - Benchmark (L) Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 15230086.695 ? 390174.190 ops/s -48.681% ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 8126310.728 ? 123661.342 ops/s -55.399% ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 1582699.233 ? 7278.744 ops/s -79.624% ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 179726.465 ? 802.333 ops/s -89.082% ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 18327.049 ? 9.506 ops/s -90.012% - Build With Reference.reachabilityFences Added And DontInline Replaced With ForceInline - Benchmark (L) Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 29839190.147 ? 576585.796 ops/s +0.546% ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 18397768.759 ? 338144.327 ops/s +0.976% ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 7746079.875 ? 101621.105 ops/s -0.278% ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 1629413.444 ? 24163.399 ops/s -1.012% ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 182250.811 ? 2028.461 ops/s -0.675% - Build With Reference.reachabilityFences Added And DontInline Removed - Benchmark (L) Mode Cnt Score Error Units Impact ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 29442980.464 ? 556324.877 ops/s -0.789% ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 18401757.539 ? 419383.901 ops/s +0.998% ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 7816766.062 ? 100144.611 ops/s +0.632% ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 1636811.564 ? 13811.447 ops/s -0.563% ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 183463.292 ? 2056.016 ops/s -0.014% Regards, Ben From: Paul Sandoz To: Ben Walsh Cc: core-libs-dev Date: 08/02/2018 16:54 Subject: Re: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup Hi Ben, Thanks. I anticipated a performance hit but not necessarily a 10x. Without looking at the generated code of the benchmark method it is hard to be sure [*], but i believe the fence is interfering with loop unrolling and/or vectorization, the comparative differences between byte and int may be related to vectorization (for byte there may be less or limited support for vectorization). How about we now try another experiment commenting out the @DontInline on the fence method and re-run the benchmarks. From Peter?s observations and Vladimir?s analysis we should be able to remove that, or even, contrary to what we initial expected when adding this feature, change to @ForceInline! Thanks, Paul. [*] If you are running on linux you can use the excellent JMH perfasm feature to dump the hot parts of HotSpots generated code. > On Feb 8, 2018, at 8:22 AM, Ben Walsh wrote: > > Hi Paul, > > Following up with the requested loop and vectorization benchmarks ... > > > (Do the vectorization benchmark results imply that the Hotspot compiler > has been unable to perform the vectorization optimisation due to the > presence of the reachabilityFence ?) > > > ----------------------------------------------------------------------------------------------------------------------- > > > Loop Benchmarking > ---- ------------ > > package org.sample; > > import org.openjdk.jmh.annotations.Benchmark; > import org.openjdk.jmh.annotations.Level; > import org.openjdk.jmh.annotations.Param; > import org.openjdk.jmh.annotations.Scope; > import org.openjdk.jmh.annotations.Setup; > import org.openjdk.jmh.annotations.State; > > import java.nio.ByteBuffer; > > @State(Scope.Benchmark) > public class ByteBufferBenchmark { > > @Param({"1", "10", "100", "1000", "10000"}) > public int L; > > @State(Scope.Benchmark) > public static class ByteBufferContainer { > > ByteBuffer bb; > > @Setup(Level.Invocation) > public void initByteBuffer() { > bb = ByteBuffer.allocateDirect(10000); > } > > ByteBuffer getByteBuffer() { > return bb; > } > } > > @Benchmark > public ByteBuffer benchmark_byte_buffer_put(ByteBufferContainer bbC) { > > ByteBuffer bb = bbC.getByteBuffer(); > > for (int i = 0; i < L; i++) { > bb.put((byte)i); > } > > return bb; > } > > } > > > Without Changes > > Benchmark (L) Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29303145.752 ? 635979.750 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 24260859.017 ? 528891.303 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 8512366.637 ? 136615.070 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1323756.037 ? 21485.369 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 145965.305 ? 1301.469 ops/s > > > With Changes > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 28893540.122 ? 754554.747 ops/s -1.398% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 15317696.355 ? 231621.608 ops/s -36.863% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 2546599.578 ? 32136.873 ops/s -70.084% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 288832.514 ? 3854.522 ops/s -78.181% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 29747.386 > ? 214.831 ops/s -79.620% > > > ----------------------------------------------------------------------------------------------------------------------- > > > Vectorization Benchmarking > ------------- ------------ > > package org.sample; > > import org.openjdk.jmh.annotations.Benchmark; > import org.openjdk.jmh.annotations.Level; > import org.openjdk.jmh.annotations.Param; > import org.openjdk.jmh.annotations.Scope; > import org.openjdk.jmh.annotations.Setup; > import org.openjdk.jmh.annotations.State; > > import java.nio.ByteBuffer; > > @State(Scope.Benchmark) > public class ByteBufferBenchmark { > > @Param({"1", "10", "100", "1000", "10000"}) > public int L; > > @State(Scope.Benchmark) > public static class ByteBufferContainer { > > ByteBuffer bb; > > @Setup(Level.Invocation) > public void initByteBuffer() { > bb = ByteBuffer.allocateDirect(4 * 10000); > > for (int i = 0; i < 10000; i++) { > bb.putInt(i); > } > } > > ByteBuffer getByteBuffer() { > return bb; > } > > } > > @Benchmark > public int benchmark_byte_buffer_put(ByteBufferContainer bbC) { > > ByteBuffer bb = bbC.getByteBuffer(); > > bb.position(0); > > int sum = 0; > > for (int i = 0; i < L; i++) { > sum += bb.getInt(); > } > > return sum; > > } > > } > > > Without Changes > > Benchmark (L) Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29677205.748 ? 544721.142 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 18219951.454 ? 320724.793 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 7767650.826 ? 121798.910 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1646075.010 ? 9804.499 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 183489.418 ? 1355.967 ops/s > > > With Changes > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 15230086.695 ? 390174.190 ops/s -48.681% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 8126310.728 ? 123661.342 ops/s -55.399% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 1582699.233 ? 7278.744 ops/s -79.624% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 179726.465 ? 802.333 ops/s -89.082% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 18327.049 > ? 9.506 ops/s -90.012% > > > > NB : For reference - for this and previous benchmarking results ... > > "Without Changes" and "With Changes" - java -version ... > > openjdk version "10-internal" 2018-03-20 > OpenJDK Runtime Environment (build 10-internal+0-adhoc.walshbp.jdk) > OpenJDK 64-Bit Server VM (build 10-internal+0-adhoc.walshbp.jdk, mixed > mode) > > > ----------------------------------------------------------------------------------------------------------------------- > > > Regards, > Ben Walsh > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Alan.Bateman at oracle.com Mon Feb 19 16:57:06 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 19 Feb 2018 16:57:06 +0000 Subject: 8196830: publicLookup().findVirtual should not return method handle to AccessibleObject.setAccessible Message-ID: <2ecbe3ae-6b8c-33fb-d18d-8ab6f258107f@oracle.com> AccessibleObject's setAccessible(boolean) is currently not caller sensitive but the overrides in Method/Field/Constructor are. This awkwardness stems from its constructor being protected and the method not being final. It is thus possible to extend the class outside of the java.lang.reflect package and override this method (at least one popular library does this). Ideally the constructor should have been package private and/or the method be final but it's not possible to change this after 20 years. The consequence of the method in the base class not being caller sensitive is that it's possible to use a minimally trusted lookup to get a method handle to the method. Paul, Mandy and I chatted about this one recently. We prototyped changes to the MH implementation to special case this method and treat it "as if" it is caller sensitive. This maximizes compatibility but has the downside that it makes it harder to audit and somewhat fragile. In the end, we concluded it would be simpler to add the @CS annotation to this method so that it is treated consistently. The downside of this is that custom AccessibleObject implementations need to override setAccessible if they want to be invoked using method handles obtained from a minimally trusted lookup. The proposed changes are simple. The removal of "checkMemberAccess" from canBeCalledVirtual is just a clean-up because this method is no longer needs special casing (it was degraded for Java SE 10 as envisaged in JEP 176). It's not the goal here to improve the performance of canBeCalledVirtual but there may be opportunities to look at that with a separate issue: ? http://cr.openjdk.java.net/~alanb/8196830/webrev/ -Alan From yumin.qi at gmail.com Mon Feb 19 18:11:09 2018 From: yumin.qi at gmail.com (yumin qi) Date: Mon, 19 Feb 2018 10:11:09 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: <3ca965ec-c472-1522-e89d-3c2092b7ed3e@oracle.com> References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> <69e8a6d9-3657-36b0-23c2-37db2775c633@oracle.com> <3ca965ec-c472-1522-e89d-3c2092b7ed3e@oracle.com> Message-ID: Yes, I am committer. Brian, do you okay with the version? If no objection, I will push it into jdk. Thanks Yumin On Mon, Feb 19, 2018 at 2:28 AM, Alan Bateman wrote: > On 19/02/2018 05:09, yumin qi wrote: > >> Thanks! >> >> I need a sponsor for pushing it to jdk. Can you or someone else help to >> push it? >> > minqi is a jdk committer. If this is you then you should be able to push > it yourself. > > -Alan. > From claes.redestad at oracle.com Tue Feb 20 10:51:15 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 20 Feb 2018 11:51:15 +0100 Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker Message-ID: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> Hi, a small regression to lambda bootstrapping came in with the recent condy merge, and it took me a while to figure out why. Before condy, the first three parameters of calls from the BSM invoker to the six parameter LambdaMetafactory::metafactory were statically known, so only the fourth through sixth param were dynamically bound to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a slew of filterArguments, rebinds, casting MHs etc). With condy, the third parameter is now an Object (in reality either a Class or a MethodType), thus not statically known. This means the MethodType sent to checkGenericInvoker will have to add a cast for this param too, thus in makePairwiseConvertByEditor we see an additional rebind, some additional time spent spinning classes etc. Effectively increasing the cost of first lambda initialization by a small amount (a couple of ms). Here came the realization that much of the static overhead of the lambda bootstrapping could be avoided altogether since we can determine and cast arguments statically for the special-but-common case of LambdaMetafactory::metafactory. By using exact type information, and even bootstrapMethod.invokeExact, no dynamic runtime checking is needed, so the time spent in makePairwiseConvertByEditor is avoided entirely. This might be a hack, but a hack that removes a large chunk of the code executed (~75% less bytecode) for the initial lambda bootstrap. Startup tests exercising lambdas show a 10-15ms improvement - the static overhead of using lambdas is now just a few milliseconds in total. Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 The patch includes a test for an experimental new metafactory method that exists only in the amber condy-folding branch. I can easily break it out and push that directly to amber once this patch syncs up there, but have tested that keeping it in here does no harm. Thanks! /Claes From forax at univ-mlv.fr Tue Feb 20 11:38:53 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 20 Feb 2018 12:38:53 +0100 (CET) Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> Message-ID: <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> Hi Claes, instead of checking each parameter of the bsmType(), why not allocating the corresponding MethodType and storing it in a static final, so you can check if the MethodType are equals using equals (as far as i remember MethodType.equals is a == in the OpenJDK implementation). in term of name why not isLambdaMetafactoryIndyBootstrapMethod and isLambdaMetafactoryCondyBoostrapMethod instead of isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? and you can remove in the signature of isLambdaMetafactoryCallSite() and replace Class by Class. cheers, R?mi ----- Mail original ----- > De: "Claes Redestad" > ?: "core-libs-dev" > Envoy?: Mardi 20 F?vrier 2018 11:51:15 > Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker > Hi, > > a small regression to lambda bootstrapping came in with the recent > condy merge, and it took me a while to figure out why. > > Before condy, the first three parameters of calls from the BSM invoker > to the six parameter LambdaMetafactory::metafactory were statically > known, so only the fourth through sixth param were dynamically bound > to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker > -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a > slew of filterArguments, rebinds, casting MHs etc). > > With condy, the third parameter is now an Object (in reality either a > Class or a MethodType), thus not statically known. This means the > MethodType sent to checkGenericInvoker will have to add a cast for > this param too, thus in makePairwiseConvertByEditor we see an > additional rebind, some additional time spent spinning classes etc. > Effectively increasing the cost of first lambda initialization by a small > amount (a couple of ms). > > Here came the realization that much of the static overhead of the > lambda bootstrapping could be avoided altogether since we can > determine and cast arguments statically for the special-but-common > case of LambdaMetafactory::metafactory. By using exact type > information, and even bootstrapMethod.invokeExact, no dynamic > runtime checking is needed, so the time spent in > makePairwiseConvertByEditor is avoided entirely. > > This might be a hack, but a hack that removes a large chunk of the > code executed (~75% less bytecode) for the initial lambda bootstrap. > Startup tests exercising lambdas show a 10-15ms improvement - the > static overhead of using lambdas is now just a few milliseconds in total. > > Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 > > The patch includes a test for an experimental new metafactory method > that exists only in the amber condy-folding branch. I can easily break it > out and push that directly to amber once this patch syncs up there, but > have tested that keeping it in here does no harm. > > Thanks! > > /Claes From claes.redestad at oracle.com Tue Feb 20 12:07:29 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 20 Feb 2018 13:07:29 +0100 Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> Message-ID: Hi R?mi, sure, MethodType.equals will do a fast == check, but then checks the param types. It sure looks cleaner, though: http://cr.openjdk.java.net/~redestad/8198418/jdk.01/ Thanks! /Claes On 2018-02-20 12:38, Remi Forax wrote: > Hi Claes, > instead of checking each parameter of the bsmType(), why not allocating the corresponding MethodType and storing it in a static final, so you can check if the MethodType are equals using equals (as far as i remember MethodType.equals is a == in the OpenJDK implementation). > > in term of name why not isLambdaMetafactoryIndyBootstrapMethod and isLambdaMetafactoryCondyBoostrapMethod instead of isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? > > and you can remove in the signature of isLambdaMetafactoryCallSite() and replace Class by Class. > > cheers, > R?mi > > ----- Mail original ----- >> De: "Claes Redestad" >> ?: "core-libs-dev" >> Envoy?: Mardi 20 F?vrier 2018 11:51:15 >> Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker >> Hi, >> >> a small regression to lambda bootstrapping came in with the recent >> condy merge, and it took me a while to figure out why. >> >> Before condy, the first three parameters of calls from the BSM invoker >> to the six parameter LambdaMetafactory::metafactory were statically >> known, so only the fourth through sixth param were dynamically bound >> to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker >> -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a >> slew of filterArguments, rebinds, casting MHs etc). >> >> With condy, the third parameter is now an Object (in reality either a >> Class or a MethodType), thus not statically known. This means the >> MethodType sent to checkGenericInvoker will have to add a cast for >> this param too, thus in makePairwiseConvertByEditor we see an >> additional rebind, some additional time spent spinning classes etc. >> Effectively increasing the cost of first lambda initialization by a small >> amount (a couple of ms). >> >> Here came the realization that much of the static overhead of the >> lambda bootstrapping could be avoided altogether since we can >> determine and cast arguments statically for the special-but-common >> case of LambdaMetafactory::metafactory. By using exact type >> information, and even bootstrapMethod.invokeExact, no dynamic >> runtime checking is needed, so the time spent in >> makePairwiseConvertByEditor is avoided entirely. >> >> This might be a hack, but a hack that removes a large chunk of the >> code executed (~75% less bytecode) for the initial lambda bootstrap. >> Startup tests exercising lambdas show a 10-15ms improvement - the >> static overhead of using lambdas is now just a few milliseconds in total. >> >> Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 >> >> The patch includes a test for an experimental new metafactory method >> that exists only in the amber condy-folding branch. I can easily break it >> out and push that directly to amber once this patch syncs up there, but >> have tested that keeping it in here does no harm. >> >> Thanks! >> >> /Claes From vladimir.x.ivanov at oracle.com Tue Feb 20 12:20:26 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 20 Feb 2018 15:20:26 +0300 Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> Message-ID: <7396edb7-4f7d-b0c0-8a09-84bd42a8413a@oracle.com> No need in MT.equals. Pointer comparison should work as well: MethodType instances are interned and all exact type checks on MethodHandles are implemented using == on their MTs. Best regards, Vladimir Ivanov On 2/20/18 3:07 PM, Claes Redestad wrote: > Hi R?mi, > > sure, MethodType.equals will do a fast == check, but then checks the > param types. It sure looks cleaner, though: > > http://cr.openjdk.java.net/~redestad/8198418/jdk.01/ > > Thanks! > > /Claes > > > On 2018-02-20 12:38, Remi Forax wrote: >> Hi Claes, >> instead of checking each parameter of the bsmType(), why not >> allocating the corresponding MethodType and storing it in a static >> final, so you can check if the MethodType are equals using equals (as >> far as i remember MethodType.equals is a == in the OpenJDK >> implementation). >> >> in term of name why not isLambdaMetafactoryIndyBootstrapMethod and >> isLambdaMetafactoryCondyBoostrapMethod instead of >> isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? >> >> and you can remove in the signature of >> isLambdaMetafactoryCallSite() and replace Class by Class. >> >> cheers, >> R?mi >> >> ----- Mail original ----- >>> De: "Claes Redestad" >>> ?: "core-libs-dev" >>> Envoy?: Mardi 20 F?vrier 2018 11:51:15 >>> Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory >>> exactly from the BootstrapMethodInvoker >>> Hi, >>> >>> a small regression to lambda bootstrapping came in with the recent >>> condy merge, and it took me a while to figure out why. >>> >>> Before condy, the first three parameters of calls from the BSM invoker >>> to the six parameter LambdaMetafactory::metafactory were statically >>> known, so only the fourth through sixth param were dynamically bound >>> to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker >>> -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a >>> slew of filterArguments, rebinds, casting MHs etc). >>> >>> With condy, the third parameter is now an Object (in reality either a >>> Class or a MethodType), thus not statically known. This means the >>> MethodType sent to checkGenericInvoker will have to add a cast for >>> this param too, thus in makePairwiseConvertByEditor we see an >>> additional rebind, some additional time spent spinning classes etc. >>> Effectively increasing the cost of first lambda initialization by a >>> small >>> amount (a couple of ms). >>> >>> Here came the realization that much of the static overhead of the >>> lambda bootstrapping could be avoided altogether since we can >>> determine and cast arguments statically for the special-but-common >>> case of LambdaMetafactory::metafactory. By using exact type >>> information, and even bootstrapMethod.invokeExact, no dynamic >>> runtime checking is needed, so the time spent in >>> makePairwiseConvertByEditor is avoided entirely. >>> >>> This might be a hack, but a hack that removes a large chunk of the >>> code executed (~75% less bytecode) for the initial lambda bootstrap. >>> Startup tests exercising lambdas show a 10-15ms improvement - the >>> static overhead of using lambdas is now just a few milliseconds in >>> total. >>> >>> Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 >>> >>> The patch includes a test for an experimental new metafactory method >>> that exists only in the amber condy-folding branch. I can easily >>> break it >>> out and push that directly to amber once this patch syncs up there, but >>> have tested that keeping it in here does no harm. >>> >>> Thanks! >>> >>> /Claes > From claes.redestad at oracle.com Tue Feb 20 12:27:43 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 20 Feb 2018 13:27:43 +0100 Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: <7396edb7-4f7d-b0c0-8a09-84bd42a8413a@oracle.com> References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> <7396edb7-4f7d-b0c0-8a09-84bd42a8413a@oracle.com> Message-ID: <8d8a924f-482e-be93-8e3f-f541a63d14b0@oracle.com> You also pointed out that if the params or return types doesn't match, we'd get a CCE sooner or later, making the return and argument checks superfluous. This all simplifies into this, then: http://cr.openjdk.java.net/~redestad/8198418/jdk.02/ Thanks! /Claes On 2018-02-20 13:20, Vladimir Ivanov wrote: > No need in MT.equals. Pointer comparison should work as well: > MethodType instances are interned and all exact type checks on > MethodHandles are implemented using == on their MTs. > > Best regards, > Vladimir Ivanov > > On 2/20/18 3:07 PM, Claes Redestad wrote: >> Hi R?mi, >> >> sure, MethodType.equals will do a fast == check, but then checks the >> param types. It sure looks cleaner, though: >> >> http://cr.openjdk.java.net/~redestad/8198418/jdk.01/ >> >> Thanks! >> >> /Claes >> >> >> On 2018-02-20 12:38, Remi Forax wrote: >>> Hi Claes, >>> instead of checking each parameter of the bsmType(), why not >>> allocating the corresponding MethodType and storing it in a static >>> final, so you can check if the MethodType are equals using equals >>> (as far as i remember MethodType.equals is a == in the OpenJDK >>> implementation). >>> >>> in term of name why not isLambdaMetafactoryIndyBootstrapMethod and >>> isLambdaMetafactoryCondyBoostrapMethod instead of >>> isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? >>> >>> and you can remove in the signature of >>> isLambdaMetafactoryCallSite() and replace Class by Class. >>> >>> cheers, >>> R?mi >>> >>> ----- Mail original ----- >>>> De: "Claes Redestad" >>>> ?: "core-libs-dev" >>>> Envoy?: Mardi 20 F?vrier 2018 11:51:15 >>>> Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory >>>> exactly from the BootstrapMethodInvoker >>>> Hi, >>>> >>>> a small regression to lambda bootstrapping came in with the recent >>>> condy merge, and it took me a while to figure out why. >>>> >>>> Before condy, the first three parameters of calls from the BSM invoker >>>> to the six parameter LambdaMetafactory::metafactory were statically >>>> known, so only the fourth through sixth param were dynamically bound >>>> to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker >>>> -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a >>>> slew of filterArguments, rebinds, casting MHs etc). >>>> >>>> With condy, the third parameter is now an Object (in reality either a >>>> Class or a MethodType), thus not statically known. This means the >>>> MethodType sent to checkGenericInvoker will have to add a cast for >>>> this param too, thus in makePairwiseConvertByEditor we see an >>>> additional rebind, some additional time spent spinning classes etc. >>>> Effectively increasing the cost of first lambda initialization by a >>>> small >>>> amount (a couple of ms). >>>> >>>> Here came the realization that much of the static overhead of the >>>> lambda bootstrapping could be avoided altogether since we can >>>> determine and cast arguments statically for the special-but-common >>>> case of LambdaMetafactory::metafactory. By using exact type >>>> information, and even bootstrapMethod.invokeExact, no dynamic >>>> runtime checking is needed, so the time spent in >>>> makePairwiseConvertByEditor is avoided entirely. >>>> >>>> This might be a hack, but a hack that removes a large chunk of the >>>> code executed (~75% less bytecode) for the initial lambda bootstrap. >>>> Startup tests exercising lambdas show a 10-15ms improvement - the >>>> static overhead of using lambdas is now just a few milliseconds in >>>> total. >>>> >>>> Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ >>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 >>>> >>>> The patch includes a test for an experimental new metafactory method >>>> that exists only in the amber condy-folding branch. I can easily >>>> break it >>>> out and push that directly to amber once this patch syncs up there, >>>> but >>>> have tested that keeping it in here does no harm. >>>> >>>> Thanks! >>>> >>>> /Claes >> From forax at univ-mlv.fr Tue Feb 20 13:25:51 2018 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Tue, 20 Feb 2018 14:25:51 +0100 (CET) Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: <8d8a924f-482e-be93-8e3f-f541a63d14b0@oracle.com> References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> <7396edb7-4f7d-b0c0-8a09-84bd42a8413a@oracle.com> <8d8a924f-482e-be93-8e3f-f541a63d14b0@oracle.com> Message-ID: <1138512006.805432.1519133151223.JavaMail.zimbra@u-pem.fr> Looks good ! R?mi ----- Mail original ----- > De: "Claes Redestad" > ?: "Vladimir Ivanov" , "Remi Forax" > Cc: "core-libs-dev" > Envoy?: Mardi 20 F?vrier 2018 13:27:43 > Objet: Re: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker > You also pointed out that if the params or return types doesn't match, > we'd get a CCE sooner or later, making the return and argument checks > superfluous. This all simplifies into this, then: > > http://cr.openjdk.java.net/~redestad/8198418/jdk.02/ > > Thanks! > > /Claes > > > On 2018-02-20 13:20, Vladimir Ivanov wrote: >> No need in MT.equals. Pointer comparison should work as well: >> MethodType instances are interned and all exact type checks on >> MethodHandles are implemented using == on their MTs. >> >> Best regards, >> Vladimir Ivanov >> >> On 2/20/18 3:07 PM, Claes Redestad wrote: >>> Hi R?mi, >>> >>> sure, MethodType.equals will do a fast == check, but then checks the >>> param types. It sure looks cleaner, though: >>> >>> http://cr.openjdk.java.net/~redestad/8198418/jdk.01/ >>> >>> Thanks! >>> >>> /Claes >>> >>> >>> On 2018-02-20 12:38, Remi Forax wrote: >>>> Hi Claes, >>>> instead of checking each parameter of the bsmType(), why not >>>> allocating the corresponding MethodType and storing it in a static >>>> final, so you can check if the MethodType are equals using equals >>>> (as far as i remember MethodType.equals is a == in the OpenJDK >>>> implementation). >>>> >>>> in term of name why not isLambdaMetafactoryIndyBootstrapMethod and >>>> isLambdaMetafactoryCondyBoostrapMethod instead of >>>> isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? >>>> >>>> and you can remove in the signature of >>>> isLambdaMetafactoryCallSite() and replace Class by Class. >>>> >>>> cheers, >>>> R?mi >>>> >>>> ----- Mail original ----- >>>>> De: "Claes Redestad" >>>>> ?: "core-libs-dev" >>>>> Envoy?: Mardi 20 F?vrier 2018 11:51:15 >>>>> Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory >>>>> exactly from the BootstrapMethodInvoker >>>>> Hi, >>>>> >>>>> a small regression to lambda bootstrapping came in with the recent >>>>> condy merge, and it took me a while to figure out why. >>>>> >>>>> Before condy, the first three parameters of calls from the BSM invoker >>>>> to the six parameter LambdaMetafactory::metafactory were statically >>>>> known, so only the fourth through sixth param were dynamically bound >>>>> to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker >>>>> -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a >>>>> slew of filterArguments, rebinds, casting MHs etc). >>>>> >>>>> With condy, the third parameter is now an Object (in reality either a >>>>> Class or a MethodType), thus not statically known. This means the >>>>> MethodType sent to checkGenericInvoker will have to add a cast for >>>>> this param too, thus in makePairwiseConvertByEditor we see an >>>>> additional rebind, some additional time spent spinning classes etc. >>>>> Effectively increasing the cost of first lambda initialization by a >>>>> small >>>>> amount (a couple of ms). >>>>> >>>>> Here came the realization that much of the static overhead of the >>>>> lambda bootstrapping could be avoided altogether since we can >>>>> determine and cast arguments statically for the special-but-common >>>>> case of LambdaMetafactory::metafactory. By using exact type >>>>> information, and even bootstrapMethod.invokeExact, no dynamic >>>>> runtime checking is needed, so the time spent in >>>>> makePairwiseConvertByEditor is avoided entirely. >>>>> >>>>> This might be a hack, but a hack that removes a large chunk of the >>>>> code executed (~75% less bytecode) for the initial lambda bootstrap. >>>>> Startup tests exercising lambdas show a 10-15ms improvement - the >>>>> static overhead of using lambdas is now just a few milliseconds in >>>>> total. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ >>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 >>>>> >>>>> The patch includes a test for an experimental new metafactory method >>>>> that exists only in the amber condy-folding branch. I can easily >>>>> break it >>>>> out and push that directly to amber once this patch syncs up there, >>>>> but >>>>> have tested that keeping it in here does no harm. >>>>> >>>>> Thanks! >>>>> >>>>> /Claes From ben_walsh at uk.ibm.com Tue Feb 20 14:11:18 2018 From: ben_walsh at uk.ibm.com (Ben Walsh) Date: Tue, 20 Feb 2018 14:11:18 +0000 Subject: Fw: [PATCH] ObjectInputStream Reading Performance Optimisation Message-ID: Still looking for a sponsor for this please ... Thanks, Ben Walsh ----- Forwarded by Ben Walsh/UK/IBM on 20/02/2018 14:04 ----- From: Ben Walsh/UK/IBM To: core-libs-dev Date: 07/02/2018 14:18 Subject: [PATCH] ObjectInputStream Reading Performance Optimisation As per the guidance here - http://mail.openjdk.java.net/pipermail/valhalla-dev/2018-February/003772.html , I am resubmitting this patch to this mailing list ... The main advantage of this optimisation is that a suitably "aware" compiler can avoid stack walking by using a private method named redirectedReadObject that can be called from a call site that was calling readObject originally (in some cases). The conditions when this can be done are essentially when we know that the caller method's class is a "user defined class" because we have that caller method/class at compile time and can check if it was loaded by a user defined class loader. If so, the compiler emits code to call the private redirectedReadObject method (instead of readObject that was there in the byte codes) and it avoids doing a stack walk to discover the latest user class loader because it has an extra parameter that is a class loader that can be passed in, in cases when the compiler knows the answer at compile time. In such cases, the compiler generates code to pass in the latest user defined class loader known to redirectedReadObject. I would like to pair with a sponsor to contribute this patch ... ----------------------------------------------------------------------------------------------------------------------- diff -r 60c19c384333 src/java.base/share/classes/java/io/ClassCache.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/java.base/share/classes/java/io/ClassCache.java Fri Feb 02 11:52:41 2018 +0000 @@ -0,0 +1,403 @@ +/* + * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Oracle designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +package java.io; + +import java.lang.ref.ReferenceQueue; +import java.lang.ref.WeakReference; +import java.security.AccessController; +import java.security.PrivilegedAction; +import java.util.Objects; +import java.util.concurrent.ConcurrentHashMap; +/* ClassCache is Primarily responsible for Caching the results of the className lookups and hence to avoid + * multiple Lookup for same Class Instance. + * ClassCache provides a ConcurrentHash based ClassCache which is looked up prior to calling the class.forName + * Method resolveClass() from ObjectInputStream uses this Cache. + * + * Caching is done only when the actually used loader for a Class is one of the Sytem loaders (ie) App Class Loader, + * System Extension loader and BootStrap loader + * + */ +final class ClassCache { +/* Main Cache for storing the Class.forName results Here Key used would be CacheKey */ + private final ConcurrentHashMap cache = + new ConcurrentHashMap(); +/* Initiating Loader to CacheKey mapping in the Cache used by Reaper thread for removal on stale Loaders */ + private final ConcurrentHashMap loaderKeys = + new ConcurrentHashMap(); + /* Keeps a link of Actual System Loader used to Initiating Loader mapping */ + private final ConcurrentHashMap canonicalLoaderRefs = + new ConcurrentHashMap(); + /* Reference Queue registered for notification on stale Loaders */ + private final ReferenceQueue staleLoaderRefs = + new ReferenceQueue(); +/* + * Constructor Populates Canonical Loader Refs with System Loader Entries and initializes the reaper thread which + * monitors the ReferenceQueue for stale loaders + */ + + public ClassCache() { + ClassLoader loader = ClassLoader.getSystemClassLoader(); + while (loader != null) { + setCanonicalSystemLoaderRef(loader); + loader = loader.getParent(); + } + setCanonicalSystemLoaderRef(null); + AccessController.doPrivileged( + new CreateReaperAction(this, staleLoaderRefs)).start(); + } + /* + * sets Canonical Loader reference for the loader + */ + + private void setCanonicalSystemLoaderRef(ClassLoader loader) { + LoaderRef newKey = new LoaderRef(loader, staleLoaderRefs, true); + assert (canonicalLoaderRefs.put(newKey, newKey) == null); + } + + /* + * get Canonical Loader reference for the loader + */ + + + LoaderRef getCanonicalLoaderRef(Object loaderObj) { + LoaderRef newKey = new LoaderRef(loaderObj, staleLoaderRefs); + + LoaderRef key = canonicalLoaderRefs.get(newKey); + if (key == null) { + key = canonicalLoaderRefs.putIfAbsent(newKey, newKey); + if (key == null) { + return newKey; + } + } + + newKey.clear(); + return key; + } +/* + * Remove unused LoaderKey, and initiates corresponding CacheKey entry the main Cache + */ + void removeStaleRef(LoaderRef loaderRef) { + canonicalLoaderRefs.remove(loaderRef); + CacheKey key = loaderKeys.remove(loaderRef); + while (key != null) { + cache.remove(key); + key = key.next; + } + } + +/* + * Identifies if the loader used to load is one of the system loaders, + * if so updates the cache and the LoaderKey, and also a StaleLoaderReference for the initiating LoaderObject + * via LoaderRef Constructor + */ + void update(CacheKey key, Class result) { + Object resultLoaderObj = + LoaderRef.getLoaderObj(result.getClassLoader()); + if (getCanonicalLoaderRef(resultLoaderObj).isSystem == false) { + return; + } + + Object oldValue = cache.replace(key, result); + assert (oldValue instanceof FutureValue) : + ("Value replaced is of type '" + oldValue.getClass().getName() + + "', not of type '" + FutureValue.class.getName() + "'."); + + LoaderRef loaderRef = key.loaderRef; + if (loaderRef.isSystem == false) { + key.next = loaderKeys.get(loaderRef); + if (key.next == null) { + key.next = loaderKeys.putIfAbsent(loaderRef, key); + if (key.next == null) return; + } + while (!loaderKeys.replace(loaderRef, key.next, key)) { + key.next = loaderKeys.get(loaderRef); + } + } + } +/* + * Creates a New Entry in the Cache + */ + private Object createEntry(CacheKey key) { + FutureValue newValue = new FutureValue(key, this); //Does actual call to class.forName as required. + Object value = cache.putIfAbsent(key, newValue); + if (value == null) value = newValue; + return value; + } +/* + * This is the entry point in to the cache from ObjectInputStream. First Lookup is done based on the className and Loader + */ + public Class get(String className, ClassLoader loader) + throws ClassNotFoundException { + LookupKey key = new LookupKey(className, loader, this); + Object value = cache.get(key); + if (value == null) { + value = createEntry(key.createCacheKey()); + } + + if (value instanceof FutureValue) { + + return ((FutureValue)value).get(); + } + + return (Class)value; + } + + /* + * FutureValue implements Future Mechanics that is required for addressing contention issues in the HashMap + */ + + private static final class FutureValue { + private final CacheKey key; + private final LoaderRef loaderRef; + private final ClassCache cache; + private Class value = null; + + FutureValue(CacheKey key, ClassCache cache) { + this.key = key; + this.loaderRef = key.loaderRef; + this.cache = cache; + } + + /* + * tries to get the value from Cache, if not available tries to do get class.forName with active loader + * and replace the entry in the cache as required + */ + Class get() throws ClassNotFoundException { + synchronized (this) { + if (value != null) return value; + value = Class.forName(key.className, false, + loaderRef.getActiveLoader()); + } + if (value != null) { + cache.update(key, value); + } + + return value; + } + } + + private static final class CreateReaperAction + implements PrivilegedAction { + private final ClassCache cache; + private final ReferenceQueue queue; + + CreateReaperAction(ClassCache cache, ReferenceQueue queue) { + this.cache = cache; + this.queue = queue; + } + + public Thread run() { + return new Reaper(cache, queue); + } + } + + private static final class Reaper extends Thread { + private final WeakReference cacheRef; + private final ReferenceQueue queue; + + Reaper(ClassCache cache, ReferenceQueue queue) { + super("ClassCache Reaper"); + this.queue = queue; + cacheRef = new WeakReference(cache, queue); + setDaemon(true); + setContextClassLoader(null); + } +/* + * Blocks on remove() on queur reference and calls processStaleRef() when any loader is removed.(non-Javadoc) + * @see java.lang.Thread#run() + */ + public void run() { + Object staleRef = null; + do { + try { + staleRef = queue.remove(); + if (staleRef == cacheRef) break; + + processStaleRef((LoaderRef)staleRef); + } catch (InterruptedException e) { } + } while (true); + } + + private void processStaleRef(LoaderRef staleRef) { + ClassCache cache = cacheRef.get(); + if (cache == null) return; + + cache.removeStaleRef(staleRef); + } + } + + /* + * The use of the loaderRefs map is to allow efficient processing + * of one Weak Reference for each stale ClassLoader, rather than one WR for each entry in the cache. + * + * CacheKey as well as Lookup Key will be refering to this LoaderRef. + * + * Initiating Class Loaders needs to be referred for both lookup and Caching,so for performance reasons + * a LoaderRef is maintained which would be used by both LookupKey and CachingKey + * (ie) LookupKey will actually store the LoaderObj, but it will be canonically refered via a loaderRed + * by the caching Key + * LoaderKey has LoaderRef Objects as well and is used to Link the Initiating Loader with the actual cache Entries + * which is used to remove Stale reference entries. + */ + + private static class LoaderRef extends WeakReference { + private static final String NULL_LOADER = new String(""); + private final int hashcode; + public final boolean isSystem; + + static Object getLoaderObj(ClassLoader loader) { + return ((loader == null) ? NULL_LOADER : loader); + } + + LoaderRef(Object loaderObj, ReferenceQueue queue) { + this(false, Objects.requireNonNull(loaderObj), queue); + } + + LoaderRef(ClassLoader loader, ReferenceQueue queue, + boolean isSystem) { + this(isSystem, getLoaderObj(loader), queue); + } + + /* + * Creates a new weak reference that refers to the given object and is registered with the given queue. + */ + + private LoaderRef(boolean isSystem, Object loaderObj, + ReferenceQueue queue) { + super(loaderObj, queue); + String loaderClassName = ((loaderObj == NULL_LOADER) ? + NULL_LOADER : loaderObj.getClass().getName()); + hashcode = (loaderClassName.hashCode() + + System.identityHashCode(loaderObj)); + this.isSystem = isSystem; + } + + public final boolean equals(Object o) { + if (this == o) return true; + if (!(o instanceof LoaderRef)) return false; + + Object loader = get(); + return ((loader != null) && (loader == ((LoaderRef)o).get())); + } + + public final int hashCode() { + return hashcode; + } + + ClassLoader getActiveLoader() { + Object loaderObj = Objects.requireNonNull(get()); + return ((loaderObj == NULL_LOADER) ? null : (ClassLoader)loaderObj); + } + } + /* + * For better clarity and to avoid multiple lookups to the cache. Key is implemented to have + * one abstract key to final sub classes which serve specific purpose + * LookupKey - This is a short lived key, not part of any hashmap and stores the strong reference to + * loaderobject + * CachingKey - uses the same hash as LookupKey and has a means to be generated from LookupKey and has reference + * to the Loaderobj via a weakreference. + */ + + private static abstract class Key { + public final String className; + protected final int hashcode; + + protected Key(String className, int hashcode) { + this.className = className; + this.hashcode = hashcode; + } + + abstract Object getLoaderObj(); + + public final boolean equals(Object o) { + if (this == o) return true; + if (!(o instanceof Key)) return false; + + Key k = (Key)o; + Object loaderObj = getLoaderObj(); + return (className.equals(k.className) && + (loaderObj != null) && (loaderObj == k.getLoaderObj())); + } + + public final int hashCode() { + return hashcode; + } + } + /* + * Lookup Key hash code is framed using loadername hash + loader's system identity hashcode. This + * is same as the hashcode maintained in CacheKey + */ + + private static final class LookupKey extends Key { + private final Object loaderObj; + private final ClassCache cache; + + private static int hashCode(String className, ClassLoader loader) { + int hashcode = className.hashCode(); + if (loader != null) { + hashcode += (loader.getClass().getName().hashCode() + + System.identityHashCode(loader)); + } + return hashcode; + } + + public LookupKey(String className, ClassLoader loader, + ClassCache cache) { + super(Objects.requireNonNull(className), + hashCode(className, loader)); + loaderObj = LoaderRef.getLoaderObj(loader); + this.cache = cache; + } + + Object getLoaderObj() { + return loaderObj; + } + + CacheKey createCacheKey() { + return new CacheKey(className, hashcode, + cache.getCanonicalLoaderRef(loaderObj)); + } + } + + /* + * CacheKey is the actual key that is stored in the cache, and it stores the weakreference + * of the Initiating loader object via loaderRef + * + */ + private static final class CacheKey extends Key { + public final LoaderRef loaderRef; + public CacheKey next = null; + + CacheKey(String className, int hashcode, LoaderRef loaderRef) { + super(className, hashcode); + this.loaderRef = loaderRef; + } + + Object getLoaderObj() { + return loaderRef.get(); + } + } +} diff -r 60c19c384333 src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java Wed Jan 31 15:15:09 2018 -0800 +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java Fri Feb 02 11:52:41 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1996, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1996, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -308,6 +308,32 @@ private ObjectInputFilter serialFilter; /** + * cache LUDCL (Latest User Defined Class Loader) till completion of + * read* requests + */ + + /* ClassCache Entry for caching class.forName results upon enableClassCaching */ + private static final ClassCache classCache; + private static final boolean isClassCachingEnabled; + static { + isClassCachingEnabled = + AccessController.doPrivileged(new GetClassCachingSettingAction()); + classCache = (isClassCachingEnabled ? new ClassCache() : null); + } + + + /** if true LUDCL/forName results would be cached, true by default */ + private static final class GetClassCachingSettingAction + implements PrivilegedAction { + public Boolean run() { + String property = + System.getProperty("openjdk.enableClassCaching", "true"); + return property.equalsIgnoreCase("true"); + } + } + private ClassLoader cachedLudcl; + + /** * Creates an ObjectInputStream that reads from the specified InputStream. * A serialization stream header is read from the stream and verified. * This constructor will block until the corresponding ObjectOutputStream @@ -412,10 +438,64 @@ public final Object readObject() throws IOException, ClassNotFoundException { + return readObjectImpl(null); + } + + /** + * Whenever jit compiler encounters processing readObject() method + * it will replace the call with redirectedReadObject() method to improve + * the performance for custom serialisation. JIT provide the class loader through + * the caller parameter to avoid the stack walk through while calling + * latestUserDefinedLoader(). + * + * @throws ClassNotFoundException if the class of a serialized object + * could not be found. + * @throws IOException if an I/O error occurs. + * + */ + + private static Object redirectedReadObject(ObjectInputStream iStream, Class caller) + throws ClassNotFoundException, IOException + { + return iStream.readObjectImpl(caller); + } + + /** + * Actual implementation of readObject method which fetches classloader using + * latestUserDefinedLoader() method if caller is null. If caller is not null which means + * jit passes the class loader info and hence avoids calling latestUserDefinedLoader() + * method call to improve the performance for custom serialisation. + * + * @throws ClassNotFoundException if the class of a serialized object + * could not be found. + * @throws IOException if an I/O error occurs. + */ + private Object readObjectImpl(Class caller) + throws ClassNotFoundException, IOException + { + if (enableOverride) { return readObjectOverride(); } + ClassLoader oldCachedLudcl = null; + boolean setCached = false; + + if ((curContext == null) && (isClassCachingEnabled)) { + oldCachedLudcl = cachedLudcl; + + // If caller is not provided, follow the standard path to get the cachedLudcl. + // Otherwise use the class loader provided by JIT as the cachedLudcl. + + if (caller == null) { + cachedLudcl = latestUserDefinedLoader(); + } else { + cachedLudcl = caller.getClassLoader(); + } + + setCached = true; + } + // if nested read, passHandle contains handle of enclosing object int outerHandle = passHandle; try { @@ -432,6 +512,9 @@ return obj; } finally { passHandle = outerHandle; + if (setCached) { + cachedLudcl = oldCachedLudcl; + } if (closed && depth == 0) { clear(); } @@ -511,6 +594,16 @@ * @since 1.4 */ public Object readUnshared() throws IOException, ClassNotFoundException { + + ClassLoader oldCachedLudcl = null; + boolean setCached = false; + + if ((curContext == null) && (isClassCachingEnabled)) { + oldCachedLudcl = cachedLudcl; + cachedLudcl = latestUserDefinedLoader(); + setCached = true; + } + // if nested read, passHandle contains handle of enclosing object int outerHandle = passHandle; try { @@ -527,6 +620,9 @@ return obj; } finally { passHandle = outerHandle; + if (setCached) { + cachedLudcl = oldCachedLudcl; + } if (closed && depth == 0) { clear(); } @@ -682,7 +778,10 @@ { String name = desc.getName(); try { - return Class.forName(name, false, latestUserDefinedLoader()); + return ((classCache == null) ? + Class.forName(name, false, latestUserDefinedLoader()) : + classCache.get(name, cachedLudcl)); + } catch (ClassNotFoundException ex) { Class cl = primClasses.get(name); if (cl != null) { ----------------------------------------------------------------------------------------------------------------------- Regards, Ben Walsh Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Alan.Bateman at oracle.com Tue Feb 20 14:31:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 20 Feb 2018 14:31:04 +0000 Subject: Fw: [PATCH] ObjectInputStream Reading Performance Optimisation In-Reply-To: References: Message-ID: <18b96932-e5f2-9c4b-bc87-1af44518cbd9@oracle.com> On 20/02/2018 14:11, Ben Walsh wrote: > Still looking for a sponsor for this please ... Hard to know what to do with this. As it stands, the patch isn't really suitable for OpenJDK as it doesn't have the HotSpot changes. In addition, I think we have to very cautious about changing very security sensitive code when we have no way to audit, test, or exercise the changes. -Alan From brian.goetz at oracle.com Tue Feb 20 15:05:33 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 20 Feb 2018 07:05:33 -0800 Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: <8d8a924f-482e-be93-8e3f-f541a63d14b0@oracle.com> References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> <7396edb7-4f7d-b0c0-8a09-84bd42a8413a@oracle.com> <8d8a924f-482e-be93-8e3f-f541a63d14b0@oracle.com> Message-ID: <4ECFA09C-9A7C-4B04-B899-2BE59FF0B258@oracle.com> Add a comment to LMF to remember to update the hack if additional sigs are added. Sent from my iPad > On Feb 20, 2018, at 4:27 AM, Claes Redestad wrote: > > You also pointed out that if the params or return types doesn't match, we'd get a CCE sooner or later, making the return and argument checks superfluous. This all simplifies into this, then: > > http://cr.openjdk.java.net/~redestad/8198418/jdk.02/ > > Thanks! > > /Claes > > >> On 2018-02-20 13:20, Vladimir Ivanov wrote: >> No need in MT.equals. Pointer comparison should work as well: MethodType instances are interned and all exact type checks on MethodHandles are implemented using == on their MTs. >> >> Best regards, >> Vladimir Ivanov >> >>> On 2/20/18 3:07 PM, Claes Redestad wrote: >>> Hi R?mi, >>> >>> sure, MethodType.equals will do a fast == check, but then checks the param types. It sure looks cleaner, though: >>> >>> http://cr.openjdk.java.net/~redestad/8198418/jdk.01/ >>> >>> Thanks! >>> >>> /Claes >>> >>> >>>> On 2018-02-20 12:38, Remi Forax wrote: >>>> Hi Claes, >>>> instead of checking each parameter of the bsmType(), why not allocating the corresponding MethodType and storing it in a static final, so you can check if the MethodType are equals using equals (as far as i remember MethodType.equals is a == in the OpenJDK implementation). >>>> >>>> in term of name why not isLambdaMetafactoryIndyBootstrapMethod and isLambdaMetafactoryCondyBoostrapMethod instead of isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? >>>> >>>> and you can remove in the signature of isLambdaMetafactoryCallSite() and replace Class by Class. >>>> >>>> cheers, >>>> R?mi >>>> >>>> ----- Mail original ----- >>>>> De: "Claes Redestad" >>>>> ?: "core-libs-dev" >>>>> Envoy?: Mardi 20 F?vrier 2018 11:51:15 >>>>> Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker >>>>> Hi, >>>>> >>>>> a small regression to lambda bootstrapping came in with the recent >>>>> condy merge, and it took me a while to figure out why. >>>>> >>>>> Before condy, the first three parameters of calls from the BSM invoker >>>>> to the six parameter LambdaMetafactory::metafactory were statically >>>>> known, so only the fourth through sixth param were dynamically bound >>>>> to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker >>>>> -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a >>>>> slew of filterArguments, rebinds, casting MHs etc). >>>>> >>>>> With condy, the third parameter is now an Object (in reality either a >>>>> Class or a MethodType), thus not statically known. This means the >>>>> MethodType sent to checkGenericInvoker will have to add a cast for >>>>> this param too, thus in makePairwiseConvertByEditor we see an >>>>> additional rebind, some additional time spent spinning classes etc. >>>>> Effectively increasing the cost of first lambda initialization by a small >>>>> amount (a couple of ms). >>>>> >>>>> Here came the realization that much of the static overhead of the >>>>> lambda bootstrapping could be avoided altogether since we can >>>>> determine and cast arguments statically for the special-but-common >>>>> case of LambdaMetafactory::metafactory. By using exact type >>>>> information, and even bootstrapMethod.invokeExact, no dynamic >>>>> runtime checking is needed, so the time spent in >>>>> makePairwiseConvertByEditor is avoided entirely. >>>>> >>>>> This might be a hack, but a hack that removes a large chunk of the >>>>> code executed (~75% less bytecode) for the initial lambda bootstrap. >>>>> Startup tests exercising lambdas show a 10-15ms improvement - the >>>>> static overhead of using lambdas is now just a few milliseconds in total. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ >>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 >>>>> >>>>> The patch includes a test for an experimental new metafactory method >>>>> that exists only in the amber condy-folding branch. I can easily break it >>>>> out and push that directly to amber once this patch syncs up there, but >>>>> have tested that keeping it in here does no harm. >>>>> >>>>> Thanks! >>>>> >>>>> /Claes >>> > From claes.redestad at oracle.com Tue Feb 20 15:46:47 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 20 Feb 2018 16:46:47 +0100 Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: <4ECFA09C-9A7C-4B04-B899-2BE59FF0B258@oracle.com> References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> <7396edb7-4f7d-b0c0-8a09-84bd42a8413a@oracle.com> <8d8a924f-482e-be93-8e3f-f541a63d14b0@oracle.com> <4ECFA09C-9A7C-4B04-B899-2BE59FF0B258@oracle.com> Message-ID: <994b12f1-c05b-e46a-7235-2ef3079e921f@oracle.com> Would injecting this before the LMF::metafactory do? Or should this be an @implNote? diff -r d8e1eab41853 src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java --- a/src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java Tue Feb 20 14:40:53 2018 +0100 +++ b/src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java Tue Feb 20 16:50:27 2018 +0100 @@ -242,6 +242,12 @@ ???? private static final Class[] EMPTY_CLASS_ARRAY = new Class[0]; ???? private static final MethodType[] EMPTY_MT_ARRAY = new MethodType[0]; +??? // LambdaMetafactory bootstrap methods are startup sensitive, and may be +??? // special cased in java.lang.invokeBootstrapMethodInvoker to ensure +??? // methods are invoked with exact type information to avoid generating +??? // code for runtime checks. Take care any changes or additions here are +??? // reflected there as appropriate. + ???? /** ????? * Facilitates the creation of simple "function objects" that implement one ????? * or more interfaces by delegation to a provided {@link MethodHandle}, /Claes On 2018-02-20 16:05, Brian Goetz wrote: > Add a comment to LMF to remember to update the hack if additional sigs are added. > > Sent from my iPad > >> On Feb 20, 2018, at 4:27 AM, Claes Redestad wrote: >> >> You also pointed out that if the params or return types doesn't match, we'd get a CCE sooner or later, making the return and argument checks superfluous. This all simplifies into this, then: >> >> http://cr.openjdk.java.net/~redestad/8198418/jdk.02/ >> >> Thanks! >> >> /Claes >> >> >>> On 2018-02-20 13:20, Vladimir Ivanov wrote: >>> No need in MT.equals. Pointer comparison should work as well: MethodType instances are interned and all exact type checks on MethodHandles are implemented using == on their MTs. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>>> On 2/20/18 3:07 PM, Claes Redestad wrote: >>>> Hi R?mi, >>>> >>>> sure, MethodType.equals will do a fast == check, but then checks the param types. It sure looks cleaner, though: >>>> >>>> http://cr.openjdk.java.net/~redestad/8198418/jdk.01/ >>>> >>>> Thanks! >>>> >>>> /Claes >>>> >>>> >>>>> On 2018-02-20 12:38, Remi Forax wrote: >>>>> Hi Claes, >>>>> instead of checking each parameter of the bsmType(), why not allocating the corresponding MethodType and storing it in a static final, so you can check if the MethodType are equals using equals (as far as i remember MethodType.equals is a == in the OpenJDK implementation). >>>>> >>>>> in term of name why not isLambdaMetafactoryIndyBootstrapMethod and isLambdaMetafactoryCondyBoostrapMethod instead of isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? >>>>> >>>>> and you can remove in the signature of isLambdaMetafactoryCallSite() and replace Class by Class. >>>>> >>>>> cheers, >>>>> R?mi >>>>> >>>>> ----- Mail original ----- >>>>>> De: "Claes Redestad" >>>>>> ?: "core-libs-dev" >>>>>> Envoy?: Mardi 20 F?vrier 2018 11:51:15 >>>>>> Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker >>>>>> Hi, >>>>>> >>>>>> a small regression to lambda bootstrapping came in with the recent >>>>>> condy merge, and it took me a while to figure out why. >>>>>> >>>>>> Before condy, the first three parameters of calls from the BSM invoker >>>>>> to the six parameter LambdaMetafactory::metafactory were statically >>>>>> known, so only the fourth through sixth param were dynamically bound >>>>>> to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker >>>>>> -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a >>>>>> slew of filterArguments, rebinds, casting MHs etc). >>>>>> >>>>>> With condy, the third parameter is now an Object (in reality either a >>>>>> Class or a MethodType), thus not statically known. This means the >>>>>> MethodType sent to checkGenericInvoker will have to add a cast for >>>>>> this param too, thus in makePairwiseConvertByEditor we see an >>>>>> additional rebind, some additional time spent spinning classes etc. >>>>>> Effectively increasing the cost of first lambda initialization by a small >>>>>> amount (a couple of ms). >>>>>> >>>>>> Here came the realization that much of the static overhead of the >>>>>> lambda bootstrapping could be avoided altogether since we can >>>>>> determine and cast arguments statically for the special-but-common >>>>>> case of LambdaMetafactory::metafactory. By using exact type >>>>>> information, and even bootstrapMethod.invokeExact, no dynamic >>>>>> runtime checking is needed, so the time spent in >>>>>> makePairwiseConvertByEditor is avoided entirely. >>>>>> >>>>>> This might be a hack, but a hack that removes a large chunk of the >>>>>> code executed (~75% less bytecode) for the initial lambda bootstrap. >>>>>> Startup tests exercising lambdas show a 10-15ms improvement - the >>>>>> static overhead of using lambdas is now just a few milliseconds in total. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ >>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 >>>>>> >>>>>> The patch includes a test for an experimental new metafactory method >>>>>> that exists only in the amber condy-folding branch. I can easily break it >>>>>> out and push that directly to amber once this patch syncs up there, but >>>>>> have tested that keeping it in here does no harm. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> /Claes From brian.goetz at oracle.com Tue Feb 20 16:06:38 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 20 Feb 2018 08:06:38 -0800 Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: <994b12f1-c05b-e46a-7235-2ef3079e921f@oracle.com> References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> <7396edb7-4f7d-b0c0-8a09-84bd42a8413a@oracle.com> <8d8a924f-482e-be93-8e3f-f541a63d14b0@oracle.com> <4ECFA09C-9A7C-4B04-B899-2BE59FF0B258@oracle.com> <994b12f1-c05b-e46a-7235-2ef3079e921f@oracle.com> Message-ID: <5006B5D0-E0AF-4DC0-A5C8-E87103E4E300@oracle.com> That?s great. Anyone maintaining this file should see it. > On Feb 20, 2018, at 7:46 AM, Claes Redestad wrote: > > Would injecting this before the LMF::metafactory do? Or should this be an @implNote? > > diff -r d8e1eab41853 src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java > --- a/src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java Tue Feb 20 14:40:53 2018 +0100 > +++ b/src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java Tue Feb 20 16:50:27 2018 +0100 > @@ -242,6 +242,12 @@ > private static final Class[] EMPTY_CLASS_ARRAY = new Class[0]; > private static final MethodType[] EMPTY_MT_ARRAY = new MethodType[0]; > > + // LambdaMetafactory bootstrap methods are startup sensitive, and may be > + // special cased in java.lang.invokeBootstrapMethodInvoker to ensure > + // methods are invoked with exact type information to avoid generating > + // code for runtime checks. Take care any changes or additions here are > + // reflected there as appropriate. > + > /** > * Facilitates the creation of simple "function objects" that implement one > * or more interfaces by delegation to a provided {@link MethodHandle}, > > /Claes > > On 2018-02-20 16:05, Brian Goetz wrote: >> Add a comment to LMF to remember to update the hack if additional sigs are added. >> >> Sent from my iPad >> >>> On Feb 20, 2018, at 4:27 AM, Claes Redestad wrote: >>> >>> You also pointed out that if the params or return types doesn't match, we'd get a CCE sooner or later, making the return and argument checks superfluous. This all simplifies into this, then: >>> >>> http://cr.openjdk.java.net/~redestad/8198418/jdk.02/ >>> >>> Thanks! >>> >>> /Claes >>> >>> >>>> On 2018-02-20 13:20, Vladimir Ivanov wrote: >>>> No need in MT.equals. Pointer comparison should work as well: MethodType instances are interned and all exact type checks on MethodHandles are implemented using == on their MTs. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>>> On 2/20/18 3:07 PM, Claes Redestad wrote: >>>>> Hi R?mi, >>>>> >>>>> sure, MethodType.equals will do a fast == check, but then checks the param types. It sure looks cleaner, though: >>>>> >>>>> http://cr.openjdk.java.net/~redestad/8198418/jdk.01/ >>>>> >>>>> Thanks! >>>>> >>>>> /Claes >>>>> >>>>> >>>>>> On 2018-02-20 12:38, Remi Forax wrote: >>>>>> Hi Claes, >>>>>> instead of checking each parameter of the bsmType(), why not allocating the corresponding MethodType and storing it in a static final, so you can check if the MethodType are equals using equals (as far as i remember MethodType.equals is a == in the OpenJDK implementation). >>>>>> >>>>>> in term of name why not isLambdaMetafactoryIndyBootstrapMethod and isLambdaMetafactoryCondyBoostrapMethod instead of isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? >>>>>> >>>>>> and you can remove in the signature of isLambdaMetafactoryCallSite() and replace Class by Class. >>>>>> >>>>>> cheers, >>>>>> R?mi >>>>>> >>>>>> ----- Mail original ----- >>>>>>> De: "Claes Redestad" >>>>>>> ?: "core-libs-dev" >>>>>>> Envoy?: Mardi 20 F?vrier 2018 11:51:15 >>>>>>> Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker >>>>>>> Hi, >>>>>>> >>>>>>> a small regression to lambda bootstrapping came in with the recent >>>>>>> condy merge, and it took me a while to figure out why. >>>>>>> >>>>>>> Before condy, the first three parameters of calls from the BSM invoker >>>>>>> to the six parameter LambdaMetafactory::metafactory were statically >>>>>>> known, so only the fourth through sixth param were dynamically bound >>>>>>> to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker >>>>>>> -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a >>>>>>> slew of filterArguments, rebinds, casting MHs etc). >>>>>>> >>>>>>> With condy, the third parameter is now an Object (in reality either a >>>>>>> Class or a MethodType), thus not statically known. This means the >>>>>>> MethodType sent to checkGenericInvoker will have to add a cast for >>>>>>> this param too, thus in makePairwiseConvertByEditor we see an >>>>>>> additional rebind, some additional time spent spinning classes etc. >>>>>>> Effectively increasing the cost of first lambda initialization by a small >>>>>>> amount (a couple of ms). >>>>>>> >>>>>>> Here came the realization that much of the static overhead of the >>>>>>> lambda bootstrapping could be avoided altogether since we can >>>>>>> determine and cast arguments statically for the special-but-common >>>>>>> case of LambdaMetafactory::metafactory. By using exact type >>>>>>> information, and even bootstrapMethod.invokeExact, no dynamic >>>>>>> runtime checking is needed, so the time spent in >>>>>>> makePairwiseConvertByEditor is avoided entirely. >>>>>>> >>>>>>> This might be a hack, but a hack that removes a large chunk of the >>>>>>> code executed (~75% less bytecode) for the initial lambda bootstrap. >>>>>>> Startup tests exercising lambdas show a 10-15ms improvement - the >>>>>>> static overhead of using lambdas is now just a few milliseconds in total. >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ >>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 >>>>>>> >>>>>>> The patch includes a test for an experimental new metafactory method >>>>>>> that exists only in the amber condy-folding branch. I can easily break it >>>>>>> out and push that directly to amber once this patch syncs up there, but >>>>>>> have tested that keeping it in here does no harm. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> /Claes > From claes.redestad at oracle.com Tue Feb 20 16:46:28 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 20 Feb 2018 17:46:28 +0100 Subject: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker In-Reply-To: <5006B5D0-E0AF-4DC0-A5C8-E87103E4E300@oracle.com> References: <03411d0e-5aa5-c317-712c-df2887b46849@oracle.com> <321568494.775093.1519126733301.JavaMail.zimbra@u-pem.fr> <7396edb7-4f7d-b0c0-8a09-84bd42a8413a@oracle.com> <8d8a924f-482e-be93-8e3f-f541a63d14b0@oracle.com> <4ECFA09C-9A7C-4B04-B899-2BE59FF0B258@oracle.com> <994b12f1-c05b-e46a-7235-2ef3079e921f@oracle.com> <5006B5D0-E0AF-4DC0-A5C8-E87103E4E300@oracle.com> Message-ID: <74fa7c38-82c6-c7ca-07af-6cd7b9612cfc@oracle.com> Thanks! Pushed. /Claes On 2018-02-20 17:06, Brian Goetz wrote: > That?s great. Anyone maintaining this file should see it. > > >> On Feb 20, 2018, at 7:46 AM, Claes Redestad wrote: >> >> Would injecting this before the LMF::metafactory do? Or should this be an @implNote? >> >> diff -r d8e1eab41853 src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java >> --- a/src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java Tue Feb 20 14:40:53 2018 +0100 >> +++ b/src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java Tue Feb 20 16:50:27 2018 +0100 >> @@ -242,6 +242,12 @@ >> private static final Class[] EMPTY_CLASS_ARRAY = new Class[0]; >> private static final MethodType[] EMPTY_MT_ARRAY = new MethodType[0]; >> >> + // LambdaMetafactory bootstrap methods are startup sensitive, and may be >> + // special cased in java.lang.invokeBootstrapMethodInvoker to ensure >> + // methods are invoked with exact type information to avoid generating >> + // code for runtime checks. Take care any changes or additions here are >> + // reflected there as appropriate. >> + >> /** >> * Facilitates the creation of simple "function objects" that implement one >> * or more interfaces by delegation to a provided {@link MethodHandle}, >> >> /Claes >> >> On 2018-02-20 16:05, Brian Goetz wrote: >>> Add a comment to LMF to remember to update the hack if additional sigs are added. >>> >>> Sent from my iPad >>> >>>> On Feb 20, 2018, at 4:27 AM, Claes Redestad wrote: >>>> >>>> You also pointed out that if the params or return types doesn't match, we'd get a CCE sooner or later, making the return and argument checks superfluous. This all simplifies into this, then: >>>> >>>> http://cr.openjdk.java.net/~redestad/8198418/jdk.02/ >>>> >>>> Thanks! >>>> >>>> /Claes >>>> >>>> >>>>> On 2018-02-20 13:20, Vladimir Ivanov wrote: >>>>> No need in MT.equals. Pointer comparison should work as well: MethodType instances are interned and all exact type checks on MethodHandles are implemented using == on their MTs. >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>>> On 2/20/18 3:07 PM, Claes Redestad wrote: >>>>>> Hi R?mi, >>>>>> >>>>>> sure, MethodType.equals will do a fast == check, but then checks the param types. It sure looks cleaner, though: >>>>>> >>>>>> http://cr.openjdk.java.net/~redestad/8198418/jdk.01/ >>>>>> >>>>>> Thanks! >>>>>> >>>>>> /Claes >>>>>> >>>>>> >>>>>>> On 2018-02-20 12:38, Remi Forax wrote: >>>>>>> Hi Claes, >>>>>>> instead of checking each parameter of the bsmType(), why not allocating the corresponding MethodType and storing it in a static final, so you can check if the MethodType are equals using equals (as far as i remember MethodType.equals is a == in the OpenJDK implementation). >>>>>>> >>>>>>> in term of name why not isLambdaMetafactoryIndyBootstrapMethod and isLambdaMetafactoryCondyBoostrapMethod instead of isLambdaMetafactoryCallSite and isLambdaMetafactoryFunction ? >>>>>>> >>>>>>> and you can remove in the signature of isLambdaMetafactoryCallSite() and replace Class by Class. >>>>>>> >>>>>>> cheers, >>>>>>> R?mi >>>>>>> >>>>>>> ----- Mail original ----- >>>>>>>> De: "Claes Redestad" >>>>>>>> ?: "core-libs-dev" >>>>>>>> Envoy?: Mardi 20 F?vrier 2018 11:51:15 >>>>>>>> Objet: [11] RFR: 8198418: Invoke LambdaMetafactory::metafactory exactly from the BootstrapMethodInvoker >>>>>>>> Hi, >>>>>>>> >>>>>>>> a small regression to lambda bootstrapping came in with the recent >>>>>>>> condy merge, and it took me a while to figure out why. >>>>>>>> >>>>>>>> Before condy, the first three parameters of calls from the BSM invoker >>>>>>>> to the six parameter LambdaMetafactory::metafactory were statically >>>>>>>> known, so only the fourth through sixth param were dynamically bound >>>>>>>> to enforce runtime type checks (MH.invoke -> MH.checkGenericInvoker >>>>>>>> -> MH.asType(MT) -> MHI.makePairwiseConvertByEditor -> generates a >>>>>>>> slew of filterArguments, rebinds, casting MHs etc). >>>>>>>> >>>>>>>> With condy, the third parameter is now an Object (in reality either a >>>>>>>> Class or a MethodType), thus not statically known. This means the >>>>>>>> MethodType sent to checkGenericInvoker will have to add a cast for >>>>>>>> this param too, thus in makePairwiseConvertByEditor we see an >>>>>>>> additional rebind, some additional time spent spinning classes etc. >>>>>>>> Effectively increasing the cost of first lambda initialization by a small >>>>>>>> amount (a couple of ms). >>>>>>>> >>>>>>>> Here came the realization that much of the static overhead of the >>>>>>>> lambda bootstrapping could be avoided altogether since we can >>>>>>>> determine and cast arguments statically for the special-but-common >>>>>>>> case of LambdaMetafactory::metafactory. By using exact type >>>>>>>> information, and even bootstrapMethod.invokeExact, no dynamic >>>>>>>> runtime checking is needed, so the time spent in >>>>>>>> makePairwiseConvertByEditor is avoided entirely. >>>>>>>> >>>>>>>> This might be a hack, but a hack that removes a large chunk of the >>>>>>>> code executed (~75% less bytecode) for the initial lambda bootstrap. >>>>>>>> Startup tests exercising lambdas show a 10-15ms improvement - the >>>>>>>> static overhead of using lambdas is now just a few milliseconds in total. >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~redestad/8198418/jdk.00/ >>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8198418 >>>>>>>> >>>>>>>> The patch includes a test for an experimental new metafactory method >>>>>>>> that exists only in the amber condy-folding branch. I can easily break it >>>>>>>> out and push that directly to amber once this patch syncs up there, but >>>>>>>> have tested that keeping it in here does no harm. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> /Claes From mandy.chung at oracle.com Tue Feb 20 17:45:11 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 20 Feb 2018 09:45:11 -0800 Subject: [11] RFR JDK-8198441: Replace native Runtime::runFinalization0 method with shared secrets Message-ID: <9f48c420-f2c5-935f-a4e2-bbcc1721894b@oracle.com> Webrev: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198441/webrev.00 This is a small cleanup that replaces the native Runtime::runFinalization0 method with shared secrets to invoke Finalizer::runFinalization in java. Mandy From mandy.chung at oracle.com Tue Feb 20 18:21:07 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 20 Feb 2018 10:21:07 -0800 Subject: 8196830: publicLookup().findVirtual should not return method handle to AccessibleObject.setAccessible In-Reply-To: <2ecbe3ae-6b8c-33fb-d18d-8ab6f258107f@oracle.com> References: <2ecbe3ae-6b8c-33fb-d18d-8ab6f258107f@oracle.com> Message-ID: <9bcbb564-d5bf-b352-46af-14ec998975a0@oracle.com> This patch looks good.? It's unfortunate that setAccessible was not final to begin with.? I agree that this fix is a good compromise with a simple fix and low incompatibility.?? Is there a CSR to review? Mandy On 2/19/18 8:57 AM, Alan Bateman wrote: > > AccessibleObject's setAccessible(boolean) is currently not caller > sensitive but the overrides in Method/Field/Constructor are. This > awkwardness stems from its constructor being protected and the method > not being final. It is thus possible to extend the class outside of > the java.lang.reflect package and override this method (at least one > popular library does this). Ideally the constructor should have been > package private and/or the method be final but it's not possible to > change this after 20 years. > > The consequence of the method in the base class not being caller > sensitive is that it's possible to use a minimally trusted lookup to > get a method handle to the method. Paul, Mandy and I chatted about > this one recently. We prototyped changes to the MH implementation to > special case this method and treat it "as if" it is caller sensitive. > This maximizes compatibility but has the downside that it makes it > harder to audit and somewhat fragile. In the end, we concluded it > would be simpler to add the @CS annotation to this method so that it > is treated consistently. The downside of this is that custom > AccessibleObject implementations need to override setAccessible if > they want to be invoked using method handles obtained from a > minimally trusted lookup. > > The proposed changes are simple. The removal of "checkMemberAccess" > from canBeCalledVirtual is just a clean-up because this method is no > longer needs special casing (it was degraded for Java SE 10 as > envisaged in JEP 176). It's not the goal here to improve the > performance of canBeCalledVirtual but there may be opportunities to > look at that with a separate issue: > http://cr.openjdk.java.net/~alanb/8196830/webrev/ > > -Alan From martinrb at google.com Tue Feb 20 18:34:42 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 20 Feb 2018 10:34:42 -0800 Subject: [11] RFR JDK-8198441: Replace native Runtime::runFinalization0 method with shared secrets In-Reply-To: <9f48c420-f2c5-935f-a4e2-bbcc1721894b@oracle.com> References: <9f48c420-f2c5-935f-a4e2-bbcc1721894b@oracle.com> Message-ID: Looks good! On Tue, Feb 20, 2018 at 9:45 AM, mandy chung wrote: > Webrev: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198441/webrev.00 > > This is a small cleanup that replaces the native Runtime::runFinalization0 > method with shared secrets to invoke Finalizer::runFinalization in java. > > Mandy > > From paul.sandoz at oracle.com Tue Feb 20 19:31:11 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 20 Feb 2018 11:31:11 -0800 Subject: 8196830: publicLookup().findVirtual should not return method handle to AccessibleObject.setAccessible In-Reply-To: <9bcbb564-d5bf-b352-46af-14ec998975a0@oracle.com> References: <2ecbe3ae-6b8c-33fb-d18d-8ab6f258107f@oracle.com> <9bcbb564-d5bf-b352-46af-14ec998975a0@oracle.com> Message-ID: Looks good, especially the test. Paul. > On Feb 20, 2018, at 10:21 AM, mandy chung wrote: > > This patch looks good. It's unfortunate that setAccessible was not final > to begin with. I agree that this fix is a good compromise with a simple > fix and low incompatibility. Is there a CSR to review? > > Mandy > > On 2/19/18 8:57 AM, Alan Bateman wrote: >> > AccessibleObject's setAccessible(boolean) is currently not caller > > sensitive but the overrides in Method/Field/Constructor are. This > awkwardness stems from its constructor being protected and the method > not being final. It is thus possible to extend the class outside of > the java.lang.reflect package and override this method (at least one > popular library does this). Ideally the constructor should have been > package private and/or the method be final but it's not possible to > change this after 20 years. > > The consequence of the method in the base class not being caller > sensitive is that it's possible to use a minimally trusted lookup to > get a method handle to the method. Paul, Mandy and I chatted about > this one recently. We prototyped changes to the MH implementation to > special case this method and treat it "as if" it is caller sensitive. > This maximizes compatibility but has the downside that it makes it > harder to audit and somewhat fragile. In the end, we concluded it > would be simpler to add the @CS annotation to this method so that it > is treated consistently. The downside of this is that custom > AccessibleObject implementations need to override setAccessible if > they want to be invoked using method handles obtained from a > minimally trusted lookup. > > The proposed changes are simple. The removal of "checkMemberAccess" > from canBeCalledVirtual is just a clean-up because this method is no > longer needs special casing (it was degraded for Java SE 10 as > envisaged in JEP 176). It's not the goal here to improve the > performance of canBeCalledVirtual but there may be opportunities to > look at that with a separate issue: > http://cr.openjdk.java.net/~alanb/8196830/webrev/ > > -Alan From kevinb at google.com Tue Feb 20 19:33:18 2018 From: kevinb at google.com (Kevin Bourrillion) Date: Tue, 20 Feb 2018 11:33:18 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: Just to add another dimension to this data: most of the usages of our repeat method (~75%) are in test code. These tests usually just want any old test string of a certain length. Repeating a single character is the obvious way to get that. Among production code usages (~25%), there are a few roughly equal use cases: ascii indentation/alignment, redaction, and Martin's expected case of "drawing" with ASCII symbols, and "other". On Thu, Feb 15, 2018 at 12:52 PM, Louis Wasserman wrote: > I don't think there's a case for demand to merit having a > repeat(CharSequence, int) at all. > > I did an analysis of usages of Guava's Strings.repeat on Google's > codebase. Users might be rolling their own implementations, too, but this > should be a very good proxy for demand. > > StringRepeat_SingleConstantChar = 4.475 K // strings with .length() == 1 > StringRepeat_SingleConstantCodePoint = 28 // strings with > .codePointCount(...) == 1 > StringRepeat_MultiCodePointConstant = 1.156 K // constant strings neither > of the above > StringRepeat_CharSequenceToString = 2 // > Strings.repeat(CharSequence.toString(), n) > StringRepeat_NoneOfTheAbove = 248 > > Notably, it seems like basically nobody needs to repeat a CharSequence -- > definitely not enough demand to merit the awkwardness of e.g. > Rope.repeat(n) inheriting a repeat returning a String. > > Based on this data, I'd recommend providing one and only one method of this > type: String.repeat(int). There's no real advantage to a static > repeat(char, int) method when the overwhelming majority of these are > constants: e.g. compare SomeUtilClass.repeat('*', n) versus "*".repeat(n). > Character.toString(c).repeat(n) isn't a bad workaround if you don't have a > constant char. There also isn't much demand for dealing with the code > point case specially; the String.repeat(int) method seems like it'd handle > that just fine. > > On Thu, Feb 15, 2018 at 11:44 AM Jim Laskey > wrote: > > > > > > > > On Feb 15, 2018, at 3:36 PM, Ivan Gerasimov > > > wrote: > > > > > > Hello! > > > > > > The link with the webrev returned 404, but I could find it at this > > location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ > > > > > > A few minor comments: > > > > > > 1) > > > > > > This check: > > > > > > 2992 final long limit = (long)count * 2L; > > > 2993 if ((long)Integer.MAX_VALUE < limit) { > > > > > > can be possibly simplified as > > > if (count > Integer.MAX_VALUE - count) { > > > > Good. > > > > > > > > 2) > > > Should String repeat(final int codepoint, final int count) be optimized > > for codepoints that can be represented with a single char? > > > > > > E.g. like this: > > > > > > public static String repeat(final int codepoint, final int count) { > > > return Character.isBmpCodePoint(codepoint)) > > > ? repeat((char) codepoint, count) > > > : (new String(Character.toChars(codepoint))).repeat(count); > > > } > > > > Yes, avoid array allocation. > > > > > > > > 3) > > > Using long arithmetic can possibly be avoided in the common path of > > repeat(final int count): > > > > > > E.g. like this: > > > > > > if (count < 0) { > > > throw new IllegalArgumentException("count is negative, " + > > count); > > > } else if (count == 1) { > > > return this; > > > } else if (count == 0) { > > > return ""; > > > } > > > final int len = value.length; > > > if (Integer.MAX_VALUE / count < len) { > > > throw new IllegalArgumentException( > > > "Resulting string exceeds maximum string length: " > + > > ((long)len * (long)count)); > > > } > > > final int limit = count * len; > > > > Good. > > > > Thank you. > > > > > > > > With kind regards, > > > Ivan > > > > > > On 2/15/18 9:20 AM, Jim Laskey wrote: > > >> This is a pre-CSR code review [1] for String repeat methods > > (Enhancement). > > >> > > >> The proposal is to introduce four new methods; > > >> > > >> 1. public String repeat(final int count) > > >> 2. public static String repeat(final char ch, final int count) > > >> 3. public static String repeat(final int codepoint, final int count) > > >> 4. public static String repeat(final CharSequence seq, final int > count) > > >> > > >> For the sake of transparency, only 1 is necessary, 2-4 are convenience > > methods. > > >> In the case of 2, ?*?.repeat(10) performs as well as > String.repeat(?*?, > > 10). > > >> 3 and 4 convert to String before calling 1. > > >> > > >> Performance runs with jmh (results as comment in [2]) show that these > > >> methods are significantly faster that StringBuilder equivalents. > > >> - fewer memory allocations > > >> - fewer char to byte array conversions > > >> - faster pyramid replication vs O(N) copying > > >> > > >> I left StringBuilder out of scope. It falls under the category of > > >> Appendables#append with repeat. A much bigger project. > > >> > > >> All comments welcome. Especially around the need for convenience > > >> methods, the JavaDoc content and expanding the tests. > > >> > > >> ? Jim > > >> > > >> [1] webrev: > > http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 > > >> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 > > >> > > >> > > > > > > -- > > > With kind regards, > > > Ivan Gerasimov > > > > > > > > -- Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb at google.com From lowasser at google.com Tue Feb 20 19:46:00 2018 From: lowasser at google.com (Louis Wasserman) Date: Tue, 20 Feb 2018 19:46:00 +0000 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: I'm with Brian: adding a separate API to make it easier to get from a codepoint to a String seems independently merited, and makes the single repeat API work well for that case. A very quick regex-powered search comes up with 183 hits in Google for (new String|String.copyValueOf|String.valueOf)(Character.toChars(..)). I do, however, recommend a separate thread for discussing that API :) On Tue, Feb 20, 2018 at 11:33 AM Kevin Bourrillion wrote: > Just to add another dimension to this data: most of the usages of our > repeat method (~75%) are in test code. These tests usually just want any > old test string of a certain length. Repeating a single character is the > obvious way to get that. > > Among production code usages (~25%), there are a few roughly equal use > cases: ascii indentation/alignment, redaction, and Martin's expected case > of "drawing" with ASCII symbols, and "other". > > > > On Thu, Feb 15, 2018 at 12:52 PM, Louis Wasserman > wrote: > >> I don't think there's a case for demand to merit having a >> repeat(CharSequence, int) at all. >> >> I did an analysis of usages of Guava's Strings.repeat on Google's >> codebase. Users might be rolling their own implementations, too, but this >> should be a very good proxy for demand. >> >> StringRepeat_SingleConstantChar = 4.475 K // strings with .length() == 1 >> StringRepeat_SingleConstantCodePoint = 28 // strings with >> .codePointCount(...) == 1 >> StringRepeat_MultiCodePointConstant = 1.156 K // constant strings neither >> of the above >> StringRepeat_CharSequenceToString = 2 // >> Strings.repeat(CharSequence.toString(), n) >> StringRepeat_NoneOfTheAbove = 248 >> >> Notably, it seems like basically nobody needs to repeat a CharSequence -- >> definitely not enough demand to merit the awkwardness of e.g. >> Rope.repeat(n) inheriting a repeat returning a String. >> >> Based on this data, I'd recommend providing one and only one method of >> this >> type: String.repeat(int). There's no real advantage to a static >> repeat(char, int) method when the overwhelming majority of these are >> constants: e.g. compare SomeUtilClass.repeat('*', n) versus "*".repeat(n). >> Character.toString(c).repeat(n) isn't a bad workaround if you don't have a >> constant char. There also isn't much demand for dealing with the code >> point case specially; the String.repeat(int) method seems like it'd handle >> that just fine. >> >> On Thu, Feb 15, 2018 at 11:44 AM Jim Laskey >> wrote: >> >> > >> > >> > > On Feb 15, 2018, at 3:36 PM, Ivan Gerasimov < >> ivan.gerasimov at oracle.com> >> > wrote: >> > > >> > > Hello! >> > > >> > > The link with the webrev returned 404, but I could find it at this >> > location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ >> > > >> > > A few minor comments: >> > > >> > > 1) >> > > >> > > This check: >> > > >> > > 2992 final long limit = (long)count * 2L; >> > > 2993 if ((long)Integer.MAX_VALUE < limit) { >> > > >> > > can be possibly simplified as >> > > if (count > Integer.MAX_VALUE - count) { >> > >> > Good. >> > >> > > >> > > 2) >> > > Should String repeat(final int codepoint, final int count) be >> optimized >> > for codepoints that can be represented with a single char? >> > > >> > > E.g. like this: >> > > >> > > public static String repeat(final int codepoint, final int count) { >> > > return Character.isBmpCodePoint(codepoint)) >> > > ? repeat((char) codepoint, count) >> > > : (new String(Character.toChars(codepoint))).repeat(count); >> > > } >> > >> > Yes, avoid array allocation. >> > >> > > >> > > 3) >> > > Using long arithmetic can possibly be avoided in the common path of >> > repeat(final int count): >> > > >> > > E.g. like this: >> > > >> > > if (count < 0) { >> > > throw new IllegalArgumentException("count is negative, " + >> > count); >> > > } else if (count == 1) { >> > > return this; >> > > } else if (count == 0) { >> > > return ""; >> > > } >> > > final int len = value.length; >> > > if (Integer.MAX_VALUE / count < len) { >> > > throw new IllegalArgumentException( >> > > "Resulting string exceeds maximum string length: >> " + >> > ((long)len * (long)count)); >> > > } >> > > final int limit = count * len; >> > >> > Good. >> > >> > Thank you. >> > >> > > >> > > With kind regards, >> > > Ivan >> > > >> > > On 2/15/18 9:20 AM, Jim Laskey wrote: >> > >> This is a pre-CSR code review [1] for String repeat methods >> > (Enhancement). >> > >> >> > >> The proposal is to introduce four new methods; >> > >> >> > >> 1. public String repeat(final int count) >> > >> 2. public static String repeat(final char ch, final int count) >> > >> 3. public static String repeat(final int codepoint, final int count) >> > >> 4. public static String repeat(final CharSequence seq, final int >> count) >> > >> >> > >> For the sake of transparency, only 1 is necessary, 2-4 are >> convenience >> > methods. >> > >> In the case of 2, ?*?.repeat(10) performs as well as >> String.repeat(?*?, >> > 10). >> > >> 3 and 4 convert to String before calling 1. >> > >> >> > >> Performance runs with jmh (results as comment in [2]) show that these >> > >> methods are significantly faster that StringBuilder equivalents. >> > >> - fewer memory allocations >> > >> - fewer char to byte array conversions >> > >> - faster pyramid replication vs O(N) copying >> > >> >> > >> I left StringBuilder out of scope. It falls under the category of >> > >> Appendables#append with repeat. A much bigger project. >> > >> >> > >> All comments welcome. Especially around the need for convenience >> > >> methods, the JavaDoc content and expanding the tests. >> > >> >> > >> ? Jim >> > >> >> > >> [1] webrev: >> > http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >> > >> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >> > >> >> > >> >> > > >> > > -- >> > > With kind regards, >> > > Ivan Gerasimov >> > > >> > >> > >> > > > > -- > Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb at google.com > From mandy.chung at oracle.com Tue Feb 20 19:57:04 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 20 Feb 2018 11:57:04 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> Message-ID: Hi David, I reworked the change in Shutdown class and uses jdk.internal.misc.VM to maintain the shutdown state, either in progress or shutdown (i.e. all shutdown hooks have been started). What do you think this revised version: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.03/ On 2/15/18 9:14 PM, David Holmes wrote: > > All other updates seem okay. I did have one further thought - in > Runtime does this change: > > ????? public void runFinalization() { > !???????? SharedSecrets.getJavaLangRefAccess().runFinalization(); > ????? } > > affect the classloading/initialization order at all? Runtime::runFinalization is not called by the system code.? So it won't be invoked during startup and hence won't change the classloading order during startup. Mandy From paul.sandoz at oracle.com Tue Feb 20 22:47:51 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 20 Feb 2018 14:47:51 -0800 Subject: RFR Passing 'null' value to lookup param of ConstantBootstraps.invoke does not throw NullPointerException Message-ID: <3F76BEF9-FACC-4D93-99F4-B9EBFC1DF235@oracle.com> Hi, Correction to the specification of ConstantBootstraps.invoke [1], the lookup parameter is not used: diff -r b75c9e2e3b1f src/java.base/share/classes/java/lang/invoke/ConstantBootstraps.java --- a/src/java.base/share/classes/java/lang/invoke/ConstantBootstraps.java Tue Feb 20 21:47:54 2018 +0100 +++ b/src/java.base/share/classes/java/lang/invoke/ConstantBootstraps.java Tue Feb 20 14:38:56 2018 -0800 @@ -209,8 +209,7 @@ * Returns the result of invoking a method handle with the provided * arguments. * - * @param lookup the lookup context describing the class performing the - * operation (normally stacked by the JVM) + * @param lookup unused * @param name unused * @param type the type of the value to be returned, which must be * compatible with the return type of the method handle CSR is here: https://bugs.openjdk.java.net/browse/JDK-8198469 Paul. [1] /** * Returns the result of invoking a method handle with the provided * arguments. * * @param lookup unused * @param name unused * @param type the type of the value to be returned, which must be * compatible with the return type of the method handle * @param handle the method handle to be invoked * @param args the arguments to pass to the method handle, as if with * {@link MethodHandle#invokeWithArguments}. Each argument may be * {@code null}. * @return the result of invoking the method handle * @throws WrongMethodTypeException if the handle's return type cannot be * adjusted to the desired type * @throws ClassCastException if an argument cannot be converted by * reference casting * @throws Throwable anything thrown by the method handle invocation */ public static Object invoke(MethodHandles.Lookup lookup, String name, Class type, MethodHandle handle, Object... args) throws Throwable { From mandy.chung at oracle.com Wed Feb 21 05:42:30 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 20 Feb 2018 21:42:30 -0800 Subject: RFR Passing 'null' value to lookup param of ConstantBootstraps.invoke does not throw NullPointerException In-Reply-To: <3F76BEF9-FACC-4D93-99F4-B9EBFC1DF235@oracle.com> References: <3F76BEF9-FACC-4D93-99F4-B9EBFC1DF235@oracle.com> Message-ID: <68d19433-795c-76f2-d0c9-3acef7a273f3@oracle.com> On 2/20/18 2:47 PM, Paul Sandoz wrote: > Hi, > > Correction to the specification of ConstantBootstraps.invoke [1], the lookup parameter is not used: > > diff -r b75c9e2e3b1f src/java.base/share/classes/java/lang/invoke/ConstantBootstraps.java > --- a/src/java.base/share/classes/java/lang/invoke/ConstantBootstraps.java Tue Feb 20 21:47:54 2018 +0100 > +++ b/src/java.base/share/classes/java/lang/invoke/ConstantBootstraps.java Tue Feb 20 14:38:56 2018 -0800 > @@ -209,8 +209,7 @@ > * Returns the result of invoking a method handle with the provided > * arguments. > * > - * @param lookup the lookup context describing the class performing the > - * operation (normally stacked by the JVM) > + * @param lookup unused > * @param name unused > * @param type the type of the value to be returned, which must be > * compatible with the return type of the method handle +1 > > CSR is here: > > https://bugs.openjdk.java.net/browse/JDK-8198469 Reviewed. Mandy From martinrb at google.com Wed Feb 21 06:08:18 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 20 Feb 2018 22:08:18 -0800 Subject: RFR: Here are some URLClassPath patches Message-ID: At Google I spend a lot of time staring unproductively at classloader code, and it's always hard for me to resist the urge to clean it up. So here are some patches that should be relatively uncontroversial, and may prepare for more radical changes later. 8198480: Improve ClassLoaders static init block http://cr.openjdk.java.net/~martin/webrevs/jdk/ClassLoaders-static/ https://bugs.openjdk.java.net/browse/JDK-8198480 8198481: Coding style cleanups for src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java http://cr.openjdk.java.net/~martin/webrevs/jdk/loader-style/ https://bugs.openjdk.java.net/browse/JDK-8198481 8198482: The URLClassPath field "urls" should be renamed to "unopenedUrls" http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-rename-urls/ https://bugs.openjdk.java.net/browse/JDK-8198482 8198484: URLClassPath should use an ArrayDeque instead of a Stack http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ https://bugs.openjdk.java.net/browse/JDK-8198484 8198485: Simplify a URLClassPath constructor http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-simplify-constructor/ https://bugs.openjdk.java.net/browse/JDK-8198485 From david.holmes at oracle.com Wed Feb 21 06:27:48 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 21 Feb 2018 16:27:48 +1000 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> Message-ID: Hi Mandy, On 21/02/2018 5:57 AM, mandy chung wrote: > Hi David, > > I reworked the change in Shutdown class and uses jdk.internal.misc.VM to > maintain the shutdown state, either in progress or shutdown (i.e. all > shutdown hooks have been started). > > What do you think this revised version: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.03/ It confuses me a bit. On the one hand I wasn't sure why we needed to bring the VM class into this, then on the other hand it perhaps made sense to have VM track shutdown states as well as initialization states. But I'm not sure about the two-phase "shutdown" state - I think I'd probably prefer a "shutdown initiated" state and a "shutdown complete" state. Then no need to duplicate the constant values for IN_PROGRESS and SHUTDOWN (which now need to be kept in sync with changes to the VM class). It would also simplify the shutdown() logic. Also shutdown(level) should be using initLevel(value) so that it fits in with the existing awaitInitLevel() logic and locking strategy! Someone may want to wait for shutdown in the future. That also deals with the locking issue in the Shutdown class because you don't need to use "synchronized(lock)" because runHooks is only called within "synchronized(Shutdown.class)". [To be fair the existing locking strategy seems confused to me as well - or at least it confuses me.] I also now realize that the changes I suggested for the Runtime.exit docs was incorrect. The existing docs state that we only halt if hooks have already been run - which corresponds to the old and new code. I, for some reason that escapes me, claimed we only cared if the hooks had been started, not completed - but that is inconsistent with old spec and the implementation. So apologies but what started out as a seemingly simple code removal, has become a lot more complicated, and confusing to me. Thanks, David ----- > On 2/15/18 9:14 PM, David Holmes wrote: >> >> All other updates seem okay. I did have one further thought - in >> Runtime does this change: >> >> ????? public void runFinalization() { >> !???????? SharedSecrets.getJavaLangRefAccess().runFinalization(); >> ????? } >> >> affect the classloading/initialization order at all? > > Runtime::runFinalization is not called by the system code.? So it won't > be invoked during startup and hence won't change the classloading order > during startup. > > Mandy From xueming.shen at oracle.com Wed Feb 21 06:31:54 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 20 Feb 2018 22:31:54 -0800 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. Message-ID: <5A8D125A.5030205@oracle.com> This draft JEP contains a proposal to use UTF-8 as the default charset for the JVM, so that APIs that depend on the default charset behave consistently cross all platforms. For more details, please see: https://bugs.openjdk.java.net/browse/JDK-8187041 Sherman From Alan.Bateman at oracle.com Wed Feb 21 07:28:19 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Feb 2018 07:28:19 +0000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: <2bd7f946-6e42-d294-c526-0a9e019f126e@oracle.com> On 21/02/2018 06:08, Martin Buchholz wrote: > : > > 8198481: Coding style cleanups for > src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > http://cr.openjdk.java.net/~martin/webrevs/jdk/loader-style/ > > https://bugs.openjdk.java.net/browse/JDK-8198481 > This changes comments on private fields from // to /**.? Are you generating javadoc on private members of internal classes or why are you changing all these fields? -Alan From Alan.Bateman at oracle.com Wed Feb 21 07:32:30 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Feb 2018 07:32:30 +0000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: On 21/02/2018 06:08, Martin Buchholz wrote: > : > > 8198482: The URLClassPath field "urls" should be renamed to "unopenedUrls" > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-rename-urls/ > > https://bugs.openjdk.java.net/browse/JDK-8198482 > I think this looks okay. -Alan From Alan.Bateman at oracle.com Wed Feb 21 07:59:57 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Feb 2018 07:59:57 +0000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: <4a2b9a02-73a5-9615-8065-365aa78ee82e@oracle.com> On 21/02/2018 06:08, Martin Buchholz wrote: > : > > 8198480: Improve ClassLoaders static init block > http://cr.openjdk.java.net/~martin/webrevs/jdk/ClassLoaders-static/ > > https://bugs.openjdk.java.net/browse/JDK-8198480 > This initializer has changed a few times and I agree it's a bit awkward to understand when there is a class path or not. So I think you're rejigging is okay, as is the rewording of the comment. The blank line at L45 was intentional, no need to remove that. -Alan From uschindler at apache.org Wed Feb 21 08:53:54 2018 From: uschindler at apache.org (Uwe Schindler) Date: Wed, 21 Feb 2018 09:53:54 +0100 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: <5A8D125A.5030205@oracle.com> References: <5A8D125A.5030205@oracle.com> Message-ID: <001301d3aaf1$827285a0$875790e0$@apache.org> Hi, > This draft JEP contains a proposal to use UTF-8 as the default charset > for the JVM, so that > APIs that depend on the default charset behave consistently cross all > platforms. > > For more details, please see: > https://bugs.openjdk.java.net/browse/JDK-8187041 Thanks for finally adding a JEP like this. Thanks also to Robert Muir for always insisting in fixing this problem! I have a few comments: The JEP should NOT cause that new APIs, which may convert between characters and bytes to no longer explicitly accept a charset. One example is the proposed ByteBuffer methods taking String. The default ones would work with UTF-8, but it should still be possible to an API user to always add a charset whenever there is a conversion between bytes and chars. This is especially important as the user may still change the default and breaking your app. Because the rule is still: Only YOU, the developer, know the charset of your stuff when you load a JAR resource file or pass a String to the network in a ByteBuffer! The biggest offenders on this is also given as an example: FileReader and FileWriter. Although both classes subclass InputStreamReader/OutputStreamWriter and just pass the right delegate to the superclass in the ctor, both classes are missing the possibility to specify a charset. Because of this, the use of FileReader and FileWriter is completely forbidden in many Apache projects (Apache Lucene, Solr, Elasticsearch, Apache TIKA,...). So I'd suggest to also fix the API here and just add the missing ctors. The Java 7+ methods in java.nio.file.Files already ignore the default charset and always use UTF-8. How to proceed with those? Should they be changed to behave to the new mechanisms? I'd suggest to not do this, as its part of the spec (to use UTF-8) and should not rely on external forces, but I wanted to bring this in. Changing the default would help many users, if they are actually using newer JDKs. For those with older versions (and compiling their code against older versions), you still have to avoid the default charsets. In addition, as you still can change the "default charset", any library developer reading resources from its own JAR file or passing Strings to network protocols cannot rely on the fact, that the default charset is really UTF-8! (a user may have changed it to something else). Because of this, Apache libraries will forbid usage of all methods using default charsets (and locales + timezones). The "changeable default" does not affect application developers (because they have in most cases control about the environment), but library developers should always be explicit! For this to work, I also want to do some "advertisement": All library projects should use the Forbidden-Apis Maven/Gradle/Ant plugin to scan their bytecode for offenders using default charsets, default locales or relying on default timezones. See the blog post about it [1] and the project page [2]. The tool is also useful to replace "jdeps" in projects with Java versions before 8, as it can scan your code for access to internal JDK APIs, too. See the documentation [3] and github wiki pages for useful examples. It may also be a good idea to mention it in the JEP as a "workaround" or "further reading". Finally: Because one can still change the default, I'd propose to deprecate all methods that use a default charset (unrelated to actually changing the default). Only if you do this, it would make tools like "forbiddenapis" irrelevant for library developers. And finally, finally: I'd also propose to change the default Locale to Locale.ROOT (same issues). The String.toLowerCase() in Turkish locales still break thousands of apps! But that's a different JEP - but I would strongly support it! Uwe [1] http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html [2] https://github.com/policeman-tools/forbidden-apis [3] https://jenkins.thetaphi.de/job/Forbidden-APIs/javadoc/ ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ From forax at univ-mlv.fr Wed Feb 21 11:04:19 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 21 Feb 2018 11:04:19 +0000 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: <001301d3aaf1$827285a0$875790e0$@apache.org> References: <5A8D125A.5030205@oracle.com> <001301d3aaf1$827285a0$875790e0$@apache.org> Message-ID: I agree with Uwe, we should deprecate all methods/constructors that relies on the default charset. And we should do that before changing to use UTF-8 by default. Remi On February 21, 2018 8:53:54 AM UTC, Uwe Schindler wrote: >Hi, > >> This draft JEP contains a proposal to use UTF-8 as the default >charset >> for the JVM, so that >> APIs that depend on the default charset behave consistently cross all >> platforms. >> >> For more details, please see: >> https://bugs.openjdk.java.net/browse/JDK-8187041 > >Thanks for finally adding a JEP like this. Thanks also to Robert Muir >for always insisting in fixing this problem! I have a few comments: > >The JEP should NOT cause that new APIs, which may convert between >characters and bytes to no longer explicitly accept a charset. One >example is the proposed ByteBuffer methods taking String. The default >ones would work with UTF-8, but it should still be possible to an API >user to always add a charset whenever there is a conversion between >bytes and chars. This is especially important as the user may still >change the default and breaking your app. Because the rule is still: >Only YOU, the developer, know the charset of your stuff when you load a >JAR resource file or pass a String to the network in a ByteBuffer! > >The biggest offenders on this is also given as an example: FileReader >and FileWriter. Although both classes subclass >InputStreamReader/OutputStreamWriter and just pass the right delegate >to the superclass in the ctor, both classes are missing the possibility >to specify a charset. Because of this, the use of FileReader and >FileWriter is completely forbidden in many Apache projects (Apache >Lucene, Solr, Elasticsearch, Apache TIKA,...). So I'd suggest to also >fix the API here and just add the missing ctors. > >The Java 7+ methods in java.nio.file.Files already ignore the default >charset and always use UTF-8. How to proceed with those? Should they be >changed to behave to the new mechanisms? I'd suggest to not do this, as >its part of the spec (to use UTF-8) and should not rely on external >forces, but I wanted to bring this in. > >Changing the default would help many users, if they are actually using >newer JDKs. For those with older versions (and compiling their code >against older versions), you still have to avoid the default charsets. >In addition, as you still can change the "default charset", any library >developer reading resources from its own JAR file or passing Strings to >network protocols cannot rely on the fact, that the default charset is >really UTF-8! (a user may have changed it to something else). Because >of this, Apache libraries will forbid usage of all methods using >default charsets (and locales + timezones). The "changeable default" >does not affect application developers (because they have in most cases >control about the environment), but library developers should always be >explicit! > >For this to work, I also want to do some "advertisement": All library >projects should use the Forbidden-Apis Maven/Gradle/Ant plugin to scan >their bytecode for offenders using default charsets, default locales or >relying on default timezones. See the blog post about it [1] and the >project page [2]. The tool is also useful to replace "jdeps" in >projects with Java versions before 8, as it can scan your code for >access to internal JDK APIs, too. See the documentation [3] and github >wiki pages for useful examples. It may also be a good idea to mention >it in the JEP as a "workaround" or "further reading". > >Finally: Because one can still change the default, I'd propose to >deprecate all methods that use a default charset (unrelated to actually >changing the default). Only if you do this, it would make tools like >"forbiddenapis" irrelevant for library developers. > >And finally, finally: I'd also propose to change the default Locale to >Locale.ROOT (same issues). The String.toLowerCase() in Turkish locales >still break thousands of apps! But that's a different JEP - but I would >strongly support it! > >Uwe > >[1] >http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html >[2] https://github.com/policeman-tools/forbidden-apis >[3] https://jenkins.thetaphi.de/job/Forbidden-APIs/javadoc/ > >----- >Uwe Schindler >uschindler at apache.org >ASF Member, Apache Lucene PMC / Committer >Bremen, Germany >http://lucene.apache.org/ -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From peter.levart at gmail.com Wed Feb 21 11:39:37 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 21 Feb 2018 12:39:37 +0100 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: <21c00794-ad49-5707-4ba5-8caccd89d074@gmail.com> Hi Martin, I checked this one... On 02/21/2018 07:08 AM, Martin Buchholz wrote: > 8198484: URLClassPath should use an ArrayDeque instead of a Stack > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ > https://bugs.openjdk.java.net/browse/JDK-8198484 I admit I had to study the Stack API first as I don't regularly use it nowadays ;-). You seem to be using the following replacements consistently: Stack vs. ArrayDequeue push(e) vs. addFirst(e) add(0, e) vs. addLast(e) isEmpty() / pop() vs. pollFirst() ...but then in push(URL[]), you use: push(e) vs. push(e) Deque.addFirst(e) is equivalent to Deque.push(e), but it would be nice to keep using the same method consistently in one class. Regards, Peter From david.lloyd at redhat.com Wed Feb 21 13:19:13 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 21 Feb 2018 07:19:13 -0600 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: <5A8D125A.5030205@oracle.com> References: <5A8D125A.5030205@oracle.com> Message-ID: I agree with Uwe and Remi; if the default is still changeable, the problem doesn't go away, it simply becomes slightly more insidious. On Wed, Feb 21, 2018 at 12:31 AM, Xueming Shen wrote: > This draft JEP contains a proposal to use UTF-8 as the default charset for > the JVM, so that > APIs that depend on the default charset behave consistently cross all > platforms. > > For more details, please see: > https://bugs.openjdk.java.net/browse/JDK-8187041 > > Sherman -- - DML From Alan.Bateman at oracle.com Wed Feb 21 13:37:26 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Feb 2018 13:37:26 +0000 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: References: <5A8D125A.5030205@oracle.com> Message-ID: <22bca8a4-5548-7af9-9770-ad7b749191c9@oracle.com> On 21/02/2018 13:19, David Lloyd wrote: > I agree with Uwe and Remi; if the default is still changeable, the > problem doesn't go away, it simply becomes slightly more insidious. > The proposal is to eventually get to the point that the default charset cannot be changed. It will take several releases to get there due to the potential compatibility impact. This draft JEP is the first step to switch to UTF-8 by default. A first step has to allow it be changed in order to keep some existing code/deployments working. Sorry this isn't clear in the JEP yet, there several clarifications to this JEP that haven't been included yet (on my list, I didn't realize it would be discussed here this week). -Alan From scolebourne at joda.org Wed Feb 21 13:41:29 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 21 Feb 2018 13:41:29 +0000 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: <22bca8a4-5548-7af9-9770-ad7b749191c9@oracle.com> References: <5A8D125A.5030205@oracle.com> <22bca8a4-5548-7af9-9770-ad7b749191c9@oracle.com> Message-ID: On 21 February 2018 at 13:37, Alan Bateman wrote: > The proposal is to eventually get to the point that the default charset > cannot be changed. It will take several releases to get there due to the > potential compatibility impact. This seems like a reasonable strategy to solve the problem. I also agree that all locations where a default charset is used need to have a method alongside that takes a CharSet, eg. FileWriter. Stephen From Alan.Bateman at oracle.com Wed Feb 21 13:55:12 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Feb 2018 13:55:12 +0000 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: References: <5A8D125A.5030205@oracle.com> <22bca8a4-5548-7af9-9770-ad7b749191c9@oracle.com> Message-ID: On 21/02/2018 13:41, Stephen Colebourne wrote: > On 21 February 2018 at 13:37, Alan Bateman wrote: >> The proposal is to eventually get to the point that the default charset >> cannot be changed. It will take several releases to get there due to the >> potential compatibility impact. > This seems like a reasonable strategy to solve the problem. > > I also agree that all locations where a default charset is used need > to have a method alongside that takes a CharSet, eg. FileWriter. > Good progress was made via JDK-8183743 [1] in Java SE 10 to add constructors and methods that take a Charset and eliminate the historical inconsistencies. The issue of legacy FileReader/FileWriter is linked from that JIRA issue. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8183743 From Alan.Bateman at oracle.com Wed Feb 21 14:07:41 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Feb 2018 14:07:41 +0000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: On 21/02/2018 06:08, Martin Buchholz wrote: > : > > 8198485: Simplify a URLClassPath constructor > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-simplify-constructor/ > > https://bugs.openjdk.java.net/browse/JDK-8198485 This one looks okay to me. -Alan From Alan.Bateman at oracle.com Wed Feb 21 14:10:21 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Feb 2018 14:10:21 +0000 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: <001301d3aaf1$827285a0$875790e0$@apache.org> References: <5A8D125A.5030205@oracle.com> <001301d3aaf1$827285a0$875790e0$@apache.org> Message-ID: On 21/02/2018 08:53, Uwe Schindler wrote: > : > The Java 7+ methods in java.nio.file.Files already ignore the default charset and always use UTF-8. How to proceed with those? Should they be changed to behave to the new mechanisms? I'd suggest to not do this, as its part of the spec (to use UTF-8) and should not rely on external forces, but I wanted to bring this in. > There is no proposal to change these methods. -Alan From rcmuir at gmail.com Wed Feb 21 14:26:09 2018 From: rcmuir at gmail.com (Robert Muir) Date: Wed, 21 Feb 2018 09:26:09 -0500 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: References: <5A8D125A.5030205@oracle.com> <22bca8a4-5548-7af9-9770-ad7b749191c9@oracle.com> Message-ID: On Wed, Feb 21, 2018 at 8:55 AM, Alan Bateman wrote: > Good progress was made via JDK-8183743 [1] in Java SE 10 to add constructors > and methods that take a Charset and eliminate the historical > inconsistencies. The issue of legacy FileReader/FileWriter is linked from > that JIRA issue. > Can we ensure we have CharsetDecoder/Encoder params too? There is unfortunately a huge difference between InputStreamReader(x, StandardCharsets.UTF_8) and InputStreamReader(x, StandardCharsets.UTF_8.newDecoder()). And the silent replacement of the "easier" one is probably not what most apps want. From uschindler at apache.org Wed Feb 21 14:58:47 2018 From: uschindler at apache.org (Uwe Schindler) Date: Wed, 21 Feb 2018 15:58:47 +0100 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: References: <5A8D125A.5030205@oracle.com> <22bca8a4-5548-7af9-9770-ad7b749191c9@oracle.com> Message-ID: <007001d3ab24$7bb63f30$7322bd90$@apache.org> Hi, Thanks Alan for the link to this issue about FileReader/Writer! Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Alan Bateman > Sent: Wednesday, February 21, 2018 2:55 PM > To: Stephen Colebourne ; core-libs- > dev at openjdk.java.net > Subject: Re: Draft JEP: To use UTF-8 as the default charset for the Java virtual > machine. > > On 21/02/2018 13:41, Stephen Colebourne wrote: > > On 21 February 2018 at 13:37, Alan Bateman > wrote: > >> The proposal is to eventually get to the point that the default charset > >> cannot be changed. It will take several releases to get there due to the > >> potential compatibility impact. > > This seems like a reasonable strategy to solve the problem. > > > > I also agree that all locations where a default charset is used need > > to have a method alongside that takes a CharSet, eg. FileWriter. > > > Good progress was made via JDK-8183743 [1] in Java SE 10 to add > constructors and methods that take a Charset and eliminate the > historical inconsistencies. The issue of legacy FileReader/FileWriter is > linked from that JIRA issue. > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8183743 From christoph.langer at sap.com Wed Feb 21 15:16:17 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 21 Feb 2018 15:16:17 +0000 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> Message-ID: <618b0785b9dd4e1ca47df9b448f30637@sap.com> Hi Bernd, would your test still fail with system property "-Djdk.xml.overrideDefaultParser=true"? I debugged a similar issue (if not the same) today and found that with change http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jaxp/rev/08a44c164993 the default behavior for Xalan changed a bit. Before that change, the services mechanism was enabled to load the SAX and DOM implementations for the use of Xalan. E.g. when TransformerFactory.newInstance() was called, the services mechanism was used and in the TransformerFactory (at least if it was the default JDK implementation), the property to use the services mechanism was set as well - wich was used to retrieve SAX and DOM parser implementations. And what hit me even more was that SAX2DOM had its private field "DocumentBuilderFactory _factory" automatically initialized by a call to DocumentBuilderFactory.newInstance() which in turn would resolve any custom DBF implementation if it were on classpath. Now, with the new implementation, the _factory in SAX2DOM will be initialized with the "overrideDefaultParser " setting coming from the Transformer Factory. So, by setting -Djdk.xml.overrideDefaultParser=true, you will restore the old behavior. @Joe: Is there some documentation for this change in default behavior that came with JDK8u161? I assume it is for higher security and cannot be reverted (e.g. by setting the feature default for Djdk.xml.overrideDefaultParser to true). Best regards Christoph > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Bernd > Sent: Dienstag, 13. Februar 2018 22:55 > To: OpenJDK Dev list > Subject: Re: Oracle Java 8u161 regression in XML Schema Factory > > Hello, > > > 2018-01-25 17:41 GMT+01:00 Se?n Coffey : > > > > > Classes nearer to those below were touched via JDK-8186080: Transform > XML > > interfaces > > http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 > > http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 > > > > This may be connected with some tools trying to redefine the classes > > perhaps. Needs more investigating. Perhaps the XMLSchemaLoader > changes are > > a factor ? > > > > I have ben able to extract a minimal reproducer. It is not related to > XMLUnit, only to powermock. If it instruments com.sun but not javax.xml > (and other combinations) then it fails. > > For details see the readme in this maven project: > > https://github.com/ecki/reproduce-schemaex > > I also found a way to make it work with both versions, so its no longer an > issue for me, but there is definitely some changes (which might also be > triggered in AppServers or OSGi containers with partially reconfigured > implementations. Not sure if you want to investigate deeper). > > Gruss > Bernd > > > Here is the stacktrace anyway: > >> > >> com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: Schema > >> factory class > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > does > >> not > >> extend from SchemaDVFactory. > >> at > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. > >> getInstance(SchemaDVFactory.java:75) > >> at > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. > >> getInstance(SchemaDVFactory.java:57) > >> at > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >> reset(XMLSchemaLoader.java:1024) > >> at > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >> loadGrammar(XMLSchemaLoader.java:556) > >> at > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >> loadGrammar(XMLSchemaLoader.java:535) > >> at > >> com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchema > >> Factory.newSchema(XMLSchemaFactory.java:254) > >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. > >> java:638) > >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. > >> java:654) > >> at > >> com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchema > >> Validator(BuilderTestHelper.java:57) > >> at > >> com.seeburger.api.test.helpers.BuilderTestHelper.validateAnd > >> Compare(BuilderTestHelper.java:73) > >> at > >> com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuild > >> erTest.java:196) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce > >> ssorImpl.java:62) > >> at > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe > >> thodAccessorImpl.java:43) > >> at java.lang.reflect.Method.invoke(Method.java:498) > >> at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod > >> (PowerMockJUnit44RunnerDelegateImpl.java:310) > >> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie. > >> java:89) > >> at > >> org.junit.internal.runners.MethodRoadie.runBeforesThenTestTh > >> enAfters(MethodRoadie.java:97) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(P > >> owerMockJUnit44RunnerDelegateImpl.java:294) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R > >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestIn > >> Super(PowerMockJUnit47RunnerDelegateImpl.java:127) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R > >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(P > >> owerMockJUnit47RunnerDelegateImpl.java:82) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThe > >> nTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) > >> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie > >> .java:87) > >> at > org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> unnerDelegateImpl.invokeTestMethod(PowerMockJUni > >> t44RunnerDelegateImpl.java:207) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> > unnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.ja > va:146) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> > unnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) > >> at > >> org.junit.internal.runners.ClassRoadie.runUnprotected(ClassR > >> oadie.java:34) > >> at > >> org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> > unnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) > >> at > >> org.powermock.modules.junit4.common.internal.impl.JUnit4Test > >> SuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) > >> at > >> org.powermock.modules.junit4.common.internal.impl.AbstractCo > >> > mmonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) > >> at > >> > org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner. > java:59) > >> at > >> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. > >> run(JUnit4TestReference.java:86) > >> at > >> org.eclipse.jdt.internal.junit.runner.TestExecution.run( > >> TestExecution.java:38) > >> at > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe > >> sts(RemoteTestRunner.java:539) > >> at > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe > >> sts(RemoteTestRunner.java:761) > >> at > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( > >> RemoteTestRunner.java:461) > >> at > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( > >> RemoteTestRunner.java:207) > >> > >> on the classpath jaxb-impl-2.2.5.jar but the specific packages are only > >> loaded from rt.jar and redefined. I asume the later is done by > Powermock. > >> > >> Line 611: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > >> C:\Program > >> Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >> Line 616: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > from > >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >> Line 617: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > from > >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >> Line 618: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > >> __JVM_DefineClass__] > >> Line 619: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > from > >> __JVM_DefineClass__] > >> Line 620: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > from > >> __JVM_DefineClass__] > >> > >> Is that something you are concerned? > >> > >> Gruss > >> Bernd > >> > > From Roger.Riggs at Oracle.com Wed Feb 21 15:48:30 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 21 Feb 2018 10:48:30 -0500 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: <22abe55b-b7e3-63d0-5442-5efb5920720a@Oracle.com> Hi Martin, Generally, I would leave code alone that does not have a good reason for changing. That goes for commenting conventions (// vs /*) too. On the style changes in URLClassPath-ArrayDeque, about declarations like: ArrayList path = new ArrayList<>(); It had been a recommended style to use the abstract type (List) on the left hand side and only use the concrete type where necessary. But with the addition of 'var' local type inference, perhaps that would an option here. (Not trying to start a style war). $.02, Roger On 2/21/2018 1:08 AM, Martin Buchholz wrote: > At Google I spend a lot of time staring unproductively at classloader code, > and it's always hard for me to resist the urge to clean it up. So here are > some patches that should be relatively uncontroversial, and may prepare for > more radical changes later. > > > 8198484: URLClassPath should use an ArrayDeque instead of a Stack > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ > https://bugs.openjdk.java.net/browse/JDK-8198484 > From volker.simonis at gmail.com Wed Feb 21 16:11:03 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 21 Feb 2018 17:11:03 +0100 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: <5A8D125A.5030205@oracle.com> References: <5A8D125A.5030205@oracle.com> Message-ID: Hi Sherman, the tricky part is really "sun.jnu.encoding" and how the VM interacts with the underlying OS. You may remember that we had an interesting discussion about this topic some time ago [1]. As far as I understand, the JEP doesn't plan to change the handling of "sun.jnu.encoding". So does this mean that the VM will still correctly start and work on system with a platform encoding different from UTF-8? I.e. will starting the VM from a path which contains characters in that special platform encoding or classpath/argument settings with characters in that special character encoding still work? If the answer will be yes (which I expect) maybe you could explain that a little more detailed in the JEP. I.e. the JEP should say that it changes the default encoding for the Java API classes and not the default encoding for natively accessing system resources. Maybe the JEP should also mention that "sun.jnu.encoding" is more or less a "read-only" property which can not be reliable set by the user on the command line (it's a chicken-egg problem: for the parsing of the command line we need the correct encoding, so it can not be reliably set on the command line). For these reasons the Summary "Use UTF-8 as the Java virtual machine's default charset ..." is a little misleading. Maybe you could rephrase to something like "Use UTF-8 as the default charset so that Java APIs that depend on the default charset behave consistently across all platforms." Thank you and best regards, Volker [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/thread.html#37516 On Wed, Feb 21, 2018 at 7:31 AM, Xueming Shen wrote: > This draft JEP contains a proposal to use UTF-8 as the default charset for > the JVM, so that > APIs that depend on the default charset behave consistently cross all > platforms. > > For more details, please see: > https://bugs.openjdk.java.net/browse/JDK-8187041 > > Sherman From xueming.shen at oracle.com Wed Feb 21 16:42:40 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 21 Feb 2018 08:42:40 -0800 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: References: <5A8D125A.5030205@oracle.com> Message-ID: <5A8DA180.5090004@oracle.com> Hi Volker, Yes, the handing of sun.jnu.encoding will not be changed. It will remain as a read-only/informative system property. sun.jnu.encoding is really an implementation details (as well as file.encoding, though in this JEP file.encoding might be used to provide a mechanism to fallback to the current/old/existing behavior, so might become a public/official interface/ system property). From API perspective the Charset.defaultCharset() is the only place to obtain the "Java virtual machine's default charset". As Alan said in previous comment, clarifications will be included in the final version based on feedback/suggestion -Sherman On 2/21/18, 8:11 AM, Volker Simonis wrote: > Hi Sherman, > > the tricky part is really "sun.jnu.encoding" and how the VM interacts > with the underlying OS. You may remember that we had an interesting > discussion about this topic some time ago [1]. > > As far as I understand, the JEP doesn't plan to change the handling of > "sun.jnu.encoding". So does this mean that the VM will still correctly > start and work on system with a platform encoding different from > UTF-8? I.e. will starting the VM from a path which contains characters > in that special platform encoding or classpath/argument settings with > characters in that special character encoding still work? If the > answer will be yes (which I expect) maybe you could explain that a > little more detailed in the JEP. I.e. the JEP should say that it > changes the default encoding for the Java API classes and not the > default encoding for natively accessing system resources. > > Maybe the JEP should also mention that "sun.jnu.encoding" is more or > less a "read-only" property which can not be reliable set by the user > on the command line (it's a chicken-egg problem: for the parsing of > the command line we need the correct encoding, so it can not be > reliably set on the command line). > > For these reasons the Summary "Use UTF-8 as the Java virtual machine's > default charset ..." is a little misleading. Maybe you could rephrase > to something like "Use UTF-8 as the default charset so that Java APIs > that depend on the default charset behave consistently across all > platforms." > > Thank you and best regards, > Volker > > [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/thread.html#37516 > > On Wed, Feb 21, 2018 at 7:31 AM, Xueming Shen wrote: >> This draft JEP contains a proposal to use UTF-8 as the default charset for >> the JVM, so that >> APIs that depend on the default charset behave consistently cross all >> platforms. >> >> For more details, please see: >> https://bugs.openjdk.java.net/browse/JDK-8187041 >> >> Sherman From mark.reinhold at oracle.com Wed Feb 21 17:09:21 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 21 Feb 2018 12:09:21 -0500 Subject: RFR: Here are some URLClassPath patches In-Reply-To: <2bd7f946-6e42-d294-c526-0a9e019f126e@oracle.com> References: <2bd7f946-6e42-d294-c526-0a9e019f126e@oracle.com> Message-ID: <20180221120921.74992813@eggemoggin.niobe.net> 2018/2/21 2:28:19 -0500, alan.bateman at oracle.com: > On 21/02/2018 06:08, Martin Buchholz wrote: >> : >> >> 8198481: Coding style cleanups for >> src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java >> http://cr.openjdk.java.net/~martin/webrevs/jdk/loader-style/ >> >> https://bugs.openjdk.java.net/browse/JDK-8198481 > > This changes comments on private fields from // to /**. Are you > generating javadoc on private members of internal classes or why are you > changing all these fields? The use of // rather than /** on private members is intentional, to make it clear to the reader that these are internal elements rather than API. You might not like it, but please don't change it. - Mark From xueming.shen at oracle.com Wed Feb 21 18:16:40 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 21 Feb 2018 10:16:40 -0800 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: References: <5A8D125A.5030205@oracle.com> <22bca8a4-5548-7af9-9770-ad7b749191c9@oracle.com> Message-ID: <5A8DB788.4010509@oracle.com> On 2/21/18, 6:26 AM, Robert Muir wrote: > On Wed, Feb 21, 2018 at 8:55 AM, Alan Bateman wrote: > >> Good progress was made via JDK-8183743 [1] in Java SE 10 to add constructors >> and methods that take a Charset and eliminate the historical >> inconsistencies. The issue of legacy FileReader/FileWriter is linked from >> that JIRA issue. >> > Can we ensure we have CharsetDecoder/Encoder params too? There is > unfortunately a huge difference between InputStreamReader(x, > StandardCharsets.UTF_8) and InputStreamReader(x, > StandardCharsets.UTF_8.newDecoder()). And the silent replacement of > the "easier" one is probably not what most apps want. Hi Robert, Understood a silent replacement might not be the desired behavior in some use scenarios. Anymore details regarding what "most apps want" when there is/are malformed/unmappable? It appears the best the underneath de/encoder can do here is to throw an IOException. Given the caller of the Reader/Writer does not have the access to the bytes of the underlying stream src (reader)/dst(writer), there is in theory impossible to do anything to recover and continue without risking data loss. The assumption here is if you want to have a fine-grained control of the de/ encoding, you might want to work with the Input/OutStream/Channel + CharsetDe/Encoder instead of Reader/Writer. No, I'm not saying we can't do Reader(CharsetDecoder)/Writer(CharsetEncoder), just wanted to know what's the real use scenario and what's the better/ best choice here. -Sherman From ecki at zusammenkunft.net Wed Feb 21 19:01:28 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Wed, 21 Feb 2018 20:01:28 +0100 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: <618b0785b9dd4e1ca47df9b448f30637@sap.com> References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> <618b0785b9dd4e1ca47df9b448f30637@sap.com> Message-ID: <5a8dc206.f8aedf0a.6fae.9f4d@mx.google.com> Hallo Christoph, Yes the Tests fail with -Djdk.xml.overrideDefaultParser=true (and false) as well. Gruss Bernd -- http://bernd.eckenfels.net Von: Langer, Christoph Gesendet: Mittwoch, 21. Februar 2018 16:16 An: Bernd; OpenJDK Dev list; huizhe.wang at oracle.com Betreff: RE: Oracle Java 8u161 regression in XML Schema Factory Hi Bernd, would your test still fail with system property "-Djdk.xml.overrideDefaultParser=true"? I debugged a similar issue (if not the same) today and found that with change http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jaxp/rev/08a44c164993 the default behavior for Xalan changed a bit. Before that change, the services mechanism was enabled to load the SAX and DOM implementations for the use of Xalan. E.g. when TransformerFactory.newInstance() was called, the services mechanism was used and in the TransformerFactory (at least if it was the default JDK implementation), the property to use the services mechanism was set as well - wich was used to retrieve SAX and DOM parser implementations. And what hit me even more was that SAX2DOM had its private field "DocumentBuilderFactory _factory" automatically initialized by a call to DocumentBuilderFactory.newInstance() which in turn would resolve any custom DBF implementation if it were on classpath. Now, with the new implementation, the _factory in SAX2DOM will be initialized with the "overrideDefaultParser " setting coming from the Transformer Factory. So, by setting -Djdk.xml.overrideDefaultParser=true, you will restore the old behavior. @Joe: Is there some documentation for this change in default behavior that came with JDK8u161? I assume it is for higher security and cannot be reverted (e.g. by setting the feature default for Djdk.xml.overrideDefaultParser to true). Best regards Christoph > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Bernd > Sent: Dienstag, 13. Februar 2018 22:55 > To: OpenJDK Dev list > Subject: Re: Oracle Java 8u161 regression in XML Schema Factory > > Hello, > > > 2018-01-25 17:41 GMT+01:00 Se?n Coffey : > > > > > Classes nearer to those below were touched via JDK-8186080: Transform > XML > > interfaces > > http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 > > http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 > > > > This may be connected with some tools trying to redefine the classes > > perhaps. Needs more investigating. Perhaps the XMLSchemaLoader > changes are > > a factor ? > > > > I have ben able to extract a minimal reproducer. It is not related to > XMLUnit, only to powermock. If it instruments com.sun but not javax.xml > (and other combinations) then it fails. > > For details see the readme in this maven project: > > https://github.com/ecki/reproduce-schemaex > > I also found a way to make it work with both versions, so its no longer an > issue for me, but there is definitely some changes (which might also be > triggered in AppServers or OSGi containers with partially reconfigured > implementations. Not sure if you want to investigate deeper). > > Gruss > Bernd > > > Here is the stacktrace anyway: > >> > >> com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: Schema > >> factory class > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > does > >> not > >> extend from SchemaDVFactory. > >> at > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. > >> getInstance(SchemaDVFactory.java:75) > >> at > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. > >> getInstance(SchemaDVFactory.java:57) > >> at > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >> reset(XMLSchemaLoader.java:1024) > >> at > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >> loadGrammar(XMLSchemaLoader.java:556) > >> at > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >> loadGrammar(XMLSchemaLoader.java:535) > >> at > >> com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchema > >> Factory.newSchema(XMLSchemaFactory.java:254) > >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. > >> java:638) > >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. > >> java:654) > >> at > >> com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchema > >> Validator(BuilderTestHelper.java:57) > >> at > >> com.seeburger.api.test.helpers.BuilderTestHelper.validateAnd > >> Compare(BuilderTestHelper.java:73) > >> at > >> com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuild > >> erTest.java:196) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce > >> ssorImpl.java:62) > >> at > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe > >> thodAccessorImpl.java:43) > >> at java.lang.reflect.Method.invoke(Method.java:498) > >> at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod > >> (PowerMockJUnit44RunnerDelegateImpl.java:310) > >> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie. > >> java:89) > >> at > >> org.junit.internal.runners.MethodRoadie.runBeforesThenTestTh > >> enAfters(MethodRoadie.java:97) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(P > >> owerMockJUnit44RunnerDelegateImpl.java:294) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R > >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestIn > >> Super(PowerMockJUnit47RunnerDelegateImpl.java:127) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R > >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(P > >> owerMockJUnit47RunnerDelegateImpl.java:82) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThe > >> nTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) > >> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie > >> .java:87) > >> at > org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> unnerDelegateImpl.invokeTestMethod(PowerMockJUni > >> t44RunnerDelegateImpl.java:207) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> > unnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.ja > va:146) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> > unnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) > >> at > >> org.junit.internal.runners.ClassRoadie.runUnprotected(ClassR > >> oadie.java:34) > >> at > >> org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) > >> at > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >> > unnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) > >> at > >> org.powermock.modules.junit4.common.internal.impl.JUnit4Test > >> SuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) > >> at > >> org.powermock.modules.junit4.common.internal.impl.AbstractCo > >> > mmonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) > >> at > >> > org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner. > java:59) > >> at > >> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. > >> run(JUnit4TestReference.java:86) > >> at > >> org.eclipse.jdt.internal.junit.runner.TestExecution.run( > >> TestExecution.java:38) > >> at > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe > >> sts(RemoteTestRunner.java:539) > >> at > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe > >> sts(RemoteTestRunner.java:761) > >> at > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( > >> RemoteTestRunner.java:461) > >> at > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( > >> RemoteTestRunner.java:207) > >> > >> on the classpath jaxb-impl-2.2.5.jar but the specific packages are only > >> loaded from rt.jar and redefined. I asume the later is done by > Powermock. > >> > >> Line 611: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > >> C:\Program > >> Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >> Line 616: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > from > >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >> Line 617: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > from > >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >> Line 618: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > >> __JVM_DefineClass__] > >> Line 619: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > from > >> __JVM_DefineClass__] > >> Line 620: [Loaded > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > from > >> __JVM_DefineClass__] > >> > >> Is that something you are concerned? > >> > >> Gruss > >> Bernd > >> > > From huizhe.wang at oracle.com Wed Feb 21 19:25:29 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 21 Feb 2018 11:25:29 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> Message-ID: <5A8DC7A9.30104@oracle.com> Hi Stuart, Thanks for the apiNote! Joe re-approved the CSR with the added apiNote. For a test that compares Latin1 vs UTF16 coders, I added tests to the compact string's CompareTo test[1], basically running the original test for String with StringBuilder/Buffer. A build with jdk_core tests passed. Here's the updated webrev: JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ [1] http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/test/jdk/java/lang/String/CompactString/CompareTo.java.sdiff.html Best, Joe On 2/14/18, 5:09 PM, Stuart Marks wrote: > Hi Joe, > > Overall, looks good. > > Are there any tests of AbstractStringBuilder.compareTo() that exercise > comparisons of the Latin1 vs UTF16 coders? > > A couple people have raised the issue of the SB's implementing > Comparable but not overriding equals(). This is unusual but > well-defined. I do think it deserves the "inconsistent with equals" > treatment. Something like the following should be added to both SB's: > > ========== > @apiNote > {@code StringBuilder} implements {@code Comparable} but does not > override {@link Object#equals equals}. Thus, the natural ordering of > {@code StringBuilder} is inconsistent with equals. Care should be > exercised if {@code StringBuilder} objects are used as keys in a > {@code SortedMap} or elements in a {@code SortedSet}. See {@link > Comparable}, {@link java.util.SortedMap SortedMap}, or {@link > java.util.SortedSet SortedSet} for more information. > ========== > > Respectively for StringBuffer. (Adjust markup to taste.) > > Thanks, > > s'marks > > > > > On 2/8/18 4:47 PM, Joe Wang wrote: >> Hi all, >> >> The CSR for the enhancement is now approved. Thanks Joe! >> >> The webrev has been updated accordingly. Please let me know if you >> have any further comment on the implementation. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> Thanks, >> Joe >> >> On 2/2/2018 12:46 PM, Joe Wang wrote: >>> Thanks Jason. Will update that accordingly. >>> >>> Best, >>> Joe >>> >>> On 2/2/2018 11:22 AM, Jason Mehrens wrote: >>>> Joe, >>>> >>>> The identity check in CS.compare makes sense. However, it won't be >>>> null hostile if we call CS.compare(null, null) and that doesn't >>>> seem right. >>>> Usually when writing comparator classes I end up with: >>>> === >>>> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >>>> return 0; >>>> } >>>> === >>>> >>>> Jason >>>> ________________________________________ >>>> From: core-libs-dev on >>>> behalf of Joe Wang >>>> Sent: Friday, February 2, 2018 1:01 PM >>>> To: core-libs-dev >>>> Subject: Re: RFR (JDK11) 8137326: Methods for comparing >>>> CharSequence, StringBuilder, and StringBuffer >>>> >>>> Hi, >>>> >>>> Thanks all for comments and suggestions. I've updated the webrev. >>>> Please >>>> review. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>> >>>> Thanks, >>>> Joe >>>> >>>> On 1/31/2018 9:31 PM, Joe Wang wrote: >>>>> Hi Tagir, >>>>> >>>>> Thanks for the comment. I will consider adding that to the javadoc >>>>> emphasizing that the comparison is performed from 0 to length() - >>>>> 1 of >>>>> the two sequences. >>>>> >>>>> Best, >>>>> Joe >>>>> >>>>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>>>> Hello! >>>>>> >>>>>> An AbstractStringBuilder#compareTo implementation is wrong. You >>>>>> cannot >>>>>> simply compare the whole byte array. Here's the test-case: >>>>>> >>>>>> public class Test { >>>>>> public static void main(String[] args) { >>>>>> StringBuilder sb1 = new StringBuilder("test1"); >>>>>> StringBuilder sb2 = new StringBuilder("test2"); >>>>>> sb1.setLength(4); >>>>>> sb2.setLength(4); >>>>>> System.out.println(sb1.compareTo(sb2)); >>>>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>>>> } >>>>>> } >>>>>> >>>>>> We truncated the stringbuilders making their content equal, so >>>>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo >>>>>> compares >>>>>> the original content (before the truncation) as truncation, of >>>>>> course, >>>>>> does not zero the truncated bytes, neither does it reallocate the >>>>>> array (unless explicitly asked via trimToSize). >>>>>> >>>>>> With best regards, >>>>>> Tagir Valeev. >>>>>> >>>>>> >>>>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>>>> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>>>> StringBuffer. >>>>>>> >>>>>>> The Comparable implementations for StringBuilder/Buffer are similar >>>>>>> to that >>>>>>> of String, allowing comparison operations between two >>>>>>> StringBuilders/Buffers, e.g. >>>>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>>>> method >>>>>>> 'compare' is added instead of implementing the Comparable >>>>>>> interface. >>>>>>> This >>>>>>> 'compare' method may take CharSequence implementations such as >>>>>>> String, >>>>>>> StringBuilder and StringBuffer, making it possible to perform >>>>>>> comparison >>>>>>> among them. The previous example for example is equivalent to >>>>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>>>> >>>>>>> Tests for java.base have been independent from each other. The new >>>>>>> tests are >>>>>>> therefore created to have no dependency on each other or sharing >>>>>>> any >>>>>>> code. >>>>>>> >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>>> >>>>>>> Thanks, >>>>>>> Joe >>> >> From mandy.chung at oracle.com Wed Feb 21 19:34:08 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 21 Feb 2018 11:34:08 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> Message-ID: <27436fb7-e7f8-bc47-924e-aeced7057fe3@oracle.com> Hi David, I think I'm clear on the implementation now (my mistake that I neglected ApplicationShutdownHooks).? Shutdown class keeps the "system shutdown hooks".? ApplicationShutdownHooks is one system hook that starts all application shutdown hooks and waits until they finish. ? java.io.Console and DeleteOnExitHook are two other system shutdown hooks to restore console echo state or support deleteOnExit. The system shutdown hooks are all run in the thread initiating the shutdown that holds the Shutdown.class. On 2/20/18 10:27 PM, David Holmes wrote: > Hi Mandy, > > On 21/02/2018 5:57 AM, mandy chung wrote: >> Hi David, >> >> I reworked the change in Shutdown class and uses jdk.internal.misc.VM >> to maintain the shutdown state, either in progress or shutdown (i.e. >> all shutdown hooks have been started). >> >> What do you think this revised version: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.03/ > > It confuses me a bit. On the one hand I wasn't sure why we needed to > bring the VM class into this, then on the other hand it perhaps made > sense to have VM track shutdown states as well as initialization states. > The latter is my motivation and I think it's a good cleanup to have VM state to indicate it's shutdown. > But I'm not sure about the two-phase "shutdown" state - I think I'd > probably prefer a "shutdown initiated" state and a "shutdown complete" > state. Then no need to duplicate the constant values for IN_PROGRESS > and SHUTDOWN (which now need to be kept in sync with changes to the VM > class). It would also simplify the shutdown() logic. > Shutdown::add method can be used to register a system hook while shutdown is in progress.? Actually currentRunningHook can be used to guard if Shutdown::add should reject to add a new hook.? I updated the webrev and no longer needs the two phases. > Also shutdown(level) should be using initLevel(value) so that it fits > in with the existing awaitInitLevel() logic and locking strategy! > Someone may want to wait for shutdown in the future. That also deals > with the locking issue in the Shutdown class because you don't need to > use "synchronized(lock)" because runHooks is only called within > "synchronized(Shutdown.class)". [To be fair the existing locking > strategy seems confused to me as well - or at least it confuses me.] I agree but I tried not to touch the existing locking strategy as it should be a separate changeset.? I prefer to add VM::shutdown method > > I also now realize that the changes I suggested for the Runtime.exit > docs was incorrect. The existing docs state that we only halt if hooks > have already been run - which corresponds to the old and new code. I, > for some reason that escapes me, claimed we only cared if the hooks > had been started, not completed - but that is inconsistent with old > spec and the implementation. > I confused myself too.? A small change will fix it: 87 *

If this method is invoked after all shutdown hooks have already 88 * been run and the status is nonzero then this method halts the 89 * virtual machine with the given status code. Otherwise, this method Mandy > So apologies but what started out as a seemingly simple code removal, > has become a lot more complicated, and confusing to me. > > Thanks, > David > ----- > >> On 2/15/18 9:14 PM, David Holmes wrote: >>> >>> All other updates seem okay. I did have one further thought - in >>> Runtime does this change: >>> >>> ????? public void runFinalization() { >>> ! SharedSecrets.getJavaLangRefAccess().runFinalization(); >>> ????? } >>> >>> affect the classloading/initialization order at all? >> >> Runtime::runFinalization is not called by the system code.? So it >> won't be invoked during startup and hence won't change the >> classloading order during startup. >> >> Mandy From martinrb at google.com Wed Feb 21 19:53:50 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 21 Feb 2018 11:53:50 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: <22abe55b-b7e3-63d0-5442-5efb5920720a@Oracle.com> References: <22abe55b-b7e3-63d0-5442-5efb5920720a@Oracle.com> Message-ID: On Wed, Feb 21, 2018 at 7:48 AM, Roger Riggs wrote: > > On the style changes in URLClassPath-ArrayDeque, about declarations like: > > ArrayList path = new ArrayList<>(); > > It had been a recommended style to use the abstract type (List) on the > left hand side > and only use the concrete type where necessary. > There's a debate about whether using concrete types in the implementation is clearer. I happen to think so. But in performance critical classes the concrete class is more important to the maintainer (I'm thinking "ArrayDeque" not "Deque" when I maintain this code), and is also likely to help the JVM. From mandy.chung at oracle.com Wed Feb 21 19:58:20 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 21 Feb 2018 11:58:20 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <27436fb7-e7f8-bc47-924e-aeced7057fe3@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> <27436fb7-e7f8-bc47-924e-aeced7057fe3@oracle.com> Message-ID: <3bc62991-7585-8b3e-8d83-7764181f9cdd@oracle.com> Here is the updated webrev: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.04/ I added some comments to clarify the implementation. Mandy On 2/21/18 11:34 AM, mandy chung wrote: > Hi David, > > I think I'm clear on the implementation now (my mistake that I > neglected ApplicationShutdownHooks).? Shutdown class keeps the "system > shutdown hooks".? ApplicationShutdownHooks is one system hook that > starts all application shutdown hooks and waits until they finish. ? > java.io.Console and DeleteOnExitHook are two other system shutdown > hooks to restore console echo state or support deleteOnExit. > > The system shutdown hooks are all run in the thread initiating the > shutdown that holds the Shutdown.class. > > On 2/20/18 10:27 PM, David Holmes wrote: >> Hi Mandy, >> >> On 21/02/2018 5:57 AM, mandy chung wrote: >>> Hi David, >>> >>> I reworked the change in Shutdown class and uses >>> jdk.internal.misc.VM to maintain the shutdown state, either in >>> progress or shutdown (i.e. all shutdown hooks have been started). >>> >>> What do you think this revised version: >>> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.03/ >> >> It confuses me a bit. On the one hand I wasn't sure why we needed to >> bring the VM class into this, then on the other hand it perhaps made >> sense to have VM track shutdown states as well as initialization states. >> > > The latter is my motivation and I think it's a good cleanup to have VM > state to indicate it's shutdown. > >> But I'm not sure about the two-phase "shutdown" state - I think I'd >> probably prefer a "shutdown initiated" state and a "shutdown >> complete" state. Then no need to duplicate the constant values for >> IN_PROGRESS and SHUTDOWN (which now need to be kept in sync with >> changes to the VM class). It would also simplify the shutdown() logic. >> > > Shutdown::add method can be used to register a system hook while > shutdown is in progress.? Actually currentRunningHook can be used to > guard if Shutdown::add should reject to add a new hook.? I updated the > webrev and no longer needs the two phases. > >> Also shutdown(level) should be using initLevel(value) so that it fits >> in with the existing awaitInitLevel() logic and locking strategy! >> Someone may want to wait for shutdown in the future. That also deals >> with the locking issue in the Shutdown class because you don't need >> to use "synchronized(lock)" because runHooks is only called within >> "synchronized(Shutdown.class)". [To be fair the existing locking >> strategy seems confused to me as well - or at least it confuses me.] > > I agree but I tried not to touch the existing locking strategy as it > should be a separate changeset.? I prefer to add VM::shutdown method >> >> I also now realize that the changes I suggested for the Runtime.exit >> docs was incorrect. The existing docs state that we only halt if >> hooks have already been run - which corresponds to the old and new >> code. I, for some reason that escapes me, claimed we only cared if >> the hooks had been started, not completed - but that is inconsistent >> with old spec and the implementation. >> > > I confused myself too.? A small change will fix it: > > 87 *

If this method is invoked after all shutdown hooks have already > 88 * been run and the status is nonzero then this method halts the > 89 * virtual machine with the given status code. Otherwise, this method > > > > Mandy > >> So apologies but what started out as a seemingly simple code removal, >> has become a lot more complicated, and confusing to me. >> >> Thanks, >> David >> ----- >> >>> On 2/15/18 9:14 PM, David Holmes wrote: >>>> >>>> All other updates seem okay. I did have one further thought - in >>>> Runtime does this change: >>>> >>>> ????? public void runFinalization() { >>>> ! SharedSecrets.getJavaLangRefAccess().runFinalization(); >>>> ????? } >>>> >>>> affect the classloading/initialization order at all? >>> >>> Runtime::runFinalization is not called by the system code.? So it >>> won't be invoked during startup and hence won't change the >>> classloading order during startup. >>> >>> Mandy > From xueming.shen at oracle.com Wed Feb 21 19:59:26 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 21 Feb 2018 11:59:26 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A8DC7A9.30104@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> <5A8DC7A9.30104@oracle.com> Message-ID: <5A8DCF9E.50508@oracle.com> Joe, AbstractStringBuilder is a mutable class. You might need to take a local copy and do the explicit boundary check before going into the StringLatin1/UTF16, especially StringUTF16. Some intrinsics, UTF16.getChar(byte[], int) for example, assume the caller performs bounds check (for performance reason). -Sherman On 2/21/18, 11:25 AM, Joe Wang wrote: > Hi Stuart, > > Thanks for the apiNote! Joe re-approved the CSR with the added apiNote. > > For a test that compares Latin1 vs UTF16 coders, I added tests to the > compact string's CompareTo test[1], basically running the original > test for String with StringBuilder/Buffer. A build with jdk_core tests > passed. > > Here's the updated webrev: > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > [1] > http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/test/jdk/java/lang/String/CompactString/CompareTo.java.sdiff.html > > Best, > Joe > > On 2/14/18, 5:09 PM, Stuart Marks wrote: >> Hi Joe, >> >> Overall, looks good. >> >> Are there any tests of AbstractStringBuilder.compareTo() that >> exercise comparisons of the Latin1 vs UTF16 coders? >> >> A couple people have raised the issue of the SB's implementing >> Comparable but not overriding equals(). This is unusual but >> well-defined. I do think it deserves the "inconsistent with equals" >> treatment. Something like the following should be added to both SB's: >> >> ========== >> @apiNote >> {@code StringBuilder} implements {@code Comparable} but does not >> override {@link Object#equals equals}. Thus, the natural ordering of >> {@code StringBuilder} is inconsistent with equals. Care should be >> exercised if {@code StringBuilder} objects are used as keys in a >> {@code SortedMap} or elements in a {@code SortedSet}. See {@link >> Comparable}, {@link java.util.SortedMap SortedMap}, or {@link >> java.util.SortedSet SortedSet} for more information. >> ========== >> >> Respectively for StringBuffer. (Adjust markup to taste.) >> >> Thanks, >> >> s'marks >> >> >> >> >> On 2/8/18 4:47 PM, Joe Wang wrote: >>> Hi all, >>> >>> The CSR for the enhancement is now approved. Thanks Joe! >>> >>> The webrev has been updated accordingly. Please let me know if you >>> have any further comment on the implementation. >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> Thanks, >>> Joe >>> >>> On 2/2/2018 12:46 PM, Joe Wang wrote: >>>> Thanks Jason. Will update that accordingly. >>>> >>>> Best, >>>> Joe >>>> >>>> On 2/2/2018 11:22 AM, Jason Mehrens wrote: >>>>> Joe, >>>>> >>>>> The identity check in CS.compare makes sense. However, it won't >>>>> be null hostile if we call CS.compare(null, null) and that doesn't >>>>> seem right. >>>>> Usually when writing comparator classes I end up with: >>>>> === >>>>> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >>>>> return 0; >>>>> } >>>>> === >>>>> >>>>> Jason >>>>> ________________________________________ >>>>> From: core-libs-dev on >>>>> behalf of Joe Wang >>>>> Sent: Friday, February 2, 2018 1:01 PM >>>>> To: core-libs-dev >>>>> Subject: Re: RFR (JDK11) 8137326: Methods for comparing >>>>> CharSequence, StringBuilder, and StringBuffer >>>>> >>>>> Hi, >>>>> >>>>> Thanks all for comments and suggestions. I've updated the webrev. >>>>> Please >>>>> review. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>> >>>>> Thanks, >>>>> Joe >>>>> >>>>> On 1/31/2018 9:31 PM, Joe Wang wrote: >>>>>> Hi Tagir, >>>>>> >>>>>> Thanks for the comment. I will consider adding that to the javadoc >>>>>> emphasizing that the comparison is performed from 0 to length() - >>>>>> 1 of >>>>>> the two sequences. >>>>>> >>>>>> Best, >>>>>> Joe >>>>>> >>>>>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>>>>> Hello! >>>>>>> >>>>>>> An AbstractStringBuilder#compareTo implementation is wrong. You >>>>>>> cannot >>>>>>> simply compare the whole byte array. Here's the test-case: >>>>>>> >>>>>>> public class Test { >>>>>>> public static void main(String[] args) { >>>>>>> StringBuilder sb1 = new StringBuilder("test1"); >>>>>>> StringBuilder sb2 = new StringBuilder("test2"); >>>>>>> sb1.setLength(4); >>>>>>> sb2.setLength(4); >>>>>>> System.out.println(sb1.compareTo(sb2)); >>>>>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> We truncated the stringbuilders making their content equal, so >>>>>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo >>>>>>> compares >>>>>>> the original content (before the truncation) as truncation, of >>>>>>> course, >>>>>>> does not zero the truncated bytes, neither does it reallocate the >>>>>>> array (unless explicitly asked via trimToSize). >>>>>>> >>>>>>> With best regards, >>>>>>> Tagir Valeev. >>>>>>> >>>>>>> >>>>>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>>>>> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>>>>> StringBuffer. >>>>>>>> >>>>>>>> The Comparable implementations for StringBuilder/Buffer are >>>>>>>> similar >>>>>>>> to that >>>>>>>> of String, allowing comparison operations between two >>>>>>>> StringBuilders/Buffers, e.g. >>>>>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>>>>> method >>>>>>>> 'compare' is added instead of implementing the Comparable >>>>>>>> interface. >>>>>>>> This >>>>>>>> 'compare' method may take CharSequence implementations such as >>>>>>>> String, >>>>>>>> StringBuilder and StringBuffer, making it possible to perform >>>>>>>> comparison >>>>>>>> among them. The previous example for example is equivalent to >>>>>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>>>>> >>>>>>>> Tests for java.base have been independent from each other. The new >>>>>>>> tests are >>>>>>>> therefore created to have no dependency on each other or >>>>>>>> sharing any >>>>>>>> code. >>>>>>>> >>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Joe >>>> >>> From rcmuir at gmail.com Wed Feb 21 20:07:07 2018 From: rcmuir at gmail.com (Robert Muir) Date: Wed, 21 Feb 2018 15:07:07 -0500 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: <5A8DB788.4010509@oracle.com> References: <5A8D125A.5030205@oracle.com> <22bca8a4-5548-7af9-9770-ad7b749191c9@oracle.com> <5A8DB788.4010509@oracle.com> Message-ID: On Wed, Feb 21, 2018 at 1:16 PM, Xueming Shen wrote: > > Hi Robert, > > Understood a silent replacement might not be the desired behavior in > some use scenarios. Anymore details regarding what "most apps want" > when there is/are malformed/unmappable? It appears the best the > underneath de/encoder can do here is to throw an IOException. Given > the caller of the Reader/Writer does not have the access to the bytes of > the underlying stream src (reader)/dst(writer), there is in theory > impossible > to do anything to recover and continue without risking data loss. The > assumption here is if you want to have a fine-grained control of the de/ > encoding, you might want to work with the Input/OutStream/Channel + > CharsetDe/Encoder instead of Reader/Writer. > > No, I'm not saying we can't do > Reader(CharsetDecoder)/Writer(CharsetEncoder), > just wanted to know what's the real use scenario and what's the better/ > best choice here. > I think the exception is the best default? This is the default behavior of python for example, unless you specifically ask for "replace" or "ignore". >>> b'\xFFabc'.decode("utf-8") Traceback (most recent call last): File "", line 1, in UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte Its also the default behavior of 'iconv' command-line tool used for converting charsets, unless you pass additional options. $ iconv -f utf-8 -t utf-8 test2.mp4 ftypisomisomiso2avc1mp41e iconv: test2.mp4:1:26: cannot convert Unfortunately in java, when using Charset or String parameters, it gives silently replacement with \uFFFD, etc. Its necessary to pass a CharsetDecoder to get an exception that something went wrong. The current situation is especially confusing as there is nothing in the javadocs to indicate that the behavior of InputStreamReader(x, Charset) and InputStreamReader(x, String) differ substantially from InputStreamReader(x, CharsetDecoder) ! I think the Charset and String parameters should default to REPORT, so the behavior of all constructors are consistent. If you want to replace, you should have to ask for it. I think replacement has use-cases but they are more "expert", e.g. web-crawling and so on. In general, wrong bytes indicate a problem and it can be very difficult to debug these issues when java hides these problems by default... From martinrb at google.com Wed Feb 21 20:30:25 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 21 Feb 2018 12:30:25 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: OK, we have a reworked set of patches. (In my corner of openjdk we generally use real javadoc on private elements.) I reverted private doc comment style to the current maddening inconsistency, except I couldn't restrain myself from fixing - // ACC used when loading classes and resources */ + // ACC used when loading classes and resources I changed ArrayDeque.push to addFirst, as Peter suggested. 8198480: Improve ClassLoaders static init block http://cr.openjdk.java.net/~martin/webrevs/jdk/ClassLoaders-static/ https://bugs.openjdk.java.net/browse/JDK-8198480 8198481: Coding style cleanups for src/java.base/share/classes/jdk/internal/loader http://cr.openjdk.java.net/~martin/webrevs/jdk/loader-style/ https://bugs.openjdk.java.net/browse/JDK-8198481 8198482: The URLClassPath field "urls" should be renamed to "unopenedUrls" http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-rename-urls/ https://bugs.openjdk.java.net/browse/JDK-8198482 8198484: URLClassPath should use an ArrayDeque instead of a Stack http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ https://bugs.openjdk.java.net/browse/JDK-8198484 8198485: Simplify a URLClassPath constructor http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-simplify-constructor/ https://bugs.openjdk.java.net/browse/JDK-8198485 From uschindler at apache.org Wed Feb 21 20:50:11 2018 From: uschindler at apache.org (Uwe Schindler) Date: Wed, 21 Feb 2018 21:50:11 +0100 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: References: <5A8D125A.5030205@oracle.com> <001301d3aaf1$827285a0$875790e0$@apache.org> Message-ID: <00e001d3ab55$9282c590$b78850b0$@apache.org> Hi Alan, > > The Java 7+ methods in java.nio.file.Files already ignore the default charset > and always use UTF-8. How to proceed with those? Should they be changed > to behave to the new mechanisms? I'd suggest to not do this, as its part of > the spec (to use UTF-8) and should not rely on external forces, but I wanted > to bring this in. > > > There is no proposal to change these methods. Thanks for clarifying! I just wanted to mention this, because those methods are different, so you should at least think about it ?? Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ From huizhe.wang at oracle.com Wed Feb 21 20:50:14 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 21 Feb 2018 12:50:14 -0800 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: <618b0785b9dd4e1ca47df9b448f30637@sap.com> References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> <618b0785b9dd4e1ca47df9b448f30637@sap.com> Message-ID: <5A8DDB86.1020006@oracle.com> > > @Joe: Is there some documentation for this change in default behavior that came with JDK8u161? I assume it is for higher security and cannot be reverted (e.g. by setting the feature default for Djdk.xml.overrideDefaultParser to true). It is indeed. It was a customer's request. Customers' complaints were that a factory created through the official API could in many cases, unknowingly to the customers, load 3rd party parsers that didn't necessarily implement the security features added since JDK7u40 and 8. For customers, this behavior was a security concern. It was particularly inconvenient to users who might have some 3rd party libraries that just happen to be in their environment. This change was not mentioned in the release notes. I'll check whether we could find a right place for a note of this change. The 8u161 release notes would have been a good place to do so. Best, Joe > > Best regards > Christoph > >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >> Behalf Of Bernd >> Sent: Dienstag, 13. Februar 2018 22:55 >> To: OpenJDK Dev list >> Subject: Re: Oracle Java 8u161 regression in XML Schema Factory >> >> Hello, >> >> >> 2018-01-25 17:41 GMT+01:00 Se?n Coffey: >> >>> Classes nearer to those below were touched via JDK-8186080: Transform >> XML >>> interfaces >>> http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 >>> http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 >>> >>> This may be connected with some tools trying to redefine the classes >>> perhaps. Needs more investigating. Perhaps the XMLSchemaLoader >> changes are >>> a factor ? >>> >> I have ben able to extract a minimal reproducer. It is not related to >> XMLUnit, only to powermock. If it instruments com.sun but not javax.xml >> (and other combinations) then it fails. >> >> For details see the readme in this maven project: >> >> https://github.com/ecki/reproduce-schemaex >> >> I also found a way to make it work with both versions, so its no longer an >> issue for me, but there is definitely some changes (which might also be >> triggered in AppServers or OSGi containers with partially reconfigured >> implementations. Not sure if you want to investigate deeper). >> >> Gruss >> Bernd >> >> >> Here is the stacktrace anyway: >>>> com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: Schema >>>> factory class >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl >> does >>>> not >>>> extend from SchemaDVFactory. >>>> at >>>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. >>>> getInstance(SchemaDVFactory.java:75) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. >>>> getInstance(SchemaDVFactory.java:57) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >>>> reset(XMLSchemaLoader.java:1024) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >>>> loadGrammar(XMLSchemaLoader.java:556) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >>>> loadGrammar(XMLSchemaLoader.java:535) >>>> at >>>> com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchema >>>> Factory.newSchema(XMLSchemaFactory.java:254) >>>> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. >>>> java:638) >>>> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. >>>> java:654) >>>> at >>>> com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchema >>>> Validator(BuilderTestHelper.java:57) >>>> at >>>> com.seeburger.api.test.helpers.BuilderTestHelper.validateAnd >>>> Compare(BuilderTestHelper.java:73) >>>> at >>>> com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuild >>>> erTest.java:196) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >>>> ssorImpl.java:62) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >>>> thodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>>> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod >>>> (PowerMockJUnit44RunnerDelegateImpl.java:310) >>>> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie. >>>> java:89) >>>> at >>>> org.junit.internal.runners.MethodRoadie.runBeforesThenTestTh >>>> enAfters(MethodRoadie.java:97) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>>> unnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(P >>>> owerMockJUnit44RunnerDelegateImpl.java:294) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R >>>> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestIn >>>> Super(PowerMockJUnit47RunnerDelegateImpl.java:127) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R >>>> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(P >>>> owerMockJUnit47RunnerDelegateImpl.java:82) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>>> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThe >>>> nTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) >>>> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie >>>> .java:87) >>>> at >> org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>>> unnerDelegateImpl.invokeTestMethod(PowerMockJUni >>>> t44RunnerDelegateImpl.java:207) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>>> >> unnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.ja >> va:146) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>>> >> unnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) >>>> at >>>> org.junit.internal.runners.ClassRoadie.runUnprotected(ClassR >>>> oadie.java:34) >>>> at >>>> org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) >>>> at >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>>> >> unnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) >>>> at >>>> org.powermock.modules.junit4.common.internal.impl.JUnit4Test >>>> SuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) >>>> at >>>> org.powermock.modules.junit4.common.internal.impl.AbstractCo >>>> >> mmonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) >>>> at >>>> >> org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner. >> java:59) >>>> at >>>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. >>>> run(JUnit4TestReference.java:86) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.TestExecution.run( >>>> TestExecution.java:38) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >>>> sts(RemoteTestRunner.java:539) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >>>> sts(RemoteTestRunner.java:761) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( >>>> RemoteTestRunner.java:461) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( >>>> RemoteTestRunner.java:207) >>>> >>>> on the classpath jaxb-impl-2.2.5.jar but the specific packages are only >>>> loaded from rt.jar and redefined. I asume the later is done by >> Powermock. >>>> Line 611: [Loaded >>>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from >>>> C:\Program >>>> Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >>>> Line 616: [Loaded >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory >> from >>>> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >>>> Line 617: [Loaded >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl >> from >>>> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >>>> Line 618: [Loaded >>>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from >>>> __JVM_DefineClass__] >>>> Line 619: [Loaded >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory >> from >>>> __JVM_DefineClass__] >>>> Line 620: [Loaded >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl >> from >>>> __JVM_DefineClass__] >>>> >>>> Is that something you are concerned? >>>> >>>> Gruss >>>> Bernd >>>> From claes.redestad at oracle.com Wed Feb 21 21:00:52 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 21 Feb 2018 22:00:52 +0100 Subject: RFR: 8198523: Refactor BootstrapMethodInvoker to further avoid runtime type checks Message-ID: Hi, BootstrapMethodInvoker can be improved a bit further by providing static information about the type argument, and additionally teaching it about StringConcatFactory BSM: Patch: http://cr.openjdk.java.net/~redestad/8198523/jdk.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8198523 This slightly improves startup on string concat tests. Thanks! /Claes From naoto.sato at oracle.com Wed Feb 21 21:13:51 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 21 Feb 2018 13:13:51 -0800 Subject: [11] RFR: JDK-8198385: Remove property sun.locale.formatasdefault Message-ID: <81803e97-2eb9-7efe-4c32-5be1e92b82f8@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8198385 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8198385/webrev.00/ The property was introduced in JDK7 for the backward compatibility, which at this point is no longer needed. Corresponding CSR is already approved. Naoto From yumin.qi at gmail.com Wed Feb 21 21:54:44 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 21 Feb 2018 13:54:44 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> <69e8a6d9-3657-36b0-23c2-37db2775c633@oracle.com> <3ca965ec-c472-1522-e89d-3c2092b7ed3e@oracle.com> Message-ID: Alan, Somehow there is a problem for me to push the changeset: pushing to ssh://minqi at hg.openjdk.java.net/jdk/jdk searching for changes remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 3 changes to 3 files remote: [jcheck fc6df5c6d4d2 2018-01-23 14:21:17 -0800] remote: remote: > Changeset: 48927:0736b1e12c70 remote: > Author: Yumin Qi remote: > Date: 2018-02-21 12:01 remote: > remote: > 8194154: System property user.dir should not be changed remote: > Summary: Cached user.dir so getCanonicalPath uses the cached value. remote: > Reviewed-by: alanb, bpb, rriggs remote: remote: Invalid changeset author: Yumin Qi remote: remote: transaction abort! remote: rollback completed remote: abort: pretxnchangegroup.0.jcheck hook failed What is the real reason for the failure? I am in following projects: Projects hsx HotSpot Express Project ? Author jdk JDK Project ? Committer jdk-updates JDK Updates Project ? Committer jdk10 JDK 10 Project ? Committer Thanks Yumin On Mon, Feb 19, 2018 at 10:11 AM, yumin qi wrote: > Yes, I am committer. > > Brian, do you okay with the version? If no objection, I will push it into > jdk. > > Thanks > Yumin > > On Mon, Feb 19, 2018 at 2:28 AM, Alan Bateman > wrote: > >> On 19/02/2018 05:09, yumin qi wrote: >> >>> Thanks! >>> >>> I need a sponsor for pushing it to jdk. Can you or someone else help to >>> push it? >>> >> minqi is a jdk committer. If this is you then you should be able to push >> it yourself. >> >> -Alan. >> > > From Roger.Riggs at Oracle.com Wed Feb 21 22:00:29 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 21 Feb 2018 17:00:29 -0500 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> <69e8a6d9-3657-36b0-23c2-37db2775c633@oracle.com> <3ca965ec-c472-1522-e89d-3c2092b7ed3e@oracle.com> Message-ID: <830cf757-b7f4-0d52-759f-fcbed9b62fc4@Oracle.com> Hi Yumin, Use your OpenJDK id? "minqi" for the Author instead of your email. Roger On 2/21/2018 4:54 PM, yumin qi wrote: > Alan, > > Somehow there is a problem for me to push the changeset: > pushing to ssh://minqi at hg.openjdk.java.net/jdk/jdk > searching for changes > remote: adding changesets > remote: adding manifests > remote: adding file changes > remote: added 1 changesets with 3 changes to 3 files > remote: [jcheck fc6df5c6d4d2 2018-01-23 14:21:17 -0800] > remote: > remote: > Changeset: 48927:0736b1e12c70 > remote: > Author: Yumin Qi > remote: > Date: 2018-02-21 12:01 > remote: > > remote: > 8194154: System property user.dir should not be changed > remote: > Summary: Cached user.dir so getCanonicalPath uses the cached > value. > remote: > Reviewed-by: alanb, bpb, rriggs > remote: > remote: Invalid changeset author: Yumin Qi > remote: > remote: transaction abort! > remote: rollback completed > remote: abort: pretxnchangegroup.0.jcheck hook failed > > What is the real reason for the failure? > I am in following projects: > > Projects > hsx HotSpot Express Project ? Author > jdk JDK Project ? Committer > jdk-updates JDK Updates > Project ? Committer > jdk10 JDK 10 Project ? Committer > > Thanks > Yumin > > On Mon, Feb 19, 2018 at 10:11 AM, yumin qi wrote: > >> Yes, I am committer. >> >> Brian, do you okay with the version? If no objection, I will push it into >> jdk. >> >> Thanks >> Yumin >> >> On Mon, Feb 19, 2018 at 2:28 AM, Alan Bateman >> wrote: >> >>> On 19/02/2018 05:09, yumin qi wrote: >>> >>>> Thanks! >>>> >>>> I need a sponsor for pushing it to jdk. Can you or someone else help to >>>> push it? >>>> >>> minqi is a jdk committer. If this is you then you should be able to push >>> it yourself. >>> >>> -Alan. >>> >> From cushon at google.com Wed Feb 21 22:18:35 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Wed, 21 Feb 2018 14:18:35 -0800 Subject: RFR JDK-8198526 Message-ID: Hello, Please review this fix to the handling of static nested classes in getAnnotatedOwnerType. webrev: http://cr.openjdk.java.net/~cushon/8198526/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8198526 From yumin.qi at gmail.com Wed Feb 21 22:25:11 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 21 Feb 2018 14:25:11 -0800 Subject: RFR: 8194154: JDK crashes parsing path string contains '//' on linux In-Reply-To: <830cf757-b7f4-0d52-759f-fcbed9b62fc4@Oracle.com> References: <8ef07993-4b72-ade3-2443-ba74fc125656@oracle.com> <888f9a12-e87b-f51b-551b-9ec05d0924a8@oracle.com> <69e8a6d9-3657-36b0-23c2-37db2775c633@oracle.com> <3ca965ec-c472-1522-e89d-3c2092b7ed3e@oracle.com> <830cf757-b7f4-0d52-759f-fcbed9b62fc4@Oracle.com> Message-ID: Roger, Thanks. Pushed. On Wed, Feb 21, 2018 at 2:00 PM, Roger Riggs wrote: > Hi Yumin, > > Use your OpenJDK id "minqi" for the Author instead of your email. > > Roger > > > > On 2/21/2018 4:54 PM, yumin qi wrote: > >> Alan, >> >> Somehow there is a problem for me to push the changeset: >> pushing to ssh://minqi at hg.openjdk.java.net/jdk/jdk >> searching for changes >> remote: adding changesets >> remote: adding manifests >> remote: adding file changes >> remote: added 1 changesets with 3 changes to 3 files >> remote: [jcheck fc6df5c6d4d2 2018-01-23 14:21:17 -0800] >> remote: >> remote: > Changeset: 48927:0736b1e12c70 >> remote: > Author: Yumin Qi >> remote: > Date: 2018-02-21 12:01 >> remote: > >> remote: > 8194154: System property user.dir should not be changed >> remote: > Summary: Cached user.dir so getCanonicalPath uses the cached >> value. >> remote: > Reviewed-by: alanb, bpb, rriggs >> remote: >> remote: Invalid changeset author: Yumin Qi >> remote: >> remote: transaction abort! >> remote: rollback completed >> remote: abort: pretxnchangegroup.0.jcheck hook failed >> >> What is the real reason for the failure? >> I am in following projects: >> >> Projects >> hsx HotSpot Express Project ? Author >> jdk JDK Project ? Committer >> jdk-updates JDK Updates >> Project ? Committer >> jdk10 JDK 10 Project ? Committer >> >> Thanks >> Yumin >> >> On Mon, Feb 19, 2018 at 10:11 AM, yumin qi wrote: >> >> Yes, I am committer. >>> >>> Brian, do you okay with the version? If no objection, I will push it into >>> jdk. >>> >>> Thanks >>> Yumin >>> >>> On Mon, Feb 19, 2018 at 2:28 AM, Alan Bateman >>> wrote: >>> >>> On 19/02/2018 05:09, yumin qi wrote: >>>> >>>> Thanks! >>>>> >>>>> I need a sponsor for pushing it to jdk. Can you or someone else help to >>>>> push it? >>>>> >>>>> minqi is a jdk committer. If this is you then you should be able to >>>> push >>>> it yourself. >>>> >>>> -Alan. >>>> >>>> >>> > From brian.burkhalter at oracle.com Wed Feb 21 22:25:21 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 21 Feb 2018 14:25:21 -0800 Subject: [11] RFR: JDK-8198385: Remove property sun.locale.formatasdefault In-Reply-To: <81803e97-2eb9-7efe-4c32-5be1e92b82f8@oracle.com> References: <81803e97-2eb9-7efe-4c32-5be1e92b82f8@oracle.com> Message-ID: <8D1E79AC-BBDE-4D5F-8DA4-7EEEDBCF9B85@oracle.com> Hi Naoto, +1 Brian On Feb 21, 2018, at 1:13 PM, naoto.sato at oracle.com wrote: > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8198385 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8198385/webrev.00/ > > The property was introduced in JDK7 for the backward compatibility, which at this point is no longer needed. Corresponding CSR is already approved. From brian.burkhalter at oracle.com Wed Feb 21 22:41:14 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 21 Feb 2018 14:41:14 -0800 Subject: RFR 8189330 Cleanup FileDescriptor implementation In-Reply-To: References: Message-ID: <4790E6BB-FE64-4629-8107-0A09DC0633D4@oracle.com> Hi Roger, I think that this looks fine. I assume the issue should be labelled ?noreg-cleanup." Brian On Feb 16, 2018, at 2:28 PM, Roger Riggs wrote: > Please review a cleanup and refactoring of FileDescriptor.java. > The Unix and Windows versions are merged to ease maintenance remove replicated code > The FileCleanable class is extracted and SocketCleanable is updated. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-fd-cleanup-8189330/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8189330 From james at lightbend.com Thu Feb 22 00:47:04 2018 From: james at lightbend.com (James Roper) Date: Thu, 22 Feb 2018 11:47:04 +1100 Subject: Reactive Streams utility API Message-ID: Hi all, This is an email to give people a heads up that we'd like to look at creating an API, in the same vein as the JDK8 Streams API, for building reactive streams (a la JDK9 juc.Flow). Our goals for this are: * To fill a gap in the JDK where if a developer wants to do even the simplest of things with a JDK9 juc.Flow, such as map or filter, they need to bring in a third party library that implements that. * To produce an API that can build Publishers, Subscribers, Processors, and complete graphs, for the purposes of consuming APIs that use reactive streams (for example, JDK9 Http Client). * To produce an API that aligns closely with ju.stream.Stream, using it for inspiration for naming, scope, general API shape, and other aspects. The purpose of this goal is to ensure familiarity of Java developers with the new API, and to limit the number of concepts Java developers need to understand to do the different types of streaming offered by the JDK. * To produce an API that can be implemented by multiple providers (including an RI in the JDK itself), using the ServiceLoader mechanism to provide and load a default implementation (while allowing custom implementations to be manually provided). There are a lot of concerns that each different streams implementation provides and implements, beyond streaming, for example monitoring/tracing, concurrency modelling, buffering strategies, performance aspects of the streams handling including fusing, and context (eg thread local) propagation. This will allow libraries to use and provide contracts based on this API without depending on a particular implementation, and allows developers to select the implementation that meets their needs. Non goals: * To produce a kitchen sink of utilities for working with reactive streams. There already exist a number of reactive streams implementations that seek to meet this goal (eg, Akka Streams, Reactor, RxJava), and once you go past the basics (map, filter, collect), and start dealing with things like fan in/out, cycles, restarting, etc, the different approaches to solving this start to vary greatly. The JDK should provide enough to be useful for typical every day streaming use cases, with developers being able to select a third party library for anything more advanced. We will update this list when we have something ready for public review. This probably won't be far off. Our hope is that we can propose this as a JEP. Regards, James -- *James Roper* *Senior Octonaut* Lightbend ? Build reactive apps! Twitter: @jroper From paul.sandoz at oracle.com Thu Feb 22 01:29:39 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 21 Feb 2018 17:29:39 -0800 Subject: RFR: 8198523: Refactor BootstrapMethodInvoker to further avoid runtime type checks In-Reply-To: References: Message-ID: +1 Paul. > On Feb 21, 2018, at 1:00 PM, Claes Redestad wrote: > > Hi, > > BootstrapMethodInvoker can be improved a bit further by providing > static information about the type argument, and additionally > teaching it about StringConcatFactory BSM: > > Patch: http://cr.openjdk.java.net/~redestad/8198523/jdk.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8198523 > > This slightly improves startup on string concat tests. > > Thanks! > > /Claes From paul.sandoz at oracle.com Thu Feb 22 01:38:53 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 21 Feb 2018 17:38:53 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: <985ABBE0-2539-40C1-9AFE-C282C6DA6FA8@oracle.com> Hi Ben, Much better :-) thanks for doing this. Before preceding with replacing @DontInline with @ForceInline i would like to appeal to Vladimir to confirm this is ok. If you send me an attached patch directly in email i can upload it as a webrev for you, it should be easier to review. (When you have two sponsored contributions you can request the author role which gives you an account on JIRA and to the cr system so you can upload webrevs [2] yourself). Paul. [1] http://openjdk.java.net/projects/#project-author [2] http://openjdk.java.net/guide/webrevHelp.html > On Feb 19, 2018, at 8:37 AM, Ben Walsh wrote: > > As requested, here are the results with modifications to the annotations > on Reference.reachabilityFence. Much more promising ? From david.holmes at oracle.com Thu Feb 22 02:08:44 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Feb 2018 12:08:44 +1000 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <3bc62991-7585-8b3e-8d83-7764181f9cdd@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> <27436fb7-e7f8-bc47-924e-aeced7057fe3@oracle.com> <3bc62991-7585-8b3e-8d83-7764181f9cdd@oracle.com> Message-ID: <70941614-3567-ba5d-3d99-bcccaa50f652@oracle.com> Hi Mandy, tl;dr: I think this is now good to go. On 22/02/2018 5:58 AM, mandy chung wrote: > Here is the updated webrev: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.04/ > > I added some comments to clarify the implementation. src/java.base/share/classes/java/lang/Shutdown.java 96 if (VM.isShutdown()|| slot <= currentRunningHook) Nit: need space after () 113 private static void runHooks() { 114 synchronized (lock) { 115 /* Guard against the possibility of a daemon thread invoking exit 116 * after DestroyJavaVM initiates the shutdown sequence 117 */ 118 if (VM.isShutdown()) return; 119 } I think this is actually impossible to hit, but that's a separate cleanup. 139 // set shutdown state 140 VM.shutdown(); Just an observation that we consider "shutdown" to be after all system shutdown hooks are run. So this (contrary to what I wrote in the CSR) narrows the window in which a concurrent exit(-1) would trigger an immediate halt rather than a hang. Which in turn strengthens the argument for just dropping that behaviour. --- src/java.base/share/classes/jdk/internal/misc/VM.java 103 public static void shutdown() { 104 initLevel(SYSTEM_SHUTDOWN); 105 } Doing this is exactly what I meant by my previous comments. Thanks, David > Mandy > > On 2/21/18 11:34 AM, mandy chung wrote: >> Hi David, >> >> I think I'm clear on the implementation now (my mistake that I >> neglected ApplicationShutdownHooks).? Shutdown class keeps the "system >> shutdown hooks".? ApplicationShutdownHooks is one system hook that >> starts all application shutdown hooks and waits until they finish. >> java.io.Console and DeleteOnExitHook are two other system shutdown >> hooks to restore console echo state or support deleteOnExit. >> >> The system shutdown hooks are all run in the thread initiating the >> shutdown that holds the Shutdown.class. >> >> On 2/20/18 10:27 PM, David Holmes wrote: >>> Hi Mandy, >>> >>> On 21/02/2018 5:57 AM, mandy chung wrote: >>>> Hi David, >>>> >>>> I reworked the change in Shutdown class and uses >>>> jdk.internal.misc.VM to maintain the shutdown state, either in >>>> progress or shutdown (i.e. all shutdown hooks have been started). >>>> >>>> What do you think this revised version: >>>> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.03/ >>> >>> It confuses me a bit. On the one hand I wasn't sure why we needed to >>> bring the VM class into this, then on the other hand it perhaps made >>> sense to have VM track shutdown states as well as initialization states. >>> >> >> The latter is my motivation and I think it's a good cleanup to have VM >> state to indicate it's shutdown. >> >>> But I'm not sure about the two-phase "shutdown" state - I think I'd >>> probably prefer a "shutdown initiated" state and a "shutdown >>> complete" state. Then no need to duplicate the constant values for >>> IN_PROGRESS and SHUTDOWN (which now need to be kept in sync with >>> changes to the VM class). It would also simplify the shutdown() logic. >>> >> >> Shutdown::add method can be used to register a system hook while >> shutdown is in progress.? Actually currentRunningHook can be used to >> guard if Shutdown::add should reject to add a new hook.? I updated the >> webrev and no longer needs the two phases. >> >>> Also shutdown(level) should be using initLevel(value) so that it fits >>> in with the existing awaitInitLevel() logic and locking strategy! >>> Someone may want to wait for shutdown in the future. That also deals >>> with the locking issue in the Shutdown class because you don't need >>> to use "synchronized(lock)" because runHooks is only called within >>> "synchronized(Shutdown.class)". [To be fair the existing locking >>> strategy seems confused to me as well - or at least it confuses me.] >> >> I agree but I tried not to touch the existing locking strategy as it >> should be a separate changeset.? I prefer to add VM::shutdown method >>> >>> I also now realize that the changes I suggested for the Runtime.exit >>> docs was incorrect. The existing docs state that we only halt if >>> hooks have already been run - which corresponds to the old and new >>> code. I, for some reason that escapes me, claimed we only cared if >>> the hooks had been started, not completed - but that is inconsistent >>> with old spec and the implementation. >>> >> >> I confused myself too.? A small change will fix it: >> >> 87 *

If this method is invoked after all shutdown hooks have already >> 88 * been run and the status is nonzero then this method halts the >> 89 * virtual machine with the given status code. Otherwise, this method >> >> >> >> Mandy >> >>> So apologies but what started out as a seemingly simple code removal, >>> has become a lot more complicated, and confusing to me. >>> >>> Thanks, >>> David >>> ----- >>> >>>> On 2/15/18 9:14 PM, David Holmes wrote: >>>>> >>>>> All other updates seem okay. I did have one further thought - in >>>>> Runtime does this change: >>>>> >>>>> ????? public void runFinalization() { >>>>> ! SharedSecrets.getJavaLangRefAccess().runFinalization(); >>>>> ????? } >>>>> >>>>> affect the classloading/initialization order at all? >>>> >>>> Runtime::runFinalization is not called by the system code.? So it >>>> won't be invoked during startup and hence won't change the >>>> classloading order during startup. >>>> >>>> Mandy >> > From paul.sandoz at oracle.com Thu Feb 22 02:18:30 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 21 Feb 2018 18:18:30 -0800 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: <39D8F43A-06BD-483B-8901-6F4444A8235F@oracle.com> Message-ID: <4B429F1D-5727-4B20-A051-E39E1E8C69AA@oracle.com> Hi Adam, While the burden is minimal there is a principle here that i think we should adhere to regarding additions to the code base: additions should have value within OpenJDK itself otherwise it can become a thin end of the wedge to more stuff (?well you added these things, why not just add these too??). So i would still be reluctant to add such methods without understanding the larger picture and what you have in mind. Can you send a pointer to your email referring in more detail to the larger change sets? This use-case might also apply in other related areas too with regards to logging/monitoring. I would be interested to understand what Java Flight Recorder (JFR) does in this regard (it being open sourced soon i believe) and how JFR might relate to what you are doing. Should we be adding JFR events to unsafe memory allocation? Can JFR efficiently access part of the Java call stack to determine the origin? Thanks, Paul. > On Feb 19, 2018, at 5:08 AM, Adam Farley8 wrote: > > Hi Paul, > > > Hi Adam, > > > > From reading the thread i cannot tell if this is part of a wider solution including some yet to be proposed HotSpot changes. > > The wider solution would need to include some Hotspot changes, yes. > I'm proposing raising a bug, committing the code we have here to > "set the stage", and then we can invest more time&energy later > if the concept goes down well and the community agrees to pursue > the full solution. > > As an aside, I tried submitting a big code set (including hotspot > changes) months ago, and I'm *still* struggling to find someone to > commit the thing, so I figured I'd try a more gradual, staged approach > this time. > > > > > As is i would be resistant to adding such standalone internal wrapper methods to Unsafe that have no apparent benefit within the OpenJDK itself since it's a maintenance burden. > > I'm hoping the fact that the methods are a single line (sans > comments, descriptors and curly braces) will minimise this burden. > > > > > Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come from a DBB by looking at the call stack frame above the unsafe call? > > > > Thanks, > > Paul. > > Yes that is possible, though I would advise against this because: > > A) Checking the call stack is expensive, and doing this every time we > allocate native memory is an easy way to slow down a program, > or rack up mips. > and > B) deciding which code path we're using based on the stack > means the DBB class+method (and anything the parsing code > mistakes for that class+method) can only ever allocate native > memory for DBBs. > > What do you think? > > Best Regards > > Adam Farley > > > > >> On Feb 14, 2018, at 3:32 AM, Adam Farley8 wrote: > >> > >> Hi All, > >> > >> Currently, diagnostic core files generated from OpenJDK seem to lump all > >> of the > >> native memory usages together, making it near-impossible for someone to > >> figure > >> out *what* is using all that memory in the event of a memory leak. > >> > >> The OpenJ9 VM has a feature which allows it to track the allocation of > >> native > >> memory for Direct Byte Buffers (DBBs), and to supply that information into > >> the > >> cores when they are generated. This makes it a *lot* easier to find out > >> what is using > >> all that native memory, making memory leak resolution less like some dark > >> art, and > >> more like logical debugging. > >> > >> To use this feature, there is a native method referenced in Unsafe.java. > >> To open > >> up this feature so that any VM can make use of it, the java code below > >> sets the > >> stage for it. This change starts letting people call DBB-specific methods > >> when > >> allocating native memory, and getting into the habit of using it. > >> > >> Thoughts? > >> > >> Best Regards > >> > >> Adam Farley > >> > >> P.S. Code: > >> > >> diff --git > >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >> @@ -85,7 +85,7 @@ > >> // Paranoia > >> return; > >> } > >> - UNSAFE.freeMemory(address); > >> + UNSAFE.freeDBBMemory(address); > >> address = 0; > >> Bits.unreserveMemory(size, capacity); > >> } > >> @@ -118,7 +118,7 @@ > >> > >> long base = 0; > >> try { > >> - base = UNSAFE.allocateMemory(size); > >> + base = UNSAFE.allocateDBBMemory(size); > >> } catch (OutOfMemoryError x) { > >> Bits.unreserveMemory(size, cap); > >> throw x; > >> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >> @@ -632,6 +632,26 @@ > >> } > >> > >> /** > >> + * Allocates a new block of native memory for DirectByteBuffers, of > >> the > >> + * given size in bytes. The contents of the memory are > >> uninitialized; > >> + * they will generally be garbage. The resulting native pointer will > >> + * never be zero, and will be aligned for all value types. Dispose > >> of > >> + * this memory by calling {@link #freeDBBMemory} or resize it with > >> + * {@link #reallocateDBBMemory}. > >> + * > >> + * @throws RuntimeException if the size is negative or too large > >> + * for the native size_t type > >> + * > >> + * @throws OutOfMemoryError if the allocation is refused by the > >> system > >> + * > >> + * @see #getByte(long) > >> + * @see #putByte(long, byte) > >> + */ > >> + public long allocateDBBMemory(long bytes) { > >> + return allocateMemory(bytes); > >> + } > >> + > >> + /** > >> * Resizes a new block of native memory, to the given size in bytes. > >> The > >> * contents of the new block past the size of the old block are > >> * uninitialized; they will generally be garbage. The resulting > >> native > >> @@ -687,6 +707,27 @@ > >> } > >> > >> /** > >> + * Resizes a new block of native memory for DirectByteBuffers, to the > >> + * given size in bytes. The contents of the new block past the size > >> of > >> + * the old block are uninitialized; they will generally be garbage. > >> The > >> + * resulting native pointer will be zero if and only if the requested > >> size > >> + * is zero. The resulting native pointer will be aligned for all > >> value > >> + * types. Dispose of this memory by calling {@link #freeDBBMemory}, > >> or > >> + * resize it with {@link #reallocateDBBMemory}. The address passed > >> to > >> + * this method may be null, in which case an allocation will be > >> performed. > >> + * > >> + * @throws RuntimeException if the size is negative or too large > >> + * for the native size_t type > >> + * > >> + * @throws OutOfMemoryError if the allocation is refused by the > >> system > >> + * > >> + * @see #allocateDBBMemory > >> + */ > >> + public long reallocateDBBMemory(long address, long bytes) { > >> + return reallocateMemory(address, bytes); > >> + } > >> + > >> + /** > >> * Sets all bytes in a given block of memory to a fixed value > >> * (usually zero). > >> * > >> @@ -918,6 +959,17 @@ > >> checkPointer(null, address); > >> } > >> > >> + /** > >> + * Disposes of a block of native memory, as obtained from {@link > >> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address > >> passed > >> + * to this method may be null, in which case no action is taken. > >> + * > >> + * @see #allocateDBBMemory > >> + */ > >> + public void freeDBBMemory(long address) { > >> + freeMemory(address); > >> + } > >> + > >> /// random queries > >> > >> /** > >> > >> Unless stated otherwise above: > >> IBM United Kingdom Limited - Registered in England and Wales with number > >> 741598. > >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From mandy.chung at oracle.com Thu Feb 22 02:28:57 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 21 Feb 2018 18:28:57 -0800 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <70941614-3567-ba5d-3d99-bcccaa50f652@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> <27436fb7-e7f8-bc47-924e-aeced7057fe3@oracle.com> <3bc62991-7585-8b3e-8d83-7764181f9cdd@oracle.com> <70941614-3567-ba5d-3d99-bcccaa50f652@oracle.com> Message-ID: <10afa8b8-963c-8049-4d85-75b75982b620@oracle.com> On 2/21/18 6:08 PM, David Holmes wrote: > Hi Mandy, > > tl;dr: I think this is now good to go. > > On 22/02/2018 5:58 AM, mandy chung wrote: >> Here is the updated webrev: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.04/ >> >> I added some comments to clarify the implementation. > > src/java.base/share/classes/java/lang/Shutdown.java > > ?96???????????????? if (VM.isShutdown()|| slot <= currentRunningHook) > > Nit: need space after () > > Fixed. > ?113 private static void runHooks() { > ?114???????? synchronized (lock) { > ?115???????????? /* Guard against the possibility of a daemon thread > invoking exit > ?116????????????? * after DestroyJavaVM initiates the shutdown sequence > ?117????????????? */ > ?118???????????? if (VM.isShutdown()) return; > ?119???????? } > > I think this is actually impossible to hit, but that's a separate > cleanup. When T1 is calling runHooks, T2 calls exit and blocks on Shutdown.class, once T1 releases the lock if VM does not halt yet, T2 will enter this method. > > ?139???????? // set shutdown state > ?140???????? VM.shutdown(); > > Just an observation that we consider "shutdown" to be after all system > shutdown hooks are run. So this (contrary to what I wrote in the CSR) > narrows the window in which a concurrent exit(-1) would trigger an > immediate halt rather than a hang. Which in turn strengthens the > argument for just dropping that behaviour. If an application hook calls exit(0), it will hang (not the thread holding the Shutdown class lock).? Do you want to file an issue to track that? > > --- > > src/java.base/share/classes/jdk/internal/misc/VM.java > > ?103???? public static void shutdown() { > ?104???????? initLevel(SYSTEM_SHUTDOWN); > ?105???? } > > Doing this is exactly what I meant by my previous comments. > Good. Thanks for the thorough review. Mandy From david.holmes at oracle.com Thu Feb 22 03:38:57 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Feb 2018 13:38:57 +1000 Subject: [11] RFR JDK-8198249: Remove deprecated Runtime::runFinalizersOnExit and System::runFinalizersOnExit In-Reply-To: <10afa8b8-963c-8049-4d85-75b75982b620@oracle.com> References: <06744ce4-e41f-75b9-08c1-791080b24ac6@oracle.com> <8cf0f461-31a6-c316-c091-28e96c7e7b95@oracle.com> <8eafc64b-d400-cc63-4b42-486ebd1c19a6@oracle.com> <27436fb7-e7f8-bc47-924e-aeced7057fe3@oracle.com> <3bc62991-7585-8b3e-8d83-7764181f9cdd@oracle.com> <70941614-3567-ba5d-3d99-bcccaa50f652@oracle.com> <10afa8b8-963c-8049-4d85-75b75982b620@oracle.com> Message-ID: <9dba6a3c-a445-3fc2-2924-dd4b3aa4a771@oracle.com> Hi Mandy, On 22/02/2018 12:28 PM, mandy chung wrote: > On 2/21/18 6:08 PM, David Holmes wrote: >> Hi Mandy, >> >> tl;dr: I think this is now good to go. >> >> On 22/02/2018 5:58 AM, mandy chung wrote: >>> Here is the updated webrev: >>> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198249/webrev.04/ >>> >>> I added some comments to clarify the implementation. >> >> src/java.base/share/classes/java/lang/Shutdown.java >> >> ?96???????????????? if (VM.isShutdown()|| slot <= currentRunningHook) >> >> Nit: need space after () >> >> > > Fixed. > >> ?113 private static void runHooks() { >> ?114???????? synchronized (lock) { >> ?115???????????? /* Guard against the possibility of a daemon thread >> invoking exit >> ?116????????????? * after DestroyJavaVM initiates the shutdown sequence >> ?117????????????? */ >> ?118???????????? if (VM.isShutdown()) return; >> ?119???????? } >> >> I think this is actually impossible to hit, but that's a separate >> cleanup. > > When T1 is calling runHooks, T2 calls exit and blocks on Shutdown.class, > once T1 releases the lock if VM does not halt yet, T2 will enter this > method. Ah! T1 is executing destroyJavaVM - and of course releases the lock eventually. Got it. >> >> ?139???????? // set shutdown state >> ?140???????? VM.shutdown(); >> >> Just an observation that we consider "shutdown" to be after all system >> shutdown hooks are run. So this (contrary to what I wrote in the CSR) >> narrows the window in which a concurrent exit(-1) would trigger an >> immediate halt rather than a hang. Which in turn strengthens the >> argument for just dropping that behaviour. > > If an application hook calls exit(0), it will hang (not the thread > holding the Shutdown class lock).? Do you want to file an issue to track > that? If an application hook calls exit(0) it will block on the Shutdown.class lock which then hangs the whole shutdown process as the thread holding that lock is doing a join() on the hook thread - logical deadlock. But that wasn't my point at all. I just think this whole "calling exit with non-zero causes immediate halt" logic is meaningless now there are no finalizers running (and potentially calling exit). Cheers, David ------- >> >> --- >> >> src/java.base/share/classes/jdk/internal/misc/VM.java >> >> ?103???? public static void shutdown() { >> ?104???????? initLevel(SYSTEM_SHUTDOWN); >> ?105???? } >> >> Doing this is exactly what I meant by my previous comments. >> > > Good. > > Thanks for the thorough review. > Mandy > From huizhe.wang at oracle.com Thu Feb 22 08:39:51 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 22 Feb 2018 00:39:51 -0800 Subject: [11] RFR: (JAXP) 8038043: Xerces Update: XInclude update In-Reply-To: References: Message-ID: Hi Aleksei, Thanks for taking the time to work on this! Looks good overall. XIncludeHandler: setupCurrentBaseURI method can be private. XIncludeTextReader: there's a very long line at 191. It would be good to fix it so that Sdiffs looks better the next time. As for the tests, I'm fine with the encoding tests plus passing all existing ones. Best, Joe On 2/16/2018 9:53 AM, Aleks Efimov wrote: > Hi, > > Please, help to review the update of XInclude related classes from the > Apache Xerces 2.11.0 source. > JBS: > ??? https://bugs.openjdk.java.net/browse/JDK-8038043 > The webrev: > ??? http://cr.openjdk.java.net/~aefimov/8038043/11/00/ > > New regression test has been added to check the updated reporting of > invalid bytes encountered during the parsing and inclusion of XML > documents. New test and other regression tests shows no failures. > > With Best Regards, > Aleksei From christoph.langer at sap.com Thu Feb 22 08:42:39 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 22 Feb 2018 08:42:39 +0000 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) Message-ID: <58519ccf88ac4517b6186af613e385fd@sap.com> Hi, please review a simple import cleanup fix for java/util/jar/Attributes.java and sun/management/jdp/JdpController.java. Bug: https://bugs.openjdk.java.net/browse/JDK-8198539 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8198539.0/ Thanks and best regards Christoph From christoph.langer at sap.com Thu Feb 22 08:45:14 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 22 Feb 2018 08:45:14 +0000 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: <5a8dc206.f8aedf0a.6fae.9f4d@mx.google.com> References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> <618b0785b9dd4e1ca47df9b448f30637@sap.com> <5a8dc206.f8aedf0a.6fae.9f4d@mx.google.com> Message-ID: Hi Bernd, ok, your issue must be something different then. Best regards Christoph > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Bernd Eckenfels > Sent: Mittwoch, 21. Februar 2018 20:01 > To: OpenJDK Dev list > Subject: Re: Oracle Java 8u161 regression in XML Schema Factory > > Hallo Christoph, > > Yes the Tests fail with -Djdk.xml.overrideDefaultParser=true (and false) as > well. > > Gruss > Bernd > -- > http://bernd.eckenfels.net > > Von: Langer, Christoph > Gesendet: Mittwoch, 21. Februar 2018 16:16 > An: Bernd; OpenJDK Dev list; huizhe.wang at oracle.com > Betreff: RE: Oracle Java 8u161 regression in XML Schema Factory > > Hi Bernd, > > would your test still fail with system property "- > Djdk.xml.overrideDefaultParser=true"? > > I debugged a similar issue (if not the same) today and found that with change > http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jaxp/rev/08a44c164993 the > default behavior for Xalan changed a bit. > > Before that change, the services mechanism was enabled to load the SAX > and DOM implementations for the use of Xalan. E.g. when > TransformerFactory.newInstance() was called, the services mechanism was > used and in the TransformerFactory (at least if it was the default JDK > implementation), the property to use the services mechanism was set as > well - wich was used to retrieve SAX and DOM parser implementations. > > And what hit me even more was that SAX2DOM had its private field > "DocumentBuilderFactory _factory" automatically initialized by a call to > DocumentBuilderFactory.newInstance() which in turn would resolve any > custom DBF implementation if it were on classpath. Now, with the new > implementation, the _factory in SAX2DOM will be initialized with the > "overrideDefaultParser " setting coming from the Transformer Factory. So, > by setting -Djdk.xml.overrideDefaultParser=true, you will restore the old > behavior. > > @Joe: Is there some documentation for this change in default behavior that > came with JDK8u161? I assume it is for higher security and cannot be > reverted (e.g. by setting the feature default for > Djdk.xml.overrideDefaultParser to true). > > Best regards > Christoph > > > -----Original Message----- > > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > > Behalf Of Bernd > > Sent: Dienstag, 13. Februar 2018 22:55 > > To: OpenJDK Dev list > > Subject: Re: Oracle Java 8u161 regression in XML Schema Factory > > > > Hello, > > > > > > 2018-01-25 17:41 GMT+01:00 Se?n Coffey : > > > > > > > > Classes nearer to those below were touched via JDK-8186080: Transform > > XML > > > interfaces > > > http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 > > > http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 > > > > > > This may be connected with some tools trying to redefine the classes > > > perhaps. Needs more investigating. Perhaps the XMLSchemaLoader > > changes are > > > a factor ? > > > > > > > I have ben able to extract a minimal reproducer. It is not related to > > XMLUnit, only to powermock. If it instruments com.sun but not javax.xml > > (and other combinations) then it fails. > > > > For details see the readme in this maven project: > > > > https://github.com/ecki/reproduce-schemaex > > > > I also found a way to make it work with both versions, so its no longer an > > issue for me, but there is definitely some changes (which might also be > > triggered in AppServers or OSGi containers with partially reconfigured > > implementations. Not sure if you want to investigate deeper). > > > > Gruss > > Bernd > > > > > > Here is the stacktrace anyway: > > >> > > >> com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: > Schema > > >> factory class > > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > > does > > >> not > > >> extend from SchemaDVFactory. > > >> at > > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. > > >> getInstance(SchemaDVFactory.java:75) > > >> at > > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. > > >> getInstance(SchemaDVFactory.java:57) > > >> at > > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > > >> reset(XMLSchemaLoader.java:1024) > > >> at > > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > > >> loadGrammar(XMLSchemaLoader.java:556) > > >> at > > >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > > >> loadGrammar(XMLSchemaLoader.java:535) > > >> at > > >> com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchema > > >> Factory.newSchema(XMLSchemaFactory.java:254) > > >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. > > >> java:638) > > >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. > > >> java:654) > > >> at > > >> com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchema > > >> Validator(BuilderTestHelper.java:57) > > >> at > > >> com.seeburger.api.test.helpers.BuilderTestHelper.validateAnd > > >> Compare(BuilderTestHelper.java:73) > > >> at > > >> com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuild > > >> erTest.java:196) > > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > >> at > > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce > > >> ssorImpl.java:62) > > >> at > > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe > > >> thodAccessorImpl.java:43) > > >> at java.lang.reflect.Method.invoke(Method.java:498) > > >> at > org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > > >> > unnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod > > >> (PowerMockJUnit44RunnerDelegateImpl.java:310) > > >> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie. > > >> java:89) > > >> at > > >> org.junit.internal.runners.MethodRoadie.runBeforesThenTestTh > > >> enAfters(MethodRoadie.java:97) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > > >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(P > > >> owerMockJUnit44RunnerDelegateImpl.java:294) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R > > >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestIn > > >> Super(PowerMockJUnit47RunnerDelegateImpl.java:127) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R > > >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(P > > >> owerMockJUnit47RunnerDelegateImpl.java:82) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > > >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThe > > >> nTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) > > >> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie > > >> .java:87) > > >> at > > org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > > >> unnerDelegateImpl.invokeTestMethod(PowerMockJUni > > >> t44RunnerDelegateImpl.java:207) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > > >> > > > unnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.ja > > va:146) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > > >> > > > unnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) > > >> at > > >> org.junit.internal.runners.ClassRoadie.runUnprotected(ClassR > > >> oadie.java:34) > > >> at > > >> > org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) > > >> at > > >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > > >> > > unnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) > > >> at > > >> org.powermock.modules.junit4.common.internal.impl.JUnit4Test > > >> SuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) > > >> at > > >> org.powermock.modules.junit4.common.internal.impl.AbstractCo > > >> > > > mmonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) > > >> at > > >> > > > org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner. > > java:59) > > >> at > > >> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. > > >> run(JUnit4TestReference.java:86) > > >> at > > >> org.eclipse.jdt.internal.junit.runner.TestExecution.run( > > >> TestExecution.java:38) > > >> at > > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe > > >> sts(RemoteTestRunner.java:539) > > >> at > > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe > > >> sts(RemoteTestRunner.java:761) > > >> at > > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( > > >> RemoteTestRunner.java:461) > > >> at > > >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( > > >> RemoteTestRunner.java:207) > > >> > > >> on the classpath jaxb-impl-2.2.5.jar but the specific packages are only > > >> loaded from rt.jar and redefined. I asume the later is done by > > Powermock. > > >> > > >> Line 611: [Loaded > > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > > >> C:\Program > > >> Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > > >> Line 616: [Loaded > > >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > > from > > >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > > >> Line 617: [Loaded > > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > > from > > >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > > >> Line 618: [Loaded > > >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > > >> __JVM_DefineClass__] > > >> Line 619: [Loaded > > >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > > from > > >> __JVM_DefineClass__] > > >> Line 620: [Loaded > > >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > > from > > >> __JVM_DefineClass__] > > >> > > >> Is that something you are concerned? > > >> > > >> Gruss > > >> Bernd > > >> > > > From christoph.langer at sap.com Thu Feb 22 08:47:45 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 22 Feb 2018 08:47:45 +0000 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: <5A8DDB86.1020006@oracle.com> References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> <618b0785b9dd4e1ca47df9b448f30637@sap.com> <5A8DDB86.1020006@oracle.com> Message-ID: <799388aea3444b16985205aa08316eb5@sap.com> Hi Joe, thanks for the clarification. It would be good to have a place of documentation where one could refer customers to. Thanks Christoph > -----Original Message----- > From: Joe Wang [mailto:huizhe.wang at oracle.com] > Sent: Mittwoch, 21. Februar 2018 21:50 > To: Langer, Christoph > Cc: Bernd ; OpenJDK Dev list dev at openjdk.java.net> > Subject: Re: Oracle Java 8u161 regression in XML Schema Factory > > > > > > @Joe: Is there some documentation for this change in default behavior > that came with JDK8u161? I assume it is for higher security and cannot be > reverted (e.g. by setting the feature default for > Djdk.xml.overrideDefaultParser to true). > > It is indeed. It was a customer's request. Customers' complaints were > that a factory created through the official API could in many cases, > unknowingly to the customers, load 3rd party parsers that didn't > necessarily implement the security features added since JDK7u40 and 8. > For customers, this behavior was a security concern. It was particularly > inconvenient to users who might have some 3rd party libraries that just > happen to be in their environment. > > This change was not mentioned in the release notes. I'll check whether > we could find a right place for a note of this change. The 8u161 release > notes would have been a good place to do so. > > Best, > Joe > > > > > Best regards > > Christoph > > > >> -----Original Message----- > >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > >> Behalf Of Bernd > >> Sent: Dienstag, 13. Februar 2018 22:55 > >> To: OpenJDK Dev list > >> Subject: Re: Oracle Java 8u161 regression in XML Schema Factory > >> > >> Hello, > >> > >> > >> 2018-01-25 17:41 GMT+01:00 Se?n Coffey: > >> > >>> Classes nearer to those below were touched via JDK-8186080: Transform > >> XML > >>> interfaces > >>> http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 > >>> http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 > >>> > >>> This may be connected with some tools trying to redefine the classes > >>> perhaps. Needs more investigating. Perhaps the XMLSchemaLoader > >> changes are > >>> a factor ? > >>> > >> I have ben able to extract a minimal reproducer. It is not related to > >> XMLUnit, only to powermock. If it instruments com.sun but not javax.xml > >> (and other combinations) then it fails. > >> > >> For details see the readme in this maven project: > >> > >> https://github.com/ecki/reproduce-schemaex > >> > >> I also found a way to make it work with both versions, so its no longer an > >> issue for me, but there is definitely some changes (which might also be > >> triggered in AppServers or OSGi containers with partially reconfigured > >> implementations. Not sure if you want to investigate deeper). > >> > >> Gruss > >> Bernd > >> > >> > >> Here is the stacktrace anyway: > >>>> com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: > Schema > >>>> factory class > >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > >> does > >>>> not > >>>> extend from SchemaDVFactory. > >>>> at > >>>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. > >>>> getInstance(SchemaDVFactory.java:75) > >>>> at > >>>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. > >>>> getInstance(SchemaDVFactory.java:57) > >>>> at > >>>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >>>> reset(XMLSchemaLoader.java:1024) > >>>> at > >>>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >>>> loadGrammar(XMLSchemaLoader.java:556) > >>>> at > >>>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. > >>>> loadGrammar(XMLSchemaLoader.java:535) > >>>> at > >>>> com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchema > >>>> Factory.newSchema(XMLSchemaFactory.java:254) > >>>> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. > >>>> java:638) > >>>> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. > >>>> java:654) > >>>> at > >>>> com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchema > >>>> Validator(BuilderTestHelper.java:57) > >>>> at > >>>> com.seeburger.api.test.helpers.BuilderTestHelper.validateAnd > >>>> Compare(BuilderTestHelper.java:73) > >>>> at > >>>> com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuild > >>>> erTest.java:196) > >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>>> at > >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce > >>>> ssorImpl.java:62) > >>>> at > >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe > >>>> thodAccessorImpl.java:43) > >>>> at java.lang.reflect.Method.invoke(Method.java:498) > >>>> at > org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >>>> > unnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod > >>>> (PowerMockJUnit44RunnerDelegateImpl.java:310) > >>>> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie. > >>>> java:89) > >>>> at > >>>> org.junit.internal.runners.MethodRoadie.runBeforesThenTestTh > >>>> enAfters(MethodRoadie.java:97) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >>>> unnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(P > >>>> owerMockJUnit44RunnerDelegateImpl.java:294) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R > >>>> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestIn > >>>> Super(PowerMockJUnit47RunnerDelegateImpl.java:127) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R > >>>> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(P > >>>> owerMockJUnit47RunnerDelegateImpl.java:82) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >>>> > unnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThe > >>>> nTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) > >>>> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie > >>>> .java:87) > >>>> at > >> org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >>>> unnerDelegateImpl.invokeTestMethod(PowerMockJUni > >>>> t44RunnerDelegateImpl.java:207) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >>>> > >> > unnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.ja > >> va:146) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >>>> > >> > unnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) > >>>> at > >>>> org.junit.internal.runners.ClassRoadie.runUnprotected(ClassR > >>>> oadie.java:34) > >>>> at > >>>> > org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) > >>>> at > >>>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R > >>>> > >> > unnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) > >>>> at > >>>> org.powermock.modules.junit4.common.internal.impl.JUnit4Test > >>>> SuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) > >>>> at > >>>> org.powermock.modules.junit4.common.internal.impl.AbstractCo > >>>> > >> > mmonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) > >>>> at > >>>> > >> > org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner. > >> java:59) > >>>> at > >>>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. > >>>> run(JUnit4TestReference.java:86) > >>>> at > >>>> org.eclipse.jdt.internal.junit.runner.TestExecution.run( > >>>> TestExecution.java:38) > >>>> at > >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe > >>>> sts(RemoteTestRunner.java:539) > >>>> at > >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe > >>>> sts(RemoteTestRunner.java:761) > >>>> at > >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( > >>>> RemoteTestRunner.java:461) > >>>> at > >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( > >>>> RemoteTestRunner.java:207) > >>>> > >>>> on the classpath jaxb-impl-2.2.5.jar but the specific packages are only > >>>> loaded from rt.jar and redefined. I asume the later is done by > >> Powermock. > >>>> Line 611: [Loaded > >>>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > >>>> C:\Program > >>>> Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >>>> Line 616: [Loaded > >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > >> from > >>>> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >>>> Line 617: [Loaded > >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > >> from > >>>> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > >>>> Line 618: [Loaded > >>>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > >>>> __JVM_DefineClass__] > >>>> Line 619: [Loaded > >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > >> from > >>>> __JVM_DefineClass__] > >>>> Line 620: [Loaded > >>>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > >> from > >>>> __JVM_DefineClass__] > >>>> > >>>> Is that something you are concerned? > >>>> > >>>> Gruss > >>>> Bernd > >>>> From thomas.stuefe at gmail.com Thu Feb 22 08:54:16 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 22 Feb 2018 09:54:16 +0100 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) In-Reply-To: <58519ccf88ac4517b6186af613e385fd@sap.com> References: <58519ccf88ac4517b6186af613e385fd@sap.com> Message-ID: Hi Christoph, Looks fine. .. Thomas On Feb 22, 2018 09:42, "Langer, Christoph" wrote: > Hi, > > please review a simple import cleanup fix for > java/util/jar/Attributes.java and sun/management/jdp/JdpController.java. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8198539 > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8198539.0/ > > Thanks and best regards > Christoph > > From andrew_m_leonard at uk.ibm.com Thu Feb 22 09:03:17 2018 From: andrew_m_leonard at uk.ibm.com (Andrew Leonard) Date: Thu, 22 Feb 2018 09:03:17 +0000 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) In-Reply-To: <58519ccf88ac4517b6186af613e385fd@sap.com> References: <58519ccf88ac4517b6186af613e385fd@sap.com> Message-ID: looks good Cheers Andrew Leonard Java Runtimes Development IBM Hursley IBM United Kingdom Ltd Phone internal: 245913, external: 01962 815913 internet email: andrew_m_leonard at uk.ibm.com From: "Langer, Christoph" To: OpenJDK Dev list , "serviceability-dev at openjdk.java.net" Cc: "andrew_m_leonard at uk.ibm.com" Date: 22/02/2018 08:42 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) Hi, please review a simple import cleanup fix for java/util/jar/Attributes.java and sun/management/jdp/JdpController.java. Bug: https://bugs.openjdk.java.net/browse/JDK-8198539 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8198539.0/ Thanks and best regards Christoph Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From nishit.jain at oracle.com Thu Feb 22 11:51:13 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Thu, 22 Feb 2018 17:21:13 +0530 Subject: [11] RFR JDK-8060094: java/util/Formatter/Basic.java failed in tr locale Message-ID: Hi, Please review the fix for JDK-8060094. Bug: https://bugs.openjdk.java.net/browse/JDK-8060094 Webrev: http://cr.openjdk.java.net/~nishjain/8060094/webrev.03/ CSR: https://bugs.openjdk.java.net/browse/JDK-8197916 Cause: The Formatter APIs were not using the correct locale for upper casing when specified during instance creation or during format() call. The default locale was getting used irrespective of whether a Locale is specified in the API. Fix: Modified the APIs to use the locale specified. Regards, Nishit Jain From Alan.Bateman at oracle.com Thu Feb 22 13:33:30 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 22 Feb 2018 13:33:30 +0000 Subject: Draft JEP: To use UTF-8 as the default charset for the Java virtual machine. In-Reply-To: <00e001d3ab55$9282c590$b78850b0$@apache.org> References: <5A8D125A.5030205@oracle.com> <001301d3aaf1$827285a0$875790e0$@apache.org> <00e001d3ab55$9282c590$b78850b0$@apache.org> Message-ID: <615f0760-cd7b-e6b5-b261-046ee0ab877e@oracle.com> On 21/02/2018 20:50, Uwe Schindler wrote: > : > Thanks for clarifying! I just wanted to mention this, because those methods are different, so you should at least think about it ?? > These methods were deliberately specified to use UTF-8 and I don't think we should change them (changing them for a release or two would cause needless breakage of course). -Alan From Alan.Bateman at oracle.com Thu Feb 22 13:34:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 22 Feb 2018 13:34:39 +0000 Subject: [11] RFR: JDK-8198385: Remove property sun.locale.formatasdefault In-Reply-To: <81803e97-2eb9-7efe-4c32-5be1e92b82f8@oracle.com> References: <81803e97-2eb9-7efe-4c32-5be1e92b82f8@oracle.com> Message-ID: <071b4dc7-0ea6-2460-9a3f-57ed18d01b9b@oracle.com> On 21/02/2018 21:13, naoto.sato at oracle.com wrote: > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8198385 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8198385/webrev.00/ > > The property was introduced in JDK7 for the backward compatibility, > which at this point is no longer needed. Corresponding CSR is already > approved. This looks okay. Are there are tests to be adjusted/removed as part of this? -Alan From leventov.ru at gmail.com Thu Feb 22 16:51:30 2018 From: leventov.ru at gmail.com (Roman Leventov) Date: Thu, 22 Feb 2018 17:51:30 +0100 Subject: [PATCH] Optimization of AbstractStringBuilder.ensureCapacityInternal() Message-ID: AbstractStringBuilder.ensureCapacityInternal() doesn't need to copy the whole underlying array, only the part until the current count. diff --git a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java @@ -140,11 +140,12 @@ * overflow, this method throws {@code OutOfMemoryError}. */ private void ensureCapacityInternal(int minimumCapacity) { + byte[] oldValue = value; // overflow-conscious code - int oldCapacity = value.length >> coder; + int oldCapacity = oldValue.length >> coder; if (minimumCapacity - oldCapacity > 0) { - value = Arrays.copyOf(value, - newCapacity(minimumCapacity) << coder); + value = new byte[newCapacity(minimumCapacity) << coder]; + System.arraycopy(oldValue, 0, value, 0, count << coder); } } From naoto.sato at oracle.com Thu Feb 22 17:12:31 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 22 Feb 2018 09:12:31 -0800 Subject: [11] RFR: JDK-8198385: Remove property sun.locale.formatasdefault In-Reply-To: <071b4dc7-0ea6-2460-9a3f-57ed18d01b9b@oracle.com> References: <81803e97-2eb9-7efe-4c32-5be1e92b82f8@oracle.com> <071b4dc7-0ea6-2460-9a3f-57ed18d01b9b@oracle.com> Message-ID: Thanks, Alan. There is no test case affected by this change, so I added "noreg" label to the jira issue. Naoto On 2/22/18 5:34 AM, Alan Bateman wrote: > On 21/02/2018 21:13, naoto.sato at oracle.com wrote: >> Hello, >> >> Please review the fix to the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8198385 >> >> The proposed changeset is located at: >> >> http://cr.openjdk.java.net/~naoto/8198385/webrev.00/ >> >> The property was introduced in JDK7 for the backward compatibility, >> which at this point is no longer needed. Corresponding CSR is already >> approved. > This looks okay. Are there are tests to be adjusted/removed as part of > this? > > -Alan From leventov.ru at gmail.com Thu Feb 22 17:23:56 2018 From: leventov.ru at gmail.com (Roman Leventov) Date: Thu, 22 Feb 2018 18:23:56 +0100 Subject: [PATCH] Optimization of AbstractStringBuilder.ensureCapacityInternal() In-Reply-To: References: Message-ID: Similar optimizations are also possible in ArrayList and Vector. On 22 February 2018 at 17:51, Roman Leventov wrote: > AbstractStringBuilder.ensureCapacityInternal() doesn't need to copy the > whole underlying array, only the part until the current count. > > diff --git a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > @@ -140,11 +140,12 @@ > * overflow, this method throws {@code OutOfMemoryError}. > */ > private void ensureCapacityInternal(int minimumCapacity) { > + byte[] oldValue = value; > // overflow-conscious code > - int oldCapacity = value.length >> coder; > + int oldCapacity = oldValue.length >> coder; > if (minimumCapacity - oldCapacity > 0) { > - value = Arrays.copyOf(value, > - newCapacity(minimumCapacity) << coder); > + value = new byte[newCapacity(minimumCapacity) << coder]; > + System.arraycopy(oldValue, 0, value, 0, count << coder); > } > } > From claes.redestad at oracle.com Thu Feb 22 17:35:18 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 22 Feb 2018 18:35:18 +0100 Subject: [PATCH] Optimization of AbstractStringBuilder.ensureCapacityInternal() In-Reply-To: References: Message-ID: <1eeb46c7-1e02-1944-9620-ccebc104c071@oracle.com> Hi, interesting - do you have any numbers showing a benefit from doing this (on various sets of input)? My concerns are that count might typically be close enough to value.length to make the difference small in practice, and that there are some (fragile) JIT optimizations to try and avoid zeroing out the newly allocated array that we have to take care not to break. /Claes On 2018-02-22 17:51, Roman Leventov wrote: > AbstractStringBuilder.ensureCapacityInternal() doesn't need to copy the > whole underlying array, only the part until the current count. > > diff --git > a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java > @@ -140,11 +140,12 @@ > * overflow, this method throws {@code OutOfMemoryError}. > */ > private void ensureCapacityInternal(int minimumCapacity) { > + byte[] oldValue = value; > // overflow-conscious code > - int oldCapacity = value.length >> coder; > + int oldCapacity = oldValue.length >> coder; > if (minimumCapacity - oldCapacity > 0) { > - value = Arrays.copyOf(value, > - newCapacity(minimumCapacity) << coder); > + value = new byte[newCapacity(minimumCapacity) << coder]; > + System.arraycopy(oldValue, 0, value, 0, count << coder); > } > } From cushon at google.com Thu Feb 22 18:48:25 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Thu, 22 Feb 2018 10:48:25 -0800 Subject: RFR: 8198526: getAnnotatedOwnerType does not handle static nested classes correctly Message-ID: (replying to edit the subject) On Wed, Feb 21, 2018 at 2:18 PM, Liam Miller-Cushon wrote: > Hello, > > Please review this fix to the handling of static nested classes > in getAnnotatedOwnerType. > > webrev: http://cr.openjdk.java.net/~cushon/8198526/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8198526 > From naoto.sato at oracle.com Thu Feb 22 19:26:59 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 22 Feb 2018 11:26:59 -0800 Subject: [11] RFR JDK-8060094: java/util/Formatter/Basic.java failed in tr locale In-Reply-To: References: Message-ID: <8118448f-b842-b802-3413-12e429878bd5@oracle.com> Hi Nishit, In the test case, the exception error messages may be more helpful if there is a distinction between the two (line 103 and 120) mentioning the formatter is using the default or specified locale. Naoto On 2/22/18 3:51 AM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8060094. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8060094 > Webrev: http://cr.openjdk.java.net/~nishjain/8060094/webrev.03/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8197916 > > Cause: The Formatter APIs were not using the correct locale for upper > casing when specified during instance creation or during format() call. > The default locale was getting used irrespective of whether a Locale is > specified in the API. > Fix: Modified the APIs to use the locale specified. > > Regards, > Nishit Jain From huizhe.wang at oracle.com Thu Feb 22 20:04:43 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 22 Feb 2018 12:04:43 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A8DCF9E.50508@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> <5A8DC7A9.30104@oracle.com> <5A8DCF9E.50508@oracle.com> Message-ID: <124fb31a-e508-0a01-98c7-dff9b345b3e6@oracle.com> Hi Sherman, Thanks for reviewing the change. Taking a local copy of the count field, but the boundary check would be almost immediately done against the field itself. Are you worrying about the count field may be out of sync with the byte array? I would think that's unlikely to happen. Whether it's StringBuilder or StringBuffer, it's not advisable/practical to use in multiple threads. In a valid usage, the count is always consistent with the byte array. Best, Joe On 2/21/2018 11:59 AM, Xueming Shen wrote: > Joe, > > AbstractStringBuilder is a mutable class. You might need to take a > local copy > and do the explicit boundary check before going into the > StringLatin1/UTF16, > especially StringUTF16. Some intrinsics, UTF16.getChar(byte[], int) > for example, > assume the caller performs bounds check (for performance reason). > > -Sherman > > On 2/21/18, 11:25 AM, Joe Wang wrote: >> Hi Stuart, >> >> Thanks for the apiNote! Joe re-approved the CSR with the added apiNote. >> >> For a test that compares Latin1 vs UTF16 coders, I added tests to the >> compact string's CompareTo test[1], basically running the original >> test for String with StringBuilder/Buffer. A build with jdk_core >> tests passed. >> >> Here's the updated webrev: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> [1] >> http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/test/jdk/java/lang/String/CompactString/CompareTo.java.sdiff.html >> >> Best, >> Joe >> >> On 2/14/18, 5:09 PM, Stuart Marks wrote: >>> Hi Joe, >>> >>> Overall, looks good. >>> >>> Are there any tests of AbstractStringBuilder.compareTo() that >>> exercise comparisons of the Latin1 vs UTF16 coders? >>> >>> A couple people have raised the issue of the SB's implementing >>> Comparable but not overriding equals(). This is unusual but >>> well-defined. I do think it deserves the "inconsistent with equals" >>> treatment. Something like the following should be added to both SB's: >>> >>> ========== >>> @apiNote >>> {@code StringBuilder} implements {@code Comparable} but does not >>> override {@link Object#equals equals}. Thus, the natural ordering of >>> {@code StringBuilder} is inconsistent with equals. Care should be >>> exercised if {@code StringBuilder} objects are used as keys in a >>> {@code SortedMap} or elements in a {@code SortedSet}. See {@link >>> Comparable}, {@link java.util.SortedMap SortedMap}, or {@link >>> java.util.SortedSet SortedSet} for more information. >>> ========== >>> >>> Respectively for StringBuffer. (Adjust markup to taste.) >>> >>> Thanks, >>> >>> s'marks >>> >>> >>> >>> >>> On 2/8/18 4:47 PM, Joe Wang wrote: >>>> Hi all, >>>> >>>> The CSR for the enhancement is now approved. Thanks Joe! >>>> >>>> The webrev has been updated accordingly. Please let me know if you >>>> have any further comment on the implementation. >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>> >>>> Thanks, >>>> Joe >>>> >>>> On 2/2/2018 12:46 PM, Joe Wang wrote: >>>>> Thanks Jason. Will update that accordingly. >>>>> >>>>> Best, >>>>> Joe >>>>> >>>>> On 2/2/2018 11:22 AM, Jason Mehrens wrote: >>>>>> Joe, >>>>>> >>>>>> The identity check in CS.compare makes sense.? However, it won't >>>>>> be null hostile if we call CS.compare(null, null) and that >>>>>> doesn't seem right. >>>>>> Usually when writing comparator classes I end up with: >>>>>> === >>>>>> if (Objects.requireNonNull(o1) == Objects.requireNonNull(o2)) { >>>>>> ???? return 0; >>>>>> } >>>>>> === >>>>>> >>>>>> Jason >>>>>> ________________________________________ >>>>>> From: core-libs-dev on >>>>>> behalf of Joe Wang >>>>>> Sent: Friday, February 2, 2018 1:01 PM >>>>>> To: core-libs-dev >>>>>> Subject: Re: RFR (JDK11) 8137326: Methods for comparing >>>>>> CharSequence, StringBuilder, and StringBuffer >>>>>> >>>>>> Hi, >>>>>> >>>>>> Thanks all for comments and suggestions. I've updated the webrev. >>>>>> Please >>>>>> review. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>> >>>>>> Thanks, >>>>>> Joe >>>>>> >>>>>> On 1/31/2018 9:31 PM, Joe Wang wrote: >>>>>>> Hi Tagir, >>>>>>> >>>>>>> Thanks for the comment. I will consider adding that to the javadoc >>>>>>> emphasizing that the comparison is performed from 0 to length() >>>>>>> - 1 of >>>>>>> the two sequences. >>>>>>> >>>>>>> Best, >>>>>>> Joe >>>>>>> >>>>>>> On 1/29/18, 8:07 PM, Tagir Valeev wrote: >>>>>>>> Hello! >>>>>>>> >>>>>>>> An AbstractStringBuilder#compareTo implementation is wrong. You >>>>>>>> cannot >>>>>>>> simply compare the whole byte array. Here's the test-case: >>>>>>>> >>>>>>>> public class Test { >>>>>>>> ??? public static void main(String[] args) { >>>>>>>> ????? StringBuilder sb1 = new StringBuilder("test1"); >>>>>>>> ????? StringBuilder sb2 = new StringBuilder("test2"); >>>>>>>> ????? sb1.setLength(4); >>>>>>>> ????? sb2.setLength(4); >>>>>>>> ????? System.out.println(sb1.compareTo(sb2)); >>>>>>>> System.out.println(sb1.toString().compareTo(sb2.toString())); >>>>>>>> ??? } >>>>>>>> } >>>>>>>> >>>>>>>> We truncated the stringbuilders making their content equal, so >>>>>>>> sb1.toString().compareTo(sb2.toString()) is 0, but compareTo >>>>>>>> compares >>>>>>>> the original content (before the truncation) as truncation, of >>>>>>>> course, >>>>>>>> does not zero the truncated bytes, neither does it reallocate the >>>>>>>> array (unless explicitly asked via trimToSize). >>>>>>>> >>>>>>>> With best regards, >>>>>>>> Tagir Valeev. >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang >>>>>>>> wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Adding methods for comparing CharSequence, StringBuilder, and >>>>>>>>> StringBuffer. >>>>>>>>> >>>>>>>>> The Comparable implementations for StringBuilder/Buffer are >>>>>>>>> similar >>>>>>>>> to that >>>>>>>>> of String, allowing comparison operations between two >>>>>>>>> StringBuilders/Buffers, e.g. >>>>>>>>> aStringBuilder.compareTo(anotherStringBuilder). >>>>>>>>> For CharSequence however, refer to the comments in JIRA, a static >>>>>>>>> method >>>>>>>>> 'compare' is added instead of implementing the Comparable >>>>>>>>> interface. >>>>>>>>> This >>>>>>>>> 'compare' method may take CharSequence implementations such as >>>>>>>>> String, >>>>>>>>> StringBuilder and StringBuffer, making it possible to perform >>>>>>>>> comparison >>>>>>>>> among them. The previous example for example is equivalent to >>>>>>>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>>>>>>> >>>>>>>>> Tests for java.base have been independent from each other. The >>>>>>>>> new >>>>>>>>> tests are >>>>>>>>> therefore created to have no dependency on each other or >>>>>>>>> sharing any >>>>>>>>> code. >>>>>>>>> >>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Joe >>>>> >>>> > From david.lloyd at redhat.com Thu Feb 22 20:08:08 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Thu, 22 Feb 2018 14:08:08 -0600 Subject: [JDK-6341887] Patch V2: java.util.zip: Add ByteBuffer methods to Inflater/Deflater Message-ID: This is the second version of the patch first discussed here [1] to add the ability to inflate/deflate to/from heap/direct byte buffers. The patch is attached, as well as benchmark results with the original and new code, and the benchmark source. The patch is also visible online at [2]. Some notable features of this version: ? Complete JavaDoc per Alan's reminder ? Misc. small bug fixes ? Removed nearly all JNI field accesses, replaced with simple bit-fields on input and output parameters to inflate* and deflate* native methods ? Fixed a problem Alan alluded to where an inflate operation can throw DataFormatException yet still have consumed some input data; this is done by way of a single field-based output parameter which is only used when this exception is thrown ? Cleaned up "finish" handling on the deflate side ? Implemented Sherman's suggestion to support both array-based and buffer-based input; no ByteBuffer wrapping is used in this version ? Updated FlaterTest to test all combinations of inflate/deflate array/heap/direct ? Per-call performance seems generally improved overall; from the JMH results: ? JDK version completed 95% of 16-byte inflate ops within 1262 ns/op (array in, array out) ? New version completes 95% of 16-byte inflate ops within 1124 ns/op (array in, array out) ? In addition, the new direct buffer modes show slightly better results at 1114 ns/op for 16-byte ops for direct buffer in & out - not accounting for possible secondary improvements resulting from avoiding Get*Critical JNI functions (as well as the obvious: avoiding the need to allocate & copy between byte arrays and direct buffers) The JMH test methods are named based on the type of input and output done for each test, where "A" signifies "array", "H" signifies "heap buffer", and "D" signifies "direct buffer". [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-February/051562.html [2] https://github.com/dmlloyd/openjdk/commit/zlib-bytebuffer-v6 -- - DML -------------- next part -------------- commit 04a83977c5b70d1a9d5850b05fc8b585356b6c2a Author: David M. Lloyd Date: Fri Feb 16 11:00:10 2018 -0600 [JDK-6341887] Update Inflater/Deflater to handle ByteBuffer diff --git a/make/mapfiles/libzip/mapfile-vers b/make/mapfiles/libzip/mapfile-vers index d711d8e17f4..11ccc2d6ecb 100644 --- a/make/mapfiles/libzip/mapfile-vers +++ b/make/mapfiles/libzip/mapfile-vers @@ -33,20 +33,28 @@ SUNWprivate_1.1 { Java_java_util_zip_CRC32_update; Java_java_util_zip_CRC32_updateBytes0; Java_java_util_zip_CRC32_updateByteBuffer0; - Java_java_util_zip_Deflater_deflateBytes; + Java_java_util_zip_Deflater_deflateBytesBytes; + Java_java_util_zip_Deflater_deflateBytesBuffer; + Java_java_util_zip_Deflater_deflateBufferBytes; + Java_java_util_zip_Deflater_deflateBufferBuffer; Java_java_util_zip_Deflater_end; Java_java_util_zip_Deflater_getAdler; Java_java_util_zip_Deflater_init; Java_java_util_zip_Deflater_initIDs; Java_java_util_zip_Deflater_reset; Java_java_util_zip_Deflater_setDictionary; + Java_java_util_zip_Deflater_setDictionaryBuffer; Java_java_util_zip_Inflater_end; Java_java_util_zip_Inflater_getAdler; - Java_java_util_zip_Inflater_inflateBytes; + Java_java_util_zip_Inflater_inflateBytesBytes; + Java_java_util_zip_Inflater_inflateBytesBuffer; + Java_java_util_zip_Inflater_inflateBufferBytes; + Java_java_util_zip_Inflater_inflateBufferBuffer; Java_java_util_zip_Inflater_init; Java_java_util_zip_Inflater_initIDs; Java_java_util_zip_Inflater_reset; Java_java_util_zip_Inflater_setDictionary; + Java_java_util_zip_Inflater_setDictionaryBuffer; ZIP_Close; ZIP_CRC32; ZIP_FreeEntry; diff --git a/src/java.base/share/classes/java/util/zip/Deflater.java b/src/java.base/share/classes/java/util/zip/Deflater.java index c75dd4a33f0..36e08a1dfc7 100644 --- a/src/java.base/share/classes/java/util/zip/Deflater.java +++ b/src/java.base/share/classes/java/util/zip/Deflater.java @@ -26,7 +26,12 @@ package java.util.zip; import java.lang.ref.Cleaner.Cleanable; +import java.nio.ByteBuffer; +import java.nio.ReadOnlyBufferException; +import java.util.Objects; + import jdk.internal.ref.CleanerFactory; +import sun.nio.ch.DirectBuffer; /** * This class provides support for general purpose compression using the @@ -92,8 +97,9 @@ import jdk.internal.ref.CleanerFactory; public class Deflater { private final DeflaterZStreamRef zsRef; - private byte[] buf = new byte[0]; - private int off, len; + private ByteBuffer input = ZipUtils.defaultBuf; + private byte[] inputArray; + private int inputPos, inputLim; private int level, strategy; private boolean setParams; private boolean finish, finished; @@ -170,9 +176,14 @@ public class Deflater { */ public static final int FULL_FLUSH = 3; + /** + * Flush mode to use at the end of output. Can only be provided by the + * user by way of {@link #finish()}. + */ + private static final int FINISH = 4; + static { ZipUtils.loadLibrary(); - initIDs(); } /** @@ -216,16 +227,14 @@ public class Deflater { * @see Deflater#needsInput */ public void setInput(byte[] b, int off, int len) { - if (b== null) { - throw new NullPointerException(); - } if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } synchronized (zsRef) { - this.buf = b; - this.off = off; - this.len = len; + this.input = null; + this.inputArray = b; + this.inputPos = off; + this.inputLim = off + len; } } @@ -239,6 +248,31 @@ public class Deflater { setInput(b, 0, b.length); } + /** + * Sets input data for compression. This should be called whenever + * needsInput() returns true indicating that more input data is required. + *

+ * The given buffer's position will be updated as deflate operations are + * performed. The input buffer may be modified (refilled) between deflate + * operations; doing so is equivalent to creating a new buffer and setting + * it with this method. + *

+ * Modifying the input buffer's contents, position, or limit concurrently with + * a deflate operation will result in undefined behavior, which may include + * incorrect operation results or operation failure. + * + * @param byteBuffer the input data bytes + * @see Deflater#needsInput + * @since 11 + */ + public void setInput(ByteBuffer byteBuffer) { + Objects.requireNonNull(byteBuffer); + synchronized (zsRef) { + this.input = byteBuffer; + this.inputArray = null; + } + } + /** * Sets preset dictionary for compression. A preset dictionary is used * when the history buffer can be predetermined. When the data is later @@ -252,9 +286,6 @@ public class Deflater { * @see Inflater#getAdler */ public void setDictionary(byte[] b, int off, int len) { - if (b == null) { - throw new NullPointerException(); - } if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } @@ -278,6 +309,37 @@ public class Deflater { setDictionary(b, 0, b.length); } + /** + * Sets preset dictionary for compression. A preset dictionary is used + * when the history buffer can be predetermined. When the data is later + * uncompressed with Inflater.inflate(), Inflater.getAdler() can be called + * in order to get the Adler-32 value of the dictionary required for + * decompression. + *

+ * The bytes in given byte buffer will be fully consumed by this method. On + * return, its position will equal its limit. + * + * @param byteBuffer the dictionary data bytes + * @see Inflater#inflate + * @see Inflater#getAdler + */ + public void setDictionary(ByteBuffer byteBuffer) { + synchronized (zsRef) { + final int position = byteBuffer.position(); + final int remaining = Math.max(byteBuffer.limit() - position, 0); + ensureOpen(); + if (byteBuffer.isDirect()) { + final long address = ((DirectBuffer) byteBuffer).address(); + setDictionaryBuffer(zsRef.address(), address + position, remaining); + } else { + final byte[] array = ZipUtils.getBufferArray(byteBuffer); + final int offset = ZipUtils.getBufferOffset(byteBuffer); + setDictionary(zsRef.address(), array, offset + position, remaining); + } + byteBuffer.position(position + remaining); + } + } + /** * Sets the compression strategy to the specified value. * @@ -331,14 +393,17 @@ public class Deflater { } /** - * Returns true if the input data buffer is empty and setInput() - * should be called in order to provide more input. + * Returns true if no data remains in the input buffer. This can + * be used to determine if one of the {@code setInput()} methods should be + * called in order to provide more input. + * * @return true if the input data buffer is empty and setInput() * should be called in order to provide more input */ public boolean needsInput() { synchronized (zsRef) { - return len <= 0; + final ByteBuffer input = this.input; + return input == null ? inputLim == inputPos : ! input.hasRemaining(); } } @@ -404,6 +469,26 @@ public class Deflater { return deflate(b, 0, b.length, NO_FLUSH); } + /** + * Compresses the input data and fills specified buffer with compressed + * data. Returns actual number of bytes of compressed data. A return value + * of 0 indicates that {@link #needsInput() needsInput} should be called + * in order to determine if more input data is required. + * + *

This method uses {@link #NO_FLUSH} as its compression flush mode. + * An invocation of this method of the form {@code deflater.deflate(b)} + * yields the same result as the invocation of + * {@code deflater.deflate(b, 0, b.length, Deflater.NO_FLUSH)}. + * + * @param output the buffer for the compressed data + * @return the actual number of bytes of compressed data written to the + * output buffer + * @since 11 + */ + public int deflate(ByteBuffer output) { + return deflate(output, NO_FLUSH); + } + /** * Compresses the input data and fills the specified buffer with compressed * data. Returns actual number of bytes of data compressed. @@ -441,6 +526,10 @@ public class Deflater { * repeatedly output to the output buffer every time this method is * invoked. * + *

If the {@link #setInput(ByteBuffer)} method was called to provide a buffer + * for input, the input buffer's position will be advanced by the number of bytes + * consumed by this operation. + * * @param b the buffer for the compressed data * @param off the start offset of the data * @param len the maximum number of bytes of compressed data @@ -452,24 +541,227 @@ public class Deflater { * @since 1.7 */ public int deflate(byte[] b, int off, int len, int flush) { - if (b == null) { - throw new NullPointerException(); - } if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } + if (flush != NO_FLUSH && flush != SYNC_FLUSH && flush != FULL_FLUSH) { + throw new IllegalArgumentException(); + } synchronized (zsRef) { ensureOpen(); - if (flush == NO_FLUSH || flush == SYNC_FLUSH || - flush == FULL_FLUSH) { - int thisLen = this.len; - int n = deflateBytes(zsRef.address(), b, off, len, flush); - bytesWritten += n; - bytesRead += (thisLen - this.len); - return n; + + final ByteBuffer input = this.input; + if (finish) { + // disregard given flush mode in this case + flush = FINISH; + } + final int params; + if (setParams) { + // bit 0: true to set params + // bit 1-2: strategy (0, 1, or 2) + // bit 3-31: level (0..9 or -1) + params = 1 | strategy << 1 | level << 3; + } else { + params = 0; + } + final int inputPos; + final long result; + if (input == null) { + inputPos = this.inputPos; + result = deflateBytesBytes(zsRef.address(), + inputArray, inputPos, inputLim - inputPos, + b, off, len, + flush, params); + } else { + inputPos = input.position(); + final int inputRem = Math.max(input.limit() - inputPos, 0); + if (input.isDirect()) { + final long inputAddress = ((DirectBuffer) input).address(); + result = deflateBufferBytes(zsRef.address(), + inputAddress + inputPos, inputRem, + b, off, len, + flush, params); + } else { + final byte[] inputArray = ZipUtils.getBufferArray(input); + final int inputOffset = ZipUtils.getBufferOffset(input); + result = deflateBytesBytes(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + b, off, len, + flush, params); + } + } + int read = (int) (result & 0x7fff_ffffL); + int written = (int) (result >>> 31 & 0x7fff_ffffL); + if ((result >>> 62 & 1) != 0) { + finished = true; + } + if (params != 0 && (result >>> 63 & 1) == 0) { + setParams = false; + } + if (input != null) { + input.position(inputPos + read); + } else { + this.inputPos = inputPos + read; } + bytesWritten += written; + bytesRead += read; + return written; + } + } + + /** + * Compresses the input data and fills the specified buffer with compressed + * data. Returns actual number of bytes of data compressed. + * + *

Compression flush mode is one of the following three modes: + * + *

    + *
  • {@link #NO_FLUSH}: allows the deflater to decide how much data + * to accumulate, before producing output, in order to achieve the best + * compression (should be used in normal use scenario). A return value + * of 0 in this flush mode indicates that {@link #needsInput()} should + * be called in order to determine if more input data is required. + * + *
  • {@link #SYNC_FLUSH}: all pending output in the deflater is flushed, + * to the specified output buffer, so that an inflater that works on + * compressed data can get all input data available so far (In particular + * the {@link #needsInput()} returns {@code true} after this invocation + * if enough output space is provided). Flushing with {@link #SYNC_FLUSH} + * may degrade compression for some compression algorithms and so it + * should be used only when necessary. + * + *
  • {@link #FULL_FLUSH}: all pending output is flushed out as with + * {@link #SYNC_FLUSH}. The compression state is reset so that the inflater + * that works on the compressed output data can restart from this point + * if previous compressed data has been damaged or if random access is + * desired. Using {@link #FULL_FLUSH} too often can seriously degrade + * compression. + *
+ * + *

In the case of {@link #FULL_FLUSH} or {@link #SYNC_FLUSH}, if + * the return value is {@code len}, the space available in output + * buffer {@code output}, this method should be invoked again with the same + * {@code flush} parameter and more output space. Make sure that + * {@code len} is greater than 6 to avoid flush marker (5 bytes) being + * repeatedly output to the output buffer every time this method is + * invoked. + * + *

On success, the position of the given {@code output} byte buffer will be + * advanced by as many bytes as were produced by the operation, which is equal + * to the number returned by this method. + * + *

If the {@link #setInput(ByteBuffer)} method was called to provide a buffer + * for input, the input buffer's position will be advanced by the number of bytes + * consumed by this operation. + * + * @param output the buffer for the compressed data + * @param flush the compression flush mode + * @return the actual number of bytes of compressed data written to + * the output buffer + * + * @throws IllegalArgumentException if the flush mode is invalid + * @since 11 + */ + public int deflate(ByteBuffer output, int flush) { + if (output.isReadOnly()) { + throw new ReadOnlyBufferException(); + } + if (flush != NO_FLUSH && flush != SYNC_FLUSH && flush != FULL_FLUSH) { throw new IllegalArgumentException(); } + synchronized (zsRef) { + ensureOpen(); + + final ByteBuffer input = this.input; + if (finish) { + // disregard given flush mode in this case + flush = FINISH; + } + final int params; + if (setParams) { + // bit 0: true to set params + // bit 1-2: strategy (0, 1, or 2) + // bit 3-31: level (0..9 or -1) + params = 1 | strategy << 1 | level << 3; + } else { + params = 0; + } + final int outputPos = output.position(); + final int outputRem = Math.max(output.limit() - outputPos, 0); + final int inputPos; + final long result; + if (input == null) { + inputPos = this.inputPos; + if (output.isDirect()) { + final long outputAddress = ((DirectBuffer) output).address(); + result = deflateBytesBuffer(zsRef.address(), + inputArray, inputPos, inputLim - inputPos, + outputAddress + outputPos, outputRem, + flush, params); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = deflateBytesBytes(zsRef.address(), + inputArray, inputPos, inputLim - inputPos, + outputArray, outputOffset + outputPos, outputRem, + flush, params); + } + } else { + inputPos = input.position(); + final int inputRem = Math.max(input.limit() - inputPos, 0); + if (input.isDirect()) { + final long inputAddress = ((DirectBuffer) input).address(); + if (output.isDirect()) { + final long outputAddress = outputPos + ((DirectBuffer) output).address(); + result = deflateBufferBuffer(zsRef.address(), + inputAddress + inputPos, inputRem, + outputAddress, outputRem, + flush, params); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = deflateBufferBytes(zsRef.address(), + inputAddress + inputPos, inputRem, + outputArray, outputOffset + outputPos, outputRem, + flush, params); + } + } else { + final byte[] inputArray = ZipUtils.getBufferArray(input); + final int inputOffset = ZipUtils.getBufferOffset(input); + if (output.isDirect()) { + final long outputAddress = ((DirectBuffer) output).address(); + result = deflateBytesBuffer(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + outputAddress + outputPos, outputRem, + flush, params); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = deflateBytesBytes(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + outputArray, outputOffset + outputPos, outputRem, + flush, params); + } + } + } + int read = (int) (result & 0x7fff_ffffL); + int written = (int) (result >>> 31 & 0x7fff_ffffL); + if ((result >>> 62 & 1) != 0) { + finished = true; + } + if (params != 0 && (result >>> 63 & 1) == 0) { + setParams = false; + } + if (input != null) { + input.position(inputPos + read); + } else { + this.inputPos = inputPos + read; + } + output.position(outputPos + written); + bytesWritten += written; + bytesRead += read; + return written; + } } /** @@ -545,7 +837,8 @@ public class Deflater { reset(zsRef.address()); finish = false; finished = false; - off = len = 0; + input = ZipUtils.defaultBuf; + inputArray = null; bytesRead = bytesWritten = 0; } } @@ -560,7 +853,7 @@ public class Deflater { public void end() { synchronized (zsRef) { zsRef.clean(); - buf = null; + input = ZipUtils.defaultBuf; } } @@ -585,11 +878,26 @@ public class Deflater { throw new NullPointerException("Deflater has been closed"); } - private static native void initIDs(); private static native long init(int level, int strategy, boolean nowrap); - private static native void setDictionary(long addr, byte[] b, int off, int len); - private native int deflateBytes(long addr, byte[] b, int off, int len, - int flush); + private static native void setDictionary(long addr, byte[] b, int off, + int len); + private static native void setDictionaryBuffer(long addr, long bufAddress, int len); + private native long deflateBytesBytes(long addr, + byte[] inputArray, int inputOff, int inputLen, + byte[] outputArray, int outputOff, int outputLen, + int flush, int params); + private native long deflateBytesBuffer(long addr, + byte[] inputArray, int inputOff, int inputLen, + long outputAddress, int outputLen, + int flush, int params); + private native long deflateBufferBytes(long addr, + long inputAddress, int inputLen, + byte[] outputArray, int outputOff, int outputLen, + int flush, int params); + private native long deflateBufferBuffer(long addr, + long inputAddress, int inputLen, + long outputAddress, int outputLen, + int flush, int params); private static native int getAdler(long addr); private static native void reset(long addr); private static native void end(long addr); diff --git a/src/java.base/share/classes/java/util/zip/Inflater.java b/src/java.base/share/classes/java/util/zip/Inflater.java index 9c6d8aa3d83..6cb76c4c44f 100644 --- a/src/java.base/share/classes/java/util/zip/Inflater.java +++ b/src/java.base/share/classes/java/util/zip/Inflater.java @@ -26,7 +26,12 @@ package java.util.zip; import java.lang.ref.Cleaner.Cleanable; +import java.nio.ByteBuffer; +import java.nio.ReadOnlyBufferException; +import java.util.Objects; + import jdk.internal.ref.CleanerFactory; +import sun.nio.ch.DirectBuffer; /** * This class provides support for general purpose decompression using the @@ -92,14 +97,19 @@ import jdk.internal.ref.CleanerFactory; public class Inflater { private final InflaterZStreamRef zsRef; - private byte[] buf = defaultBuf; - private int off, len; + private ByteBuffer input = ZipUtils.defaultBuf; + private byte[] inputArray; + private int inputPos, inputLim; private boolean finished; private boolean needDict; private long bytesRead; private long bytesWritten; - private static final byte[] defaultBuf = new byte[0]; + /** + * This field is used as an "out" parameter from JNI when a + * {@link DataFormatException} is thrown during the inflate operation. + */ + private int inputConsumed; static { ZipUtils.loadLibrary(); @@ -138,16 +148,14 @@ public class Inflater { * @see Inflater#needsInput */ public void setInput(byte[] b, int off, int len) { - if (b == null) { - throw new NullPointerException(); - } if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } synchronized (zsRef) { - this.buf = b; - this.off = off; - this.len = len; + this.input = null; + this.inputArray = b; + this.inputPos = off; + this.inputLim = off + len; } } @@ -162,6 +170,32 @@ public class Inflater { setInput(b, 0, b.length); } + /** + * Sets input data for decompression. Should be called whenever + * needsInput() returns true indicating that more input data is + * required. + *

+ * The given buffer's position will be updated as inflate operations are + * performed. The input buffer may be modified (refilled) between inflate + * operations; doing so is equivalent to creating a new buffer and setting + * it with this method. + *

+ * Modifying the input buffer's contents, position, or limit concurrently with + * an inflate operation will result in undefined behavior, which may include + * incorrect operation results or operation failure. + * + * @param byteBuffer the input data bytes + * @see Inflater#needsInput + * @since 11 + */ + public void setInput(ByteBuffer byteBuffer) { + Objects.requireNonNull(byteBuffer); + synchronized (zsRef) { + this.input = byteBuffer; + this.inputArray = null; + } + } + /** * Sets the preset dictionary to the given array of bytes. Should be * called when inflate() returns 0 and needsDictionary() returns true @@ -174,9 +208,6 @@ public class Inflater { * @see Inflater#getAdler */ public void setDictionary(byte[] b, int off, int len) { - if (b == null) { - throw new NullPointerException(); - } if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } @@ -200,6 +231,38 @@ public class Inflater { setDictionary(b, 0, b.length); } + /** + * Sets the preset dictionary to the given array of bytes. Should be + * called when inflate() returns 0 and needsDictionary() returns true + * indicating that a preset dictionary is required. The method getAdler() + * can be used to get the Adler-32 value of the dictionary needed. + *

+ * The bytes in given byte buffer will be fully consumed by this method. On + * return, its position will equal its limit. + * + * @param byteBuffer the dictionary data bytes + * @see Inflater#needsDictionary + * @see Inflater#getAdler + * @since 11 + */ + public void setDictionary(ByteBuffer byteBuffer) { + synchronized (zsRef) { + final int position = byteBuffer.position(); + final int remaining = Math.max(byteBuffer.limit() - position, 0); + ensureOpen(); + if (byteBuffer.isDirect()) { + final long address = ((DirectBuffer) byteBuffer).address(); + setDictionaryBuffer(zsRef.address(), address + position, remaining); + } else { + final byte[] array = ZipUtils.getBufferArray(byteBuffer); + final int offset = ZipUtils.getBufferOffset(byteBuffer); + setDictionary(zsRef.address(), array, offset + position, remaining); + } + byteBuffer.position(position + remaining); + needDict = false; + } + } + /** * Returns the total number of bytes remaining in the input buffer. * This can be used to find out what bytes still remain in the input @@ -208,19 +271,22 @@ public class Inflater { */ public int getRemaining() { synchronized (zsRef) { - return len; + final ByteBuffer input = this.input; + return input == null ? inputLim - inputPos : input.remaining(); } } /** * Returns true if no data remains in the input buffer. This can - * be used to determine if #setInput should be called in order - * to provide more input. + * be used to determine if one of the {@code setInput()} methods should be + * called in order to provide more input. + * * @return true if no data remains in the input buffer */ public boolean needsInput() { synchronized (zsRef) { - return len <= 0; + final ByteBuffer input = this.input; + return input == null ? inputLim == inputPos : ! input.hasRemaining(); } } @@ -254,30 +320,83 @@ public class Inflater { * determine if more input data or a preset dictionary is required. * In the latter case, getAdler() can be used to get the Adler-32 * value of the dictionary required. + *

+ * If the {@link #setInput(ByteBuffer)} method was called to provide a buffer + * for input, the input buffer's position will be advanced by the number of bytes + * consumed by this operation, even in the event that an exception is thrown. + * * @param b the buffer for the uncompressed data * @param off the start offset of the data * @param len the maximum number of uncompressed bytes * @return the actual number of uncompressed bytes - * @exception DataFormatException if the compressed data format is invalid + * @throws DataFormatException if the compressed data format is invalid * @see Inflater#needsInput * @see Inflater#needsDictionary */ public int inflate(byte[] b, int off, int len) throws DataFormatException { - if (b == null) { - throw new NullPointerException(); - } if (off < 0 || len < 0 || off > b.length - len) { throw new ArrayIndexOutOfBoundsException(); } synchronized (zsRef) { ensureOpen(); - int thisLen = this.len; - int n = inflateBytes(zsRef.address(), b, off, len); - bytesWritten += n; - bytesRead += (thisLen - this.len); - return n; + final ByteBuffer input = this.input; + final long result; + final int inputPos; + if (input == null) { + inputPos = this.inputPos; + try { + result = inflateBytesBytes(zsRef.address(), + inputArray, inputPos, inputLim - inputPos, + b, off, len); + } catch (DataFormatException e) { + int read = inputConsumed; + this.inputPos = inputPos + read; + bytesRead += read; + inputConsumed = 0; + throw e; + } + } else { + inputPos = input.position(); + try { + final int inputRem = Math.max(input.limit() - inputPos, 0); + if (input.isDirect()) { + final long inputAddress = ((DirectBuffer) input).address(); + result = inflateBufferBytes(zsRef.address(), + inputAddress + inputPos, inputRem, + b, off, len); + } else { + final byte[] inputArray = ZipUtils.getBufferArray(input); + final int inputOffset = ZipUtils.getBufferOffset(input); + result = inflateBytesBytes(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + b, off, len); + } + } catch (DataFormatException e) { + int read = inputConsumed; + input.position(inputPos + read); + bytesRead += read; + inputConsumed = 0; + throw e; + } + } + int read = (int) (result & 0x7fff_ffffL); + int written = (int) (result >>> 31 & 0x7fff_ffffL); + if ((result >>> 62 & 1) != 0) { + finished = true; + } + if ((result >>> 63 & 1) != 0) { + needDict = true; + } + if (input != null) { + input.position(inputPos + read); + } else { + this.inputPos = inputPos + read; + } + bytesWritten += written; + bytesRead += read; + return written; } } @@ -288,9 +407,14 @@ public class Inflater { * determine if more input data or a preset dictionary is required. * In the latter case, getAdler() can be used to get the Adler-32 * value of the dictionary required. + *

+ * If the {@link #setInput(ByteBuffer)} method was called to provide a buffer + * for input, the input buffer's position will be advanced by the number of bytes + * consumed by this operation, even in the event that an exception is thrown. + * * @param b the buffer for the uncompressed data * @return the actual number of uncompressed bytes - * @exception DataFormatException if the compressed data format is invalid + * @throws DataFormatException if the compressed data format is invalid * @see Inflater#needsInput * @see Inflater#needsDictionary */ @@ -298,6 +422,125 @@ public class Inflater { return inflate(b, 0, b.length); } + /** + * Uncompresses bytes into specified buffer. Returns actual number + * of bytes uncompressed. A return value of 0 indicates that + * needsInput() or needsDictionary() should be called in order to + * determine if more input data or a preset dictionary is required. + * In the latter case, getAdler() can be used to get the Adler-32 + * value of the dictionary required. + *

+ * On success, the position of the given {@code output} byte buffer will be + * advanced by as many bytes as were produced by the operation, which is equal + * to the number returned by this method. + *

+ * If the {@link #setInput(ByteBuffer)} method was called to provide a buffer + * for input, the input buffer's position will be advanced by the number of bytes + * consumed by this operation, even in the event that an exception is thrown. + * + * @param output the buffer for the uncompressed data + * @return the actual number of uncompressed bytes + * @throws DataFormatException if the compressed data format is invalid + * @throws ReadOnlyBufferException if the given output buffer is read-only + * @see Inflater#needsInput + * @see Inflater#needsDictionary + * @since 11 + */ + public int inflate(ByteBuffer output) throws DataFormatException { + if (output.isReadOnly()) { + throw new ReadOnlyBufferException(); + } + synchronized (zsRef) { + ensureOpen(); + final ByteBuffer input = this.input; + final long result; + final int inputPos; + final int outputPos = output.position(); + final int outputRem = Math.max(output.limit() - outputPos, 0); + if (input == null) { + inputPos = this.inputPos; + try { + if (output.isDirect()) { + final long outputAddress = ((DirectBuffer) output).address(); + result = inflateBytesBuffer(zsRef.address(), + inputArray, inputPos, inputLim - inputPos, + outputAddress + outputPos, outputRem); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = inflateBytesBytes(zsRef.address(), + inputArray, inputPos, inputLim - inputPos, + outputArray, outputOffset + outputPos, outputRem); + } + } catch (DataFormatException e) { + int read = inputConsumed; + this.inputPos = inputPos + read; + bytesRead += read; + inputConsumed = 0; + throw e; + } + } else { + inputPos = input.position(); + final int inputRem = Math.max(input.limit() - inputPos, 0); + try { + if (input.isDirect()) { + final long inputAddress = ((DirectBuffer) input).address(); + if (output.isDirect()) { + final long outputAddress = ((DirectBuffer) output).address(); + result = inflateBufferBuffer(zsRef.address(), + inputAddress + inputPos, inputRem, + outputAddress + outputPos, outputRem); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = inflateBufferBytes(zsRef.address(), + inputAddress + inputPos, inputRem, + outputArray, outputOffset + outputPos, outputRem); + } + } else { + final byte[] inputArray = ZipUtils.getBufferArray(input); + final int inputOffset = ZipUtils.getBufferOffset(input); + if (output.isDirect()) { + final long outputAddress = ((DirectBuffer) output).address(); + result = inflateBytesBuffer(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + outputAddress + outputPos, outputRem); + } else { + final byte[] outputArray = ZipUtils.getBufferArray(output); + final int outputOffset = ZipUtils.getBufferOffset(output); + result = inflateBytesBytes(zsRef.address(), + inputArray, inputOffset + inputPos, inputRem, + outputArray, outputOffset + outputPos, outputRem); + } + } + } catch (DataFormatException e) { + int read = inputConsumed; + input.position(inputPos + read); + bytesRead += read; + inputConsumed = 0; + throw e; + } + } + int read = (int) (result & 0x7fff_ffffL); + int written = (int) (result >>> 31 & 0x7fff_ffffL); + if ((result >>> 62 & 1) != 0) { + finished = true; + } + if ((result >>> 63 & 1) != 0) { + needDict = true; + } + if (input != null) { + input.position(inputPos + read); + } else { + this.inputPos = inputPos + read; + } + output.position(outputPos + written); + bytesWritten += written; + bytesRead += read; + return written; + } + } + /** * Returns the ADLER-32 value of the uncompressed data. * @return the ADLER-32 value of the uncompressed data @@ -368,10 +611,10 @@ public class Inflater { synchronized (zsRef) { ensureOpen(); reset(zsRef.address()); - buf = defaultBuf; + input = ZipUtils.defaultBuf; + inputArray = null; finished = false; needDict = false; - off = len = 0; bytesRead = bytesWritten = 0; } } @@ -386,7 +629,8 @@ public class Inflater { public void end() { synchronized (zsRef) { zsRef.clean(); - buf = null; + input = ZipUtils.defaultBuf; + inputArray = null; } } @@ -416,18 +660,23 @@ public class Inflater { throw new NullPointerException("Inflater has been closed"); } - boolean ended() { - synchronized (zsRef) { - return zsRef.address() == 0; - } - } - private static native void initIDs(); private static native long init(boolean nowrap); private static native void setDictionary(long addr, byte[] b, int off, int len); - private native int inflateBytes(long addr, byte[] b, int off, int len) - throws DataFormatException; + private static native void setDictionaryBuffer(long addr, long bufAddress, int len); + private native long inflateBytesBytes(long addr, + byte[] inputArray, int inputOff, int inputLen, + byte[] outputArray, int outputOff, int outputLen) throws DataFormatException; + private native long inflateBytesBuffer(long addr, + byte[] inputArray, int inputOff, int inputLen, + long outputAddress, int outputLen) throws DataFormatException; + private native long inflateBufferBytes(long addr, + long inputAddress, int inputLen, + byte[] outputArray, int outputOff, int outputLen) throws DataFormatException; + private native long inflateBufferBuffer(long addr, + long inputAddress, int inputLen, + long outputAddress, int outputLen) throws DataFormatException; private static native int getAdler(long addr); private static native void reset(long addr); private static native void end(long addr); diff --git a/src/java.base/share/classes/java/util/zip/ZipUtils.java b/src/java.base/share/classes/java/util/zip/ZipUtils.java index 45c5d8dbb67..07e64e4fa92 100644 --- a/src/java.base/share/classes/java/util/zip/ZipUtils.java +++ b/src/java.base/share/classes/java/util/zip/ZipUtils.java @@ -25,6 +25,8 @@ package java.util.zip; +import java.nio.Buffer; +import java.nio.ByteBuffer; import java.nio.file.attribute.FileTime; import java.security.AccessController; import java.security.PrivilegedAction; @@ -37,6 +39,9 @@ import java.util.concurrent.TimeUnit; import static java.util.zip.ZipConstants.ENDHDR; +import jdk.internal.misc.Unsafe; +import sun.nio.ch.DirectBuffer; + class ZipUtils { // used to adjust values between Windows and java epoch @@ -45,6 +50,8 @@ class ZipUtils { // used to indicate the corresponding windows time is not available public static final long WINDOWS_TIME_NOT_AVAILABLE = Long.MIN_VALUE; + static final ByteBuffer defaultBuf = ByteBuffer.allocateDirect(0); + /** * Converts Windows time (in microseconds, UTC/GMT) time to FileTime. */ @@ -281,4 +288,17 @@ class ZipUtils { AccessController.doPrivileged(pa); } } + + private static final Unsafe unsafe = Unsafe.getUnsafe(); + + private static final long byteBufferArrayOffset = unsafe.objectFieldOffset(ByteBuffer.class, "hb"); + private static final long byteBufferOffsetOffset = unsafe.objectFieldOffset(ByteBuffer.class, "offset"); + + static byte[] getBufferArray(ByteBuffer byteBuffer) { + return (byte[]) unsafe.getObject(byteBuffer, byteBufferArrayOffset); + } + + static int getBufferOffset(ByteBuffer byteBuffer) { + return unsafe.getInt(byteBuffer, byteBufferOffsetOffset); + } } diff --git a/src/java.base/share/native/libzip/Deflater.c b/src/java.base/share/native/libzip/Deflater.c index b666a16145a..b56df5ecc1b 100644 --- a/src/java.base/share/native/libzip/Deflater.c +++ b/src/java.base/share/native/libzip/Deflater.c @@ -38,34 +38,6 @@ #define DEF_MEM_LEVEL 8 -static jfieldID levelID; -static jfieldID strategyID; -static jfieldID setParamsID; -static jfieldID finishID; -static jfieldID finishedID; -static jfieldID bufID, offID, lenID; - -JNIEXPORT void JNICALL -Java_java_util_zip_Deflater_initIDs(JNIEnv *env, jclass cls) -{ - levelID = (*env)->GetFieldID(env, cls, "level", "I"); - CHECK_NULL(levelID); - strategyID = (*env)->GetFieldID(env, cls, "strategy", "I"); - CHECK_NULL(strategyID); - setParamsID = (*env)->GetFieldID(env, cls, "setParams", "Z"); - CHECK_NULL(setParamsID); - finishID = (*env)->GetFieldID(env, cls, "finish", "Z"); - CHECK_NULL(finishID); - finishedID = (*env)->GetFieldID(env, cls, "finished", "Z"); - CHECK_NULL(finishedID); - bufID = (*env)->GetFieldID(env, cls, "buf", "[B"); - CHECK_NULL(bufID); - offID = (*env)->GetFieldID(env, cls, "off", "I"); - CHECK_NULL(offID); - lenID = (*env)->GetFieldID(env, cls, "len", "I"); - CHECK_NULL(lenID); -} - JNIEXPORT jlong JNICALL Java_java_util_zip_Deflater_init(JNIEnv *env, jclass cls, jint level, jint strategy, jboolean nowrap) @@ -104,17 +76,9 @@ Java_java_util_zip_Deflater_init(JNIEnv *env, jclass cls, jint level, } } -JNIEXPORT void JNICALL -Java_java_util_zip_Deflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, - jarray b, jint off, jint len) +static void doSetDictionary(JNIEnv *env, jlong addr, jbyte *buf, jint len) { - Bytef *buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - int res; - if (buf == 0) {/* out of memory */ - return; - } - res = deflateSetDictionary((z_stream *)jlong_to_ptr(addr), buf + off, len); - (*env)->ReleasePrimitiveArrayCritical(env, b, buf, 0); + int res = deflateSetDictionary(jlong_to_ptr(addr), (Bytef *) buf, len); switch (res) { case Z_OK: break; @@ -127,94 +91,169 @@ Java_java_util_zip_Deflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, } } -JNIEXPORT jint JNICALL -Java_java_util_zip_Deflater_deflateBytes(JNIEnv *env, jobject this, jlong addr, - jarray b, jint off, jint len, jint flush) +JNIEXPORT void JNICALL +Java_java_util_zip_Deflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, + jbyteArray b, jint off, jint len) +{ + jbyte *buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); + if (buf == NULL) /* out of memory */ + return; + doSetDictionary(env, addr, buf + off, len); + (*env)->ReleasePrimitiveArrayCritical(env, b, buf, 0); +} + +JNIEXPORT void JNICALL +Java_java_util_zip_Deflater_setDictionaryBuffer(JNIEnv *env, jclass cls, jlong addr, + jlong bufferAddr, jint len) +{ + jbyte *buf = jlong_to_ptr(bufferAddr); + doSetDictionary(env, addr, buf, len); +} + +static jlong doDeflate(JNIEnv *env, jobject this, jlong addr, + jbyte *input, jint inputLen, + jbyte *output, jint outputLen, + jint flush, jint params) { z_stream *strm = jlong_to_ptr(addr); + jint inputUsed = 0, outputUsed = 0; - jarray this_buf = (*env)->GetObjectField(env, this, bufID); - jint this_off = (*env)->GetIntField(env, this, offID); - jint this_len = (*env)->GetIntField(env, this, lenID); - jbyte *in_buf; - jbyte *out_buf; - int res; - if ((*env)->GetBooleanField(env, this, setParamsID)) { - int level = (*env)->GetIntField(env, this, levelID); - int strategy = (*env)->GetIntField(env, this, strategyID); - in_buf = (*env)->GetPrimitiveArrayCritical(env, this_buf, 0); - if (in_buf == NULL) { - // Throw OOME only when length is not zero - if (this_len != 0 && (*env)->ExceptionOccurred(env) == NULL) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - out_buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - if (out_buf == NULL) { - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); - if (len != 0 && (*env)->ExceptionOccurred(env) == NULL) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } + strm->next_in = (Bytef *) input; + strm->next_out = (Bytef *) output; + strm->avail_in = inputLen; + strm->avail_out = outputLen; + + int finished = 0; + int setParams = params & 1; - strm->next_in = (Bytef *) (in_buf + this_off); - strm->next_out = (Bytef *) (out_buf + off); - strm->avail_in = this_len; - strm->avail_out = len; - res = deflateParams(strm, level, strategy); - (*env)->ReleasePrimitiveArrayCritical(env, b, out_buf, 0); - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); + if (setParams) { + int strategy = (params >> 1) & 3; + int level = params >> 3; + int res = deflateParams(strm, level, strategy); switch (res) { case Z_OK: - (*env)->SetBooleanField(env, this, setParamsID, JNI_FALSE); + setParams = 0; + /* fall through */ case Z_BUF_ERROR: - this_off += this_len - strm->avail_in; - (*env)->SetIntField(env, this, offID, this_off); - (*env)->SetIntField(env, this, lenID, strm->avail_in); - return (jint) (len - strm->avail_out); + inputUsed = inputLen - strm->avail_in; + outputUsed = outputLen - strm->avail_out; + break; default: JNU_ThrowInternalError(env, strm->msg); return 0; } } else { - jboolean finish = (*env)->GetBooleanField(env, this, finishID); - in_buf = (*env)->GetPrimitiveArrayCritical(env, this_buf, 0); - if (in_buf == NULL) { - if (this_len != 0) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - out_buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - if (out_buf == NULL) { - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); - if (len != 0) - JNU_ThrowOutOfMemoryError(env, 0); - - return 0; - } - - strm->next_in = (Bytef *) (in_buf + this_off); - strm->next_out = (Bytef *) (out_buf + off); - strm->avail_in = this_len; - strm->avail_out = len; - res = deflate(strm, finish ? Z_FINISH : flush); - (*env)->ReleasePrimitiveArrayCritical(env, b, out_buf, 0); - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); + int res = deflate(strm, flush); switch (res) { case Z_STREAM_END: - (*env)->SetBooleanField(env, this, finishedID, JNI_TRUE); + finished = 1; /* fall through */ case Z_OK: case Z_BUF_ERROR: - this_off += this_len - strm->avail_in; - (*env)->SetIntField(env, this, offID, this_off); - (*env)->SetIntField(env, this, lenID, strm->avail_in); - return len - strm->avail_out; + inputUsed = inputLen - strm->avail_in; + outputUsed = outputLen - strm->avail_out; + break; default: JNU_ThrowInternalError(env, strm->msg); return 0; } } + return ((jlong)inputUsed) | (((jlong)outputUsed) << 31) | (((jlong)finished) << 62) | (((jlong)setParams) << 63); +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Deflater_deflateBytesBytes(JNIEnv *env, jobject this, jlong addr, + jbyteArray inputArray, jint inputOff, jint inputLen, + jbyteArray outputArray, jint outputOff, jint outputLen, + jint flush, jint params) +{ + jbyte *input = (*env)->GetPrimitiveArrayCritical(env, inputArray, 0); + if (input == NULL) { + if (inputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + jbyte *output = (*env)->GetPrimitiveArrayCritical(env, outputArray, 0); + if (output == NULL) { + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + if (outputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + + jlong retVal = doDeflate(env, this, addr, + input + inputOff, inputLen, + output + outputOff, outputLen, + flush, params); + + (*env)->ReleasePrimitiveArrayCritical(env, outputArray, output, 0); + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + + return retVal; +} + + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Deflater_deflateBytesBuffer(JNIEnv *env, jobject this, jlong addr, + jbyteArray inputArray, jint inputOff, jint inputLen, + jlong outputBuffer, jint outputLen, + jint flush, jint params) +{ + jbyte *input = (*env)->GetPrimitiveArrayCritical(env, inputArray, 0); + if (input == NULL) { + if (inputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + jbyte *output = jlong_to_ptr(outputBuffer); + + jlong retVal = doDeflate(env, this, addr, + input + inputOff, inputLen, + output, outputLen, + flush, params); + + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Deflater_deflateBufferBytes(JNIEnv *env, jobject this, jlong addr, + jlong inputBuffer, jint inputLen, + jbyteArray outputArray, jint outputOff, jint outputLen, + jint flush, jint params) +{ + jbyte *input = jlong_to_ptr(inputBuffer); + jbyte *output = (*env)->GetPrimitiveArrayCritical(env, outputArray, 0); + if (output == NULL) { + if (outputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + + jlong retVal = doDeflate(env, this, addr, + input, inputLen, + output + outputOff, outputLen, + flush, params); + + (*env)->ReleasePrimitiveArrayCritical(env, outputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Deflater_deflateBufferBuffer(JNIEnv *env, jobject this, jlong addr, + jlong inputBuffer, jint inputLen, + jlong outputBuffer, jint outputLen, + jint flush, jint params) +{ + jbyte *input = jlong_to_ptr(inputBuffer); + jbyte *output = jlong_to_ptr(outputBuffer); + + return doDeflate(env, this, addr, + input, inputLen, + output, outputLen, + flush, params); } JNIEXPORT jint JNICALL diff --git a/src/java.base/share/native/libzip/Inflater.c b/src/java.base/share/native/libzip/Inflater.c index 2e21d084b39..4999b5467a0 100644 --- a/src/java.base/share/native/libzip/Inflater.c +++ b/src/java.base/share/native/libzip/Inflater.c @@ -42,23 +42,13 @@ #define ThrowDataFormatException(env, msg) \ JNU_ThrowByName(env, "java/util/zip/DataFormatException", msg) -static jfieldID needDictID; -static jfieldID finishedID; -static jfieldID bufID, offID, lenID; +static jfieldID inputConsumedID; JNIEXPORT void JNICALL Java_java_util_zip_Inflater_initIDs(JNIEnv *env, jclass cls) { - needDictID = (*env)->GetFieldID(env, cls, "needDict", "Z"); - CHECK_NULL(needDictID); - finishedID = (*env)->GetFieldID(env, cls, "finished", "Z"); - CHECK_NULL(finishedID); - bufID = (*env)->GetFieldID(env, cls, "buf", "[B"); - CHECK_NULL(bufID); - offID = (*env)->GetFieldID(env, cls, "off", "I"); - CHECK_NULL(offID); - lenID = (*env)->GetFieldID(env, cls, "len", "I"); - CHECK_NULL(lenID); + inputConsumedID = (*env)->GetFieldID(env, cls, "inputConsumed", "I"); + CHECK_NULL(inputConsumedID); } JNIEXPORT jlong JNICALL @@ -94,16 +84,9 @@ Java_java_util_zip_Inflater_init(JNIEnv *env, jclass cls, jboolean nowrap) } } -JNIEXPORT void JNICALL -Java_java_util_zip_Inflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, - jarray b, jint off, jint len) +static void doSetDictionary(JNIEnv *env, jlong addr, jbyte *buf, jint len) { - Bytef *buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - int res; - if (buf == 0) /* out of memory */ - return; - res = inflateSetDictionary(jlong_to_ptr(addr), buf + off, len); - (*env)->ReleasePrimitiveArrayCritical(env, b, buf, 0); + int res = inflateSetDictionary(jlong_to_ptr(addr), (Bytef *) buf, len); switch (res) { case Z_OK: break; @@ -117,68 +100,155 @@ Java_java_util_zip_Inflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, } } -JNIEXPORT jint JNICALL -Java_java_util_zip_Inflater_inflateBytes(JNIEnv *env, jobject this, jlong addr, - jarray b, jint off, jint len) +JNIEXPORT void JNICALL +Java_java_util_zip_Inflater_setDictionary(JNIEnv *env, jclass cls, jlong addr, + jbyteArray b, jint off, jint len) +{ + jbyte *buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); + if (buf == NULL) /* out of memory */ + return; + doSetDictionary(env, addr, buf + off, len); + (*env)->ReleasePrimitiveArrayCritical(env, b, buf, 0); +} + +JNIEXPORT void JNICALL +Java_java_util_zip_Inflater_setDictionaryBuffer(JNIEnv *env, jclass cls, jlong addr, + jlong bufferAddr, jint len) +{ + jbyte *buf = jlong_to_ptr(bufferAddr); + doSetDictionary(env, addr, buf, len); +} + +static jlong doInflate(JNIEnv *env, jobject this, jlong addr, + jbyte *input, jint inputLen, + jbyte *output, jint outputLen) { z_stream *strm = jlong_to_ptr(addr); - jarray this_buf = (jarray)(*env)->GetObjectField(env, this, bufID); - jint this_off = (*env)->GetIntField(env, this, offID); - jint this_len = (*env)->GetIntField(env, this, lenID); + jint inputUsed = 0, outputUsed = 0; - jbyte *in_buf; - jbyte *out_buf; - int ret; + strm->next_in = (Bytef *) input; + strm->next_out = (Bytef *) output; + strm->avail_in = inputLen; + strm->avail_out = outputLen; - in_buf = (*env)->GetPrimitiveArrayCritical(env, this_buf, 0); - if (in_buf == NULL) { - if (this_len != 0 && (*env)->ExceptionOccurred(env) == NULL) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - out_buf = (*env)->GetPrimitiveArrayCritical(env, b, 0); - if (out_buf == NULL) { - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); - if (len != 0 && (*env)->ExceptionOccurred(env) == NULL) - JNU_ThrowOutOfMemoryError(env, 0); - return 0; - } - strm->next_in = (Bytef *) (in_buf + this_off); - strm->next_out = (Bytef *) (out_buf + off); - strm->avail_in = this_len; - strm->avail_out = len; - ret = inflate(strm, Z_PARTIAL_FLUSH); - (*env)->ReleasePrimitiveArrayCritical(env, b, out_buf, 0); - (*env)->ReleasePrimitiveArrayCritical(env, this_buf, in_buf, 0); + int ret = inflate(strm, Z_PARTIAL_FLUSH); + int finished = 0; + int needDict = 0; switch (ret) { case Z_STREAM_END: - (*env)->SetBooleanField(env, this, finishedID, JNI_TRUE); + finished = 1; /* fall through */ case Z_OK: - this_off += this_len - strm->avail_in; - (*env)->SetIntField(env, this, offID, this_off); - (*env)->SetIntField(env, this, lenID, strm->avail_in); - return (jint) (len - strm->avail_out); + inputUsed = inputLen - strm->avail_in; + outputUsed = outputLen - strm->avail_out; + break; case Z_NEED_DICT: - (*env)->SetBooleanField(env, this, needDictID, JNI_TRUE); + needDict = 1; /* Might have consumed some input here! */ - this_off += this_len - strm->avail_in; - (*env)->SetIntField(env, this, offID, this_off); - (*env)->SetIntField(env, this, lenID, strm->avail_in); - return 0; + inputUsed = inputLen - strm->avail_in; + break; case Z_BUF_ERROR: - return 0; + break; case Z_DATA_ERROR: + inputUsed = inputLen - strm->avail_in; + (*env)->SetIntField(env, this, inputConsumedID, inputUsed); ThrowDataFormatException(env, strm->msg); - return 0; + break; case Z_MEM_ERROR: JNU_ThrowOutOfMemoryError(env, 0); - return 0; + break; default: JNU_ThrowInternalError(env, strm->msg); - return 0; + break; + } + return ((jlong)inputUsed) | (((jlong)outputUsed) << 31) | (((jlong)finished) << 62) | (((jlong)needDict) << 63); +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Inflater_inflateBytesBytes(JNIEnv *env, jobject this, jlong addr, + jbyteArray inputArray, jint inputOff, jint inputLen, + jbyteArray outputArray, jint outputOff, jint outputLen) +{ + jbyte *input = (*env)->GetPrimitiveArrayCritical(env, inputArray, 0); + if (input == NULL) { + if (inputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + jbyte *output = (*env)->GetPrimitiveArrayCritical(env, outputArray, 0); + if (output == NULL) { + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + if (outputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + + jlong retVal = doInflate(env, this, addr, + input + inputOff, inputLen, + output + outputOff, outputLen); + + (*env)->ReleasePrimitiveArrayCritical(env, outputArray, output, 0); + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Inflater_inflateBytesBuffer(JNIEnv *env, jobject this, jlong addr, + jbyteArray inputArray, jint inputOff, jint inputLen, + jlong outputBuffer, jint outputLen) +{ + jbyte *input = (*env)->GetPrimitiveArrayCritical(env, inputArray, 0); + if (input == NULL) { + if (inputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; + } + jbyte *output = jlong_to_ptr(outputBuffer); + + jlong retVal = doInflate(env, this, addr, + input + inputOff, inputLen, + output, outputLen); + + (*env)->ReleasePrimitiveArrayCritical(env, inputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Inflater_inflateBufferBytes(JNIEnv *env, jobject this, jlong addr, + jlong inputBuffer, jint inputLen, + jbyteArray outputArray, jint outputOff, jint outputLen) +{ + jbyte *input = jlong_to_ptr(inputBuffer); + jbyte *output = (*env)->GetPrimitiveArrayCritical(env, outputArray, 0); + if (output == NULL) { + if (outputLen != 0 && (*env)->ExceptionOccurred(env) == NULL) + JNU_ThrowOutOfMemoryError(env, 0); + return 0L; } + + jlong retVal = doInflate(env, this, addr, + input, inputLen, + output + outputOff, outputLen); + + (*env)->ReleasePrimitiveArrayCritical(env, outputArray, input, 0); + + return retVal; +} + +JNIEXPORT jlong JNICALL +Java_java_util_zip_Inflater_inflateBufferBuffer(JNIEnv *env, jobject this, jlong addr, + jlong inputBuffer, jint inputLen, + jlong outputBuffer, jint outputLen) +{ + jbyte *input = jlong_to_ptr(inputBuffer); + jbyte *output = jlong_to_ptr(outputBuffer); + + return doInflate(env, this, addr, + input, inputLen, + output, outputLen); } JNIEXPORT jint JNICALL diff --git a/test/jdk/java/util/zip/FlaterTest.java b/test/jdk/java/util/zip/FlaterTest.java index 7245440d033..b5aff0319b3 100644 --- a/test/jdk/java/util/zip/FlaterTest.java +++ b/test/jdk/java/util/zip/FlaterTest.java @@ -29,7 +29,6 @@ * @key randomness */ -import java.io.*; import java.nio.*; import java.util.*; import java.util.zip.*; @@ -41,35 +40,37 @@ import java.util.zip.*; */ public class FlaterTest extends Thread { private static final int DATA_LEN = 1024 * 128; - private static byte[] data; + + private static ByteBuffer dataDirect; + private static ByteBuffer dataHeap; // If true, print extra info. private static final boolean debug = false; // Set of Flater threads running. - private static Set flaters = - Collections.synchronizedSet(new HashSet()); + private static Set flaters = + Collections.synchronizedSet(new HashSet<>()); /** Fill in {@code data} with random values. */ static void createData() { - ByteBuffer bb = ByteBuffer.allocate(8); - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - for (int i = 0; i < DATA_LEN; i++) { - bb.putDouble(0, Math.random()); - baos.write(bb.array(), 0, 8); + ByteBuffer bb = ByteBuffer.allocateDirect(DATA_LEN * 8); + for (int i = 0; i < DATA_LEN * 8; i += 8) { + bb.putDouble(i, Math.random()); } - data = baos.toByteArray(); - if (debug) System.out.println("data length is " + data.length); + dataDirect = bb; + final ByteBuffer hb = ByteBuffer.allocate(bb.capacity()); + hb.duplicate().put(bb.duplicate()); + dataHeap = hb; + if (debug) System.out.println("data length is " + bb.capacity()); } /** @return the length of the deflated {@code data}. */ - private static int getDeflatedLength() throws Throwable { - int rc = 0; + private static int getDeflatedLength() { Deflater deflater = new Deflater(); - deflater.setInput(data); + deflater.setInput(dataDirect.duplicate()); deflater.finish(); - byte[] out = new byte[data.length]; - rc = deflater.deflate(out); + byte[] out = new byte[dataDirect.capacity()]; + int rc = deflater.deflate(out); deflater.end(); if (debug) System.out.println("deflatedLength is " + rc); return rc; @@ -78,70 +79,98 @@ public class FlaterTest extends Thread { /** Compares given bytes with those in {@code data}. * @throws Exception if given bytes don't match {@code data}. */ - private static void validate(byte[] buf, int offset, int len) throws Exception { + private static void validate(ByteBuffer buf, int offset, int len) throws Exception { for (int i = 0; i < len; i++ ) { - if (buf[i] != data[offset+i]) { + if (buf.get(i) != dataDirect.get(offset+i)) { throw new Exception("mismatch at " + (offset + i)); } } } - public static void realMain(String[] args) throws Throwable { - createData(); + public static void realMain(String[] args) { int numThreads = args.length > 0 ? Integer.parseInt(args[0]) : 5; - new FlaterTest().go(numThreads); + createData(); + for (int srcMode = 0; srcMode <= 2; srcMode ++) { + for (int dstMode = 0; dstMode <= 2; dstMode ++) { + new FlaterTest().go(numThreads, srcMode, dstMode); + } + } } - private synchronized void go(int numThreads) throws Throwable { + private synchronized void go(int numThreads, int srcMode, int dstMode) { int deflatedLength = getDeflatedLength(); long time = System.currentTimeMillis(); for (int i = 0; i < numThreads; i++) { - Flater f = new Flater(deflatedLength); + Flater f = new Flater(deflatedLength, srcMode, dstMode); flaters.add(f); f.start(); } - while (flaters.size() != 0) { - try { - Thread.currentThread().sleep(10); - } catch (InterruptedException ex) { - unexpected(ex); + synchronized (flaters) { + while (flaters.size() != 0) { + try { + flaters.wait(); + } catch (InterruptedException ex) { + unexpected(ex); + } } } time = System.currentTimeMillis() - time; System.out.println("Time needed for " + numThreads - + " threads to deflate/inflate: " + time + " ms."); + + " threads to deflate/inflate: " + time + " ms (srcMode="+srcMode+",dstMode="+dstMode+")"); } /** Deflates and inflates data. */ static class Flater extends Thread { private final int deflatedLength; + private final int srcMode, dstMode; - private Flater(int length) { + private Flater(int length, int srcMode, int dstMode) { this.deflatedLength = length; + this.srcMode = srcMode; + this.dstMode = dstMode; } /** Deflates and inflates {@code data}. */ public void run() { if (debug) System.out.println(getName() + " starting run()"); try { - byte[] deflated = DeflateData(deflatedLength); + ByteBuffer deflated = DeflateData(deflatedLength); InflateData(deflated); } catch (Throwable t) { t.printStackTrace(); fail(getName() + " failed"); } finally { - flaters.remove(this); + synchronized (flaters) { + flaters.remove(this); + if (flaters.isEmpty()) { + flaters.notifyAll(); + } + } } } /** Returns a copy of {@code data} in deflated form. */ - private byte[] DeflateData(int length) throws Throwable { + private ByteBuffer DeflateData(int length) { Deflater deflater = new Deflater(); - deflater.setInput(data); + if (srcMode == 0) { + deflater.setInput(dataHeap.array()); + } else if (srcMode == 1) { + deflater.setInput(dataHeap.duplicate()); + } else { + assert srcMode == 2; + deflater.setInput(dataDirect.duplicate()); + } deflater.finish(); - byte[] out = new byte[length]; - deflater.deflate(out); + ByteBuffer out = dstMode == 2 ? ByteBuffer.allocateDirect(length) : ByteBuffer.allocate(length); + int deflated; + if (dstMode == 0) { + deflated = deflater.deflate(out.array(), 0, length); + out.position(deflated); + } else { + deflater.deflate(out); + } + out.flip(); return out; } @@ -149,14 +178,30 @@ public class FlaterTest extends Thread { * inflation. * @throws Exception if inflated bytes don't match {@code data}. */ - private void InflateData(byte[] bytes) throws Throwable { + private void InflateData(ByteBuffer bytes) throws Throwable { Inflater inflater = new Inflater(); - inflater.setInput(bytes, 0, bytes.length); + if (dstMode == 0) { + inflater.setInput(bytes.array(), 0, bytes.remaining()); + } else { + inflater.setInput(bytes); + } + if (inflater.getRemaining() == 0) { + throw new Exception("Nothing to inflate (bytes=" + bytes + ")"); + } int len = 1024 * 8; int offset = 0; + ByteBuffer buf = srcMode == 2 ? ByteBuffer.allocateDirect(len) : ByteBuffer.allocate(len); while (inflater.getRemaining() > 0) { - byte[] buf = new byte[len]; - int inflated = inflater.inflate(buf, 0, len); + buf.clear(); + int inflated; + if (srcMode == 0) { + inflated = inflater.inflate(buf.array(), 0, buf.remaining()); + } else { + inflated = inflater.inflate(buf); + } + if (inflated == 0) { + throw new Exception("Nothing inflated (dst=" + buf + ",offset=" + offset + ",rem=" + inflater.getRemaining() + ",srcMode="+srcMode+",dstMode="+dstMode+")"); + } validate(buf, offset, inflated); offset += inflated; } -------------- next part -------------- # Run complete. Total time: 00:01:14 Benchmark (opSize) Mode Cnt Score Error Units InflateTests.testAIAO 16 sample 297564 997.985 ? 1.602 ns/op InflateTests.testAIAO:testAIAO?p0.00 16 sample 383.000 ns/op InflateTests.testAIAO:testAIAO?p0.50 16 sample 978.000 ns/op InflateTests.testAIAO:testAIAO?p0.90 16 sample 1190.000 ns/op InflateTests.testAIAO:testAIAO?p0.95 16 sample 1262.000 ns/op InflateTests.testAIAO:testAIAO?p0.99 16 sample 1626.000 ns/op InflateTests.testAIAO:testAIAO?p0.999 16 sample 3828.000 ns/op InflateTests.testAIAO:testAIAO?p0.9999 16 sample 6729.948 ns/op InflateTests.testAIAO:testAIAO?p1.00 16 sample 29760.000 ns/op InflateTests.testAIAO 128 sample 286720 4274.280 ? 12.714 ns/op InflateTests.testAIAO:testAIAO?p0.00 128 sample 3280.000 ns/op InflateTests.testAIAO:testAIAO?p0.50 128 sample 4224.000 ns/op InflateTests.testAIAO:testAIAO?p0.90 128 sample 4600.000 ns/op InflateTests.testAIAO:testAIAO?p0.95 128 sample 4728.000 ns/op InflateTests.testAIAO:testAIAO?p0.99 128 sample 6176.000 ns/op InflateTests.testAIAO:testAIAO?p0.999 128 sample 8656.000 ns/op InflateTests.testAIAO:testAIAO?p0.9999 128 sample 16746.493 ns/op InflateTests.testAIAO:testAIAO?p1.00 128 sample 1075200.000 ns/op InflateTests.testAIAO 1024 sample 160496 30884.232 ? 14.240 ns/op InflateTests.testAIAO:testAIAO?p0.00 1024 sample 28160.000 ns/op InflateTests.testAIAO:testAIAO?p0.50 1024 sample 30656.000 ns/op InflateTests.testAIAO:testAIAO?p0.90 1024 sample 31680.000 ns/op InflateTests.testAIAO:testAIAO?p0.95 1024 sample 32512.000 ns/op InflateTests.testAIAO:testAIAO?p0.99 1024 sample 39936.000 ns/op InflateTests.testAIAO:testAIAO?p0.999 1024 sample 48320.000 ns/op InflateTests.testAIAO:testAIAO?p0.9999 1024 sample 54646.458 ns/op InflateTests.testAIAO:testAIAO?p1.00 1024 sample 94208.000 ns/op -------------- next part -------------- # Run complete. Total time: 00:09:20 Benchmark (opSize) Mode Cnt Score Error Units InflateTests.testAIAO 16 sample 317476 931.285 ? 1.366 ns/op InflateTests.testAIAO:testAIAO?p0.00 16 sample 322.000 ns/op InflateTests.testAIAO:testAIAO?p0.50 16 sample 914.000 ns/op InflateTests.testAIAO:testAIAO?p0.90 16 sample 1124.000 ns/op InflateTests.testAIAO:testAIAO?p0.95 16 sample 1190.000 ns/op InflateTests.testAIAO:testAIAO?p0.99 16 sample 1462.000 ns/op InflateTests.testAIAO:testAIAO?p0.999 16 sample 3666.092 ns/op InflateTests.testAIAO:testAIAO?p0.9999 16 sample 5920.368 ns/op InflateTests.testAIAO:testAIAO?p1.00 16 sample 17248.000 ns/op InflateTests.testAIAO 128 sample 291550 4198.369 ? 2.803 ns/op InflateTests.testAIAO:testAIAO?p0.00 128 sample 3200.000 ns/op InflateTests.testAIAO:testAIAO?p0.50 128 sample 4152.000 ns/op InflateTests.testAIAO:testAIAO?p0.90 128 sample 4520.000 ns/op InflateTests.testAIAO:testAIAO?p0.95 128 sample 4664.000 ns/op InflateTests.testAIAO:testAIAO?p0.99 128 sample 6232.000 ns/op InflateTests.testAIAO:testAIAO?p0.999 128 sample 8128.000 ns/op InflateTests.testAIAO:testAIAO?p0.9999 128 sample 11853.518 ns/op InflateTests.testAIAO:testAIAO?p1.00 128 sample 46208.000 ns/op InflateTests.testAIAO 1024 sample 161199 30768.958 ? 12.730 ns/op InflateTests.testAIAO:testAIAO?p0.00 1024 sample 28032.000 ns/op InflateTests.testAIAO:testAIAO?p0.50 1024 sample 30560.000 ns/op InflateTests.testAIAO:testAIAO?p0.90 1024 sample 31616.000 ns/op InflateTests.testAIAO:testAIAO?p0.95 1024 sample 32448.000 ns/op InflateTests.testAIAO:testAIAO?p0.99 1024 sample 37120.000 ns/op InflateTests.testAIAO:testAIAO?p0.999 1024 sample 47667.200 ns/op InflateTests.testAIAO:testAIAO?p0.9999 1024 sample 56488.960 ns/op InflateTests.testAIAO:testAIAO?p1.00 1024 sample 90624.000 ns/op InflateTests.testAIDO 16 sample 327750 904.254 ? 1.392 ns/op InflateTests.testAIDO:testAIDO?p0.00 16 sample 274.000 ns/op InflateTests.testAIDO:testAIDO?p0.50 16 sample 885.000 ns/op InflateTests.testAIDO:testAIDO?p0.90 16 sample 1098.000 ns/op InflateTests.testAIDO:testAIDO?p0.95 16 sample 1168.000 ns/op InflateTests.testAIDO:testAIDO?p0.99 16 sample 1508.000 ns/op InflateTests.testAIDO:testAIDO?p0.999 16 sample 3728.000 ns/op InflateTests.testAIDO:testAIDO?p0.9999 16 sample 6113.799 ns/op InflateTests.testAIDO:testAIDO?p1.00 16 sample 9904.000 ns/op InflateTests.testAIDO 128 sample 292247 4189.979 ? 8.170 ns/op InflateTests.testAIDO:testAIDO?p0.00 128 sample 3168.000 ns/op InflateTests.testAIDO:testAIDO?p0.50 128 sample 4128.000 ns/op InflateTests.testAIDO:testAIDO?p0.90 128 sample 4536.000 ns/op InflateTests.testAIDO:testAIDO?p0.95 128 sample 4688.000 ns/op InflateTests.testAIDO:testAIDO?p0.99 128 sample 6512.000 ns/op InflateTests.testAIDO:testAIDO?p0.999 128 sample 8432.000 ns/op InflateTests.testAIDO:testAIDO?p0.9999 128 sample 12944.000 ns/op InflateTests.testAIDO:testAIDO?p1.00 128 sample 674816.000 ns/op InflateTests.testAIDO 1024 sample 158182 31271.382 ? 28.774 ns/op InflateTests.testAIDO:testAIDO?p0.00 1024 sample 28160.000 ns/op InflateTests.testAIDO:testAIDO?p0.50 1024 sample 30944.000 ns/op InflateTests.testAIDO:testAIDO?p0.90 1024 sample 32384.000 ns/op InflateTests.testAIDO:testAIDO?p0.95 1024 sample 33408.000 ns/op InflateTests.testAIDO:testAIDO?p0.99 1024 sample 41600.000 ns/op InflateTests.testAIDO:testAIDO?p0.999 1024 sample 50368.000 ns/op InflateTests.testAIDO:testAIDO?p0.9999 1024 sample 58740.205 ns/op InflateTests.testAIDO:testAIDO?p1.00 1024 sample 786432.000 ns/op InflateTests.testAIHO 16 sample 319679 922.777 ? 1.386 ns/op InflateTests.testAIHO:testAIHO?p0.00 16 sample 314.000 ns/op InflateTests.testAIHO:testAIHO?p0.50 16 sample 905.000 ns/op InflateTests.testAIHO:testAIHO?p0.90 16 sample 1114.000 ns/op InflateTests.testAIHO:testAIHO?p0.95 16 sample 1180.000 ns/op InflateTests.testAIHO:testAIHO?p0.99 16 sample 1492.000 ns/op InflateTests.testAIHO:testAIHO?p0.999 16 sample 3713.280 ns/op InflateTests.testAIHO:testAIHO?p0.9999 16 sample 5747.328 ns/op InflateTests.testAIHO:testAIHO?p1.00 16 sample 22304.000 ns/op InflateTests.testAIHO 128 sample 288260 4244.463 ? 3.200 ns/op InflateTests.testAIHO:testAIHO?p0.00 128 sample 3220.000 ns/op InflateTests.testAIHO:testAIHO?p0.50 128 sample 4184.000 ns/op InflateTests.testAIHO:testAIHO?p0.90 128 sample 4600.000 ns/op InflateTests.testAIHO:testAIHO?p0.95 128 sample 4768.000 ns/op InflateTests.testAIHO:testAIHO?p0.99 128 sample 6576.000 ns/op InflateTests.testAIHO:testAIHO?p0.999 128 sample 8544.000 ns/op InflateTests.testAIHO:testAIHO?p0.9999 128 sample 12267.819 ns/op InflateTests.testAIHO:testAIHO?p1.00 128 sample 48128.000 ns/op InflateTests.testAIHO 1024 sample 158861 31168.797 ? 24.568 ns/op InflateTests.testAIHO:testAIHO?p0.00 1024 sample 28160.000 ns/op InflateTests.testAIHO:testAIHO?p0.50 1024 sample 30880.000 ns/op InflateTests.testAIHO:testAIHO?p0.90 1024 sample 32128.000 ns/op InflateTests.testAIHO:testAIHO?p0.95 1024 sample 33088.000 ns/op InflateTests.testAIHO:testAIHO?p0.99 1024 sample 40832.000 ns/op InflateTests.testAIHO:testAIHO?p0.999 1024 sample 51712.000 ns/op InflateTests.testAIHO:testAIHO?p0.9999 1024 sample 61326.566 ns/op InflateTests.testAIHO:testAIHO?p1.00 1024 sample 873472.000 ns/op InflateTests.testDIAO 16 sample 333837 879.467 ? 1.301 ns/op InflateTests.testDIAO:testDIAO?p0.00 16 sample 260.000 ns/op InflateTests.testDIAO:testDIAO?p0.50 16 sample 863.000 ns/op InflateTests.testDIAO:testDIAO?p0.90 16 sample 1070.000 ns/op InflateTests.testDIAO:testDIAO?p0.95 16 sample 1136.000 ns/op InflateTests.testDIAO:testDIAO?p0.99 16 sample 1402.000 ns/op InflateTests.testDIAO:testDIAO?p0.999 16 sample 3596.648 ns/op InflateTests.testDIAO:testDIAO?p0.9999 16 sample 5737.859 ns/op InflateTests.testDIAO:testDIAO?p1.00 16 sample 15312.000 ns/op InflateTests.testDIAO 128 sample 293782 4160.117 ? 2.942 ns/op InflateTests.testDIAO:testDIAO?p0.00 128 sample 3164.000 ns/op InflateTests.testDIAO:testDIAO?p0.50 128 sample 4104.000 ns/op InflateTests.testDIAO:testDIAO?p0.90 128 sample 4488.000 ns/op InflateTests.testDIAO:testDIAO?p0.95 128 sample 4640.000 ns/op InflateTests.testDIAO:testDIAO?p0.99 128 sample 6192.000 ns/op InflateTests.testDIAO:testDIAO?p0.999 128 sample 8145.736 ns/op InflateTests.testDIAO:testDIAO?p0.9999 128 sample 12701.842 ns/op InflateTests.testDIAO:testDIAO?p1.00 128 sample 46912.000 ns/op InflateTests.testDIAO 1024 sample 160960 30808.034 ? 23.151 ns/op InflateTests.testDIAO:testDIAO?p0.00 1024 sample 28032.000 ns/op InflateTests.testDIAO:testDIAO?p0.50 1024 sample 30496.000 ns/op InflateTests.testDIAO:testDIAO?p0.90 1024 sample 31584.000 ns/op InflateTests.testDIAO:testDIAO?p0.95 1024 sample 32480.000 ns/op InflateTests.testDIAO:testDIAO?p0.99 1024 sample 41664.000 ns/op InflateTests.testDIAO:testDIAO?p0.999 1024 sample 48066.496 ns/op InflateTests.testDIAO:testDIAO?p0.9999 1024 sample 56243.699 ns/op InflateTests.testDIAO:testDIAO?p1.00 1024 sample 849920.000 ns/op InflateTests.testDIDO 16 sample 347580 849.776 ? 1.366 ns/op InflateTests.testDIDO:testDIDO?p0.00 16 sample 234.000 ns/op InflateTests.testDIDO:testDIDO?p0.50 16 sample 831.000 ns/op InflateTests.testDIDO:testDIDO?p0.90 16 sample 1044.000 ns/op InflateTests.testDIDO:testDIDO?p0.95 16 sample 1114.000 ns/op InflateTests.testDIDO:testDIDO?p0.99 16 sample 1460.000 ns/op InflateTests.testDIDO:testDIDO?p0.999 16 sample 3640.000 ns/op InflateTests.testDIDO:testDIDO?p0.9999 16 sample 6131.870 ns/op InflateTests.testDIDO:testDIDO?p1.00 16 sample 13696.000 ns/op InflateTests.testDIDO 128 sample 295207 4152.105 ? 3.356 ns/op InflateTests.testDIDO:testDIDO?p0.00 128 sample 3136.000 ns/op InflateTests.testDIDO:testDIDO?p0.50 128 sample 4096.000 ns/op InflateTests.testDIDO:testDIDO?p0.90 128 sample 4496.000 ns/op InflateTests.testDIDO:testDIDO?p0.95 128 sample 4656.000 ns/op InflateTests.testDIDO:testDIDO?p0.99 128 sample 6432.000 ns/op InflateTests.testDIDO:testDIDO?p0.999 128 sample 8672.000 ns/op InflateTests.testDIDO:testDIDO?p0.9999 128 sample 13757.338 ns/op InflateTests.testDIDO:testDIDO?p1.00 128 sample 70016.000 ns/op InflateTests.testDIDO 1024 sample 158555 31216.219 ? 17.789 ns/op InflateTests.testDIDO:testDIDO?p0.00 1024 sample 28160.000 ns/op InflateTests.testDIDO:testDIDO?p0.50 1024 sample 30880.000 ns/op InflateTests.testDIDO:testDIDO?p0.90 1024 sample 32384.000 ns/op InflateTests.testDIDO:testDIDO?p0.95 1024 sample 33216.000 ns/op InflateTests.testDIDO:testDIDO?p0.99 1024 sample 42624.000 ns/op InflateTests.testDIDO:testDIDO?p0.999 1024 sample 50624.000 ns/op InflateTests.testDIDO:testDIDO?p0.9999 1024 sample 57527.450 ns/op InflateTests.testDIDO:testDIDO?p1.00 1024 sample 355840.000 ns/op InflateTests.testDIHO 16 sample 332135 890.586 ? 1.392 ns/op InflateTests.testDIHO:testDIHO?p0.00 16 sample 278.000 ns/op InflateTests.testDIHO:testDIHO?p0.50 16 sample 872.000 ns/op InflateTests.testDIHO:testDIHO?p0.90 16 sample 1084.000 ns/op InflateTests.testDIHO:testDIHO?p0.95 16 sample 1154.000 ns/op InflateTests.testDIHO:testDIHO?p0.99 16 sample 1484.000 ns/op InflateTests.testDIHO:testDIHO?p0.999 16 sample 3688.000 ns/op InflateTests.testDIHO:testDIHO?p0.9999 16 sample 5728.000 ns/op InflateTests.testDIHO:testDIHO?p1.00 16 sample 35456.000 ns/op InflateTests.testDIHO 128 sample 291971 4195.645 ? 2.912 ns/op InflateTests.testDIHO:testDIHO?p0.00 128 sample 3176.000 ns/op InflateTests.testDIHO:testDIHO?p0.50 128 sample 4136.000 ns/op InflateTests.testDIHO:testDIHO?p0.90 128 sample 4544.000 ns/op InflateTests.testDIHO:testDIHO?p0.95 128 sample 4696.000 ns/op InflateTests.testDIHO:testDIHO?p0.99 128 sample 6464.000 ns/op InflateTests.testDIHO:testDIHO?p0.999 128 sample 8480.000 ns/op InflateTests.testDIHO:testDIHO?p0.9999 128 sample 12906.138 ns/op InflateTests.testDIHO:testDIHO?p1.00 128 sample 31840.000 ns/op InflateTests.testDIHO 1024 sample 158528 31191.515 ? 19.858 ns/op InflateTests.testDIHO:testDIHO?p0.00 1024 sample 28032.000 ns/op InflateTests.testDIHO:testDIHO?p0.50 1024 sample 30848.000 ns/op InflateTests.testDIHO:testDIHO?p0.90 1024 sample 32256.000 ns/op InflateTests.testDIHO:testDIHO?p0.95 1024 sample 33344.000 ns/op InflateTests.testDIHO:testDIHO?p0.99 1024 sample 42304.000 ns/op InflateTests.testDIHO:testDIHO?p0.999 1024 sample 50688.000 ns/op InflateTests.testDIHO:testDIHO?p0.9999 1024 sample 57044.730 ns/op InflateTests.testDIHO:testDIHO?p1.00 1024 sample 484352.000 ns/op InflateTests.testHIAO 16 sample 319309 920.649 ? 1.436 ns/op InflateTests.testHIAO:testHIAO?p0.00 16 sample 309.000 ns/op InflateTests.testHIAO:testHIAO?p0.50 16 sample 904.000 ns/op InflateTests.testHIAO:testHIAO?p0.90 16 sample 1112.000 ns/op InflateTests.testHIAO:testHIAO?p0.95 16 sample 1178.000 ns/op InflateTests.testHIAO:testHIAO?p0.99 16 sample 1440.000 ns/op InflateTests.testHIAO:testHIAO?p0.999 16 sample 3684.000 ns/op InflateTests.testHIAO:testHIAO?p0.9999 16 sample 6137.104 ns/op InflateTests.testHIAO:testHIAO?p1.00 16 sample 39104.000 ns/op InflateTests.testHIAO 128 sample 289823 4225.390 ? 3.155 ns/op InflateTests.testHIAO:testHIAO?p0.00 128 sample 3204.000 ns/op InflateTests.testHIAO:testHIAO?p0.50 128 sample 4152.000 ns/op InflateTests.testHIAO:testHIAO?p0.90 128 sample 4568.000 ns/op InflateTests.testHIAO:testHIAO?p0.95 128 sample 4776.000 ns/op InflateTests.testHIAO:testHIAO?p0.99 128 sample 6416.000 ns/op InflateTests.testHIAO:testHIAO?p0.999 128 sample 8336.000 ns/op InflateTests.testHIAO:testHIAO?p0.9999 128 sample 12528.845 ns/op InflateTests.testHIAO:testHIAO?p1.00 128 sample 46336.000 ns/op InflateTests.testHIAO 1024 sample 161249 30745.929 ? 12.827 ns/op InflateTests.testHIAO:testHIAO?p0.00 1024 sample 28032.000 ns/op InflateTests.testHIAO:testHIAO?p0.50 1024 sample 30560.000 ns/op InflateTests.testHIAO:testHIAO?p0.90 1024 sample 31552.000 ns/op InflateTests.testHIAO:testHIAO?p0.95 1024 sample 32288.000 ns/op InflateTests.testHIAO:testHIAO?p0.99 1024 sample 37952.000 ns/op InflateTests.testHIAO:testHIAO?p0.999 1024 sample 47552.000 ns/op InflateTests.testHIAO:testHIAO?p0.9999 1024 sample 53752.000 ns/op InflateTests.testHIAO:testHIAO?p1.00 1024 sample 90752.000 ns/op InflateTests.testHIDO 16 sample 332361 891.695 ? 6.406 ns/op InflateTests.testHIDO:testHIDO?p0.00 16 sample 271.000 ns/op InflateTests.testHIDO:testHIDO?p0.50 16 sample 870.000 ns/op InflateTests.testHIDO:testHIDO?p0.90 16 sample 1080.000 ns/op InflateTests.testHIDO:testHIDO?p0.95 16 sample 1148.000 ns/op InflateTests.testHIDO:testHIDO?p0.99 16 sample 1440.000 ns/op InflateTests.testHIDO:testHIDO?p0.999 16 sample 3788.000 ns/op InflateTests.testHIDO:testHIDO?p0.9999 16 sample 7012.221 ns/op InflateTests.testHIDO:testHIDO?p1.00 16 sample 618496.000 ns/op InflateTests.testHIDO 128 sample 290853 4198.779 ? 2.973 ns/op InflateTests.testHIDO:testHIDO?p0.00 128 sample 3168.000 ns/op InflateTests.testHIDO:testHIDO?p0.50 128 sample 4136.000 ns/op InflateTests.testHIDO:testHIDO?p0.90 128 sample 4544.000 ns/op InflateTests.testHIDO:testHIDO?p0.95 128 sample 4720.000 ns/op InflateTests.testHIDO:testHIDO?p0.99 128 sample 6448.000 ns/op InflateTests.testHIDO:testHIDO?p0.999 128 sample 8400.000 ns/op InflateTests.testHIDO:testHIDO?p0.9999 128 sample 11518.634 ns/op InflateTests.testHIDO:testHIDO?p1.00 128 sample 47936.000 ns/op InflateTests.testHIDO 1024 sample 158712 31162.902 ? 17.625 ns/op InflateTests.testHIDO:testHIDO?p0.00 1024 sample 28128.000 ns/op InflateTests.testHIDO:testHIDO?p0.50 1024 sample 30848.000 ns/op InflateTests.testHIDO:testHIDO?p0.90 1024 sample 32192.000 ns/op InflateTests.testHIDO:testHIDO?p0.95 1024 sample 33216.000 ns/op InflateTests.testHIDO:testHIDO?p0.99 1024 sample 42048.000 ns/op InflateTests.testHIDO:testHIDO?p0.999 1024 sample 50560.000 ns/op InflateTests.testHIDO:testHIDO?p0.9999 1024 sample 59121.421 ns/op InflateTests.testHIDO:testHIDO?p1.00 1024 sample 323072.000 ns/op InflateTests.testHIHO 16 sample 318112 927.173 ? 1.529 ns/op InflateTests.testHIHO:testHIHO?p0.00 16 sample 322.000 ns/op InflateTests.testHIHO:testHIHO?p0.50 16 sample 907.000 ns/op InflateTests.testHIHO:testHIHO?p0.90 16 sample 1120.000 ns/op InflateTests.testHIHO:testHIHO?p0.95 16 sample 1192.000 ns/op InflateTests.testHIHO:testHIHO?p0.99 16 sample 1530.000 ns/op InflateTests.testHIHO:testHIHO?p0.999 16 sample 3772.000 ns/op InflateTests.testHIHO:testHIHO?p0.9999 16 sample 6131.019 ns/op InflateTests.testHIHO:testHIHO?p1.00 16 sample 48512.000 ns/op InflateTests.testHIHO 128 sample 287766 4252.329 ? 12.322 ns/op InflateTests.testHIHO:testHIHO?p0.00 128 sample 3188.000 ns/op InflateTests.testHIHO:testHIHO?p0.50 128 sample 4176.000 ns/op InflateTests.testHIHO:testHIHO?p0.90 128 sample 4592.000 ns/op InflateTests.testHIHO:testHIHO?p0.95 128 sample 4776.000 ns/op InflateTests.testHIHO:testHIHO?p0.99 128 sample 6674.640 ns/op InflateTests.testHIHO:testHIHO?p0.999 128 sample 9200.000 ns/op InflateTests.testHIHO:testHIHO?p0.9999 128 sample 14151.301 ns/op InflateTests.testHIHO:testHIHO?p1.00 128 sample 835584.000 ns/op InflateTests.testHIHO 1024 sample 159210 31110.602 ? 21.011 ns/op InflateTests.testHIHO:testHIHO?p0.00 1024 sample 28192.000 ns/op InflateTests.testHIHO:testHIHO?p0.50 1024 sample 30848.000 ns/op InflateTests.testHIHO:testHIHO?p0.90 1024 sample 32064.000 ns/op InflateTests.testHIHO:testHIHO?p0.95 1024 sample 33024.000 ns/op InflateTests.testHIHO:testHIHO?p0.99 1024 sample 38144.000 ns/op InflateTests.testHIHO:testHIHO?p0.999 1024 sample 49650.496 ns/op InflateTests.testHIHO:testHIHO?p0.9999 1024 sample 56719.149 ns/op InflateTests.testHIHO:testHIHO?p1.00 1024 sample 765952.000 ns/op From xueming.shen at oracle.com Thu Feb 22 20:51:17 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 22 Feb 2018 12:51:17 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <124fb31a-e508-0a01-98c7-dff9b345b3e6@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> <5A8DC7A9.30104@oracle.com> <5A8DCF9E.50508@oracle.com> <124fb31a-e508-0a01-98c7-dff9b345b3e6@oracle.com> Message-ID: <5A8F2D45.1070906@oracle.com> On 2/22/18, 12:04 PM, Joe Wang wrote: > Hi Sherman, > > Thanks for reviewing the change. > > Taking a local copy of the count field, but the boundary check would > be almost immediately done against the field itself. Are you worrying > about the count field may be out of sync with the byte array? I would > think that's unlikely to happen. Whether it's StringBuilder or > StringBuffer, it's not advisable/practical to use in multiple threads. > In a valid usage, the count is always consistent with the byte array. > Hi Joe, It might not be a "valid usage" but it did happen and when it happens it might just crash the vm without those boundary checks. It's especially true for those intrinsics methods with explicit comments "intrinsic performs no bounds checks". In this case, the StringUTF16.getChar() is being called in new public method StringUTF16.compareTo(byte[], byte[], int, int) without appropriate boundary check. In the "old" code the "index" is guaranteed to be within [0, len) in StringUTF16.compareTo(byte[], byte[]), so it's safe. A real case for such scenario can be found in JDK-8158168 [1], for example. -Sherman [1] https://bugs.openjdk.java.net/browse/JDK-8158168 From cushon at google.com Thu Feb 22 23:20:41 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Thu, 22 Feb 2018 15:20:41 -0800 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present Message-ID: Hello, Please consider this fix for JDK-7183985. bug: https://bugs.openjdk.java.net/browse/JDK-7183985 webrev: http://cr.openjdk.java.net/~cushon/7183985/webrev.01/ I started a CSR for the change: https://bugs.openjdk.java.net/browse/JDK-8198584 We have been using the fix at Google for about two years, and there has been no compatibility impact. I found very few places ArrayStoreException was being explicitly handled, and none that were depending on the current behaviour of getAnnotation(). There was some previous discussion of the bug on core-libs-dev: * http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-April/040590.html * http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-October/021798.html Thanks, Liam From david.holmes at oracle.com Fri Feb 23 00:34:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Feb 2018 10:34:52 +1000 Subject: [11] RFR JDK-8198441: Replace native Runtime::runFinalization0 method with shared secrets In-Reply-To: <9f48c420-f2c5-935f-a4e2-bbcc1721894b@oracle.com> References: <9f48c420-f2c5-935f-a4e2-bbcc1721894b@oracle.com> Message-ID: <63240244-725b-35df-8d15-045d118e2d69@oracle.com> Looks good. Thanks, David On 21/02/2018 3:45 AM, mandy chung wrote: > Webrev: > ? http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198441/webrev.00 > > This is a small cleanup that replaces the native Runtime::runFinalization0 > method with shared secrets to invoke Finalizer::runFinalization in java. > > Mandy > From huizhe.wang at oracle.com Fri Feb 23 02:07:14 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 22 Feb 2018 18:07:14 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A8F2D45.1070906@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> <5A8DC7A9.30104@oracle.com> <5A8DCF9E.50508@oracle.com> <124fb31a-e508-0a01-98c7-dff9b345b3e6@oracle.com> <5A8F2D45.1070906@oracle.com> Message-ID: <5A8F7752.9000302@oracle.com> On 2/22/18, 12:51 PM, Xueming Shen wrote: > On 2/22/18, 12:04 PM, Joe Wang wrote: >> Hi Sherman, >> >> Thanks for reviewing the change. >> >> Taking a local copy of the count field, but the boundary check would >> be almost immediately done against the field itself. Are you worrying >> about the count field may be out of sync with the byte array? I would >> think that's unlikely to happen. Whether it's StringBuilder or >> StringBuffer, it's not advisable/practical to use in multiple >> threads. In a valid usage, the count is always consistent with the >> byte array. >> > > Hi Joe, > > It might not be a "valid usage" but it did happen and when it happens > it might just crash the > vm without those boundary checks. It's especially true for those > intrinsics methods with explicit > comments "intrinsic performs no bounds checks". In this case, the > StringUTF16.getChar() is being > called in new public method StringUTF16.compareTo(byte[], byte[], int, > int) without appropriate > boundary check. In the "old" code the "index" is guaranteed to be > within [0, len) in > StringUTF16.compareTo(byte[], byte[]), so it's safe. A real case for > such scenario can be found in > JDK-8158168 [1], for example. Thanks for the pointer! The email thread helps a lot. I've updated the webrev with a boundary check in ASB (AbstractStringBuilder line 106, 107), and then a note to StringUTF16.compareTo (StringUTF16 line 280). Hopefully this is sufficient. Didn't want to add any check in StringUTF16 since that may affect the original two-arg method. JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ -Joe > > -Sherman > > > [1] https://bugs.openjdk.java.net/browse/JDK-8158168 From leonid.mesnik at oracle.com Fri Feb 23 02:41:08 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 22 Feb 2018 18:41:08 -0800 Subject: RFR: 8197901: Crash during GC when logging level is debug Message-ID: Hi Could you please review following fix which update implementation of Klass::external_name for anonymous classes. Previously external_name tried to add hashcode of corresponding java_mirror for InstanceKlass if it exists. However the java_mirror could be incorrect during GC. Also external_name might tries to calculate hash_code if it was not ?pre-calculated? during class verification. See JDK-8197442 [Graal] runtime/Metaspace/DefineClass.java crashes with "biases should not be seen by VM thread here" The suggested fix is to print address of corresponding InstanceKlass instead of hashcode. It allows to identify anonymous classes and allows to use external_name at any time. The hashcode for java_mirror is still pre-calculated in verifier.cpp since ik->java_mirror()->identity_hash() still might be used during safepoint. As a regression test I updated one of tests which redefine classes and easily reproduce problem when executed with full logging enabled. Test java/lang/StackWalker/VerifyStackTrace.java is update to match new pattern. webrev: http://cr.openjdk.java.net/~lmesnik/8197901/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8197901 Leonid From david.holmes at oracle.com Fri Feb 23 04:26:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Feb 2018 14:26:54 +1000 Subject: RFR: 8197901: Crash during GC when logging level is debug In-Reply-To: References: Message-ID: Hi Leonid, Looks fine. Please also add this bug id to @bug in test/jdk/java/lang/StackWalker/VerifyStackTrace.java Thanks, David On 23/02/2018 12:41 PM, Leonid Mesnik wrote: > Hi > > Could you please review following fix which update implementation of Klass::external_name for anonymous classes. > Previously external_name tried to add hashcode of corresponding java_mirror for InstanceKlass if it exists. However the java_mirror could be incorrect during GC. Also external_name might tries to calculate hash_code if it was not ?pre-calculated? during class verification. See JDK-8197442 [Graal] runtime/Metaspace/DefineClass.java crashes with "biases should not be seen by VM thread here" > > The suggested fix is to print address of corresponding InstanceKlass instead of hashcode. It allows to identify anonymous classes and allows to use external_name at any time. The hashcode for java_mirror is still pre-calculated in verifier.cpp since ik->java_mirror()->identity_hash() still might be used during safepoint. As a regression test I updated one of tests which redefine classes and easily reproduce problem when executed with full logging enabled. > Test java/lang/StackWalker/VerifyStackTrace.java is update to match new pattern. > > webrev: http://cr.openjdk.java.net/~lmesnik/8197901/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8197901 > > Leonid > From nishit.jain at oracle.com Fri Feb 23 08:33:28 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Fri, 23 Feb 2018 14:03:28 +0530 Subject: [11] RFR JDK-8060094: java/util/Formatter/Basic.java failed in tr locale In-Reply-To: <8118448f-b842-b802-3413-12e429878bd5@oracle.com> References: <8118448f-b842-b802-3413-12e429878bd5@oracle.com> Message-ID: <2c20ae94-7cbc-9991-8ec5-1aa3b92177db@oracle.com> Thanks Naoto, Please check the updated webrev http://cr.openjdk.java.net/~nishjain/8060094/webrev.04/ Edits made: In FormatLocale.java, clarified the exception messages about the locale used and removed an unused import. Regards, Nishit Jain On 23-02-2018 00:56, Naoto Sato wrote: > Hi Nishit, > > In the test case, the exception error messages may be more helpful if > there is a distinction between the two (line 103 and 120) mentioning > the formatter is using the default or specified locale. > > Naoto > > On 2/22/18 3:51 AM, Nishit Jain wrote: >> Hi, >> >> Please review the fix for JDK-8060094. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8060094 >> Webrev: http://cr.openjdk.java.net/~nishjain/8060094/webrev.03/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8197916 >> >> Cause: The Formatter APIs were not using the correct locale for upper >> casing when specified during instance creation or during format() >> call. The default locale was getting used irrespective of whether a >> Locale is specified in the API. >> Fix: Modified the APIs to use the locale specified. >> >> Regards, >> Nishit Jain From christoph.langer at sap.com Fri Feb 23 09:05:06 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 23 Feb 2018 09:05:06 +0000 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) In-Reply-To: References: <58519ccf88ac4517b6186af613e385fd@sap.com> Message-ID: Thanks, Thomas. From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] Sent: Donnerstag, 22. Februar 2018 09:54 To: Langer, Christoph Cc: Java Core Libs ; serviceability-dev at openjdk.java.net; andrew_m_leonard at uk.ibm.com Subject: Re: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) Hi Christoph, Looks fine. .. Thomas On Feb 22, 2018 09:42, "Langer, Christoph" > wrote: Hi, please review a simple import cleanup fix for java/util/jar/Attributes.java and sun/management/jdp/JdpController.java. Bug: https://bugs.openjdk.java.net/browse/JDK-8198539 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8198539.0/ Thanks and best regards Christoph From adam.farley at uk.ibm.com Fri Feb 23 12:09:17 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Fri, 23 Feb 2018 12:09:17 +0000 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: <4B429F1D-5727-4B20-A051-E39E1E8C69AA@oracle.com> References: <39D8F43A-06BD-483B-8901-6F4444A8235F@oracle.com> <4B429F1D-5727-4B20-A051-E39E1E8C69AA@oracle.com> Message-ID: Hi Paul, The larger picture for (read: effect of) these changes is best explained in my email here: http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-February/051441.html See the hyperlink I posted, and the few lines before it. Unfortunately the only understanding I have regarding the workings of the native code is would be derived from the OpenJ9 implimentation. I figured I wouldn't be thanked for posting that code here, so I posted what code I could share, with the additional note that the Hotspot native side of this should be implimented by: 1) Turning those Unsafe.java methods into native methods, and make them abstract (descriptor only, for the uninitiated). 2) Find the Hotspot native code for native memory allocation, reallocation, and freeing. Basically create a method that stores the sum total amount of native memory used by the DBBs, and then calls the regular allocate/reallocate/free methods. E.g. NM Allocate (Java) - NM Allocate (Native) DBB NM Allocate (Java) - DBB NM Allocate (Native) - NM allocate (Native) 3) Find the code that prints the current native memory usage in core files, and add a similar bit to show the native memory usage for DBBs as a subset (see the aforementioned linked link for an example). This seems like a straightforward task, though that's easy for me to say. :) Does that answer your question? Also, I'm unfamiliar with Java Flight Recorder. Are other developers on the list familiar with JFR that can snwer this? I'll put the message in IRC as well, and update here if I get any answers. Best Regards Adam Farley From: Paul Sandoz To: Adam Farley8 Cc: core-libs-dev , hotspot-dev developers Date: 22/02/2018 02:20 Subject: Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers Hi Adam, While the burden is minimal there is a principle here that i think we should adhere to regarding additions to the code base: additions should have value within OpenJDK itself otherwise it can become a thin end of the wedge to more stuff (?well you added these things, why not just add these too??). So i would still be reluctant to add such methods without understanding the larger picture and what you have in mind. Can you send a pointer to your email referring in more detail to the larger change sets? This use-case might also apply in other related areas too with regards to logging/monitoring. I would be interested to understand what Java Flight Recorder (JFR) does in this regard (it being open sourced soon i believe) and how JFR might relate to what you are doing. Should we be adding JFR events to unsafe memory allocation? Can JFR efficiently access part of the Java call stack to determine the origin? Thanks, Paul. On Feb 19, 2018, at 5:08 AM, Adam Farley8 wrote: Hi Paul, > Hi Adam, > > From reading the thread i cannot tell if this is part of a wider solution including some yet to be proposed HotSpot changes. The wider solution would need to include some Hotspot changes, yes. I'm proposing raising a bug, committing the code we have here to "set the stage", and then we can invest more time&energy later if the concept goes down well and the community agrees to pursue the full solution. As an aside, I tried submitting a big code set (including hotspot changes) months ago, and I'm *still* struggling to find someone to commit the thing, so I figured I'd try a more gradual, staged approach this time. > > As is i would be resistant to adding such standalone internal wrapper methods to Unsafe that have no apparent benefit within the OpenJDK itself since it's a maintenance burden. I'm hoping the fact that the methods are a single line (sans comments, descriptors and curly braces) will minimise this burden. > > Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come from a DBB by looking at the call stack frame above the unsafe call? > > Thanks, > Paul. Yes that is possible, though I would advise against this because: A) Checking the call stack is expensive, and doing this every time we allocate native memory is an easy way to slow down a program, or rack up mips. and B) deciding which code path we're using based on the stack means the DBB class+method (and anything the parsing code mistakes for that class+method) can only ever allocate native memory for DBBs. What do you think? Best Regards Adam Farley > >> On Feb 14, 2018, at 3:32 AM, Adam Farley8 wrote: >> >> Hi All, >> >> Currently, diagnostic core files generated from OpenJDK seem to lump all >> of the >> native memory usages together, making it near-impossible for someone to >> figure >> out *what* is using all that memory in the event of a memory leak. >> >> The OpenJ9 VM has a feature which allows it to track the allocation of >> native >> memory for Direct Byte Buffers (DBBs), and to supply that information into >> the >> cores when they are generated. This makes it a *lot* easier to find out >> what is using >> all that native memory, making memory leak resolution less like some dark >> art, and >> more like logical debugging. >> >> To use this feature, there is a native method referenced in Unsafe.java. >> To open >> up this feature so that any VM can make use of it, the java code below >> sets the >> stage for it. This change starts letting people call DBB-specific methods >> when >> allocating native memory, and getting into the habit of using it. >> >> Thoughts? >> >> Best Regards >> >> Adam Farley >> >> P.S. Code: >> >> diff --git >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> @@ -85,7 +85,7 @@ >> // Paranoia >> return; >> } >> - UNSAFE.freeMemory(address); >> + UNSAFE.freeDBBMemory(address); >> address = 0; >> Bits.unreserveMemory(size, capacity); >> } >> @@ -118,7 +118,7 @@ >> >> long base = 0; >> try { >> - base = UNSAFE.allocateMemory(size); >> + base = UNSAFE.allocateDBBMemory(size); >> } catch (OutOfMemoryError x) { >> Bits.unreserveMemory(size, cap); >> throw x; >> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java >> @@ -632,6 +632,26 @@ >> } >> >> /** >> + * Allocates a new block of native memory for DirectByteBuffers, of >> the >> + * given size in bytes. The contents of the memory are >> uninitialized; >> + * they will generally be garbage. The resulting native pointer will >> + * never be zero, and will be aligned for all value types. Dispose >> of >> + * this memory by calling {@link #freeDBBMemory} or resize it with >> + * {@link #reallocateDBBMemory}. >> + * >> + * @throws RuntimeException if the size is negative or too large >> + * for the native size_t type >> + * >> + * @throws OutOfMemoryError if the allocation is refused by the >> system >> + * >> + * @see #getByte(long) >> + * @see #putByte(long, byte) >> + */ >> + public long allocateDBBMemory(long bytes) { >> + return allocateMemory(bytes); >> + } >> + >> + /** >> * Resizes a new block of native memory, to the given size in bytes. >> The >> * contents of the new block past the size of the old block are >> * uninitialized; they will generally be garbage. The resulting >> native >> @@ -687,6 +707,27 @@ >> } >> >> /** >> + * Resizes a new block of native memory for DirectByteBuffers, to the >> + * given size in bytes. The contents of the new block past the size >> of >> + * the old block are uninitialized; they will generally be garbage. >> The >> + * resulting native pointer will be zero if and only if the requested >> size >> + * is zero. The resulting native pointer will be aligned for all >> value >> + * types. Dispose of this memory by calling {@link #freeDBBMemory}, >> or >> + * resize it with {@link #reallocateDBBMemory}. The address passed >> to >> + * this method may be null, in which case an allocation will be >> performed. >> + * >> + * @throws RuntimeException if the size is negative or too large >> + * for the native size_t type >> + * >> + * @throws OutOfMemoryError if the allocation is refused by the >> system >> + * >> + * @see #allocateDBBMemory >> + */ >> + public long reallocateDBBMemory(long address, long bytes) { >> + return reallocateMemory(address, bytes); >> + } >> + >> + /** >> * Sets all bytes in a given block of memory to a fixed value >> * (usually zero). >> * >> @@ -918,6 +959,17 @@ >> checkPointer(null, address); >> } >> >> + /** >> + * Disposes of a block of native memory, as obtained from {@link >> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The address >> passed >> + * to this method may be null, in which case no action is taken. >> + * >> + * @see #allocateDBBMemory >> + */ >> + public void freeDBBMemory(long address) { >> + freeMemory(address); >> + } >> + >> /// random queries >> >> /** >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Alan.Bateman at oracle.com Fri Feb 23 14:28:47 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 23 Feb 2018 14:28:47 +0000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: Just getting to the updated webrev now. On 21/02/2018 20:30, Martin Buchholz wrote: > : > > > 8198480: Improve ClassLoaders static init block > http://cr.openjdk.java.net/~martin/webrevs/jdk/ClassLoaders-static/ > > https://bugs.openjdk.java.net/browse/JDK-8198480 The value of jdk.module.main is the name of the "initial module" so better to use that instead term of "main module". > > 8198481: Coding style cleanups for > src/java.base/share/classes/jdk/internal/loader > http://cr.openjdk.java.net/~martin/webrevs/jdk/loader-style/ > > https://bugs.openjdk.java.net/browse/JDK-8198481 Looks okay. > > 8198482: The URLClassPath field "urls" should be renamed to "unopenedUrls" > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-rename-urls/ > > https://bugs.openjdk.java.net/browse/JDK-8198482 Looks okay. > > 8198484: URLClassPath should use an ArrayDeque instead of a Stack > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ > > https://bugs.openjdk.java.net/browse/JDK-8198484 Can copyToArrayDeque use addAll? > > 8198485: Simplify a URLClassPath constructor > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-simplify-constructor/ > > https://bugs.openjdk.java.net/browse/JDK-8198485 > Looks okay. From peter.levart at gmail.com Fri Feb 23 15:28:34 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 23 Feb 2018 16:28:34 +0100 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: <39D8F43A-06BD-483B-8901-6F4444A8235F@oracle.com> <4B429F1D-5727-4B20-A051-E39E1E8C69AA@oracle.com> Message-ID: Hi Adam, Did you know that native memory is already tracked on the Java side for direct ByteBuffers? See class java.nio.Bits. Could you make use of it? Regards, Peter On 23 Feb 2018 1:09 pm, "Adam Farley8" wrote: > Hi Paul, > > The larger picture for (read: effect of) these changes is best explained > in my email here: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018- > February/051441.html > > See the hyperlink I posted, and the few lines before it. > > Unfortunately the only understanding I have regarding the workings of the > native code > is would be derived from the OpenJ9 implimentation. I figured I wouldn't > be thanked for > posting that code here, so I posted what code I could share, with the > additional note > that the Hotspot native side of this should be implimented by: > > 1) Turning those Unsafe.java methods into native methods, and make them > abstract > (descriptor only, for the uninitiated). > > 2) Find the Hotspot native code for native memory allocation, > reallocation, and > freeing. Basically create a method that stores the sum total amount of > native memory > used by the DBBs, and then calls the regular allocate/reallocate/free > methods. > > E.g. > > NM Allocate (Java) - NM Allocate (Native) > > DBB NM Allocate (Java) - DBB NM Allocate (Native) - NM allocate (Native) > > 3) Find the code that prints the current native memory usage in core > files, and > add a similar bit to show the native memory usage for DBBs as a subset > (see the > aforementioned linked link for an example). > > This seems like a straightforward task, though that's easy for me to say. > :) > > Does that answer your question? > > Also, I'm unfamiliar with Java Flight Recorder. Are other developers on > the list > familiar with JFR that can snwer this? I'll put the message in IRC as > well, and > update here if I get any answers. > > Best Regards > > Adam Farley > > > > From: Paul Sandoz > To: Adam Farley8 > Cc: core-libs-dev , hotspot-dev > developers > Date: 22/02/2018 02:20 > Subject: Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track > Native Memory Usage for Direct Byte Buffers > > > > Hi Adam, > > While the burden is minimal there is a principle here that i think we > should adhere to regarding additions to the code base: additions should > have value within OpenJDK itself otherwise it can become a thin end of the > wedge to more stuff (?well you added these things, why not just add these > too??). > > So i would still be reluctant to add such methods without understanding > the larger picture and what you have in mind. > > Can you send a pointer to your email referring in more detail to the > larger change sets? > > This use-case might also apply in other related areas too with regards to > logging/monitoring. I would be interested to understand what Java Flight > Recorder (JFR) does in this regard (it being open sourced soon i believe) > and how JFR might relate to what you are doing. Should we be adding JFR > events to unsafe memory allocation? Can JFR efficiently access part of the > Java call stack to determine the origin? > > Thanks, > Paul. > > On Feb 19, 2018, at 5:08 AM, Adam Farley8 wrote: > > Hi Paul, > > > Hi Adam, > > > > From reading the thread i cannot tell if this is part of a wider > solution including some yet to be proposed HotSpot changes. > > The wider solution would need to include some Hotspot changes, yes. > I'm proposing raising a bug, committing the code we have here to > "set the stage", and then we can invest more time&energy later > if the concept goes down well and the community agrees to pursue > the full solution. > > As an aside, I tried submitting a big code set (including hotspot > changes) months ago, and I'm *still* struggling to find someone to > commit the thing, so I figured I'd try a more gradual, staged approach > this time. > > > > > As is i would be resistant to adding such standalone internal wrapper > methods to Unsafe that have no apparent benefit within the OpenJDK itself > since it's a maintenance burden. > > I'm hoping the fact that the methods are a single line (sans > comments, descriptors and curly braces) will minimise this burden. > > > > > Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come > from a DBB by looking at the call stack frame above the unsafe call? > > > > Thanks, > > Paul. > > Yes that is possible, though I would advise against this because: > > A) Checking the call stack is expensive, and doing this every time we > allocate native memory is an easy way to slow down a program, > or rack up mips. > and > B) deciding which code path we're using based on the stack > means the DBB class+method (and anything the parsing code > mistakes for that class+method) can only ever allocate native > memory for DBBs. > > What do you think? > > Best Regards > > Adam Farley > > > > >> On Feb 14, 2018, at 3:32 AM, Adam Farley8 > wrote: > >> > >> Hi All, > >> > >> Currently, diagnostic core files generated from OpenJDK seem to lump > all > >> of the > >> native memory usages together, making it near-impossible for someone to > > >> figure > >> out *what* is using all that memory in the event of a memory leak. > >> > >> The OpenJ9 VM has a feature which allows it to track the allocation of > >> native > >> memory for Direct Byte Buffers (DBBs), and to supply that information > into > >> the > >> cores when they are generated. This makes it a *lot* easier to find out > > >> what is using > >> all that native memory, making memory leak resolution less like some > dark > >> art, and > >> more like logical debugging. > >> > >> To use this feature, there is a native method referenced in > Unsafe.java. > >> To open > >> up this feature so that any VM can make use of it, the java code below > >> sets the > >> stage for it. This change starts letting people call DBB-specific > methods > >> when > >> allocating native memory, and getting into the habit of using it. > >> > >> Thoughts? > >> > >> Best Regards > >> > >> Adam Farley > >> > >> P.S. Code: > >> > >> diff --git > >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >> --- > a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >> +++ > b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > >> @@ -85,7 +85,7 @@ > >> // Paranoia > >> return; > >> } > >> - UNSAFE.freeMemory(address); > >> + UNSAFE.freeDBBMemory(address); > >> address = 0; > >> Bits.unreserveMemory(size, capacity); > >> } > >> @@ -118,7 +118,7 @@ > >> > >> long base = 0; > >> try { > >> - base = UNSAFE.allocateMemory(size); > >> + base = UNSAFE.allocateDBBMemory(size); > >> } catch (OutOfMemoryError x) { > >> Bits.unreserveMemory(size, cap); > >> throw x; > >> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java > >> @@ -632,6 +632,26 @@ > >> } > >> > >> /** > >> + * Allocates a new block of native memory for DirectByteBuffers, > of > >> the > >> + * given size in bytes. The contents of the memory are > >> uninitialized; > >> + * they will generally be garbage. The resulting native pointer > will > >> + * never be zero, and will be aligned for all value types. Dispose > > >> of > >> + * this memory by calling {@link #freeDBBMemory} or resize it with > > >> + * {@link #reallocateDBBMemory}. > >> + * > >> + * @throws RuntimeException if the size is negative or too large > >> + * for the native size_t type > >> + * > >> + * @throws OutOfMemoryError if the allocation is refused by the > >> system > >> + * > >> + * @see #getByte(long) > >> + * @see #putByte(long, byte) > >> + */ > >> + public long allocateDBBMemory(long bytes) { > >> + return allocateMemory(bytes); > >> + } > >> + > >> + /** > >> * Resizes a new block of native memory, to the given size in > bytes. > >> The > >> * contents of the new block past the size of the old block are > >> * uninitialized; they will generally be garbage. The resulting > >> native > >> @@ -687,6 +707,27 @@ > >> } > >> > >> /** > >> + * Resizes a new block of native memory for DirectByteBuffers, to > the > >> + * given size in bytes. The contents of the new block past the > size > >> of > >> + * the old block are uninitialized; they will generally be > garbage. > >> The > >> + * resulting native pointer will be zero if and only if the > requested > >> size > >> + * is zero. The resulting native pointer will be aligned for all > >> value > >> + * types. Dispose of this memory by calling {@link > #freeDBBMemory}, > >> or > >> + * resize it with {@link #reallocateDBBMemory}. The address > passed > >> to > >> + * this method may be null, in which case an allocation will be > >> performed. > >> + * > >> + * @throws RuntimeException if the size is negative or too large > >> + * for the native size_t type > >> + * > >> + * @throws OutOfMemoryError if the allocation is refused by the > >> system > >> + * > >> + * @see #allocateDBBMemory > >> + */ > >> + public long reallocateDBBMemory(long address, long bytes) { > >> + return reallocateMemory(address, bytes); > >> + } > >> + > >> + /** > >> * Sets all bytes in a given block of memory to a fixed value > >> * (usually zero). > >> * > >> @@ -918,6 +959,17 @@ > >> checkPointer(null, address); > >> } > >> > >> + /** > >> + * Disposes of a block of native memory, as obtained from {@link > >> + * #allocateDBBMemory} or {@link #reallocateDBBMemory}. The > address > >> passed > >> + * to this method may be null, in which case no action is taken. > >> + * > >> + * @see #allocateDBBMemory > >> + */ > >> + public void freeDBBMemory(long address) { > >> + freeMemory(address); > >> + } > >> + > >> /// random queries > >> > >> /** > >> > >> Unless stated otherwise above: > >> IBM United Kingdom Limited - Registered in England and Wales with > number > >> 741598. > >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU > > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > From leventov.ru at gmail.com Fri Feb 23 16:30:36 2018 From: leventov.ru at gmail.com (Roman Leventov) Date: Fri, 23 Feb 2018 17:30:36 +0100 Subject: [PATCH] Optimization of AbstractStringBuilder.ensureCapacityInternal() In-Reply-To: <1eeb46c7-1e02-1944-9620-ccebc104c071@oracle.com> References: <1eeb46c7-1e02-1944-9620-ccebc104c071@oracle.com> Message-ID: Hi Claes, indeed, seems that this change breaks the zeroing optimization, so ensureCapacityInternal() becomes slower when the char count is comparable with the array length. Thanks. On 22 February 2018 at 18:35, Claes Redestad wrote: > Hi, > > interesting - do you have any numbers showing a benefit from doing this > (on various sets of input)? > > My concerns are that count might typically be close enough to value.length > to make the difference small in practice, and that there are some (fragile) > JIT optimizations to try and avoid zeroing out the newly allocated array > that we have to take care not to break. > > /Claes > > > On 2018-02-22 17:51, Roman Leventov wrote: > >> AbstractStringBuilder.ensureCapacityInternal() doesn't need to copy the >> whole underlying array, only the part until the current count. >> >> diff --git >> a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> --- a/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> +++ b/src/java.base/share/classes/java/lang/AbstractStringBuilder.java >> @@ -140,11 +140,12 @@ >> * overflow, this method throws {@code OutOfMemoryError}. >> */ >> private void ensureCapacityInternal(int minimumCapacity) { >> + byte[] oldValue = value; >> // overflow-conscious code >> - int oldCapacity = value.length >> coder; >> + int oldCapacity = oldValue.length >> coder; >> if (minimumCapacity - oldCapacity > 0) { >> - value = Arrays.copyOf(value, >> - newCapacity(minimumCapacity) << coder); >> + value = new byte[newCapacity(minimumCapacity) << coder]; >> + System.arraycopy(oldValue, 0, value, 0, count << coder); >> } >> } >> > > From Alan.Bateman at oracle.com Fri Feb 23 17:39:20 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 23 Feb 2018 17:39:20 +0000 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: Message-ID: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> On 22/02/2018 23:20, Liam Miller-Cushon wrote: > Hello, > > > Please consider this fix for JDK-7183985. > > > bug: https://bugs.openjdk.java.net/browse/JDK-7183985 > > webrev: http://cr.openjdk.java.net/~cushon/7183985/webrev.01/ > > > I started a CSR for the change: > https://bugs.openjdk.java.net/browse/JDK-8198584 > > We have been using the fix at Google for about two years, and there has > been no compatibility impact. I found very few places ArrayStoreException > was being explicitly handled, and none that were depending on the current > behaviour of getAnnotation(). > > > There was some previous discussion of the bug on core-libs-dev: > Yes, this one comes up every few years. I'm hoping Joe Darcy will reply to your review with any background or issues from when this came up in the past. From a distance then retrofitting AnnotatedElement getXXX methods to throw TypeNotPresentException seems reasonable, I'm less sure about the isAnnotationPresent method as it might be surprising for that to fail. -Alan From Alan.Bateman at oracle.com Fri Feb 23 17:44:20 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 23 Feb 2018 17:44:20 +0000 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: <39D8F43A-06BD-483B-8901-6F4444A8235F@oracle.com> <4B429F1D-5727-4B20-A051-E39E1E8C69AA@oracle.com> Message-ID: <712770ae-72ee-10b8-cd93-fb1fc34bd5b3@oracle.com> On 23/02/2018 15:28, Peter Levart wrote: > Hi Adam, > > Did you know that native memory is already tracked on the Java side for > direct ByteBuffers? See class java.nio.Bits. Could you make use of it? > Right, these are the fields that are exposed at runtime via BufferPoolMXBean. A SA based tool could read from a core file. I can't tell if this is enough for Adam, it may be that the his tool reveals more details on the buffers in the pools. -Alan From xuelei.fan at oracle.com Fri Feb 23 17:46:45 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 23 Feb 2018 09:46:45 -0800 Subject: RFR 8181594: Efficient and constant-time modular arithmetic In-Reply-To: <7bd6c1ae-4173-4384-fc57-66d1b185ba56@oracle.com> References: <7bd6c1ae-4173-4384-fc57-66d1b185ba56@oracle.com> Message-ID: ArrayUtil.java: =============== I'm not very sure how widely this utilities will be used in the future. Looks like only BigIntegerModuloP uses this classes. I may prefer to define private methods for byte array swap in BigIntegerModuloP. BigIntegerModuloP.java: ======================= As this is a class for testing or ptototype purpose, it might not be a part of JDK products, like JRE. Would you mind move it to a test package if you want to keep it? IntegerModuloP, IntegerModuloP_Base and MutableIntegerModuloP ============================================================= In the security package/context, it may make sense to use "IntegerModulo" for the referring to "integers modulo a prime value". The class name of "IntegerModuloP_Base" is not a general Java coding style. I may prefer a little bit name changes like: IntegerModuloP_Base -> IntegerModulo IntegerModuloP -> ImmutableIntegerModulo MutableIntegerModuloP -> MutableIntegerModulo IntegerFieldModuloP -> IntegerModuloField (?) MutableIntegerModuloP.java ========================== void conditionalSwapWith(MutableIntegerModuloP b, int swap); As the 'swap' parameter can only be 0 or 1, could it be a boolean parameter? Except the conditionalSwapWith() method, I did not get the points why we need a mutable version. Would you please have more description of this requirement? IntegerModuloP_Base.java ======================== default byte[] addModPowerTwo(IntegerModuloP_Base b, int len) void addModPowerTwo(IntegerModuloP_Base b, byte[] result); For the first sign of the method names, I thought it is to calculate as "(this + b) ^ 2 mod m". Besides, what's the benefits of the two methods? Could we just use: this.add(b).asByteArray() I guess, but not very sure, it is for constant time calculation. If the function is required, could it be renamed as: // the result is inside of the size range IntegerModuloP addModSize(IntegerModuloP_Base b, int size) Or // the result is wrapped if outside of the size range IntegerModuloP addOnWrap(IntegerModuloP_Base b, int size) and the use may look like: this.addModSize(b, size).asByteArray() Will review the rest when I understand more about the interfaces design. Thanks, Xuelei On 1/30/2018 8:52 AM, Adam Petcher wrote: > +core-libs-dev > > > On 1/26/2018 4:06 PM, Adam Petcher wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8181594 >> Webrev: http://cr.openjdk.java.net/~apetcher/8181594/webrev.00/ >> >> This is a code review for the field arithmetic that will be used in >> implementations of X25519/X448 key agreement, the Poly1305 >> authenticator, and EdDSA signatures. I believe that the library has >> all the features necessary for X25519/X448 and Poly1305, and I expect >> at most a couple of minor enhancements will be required to support >> EdDSA. There is no public API for this library, so we can change it in >> the future to suit the needs of new algorithms without breaking >> compatibility with external code. Still, I made an attempt to clearly >> structure and document the (internal) API, and I want to make sure it >> is understandable and easy to use. >> >> This is not a general-purpose modular arithmetic library. It will only >> work well in circumstances where the sequence of operations is >> restricted, and where the prime that defines the field has some useful >> structure. Moreover, each new field will require some field-specific >> code that takes into account the structure of the prime and the way >> the field is used in the application. The initial implementation >> includes a field for Poly1305 and the fields for X25519/X448 which >> should also work for EdDSA. >> >> The benefits of using this library are that it is much more efficient >> than using similar operations in BigInteger. Also, many operations are >> branch-free, making them suitable for use in a side-channel resistant >> implementation that does not branch on secrets. >> >> To provide some context, I have attached a code snippet describing how >> this library can be used. The snippet is the constant-time Montgomery >> ladder from my X25519/X448 implementation, which I expect to be out >> for review soon. X25519/X448 only uses standard arithmetic operations, >> and the more unusual features (e.g. add modulo a power of 2) are >> needed by Poly1305. >> >> The field arithmetic (for all fields) is implemented using a 32-bit >> representation similar to the one described in the Ed448 paper[1] (in >> the "Implementation on 32-bit platforms" section). Though my >> implementation uses signed limbs, and grade-school multiplication >> instead of Karatsuba. The argument for correctness is essentially the >> same for all three fields: the magnitude of each 64-bit limb is at >> most 2^(k-1) after reduction, except for the last limb which may have >> a magnitude of up to 2^k. The values of k are between 26 to 28 >> (depending on the field), and we can calculate that the maximum >> magnitude for any limb during an add-multiply-carry-reduce sequence is >> always less than 2^63. Therefore, no overflow occurs and all >> operations are correct. >> >> Process note: this enhancement is part of JEP 324 (Key Agreement with >> Curve25519 and Curve448). When this code review is complete, nothing >> will happen until all other work for this JEP is complete, and the JEP >> is accepted as part of some release. This means that this code will be >> pushed to the repo along with the X25519/X448 code that uses it. >> >> [1] https://eprint.iacr.org/2015/625.pdf >> >> >> > From naoto.sato at oracle.com Fri Feb 23 17:40:01 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 23 Feb 2018 09:40:01 -0800 Subject: [11] RFR JDK-8060094: java/util/Formatter/Basic.java failed in tr locale In-Reply-To: <2c20ae94-7cbc-9991-8ec5-1aa3b92177db@oracle.com> References: <8118448f-b842-b802-3413-12e429878bd5@oracle.com> <2c20ae94-7cbc-9991-8ec5-1aa3b92177db@oracle.com> Message-ID: +1 Naoto On 2/23/18 12:33 AM, Nishit Jain wrote: > Thanks Naoto, > > Please check the updated webrev > http://cr.openjdk.java.net/~nishjain/8060094/webrev.04/ > > Edits made: In FormatLocale.java, clarified the exception messages about > the locale used and removed an unused import. > > Regards, > Nishit Jain > On 23-02-2018 00:56, Naoto Sato wrote: >> Hi Nishit, >> >> In the test case, the exception error messages may be more helpful if >> there is a distinction between the two (line 103 and 120) mentioning >> the formatter is using the default or specified locale. >> >> Naoto >> >> On 2/22/18 3:51 AM, Nishit Jain wrote: >>> Hi, >>> >>> Please review the fix for JDK-8060094. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8060094 >>> Webrev: http://cr.openjdk.java.net/~nishjain/8060094/webrev.03/ >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197916 >>> >>> Cause: The Formatter APIs were not using the correct locale for upper >>> casing when specified during instance creation or during format() >>> call. The default locale was getting used irrespective of whether a >>> Locale is specified in the API. >>> Fix: Modified the APIs to use the locale specified. >>> >>> Regards, >>> Nishit Jain > From joe.darcy at oracle.com Fri Feb 23 18:06:29 2018 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 23 Feb 2018 10:06:29 -0800 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> Message-ID: On 2/23/2018 9:39 AM, Alan Bateman wrote: > On 22/02/2018 23:20, Liam Miller-Cushon wrote: >> Hello, >> >> >> Please consider this fix for JDK-7183985. >> >> >> bug: https://bugs.openjdk.java.net/browse/JDK-7183985 >> >> webrev: http://cr.openjdk.java.net/~cushon/7183985/webrev.01/ >> >> >> I started a CSR for the change: >> https://bugs.openjdk.java.net/browse/JDK-8198584 >> >> We have been using the fix at Google for about two years, and there has >> been no compatibility impact. I found very few places >> ArrayStoreException >> was being explicitly handled, and none that were depending on the >> current >> behaviour of getAnnotation(). >> >> >> There was some previous discussion of the bug on core-libs-dev: >> > Yes, this one comes up every few years. I'm hoping Joe Darcy will > reply to your review with any background or issues from when this came > up in the past. From a distance then retrofitting AnnotatedElement > getXXX methods to throw TypeNotPresentException seems reasonable, I'm > less sure about the isAnnotationPresent method as it might be > surprising for that to fail. > On my list! Cheers, -Joe From Roger.Riggs at Oracle.com Fri Feb 23 19:39:07 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 23 Feb 2018 14:39:07 -0500 Subject: RFR 8198645 Use System.lineSeparator() instead of getProperty("line.separator") Message-ID: Please review cleanup replacements of System.getProperty("line.separator") with System.lineSeparator(). It uses the line separator from System instead of looking it up in the properties each time. Also fixed one javadoc @see reference. The affected files are in several packages: com/sun/crypto/provider/ java/util/regex/ jdk/internal/util/xml/impl/ Webrev: ? http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ Thanks, Roger From lance.andersen at oracle.com Fri Feb 23 19:42:26 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 23 Feb 2018 14:42:26 -0500 Subject: RFR 8198645 Use System.lineSeparator() instead of getProperty("line.separator") In-Reply-To: References: Message-ID: <98AC376E-5DDF-4E21-B198-0A3DE4777CAF@oracle.com> Looks good Roger > On Feb 23, 2018, at 2:39 PM, Roger Riggs wrote: > > Please review cleanup replacements of System.getProperty("line.separator") with System.lineSeparator(). > It uses the line separator from System instead of looking it up in the properties each time. > Also fixed one javadoc @see reference. > > The affected files are in several packages: > > com/sun/crypto/provider/ > java/util/regex/ > jdk/internal/util/xml/impl/ > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ > > Thanks, Roger > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From xuelei.fan at oracle.com Fri Feb 23 19:46:02 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 23 Feb 2018 11:46:02 -0800 Subject: RFR 8198645 Use System.lineSeparator() instead of getProperty("line.separator") In-Reply-To: References: Message-ID: <6e3c1ae2-5f3b-8cc5-4ab3-6c1cfda7d790@oracle.com> Looks fine to me. Thanks! Xuelei On 2/23/2018 11:39 AM, Roger Riggs wrote: > Please review cleanup replacements of > System.getProperty("line.separator") with System.lineSeparator(). > It uses the line separator from System instead of looking it up in the > properties each time. > Also fixed one javadoc @see reference. > > The affected files are in several packages: > > ?? com/sun/crypto/provider/ > ?? java/util/regex/ > ?? jdk/internal/util/xml/impl/ > > Webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ > > Thanks, Roger > > From xueming.shen at oracle.com Fri Feb 23 19:50:57 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 23 Feb 2018 11:50:57 -0800 Subject: RFR 8198645 Use System.lineSeparator() instead of getProperty("line.separator") In-Reply-To: References: Message-ID: <5A9070A1.1030909@oracle.com> +1 On 02/23/2018 11:39 AM, Roger Riggs wrote: > Please review cleanup replacements of System.getProperty("line.separator") with System.lineSeparator(). > It uses the line separator from System instead of looking it up in the properties each time. > Also fixed one javadoc @see reference. > > The affected files are in several packages: > > com/sun/crypto/provider/ > java/util/regex/ > jdk/internal/util/xml/impl/ > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ > > Thanks, Roger > > From Roger.Riggs at Oracle.com Fri Feb 23 19:58:07 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 23 Feb 2018 14:58:07 -0500 Subject: JDK-6372077: JarFile.getManifest() should handle manifest attribute names up to 70 bytes In-Reply-To: References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> <02abcb68-2294-c733-5048-b1f81ee99435@oracle.com> <88b89995-cfe0-d439-d63d-699c628e2315@paratix.ch> <07e62d8d-ae3a-9ffa-d426-81ab548ad252@Oracle.com> <9a08b2c0-3659-1833-f113-23aaceca84bc@Oracle.com> Message-ID: <39bd4f58-bdf6-8fff-4c21-37c2b1f4a340@Oracle.com> Please review this contribution from Philipp Kunz to handle manifest attribute names up to 70 bytes. The change passes the available regression tests. Manifest handling is somewhat sensitive so an additional review is appreciated. Webrev: (rebased from the original patch of 2/22/18) ??? http://cr.openjdk.java.net/~rriggs/webrev-jar-6372077.patch/ Issue: ?? https://bugs.openjdk.java.net/browse/JDK-6372077 Thanks, Roger On 2/2/2018 1:52 PM, Philipp Kunz wrote: > Hi Roger > > Glad to send the patch. > I also tried to write a meaningful and useful test. Please tell me > ruthlessly if it makes sense or what not. > Looking forward to progress in a bug that has been open for more than > 10 years now. > > Philipp > > > On 22.01.2018 21:03, Roger Riggs wrote: >> Hi Philipp, >> >> I'm tending to agree with the suggestion about line length >> interpretation. >> To meet OpenJDK IP requirements, please attach the .patch file or >> include it in the body >> of the message. >> >> Thanks, Roger >> >> On 12/18/2017 11:17 PM, Philipp Kunz wrote: >>> Hi Roger, >>> >>> My suggested and also preferred approach is to read the manifest >>> specification [1] in a way such that the line breaks are not >>> included when counting the maximum line length. The specification >>> does not state explicitly whether or not to include line breaks >>> within the maximum line length limit but the following sentence from >>> the specifications gives a hint: >>> >>> ??? Because header names cannot be continued, the maximum length of a >>> ??? header name is 70 bytes (there must be a colon and a SPACE after >>> the >>> ??? name). >>> >>> Above statement can be true only if line breaks are not counted for >>> the maximum line length limit. Assuming so would in my opinion allow >>> to understand the complete manifest specification without a conflict >>> and effectively result in wider manifest files (maximum each line), >>> wider by two bytes of a line break. >>> >>> In the meantime since the mail you replied to, I created a patch [3] >>> mentioned in [2] which might be useful provided the manifest >>> specification discussion is resolved. >>> >>> Regards, >>> Philipp >>> >>> >>> [1] >>> https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Notes_on_Manifest_and_Signature_Files >>> / >>> https://docs.oracle.com/javase/9/docs/specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files >>> [2] >>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050500.html >>> [3] http://files.paratix.ch/jdk/6372077/webrev.01/ >>> >>> >>> >>> On 18.12.2017 16:46, Roger Riggs wrote: >>>> Hi Phillip, >>>> >>>> Sorry for the silence... >>>> >>>> I/we haven't had time to full understand the ramifications of the >>>> change you propose. >>>> It seems there is a difficult/unresolvable conflict in the >>>> specifications between the line length >>>> requirements and the header specs. >>>> >>>> Regards, Roger >>>> >>>> On 11/21/2017 1:18 AM, Philipp Kunz wrote: >>>>> Hi everyone, >>>>> >>>>> I haven't got any reply now for around three weeks and now i start >>>>> to wonder if I just missed it or if I should refine my approach to >>>>> find a sponsor or if it helped if I presented a ready patch or if >>>>> noone considers this important enough or anything else whatever. >>>>> This is only my second contribution hence I don't know the >>>>> procedure well. >>>>> >>>>> One point maybe worth to point out again is that I don't want to >>>>> support manifest headers longer than 70 character, just up to 70, >>>>> which has always been the intention but has only worked up to 68. >>>>> This might have been written confusingly in my last email. >>>>> >>>>> As far as I understood, I should first get a sponsor. In any case, >>>>> is there any suggestion for how to proceed? >>>>> >>>>> Regards, >>>>> Philipp >>>>> >>>>> >>>>> >>>>> On 03.11.2017 00:04, Philipp Kunz wrote: >>>>>> Hi Sean and Max and all others, >>>>>> >>>>>> Thank you Sean for the hint about the right mailing list. And >>>>>> thanks also for his hint to Max to make smaller portions of changes. >>>>>> >>>>>> I would like to contribute a fix for JDK-6372077 which is about >>>>>> JarFile.getManifest() should handle manifest attribute name[s >>>>>> longer than] 70 bytes. >>>>>> >>>>>> It looks like the bug is caused by Manifest.make72Safe breaking >>>>>> lines at 70 bytes instead of 72 despite its comment and name >>>>>> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Manifest.java#l176).The >>>>>> resulting StringBuffer has then lines of 72 bytes maximum each >>>>>> including the following line break. Without the line break that >>>>>> leaves 70 bytes of characters per line. On the other hand, header >>>>>> names can be up to 70 bytes (only single-byte utf-8 characters) >>>>>> and cannot be broken across a line break and need to be followed >>>>>> by a colon and a space which must be on the same line too >>>>>> according to the specification. When breaking at 70 bytes >>>>>> excluding the line break, however, long header names don't fit in >>>>>> one line together with the colon space delimiter because there is >>>>>> not sufficient space. >>>>>> Manifests with names up to 70 bytes long can still be written >>>>>> without immediate exception but the resulting manifests are >>>>>> illegal in my opinion. When later reading such manifests >>>>>> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Attributes.java#l406), >>>>>> an error occurs as a consequence of the bad manifest. This is >>>>>> more or less the current situation and also what JDK-6372077 >>>>>> already knew. >>>>>> >>>>>> ?--> After all, in order to fix it, i'd like to propose to make >>>>>> manifest file lines wider by two bytes. >>>>>> >>>>>> The only proper alternative that came into my mind would be to >>>>>> change the manifest specification and reduce the maximum header >>>>>> name length there by two and also in the code. If that would >>>>>> break any existing code i guess that would affect code only that >>>>>> produced invalid manifests and would be acceptable. >>>>>> >>>>>> Supporting all existing and possibly invalid manifests would mean >>>>>> to add support for reading headers the delimiters of which are >>>>>> broken onto the next line which I consider too complex with >>>>>> respect to the value added and even more so considering that >>>>>> those invalid manifest can be assumed to have been detected as >>>>>> such by reading them and also because it would be a feature that >>>>>> would be used less and less over time if the code to write >>>>>> manifest is changed at the same time to produce only valid >>>>>> manifests in the discussed respect here. I don't think this >>>>>> should be actually done. >>>>>> >>>>>> Before I actually do the leg work, i'd like to ask, if there are >>>>>> concerns or objections or tips for such a change or if anyone can >>>>>> or cannot follow the reasoning and the conclusion to make >>>>>> manifests 2 bytes wider or if i missed an important point >>>>>> altogether. >>>>>> >>>>>> In case there will be a consent about how to solve this, would >>>>>> someone volunteer to sponsor? That may be less urgent at the >>>>>> moment than the question above about how to proceed. >>>>>> >>>>>> Philipp >>>>>> >>>>>> >>>>>> On 12.10.2017 22:32, Sean Mullan wrote: >>>>>>> Hi Phillip, >>>>>>> >>>>>>> All of these bugs are in the core-libs area, so I am copying the >>>>>>> core-libs-dev list since that is where they should be discussed >>>>>>> and reviewed. I have -bcc-ed security-dev (where this was >>>>>>> originally posted). >>>>>>> >>>>>>> --Sean >>>>>>> >>>>>>> On 10/2/17 1:24 PM, Philipp Kunz wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> While fixing JDK-6695402 I came across other related bugs to >>>>>>>> manifests such as JDK-6372077, JDK-6202130, JDK-8176843, >>>>>>>> JDK-4842483, and JDK-6443578 which all relate to manifest >>>>>>>> reading and writing. Somewhere bug 7071055 is mentioned but I >>>>>>>> cannot find anywhere. Another group of bugs, JDK-6910466, >>>>>>>> JDK-4935610, and JDK-4271239 concern the mandatory manifest >>>>>>>> main attributes Manifest-Version or Signature-Version and at >>>>>>>> first glance are duplicates. If you know of more known bugs, >>>>>>>> not necessarily present in jira, I'd be glad to get notified. >>>>>>>> >>>>>>>> There are also some comments about utf handling and line >>>>>>>> breaking in the code of Manifest: >>>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l299 >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l327 >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l370 >>>>>>>> >>>>>>>> >>>>>>>> Furthermore, Attributes.map should declare appropriate type >>>>>>>> parameters: >>>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l61 >>>>>>>> >>>>>>>> The specification would also require that `header names must >>>>>>>> not start with _, - or "From"` >>>>>>>> (http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Section-Specification) >>>>>>>> but I would opt not to add this as a hard restriction because I >>>>>>>> guess it can be assumed that such names are in use now after >>>>>>>> having been working for years. A warning to a logger might be >>>>>>>> conceivable such as in >>>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l424 >>>>>>>> >>>>>>>> Attribute values are never checked at all and invalid >>>>>>>> characters such as line breaks are never detected except that >>>>>>>> when reading the manifest again the values are cut off. >>>>>>>> The tab character (U+0008) does not work in manifest values. >>>>>>>> I suspect that there are also issues regarding the iteration >>>>>>>> order but I haven't got a prove yet unlike for the other points >>>>>>>> above: >>>>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Manifest.java#l54 >>>>>>>> >>>>>>>> There is duplicated or very similar code in Attributes and >>>>>>>> Manifest: Attributes.write-Manifest.write-Attributes.writeMain >>>>>>>> and Attributes.read-Manifest.read. >>>>>>>> Resolving JDK-6202130 would have the additional benefit to be >>>>>>>> able to view manifests with any utf-8 capable editor even if >>>>>>>> multi-byte characters break across lines. >>>>>>>> >>>>>>>> Fixing these issues individually looks like more complicated >>>>>>>> work than fixing them all at once, I guess, also because of a >>>>>>>> very low current test coverage. So I'd suggest to add some >>>>>>>> thorough tests along with fixing these issues. But before I >>>>>>>> start I would like to get some guidance, assistance or at least >>>>>>>> confirmation on how to proceed. I'm new to open jdk and have >>>>>>>> submitted only one patch so far. >>>>>>>> >>>>>>>> Is it ok to add tests for things that have worked before? >>>>>>>> Is it ok to refactor duplicated code just for the added value >>>>>>>> to reduce effort for testing? >>>>>>>> I assume compatibility to and from existing manifests is the >>>>>>>> highest priority, correct? This would also be the hard part in >>>>>>>> adding as complete test coverage as possible. What would be >>>>>>>> acceptable criteria to meet? >>>>>>>> Why does Attributes not extend LinkedHashMap and why does >>>>>>>> Manifest not extend HashMap? Any objection? >>>>>>>> While I would not want to write code that looks slow or change >>>>>>>> more than necessary one can never know before having >>>>>>>> performance actually measured. Is there some way this is dealt >>>>>>>> with or should I wait for such feedback until after patch >>>>>>>> submission? >>>>>>>> >>>>>>>> Would someone sponsor? >>>>>>>> >>>>>>>> Regards, >>>>>>>> Philipp >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Paratix GmbH >>>>>>>> St Peterhofstatt 11 >>>>>>>> 8001 Z?rich >>>>>>>> >>>>>>>> +41 (0)76 397 79 35 >>>>>>>> philipp.kunz at paratix.ch >>>>>> >>>>>> -- >>>>>> >>>>>> >>>>>> Gruss Philipp >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Paratix GmbH >>>>>> St Peterhofstatt 11 >>>>>> 8001 Z?rich >>>>>> >>>>>> +41 (0)76 397 79 35 >>>>>> philipp.kunz at paratix.ch >>>>> >>>> >>> >> > > -- > > > Gruss Philipp > > > > ------------------------------------------------------------------------ > > > > Paratix GmbH > St Peterhofstatt 11 > 8001 Z?rich > > +41 (0)76 397 79 35 > philipp.kunz at paratix.ch From Roger.Riggs at Oracle.com Fri Feb 23 20:33:22 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 23 Feb 2018 15:33:22 -0500 Subject: RFR 8198645 Use System.lineSeparator() instead of getProperty("line.separator") In-Reply-To: <5A9070A1.1030909@oracle.com> References: <5A9070A1.1030909@oracle.com> Message-ID: <2ab58014-ebce-ab2e-caeb-f96b63cabe08@Oracle.com> There are two more changes in the java.xml package. (Thanks Joe) Webrev updated in place. ? http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ Thanks, Roger On 2/23/2018 2:50 PM, Xueming Shen wrote: > +1 > > On 02/23/2018 11:39 AM, Roger Riggs wrote: >> Please review cleanup replacements of >> System.getProperty("line.separator") with System.lineSeparator(). >> It uses the line separator from System instead of looking it up in >> the properties each time. >> Also fixed one javadoc @see reference. >> >> The affected files are in several packages: >> >> ?? com/sun/crypto/provider/ >> ?? java/util/regex/ >> ?? jdk/internal/util/xml/impl/ >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ >> >> Thanks, Roger >> >> > From lance.andersen at oracle.com Fri Feb 23 20:37:28 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 23 Feb 2018 15:37:28 -0500 Subject: RFR 8198645 Use System.lineSeparator() instead of getProperty("line.separator") In-Reply-To: <2ab58014-ebce-ab2e-caeb-f96b63cabe08@Oracle.com> References: <5A9070A1.1030909@oracle.com> <2ab58014-ebce-ab2e-caeb-f96b63cabe08@Oracle.com> Message-ID: <49CE557D-E466-4A80-BAA3-A78AB35B0625@oracle.com> still looks good > On Feb 23, 2018, at 3:33 PM, Roger Riggs wrote: > > There are two more changes in the java.xml package. (Thanks Joe) > Webrev updated in place. > > http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ > > Thanks, Roger > > On 2/23/2018 2:50 PM, Xueming Shen wrote: >> +1 >> >> On 02/23/2018 11:39 AM, Roger Riggs wrote: >>> Please review cleanup replacements of System.getProperty("line.separator") with System.lineSeparator(). >>> It uses the line separator from System instead of looking it up in the properties each time. >>> Also fixed one javadoc @see reference. >>> >>> The affected files are in several packages: >>> >>> com/sun/crypto/provider/ >>> java/util/regex/ >>> jdk/internal/util/xml/impl/ >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ >>> >>> Thanks, Roger >>> >>> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From huizhe.wang at oracle.com Fri Feb 23 21:41:22 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 23 Feb 2018 13:41:22 -0800 Subject: RFR 8198645 Use System.lineSeparator() instead of getProperty("line.separator") In-Reply-To: <2ab58014-ebce-ab2e-caeb-f96b63cabe08@Oracle.com> References: <5A9070A1.1030909@oracle.com> <2ab58014-ebce-ab2e-caeb-f96b63cabe08@Oracle.com> Message-ID: <13bbfaf6-b79f-be04-fcff-786454b32a12@oracle.com> Looks good to me too. Would you want to update the Copyright year? For the apache file (ToStream.java), it would also be good to update the @LastModified tag. Thanks, Joe On 2/23/2018 12:33 PM, Roger Riggs wrote: > There are two more changes in the java.xml package. (Thanks Joe) > Webrev updated in place. > > http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ > > Thanks, Roger > > On 2/23/2018 2:50 PM, Xueming Shen wrote: >> +1 >> >> On 02/23/2018 11:39 AM, Roger Riggs wrote: >>> Please review cleanup replacements of >>> System.getProperty("line.separator") with System.lineSeparator(). >>> It uses the line separator from System instead of looking it up in >>> the properties each time. >>> Also fixed one javadoc @see reference. >>> >>> The affected files are in several packages: >>> >>> ?? com/sun/crypto/provider/ >>> ?? java/util/regex/ >>> ?? jdk/internal/util/xml/impl/ >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-line-separator-8198645/ >>> >>> Thanks, Roger >>> >>> >> > From leonid.mesnik at oracle.com Fri Feb 23 21:48:27 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 23 Feb 2018 13:48:27 -0800 Subject: RFR: 8197901: Crash during GC when logging level is debug In-Reply-To: References: Message-ID: <49394AE1-14AE-4743-A964-E203B09D0201@oracle.com> Thank you for review. I will add @bug info in the test. Leonid > On Feb 22, 2018, at 8:26 PM, David Holmes wrote: > > Hi Leonid, > > Looks fine. Please also add this bug id to @bug in > > test/jdk/java/lang/StackWalker/VerifyStackTrace.java > > Thanks, > David > > On 23/02/2018 12:41 PM, Leonid Mesnik wrote: >> Hi >> Could you please review following fix which update implementation of Klass::external_name for anonymous classes. >> Previously external_name tried to add hashcode of corresponding java_mirror for InstanceKlass if it exists. However the java_mirror could be incorrect during GC. Also external_name might tries to calculate hash_code if it was not ?pre-calculated? during class verification. See JDK-8197442 [Graal] runtime/Metaspace/DefineClass.java crashes with "biases should not be seen by VM thread here" >> The suggested fix is to print address of corresponding InstanceKlass instead of hashcode. It allows to identify anonymous classes and allows to use external_name at any time. The hashcode for java_mirror is still pre-calculated in verifier.cpp since ik->java_mirror()->identity_hash() still might be used during safepoint. As a regression test I updated one of tests which redefine classes and easily reproduce problem when executed with full logging enabled. >> Test java/lang/StackWalker/VerifyStackTrace.java is update to match new pattern. >> webrev: http://cr.openjdk.java.net/~lmesnik/8197901/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8197901 >> Leonid From martinrb at google.com Fri Feb 23 21:50:57 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 23 Feb 2018 13:50:57 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: On Fri, Feb 23, 2018 at 6:28 AM, Alan Bateman wrote: > Just getting to the updated webrev now. > > On 21/02/2018 20:30, Martin Buchholz wrote: > > : > > > 8198480: Improve ClassLoaders static init block > http://cr.openjdk.java.net/~martin/webrevs/jdk/ClassLoaders-static/ > https://bugs.openjdk.java.net/browse/JDK-8198480 > > The value of jdk.module.main is the name of the "initial module" so better > to use that instead term of "main module". > > Done, but I see elsewhere the same convention, that jigsaw team might want to make consistent: --- String mainModule; String mainModule = System.getProperty("jdk.module.main"); * {@code jdk.module.main} * The module name of the initial/main module * {@code jdk.module.main.class} * The main class name of the initial module SetMainModule(const char *s) --- From mandy.chung at oracle.com Fri Feb 23 21:57:32 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 23 Feb 2018 13:57:32 -0800 Subject: [11] RFR JDK-8198653: ClassLoader::getSystemClassLoader throws InternalError when called after shutdown Message-ID: <9612a466-1d32-a5af-7111-f570e4737cb3@oracle.com> JDK-8198249 added a new shutdown VM initLevel. ClassLoader::getSystemClassLoader should be updated to handle the new case. I checked all other callers of VM::initLevel and no other place needs update. Thanks Mandy diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java --- a/src/java.base/share/classes/java/lang/ClassLoader.java +++ b/src/java.base/share/classes/java/lang/ClassLoader.java @@ -1922,7 +1922,7 @@ case 3: String msg = "getSystemClassLoader cannot be called during the system class loader instantiation"; throw new IllegalStateException(msg); - case 4: + default: // system fully initialized assert VM.isBooted() && scl != null; SecurityManager sm = System.getSecurityManager(); @@ -1930,8 +1930,6 @@ checkClassLoaderPermission(scl, Reflection.getCallerClass()); } return scl; - default: - throw new InternalError("should not reach here"); } } From martinrb at google.com Fri Feb 23 22:09:02 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 23 Feb 2018 14:09:02 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: [+Paul] On Fri, Feb 23, 2018 at 6:28 AM, Alan Bateman wrote: > > 8198484: URLClassPath should use an ArrayDeque instead of a Stack > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ > https://bugs.openjdk.java.net/browse/JDK-8198484 > > Can copyToArrayDeque use addAll? > Not directly, because addAll uses a lambda, and it's too early in the bootstrap for lambdas. We could delambdafy ArrayDeque, plausibly because ArrayDeque is a super-core class, perhaps reengineering ArrayDeque(Collection) and/or addAll(Collection). Or perhaps lambda team has a plan to allow lambdas to be used in corest of core java sometime soon? From joe.darcy at oracle.com Sat Feb 24 00:29:56 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Fri, 23 Feb 2018 16:29:56 -0800 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> Message-ID: <5A90B204.4070005@oracle.com> Hello, A few initial comments, not my final review. Objects implementing the AnnotatedElement interface are also created in javac during annotation processing. As a secondary concern, it would be good to be have behavior between both javac and runtime annotations consistent when possible. (My own to-do list includes at least once such alignment JDK-8164819: "Make javac's toString() on annotation objects consistent with core reflection".) Even in javac we've moved away from test and directory names based on bugid. I'd recommend incorporating these regression tests into the existing tests in test/jdk/java/lang/annotation/Missing or creating a subdirectory for conditions, etc.. It would be worth verifying whether or not this change also covers java.lang.reflect.Executable.getParameterAnnotations(), more specifically the implementation of that method in Method and Constructor. The method getParameterAnnotations() is much less used than getAnnotations and the other methods on the AnnotatedElement interface, but still part of the annotations feature. (As a follow-up refactoring, it might be worthwhile to replace the interior of the three parseFooArray methods to a shared method that is passed a lambda for the "Object value = parseFooValue(/*args to get foo*/...);" logic.) Thanks, -Joe On 2/23/2018 10:06 AM, joe darcy wrote: > On 2/23/2018 9:39 AM, Alan Bateman wrote: >> On 22/02/2018 23:20, Liam Miller-Cushon wrote: >>> Hello, >>> >>> >>> Please consider this fix for JDK-7183985. >>> >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-7183985 >>> >>> webrev: http://cr.openjdk.java.net/~cushon/7183985/webrev.01/ >>> >>> >>> I started a CSR for the change: >>> https://bugs.openjdk.java.net/browse/JDK-8198584 >>> >>> We have been using the fix at Google for about two years, and there has >>> been no compatibility impact. I found very few places >>> ArrayStoreException >>> was being explicitly handled, and none that were depending on the >>> current >>> behaviour of getAnnotation(). >>> >>> >>> There was some previous discussion of the bug on core-libs-dev: >>> >> Yes, this one comes up every few years. I'm hoping Joe Darcy will >> reply to your review with any background or issues from when this >> came up in the past. From a distance then retrofitting >> AnnotatedElement getXXX methods to throw TypeNotPresentException >> seems reasonable, I'm less sure about the isAnnotationPresent method >> as it might be surprising for that to fail. >> > > On my list! > > Cheers, > > -Joe > From xueming.shen at oracle.com Sat Feb 24 01:26:28 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 23 Feb 2018 17:26:28 -0800 Subject: RFR JDK-8187653: Lock in CoderResult.Cache becomes performance bottleneck Message-ID: <5A90BF44.3030005@oracle.com> Hi, Please help review the proposed change to remove the potential performance bottleneck in CoderResult caused by the "over" synchronization the cache mechanism. issue: https://bugs.openjdk.java.net/browse/JDK-8187653 webrev: http://cr.openjdk.java.net/~sherman/8187653/webrev Notes: While the original test case (new String/String.getBytes() with UTF8 as showed in the stacktrace) described in the bug report might no longer be reproducible in jdk9, as we have been optimizing lots of String related char[]/byte[] coding path away from the paths that use CoderResult. But it is still a concern/issue for the "general" CharsetDe/Encoder.de/encode() operation, in which the malformed or unmappable CoderResult object is returned. As showed in the "[synchronized]" section in bm.scores [1], which is from the simple jmh benchmark test CoderResultBM.java [2], the sores are getting significant worse when the number of concurrent de/encoding threads gets bigger. It appears the easy fix is to replace the sync mechanism from "method synchronized + HashMap" to "ConcurrentHashMap" solves the problem, as showed in the same bm result [1] in [ConcurrentHashMap] section, because most of the accesses to the caching HashMap is read operation. The ConcurrentHahsMap's "almost non-block for retrieval operation" really helps here. There is another fact that might help us optimize further here. For most of our charsets in JDK repository (with couple exceptions), the length of malformed and unmappable CoderResult that these charsets might trigger actually is never longer than 4. So we might not need to access the ConcurrentHashMap cache at all in normal use scenario. I'm putting a CoderResult[4] on top the hashmap cache in proposed webrev. It does not improve the performance significantly, but when the thread number gets bigger, a 10%+ improvement is observed. So I would assume it might be something worth doing, given its cost is really low. Thanks, Sherman [1] http://cr.openjdk.java.net/~sherman/8187653/bm.scores [2] http://cr.openjdk.java.net/~sherman/8187653/CoderResultBM.java [3] http://cr.openjdk.java.net/~sherman/8187653/bm.scores.ms932 From david.holmes at oracle.com Sat Feb 24 06:18:58 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 24 Feb 2018 16:18:58 +1000 Subject: [11] RFR JDK-8198653: ClassLoader::getSystemClassLoader throws InternalError when called after shutdown In-Reply-To: <9612a466-1d32-a5af-7111-f570e4737cb3@oracle.com> References: <9612a466-1d32-a5af-7111-f570e4737cb3@oracle.com> Message-ID: Looks good. Is there an existing test that caught this? Thanks, David On 24/02/2018 7:57 AM, mandy chung wrote: > JDK-8198249 added a new shutdown VM initLevel. > ClassLoader::getSystemClassLoader > should be updated to handle the new case.? I checked all other callers of > VM::initLevel and no other place needs update. > > Thanks > Mandy > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java > b/src/java.base/share/classes/java/lang/ClassLoader.java > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > @@ -1922,7 +1922,7 @@ > ???????????? case 3: > ???????????????? String msg = "getSystemClassLoader cannot be called > during the system class loader instantiation"; > ???????????????? throw new IllegalStateException(msg); > -??????????? case 4: > +??????????? default: > ???????????????? // system fully initialized > ???????????????? assert VM.isBooted() && scl != null; > ???????????????? SecurityManager sm = System.getSecurityManager(); > @@ -1930,8 +1930,6 @@ > ???????????????????? checkClassLoaderPermission(scl, > Reflection.getCallerClass()); > ???????????????? } > ???????????????? return scl; > -??????????? default: > -??????????????? throw new InternalError("should not reach here"); > ???????? } > ???? } > From mandy.chung at oracle.com Sat Feb 24 06:41:36 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 23 Feb 2018 22:41:36 -0800 Subject: [11] RFR JDK-8198653: ClassLoader::getSystemClassLoader throws InternalError when called after shutdown In-Reply-To: References: <9612a466-1d32-a5af-7111-f570e4737cb3@oracle.com> Message-ID: Yes, test/hotspot/jtreg/compiler/jvmci/events/JvmciShutdownEventTest.java. This test passes with the change. Mandy On 2/23/18 10:18 PM, David Holmes wrote: > Looks good. > > Is there an existing test that caught this? > > Thanks, > David > > On 24/02/2018 7:57 AM, mandy chung wrote: >> JDK-8198249 added a new shutdown VM initLevel.? >> ClassLoader::getSystemClassLoader >> should be updated to handle the new case.? I checked all other >> callers of >> VM::initLevel and no other place needs update. >> >> Thanks >> Mandy >> >> diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java >> b/src/java.base/share/classes/java/lang/ClassLoader.java >> --- a/src/java.base/share/classes/java/lang/ClassLoader.java >> +++ b/src/java.base/share/classes/java/lang/ClassLoader.java >> @@ -1922,7 +1922,7 @@ >> ????????????? case 3: >> ????????????????? String msg = "getSystemClassLoader cannot be called >> during the system class loader instantiation"; >> ????????????????? throw new IllegalStateException(msg); >> -??????????? case 4: >> +??????????? default: >> ????????????????? // system fully initialized >> ????????????????? assert VM.isBooted() && scl != null; >> ????????????????? SecurityManager sm = System.getSecurityManager(); >> @@ -1930,8 +1930,6 @@ >> ????????????????????? checkClassLoaderPermission(scl, >> Reflection.getCallerClass()); >> ????????????????? } >> ????????????????? return scl; >> -??????????? default: >> -??????????????? throw new InternalError("should not reach here"); >> ????????? } >> ????? } >> From cushon at google.com Sun Feb 25 01:14:55 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Sat, 24 Feb 2018 17:14:55 -0800 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: <5A90B204.4070005@oracle.com> References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> <5A90B204.4070005@oracle.com> Message-ID: Hi, thanks for the comments. The updated webrev is at: http://cr.openjdk.java.net/~cushon/7183985/webrev.02/ On Fri, Feb 23, 2018 at 4:29 PM, Joseph D. Darcy wrote: > Objects implementing the AnnotatedElement interface are also created in > javac during annotation processing. As a secondary concern, it would be > good to be have behavior between both javac and runtime annotations > consistent when possible. (My own to-do list includes at least once such > alignment JDK-8164819: "Make javac's toString() on annotation objects > consistent with core reflection".) > Do you have a pointer to where that happens? There's javax.lang.model. AnnotatedConstruct#getAnnotation, which isn't affected by this issue--it throws MirroredTypesExceptionProxy when accessing an annotation value that is an array of class literals, regardless of whether the elements' classes are missing. I'm not seeing implementations of AnnotatedElement in langtools. > Even in javac we've moved away from test and directory names based on > bugid. I'd recommend incorporating these regression tests into the existing > tests in > test/jdk/java/lang/annotation/Missing > Thanks for the reminder, done. > It would be worth verifying whether or not this change also covers > java.lang.reflect.Executable.getParameterAnnotations(), more specifically > the implementation of that method in Method and Constructor. The method > getParameterAnnotations() is much less used than getAnnotations and the > other methods on the AnnotatedElement interface, but still part of the > annotations feature. > Done. > (As a follow-up refactoring, it might be worthwhile to replace the > interior of the three parseFooArray methods to a shared method that is > passed a lambda for the "Object value = parseFooValue(/*args to get > foo*/...);" logic.) > Sounds good, I filed: https://bugs.openjdk.java.net/browse/JDK-8198669 From claes.redestad at oracle.com Sun Feb 25 23:51:23 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 26 Feb 2018 00:51:23 +0100 Subject: RFR: 8198492: java/lang/StackWalker/CallerFromMain.java failed timeout. Message-ID: Hi, the JDK-8198418[1] improvements to lambda bootstrapping meant initialization changed around to allow the possibility of a bootstrap race, which made it possible to cause a class loading deadlock when different threads try to initialize classes like SimpleMethodHandle and SpeciesData at the same time. Making sure the common ancestor, BoundMethodHandle, is initialized using the same means before going into the synchronized block in LambdaForm:createFormsFor seems to be enough to ensure this race can always be resolved peacefully: http://cr.openjdk.java.net/~redestad/8198492/jdk.00/ Testing: ~5000 runs of the affected tests. /Claes [1] https://bugs.openjdk.java.net/browse/JDK-8198418 From Alan.Bateman at oracle.com Mon Feb 26 11:21:16 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 26 Feb 2018 11:21:16 +0000 Subject: RFR: 8198492: java/lang/StackWalker/CallerFromMain.java failed timeout. In-Reply-To: References: Message-ID: <5ce53ce4-eb81-f4b7-52a5-89e1b33c2cb6@oracle.com> On 25/02/2018 23:51, Claes Redestad wrote: > Hi, > > the JDK-8198418[1] improvements to lambda bootstrapping meant > initialization changed around to allow the possibility of a > bootstrap race, which made it possible to cause a class loading > deadlock when different threads try to initialize classes like > SimpleMethodHandle and SpeciesData at the same time. > > Making sure the common ancestor, BoundMethodHandle, is initialized > using the same means before going into the synchronized block in > LambdaForm:createFormsFor seems to be enough to ensure this race > can always be resolved peacefully: > > http://cr.openjdk.java.net/~redestad/8198492/jdk.00/ This looks okay to me to fix the current issue. There are several other bugs that look similar and I expect they will go away once you get this pushed. It might be useful to move JDK-8198492 to the java.lang.invoke subcomponent, and maybe change the summary so that it's clearer for anyone looking at in the future. -Alan From claes.redestad at oracle.com Mon Feb 26 11:26:47 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 26 Feb 2018 12:26:47 +0100 Subject: RFR: 8198492: java/lang/StackWalker/CallerFromMain.java failed timeout. In-Reply-To: <5ce53ce4-eb81-f4b7-52a5-89e1b33c2cb6@oracle.com> References: <5ce53ce4-eb81-f4b7-52a5-89e1b33c2cb6@oracle.com> Message-ID: On 2018-02-26 12:21, Alan Bateman wrote: >> >> http://cr.openjdk.java.net/~redestad/8198492/jdk.00/ > This looks okay to me to fix the current issue. There are several > other bugs that look similar and I expect they will go away once you > get this pushed. It might be useful to move JDK-8198492 to the > java.lang.invoke subcomponent, and maybe change the summary so that > it's clearer for anyone looking at in the future. Thanks! How about "Bootstrapping java.lang.invoke can cause deadlock after JDK-8198418"? /Claes From Alan.Bateman at oracle.com Mon Feb 26 11:50:00 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 26 Feb 2018 11:50:00 +0000 Subject: JDK-6372077: JarFile.getManifest() should handle manifest attribute names up to 70 bytes In-Reply-To: <39bd4f58-bdf6-8fff-4c21-37c2b1f4a340@Oracle.com> References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> <02abcb68-2294-c733-5048-b1f81ee99435@oracle.com> <88b89995-cfe0-d439-d63d-699c628e2315@paratix.ch> <07e62d8d-ae3a-9ffa-d426-81ab548ad252@Oracle.com> <9a08b2c0-3659-1833-f113-23aaceca84bc@Oracle.com> <39bd4f58-bdf6-8fff-4c21-37c2b1f4a340@Oracle.com> Message-ID: On 23/02/2018 19:58, Roger Riggs wrote: > Please review this contribution from Philipp Kunz to handle manifest > attribute names up to 70 bytes. > > The change passes the available regression tests. > Manifest handling is somewhat sensitive so an additional review is > appreciated. > > Webrev: (rebased from the original patch of 2/22/18) > ??? http://cr.openjdk.java.net/~rriggs/webrev-jar-6372077.patch/ > > Issue: > ?? https://bugs.openjdk.java.net/browse/JDK-6372077 I looked at the changes to the manifest code and I think they are okay. I don't have time to study the tests in detail. I think it would be useful to copy the clarification note "(there must be a colon and a SPACE after the name)" from the JAR file spec to the equivalent sentence in the Attributes class description. That will help anyone wondering why the limit is 70 rather than 72. This change shouldn't need a CSR as it's not adding any new assertions. -Alan From aleksej.efimov at oracle.com Mon Feb 26 15:34:38 2018 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Mon, 26 Feb 2018 15:34:38 +0000 Subject: [11] RFR: (JAXP) 8038043: Xerces Update: XInclude update In-Reply-To: References: Message-ID: <3492cb38-3b7c-bce2-ac8f-e81645a69a78@oracle.com> Hi Joe, Thank you for the review. I've updated XIncludeHandler and XIncludeTextReader files per your suggestions. XIncludeTextReader:191 line is still shown in Sdiffs as one line, but patch contains correct change that splits it. New webrev: http://cr.openjdk.java.net/~aefimov/8038043/11/01 Best, Aleksei On 02/22/2018 08:39 AM, Joe Wang wrote: > Hi Aleksei, > > Thanks for taking the time to work on this! > > Looks good overall. > > XIncludeHandler: setupCurrentBaseURI method can be private. > > XIncludeTextReader: there's a very long line at 191. It would be good > to fix it so that Sdiffs looks better the next time. > > As for the tests, I'm fine with the encoding tests plus passing all > existing ones. > > Best, > Joe > > On 2/16/2018 9:53 AM, Aleks Efimov wrote: >> Hi, >> >> Please, help to review the update of XInclude related classes from >> the Apache Xerces 2.11.0 source. >> JBS: >> ??? https://bugs.openjdk.java.net/browse/JDK-8038043 >> The webrev: >> ??? http://cr.openjdk.java.net/~aefimov/8038043/11/00/ >> >> New regression test has been added to check the updated reporting of >> invalid bytes encountered during the parsing and inclusion of XML >> documents. New test and other regression tests shows no failures. >> >> With Best Regards, >> Aleksei > From scolebourne at joda.org Mon Feb 26 16:41:38 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 26 Feb 2018 16:41:38 +0000 Subject: Weird timezone issues with Timestamp.valueOf(String s) In-Reply-To: References: <004c01d3a1b7$e1efe9c0$a5cfbd40$@gmail.com> Message-ID: Just to note that you really need someone from the JDBC maintenance group to comment. However, my understanding is that Timestamp conceptually represents a date and time **without reference to a time-zone**, exactly the same as LocalDateTime. But the implementation is poor, because it stores it using epoch millis relative to the local time zone. In general, I prefer to run server-side systems only in UTC to avoid issues like this. Stephen On 9 February 2018 at 17:15, Martin Buchholz wrote: > [redirect to core-libs-dev] > > On Fri, Feb 9, 2018 at 7:08 AM, wrote: > >> Not that I encourage using date/time classes from the packages of either >> java.util or java.sql, but sometimes it happens under the hood. >> >> >> >> We found a weird issue with timestamps. >> >> In our (PostgreSQL) database we have a column of SQL type TIMESTAMP - no >> timezone. >> >> >> >> When JPA fills our entity field with a java.sql.Timestamp value, it is >> given >> an inherent timezone of the system it's running on; in our case it was CET >> (UTC +1). >> >> Now this timezone isn't immediately obvious, because if you print it to >> system out, it seems to have the correct time. However when you convert it >> with .toInstant() the timezone rears its ugly head. >> >> >> >> I digged deeper and found Timestamp.valueOf(String s), it does a lot of >> magic, but in the end it calls its own deprecated constructor new >> Timestamp(int year, int month, int date, int hour, int minute, int second, >> int nano). >> >> That constructor calls the deprecated constructor of java.util.Date(int >> year, int month, int date, int hour, int minute, int second) and that is >> the one doing something with the current timezone on the system. >> >> The method Timestamp.valueOf(String s) itself however, is not deprecated >> and >> I find this odd. >> >> More odd is that Timestamp.valueOf(LocalDateTime dateTime) has a >> @SuppresWarnings("deprecation") annotation. >> >> >> >> Thus I found that Timestamp.valueOf("2017-10-04 00:00:00") on a system >> running in CET timezone yields a different result than one running in UTC >> timezone. >> >> Here is some example code where I put the output of the println's in >> comments. >> >> >> >> >> TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("Europe/Amsterdam"))); >> >> Timestamp fromStringCet = Timestamp.valueOf("2017-10-04 00:00:00"); >> >> System.out.println(fromStringCet); // 2017-10-04 00:00:00.0 >> >> System.out.println(fromStringCet.toInstant()); // 2017-10-03T22:00:00Z >> >> >> >> TimeZone.setDefault(TimeZone.getTimeZone(ZoneOffset.UTC)); >> >> Timestamp fromStringUtc = Timestamp.valueOf("2017-10-04 00:00:00"); >> >> System.out.println(fromStringUtc); // 2017-10-04 00:00:00.0 >> >> System.out.println(fromStringUtc.toInstant()); // 2017-10-04T00:00:00Z >> >> >> >> System.out.println(fromStringCet.equals(fromStringUtc)); // false >> >> >> >> LocalDateTime localDateTime = LocalDateTime.of(2017, 10, 4, 0, 0, 0); >> >> >> >> >> TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("Europe/Amsterdam"))); >> >> Timestamp fromLocalDateTimeCet = Timestamp.valueOf(localDateTime); >> >> System.out.println(fromLocalDateTimeCet.toInstant()); // >> 2017-10-03T22:00:00Z >> >> >> >> TimeZone.setDefault(TimeZone.getTimeZone(ZoneOffset.UTC)); >> >> Timestamp fromLocalDateTimeUtc = Timestamp.valueOf(localDateTime); >> >> System.out.println(fromLocalDateTimeUtc.toInstant()); // 2017-10-04 >> 00:00:00.0 >> >> >> >> System.out.println(fromLocalDateTimeCet.equals(fromLocalDateTimeUtc)); >> // false >> >> >> >> So what to do? >> >> >> >> Some options are: >> >> * Make Timestamp.valueOf(String s) deprecated? >> * Always use UTC when doing the implicit conversion >> >> >> >> Thoughts? >> >> >> >> Dave Franken >> >> >> >> From mandy.chung at oracle.com Mon Feb 26 16:46:16 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 26 Feb 2018 08:46:16 -0800 Subject: RFR: 8198492: java/lang/StackWalker/CallerFromMain.java failed timeout. In-Reply-To: References: Message-ID: <80a2cd18-0b76-7fdd-a3d6-99207758eff9@oracle.com> On 2/25/18 3:51 PM, Claes Redestad wrote: > Hi, > > the JDK-8198418[1] improvements to lambda bootstrapping meant > initialization changed around to allow the possibility of a > bootstrap race, which made it possible to cause a class loading > deadlock when different threads try to initialize classes like > SimpleMethodHandle and SpeciesData at the same time. > > Making sure the common ancestor, BoundMethodHandle, is initialized > using the same means before going into the synchronized block in > LambdaForm:createFormsFor seems to be enough to ensure this race > can always be resolved peacefully: > > http://cr.openjdk.java.net/~redestad/8198492/jdk.00/ This looks okay. Mandy From huizhe.wang at oracle.com Mon Feb 26 17:11:56 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 26 Feb 2018 09:11:56 -0800 Subject: [11] RFR: (JAXP) 8038043: Xerces Update: XInclude update In-Reply-To: <3492cb38-3b7c-bce2-ac8f-e81645a69a78@oracle.com> References: <3492cb38-3b7c-bce2-ac8f-e81645a69a78@oracle.com> Message-ID: <3597cb39-c681-9c83-f078-4097e270d401@oracle.com> Looks good. Thanks! Best, Joe On 2/26/2018 7:34 AM, Aleks Efimov wrote: > Hi Joe, > > Thank you for the review. I've updated XIncludeHandler and > XIncludeTextReader files per your suggestions. > XIncludeTextReader:191 line is still shown in Sdiffs as one line, but > patch contains correct change that splits it. > > New webrev: http://cr.openjdk.java.net/~aefimov/8038043/11/01 > > Best, > Aleksei > > > On 02/22/2018 08:39 AM, Joe Wang wrote: >> Hi Aleksei, >> >> Thanks for taking the time to work on this! >> >> Looks good overall. >> >> XIncludeHandler: setupCurrentBaseURI method can be private. >> >> XIncludeTextReader: there's a very long line at 191. It would be good >> to fix it so that Sdiffs looks better the next time. >> >> As for the tests, I'm fine with the encoding tests plus passing all >> existing ones. >> >> Best, >> Joe >> >> On 2/16/2018 9:53 AM, Aleks Efimov wrote: >>> Hi, >>> >>> Please, help to review the update of XInclude related classes from >>> the Apache Xerces 2.11.0 source. >>> JBS: >>> ??? https://bugs.openjdk.java.net/browse/JDK-8038043 >>> The webrev: >>> ??? http://cr.openjdk.java.net/~aefimov/8038043/11/00/ >>> >>> New regression test has been added to check the updated reporting of >>> invalid bytes encountered during the parsing and inclusion of XML >>> documents. New test and other regression tests shows no failures. >>> >>> With Best Regards, >>> Aleksei >> > From paul.sandoz at oracle.com Mon Feb 26 17:28:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 26 Feb 2018 09:28:17 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: > On Feb 23, 2018, at 2:09 PM, Martin Buchholz wrote: > > [+Paul] > > On Fri, Feb 23, 2018 at 6:28 AM, Alan Bateman wrote: >> >> 8198484: URLClassPath should use an ArrayDeque instead of a Stack >> http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ >> https://bugs.openjdk.java.net/browse/JDK-8198484 > Can copyToArrayDeque use addAll? > > On Feb 23, 2018, at 2:09 PM, Martin Buchholz wrote: > Not directly, because addAll uses a lambda, and it's too early in the bootstrap for lambdas. > > We could delambdafy ArrayDeque, plausibly because ArrayDeque is a super-core class, perhaps reengineering ArrayDeque(Collection) and/or addAll(Collection). > Some data on how many lambdas/methods refs are used by the core collection classes could help. I would be wary of going on a lambda purge right now and biasing certain collection classes towards their use at startup if we can avoid that with careful management. I would be tempted to drop the method copyToArrayDeque and just rely on the push method (even though it uses synchronized), then add a comment to the unopenedUrls field and/or in the constructor. > Or perhaps lambda team has a plan to allow lambdas to be used in corest of core java sometime soon? No current plans to more formally attack this gnarly problem. A start would be to gather data on the VM boot code-paths to better understand the sensitive areas, from that we might consider black listing some classes so a compiler failure is produced. IIRC the circularity errors can often confusing, it might be possible to place a check for the required VM init level, thereby producing a clearer runtime error. Lazier initialization might help too (i hope with constant dynamic stuff, where even though it?s using j.l.invoke it will move stuff off the critical path, while also spreading the cost of startup). Paul. From adam.petcher at oracle.com Mon Feb 26 18:39:52 2018 From: adam.petcher at oracle.com (Adam Petcher) Date: Mon, 26 Feb 2018 13:39:52 -0500 Subject: RFR 8181594: Efficient and constant-time modular arithmetic In-Reply-To: References: <7bd6c1ae-4173-4384-fc57-66d1b185ba56@oracle.com> Message-ID: <9021dd99-4ac1-f7a1-fc91-ab0a9cf1685b@oracle.com> Thanks for the initial feedback. Here is the latest webrev: http://cr.openjdk.java.net/~apetcher/8181594/webrev.01/ See inline below. On 2/23/2018 12:46 PM, Xuelei Fan wrote: > ArrayUtil.java: > =============== > I'm not very sure how widely this utilities will be used in the > future. Looks like only BigIntegerModuloP uses this classes.? I may > prefer to define private methods for byte array swap in > BigIntegerModuloP. It is also used by XDHPublicKeyImpl (in the XDH code review). XDH public keys are represented as BigInteger, and I use the array reverse method to convert encoded keys to BigInteger. > > BigIntegerModuloP.java: > ======================= > As this is a class for testing or ptototype purpose,? it might not be > a part of JDK products, like JRE.? Would you mind move it to a test > package if you want to keep it? Good idea. I moved it into the same directory as TestIntegerModuloP.java. > > IntegerModuloP, IntegerModuloP_Base and MutableIntegerModuloP > ============================================================= > In the security package/context, it may make sense to use > "IntegerModulo" for the referring to "integers modulo a prime value". In ECC, we also have curves over binary fields, and some methods in these interfaces/classes will not work correctly if the number of field elements is not prime (e.g. Fermat's little theorem is used to calculate a multiplicative inverse). So I would prefer that the names of these interfaces contain some string (e.g. "Prime" or "P") to make it clear that they only represent prime fields. > The class name of "IntegerModuloP_Base" is not a general Java coding > style.? I may prefer a little bit name changes like: > ???? IntegerModuloP_Base?? -> IntegerModulo > ???? IntegerModuloP??????? -> ImmutableIntegerModulo > ???? MutableIntegerModuloP -> MutableIntegerModulo > > ???? IntegerFieldModuloP?? -> IntegerModuloField (?) > I'm not fond of the current names, either. My goal with the naming was to indicate that "IntegerModuloP_Base" shouldn't be used in most cases, because you don't know whether it is mutable or not. So the typical interface to use is the immutable one, and the mutable one should only be used when mutability is necessary. Your suggested naming is more conventional, but I think it loses this aspect of my naming system. It's only an internal API, so I'm not too picky about the names, and I have no problem changing it as you suggest. But I'd like to see if anyone else has any suggestions about the names, first. > > MutableIntegerModuloP.java > ========================== > void conditionalSwapWith(MutableIntegerModuloP b, int swap); > As the 'swap' parameter can only be 0 or 1, could it be a boolean > parameter? I couldn't come up with a way to implement this without branching when the swap parameter is boolean. See IntegerPolynomial.conditionalSwap to see how this is implemented in arithmetic with an int swap argument. If you (or anyone) can think of a way to do this with boolean, let me know. I added a sentence to the comment above conditionalSwapWith that describes why it is an int instead of a boolean. > > Except the conditionalSwapWith() method, I did not get the points why > we need a mutable version.? Would you please have more description of > this requirement? The comment above the class definition has this sentence: "This interface can be used to improve performance and avoid the allocation of a large number of temporary objects." Do you need more information than this in the comments? The performance motivation is so that a.add(b).multiply(c)... can be done without allocating a new buffer for each operation. For example, without mutable field elements, an X25519 point multiplication would allocate around 4,300 temporary arrays totaling 350,000 bytes. If I remember correctly, switching the X25519 implementation to mutable field elements reduced the point multiplication time by about half. > > > IntegerModuloP_Base.java > ======================== > default byte[] addModPowerTwo(IntegerModuloP_Base b, int len) > void addModPowerTwo(IntegerModuloP_Base b, byte[] result); > > For the first sign of the method names, I thought it is to calculate > as "(this + b) ^ 2 mod m". To be precise, it calculates "((this % p) + (b % p)) % 2^m" (where p is the prime that defines the field, and m is the desired length, in bits). Note that the addition here is normal integer addition (not addition in GF(p)). This operation is not used in XDH, but it is used in Poly1305 to add the AES encryption of a nonce to a field element. So you can get more information about this operation by reading the Poly1305 paper/RFC. > Besides, what's the benefits of the two methods?? Could we just use: > ????? this.add(b).asByteArray() No, because that would calculate "((this + b) mod p) mod 2^m". The value of (this + b) can be larger than p, so this would not produce the desired result. > > I guess, but not very sure, it is for constant time calculation. If > the function is required, could it be renamed as: > > ????? // the result is inside of the size range > ????? IntegerModuloP addModSize(IntegerModuloP_Base b, int size) > Or > ????? // the result is wrapped if outside of the size range > ????? IntegerModuloP addOnWrap(IntegerModuloP_Base b, int size) > > and the use may look like: > ????? this.addModSize(b, size).asByteArray() > Any attempt to perform the addition in IntegerModuloP and then pull out the byte array will not work. This class can only represent field elements, so the sum would be in the field, which is not what we want. > > Will review the rest when I understand more about the interfaces design. > > Thanks, > Xuelei > > On 1/30/2018 8:52 AM, Adam Petcher wrote: >> +core-libs-dev >> >> >> On 1/26/2018 4:06 PM, Adam Petcher wrote: >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8181594 >>> Webrev: http://cr.openjdk.java.net/~apetcher/8181594/webrev.00/ >>> >>> This is a code review for the field arithmetic that will be used in >>> implementations of X25519/X448 key agreement, the Poly1305 >>> authenticator, and EdDSA signatures. I believe that the library has >>> all the features necessary for X25519/X448 and Poly1305, and I >>> expect at most a couple of minor enhancements will be required to >>> support EdDSA. There is no public API for this library, so we can >>> change it in the future to suit the needs of new algorithms without >>> breaking compatibility with external code. Still, I made an attempt >>> to clearly structure and document the (internal) API, and I want to >>> make sure it is understandable and easy to use. >>> >>> This is not a general-purpose modular arithmetic library. It will >>> only work well in circumstances where the sequence of operations is >>> restricted, and where the prime that defines the field has some >>> useful structure. Moreover, each new field will require some >>> field-specific code that takes into account the structure of the >>> prime and the way the field is used in the application. The initial >>> implementation includes a field for Poly1305 and the fields for >>> X25519/X448 which should also work for EdDSA. >>> >>> The benefits of using this library are that it is much more >>> efficient than using similar operations in BigInteger. Also, many >>> operations are branch-free, making them suitable for use in a >>> side-channel resistant implementation that does not branch on secrets. >>> >>> To provide some context, I have attached a code snippet describing >>> how this library can be used. The snippet is the constant-time >>> Montgomery ladder from my X25519/X448 implementation, which I expect >>> to be out for review soon. X25519/X448 only uses standard arithmetic >>> operations, and the more unusual features (e.g. add modulo a power >>> of 2) are needed by Poly1305. >>> >>> The field arithmetic (for all fields) is implemented using a 32-bit >>> representation similar to the one described in the Ed448 paper[1] >>> (in the "Implementation on 32-bit platforms" section). Though my >>> implementation uses signed limbs, and grade-school multiplication >>> instead of Karatsuba. The argument for correctness is essentially >>> the same for all three fields: the magnitude of each 64-bit limb is >>> at most 2^(k-1) after reduction, except for the last limb which may >>> have a magnitude of up to 2^k. The values of k are between 26 to 28 >>> (depending on the field), and we can calculate that the maximum >>> magnitude for any limb during an add-multiply-carry-reduce sequence >>> is always less than 2^63. Therefore, no overflow occurs and all >>> operations are correct. >>> >>> Process note: this enhancement is part of JEP 324 (Key Agreement >>> with Curve25519 and Curve448). When this code review is complete, >>> nothing will happen until all other work for this JEP is complete, >>> and the JEP is accepted as part of some release. This means that >>> this code will be pushed to the repo along with the X25519/X448 code >>> that uses it. >>> >>> [1] https://eprint.iacr.org/2015/625.pdf >>> >>> >>> >> From martinrb at google.com Mon Feb 26 21:25:57 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 26 Feb 2018 13:25:57 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: On Mon, Feb 26, 2018 at 9:28 AM, Paul Sandoz wrote: > > > > On Feb 23, 2018, at 2:09 PM, Martin Buchholz > wrote: > > > > [+Paul] > > > > On Fri, Feb 23, 2018 at 6:28 AM, Alan Bateman > wrote: > >> > >> 8198484: URLClassPath should use an ArrayDeque instead of a Stack > >> http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ > >> https://bugs.openjdk.java.net/browse/JDK-8198484 > > Can copyToArrayDeque use addAll? > > > > > > On Feb 23, 2018, at 2:09 PM, Martin Buchholz > wrote: > > > Not directly, because addAll uses a lambda, and it's too early in the > bootstrap for lambdas. > > > > We could delambdafy ArrayDeque, plausibly because ArrayDeque is a > super-core class, perhaps reengineering ArrayDeque(Collection) and/or > addAll(Collection). > > > > Some data on how many lambdas/methods refs are used by the core collection > classes could help. I would be wary of going on a lambda purge right now > and biasing certain collection classes towards their use at startup if we > can avoid that with careful management. > > I would be tempted to drop the method copyToArrayDeque and just rely on > the push method (even though it uses synchronized), then add a comment to > the unopenedUrls field and/or in the constructor. > > I prefer to keep things as I have them for now. Calling push requires an extra copy to create the array, and even though that overhead is swamped by other inefficiencies, it's a tax on every single java program (and the tax is higher at Google, where we have trouble fitting the classpath into Linux' 128 kb per-arg command line length limit). From paul.sandoz at oracle.com Mon Feb 26 22:01:06 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 26 Feb 2018 14:01:06 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: <8132A6F4-F92F-4788-AC62-E8FC160C1FF8@oracle.com> > On Feb 26, 2018, at 1:25 PM, Martin Buchholz wrote: > > > > On Mon, Feb 26, 2018 at 9:28 AM, Paul Sandoz > wrote: > > > > On Feb 23, 2018, at 2:09 PM, Martin Buchholz > wrote: > > > > [+Paul] > > > > On Fri, Feb 23, 2018 at 6:28 AM, Alan Bateman > wrote: > >> > >> 8198484: URLClassPath should use an ArrayDeque instead of a Stack > >> http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ > >> https://bugs.openjdk.java.net/browse/JDK-8198484 > > Can copyToArrayDeque use addAll? > > > > > > On Feb 23, 2018, at 2:09 PM, Martin Buchholz > wrote: > > > Not directly, because addAll uses a lambda, and it's too early in the bootstrap for lambdas. > > > > We could delambdafy ArrayDeque, plausibly because ArrayDeque is a super-core class, perhaps reengineering ArrayDeque(Collection) and/or addAll(Collection). > > > > Some data on how many lambdas/methods refs are used by the core collection classes could help. I would be wary of going on a lambda purge right now and biasing certain collection classes towards their use at startup if we can avoid that with careful management. > > I would be tempted to drop the method copyToArrayDeque and just rely on the push method (even though it uses synchronized), then add a comment to the unopenedUrls field and/or in the constructor. > > > I prefer to keep things as I have them for now. Calling push requires an extra copy to create the array, Ok, fair point, Paul. > and even though that overhead is swamped by other inefficiencies, it's a tax on every single java program (and the tax is higher at Google, where we have trouble fitting the classpath into Linux' 128 kb per-arg command line length limit). > From mandy.chung at oracle.com Mon Feb 26 23:06:08 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 26 Feb 2018 15:06:08 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: Message-ID: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> On 2/21/18 12:30 PM, Martin Buchholz wrote: > OK, we have a reworked set of patches. > > (In my corner of openjdk we generally use real javadoc on private > elements.) > > I reverted private doc comment style to the current maddening > inconsistency, except I couldn't restrain myself from fixing > > -? ? // ACC used when loading classes and resources */ > +? ? // ACC used when loading classes and resources > > I changed ArrayDeque.push to addFirst, as Peter suggested. > > > 8198480: Improve ClassLoaders static init block > http://cr.openjdk.java.net/~martin/webrevs/jdk/ClassLoaders-static/ > > https://bugs.openjdk.java.net/browse/JDK-8198480 > Can you rename initialModuleName to mainModule as Alan suggests (also in the comment)? line 63-64: ident further to the right as line 62 is the condition. > 8198481: Coding style cleanups for > src/java.base/share/classes/jdk/internal/loader > http://cr.openjdk.java.net/~martin/webrevs/jdk/loader-style/ > > https://bugs.openjdk.java.net/browse/JDK-8198481 Looks okay. > > 8198482: The URLClassPath field "urls" should be renamed to "unopenedUrls" > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-rename-urls/ > > https://bugs.openjdk.java.net/browse/JDK-8198482 > +1 > 8198484: URLClassPath should use an ArrayDeque instead of a Stack > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-ArrayDeque/ > > https://bugs.openjdk.java.net/browse/JDK-8198484 > Looks okay.? I also think no need to have a separate copyToArrayDeque method and just inline in the constructor. > 8198485: Simplify a URLClassPath constructor > http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassPath-simplify-constructor/ > > https://bugs.openjdk.java.net/browse/JDK-8198485 > +1 Mandy From martinrb at google.com Mon Feb 26 23:10:49 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 26 Feb 2018 15:10:49 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> Message-ID: On Mon, Feb 26, 2018 at 3:06 PM, mandy chung wrote: > > 8198480: Improve ClassLoaders static init block > http://cr.openjdk.java.net/~martin/webrevs/jdk/ClassLoaders-static/ > https://bugs.openjdk.java.net/browse/JDK-8198480 > > > Can you rename initialModuleName to mainModule as Alan suggests (also in > the comment)? > It's the other way around - I renamed it from mainModuleName to initialModuleName based on Alan's comment. (and the code elsewhere continues to be inconsistent on this) From mandy.chung at oracle.com Mon Feb 26 23:14:56 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 26 Feb 2018 15:14:56 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> Message-ID: On 2/26/18 3:10 PM, Martin Buchholz wrote: > > > On Mon, Feb 26, 2018 at 3:06 PM, mandy chung > wrote: > >> >> 8198480: Improve ClassLoaders static init block >> http://cr.openjdk.java.net/~martin/webrevs/jdk/ClassLoaders-static/ >> >> https://bugs.openjdk.java.net/browse/JDK-8198480 >> >> > > Can you rename initialModuleName to mainModule as Alan suggests > (also in the comment)? > > > It's the other way around - I renamed it from mainModuleName to > initialModuleName based on Alan's comment.? (and the code elsewhere > continues to be inconsistent on this) OK.?? I agree we should follow up and make them consistent. Mandy From martinrb at google.com Mon Feb 26 23:26:35 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 26 Feb 2018 15:26:35 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> Message-ID: On Mon, Feb 26, 2018 at 3:06 PM, mandy chung wrote: > > > On 2/21/18 12:30 PM, Martin Buchholz wrote: > line 63-64: ident further to the right as line 62 is the condition. > > Whitespace rejiggered. > Looks okay. I also think no need to have a separate copyToArrayDeque > method and just inline in the constructor. > It's used twice. Also, it's likely to be replaced someday when we decide what to do with lambdas, so good to keep as a separate method. From mandy.chung at oracle.com Mon Feb 26 23:38:43 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 26 Feb 2018 15:38:43 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> Message-ID: <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> On 2/26/18 3:26 PM, Martin Buchholz wrote: > > > Looks okay.? I also think no need to have a separate > copyToArrayDeque method and just inline in the constructor. > > > It's used twice.? Also, it's likely to be replaced someday when we > decide what to do with lambdas, so good to keep as a separate method. The first use can be replaced with very simple code: ArrayList path = new ArrayList<>(urls.length); ArrayDeque unopenedUrls = new ArrayDeque<>(urls.length); for (URL url : urls) { path.add(url); unopenedUrls.add(url); } Mandy From stuart.marks at oracle.com Tue Feb 27 00:23:02 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 26 Feb 2018 16:23:02 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <9543183a-65ec-8506-a955-eb90ec3cd6c8@oracle.com> Message-ID: To close the loop on this, I've reopened JDK-4993841, which requests adding an API Character.toString(int) which converts an int codepoint value to a String. (This seems like the obvious API, in parallel with Character.toString(char), but of course alternatives could be considered.) s'marks On 2/20/18 11:46 AM, Louis Wasserman wrote: > I'm with Brian: adding a separate API to make it easier to get from a > codepoint to a String seems independently merited, and makes the single > repeat API work well for that case. A very quick regex-powered search > comes up with 183 hits in Google for (new > String|String.copyValueOf|String.valueOf)(Character.toChars(..)). > > I do, however, recommend a separate thread for discussing that API :) > > On Tue, Feb 20, 2018 at 11:33 AM Kevin Bourrillion > wrote: > >> Just to add another dimension to this data: most of the usages of our >> repeat method (~75%) are in test code. These tests usually just want any >> old test string of a certain length. Repeating a single character is the >> obvious way to get that. >> >> Among production code usages (~25%), there are a few roughly equal use >> cases: ascii indentation/alignment, redaction, and Martin's expected case >> of "drawing" with ASCII symbols, and "other". >> >> >> >> On Thu, Feb 15, 2018 at 12:52 PM, Louis Wasserman >> wrote: >> >>> I don't think there's a case for demand to merit having a >>> repeat(CharSequence, int) at all. >>> >>> I did an analysis of usages of Guava's Strings.repeat on Google's >>> codebase. Users might be rolling their own implementations, too, but this >>> should be a very good proxy for demand. >>> >>> StringRepeat_SingleConstantChar = 4.475 K // strings with .length() == 1 >>> StringRepeat_SingleConstantCodePoint = 28 // strings with >>> .codePointCount(...) == 1 >>> StringRepeat_MultiCodePointConstant = 1.156 K // constant strings neither >>> of the above >>> StringRepeat_CharSequenceToString = 2 // >>> Strings.repeat(CharSequence.toString(), n) >>> StringRepeat_NoneOfTheAbove = 248 >>> >>> Notably, it seems like basically nobody needs to repeat a CharSequence -- >>> definitely not enough demand to merit the awkwardness of e.g. >>> Rope.repeat(n) inheriting a repeat returning a String. >>> >>> Based on this data, I'd recommend providing one and only one method of >>> this >>> type: String.repeat(int). There's no real advantage to a static >>> repeat(char, int) method when the overwhelming majority of these are >>> constants: e.g. compare SomeUtilClass.repeat('*', n) versus "*".repeat(n). >>> Character.toString(c).repeat(n) isn't a bad workaround if you don't have a >>> constant char. There also isn't much demand for dealing with the code >>> point case specially; the String.repeat(int) method seems like it'd handle >>> that just fine. >>> >>> On Thu, Feb 15, 2018 at 11:44 AM Jim Laskey >>> wrote: >>> >>>> >>>> >>>>> On Feb 15, 2018, at 3:36 PM, Ivan Gerasimov < >>> ivan.gerasimov at oracle.com> >>>> wrote: >>>>> >>>>> Hello! >>>>> >>>>> The link with the webrev returned 404, but I could find it at this >>>> location: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-00/ >>>>> >>>>> A few minor comments: >>>>> >>>>> 1) >>>>> >>>>> This check: >>>>> >>>>> 2992 final long limit = (long)count * 2L; >>>>> 2993 if ((long)Integer.MAX_VALUE < limit) { >>>>> >>>>> can be possibly simplified as >>>>> if (count > Integer.MAX_VALUE - count) { >>>> >>>> Good. >>>> >>>>> >>>>> 2) >>>>> Should String repeat(final int codepoint, final int count) be >>> optimized >>>> for codepoints that can be represented with a single char? >>>>> >>>>> E.g. like this: >>>>> >>>>> public static String repeat(final int codepoint, final int count) { >>>>> return Character.isBmpCodePoint(codepoint)) >>>>> ? repeat((char) codepoint, count) >>>>> : (new String(Character.toChars(codepoint))).repeat(count); >>>>> } >>>> >>>> Yes, avoid array allocation. >>>> >>>>> >>>>> 3) >>>>> Using long arithmetic can possibly be avoided in the common path of >>>> repeat(final int count): >>>>> >>>>> E.g. like this: >>>>> >>>>> if (count < 0) { >>>>> throw new IllegalArgumentException("count is negative, " + >>>> count); >>>>> } else if (count == 1) { >>>>> return this; >>>>> } else if (count == 0) { >>>>> return ""; >>>>> } >>>>> final int len = value.length; >>>>> if (Integer.MAX_VALUE / count < len) { >>>>> throw new IllegalArgumentException( >>>>> "Resulting string exceeds maximum string length: >>> " + >>>> ((long)len * (long)count)); >>>>> } >>>>> final int limit = count * len; >>>> >>>> Good. >>>> >>>> Thank you. >>>> >>>>> >>>>> With kind regards, >>>>> Ivan >>>>> >>>>> On 2/15/18 9:20 AM, Jim Laskey wrote: >>>>>> This is a pre-CSR code review [1] for String repeat methods >>>> (Enhancement). >>>>>> >>>>>> The proposal is to introduce four new methods; >>>>>> >>>>>> 1. public String repeat(final int count) >>>>>> 2. public static String repeat(final char ch, final int count) >>>>>> 3. public static String repeat(final int codepoint, final int count) >>>>>> 4. public static String repeat(final CharSequence seq, final int >>> count) >>>>>> >>>>>> For the sake of transparency, only 1 is necessary, 2-4 are >>> convenience >>>> methods. >>>>>> In the case of 2, ?*?.repeat(10) performs as well as >>> String.repeat(?*?, >>>> 10). >>>>>> 3 and 4 convert to String before calling 1. >>>>>> >>>>>> Performance runs with jmh (results as comment in [2]) show that these >>>>>> methods are significantly faster that StringBuilder equivalents. >>>>>> - fewer memory allocations >>>>>> - fewer char to byte array conversions >>>>>> - faster pyramid replication vs O(N) copying >>>>>> >>>>>> I left StringBuilder out of scope. It falls under the category of >>>>>> Appendables#append with repeat. A much bigger project. >>>>>> >>>>>> All comments welcome. Especially around the need for convenience >>>>>> methods, the JavaDoc content and expanding the tests. >>>>>> >>>>>> ? Jim >>>>>> >>>>>> [1] webrev: >>>> http://cr.openjdk.java.net//oj/home/jlaskey/8197594/webrev-00 >>>>>> [2] jbs: https://bugs.openjdk.java.net/browse/JDK-8197594 >>>>>> >>>>>> >>>>> >>>>> -- >>>>> With kind regards, >>>>> Ivan Gerasimov >>>>> >>>> >>>> >>> >> >> >> >> -- >> Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb at google.com >> From stuart.marks at oracle.com Tue Feb 27 00:57:30 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 26 Feb 2018 16:57:30 -0800 Subject: RFR: 8197594 - String and character repeat In-Reply-To: References: <4A9DE3C2-A58E-474D-8CE6-E1CBD317F78C@oracle.com> <2f6ff46d-ea2e-b4ee-5760-99f98f500393@oracle.com> <5FB78E84-C701-48A2-8F99-9F721DF2A111@oracle.com> <8C10050B-DA2B-4F4C-B0E7-FBFA5683390C@oracle.com> Message-ID: <85d3279a-979b-3b28-f494-264543e6ecfb@oracle.com> On 2/18/18 1:37 AM, James Laskey wrote: > Didn?t I hear someone mentioning ?\U1D11A? at some point? On 2/19/18 7:55 AM, Martin Buchholz wrote: > Oops, I already got it wrong - it's already at 6 hex digits because there are 17 > planes, not 16.? MAX_CODE_POINT is U+10FFFF. > Yes, we need a variable width syntax like regex?\x{h...h} Yeah, there are a bunch of syntactic alternatives to consider. An "obvious" alternative to "\uxxxx" is "\Uxxxxxx" which works if you're always willing to specify six digits (or to have some weird non-delimited but variable-length sequence, such as the existing octal escapes for chars (does anybody use those (see JLS 3.10.6)?)) The difference between \u and \U is rather subtle, though. Or a delimited sequence such as used by regex might be an alternative. > And java regex also supports > ? \N{name}The character with Unicode character name 'name' > so we could do the same for the java language. > Although it would be a little weird to have every Unicode update make some > previously invalid source files valid. > > We could also say "It's 2018 and UTF-8 has won" and simply use UTF-8 in source > files directly.? ?No Unicode escapes needed. Even if one is willing to have a source file in UTF-8 (as opposed to say, ASCII) there are things in Unicode that are really hard to edit. For example, there are zero-width spaces, joiners, non-joiners, directionality markers, etc. I think escapes are the bare minimum. Some kind of name-based interpolation would be useful, but the actual Unicode names are rather unwieldy. Maybe something like HTML entities would be worthwhile to investigate, though probably with a different syntax. s'marks From martinrb at google.com Tue Feb 27 01:00:17 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 26 Feb 2018 17:00:17 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> Message-ID: On Mon, Feb 26, 2018 at 3:38 PM, mandy chung wrote: > > > On 2/26/18 3:26 PM, Martin Buchholz wrote: > > > Looks okay. I also think no need to have a separate copyToArrayDeque >> method and just inline in the constructor. >> > > It's used twice. Also, it's likely to be replaced someday when we decide > what to do with lambdas, so good to keep as a separate method. > > > The first use can be replaced with very simple code: > > ArrayList path = new ArrayList<>(urls.length); > ArrayDeque unopenedUrls = new ArrayDeque<>(urls.length); > for (URL url : urls) { > path.add(url); > unopenedUrls.add(url); > } > > I had trouble deciding which way was better, so let's do it your way! toArrayDeque is gone. From igor.ignatyev at oracle.com Tue Feb 27 01:25:26 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 26 Feb 2018 17:25:26 -0800 Subject: RFR(XXS) : 8190679 : java/util/Arrays/TimSortStackSize2.java fails with "Initial heap size set to a larger value than the maximum heap size" Message-ID: <02AFE3F4-5D26-49B6-8787-B04817EDE6C1@oracle.com> http://cr.openjdk.java.net/~iignatyev//8190679/webrev.00/index.html > 9 lines changed: 2 ins; 0 del; 7 mod; Hi all, could you please review the patch for TimSortStackSize2 test? the test failed when externally passed (via -javaoption or -vmoption) -Xmx value is less than 770m or 385m, depending on UseCompressedOops. it happened because the test explicitly set Xms value, but didn't set Xmx. now, the test sets Xmx as Xms times 2. PS as it mostly affects hotspot testing, the patch will be pushed to jdk/hs. webrev: http://cr.openjdk.java.net/~iignatyev//8190679/webrev.00/index.html testing: java/util/Arrays/TimSortStackSize2.java w/ and w/o externally provided Xmx value JBS: https://bugs.openjdk.java.net/browse/JDK-8190679 Thanks, -- Igor From paul.sandoz at oracle.com Tue Feb 27 02:50:08 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 26 Feb 2018 18:50:08 -0800 Subject: [PATCH] Reduce Chance Of Mistakenly Early Backing Memory Cleanup In-Reply-To: References: <7215566D-A8BA-4D51-8262-E70E7FA55982@oracle.com> Message-ID: Hi Ben, Here is the webrev online: http://cr.openjdk.java.net/~psandoz/jdk/buffer-reachability-fence/webrev/index.html (I don?t know if you have any colleagues with author or above roles in OpenJDK to upload for you, it might be faster.) Reference.java ? 423 @ForceInline 424 public static void reachabilityFence(Object ref) { 425 // Does nothing, because this method is annotated with @DontInline 426 // HotSpot needs to retain the ref and not GC it before a call to this 427 // method 428 } We need to update the comment, preferably using a summary of Vladimir?s analysis. Direct-X-Buffer-bin.java.template ? 34 private $type$ get$Type$(long a) { 35 $memtype$ x = UNSAFE.get$Memtype$Unaligned(null, a, bigEndian); 36 $type$ y = $fromBits$(x); 37 Reference.reachabilityFence(this); 38 return y; 39 } It?s overkill in the above case but for good practice reasons i recommend for all usages using a try/finally block as suggested in the JavaDoc for Reference.reachabilityFence. Direct-X-Buffer.java.template ? 260 public $type$ get() { 261 return $fromBits$($swap$(UNSAFE.get$Swaptype$(ix(nextGetIndex())))); 262 } 263 264 public $type$ get(int i) { 265 return $fromBits$($swap$(UNSAFE.get$Swaptype$(ix(checkIndex(i))))); 266 } 267 268 #if[streamableType] 269 $type$ getUnchecked(int i) { 270 return $fromBits$($swap$(UNSAFE.get$Swaptype$(ix(i)))); 271 } 272 #end[streamableType] Missing fences. We also need to look carefully at the bulk operations as well, you have a fence for the bulk put accepting a ByteBuffer although a fence is likely required on the src as well. 506 byte _get(int i) { // package-private 507 return UNSAFE.getByte(address + i); 508 } AFAICT the _get and _put methods are no longer used and the code could be deleted (left over from other refactoring to the view classes). Thanks, Paul. > On Feb 19, 2018, at 8:37 AM, Ben Walsh wrote: > > As requested, here are the results with modifications to the annotations > on Reference.reachabilityFence. Much more promising ... > > > * Benchmark 1 * > > Test Code : > > package org.sample; > > import org.openjdk.jmh.annotations.Benchmark; > import org.openjdk.jmh.annotations.Level; > import org.openjdk.jmh.annotations.Scope; > import org.openjdk.jmh.annotations.Setup; > import org.openjdk.jmh.annotations.State; > > import java.nio.ByteBuffer; > > public class ByteBufferBenchmark { > > @State(Scope.Benchmark) > public static class ByteBufferContainer { > > ByteBuffer bb; > > @Setup(Level.Invocation) > public void initByteBuffer() { > bb = ByteBuffer.allocateDirect(1); > } > > ByteBuffer getByteBuffer() { > return bb; > } > } > > @Benchmark > public void benchmark_byte_buffer_put(ByteBufferContainer bbC) { > > bbC.getByteBuffer().put((byte)42); > } > > } > > Results : > > - Unmodified Build - > > Benchmark Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 35604933.518 ? > 654975.515 ops/s > > - Build With Reference.reachabilityFences Added - > > Benchmark Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 33100911.857 ? > 747461.951 ops/s -7.033% > > - Build With Reference.reachabilityFences Added And DontInline Replaced > With ForceInline - > > Benchmark Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 34836320.294 ? > 640188.408 ops/s -2.159% > > - Build With Reference.reachabilityFences Added And DontInline Removed - > > Benchmark Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put thrpt 200 34740015.332 ? > 556578.542 ops/s -2.429% > > > * Benchmark 2 * > > Test Code : > > package org.sample; > > import org.openjdk.jmh.annotations.Benchmark; > import org.openjdk.jmh.annotations.Level; > import org.openjdk.jmh.annotations.Param; > import org.openjdk.jmh.annotations.Scope; > import org.openjdk.jmh.annotations.Setup; > import org.openjdk.jmh.annotations.State; > > import java.nio.ByteBuffer; > > @State(Scope.Benchmark) > public class ByteBufferBenchmark { > > @Param({"1", "10", "100", "1000", "10000"}) > public int L; > > @State(Scope.Benchmark) > public static class ByteBufferContainer { > > ByteBuffer bb; > > @Setup(Level.Invocation) > public void initByteBuffer() { > bb = ByteBuffer.allocateDirect(10000); > } > > ByteBuffer getByteBuffer() { > return bb; > } > } > > @Benchmark > public ByteBuffer benchmark_byte_buffer_put(ByteBufferContainer bbC) { > > ByteBuffer bb = bbC.getByteBuffer(); > > for (int i = 0; i < L; i++) { > bb.put((byte)i); > } > > return bb; > } > > } > > Results : > > - Unmodified Build - > > Benchmark (L) Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29303145.752 ? 635979.750 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 24260859.017 ? 528891.303 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 8512366.637 ? 136615.070 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1323756.037 ? 21485.369 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 145965.305 ? 1301.469 ops/s > > - Build With Reference.reachabilityFences Added - > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 28893540.122 ? 754554.747 ops/s -1.398% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 15317696.355 ? 231621.608 ops/s -36.863% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 2546599.578 ? 32136.873 ops/s -70.084% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 288832.514 ? 3854.522 ops/s -78.181% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 29747.386 > ? 214.831 ops/s -79.620% > > - Build With Reference.reachabilityFences Added And DontInline Replaced > With ForceInline - > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29372326.859 ? 525988.179 ops/s +0.236% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 24326735.480 ? 484358.862 ops/s +0.272% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 8492692.912 ? 120924.878 ops/s -0.231% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1332131.417 ? 14981.587 ops/s +0.633% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 144990.569 ? 1518.877 ops/s -0.668% > > - Build With Reference.reachabilityFences Added And DontInline Removed - > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29842696.017 ? 462902.634 ops/s +1.841% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 24842729.069 ? 436174.452 ops/s +2.398% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 8518393.953 ? 129254.536 ops/s +0.071% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1344772.370 ? 15916.867 ops/s +1.588% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 145087.256 ? 1277.491 ops/s -0.602% > > > * Benchmark 3 * > > Test Code : > > package org.sample; > > import org.openjdk.jmh.annotations.Benchmark; > import org.openjdk.jmh.annotations.Level; > import org.openjdk.jmh.annotations.Param; > import org.openjdk.jmh.annotations.Scope; > import org.openjdk.jmh.annotations.Setup; > import org.openjdk.jmh.annotations.State; > > import java.nio.ByteBuffer; > > @State(Scope.Benchmark) > public class ByteBufferBenchmark { > > @Param({"1", "10", "100", "1000", "10000"}) > public int L; > > @State(Scope.Benchmark) > public static class ByteBufferContainer { > > ByteBuffer bb; > > @Setup(Level.Invocation) > public void initByteBuffer() { > bb = ByteBuffer.allocateDirect(4 * 10000); > > for (int i = 0; i < 10000; i++) { > bb.putInt(i); > } > } > > ByteBuffer getByteBuffer() { > return bb; > } > > } > > @Benchmark > public int benchmark_byte_buffer_put(ByteBufferContainer bbC) { > > ByteBuffer bb = bbC.getByteBuffer(); > > bb.position(0); > > int sum = 0; > > for (int i = 0; i < L; i++) { > sum += bb.getInt(); > } > > return sum; > > } > > } > > Results : > > - Unmodified Build - > > Benchmark (L) Mode Cnt Score > Error Units > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29677205.748 ? 544721.142 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 18219951.454 ? 320724.793 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 7767650.826 ? 121798.910 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1646075.010 ? 9804.499 ops/s > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 183489.418 ? 1355.967 ops/s > > - Build With Reference.reachabilityFences Added - > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 15230086.695 ? 390174.190 ops/s -48.681% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 8126310.728 ? 123661.342 ops/s -55.399% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 1582699.233 ? 7278.744 ops/s -79.624% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 179726.465 ? 802.333 ops/s -89.082% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 18327.049 > ? 9.506 ops/s -90.012% > > - Build With Reference.reachabilityFences Added And DontInline Replaced > With ForceInline - > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29839190.147 ? 576585.796 ops/s +0.546% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 18397768.759 ? 338144.327 ops/s +0.976% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 7746079.875 ? 101621.105 ops/s -0.278% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1629413.444 ? 24163.399 ops/s -1.012% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 182250.811 ? 2028.461 ops/s -0.675% > > - Build With Reference.reachabilityFences Added And DontInline Removed - > > Benchmark (L) Mode Cnt Score > Error Units Impact > ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 > 29442980.464 ? 556324.877 ops/s -0.789% > ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 > 18401757.539 ? 419383.901 ops/s +0.998% > ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 > 7816766.062 ? 100144.611 ops/s +0.632% > ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 > 1636811.564 ? 13811.447 ops/s -0.563% > ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 183463.292 ? 2056.016 ops/s -0.014% > > > Regards, > Ben > > > > From: Paul Sandoz > To: Ben Walsh > Cc: core-libs-dev > Date: 08/02/2018 16:54 > Subject: Re: [PATCH] Reduce Chance Of Mistakenly Early Backing > Memory Cleanup > > > > Hi Ben, > > Thanks. I anticipated a performance hit but not necessarily a 10x. Without > looking at the generated code of the benchmark method it is hard to be > sure [*], but i believe the fence is interfering with loop unrolling > and/or vectorization, the comparative differences between byte and int may > be related to vectorization (for byte there may be less or limited support > for vectorization). > > How about we now try another experiment commenting out the @DontInline on > the fence method and re-run the benchmarks. From Peter?s observations and > Vladimir?s analysis we should be able to remove that, or even, contrary to > what we initial expected when adding this feature, change to @ForceInline! > > Thanks, > Paul. > > [*] If you are running on linux you can use the excellent JMH perfasm > feature to dump the hot parts of HotSpots generated code. > >> On Feb 8, 2018, at 8:22 AM, Ben Walsh wrote: >> >> Hi Paul, >> >> Following up with the requested loop and vectorization benchmarks ... >> >> >> (Do the vectorization benchmark results imply that the Hotspot compiler >> has been unable to perform the vectorization optimisation due to the >> presence of the reachabilityFence ?) >> >> >> > ----------------------------------------------------------------------------------------------------------------------- >> >> >> Loop Benchmarking >> ---- ------------ >> >> package org.sample; >> >> import org.openjdk.jmh.annotations.Benchmark; >> import org.openjdk.jmh.annotations.Level; >> import org.openjdk.jmh.annotations.Param; >> import org.openjdk.jmh.annotations.Scope; >> import org.openjdk.jmh.annotations.Setup; >> import org.openjdk.jmh.annotations.State; >> >> import java.nio.ByteBuffer; >> >> @State(Scope.Benchmark) >> public class ByteBufferBenchmark { >> >> @Param({"1", "10", "100", "1000", "10000"}) >> public int L; >> >> @State(Scope.Benchmark) >> public static class ByteBufferContainer { >> >> ByteBuffer bb; >> >> @Setup(Level.Invocation) >> public void initByteBuffer() { >> bb = ByteBuffer.allocateDirect(10000); >> } >> >> ByteBuffer getByteBuffer() { >> return bb; >> } >> } >> >> @Benchmark >> public ByteBuffer benchmark_byte_buffer_put(ByteBufferContainer bbC) > { >> >> ByteBuffer bb = bbC.getByteBuffer(); >> >> for (int i = 0; i < L; i++) { >> bb.put((byte)i); >> } >> >> return bb; >> } >> >> } >> >> >> Without Changes >> >> Benchmark (L) Mode Cnt Score >> Error Units >> ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 >> 29303145.752 ? 635979.750 ops/s >> ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 >> 24260859.017 ? 528891.303 ops/s >> ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 >> 8512366.637 ? 136615.070 ops/s >> ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 >> 1323756.037 ? 21485.369 ops/s >> ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 >> 145965.305 ? 1301.469 ops/s >> >> >> With Changes >> >> Benchmark (L) Mode Cnt Score >> Error Units Impact >> ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 >> 28893540.122 ? 754554.747 ops/s -1.398% >> ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 >> 15317696.355 ? 231621.608 ops/s -36.863% >> ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 >> 2546599.578 ? 32136.873 ops/s -70.084% >> ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 >> 288832.514 ? 3854.522 ops/s -78.181% >> ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 29747.386 >> ? 214.831 ops/s -79.620% >> >> >> > ----------------------------------------------------------------------------------------------------------------------- >> >> >> Vectorization Benchmarking >> ------------- ------------ >> >> package org.sample; >> >> import org.openjdk.jmh.annotations.Benchmark; >> import org.openjdk.jmh.annotations.Level; >> import org.openjdk.jmh.annotations.Param; >> import org.openjdk.jmh.annotations.Scope; >> import org.openjdk.jmh.annotations.Setup; >> import org.openjdk.jmh.annotations.State; >> >> import java.nio.ByteBuffer; >> >> @State(Scope.Benchmark) >> public class ByteBufferBenchmark { >> >> @Param({"1", "10", "100", "1000", "10000"}) >> public int L; >> >> @State(Scope.Benchmark) >> public static class ByteBufferContainer { >> >> ByteBuffer bb; >> >> @Setup(Level.Invocation) >> public void initByteBuffer() { >> bb = ByteBuffer.allocateDirect(4 * 10000); >> >> for (int i = 0; i < 10000; i++) { >> bb.putInt(i); >> } >> } >> >> ByteBuffer getByteBuffer() { >> return bb; >> } >> >> } >> >> @Benchmark >> public int benchmark_byte_buffer_put(ByteBufferContainer bbC) { >> >> ByteBuffer bb = bbC.getByteBuffer(); >> >> bb.position(0); >> >> int sum = 0; >> >> for (int i = 0; i < L; i++) { >> sum += bb.getInt(); >> } >> >> return sum; >> >> } >> >> } >> >> >> Without Changes >> >> Benchmark (L) Mode Cnt Score >> Error Units >> ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 >> 29677205.748 ? 544721.142 ops/s >> ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 >> 18219951.454 ? 320724.793 ops/s >> ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 >> 7767650.826 ? 121798.910 ops/s >> ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 >> 1646075.010 ? 9804.499 ops/s >> ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 >> 183489.418 ? 1355.967 ops/s >> >> >> With Changes >> >> Benchmark (L) Mode Cnt Score >> Error Units Impact >> ByteBufferBenchmark.benchmark_byte_buffer_put 1 thrpt 200 >> 15230086.695 ? 390174.190 ops/s -48.681% >> ByteBufferBenchmark.benchmark_byte_buffer_put 10 thrpt 200 >> 8126310.728 ? 123661.342 ops/s -55.399% >> ByteBufferBenchmark.benchmark_byte_buffer_put 100 thrpt 200 >> 1582699.233 ? 7278.744 ops/s -79.624% >> ByteBufferBenchmark.benchmark_byte_buffer_put 1000 thrpt 200 >> 179726.465 ? 802.333 ops/s -89.082% >> ByteBufferBenchmark.benchmark_byte_buffer_put 10000 thrpt 200 > 18327.049 >> ? 9.506 ops/s -90.012% >> >> >> >> NB : For reference - for this and previous benchmarking results ... >> >> "Without Changes" and "With Changes" - java -version ... >> >> openjdk version "10-internal" 2018-03-20 >> OpenJDK Runtime Environment (build 10-internal+0-adhoc.walshbp.jdk) >> OpenJDK 64-Bit Server VM (build 10-internal+0-adhoc.walshbp.jdk, mixed >> mode) >> >> >> > ----------------------------------------------------------------------------------------------------------------------- >> >> >> Regards, >> Ben Walsh >> > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > From david.holmes at oracle.com Tue Feb 27 05:00:45 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 27 Feb 2018 15:00:45 +1000 Subject: RFR(XXS) : 8190679 : java/util/Arrays/TimSortStackSize2.java fails with "Initial heap size set to a larger value than the maximum heap size" In-Reply-To: <02AFE3F4-5D26-49B6-8787-B04817EDE6C1@oracle.com> References: <02AFE3F4-5D26-49B6-8787-B04817EDE6C1@oracle.com> Message-ID: Hi Igor, On 27/02/2018 11:25 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8190679/webrev.00/index.html >> 9 lines changed: 2 ins; 0 del; 7 mod; > > Hi all, > > could you please review the patch for TimSortStackSize2 test? > > the test failed when externally passed (via -javaoption or -vmoption) -Xmx value is less than 770m or 385m, depending on UseCompressedOops. it happened because the test explicitly set Xms value, but didn't set Xmx. > now, the test sets Xmx as Xms times 2. I'm not happy with setting Xmx at 2 times Xms - that seems to be setting ourselves up for another case where we can't set -Xmx at startup. This test has encountered problems in the past with external flag settings - see in particular the review thread for JDK-8075071: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-March/032316.html Will the test pass if we simply set -Xmx and -Xms to the same? Or (equivalently based on on previous review discussions) just set -Xmx instead of -Xms? Thanks, David > PS as it mostly affects hotspot testing, the patch will be pushed to jdk/hs. > > webrev: http://cr.openjdk.java.net/~iignatyev//8190679/webrev.00/index.html > testing: java/util/Arrays/TimSortStackSize2.java w/ and w/o externally provided Xmx value > JBS: https://bugs.openjdk.java.net/browse/JDK-8190679 > > Thanks, > -- Igor > From xu.y.yin at oracle.com Tue Feb 27 06:51:13 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 27 Feb 2018 14:51:13 +0800 Subject: [JDK 11] RFR 8196759: Move two java/text/Normalizer tests into OpenJDK Message-ID: <27D165CF-D330-4FB8-A6DC-4F861B9471ED@oracle.com> Please review the changes to move two java/text/Normalizer tests to OpenJDK jdk_text group, thanks bug: https://bugs.openjdk.java.net/browse/JDK-8196759 webrev: http://cr.openjdk.java.net/~xiaofeya/8196759/webrev.00/ Regards, Chris From adam.farley at uk.ibm.com Tue Feb 27 11:53:18 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Tue, 27 Feb 2018 11:53:18 +0000 Subject: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: <712770ae-72ee-10b8-cd93-fb1fc34bd5b3@oracle.com> References: <39D8F43A-06BD-483B-8901-6F4444A8235F@oracle.com> <4B429F1D-5727-4B20-A051-E39E1E8C69AA@oracle.com> <712770ae-72ee-10b8-cd93-fb1fc34bd5b3@oracle.com> Message-ID: Hi Alan, Peter, The main bit of data I wanted out of this was the sum total of native memory being used to store DirectByteBuffers, and to have that information printed in diagnostic cores. Thanks for pointing out Bits. I will investigate if there is a way to make that data available to the VM when a diagnostic core is generated (I'm poking through SharedSecrets and JavaNioAccess now) without running java code. Worst case scenario, we don't get this feature, and at least we can retrieve this information from the core by using, as Alan suggests, an SA- based tool to retrieve the state of the Bits variables at crash-time. Best Regards Adam Farley From: Alan Bateman To: Peter Levart , Adam Farley8 Cc: "hotspot-dev at openjdk.java.net developers" , core-libs-dev Date: 23/02/2018 17:52 Subject: Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers On 23/02/2018 15:28, Peter Levart wrote: > Hi Adam, > > Did you know that native memory is already tracked on the Java side for > direct ByteBuffers? See class java.nio.Bits. Could you make use of it? > Right, these are the fields that are exposed at runtime via BufferPoolMXBean. A SA based tool could read from a core file. I can't tell if this is enough for Adam, it may be that the his tool reveals more details on the buffers in the pools. -Alan Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From adam.farley at uk.ibm.com Tue Feb 27 15:04:48 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Tue, 27 Feb 2018 15:04:48 +0000 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java Message-ID: Resending. Bump. :) On 14/02/2018 14:13, Adam Farley8 wrote: >> Hi All, >> >> -- Short version -- >> >> Could a committer please take the fix for JDK-8190187 (full code included >> in the bug) and: >> >> 1) Complete the CSR process for the new JNI Return code. >> 2) Commit the changes that contain (a) the new return code, and (b) the >> non-Hotspot code that handles the new code. > The patches attached to the JIRA issue are missing the changes to the > JVM TI spec (jvmti.xml). I'm not seeing the JNI return codes in that file. Are you after one of those dated updates near the bottom? e.g. ``` Added JNI_SILENT_EXIT return code for early exits without errors. java.c's ParseArguments function now sets pret value to 2 for a NULL pwhat. This allows us to clearly identify when no class was set, but no other error has occurred. This is undone in java.c's ContinueInNewThread method, so the surface behaviour does not change. ``` > There is also text to be written for the JNI spec if this proposal goes > ahead. I assume you mean the "RETURNS" section of the JNI_CreateJavaVM bit on the invocation.html web page. Something like this? ``` RETURNS: Returns JNI_OK on success; returns a suitable JNI error code (a negative number) on failure. The sole exception is a silent exit, which returns JNI error code JNI_SILENT_EXIT. This indicates that the VM cannot be used, but that this is the intended behaviour for the arguments used. E.g. -Xlog:help (which prints help output and then destroys the VM) ``` > > I don't agree that the launcher should be looking for > "-agentlib:jdwp=help" in the command line as it's just one of many ways > that the debugger agent might be started (e.g. -Xrunjdwp:, > _JAVA_TOOLS_OPTIONS, ...). We can avoid that by finding a way around this line in ContinueInNewThread (java.c): ``` return (ret != 0) ? ret : rslt; ``` I have devised a means to do this, as outlined in the jvmti.xml change above. I put the changes into a recent clone of jdk/jdk, and attached the hg diff, along with an improved test. If you have Author or Committer privileges, could you add the files to the bug please? > >> Backporting would be appreciated, but is optional. > I don't think these changes are appropriate to backport as they include > specification changes/ > >-Alan This is fair. - Adam Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Alan.Bateman at oracle.com Tue Feb 27 15:20:23 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 27 Feb 2018 15:20:23 +0000 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java In-Reply-To: References: Message-ID: On 27/02/2018 15:04, Adam Farley8 wrote: > Resending. Bump. :) > > On 14/02/2018 14:13, Adam Farley8 wrote: > >> Hi All, > >> > >> -- Short version -- > >> > >> Could a committer please take the fix for JDK-8190187 (full code > included > >> in the bug) and: > >> > >> 1) Complete the CSR process for the new JNI Return code. > >> 2) Commit the changes that contain (a) the new return code, and (b) the > >> non-Hotspot code that handles the new code. > > The patches attached to the JIRA issue are missing the changes to the > > JVM TI spec (jvmti.xml). > > I'm not seeing the JNI return codes in that file. Are you after one of > those > dated updates near the bottom? > e.g. > ``` > > Added JNI_SILENT_EXIT return code for early exits without errors. > java.c's ParseArguments function now sets pret value to 2 for a NULL > pwhat. > ? ?This allows us to clearly identify when no class was set, but no > other error has occurred. > ? ?This is undone in java.c's ContinueInNewThread method, so the > surface behaviour does not change. > > ``` The "Agent Start-Up" section is the section to look at. The important part is: "The return value from|Agent_OnLoad|or|Agent_OnLoad_|is used to indicate an error. Any value other than zero indicates an error and causes termination of the VM." If there is special return value to mean "VM terminates without error" then this part of the spec will need to be adjusted. An additional point is that you can start several agents from the command line, does the VM terminate after it has started all agents or does it terminate when the first agent returns asks the VM to terminate quietly? > > > > There is also text to be written for the JNI spec if this proposal goes > > ahead. > > I assume you mean the "RETURNS" section of the JNI_CreateJavaVM > bit on the invocation.html web page. Something like this? > > ``` > RETURNS: > Returns JNI_OK on success; returns a suitable JNI error code (a > negative number) on failure. > > The sole exception is a silent exit, which returns JNI error code > JNI_SILENT_EXIT. > This indicates that the VM cannot be used, but that this is the > intended behaviour for the > arguments used. E.g. -Xlog:help (which prints help output and then > destroys the VM) > ``` Yes, this is the place that will need changes. > > > > > > I don't agree that the launcher should be looking for > > "-agentlib:jdwp=help" in the command line as it's just one of many ways > > that the debugger agent might be started (e.g. -Xrunjdwp:, > > _JAVA_TOOLS_OPTIONS, ...). > > We can avoid that by finding a way around this line in > ContinueInNewThread (java.c): > > ``` > return (ret != 0) ? ret : rslt; > ``` > > I have devised a means to do this, as outlined in the jvmti.xml change > above. I put the > changes into a recent clone of jdk/jdk, and attached the hg diff, > along with an improved > test. The launcher should only need to look at the return value from JNI CraeateJavaVM. I don't think it should do any special handling for the JDWP or other agents (there are just too many ways to inject command lines and the launcher cannot be expected to handle all of them). -Alan From claes.redestad at oracle.com Tue Feb 27 15:42:01 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 27 Feb 2018 16:42:01 +0100 Subject: RFR: 8198755: Reduce cost of InvokerBytecodeGenerator::isStaticallyInvocable/-Nameable Message-ID: <3c5d6248-2fd8-bc3c-7e86-d1e3ee30ab70@oracle.com> Hi, when generating LF classes with InvokerBytecodeGenerator, almost 5% of executed code is concerned with determining if members and parameters are statically invocable and/or nameable, which we can assert is always true for MethodHandle arguments. Since MethodHandle targets and parameters are common, it's a profitable startup optimization to add a fast-path test for this. Webrev: http://cr.openjdk.java.net/~redestad/8198755/jdk.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8198755 Thanks! /Claes From paul.sandoz at oracle.com Tue Feb 27 16:42:16 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 27 Feb 2018 08:42:16 -0800 Subject: RFR: 8198755: Reduce cost of InvokerBytecodeGenerator::isStaticallyInvocable/-Nameable In-Reply-To: <3c5d6248-2fd8-bc3c-7e86-d1e3ee30ab70@oracle.com> References: <3c5d6248-2fd8-bc3c-7e86-d1e3ee30ab70@oracle.com> Message-ID: <9F446882-D55E-42F0-B5B8-34E7C8A0C742@oracle.com> > On Feb 27, 2018, at 7:42 AM, Claes Redestad wrote: > > Hi, > > when generating LF classes with InvokerBytecodeGenerator, almost 5% of executed code is concerned with determining if members and parameters are statically invocable and/or nameable, which we can assert is always true for MethodHandle arguments. > IIRC this is for the case where we can reliably crack open the method handle (from it?s member name) and invoke it directly? > Since MethodHandle targets and parameters are common, it's a profitable startup optimization to add a fast-path test for this. > > Webrev: http://cr.openjdk.java.net/~redestad/8198755/jdk.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8198755 > 957 for (int i = 0; i < mtype.parameterCount(); i++) 958 if (!isStaticallyNameable(mtype.parameterType(i))) Revert back to: for (Class ptype : mtype.parameterArray()) if (!isStaticallyNameable(ptype)) ? Paul. From john.r.rose at oracle.com Tue Feb 27 16:53:15 2018 From: john.r.rose at oracle.com (John Rose) Date: Tue, 27 Feb 2018 08:53:15 -0800 Subject: RFR: 8198755: Reduce cost of InvokerBytecodeGenerator::isStaticallyInvocable/-Nameable In-Reply-To: <9F446882-D55E-42F0-B5B8-34E7C8A0C742@oracle.com> References: <3c5d6248-2fd8-bc3c-7e86-d1e3ee30ab70@oracle.com> <9F446882-D55E-42F0-B5B8-34E7C8A0C742@oracle.com> Message-ID: Looks good. On Feb 27, 2018, at 8:42 AM, Paul Sandoz wrote: > > > >> On Feb 27, 2018, at 7:42 AM, Claes Redestad wrote: >> >> Hi, >> >> when generating LF classes with InvokerBytecodeGenerator, almost 5% of executed code is concerned with determining if members and parameters are statically invocable and/or nameable, which we can assert is always true for MethodHandle arguments. >> > > IIRC this is for the case where we can reliably crack open the method handle (from it?s member name) and invoke it directly? Yes, invoked directly from inside LF code in java.lang.invoke. The 'member' parameter already gives evidence that somebody has gotten the right to call the method indirectly via a MH, and the LF generator wants to lower that to a direct invocation at bytecode level. That's why MH package-private members are called directly. > >> Since MethodHandle targets and parameters are common, it's a profitable startup optimization to add a fast-path test for this. >> >> Webrev: http://cr.openjdk.java.net/~redestad/8198755/jdk.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8198755 >> > > > 957 for (int i = 0; i < mtype.parameterCount(); i++) > 958 if (!isStaticallyNameable(mtype.parameterType(i))) > > Revert back to: > > for (Class ptype : mtype.parameterArray()) > if (!isStaticallyNameable(ptype)) > > ? +1 ? John From claes.redestad at oracle.com Tue Feb 27 18:01:36 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 27 Feb 2018 19:01:36 +0100 Subject: RFR: 8198755: Reduce cost of InvokerBytecodeGenerator::isStaticallyInvocable/-Nameable In-Reply-To: References: <3c5d6248-2fd8-bc3c-7e86-d1e3ee30ab70@oracle.com> <9F446882-D55E-42F0-B5B8-34E7C8A0C742@oracle.com> Message-ID: On 2018-02-27 17:53, John Rose wrote: > Looks good. Thanks! I'll revert for-loop (it was a remnant of an earlier experiment to see if getting rid of the ptypes.clone() mattered for startup: it didn't, really) /Claes From naoto.sato at oracle.com Tue Feb 27 18:39:49 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 27 Feb 2018 10:39:49 -0800 Subject: [JDK 11] RFR 8196759: Move two java/text/Normalizer tests into OpenJDK In-Reply-To: <27D165CF-D330-4FB8-A6DC-4F861B9471ED@oracle.com> References: <27D165CF-D330-4FB8-A6DC-4F861B9471ED@oracle.com> Message-ID: <8dfd330f-bc8b-c7cd-cb73-c1254558c515@oracle.com> Looks good. Naoto On 2/26/18 10:51 PM, Chris Yin wrote: > Please review the changes to move two java/text/Normalizer tests to OpenJDK jdk_text group, thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8196759 > webrev: http://cr.openjdk.java.net/~xiaofeya/8196759/webrev.00/ > > Regards, > Chris > From huizhe.wang at oracle.com Tue Feb 27 19:37:04 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 27 Feb 2018 11:37:04 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A8F7752.9000302@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> <5A8DC7A9.30104@oracle.com> <5A8DCF9E.50508@oracle.com> <124fb31a-e508-0a01-98c7-dff9b345b3e6@oracle.com> <5A8F2D45.1070906@oracle.com> <5A8F7752.9000302@oracle.com> Message-ID: Hi Sherman and all, Thanks for the further reviews! Here's the latest webrev with boundary checks in StringLatin1/StringUTF16: JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ Best, Joe On 2/22/2018 6:07 PM, Joe Wang wrote: > > > On 2/22/18, 12:51 PM, Xueming Shen wrote: >> On 2/22/18, 12:04 PM, Joe Wang wrote: >>> Hi Sherman, >>> >>> Thanks for reviewing the change. >>> >>> Taking a local copy of the count field, but the boundary check would >>> be almost immediately done against the field itself. Are you >>> worrying about the count field may be out of sync with the byte >>> array? I would think that's unlikely to happen. Whether it's >>> StringBuilder or StringBuffer, it's not advisable/practical to use >>> in multiple threads. In a valid usage, the count is always >>> consistent with the byte array. >>> >> >> Hi Joe, >> >> It might not be a "valid usage" but it did happen and when it happens >> it might just crash the >> vm without those boundary checks. It's especially true for those >> intrinsics methods with explicit >> comments "intrinsic performs no bounds checks". In this case, the >> StringUTF16.getChar() is being >> called in new public method StringUTF16.compareTo(byte[], byte[], >> int, int) without appropriate >> boundary check. In the "old" code the "index" is guaranteed to be >> within [0, len) in >> StringUTF16.compareTo(byte[], byte[]), so it's safe. A real case for >> such scenario can be found in >> JDK-8158168 [1], for example. > > Thanks for the pointer! The email thread helps a lot. I've updated the > webrev with a boundary check in ASB (AbstractStringBuilder line 106, > 107), and then a note to StringUTF16.compareTo (StringUTF16 line 280). > Hopefully this is sufficient. Didn't want to add any check in > StringUTF16 since that may affect the original two-arg method. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > -Joe > >> >> -Sherman >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8158168 From xueming.shen at oracle.com Tue Feb 27 20:15:07 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 27 Feb 2018 12:15:07 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> <5A8DC7A9.30104@oracle.com> <5A8DCF9E.50508@oracle.com> <124fb31a-e508-0a01-98c7-dff9b345b3e6@oracle.com> <5A8F2D45.1070906@oracle.com> <5A8F7752.9000302@oracle.com> Message-ID: <5A95BC4B.4080303@oracle.com> +1 On 2/27/18, 11:37 AM, Joe Wang wrote: > Hi Sherman and all, > > Thanks for the further reviews! > > Here's the latest webrev with boundary checks in > StringLatin1/StringUTF16: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Best, > Joe > > On 2/22/2018 6:07 PM, Joe Wang wrote: >> >> >> On 2/22/18, 12:51 PM, Xueming Shen wrote: >>> On 2/22/18, 12:04 PM, Joe Wang wrote: >>>> Hi Sherman, >>>> >>>> Thanks for reviewing the change. >>>> >>>> Taking a local copy of the count field, but the boundary check >>>> would be almost immediately done against the field itself. Are you >>>> worrying about the count field may be out of sync with the byte >>>> array? I would think that's unlikely to happen. Whether it's >>>> StringBuilder or StringBuffer, it's not advisable/practical to use >>>> in multiple threads. In a valid usage, the count is always >>>> consistent with the byte array. >>>> >>> >>> Hi Joe, >>> >>> It might not be a "valid usage" but it did happen and when it >>> happens it might just crash the >>> vm without those boundary checks. It's especially true for those >>> intrinsics methods with explicit >>> comments "intrinsic performs no bounds checks". In this case, the >>> StringUTF16.getChar() is being >>> called in new public method StringUTF16.compareTo(byte[], byte[], >>> int, int) without appropriate >>> boundary check. In the "old" code the "index" is guaranteed to be >>> within [0, len) in >>> StringUTF16.compareTo(byte[], byte[]), so it's safe. A real case for >>> such scenario can be found in >>> JDK-8158168 [1], for example. >> >> Thanks for the pointer! The email thread helps a lot. I've updated >> the webrev with a boundary check in ASB (AbstractStringBuilder line >> 106, 107), and then a note to StringUTF16.compareTo (StringUTF16 line >> 280). Hopefully this is sufficient. Didn't want to add any check in >> StringUTF16 since that may affect the original two-arg method. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> -Joe >> >>> >>> -Sherman >>> >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8158168 > From igor.ignatyev at oracle.com Tue Feb 27 21:17:12 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 27 Feb 2018 13:17:12 -0800 Subject: RFR(XXS) : 8190679 : java/util/Arrays/TimSortStackSize2.java fails with "Initial heap size set to a larger value than the maximum heap size" In-Reply-To: References: <02AFE3F4-5D26-49B6-8787-B04817EDE6C1@oracle.com> Message-ID: Hi David, I have set Xmx equal to Xms, the test passes w/ different externally passed combinations of Xmx, Xms and UseCompressedOops. http://cr.openjdk.java.net/~iignatyev//8190679/webrev.01/index.html Thanks, -- Igor > On Feb 26, 2018, at 9:00 PM, David Holmes wrote: > > Hi Igor, > > On 27/02/2018 11:25 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8190679/webrev.00/index.html >>> 9 lines changed: 2 ins; 0 del; 7 mod; >> Hi all, >> could you please review the patch for TimSortStackSize2 test? >> the test failed when externally passed (via -javaoption or -vmoption) -Xmx value is less than 770m or 385m, depending on UseCompressedOops. it happened because the test explicitly set Xms value, but didn't set Xmx. >> now, the test sets Xmx as Xms times 2. > > I'm not happy with setting Xmx at 2 times Xms - that seems to be setting ourselves up for another case where we can't set -Xmx at startup. This test has encountered problems in the past with external flag settings - see in particular the review thread for JDK-8075071: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-March/032316.html > > Will the test pass if we simply set -Xmx and -Xms to the same? Or (equivalently based on on previous review discussions) just set -Xmx instead of -Xms? > > Thanks, > David > >> PS as it mostly affects hotspot testing, the patch will be pushed to jdk/hs. >> webrev: http://cr.openjdk.java.net/~iignatyev//8190679/webrev.00/index.html >> testing: java/util/Arrays/TimSortStackSize2.java w/ and w/o externally provided Xmx value >> JBS: https://bugs.openjdk.java.net/browse/JDK-8190679 >> Thanks, >> -- Igor From joe.darcy at oracle.com Wed Feb 28 01:18:17 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 27 Feb 2018 17:18:17 -0800 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> <5A90B204.4070005@oracle.com> Message-ID: <5A960359.4070507@oracle.com> Hi Liam, On 2/24/2018 5:14 PM, Liam Miller-Cushon wrote: > Hi, thanks for the comments. > > The updated webrev is at: > http://cr.openjdk.java.net/~cushon/7183985/webrev.02/ > > > On Fri, Feb 23, 2018 at 4:29 PM, Joseph D. Darcy > wrote: > > Objects implementing the AnnotatedElement interface are also > created in javac during annotation processing. As a secondary > concern, it would be good to be have behavior between both javac > and runtime annotations consistent when possible. (My own to-do > list includes at least once such alignment JDK-8164819: "Make > javac's toString() on annotation objects consistent with core > reflection".) > > > Do you have a pointer to where that happens? There's > javax.lang.model.AnnotatedConstruct#getAnnotation, which isn't > affected by this issue--it throws MirroredTypesExceptionProxy when > accessing an annotation value that is an array of class literals, > regardless of whether the elements' classes are missing. I'm not > seeing implementations of AnnotatedElement in langtools. Sorry, I misremembered the situation in javac. For annotation processing in javac, AnnotatedElement is not a factor, but the class src/jdk.compiler/share/classes/com/sun/tools/javac/model/AnnotationProxyMaker.java in javac does create annotation proxies, which is the same general technique used to create the annotation objects at runtime for core reflection. These annotation objects in javac for javax.lang.model and annotation processing, do not follow the full contract of java.lang.reflect.AnnotatedElement. In particular, the annotations returned are not necessarily serializable. The intersection of methods on AnnotatedConstruct and AnnotatedElement is only a proper subset of the methods on AnnotatedElement; however, getAnnotation?(Class annotationClass) is in the intersection. > Even in javac we've moved away from test and directory names based > on bugid. I'd recommend incorporating these regression tests into > the existing tests in > test/jdk/java/lang/annotation/Missing > > > Thanks for the reminder, done. I believe the new tests could reuse some of the existing types in test/jdk/java/lang/annotation/Missing. For example, the new MissingAnnotation.java is an alpha-rename of the existing Missing.java. If such sharing is not practical, then I'd recommend putting the new files into a subdirectory under underneath test/.../annotation/Missing (otherwise it will be confusing to edit these tests in the future since too many file names will start with "Missing".) In MissingClassArrayElementTest.java: 71 public static void main(String[] args) throws Exception { 72 ClassArrayAnnotation[] outer = Test.class.getAnnotation(AnnotationAnnotation.class).value(); 73 // The second entry in the values array can be loaded successfully 74 assertEquals(Arrays.toString(outer[1].value()), "[class java.lang.String]"); Something like assertEquals(Arrays.equals(outer[1].value(), {String.class}) would be more robust against any future changes in Class.toString. Likewise for the analogous comparisons. > It would be worth verifying whether or not this change also covers > java.lang.reflect.Executable.getParameterAnnotations(), more > specifically the implementation of that method in Method and > Constructor. The method getParameterAnnotations() is much less > used than getAnnotations and the other methods on the > AnnotatedElement interface, but still part of the annotations feature. > > > Done. Thanks. Cheers, -Joe From xu.y.yin at oracle.com Wed Feb 28 01:52:05 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Wed, 28 Feb 2018 09:52:05 +0800 Subject: [JDK 11] RFR 8196759: Move two java/text/Normalizer tests into OpenJDK In-Reply-To: <8dfd330f-bc8b-c7cd-cb73-c1254558c515@oracle.com> References: <27D165CF-D330-4FB8-A6DC-4F861B9471ED@oracle.com> <8dfd330f-bc8b-c7cd-cb73-c1254558c515@oracle.com> Message-ID: <0FF1D214-DB0D-4350-909B-28627AD7CEB3@oracle.com> Thank you, Naoto Regards, Chris > On 28 Feb 2018, at 2:39 AM, Naoto Sato wrote: > > Looks good. > > Naoto > > On 2/26/18 10:51 PM, Chris Yin wrote: >> Please review the changes to move two java/text/Normalizer tests to OpenJDK jdk_text group, thanks >> bug: https://bugs.openjdk.java.net/browse/JDK-8196759 >> webrev: http://cr.openjdk.java.net/~xiaofeya/8196759/webrev.00/ >> Regards, >> Chris From martinrb at google.com Wed Feb 28 03:02:02 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 27 Feb 2018 19:02:02 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> Message-ID: I should probably do more testing than usual when touching classloading... We have a jdi test that does this (hg blame finds duke as only author): public static ClassLoader classLoaderValue; { try { urls[0] = new URL("hi there"); } catch (java.net.MalformedURLException ee) { } classLoaderValue = new URLClassLoader(urls); } The assignment to urls[0] never succeeds (!), so URLClassLoader(URL[]) is called with an array holding a null. Which should cause a NPE, but did not prior to JDK-8198484 . ArrayDeque does not permit null elements! Should we accept the NPE as a bug fix and file a retroactive CSR for that, or should we continue to compatibly tolerate null URLs (yuk)? Can we get jdi team to fix their dodgy tests? From joe.darcy at oracle.com Wed Feb 28 03:11:38 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 27 Feb 2018 19:11:38 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> Message-ID: <5A961DEA.2090909@oracle.com> Hi Martin, Until we figure out the best course of action, please problem list the failing test. Thanks, -Joe On 2/27/2018 7:02 PM, Martin Buchholz wrote: > I should probably do more testing than usual when touching classloading... > > We have a jdi test that does this (hg blame finds duke as only author): > > public static ClassLoader classLoaderValue; > { > try { > urls[0] = new URL("hi there"); > } catch (java.net.MalformedURLException ee) { > } > classLoaderValue = new URLClassLoader(urls); > } > > The assignment to urls[0] never succeeds (!), so URLClassLoader(URL[]) is > called with an array holding a null. Which should cause a NPE, but did not > prior to JDK-8198484 . > ArrayDeque does not permit null elements! > > Should we accept the NPE as a bug fix and file a retroactive CSR for that, > or should we continue to compatibly tolerate null URLs (yuk)? > > Can we get jdi team to fix their dodgy tests? From david.holmes at oracle.com Wed Feb 28 03:11:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Feb 2018 13:11:52 +1000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> Message-ID: <524b46ed-d391-b47b-0b66-0087f8b6bdea@oracle.com> Hi Martin, On 28/02/2018 1:02 PM, Martin Buchholz wrote: > I should probably do more testing than usual when touching classloading... > > We have a jdi test that does this (hg blame finds duke as only author): > > ? ? public static ClassLoader classLoaderValue; > ? ? { > ? ? ? ? try { > ? ? ? ? ? ? urls[0] = new URL("hi there"); > ? ? ? ? } catch (java.net.MalformedURLException ee) { > ? ? ? ? } > ? ? ? ? classLoaderValue = new URLClassLoader(urls); > ? ? } > > The assignment to urls[0] never succeeds (!), so URLClassLoader(URL[]) > is called with an array holding a null.? Which should cause a NPE, but > did not prior to JDK-8198484 > . > ArrayDeque does not permit null elements! > > Should we accept the NPE as a bug fix and file a retroactive CSR for > that, or should we continue to compatibly tolerate null URLs (yuk)? As I wrote in the bug report the tolerance for nulls is very limited. Seems you can have them in the initial URL[] but if you actually try to use one you get NPE. So accepting a null entry in the URL[] seems rather pointless - and addURL explicitly states that null is ignored. So I'd argue for the CSR so we can at least consider whether we need tolerate this. > Can we get jdi team to fix their dodgy tests? Yes. Depending on which way we go we may need a new bug, otherwise this one can be assigned back to hotspot->svc. I'm also perplexed by the test logic. Thanks, David From david.holmes at oracle.com Wed Feb 28 04:33:29 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Feb 2018 14:33:29 +1000 Subject: RFR(XXS) : 8190679 : java/util/Arrays/TimSortStackSize2.java fails with "Initial heap size set to a larger value than the maximum heap size" In-Reply-To: References: <02AFE3F4-5D26-49B6-8787-B04817EDE6C1@oracle.com> Message-ID: On 28/02/2018 7:17 AM, Igor Ignatyev wrote: > Hi David, > > I have set Xmx equal to Xms, the test passes w/ different externally > passed combinations of Xmx, Xms and UseCompressedOops. > > http://cr.openjdk.java.net/~iignatyev//8190679/webrev.01/index.html Looks good! Thanks, David > Thanks, > -- Igor > >> On Feb 26, 2018, at 9:00 PM, David Holmes > > wrote: >> >> Hi Igor, >> >> On 27/02/2018 11:25 AM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8190679/webrev.00/index.html >>>> 9 lines changed: 2 ins; 0 del; 7 mod; >>> Hi all, >>> could you please review the patch for TimSortStackSize2 test? >>> the test failed when externally passed (via -javaoption or -vmoption) >>> -Xmx value is less than 770m or 385m, depending on UseCompressedOops. >>> it happened because the test explicitly set Xms value, but didn't set >>> Xmx. >>> now, the test sets Xmx as Xms times 2. >> >> I'm not happy with setting Xmx at 2 times Xms - that seems to be >> setting ourselves up for another case where we can't set -Xmx at >> startup. This test has encountered problems in the past with external >> flag settings - see in particular the review thread for JDK-8075071: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-March/032316.html >> >> Will the test pass if we simply set -Xmx and -Xms to the same? Or >> (equivalently based on on previous review discussions) just set -Xmx >> instead of -Xms? >> >> Thanks, >> David >> >>> PS as it mostly affects hotspot testing, the patch will be pushed to >>> jdk/hs. >>> webrev: >>> http://cr.openjdk.java.net/~iignatyev//8190679/webrev.00/index.html >>> testing: java/util/Arrays/TimSortStackSize2.java ?w/ and w/o >>> externally provided Xmx value >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8190679 >>> Thanks, >>> -- Igor > From martinrb at google.com Wed Feb 28 05:03:57 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 27 Feb 2018 21:03:57 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: <524b46ed-d391-b47b-0b66-0087f8b6bdea@oracle.com> References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> <524b46ed-d391-b47b-0b66-0087f8b6bdea@oracle.com> Message-ID: 8198808: jdi tests failing after JDK-8198484 http://cr.openjdk.java.net/~martin/webrevs/jdk/jdi-ProblemList/ https://bugs.openjdk.java.net/browse/JDK-8198808 From david.holmes at oracle.com Wed Feb 28 05:28:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Feb 2018 15:28:30 +1000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> <524b46ed-d391-b47b-0b66-0087f8b6bdea@oracle.com> Message-ID: <45c4cd17-a63d-0327-b76e-6258389ca243@oracle.com> Looks good. Thanks, David On 28/02/2018 3:03 PM, Martin Buchholz wrote: > 8198808: jdi tests failing after JDK-8198484 > http://cr.openjdk.java.net/~martin/webrevs/jdk/jdi-ProblemList/ > https://bugs.openjdk.java.net/browse/JDK-8198808 > From mandy.chung at oracle.com Wed Feb 28 05:37:28 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 27 Feb 2018 21:37:28 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> <524b46ed-d391-b47b-0b66-0087f8b6bdea@oracle.com> Message-ID: On 2/27/18 9:03 PM, Martin Buchholz wrote: > 8198808: jdi tests failing after JDK-8198484 > http://cr.openjdk.java.net/~martin/webrevs/jdk/jdi-ProblemList/ > > https://bugs.openjdk.java.net/browse/JDK-8198808 > +1 Mandy From amy.lu at oracle.com Wed Feb 28 08:53:33 2018 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 28 Feb 2018 16:53:33 +0800 Subject: [JDK 11] Problem list tools/jimage/JImageExtractTest.java for macosx-all Message-ID: <7e1205c3-c185-a854-7983-e88befd63bd1@oracle.com> Please review the patch to problem list tools/jimage/JImageExtractTest.java for macosx-all. This test fails frequently (observed at Mac) and should be problem listed before?JDK-8198819?fixed. bug: https://bugs.openjdk.java.net/browse/JDK-8198820 webrev: http://cr.openjdk.java.net/~amlu/8198820/webrev.00/ Thanks, Amy --- old/test/jdk/ProblemList.txt 2018-02-28 16:48:06.000000000 +0800 +++ new/test/jdk/ProblemList.txt 2018-02-28 16:48:06.000000000 +0800 @@ -467,7 +467,7 @@ tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all -tools/jimage/JImageExtractTest.java 8198405 windows-all +tools/jimage/JImageExtractTest.java 8198405,8198819 windows-all,macosx-all tools/jimage/JImageListTest.java 8198405 windows-all ############################################################################ From Alan.Bateman at oracle.com Wed Feb 28 10:23:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 28 Feb 2018 10:23:39 +0000 Subject: [JDK 11] Problem list tools/jimage/JImageExtractTest.java for macosx-all In-Reply-To: <7e1205c3-c185-a854-7983-e88befd63bd1@oracle.com> References: <7e1205c3-c185-a854-7983-e88befd63bd1@oracle.com> Message-ID: <299dc67c-c0c7-38d6-3728-9d82ff509fc1@oracle.com> On 28/02/2018 08:53, Amy Lu wrote: > Please review the patch to problem list > tools/jimage/JImageExtractTest.java for macosx-all. > > This test fails frequently (observed at Mac) and should be problem > listed before?JDK-8198819?fixed. > > bug: https://bugs.openjdk.java.net/browse/JDK-8198820 > webrev: http://cr.openjdk.java.net/~amlu/8198820/webrev.00/ This looks okay, also no objection to excluding it on all platforms as it is seems to fail periodically on Solaris too. -Alan From amy.lu at oracle.com Wed Feb 28 11:29:58 2018 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 28 Feb 2018 19:29:58 +0800 Subject: [JDK 11] Problem list tools/jimage/JImageExtractTest.java for macosx-all In-Reply-To: <299dc67c-c0c7-38d6-3728-9d82ff509fc1@oracle.com> References: <7e1205c3-c185-a854-7983-e88befd63bd1@oracle.com> <299dc67c-c0c7-38d6-3728-9d82ff509fc1@oracle.com> Message-ID: <366fbae0-4dd7-b15b-6717-c0aa6e93dc11@oracle.com> On 28/02/2018 6:23 PM, Alan Bateman wrote: > > > On 28/02/2018 08:53, Amy Lu wrote: >> Please review the patch to problem list >> tools/jimage/JImageExtractTest.java for macosx-all. >> >> This test fails frequently (observed at Mac) and should be problem >> listed before?JDK-8198819?fixed. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8198820 >> webrev: http://cr.openjdk.java.net/~amlu/8198820/webrev.00/ > This looks okay, also no objection to excluding it on all platforms as > it is seems to fail periodically on Solaris too. Thank you Alan! Changed to problem list for generic-all and pushed. Thanks, Amy > > -Alan From cello86 at gmail.com Wed Feb 28 11:40:36 2018 From: cello86 at gmail.com (Marcello Lorenzi) Date: Wed, 28 Feb 2018 12:40:36 +0100 Subject: sun.rmi.transport.tcp.responseTimeout ignored Message-ID: Hi All, We started a new Java standalone application with OpenJDK Runtime Environment (build 1.8.0_161-b14). This application has a network problem to connect to a RMI service and the application threads remained in hang status. We tried to apply the parameter sun.rmi.transport.tcp.responseTimeout but the threads on JVM side remained in stuck and the connection to the RMI server was in SYN_SENT status. Could it be related with bug https://bugs.openjdk.java.net/browse/JDK-8151212? Thanks, Marcello From claes.redestad at oracle.com Wed Feb 28 13:16:28 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 28 Feb 2018 14:16:28 +0100 Subject: RFR: 8198831: Lazy initialization of ValueConversions MethodHandles Message-ID: Hi, this patch makes the lookup of various MH in sun.invoke.util.ValueConversions lazy, realizing a tiny startup optimization to various apps. Webrev: http://cr.openjdk.java.net/~redestad/8198831/jdk.00/ Bug:??? https://bugs.openjdk.java.net/browse/JDK-8198831 Thanks! /Claes From shade at redhat.com Wed Feb 28 13:48:51 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 28 Feb 2018 14:48:51 +0100 Subject: RFR: 8198831: Lazy initialization of ValueConversions MethodHandles In-Reply-To: References: Message-ID: <28726598-5a6c-7320-06b9-bccd128ab61d@redhat.com> On 02/28/2018 02:16 PM, Claes Redestad wrote: > this patch makes the lookup of various MH in sun.invoke.util.ValueConversions lazy, realizing a tiny > startup optimization to various apps. > > Webrev: http://cr.openjdk.java.net/~redestad/8198831/jdk.00/ Looks good to me. -Aleksey From Alan.Bateman at oracle.com Wed Feb 28 14:17:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 28 Feb 2018 14:17:11 +0000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> Message-ID: On 28/02/2018 03:02, Martin Buchholz wrote: > I should probably do more testing than usual when touching classloading... > > We have a jdi test that does this (hg blame finds duke as only author): > > ? ? public static ClassLoader classLoaderValue; > ? ? { > ? ? ? ? try { > ? ? ? ? ? ? urls[0] = new URL("hi there"); > ? ? ? ? } catch (java.net.MalformedURLException ee) { > ? ? ? ? } > ? ? ? ? classLoaderValue = new URLClassLoader(urls); > ? ? } > > : > > Can we get jdi team to fix their dodgy tests? I checked pre OpenJDK history but I'm none the wiser. I assume the original author of this test assumed that urls[0] would be set to placeholder URL, it's just the MalformedURLException (which is must catch because it's a checked exception) masked the test bug. java.net.URLClassLoader and its implementation in URLClassPath implementation haven't had a lot of TLC which probably explains why the null issue didn't surface before. We also need to change the one-arg constructor to use varargs to avoid needing to create an explicit array for the common case that you want to create it with a single URL. -Alan From claes.redestad at oracle.com Wed Feb 28 14:22:49 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 28 Feb 2018 15:22:49 +0100 Subject: RFR: 8198831: Lazy initialization of ValueConversions MethodHandles In-Reply-To: <28726598-5a6c-7320-06b9-bccd128ab61d@redhat.com> References: <28726598-5a6c-7320-06b9-bccd128ab61d@redhat.com> Message-ID: On 2018-02-28 14:48, Aleksey Shipilev wrote: > On 02/28/2018 02:16 PM, Claes Redestad wrote: >> this patch makes the lookup of various MH in sun.invoke.util.ValueConversions lazy, realizing a tiny >> startup optimization to various apps. >> >> Webrev: http://cr.openjdk.java.net/~redestad/8198831/jdk.00/ > Looks good to me. Thanks, Aleksey! /Claes From Roger.Riggs at Oracle.com Wed Feb 28 14:45:37 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 28 Feb 2018 09:45:37 -0500 Subject: RFR 8198697: Simplify platform encoding initialization tweak Message-ID: <0ea3699b-e4ab-ceca-e0d7-d41c2ed675f4@Oracle.com> Hi, In an effort to untangle some of the issues with property initialization I was looking at the platform encoding initialization and found a simplification. Currently, the initialization occurs as a side effect of the first call to JNU_NewStringPlatform and involves a upcall to get sun.jnu.encoding from the system properties.? The value is cached for later use. The native System.initProperties determines the platform specific encoding via java_props_md.c and does an upcall to set the sun.jnu.encoding system property, taking care to do it before the first string that needs platform encoding. The change directly initializes the platform encoding fast path before it is needed to encode platform strings. And moves the setting sun.jnu.encoding system property after the command line arguments are inserted keeping it from being overridden by -D on the command line that can only confuse confusion with code that later reads the property. Please review and comment. webrev: ?? http://cr.openjdk.java.net/~rriggs/webrev-simplify-jnu-8198697/ Issue: ?? https://bugs.openjdk.java.net/browse/JDK-8198697 Thanks, Roger From Roger.Riggs at Oracle.com Wed Feb 28 15:41:25 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 28 Feb 2018 10:41:25 -0500 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A95BC4B.4080303@oracle.com> References: <5A6A99B7.1050703@oracle.com> <5A72A646.7040101@oracle.com> <2c4d2612-4937-05a9-0227-f1c0c0eeb0ce@oracle.com> <092b68a0-ffa1-0277-6f84-5d7db5239a8d@oracle.com> <5A8DC7A9.30104@oracle.com> <5A8DCF9E.50508@oracle.com> <124fb31a-e508-0a01-98c7-dff9b345b3e6@oracle.com> <5A8F2D45.1070906@oracle.com> <5A8F7752.9000302@oracle.com> <5A95BC4B.4080303@oracle.com> Message-ID: <77bbe740-ed51-f839-b878-f5036564ba79@Oracle.com> +1; looks good On 2/27/2018 3:15 PM, Xueming Shen wrote: > +1 > > On 2/27/18, 11:37 AM, Joe Wang wrote: >> Hi Sherman and all, >> >> Thanks for the further reviews! >> >> Here's the latest webrev with boundary checks in >> StringLatin1/StringUTF16: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> Best, >> Joe >> >> On 2/22/2018 6:07 PM, Joe Wang wrote: >>> >>> >>> On 2/22/18, 12:51 PM, Xueming Shen wrote: >>>> On 2/22/18, 12:04 PM, Joe Wang wrote: >>>>> Hi Sherman, >>>>> >>>>> Thanks for reviewing the change. >>>>> >>>>> Taking a local copy of the count field, but the boundary check >>>>> would be almost immediately done against the field itself. Are you >>>>> worrying about the count field may be out of sync with the byte >>>>> array? I would think that's unlikely to happen. Whether it's >>>>> StringBuilder or StringBuffer, it's not advisable/practical to use >>>>> in multiple threads. In a valid usage, the count is always >>>>> consistent with the byte array. >>>>> >>>> >>>> Hi Joe, >>>> >>>> It might not be a "valid usage" but it did happen and when it >>>> happens it might just crash the >>>> vm without those boundary checks. It's especially true for those >>>> intrinsics methods with explicit >>>> comments "intrinsic performs no bounds checks". In this case, the >>>> StringUTF16.getChar() is being >>>> called in new public method StringUTF16.compareTo(byte[], byte[], >>>> int, int) without appropriate >>>> boundary check. In the "old" code the "index" is guaranteed to be >>>> within [0, len) in >>>> StringUTF16.compareTo(byte[], byte[]), so it's safe. A real case >>>> for such scenario can be found in >>>> JDK-8158168 [1], for example. >>> >>> Thanks for the pointer! The email thread helps a lot. I've updated >>> the webrev with a boundary check in ASB (AbstractStringBuilder line >>> 106, 107), and then a note to StringUTF16.compareTo (StringUTF16 >>> line 280). Hopefully this is sufficient. Didn't want to add any >>> check in StringUTF16 since that may affect the original two-arg method. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> -Joe >>> >>>> >>>> -Sherman >>>> >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8158168 >> > From james.laskey at oracle.com Wed Feb 28 16:31:23 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 28 Feb 2018 12:31:23 -0400 Subject: RFR: JDK-8197594: String#repeat Message-ID: <55005B6D-CCD8-4B20-BD62-57DAAFCE8E56@oracle.com> Introduction of a new instance method String::repeat to allow an efficient and concise approach for generating repeated character sequences as strings. Performance information in JBS. Thank you. Cheers, ? Jim JBS: https://bugs.openjdk.java.net/browse/JDK-8197594 CSR: https://bugs.openjdk.java.net/browse/JDK-8198296 Webrev: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-02/index.html JavaDoc: http://cr.openjdk.java.net/~jlaskey/8197594/String.html From Randy.Crihfield at Oracle.com Wed Feb 28 16:57:28 2018 From: Randy.Crihfield at Oracle.com (Randy Crihfield) Date: Wed, 28 Feb 2018 11:57:28 -0500 Subject: RFR: 8193660 Check SOURCE line in "release" file for closedjdk In-Reply-To: <5A85C26D.7010800@Oracle.com> References: <5A85C26D.7010800@Oracle.com> Message-ID: <5A96DF78.4030109@Oracle.com> poke poke can someone please review? Thanks Randy On 02/15/18 12:25 PM, Randy Crihfield wrote: > I need to revise the resource file test created for JDK-8192837 > > The new bug is https://bugs.openjdk.java.net/browse/JDK-8193660 > > The webrev is located at > http://cr.openjdk.java.net/~shurailine/8193660/webrev.00/ > > > Any comments/suggestions are welcome, also I will need a sponsor for > it at the end? > > Randy From Alan.Bateman at oracle.com Wed Feb 28 16:58:58 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 28 Feb 2018 16:58:58 +0000 Subject: RFR: 8193660 Check SOURCE line in "release" file for closedjdk In-Reply-To: <5A96DF78.4030109@Oracle.com> References: <5A85C26D.7010800@Oracle.com> <5A96DF78.4030109@Oracle.com> Message-ID: <1b16da8d-f227-153d-64b6-97bc971d31e3@oracle.com> On 28/02/2018 16:57, Randy Crihfield wrote: > > poke poke can someone please review?? Thanks > The `release` is a properties file so I assume you can simplify the test by using Properties.load and then testing if the "SOURCE" key is present and whether it has the expected value. -Alan. From martinrb at google.com Wed Feb 28 18:04:41 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 28 Feb 2018 10:04:41 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> Message-ID: On Wed, Feb 28, 2018 at 6:17 AM, Alan Bateman wrote: > On 28/02/2018 03:02, Martin Buchholz wrote: > > I should probably do more testing than usual when touching classloading... > > We have a jdi test that does this (hg blame finds duke as only author): > > public static ClassLoader classLoaderValue; > { > try { > urls[0] = new URL("hi there"); > } catch (java.net.MalformedURLException ee) { > } > classLoaderValue = new URLClassLoader(urls); > } > > : > > Can we get jdi team to fix their dodgy tests? > > I checked pre OpenJDK history but I'm none the wiser. I assume the > original author of this test assumed that urls[0] would be set to > placeholder URL, it's just the MalformedURLException (which is must catch > because it's a checked exception) masked the test bug. > Yes, it's likely this is a classic example of "impossible exception swallowing" that is no longer considered acceptable. Of course converting to an unchecked exception like AssertionError would have been correct and would have caught the mistake. > java.net.URLClassLoader and its implementation in URLClassPath > implementation haven't had a lot of TLC which probably explains why the > null issue didn't surface before. We also need to change the one-arg > constructor to use varargs to avoid needing to create an explicit array for > the common case that you want to create it with a single URL. > Too bad the URL[] argument is not consistently the last one, else one could convert them all to varargs. Varargs-friendliness seems like an independent change. I'm not really a java.net guy (yet?) - I invite you to take over this sub-task of "improving URLClassLoader's constructors". From Alan.Bateman at oracle.com Wed Feb 28 18:09:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 28 Feb 2018 18:09:22 +0000 Subject: RFR: Here are some URLClassPath patches In-Reply-To: References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> Message-ID: <90892fc9-31f0-1da8-46f9-56c91865cb99@oracle.com> On 28/02/2018 18:04, Martin Buchholz wrote: > : > > Too bad the URL[] argument is not consistently the last one, else one > could? convert them all to varargs. Varargs-friendliness seems like an > independent change. Yes, completely separate issue and one that is already tracked: https://bugs.openjdk.java.net/browse/JDK-8068425 > > I'm not really a java.net guy (yet?) - I invite you > to take over this sub-task of "improving URLClassLoader's constructors". From Randy.Crihfield at Oracle.com Wed Feb 28 18:15:37 2018 From: Randy.Crihfield at Oracle.com (Randy Crihfield) Date: Wed, 28 Feb 2018 13:15:37 -0500 Subject: RFR: 8193660 Check SOURCE line in "release" file for closedjdk In-Reply-To: <1b16da8d-f227-153d-64b6-97bc971d31e3@oracle.com> References: <5A85C26D.7010800@Oracle.com> <5A96DF78.4030109@Oracle.com> <1b16da8d-f227-153d-64b6-97bc971d31e3@oracle.com> Message-ID: <5A96F1C9.2060403@Oracle.com> On 02/28/18 11:58 AM, Alan Bateman wrote: > On 28/02/2018 16:57, Randy Crihfield wrote: >> >> poke poke can someone please review? Thanks >> > The `release` is a properties file so I assume you can simplify the > test by using Properties.load and then testing if the "SOURCE" key is > present and whether it has the expected value. > > -Alan. I could though I actually already wrote the test as requested initially and read the file as desired at that time, which is in the workspace. The current incantation is to expand the test to also check a closed build and streamline, rather than write two tests. I would like to get approval as it is, but if I need to go back and change existing behavior I will. Randy From lance.andersen at oracle.com Wed Feb 28 18:25:55 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 28 Feb 2018 13:25:55 -0500 Subject: RFR JDK-8197533 move javax.transaction.xa into its own module Message-ID: Hi all, This RFR request moves the javax.transaction.xa package out of the java.sql module and into its own module java.transaction.xa. One of the motivators for this change is due to the fact JSR 907 1.3 MR indicated that the javax.transaction.xa package will be subsumed by Java SE. There should be no compatibility issues with this change. Any module that `requires java.sql` will continue to have access to the public classes in the javax.transaction.xa package at both compile-time and run-time. The CSR has been approved The webrev can be found at: http://cr.openjdk.java.net/~lancea/8197533/webrev.00/ Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From paul.sandoz at oracle.com Wed Feb 28 18:40:38 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 28 Feb 2018 10:40:38 -0800 Subject: RFR JDK-8197533 move javax.transaction.xa into its own module In-Reply-To: References: Message-ID: Compatible module refactoring in action! Looks good, one comment: test/jdk/javax/transaction/xa/testng/JavaSqlModuleDriver.java This is not a valid Java source file can you merge the jtreg meta data into XAExceptionTests instead? Paul. > On Feb 28, 2018, at 10:25 AM, Lance Andersen wrote: > > Hi all, > > This RFR request moves the javax.transaction.xa package out of the java.sql module and into its own module java.transaction.xa. One of the motivators for this change is due to the fact JSR 907 1.3 MR indicated that the javax.transaction.xa package will be subsumed by Java SE. > > There should be no compatibility issues with this change. Any module that `requires java.sql` will continue to have access to the public classes in the javax.transaction.xa package at both compile-time and run-time. > > > The CSR has been approved > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8197533/webrev.00/ > > Best > Lance > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From lance.andersen at oracle.com Wed Feb 28 19:20:45 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 28 Feb 2018 14:20:45 -0500 Subject: RFR JDK-8197533 move javax.transaction.xa into its own module In-Reply-To: References: Message-ID: <0EC6FFD0-31EA-4D24-B82D-C4D5DCAB8E65@oracle.com> Hi Paul, Thank you for the review. > On Feb 28, 2018, at 1:40 PM, Paul Sandoz wrote: > > Compatible module refactoring in action! > > Looks good, one comment: > > test/jdk/javax/transaction/xa/testng/JavaSqlModuleDriver.java > > This is not a valid Java source file can you merge the jtreg meta data into XAExceptionTests instead? As we discussed offline, I will change the above file and Driver.java. I naively assumed this was OK as the change to add Driver.java was made after I had originally added these tests in 2015. Best Lance > > Paul. > >> On Feb 28, 2018, at 10:25 AM, Lance Andersen > wrote: >> >> Hi all, >> >> This RFR request moves the javax.transaction.xa package out of the java.sql module and into its own module java.transaction.xa. One of the motivators for this change is due to the fact JSR 907 1.3 MR indicated that the javax.transaction.xa package will be subsumed by Java SE. >> >> There should be no compatibility issues with this change. Any module that `requires java.sql` will continue to have access to the public classes in the javax.transaction.xa package at both compile-time and run-time. >> >> >> The CSR has been approved >> >> The webrev can be found at: http://cr.openjdk.java.net/~lancea/8197533/webrev.00/ >> >> Best >> Lance >> > >> > > >> >Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Alan.Bateman at oracle.com Wed Feb 28 19:43:59 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 28 Feb 2018 19:43:59 +0000 Subject: RFR JDK-8197533 move javax.transaction.xa into its own module In-Reply-To: References: Message-ID: On 28/02/2018 18:25, Lance Andersen wrote: > Hi all, > > This RFR request moves the javax.transaction.xa package out of the java.sql module and into its own module java.transaction.xa. One of the motivators for this change is due to the fact JSR 907 1.3 MR indicated that the javax.transaction.xa package will be subsumed by Java SE. > > There should be no compatibility issues with this change. Any module that `requires java.sql` will continue to have access to the public classes in the javax.transaction.xa package at both compile-time and run-time. > > I skipped the tests but everything else looks good. It is a compatible change as you noted. Is there any XA text from the original JTA spec that should be added to the module description as part of this? Another way to ask this is whether the JTA 1.3 drops any text dealing with the XA part. -Alan From lance.andersen at oracle.com Wed Feb 28 19:54:56 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 28 Feb 2018 14:54:56 -0500 Subject: RFR JDK-8197533 move javax.transaction.xa into its own module In-Reply-To: References: Message-ID: <98DD56F9-A3F0-4D1D-BCCA-C72220225074@oracle.com> > On Feb 28, 2018, at 2:43 PM, Alan Bateman wrote: > > On 28/02/2018 18:25, Lance Andersen wrote: >> Hi all, >> >> This RFR request moves the javax.transaction.xa package out of the java.sql module and into its own module java.transaction.xa. One of the motivators for this change is due to the fact JSR 907 1.3 MR indicated that the javax.transaction.xa package will be subsumed by Java SE. >> >> There should be no compatibility issues with this change. Any module that `requires java.sql` will continue to have access to the public classes in the javax.transaction.xa package at both compile-time and run-time. >> >> > I skipped the tests but everything else looks good. It is a compatible change as you noted. Thank you Alan > > Is there any XA text from the original JTA spec that should be added to the module description as part of this? Another way to ask this is whether the JTA 1.3 drops any text dealing with the XA part. Still waiting to see what changes are made to the PDF spec, that is still needing to be completed. So I think for now, we go with what we have and can circle back if needed. Best Lance > > -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From patrick at reini.net Wed Feb 28 20:50:50 2018 From: patrick at reini.net (Patrick Reinhart) Date: Wed, 28 Feb 2018 21:50:50 +0100 Subject: RFR 8196298 Add null Reader and Writer Message-ID: <1DA3A26E-F18C-4621-BC6B-4165CC2CECA5@reini.net> Hi every boy that reviewed I tried to incorporate all feedback I received so far and updated the webrev accordingly: http://cr.openjdk.java.net/~reinhapa/reviews/8196298/webrev.01 -Patrick From lance.andersen at oracle.com Wed Feb 28 21:22:15 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 28 Feb 2018 16:22:15 -0500 Subject: RFR JDK-8197533 move javax.transaction.xa into its own module In-Reply-To: <0EC6FFD0-31EA-4D24-B82D-C4D5DCAB8E65@oracle.com> References: <0EC6FFD0-31EA-4D24-B82D-C4D5DCAB8E65@oracle.com> Message-ID: <144F8EF0-0020-4577-8F9B-3FC86AF8ED4A@oracle.com> > On Feb 28, 2018, at 2:20 PM, Lance Andersen wrote: > > Hi Paul, > > Thank you for the review. >> On Feb 28, 2018, at 1:40 PM, Paul Sandoz > wrote: >> >> Compatible module refactoring in action! >> >> Looks good, one comment: >> >> test/jdk/javax/transaction/xa/testng/JavaSqlModuleDriver.java >> >> This is not a valid Java source file can you merge the jtreg meta data into XAExceptionTests instead? > > As we discussed offline, I will change the above file and Driver.java. I naively assumed this was OK as the change to add Driver.java was made after I had originally added these tests in 2015. http://cr.openjdk.java.net/~lancea/8197533/webrev.01/ has the updated tests Best Lance > > Best > Lance >> >> Paul. >> >>> On Feb 28, 2018, at 10:25 AM, Lance Andersen >> wrote: >>> >>> Hi all, >>> >>> This RFR request moves the javax.transaction.xa package out of the java.sql module and into its own module java.transaction.xa. One of the motivators for this change is due to the fact JSR 907 1.3 MR indicated that the javax.transaction.xa package will be subsumed by Java SE. >>> >>> There should be no compatibility issues with this change. Any module that `requires java.sql` will continue to have access to the public classes in the javax.transaction.xa package at both compile-time and run-time. >>> >>> >>> The CSR has been approved >>> >>> The webrev can be found at: http://cr.openjdk.java.net/~lancea/8197533/webrev.00/ >>> >>> Best >>> Lance >>> >> >>> >> >> >>> >>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com > >> > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From magnus.ihse.bursie at oracle.com Wed Feb 28 21:51:53 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 28 Feb 2018 22:51:53 +0100 Subject: RFR JDK-8197533 move javax.transaction.xa into its own module In-Reply-To: References: Message-ID: On 2018-02-28 19:25, Lance Andersen wrote: > Hi all, > > This RFR request moves the javax.transaction.xa package out of the java.sql module and into its own module java.transaction.xa. One of the motivators for this change is due to the fact JSR 907 1.3 MR indicated that the javax.transaction.xa package will be subsumed by Java SE. > > There should be no compatibility issues with this change. Any module that `requires java.sql` will continue to have access to the public classes in the javax.transaction.xa package at both compile-time and run-time. > > > The CSR has been approved > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8197533/webrev.00/ Build changes look good. /Magnus > > Best > Lance > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From joe.darcy at oracle.com Wed Feb 28 21:55:15 2018 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 28 Feb 2018 13:55:15 -0800 Subject: RFR JDK-8197533 move javax.transaction.xa into its own module In-Reply-To: <144F8EF0-0020-4577-8F9B-3FC86AF8ED4A@oracle.com> References: <0EC6FFD0-31EA-4D24-B82D-C4D5DCAB8E65@oracle.com> <144F8EF0-0020-4577-8F9B-3FC86AF8ED4A@oracle.com> Message-ID: <5da666dc-07e7-17ee-80bc-25fa08aa1830@oracle.com> Hi Lance, I'd prefer if src/java.sql/share/classes/javax/transaction/xa/package.html was replaced by a package-info.java files as opposed to another package.html file in the new module. Thanks, -Joe On 2/28/2018 1:22 PM, Lance Andersen wrote: >> On Feb 28, 2018, at 2:20 PM, Lance Andersen wrote: >> >> Hi Paul, >> >> Thank you for the review. >>> On Feb 28, 2018, at 1:40 PM, Paul Sandoz > wrote: >>> >>> Compatible module refactoring in action! >>> >>> Looks good, one comment: >>> >>> test/jdk/javax/transaction/xa/testng/JavaSqlModuleDriver.java >>> >>> This is not a valid Java source file can you merge the jtreg meta data into XAExceptionTests instead? >> As we discussed offline, I will change the above file and Driver.java. I naively assumed this was OK as the change to add Driver.java was made after I had originally added these tests in 2015. > http://cr.openjdk.java.net/~lancea/8197533/webrev.01/ has the updated tests > Best > Lance >> Best >> Lance >>> Paul. >>> >>>> On Feb 28, 2018, at 10:25 AM, Lance Andersen >> wrote: >>>> >>>> Hi all, >>>> >>>> This RFR request moves the javax.transaction.xa package out of the java.sql module and into its own module java.transaction.xa. One of the motivators for this change is due to the fact JSR 907 1.3 MR indicated that the javax.transaction.xa package will be subsumed by Java SE. >>>> >>>> There should be no compatibility issues with this change. Any module that `requires java.sql` will continue to have access to the public classes in the javax.transaction.xa package at both compile-time and run-time. >>>> >>>> >>>> The CSR has been approved >>>> >>>> The webrev can be found at: http://cr.openjdk.java.net/~lancea/8197533/webrev.00/ >>>> >>>> Best >>>> Lance >>>> >> >>>> >> >> >>>> >>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>> Oracle Java Engineering >>>> 1 Network Drive >>>> Burlington, MA 01803 >>>> Lance.Andersen at oracle.com > >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From stuart.marks at oracle.com Wed Feb 28 22:21:14 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 28 Feb 2018 14:21:14 -0800 Subject: RFR: JDK-8197594: String#repeat In-Reply-To: <55005B6D-CCD8-4B20-BD62-57DAAFCE8E56@oracle.com> References: <55005B6D-CCD8-4B20-BD62-57DAAFCE8E56@oracle.com> Message-ID: Hi Jim, Implementation looks good. I'd suggest a couple small editorial changes to the spec: 2966 /** 2967 * Returns a string whose value is the concatenation of this 2968 * string repeated {@code count} times. 2969 *

2970 * If count or length is zero then the empty string is returned. I went looking for a 'length' parameter, until I realized it means the length of *this string*. So maybe clarify that. (And also line 2977.) 2971 *

2972 * This method may be used to create space padding for 2973 * formatting text or zero padding for formatting numbers. I don't think having this text is necessary, but if you want it, I'd suggest putting it into an @apiNote. 2974 * @param count number of times to repeat 2975 * @return A string composed of this string repeated 2976 * {@code count} times or the empty string if count 2977 * or length is zero. Thanks, s'marks On 2/28/18 8:31 AM, Jim Laskey wrote: > Introduction of a new instance method String::repeat to allow an efficient and concise approach for generating repeated character sequences as strings. > > Performance information in JBS. > > Thank you. > > Cheers, > > ? Jim > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8197594 > CSR: https://bugs.openjdk.java.net/browse/JDK-8198296 > > Webrev: http://cr.openjdk.java.net/~jlaskey/8197594/webrev-02/index.html > JavaDoc: http://cr.openjdk.java.net/~jlaskey/8197594/String.html > > > From cushon at google.com Wed Feb 28 23:16:03 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Wed, 28 Feb 2018 15:16:03 -0800 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: <5A960359.4070507@oracle.com> References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> <5A90B204.4070005@oracle.com> <5A960359.4070507@oracle.com> Message-ID: Hi Joe, The updated webrev is at: http://cr.openjdk.java.net/~cushon/7183985/webrev.03/ I added some more test coverage for arrays that contain both missing and non-missing classes, and for arrays that contain multiple different missing classes. In the latter case the exception proxy for the first missing class is returned. I also consolidate the 'typeError' flag and exceptionProxy, and just check if the latter is set. On Tue, Feb 27, 2018 at 5:18 PM, Joseph D. Darcy wrote: > Sorry, I misremembered the situation in javac. For annotation processing > in javac, AnnotatedElement is not a factor, but the class > > src/jdk.compiler/share/classes/com/sun/tools/javac/ > model/AnnotationProxyMaker.java > > in javac does create annotation proxies, which is the same general > technique used to create the annotation objects at runtime for core > reflection. These annotation objects in javac for javax.lang.model and > annotation processing, do not follow the full contract of java.lang.reflect.AnnotatedElement. > In particular, the annotations returned are not necessarily serializable. > The intersection of methods on AnnotatedConstruct and AnnotatedElement is > only a proper subset of the methods on AnnotatedElement; however, > getAnnotation?(Class annotationClass) is in the intersection. > Understood, thanks. I did some manual testing. There are currently some issues related to JDK-8187950, but with Jan's pending fix for that bug everything looks good to me. Here's a demo: https://gist.github.com/cushon/804021a520cec1832594c6e1138cebdd. The API correctly models an array of Class values where some classes are missing by using ErrorTypes for the elements whose classes are missing. > I believe the new tests could reuse some of the existing types in > test/jdk/java/lang/annotation/Missing. For example, the new > MissingAnnotation.java is an alpha-rename of the existing Missing.java. If > such sharing is not practical, then I'd recommend putting the new files > into a subdirectory under underneath test/.../annotation/Missing (otherwise > it will be confusing to edit these tests in the future since too many file > names will start with "Missing".) > I moved the new tests to a subdirectory. I think the rename of Missing to MissingAnnotation is helpful to distinguish it from the inputs for the class and enum tests, and I wanted to avoid refactoring the existing test case, but if you have additional suggestions about the best way to structure this I'm happy to reorganize things. > Something like > > assertEquals(Arrays.equals(outer[1].value(), {String.class}) > > would be more robust against any future changes in Class.toString. > Likewise for the analogous comparisons. > Done. Thanks, Liam From martinrb at google.com Wed Feb 28 23:58:01 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 28 Feb 2018 15:58:01 -0800 Subject: RFR: Here are some URLClassPath patches In-Reply-To: <90892fc9-31f0-1da8-46f9-56c91865cb99@oracle.com> References: <912a66bb-45d3-0248-7f8e-b43da9b33ff9@oracle.com> <684a8186-6d29-1e52-34cd-3ceb13b2ff4f@oracle.com> <90892fc9-31f0-1da8-46f9-56c91865cb99@oracle.com> Message-ID: OK, that was weird... All I did for testing was follow // This section should be uncommented if 8026517 is fixed. but ... 8026517 is marked fixed! So switching to ArrayDeque accidentally fixed 8026517 for real?! 8198810: URLClassLoader does not specify behavior when URL array contains null http://cr.openjdk.java.net/~martin/webrevs/jdk/URLClassLoader-NPE/ https://bugs.openjdk.java.net/browse/JDK-8198810