From m-matsushima at bk.jp.nec.com Fri Sep 1 02:34:30 2017 From: m-matsushima at bk.jp.nec.com (Mitsuru Matsushima) Date: Fri, 1 Sep 2017 02:34:30 +0000 Subject: [10] RFR 8180469: Wrong short form text for supplemental Japanese era In-Reply-To: <04f6fa07-019f-27f3-5971-b2b27d40ca0c@oracle.com> References: <04f6fa07-019f-27f3-5971-b2b27d40ca0c@oracle.com> Message-ID: Hi Naoto-san, The fix looks good, though I'm not a reviewer... By the way, I may have forgotten to inform you that there exist an issue at the short form of SimpleDateFormat has an issue. The SimpleDateFormat class is only capable to treat two form, Short and Long. At JDK9, the CLDR Provider become to default, the provider returns the same value for the Short form and the Long form. So, the behavior of SimpleDateFormat is incompatible to previous versions. (See the Comparison table, I described before.) --- Mitsuru > -----Original Message----- > From: i18n-dev [mailto:i18n-dev-bounces at openjdk.java.net] On Behalf Of Naoto Sato > Sent: Thursday, August 31, 2017 7:56 AM > To: core-libs-dev ; i18n-dev > Subject: [10] RFR 8180469: Wrong short form text for supplemental Japanese era > > Hi, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8180469 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8180469/webrev.00/ > > The problem was caused by the difference of the Era display name for "SHORT" style between java.time and java.util.Calendar. > > Naoto From mandy.chung at oracle.com Fri Sep 1 05:39:54 2017 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 31 Aug 2017 22:39:54 -0700 Subject: Review Request JDK-8186050: StackFrame should provide the method signature In-Reply-To: <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> References: <5ad2e4a3-3fd3-0df4-c6c4-0b9ca8068a4e@oracle.com> <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> Message-ID: <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> Updated webrev: http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.01/index.html This introduces two new methods, StackFrame::getMethodType and StackFrame::getDescriptor. Mandy On 8/30/17 12:25 AM, Remi Forax wrote: > Hi Mandy, > thanks for taking care of this. > > In my opinion, we should provide both getMethodType() and getDescriptor(), > getDescriptor() is handy for logging (finding the right overload when line numbers are not present) and getMethodType() is the one you whant if you want to inspect the runtime view of the stack frames (and by example interact with java.lang.invoke). For me, it's the same reason that give us getDeclaringClass() and getClassName() in the current API. > > So getDescriptor() can be called with no restriction but getMethodType() requires RETAIN_CLASS_REFERENCE. > > regards, > R?mi > > ----- Mail original ----- >> De: "mandy chung" >> ?: "core-libs-dev" >> Envoy?: Mardi 29 Ao?t 2017 00:57:28 >> Objet: Review Request JDK-8186050: StackFrame should provide the method signature >> Method signature is missing in the StackFrame API. This proposes to add >> StackFrame::getMethodDescriptor method to return the method descriptor >> in a stack frame. >> >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.00/index.html >> >> There are a couple options how to present the method signature in the >> API level: >> 1. Class[] getParameterTypes() and Class getReturnTypes() similiar >> to what java.lang.reflect.Method has. >> 2. java.lang.invoke.MethodType >> 3. a String representation (i) comma-separated list of the method's >> formal parameter types (ii) bytecode method descriptor as specified in JVMS >> >> Returning Class instance should require to add a new StackWalker >> option to access to the parameter types and return type for option #1 >> and #2. StackFrame::getDeclaringClass requires the stack walker to have >> the RETAIN_CLASS_REFERENCE capability. >> >> Option #2 returning MethodType is handy while java.lang would reference >> a type in java.lang.invoke. >> >> Option #3 requires the caller to parse the return string and call >> Class.forName to get the Class instance. OTOH >> MethodType::fromMethodDescriptorString method that returns MethodType >> from a bytecode method descriptor string. >> >> Method signature is for information for typical cases. Getting Class >> for the parameter types and return type would be a niche case. I think >> returning the method descriptor string is a good option - keep the API >> simple and can use MethodType::fromMethodDescriptorString to get back >> the types if needed. >> >> thanks >> Mandy From shade at redhat.com Fri Sep 1 06:14:29 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 1 Sep 2017 08:14:29 +0200 Subject: RFR: 8187033: [PPC] Imporve performance of ObjectStreamClass.getClassDataLayout() In-Reply-To: References:

Message-ID: <9cee904a-aa3a-433f-0763-618bf385088d@redhat.com> On 08/31/2017 09:39 PM, Hans Boehm wrote: >> I guess you can make VarHandle.fullFence() between getClassDataLayout0() and storing it to the >> non-volatile field... > > ... with the usual warning about this sort of thing: > > According to the JMM, it's not guaranteed to work, because the reader-side guarantees are not > there. In practice, you're relying on dependency-based ordering, which the compiler is currently > unlikely to break in this case. But future implementations may. Right. > I presume the real concern here is the cost on the reader side? Presumably that could be reduced > with a VarHandle getAcquire(). I believe that eliminates the heavy-weight sync, and just keeps > an lwsync. Imperfect, but better. Oh right! This is exactly why acq/rel exist. Since OSC is a heavily used class, the Unsafe counterparts might be better: private static final Unsafe U = ...; private static final long CDS_OFFSET = U.objectFieldOffset(...); private volatile ClassDataSlot[] dataLayout; // volatile for safety of naked reads ClassDataSlot[] getClassDataLayout() throws InvalidClassException { ClassDataSlot[] slots = U.getObjectAcquire(this, CDS_OFFSET); if (slots == null) { slots = getClassDataLayout0(); U.putObjectRelease(this, CDS_OFFSET, slots); } return slots; } Ogata, please try if that indeed helps on POWER? Thanks, -Aleksey From OGATAK at jp.ibm.com Fri Sep 1 06:53:23 2017 From: OGATAK at jp.ibm.com (Kazunori Ogata) Date: Fri, 1 Sep 2017 15:53:23 +0900 Subject: RFR: 8187033: [PPC] Imporve performance of ObjectStreamClass.getClassDataLayout() In-Reply-To: <9cee904a-aa3a-433f-0763-618bf385088d@redhat.com> References:

<9cee904a-aa3a-433f-0763-618bf385088d@redhat.com> Message-ID: Hi Aleksey and Hans, Thank you for your comments. I'll try to see how much Unsafe approach improves performance. I'm now thinking the approach to use final that Aleksey mentioned in the first reply is a good one. I checked the JIT generated code. It puts MEMBAR-store-store and MEMBAR-release before storing ClassDataSlot array to the final field "slots", and there is no MEMBAR when reading dataLayout or dataLayout.slots. Since dataLayout is heavily read than written, I think it is preferable if we can put all overhead into the writing side. But I'll see how performance changes with lwsync on reading it. private DataLayout dataLayout; /** * Class to ensure elements of ClassDataSlot[] are visible to other * threads. The "final" qualifier of the variable slots is necessary. */ private static class DataLayout { final ClassDataSlot[] slots; DataLayout(ClassDataSlot[] s) { slots = s; } } ClassDataSlot[] getClassDataLayout() throws InvalidClassException { // REMIND: synchronize instead of relying on volatile? if (dataLayout == null) { ClassDataSlot[] slots = getClassDataLayout0(); dataLayout = new DataLayout(slots); } return dataLayout.slots; } Regards, Ogata From: Aleksey Shipilev To: Hans Boehm Cc: Kazunori Ogata , core-libs-dev Date: 2017/09/01 15:14 Subject: Re: RFR: 8187033: [PPC] Imporve performance of ObjectStreamClass.getClassDataLayout() On 08/31/2017 09:39 PM, Hans Boehm wrote: >> I guess you can make VarHandle.fullFence() between getClassDataLayout0() and storing it to the >> non-volatile field... > > ... with the usual warning about this sort of thing: > > According to the JMM, it's not guaranteed to work, because the reader-side guarantees are not > there. In practice, you're relying on dependency-based ordering, which the compiler is currently > unlikely to break in this case. But future implementations may. Right. > I presume the real concern here is the cost on the reader side? Presumably that could be reduced > with a VarHandle getAcquire(). I believe that eliminates the heavy-weight sync, and just keeps > an lwsync. Imperfect, but better. Oh right! This is exactly why acq/rel exist. Since OSC is a heavily used class, the Unsafe counterparts might be better: private static final Unsafe U = ...; private static final long CDS_OFFSET = U.objectFieldOffset(...); private volatile ClassDataSlot[] dataLayout; // volatile for safety of naked reads ClassDataSlot[] getClassDataLayout() throws InvalidClassException { ClassDataSlot[] slots = U.getObjectAcquire(this, CDS_OFFSET); if (slots == null) { slots = getClassDataLayout0(); U.putObjectRelease(this, CDS_OFFSET, slots); } return slots; } Ogata, please try if that indeed helps on POWER? Thanks, -Aleksey [attachment "signature.asc" deleted by Kazunori Ogata/Japan/IBM] From forax at univ-mlv.fr Fri Sep 1 09:57:30 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 1 Sep 2017 11:57:30 +0200 (CEST) Subject: Review Request JDK-8186050: StackFrame should provide the method signature In-Reply-To: <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> References: <5ad2e4a3-3fd3-0df4-c6c4-0b9ca8068a4e@oracle.com> <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> Message-ID: <1028258365.2136415.1504259850363.JavaMail.zimbra@u-pem.fr> Hi Mandy, good for me. in Basic.java, in methodTypes(), instead of using a new HashMap, you can use Map.of(). cheers, R?mi ----- Mail original ----- > De: "mandy chung" > ?: "Remi Forax" > Cc: "core-libs-dev" > Envoy?: Vendredi 1 Septembre 2017 07:39:54 > Objet: Re: Review Request JDK-8186050: StackFrame should provide the method signature > Updated webrev: > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.01/index.html > > This introduces two new methods, StackFrame::getMethodType and > StackFrame::getDescriptor. > > Mandy > > On 8/30/17 12:25 AM, Remi Forax wrote: >> Hi Mandy, >> thanks for taking care of this. >> >> In my opinion, we should provide both getMethodType() and getDescriptor(), >> getDescriptor() is handy for logging (finding the right overload when line >> numbers are not present) and getMethodType() is the one you whant if you want >> to inspect the runtime view of the stack frames (and by example interact with >> java.lang.invoke). For me, it's the same reason that give us >> getDeclaringClass() and getClassName() in the current API. >> >> So getDescriptor() can be called with no restriction but getMethodType() >> requires RETAIN_CLASS_REFERENCE. >> >> regards, >> R?mi >> >> ----- Mail original ----- >>> De: "mandy chung" >>> ?: "core-libs-dev" >>> Envoy?: Mardi 29 Ao?t 2017 00:57:28 >>> Objet: Review Request JDK-8186050: StackFrame should provide the method >>> signature >>> Method signature is missing in the StackFrame API. This proposes to add >>> StackFrame::getMethodDescriptor method to return the method descriptor >>> in a stack frame. >>> >>> Webrev at: >>> http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.00/index.html >>> >>> There are a couple options how to present the method signature in the >>> API level: >>> 1. Class[] getParameterTypes() and Class getReturnTypes() similiar >>> to what java.lang.reflect.Method has. >>> 2. java.lang.invoke.MethodType >>> 3. a String representation (i) comma-separated list of the method's >>> formal parameter types (ii) bytecode method descriptor as specified in JVMS >>> >>> Returning Class instance should require to add a new StackWalker >>> option to access to the parameter types and return type for option #1 >>> and #2. StackFrame::getDeclaringClass requires the stack walker to have >>> the RETAIN_CLASS_REFERENCE capability. >>> >>> Option #2 returning MethodType is handy while java.lang would reference >>> a type in java.lang.invoke. >>> >>> Option #3 requires the caller to parse the return string and call >>> Class.forName to get the Class instance. OTOH >>> MethodType::fromMethodDescriptorString method that returns MethodType >>> from a bytecode method descriptor string. >>> >>> Method signature is for information for typical cases. Getting Class >>> for the parameter types and return type would be a niche case. I think >>> returning the method descriptor string is a good option - keep the API >>> simple and can use MethodType::fromMethodDescriptorString to get back >>> the types if needed. >>> >>> thanks > >> Mandy From Alan.Bateman at oracle.com Fri Sep 1 13:44:55 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 1 Sep 2017 14:44:55 +0100 Subject: Review Request JDK-8186050: StackFrame should provide the method signature In-Reply-To: <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> References: <5ad2e4a3-3fd3-0df4-c6c4-0b9ca8068a4e@oracle.com> <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> Message-ID: On 01/09/2017 06:39, mandy chung wrote: > Updated webrev: > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.01/index.html > > > This introduces two new methods, StackFrame::getMethodType and > StackFrame::getDescriptor. The two new methods look good. -Alan From Alan.Bateman at oracle.com Fri Sep 1 14:36:44 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 1 Sep 2017 15:36:44 +0100 Subject: RFR JDK-8186751: Add ISO-8859-16 Charset support In-Reply-To: <59A7711F.9060801@oracle.com> References: <59A7711F.9060801@oracle.com> Message-ID: <13fc862e-60d8-d6e7-df48-28f62f2e451e@oracle.com> On 31/08/2017 03:14, Xueming Shen wrote: > Hi, > > Please help review the changeset to add the 8859_1 charset support. > > issue: https://bugs.openjdk.java.net/browse/JDK-8186751 > webrev: http://cr.openjdk.java.net/~sherman/8186751/webrev Looks good. -Alan From mandy.chung at oracle.com Fri Sep 1 16:40:24 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Sep 2017 09:40:24 -0700 Subject: Review Request JDK-8186050: StackFrame should provide the method signature In-Reply-To: <1028258365.2136415.1504259850363.JavaMail.zimbra@u-pem.fr> References: <5ad2e4a3-3fd3-0df4-c6c4-0b9ca8068a4e@oracle.com> <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> <1028258365.2136415.1504259850363.JavaMail.zimbra@u-pem.fr> Message-ID: <3828f1ef-9591-0edf-950f-8e884eca3b84@oracle.com> On 9/1/17 2:57 AM, forax at univ-mlv.fr wrote: > Hi Mandy, > good for me. > > in Basic.java, in methodTypes(), instead of using a new HashMap, you can use Map.of(). Yes will fix it before push. Mandy > cheers, > R?mi > > ----- Mail original ----- >> De: "mandy chung" >> ?: "Remi Forax" >> Cc: "core-libs-dev" >> Envoy?: Vendredi 1 Septembre 2017 07:39:54 >> Objet: Re: Review Request JDK-8186050: StackFrame should provide the method signature >> Updated webrev: >> http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.01/index.html >> >> This introduces two new methods, StackFrame::getMethodType and >> StackFrame::getDescriptor. >> >> Mandy >> >> On 8/30/17 12:25 AM, Remi Forax wrote: >>> Hi Mandy, >>> thanks for taking care of this. >>> >>> In my opinion, we should provide both getMethodType() and getDescriptor(), >>> getDescriptor() is handy for logging (finding the right overload when line >>> numbers are not present) and getMethodType() is the one you whant if you want >>> to inspect the runtime view of the stack frames (and by example interact with >>> java.lang.invoke). For me, it's the same reason that give us >>> getDeclaringClass() and getClassName() in the current API. >>> >>> So getDescriptor() can be called with no restriction but getMethodType() >>> requires RETAIN_CLASS_REFERENCE. >>> >>> regards, >>> R?mi >>> >>> ----- Mail original ----- >>>> De: "mandy chung" >>>> ?: "core-libs-dev" >>>> Envoy?: Mardi 29 Ao?t 2017 00:57:28 >>>> Objet: Review Request JDK-8186050: StackFrame should provide the method >>>> signature >>>> Method signature is missing in the StackFrame API. This proposes to add >>>> StackFrame::getMethodDescriptor method to return the method descriptor >>>> in a stack frame. >>>> >>>> Webrev at: >>>> http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.00/index.html >>>> >>>> There are a couple options how to present the method signature in the >>>> API level: >>>> 1. Class[] getParameterTypes() and Class getReturnTypes() similiar >>>> to what java.lang.reflect.Method has. >>>> 2. java.lang.invoke.MethodType >>>> 3. a String representation (i) comma-separated list of the method's >>>> formal parameter types (ii) bytecode method descriptor as specified in JVMS >>>> >>>> Returning Class instance should require to add a new StackWalker >>>> option to access to the parameter types and return type for option #1 >>>> and #2. StackFrame::getDeclaringClass requires the stack walker to have >>>> the RETAIN_CLASS_REFERENCE capability. >>>> >>>> Option #2 returning MethodType is handy while java.lang would reference >>>> a type in java.lang.invoke. >>>> >>>> Option #3 requires the caller to parse the return string and call >>>> Class.forName to get the Class instance. OTOH >>>> MethodType::fromMethodDescriptorString method that returns MethodType >>>> from a bytecode method descriptor string. >>>> >>>> Method signature is for information for typical cases. Getting Class >>>> for the parameter types and return type would be a niche case. I think >>>> returning the method descriptor string is a good option - keep the API >>>> simple and can use MethodType::fromMethodDescriptorString to get back >>>> the types if needed. >>>> >>>> thanks >>>> Mandy From mandy.chung at oracle.com Fri Sep 1 16:46:36 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Sep 2017 09:46:36 -0700 Subject: Review Request JDK-8186050: StackFrame should provide the method signature In-Reply-To: <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> References: <5ad2e4a3-3fd3-0df4-c6c4-0b9ca8068a4e@oracle.com> <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> Message-ID: Please also review CSR: https://bugs.openjdk.java.net/browse/JDK-8186872 Mandy On 8/31/17 10:39 PM, mandy chung wrote: > Updated webrev: > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.01/index.html > > > This introduces two new methods, StackFrame::getMethodType and > StackFrame::getDescriptor. > > Mandy From brent.christian at oracle.com Fri Sep 1 17:21:15 2017 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 1 Sep 2017 10:21:15 -0700 Subject: Review Request JDK-8186050: StackFrame should provide the method signature In-Reply-To: <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> References: <5ad2e4a3-3fd3-0df4-c6c4-0b9ca8068a4e@oracle.com> <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> Message-ID: <92bf1dfa-c229-78cd-e99b-153d4b9d505f@oracle.com> Looks good. I have only one (minor) comment. In the new Javadoc for getMethodType(): StackWalker.java: 133 * Returns {@link MethodType} representing the parameter types and The docs for similar methods all begin with, "Returns _the_ ..." -Brent On 08/31/2017 10:39 PM, mandy chung wrote: > Updated webrev: > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.01/index.html > > > This introduces two new methods, StackFrame::getMethodType and > StackFrame::getDescriptor. > > Mandy > > On 8/30/17 12:25 AM, Remi Forax wrote: >> Hi Mandy, >> thanks for taking care of this. >> >> In my opinion, we should provide both getMethodType() and >> getDescriptor(), >> getDescriptor() is handy for logging (finding the right overload when >> line numbers are not present) and getMethodType() is the one you whant >> if you want to inspect the runtime view of the stack frames (and by >> example interact with java.lang.invoke). For me, it's the same reason >> that give us getDeclaringClass() and getClassName() in the current API. >> >> So getDescriptor() can be called with no restriction but >> getMethodType() requires RETAIN_CLASS_REFERENCE. >> >> regards, >> R?mi >> >> ----- Mail original ----- >>> De: "mandy chung" >>> ?: "core-libs-dev" >>> Envoy?: Mardi 29 Ao?t 2017 00:57:28 >>> Objet: Review Request JDK-8186050: StackFrame should provide the >>> method signature >>> Method signature is missing in the StackFrame API. This proposes to add >>> StackFrame::getMethodDescriptor method to return the method descriptor >>> in a stack frame. >>> >>> Webrev at: >>> http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.00/index.html >>> >>> >>> There are a couple options how to present the method signature in the >>> API level: >>> 1. Class[] getParameterTypes() and Class getReturnTypes() similiar >>> to what java.lang.reflect.Method has. >>> 2. java.lang.invoke.MethodType >>> 3. a String representation (i) comma-separated list of the method's >>> formal parameter types (ii) bytecode method descriptor as specified >>> in JVMS >>> >>> Returning Class instance should require to add a new StackWalker >>> option to access to the parameter types and return type for option #1 >>> and #2. StackFrame::getDeclaringClass requires the stack walker to have >>> the RETAIN_CLASS_REFERENCE capability. >>> >>> Option #2 returning MethodType is handy while java.lang would reference >>> a type in java.lang.invoke. >>> >>> Option #3 requires the caller to parse the return string and call >>> Class.forName to get the Class instance. OTOH >>> MethodType::fromMethodDescriptorString method that returns MethodType >>> from a bytecode method descriptor string. >>> >>> Method signature is for information for typical cases. Getting Class >>> for the parameter types and return type would be a niche case. I think >>> returning the method descriptor string is a good option - keep the API >>> simple and can use MethodType::fromMethodDescriptorString to get back >>> the types if needed. >>> >>> thanks >>> Mandy > From mandy.chung at oracle.com Fri Sep 1 17:42:58 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Sep 2017 10:42:58 -0700 Subject: Review Request JDK-8186050: StackFrame should provide the method signature In-Reply-To: <92bf1dfa-c229-78cd-e99b-153d4b9d505f@oracle.com> References: <5ad2e4a3-3fd3-0df4-c6c4-0b9ca8068a4e@oracle.com> <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> <92bf1dfa-c229-78cd-e99b-153d4b9d505f@oracle.com> Message-ID: <0be2b47d-a045-be77-9a96-64dbd00e2109@oracle.com> On 9/1/17 10:21 AM, Brent Christian wrote: > Looks good.? I have only one (minor) comment. > > In the new Javadoc for getMethodType(): > > StackWalker.java: > > ?133????????? * Returns {@link MethodType} representing the parameter > types and > > > The docs for similar methods all begin with, "Returns _the_ ..." Fixed.? Thanks for the review. Mandy From naoto.sato at oracle.com Fri Sep 1 17:48:32 2017 From: naoto.sato at oracle.com (Naoto Sato) Date: Fri, 1 Sep 2017 10:48:32 -0700 Subject: [10] RFR 8180469: Wrong short form text for supplemental Japanese era In-Reply-To: References: <04f6fa07-019f-27f3-5971-b2b27d40ca0c@oracle.com> Message-ID: <5a6c018f-1292-a49b-a015-bd3154ff27cc@oracle.com> Hi Mitsuru-san, Yes, I remember we discussed on this issue before. The reason that LONG and SHORT names for Japanese era are the same is that CLDR's era names are not very consistent on length. They have "eraNames", "eraAbbr", and "eraNarrow" variations. We simply assign LONG to eraNames and SHORT to eraAbbr in SimpleDateFormat. Possibly the right solution is to provide "narrow" option in SimpleDateFormat, but it would be breaking the compatibility (text length of those pattern characters just have two options, one is 4 or greater (=LONG), and the other is less than 4 (=SHORT)). So, considering these, I have a couple of options. One is to use the newer java.time.format APIs which can correctly handle this, or use the JDK8 locale data by specifying -Djava.locale.providers=COMPAT at runtime. HTH, Naoto On 8/31/17 7:34 PM, Mitsuru Matsushima wrote: > Hi Naoto-san, > > The fix looks good, though I'm not a reviewer... > > By the way, I may have forgotten to inform you that there exist an issue at the short form of SimpleDateFormat has an issue. > > The SimpleDateFormat class is only capable to treat two form, Short and Long. > At JDK9, the CLDR Provider become to default, the provider returns the same value for the Short form and the Long form. > So, the behavior of SimpleDateFormat is incompatible to previous versions. > (See the Comparison table, I described before.) > > --- > Mitsuru > >> -----Original Message----- >> From: i18n-dev [mailto:i18n-dev-bounces at openjdk.java.net] On Behalf Of Naoto Sato >> Sent: Thursday, August 31, 2017 7:56 AM >> To: core-libs-dev ; i18n-dev >> Subject: [10] RFR 8180469: Wrong short form text for supplemental Japanese era >> >> Hi, >> >> Please review the fix to the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8180469 >> >> The proposed changeset is located at: >> >> http://cr.openjdk.java.net/~naoto/8180469/webrev.00/ >> >> The problem was caused by the difference of the Era display name for "SHORT" style between java.time and java.util.Calendar. >> >> Naoto From Ulf.Zibis at CoSoCo.de Fri Sep 1 18:22:39 2017 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Fri, 1 Sep 2017 20:22:39 +0200 Subject: RFR JDK-8186751: Add ISO-8859-16 Charset support In-Reply-To: <59A87FC0.4090505@oracle.com> References: <59A7711F.9060801@oracle.com> <59A87FC0.4090505@oracle.com> Message-ID: Hi again, Am 31.08.2017 um 23:29 schrieb Xueming Shen: > The 8859-16 aliases are the copy/paste from > https://www.iana.org/assignments/character-sets/character-sets.xhtml > > From the same page we have the followings for 8859-15 > ISO_8859-15 > Latin-9 > csISO885915 > > It appears only the first one is listed in our alias list (explicitly > commented). > Don't have the memory for the history (8859-15 was added a long time ago), > not sure whether the other two were missed at the very beginning or were > added later in the iana. Since we are here, I added them in. Fine! > http://cr.openjdk.java.net/~sherman/8186751/webrev > (the old one is copied to webrev.00) - I think, the comment should now be in plural: # IANA alias*es *- Also I think, it would make sense to at least add "ISO8859-16" and "ISO8859_16" as # Other aliases - "ISO-8859-15" must not be listed as alias, as it already is the canonical name. - Not sure, where "LATIN0" and "csISOlatin0" come from. -Ulf From fguillaume at nuxeo.com Fri Sep 1 19:16:13 2017 From: fguillaume at nuxeo.com (Florent Guillaume) Date: Fri, 1 Sep 2017 21:16:13 +0200 Subject: RFR JDK-8186751: Add ISO-8859-16 Charset support In-Reply-To: References: <59A7711F.9060801@oracle.com> <59A87FC0.4090505@oracle.com> Message-ID: Hi, >From https://www.cs.tut.fi/~jkorpela/latin9.html: > Originally (http://www.indigo.ie/egt/standards/iso8859/latin00.html) the project > which lead to the creation of ISO Latin 9 used the working name "Latin Alphabet > Number Zero" for it. Therefore it has often been referred to as "Latin 0". I don't think it should be included in the aliases. Florent On Fri, Sep 1, 2017 at 8:22 PM, Ulf Zibis wrote: > Hi again, > > Am 31.08.2017 um 23:29 schrieb Xueming Shen: > > The 8859-16 aliases are the copy/paste from > https://www.iana.org/assignments/character-sets/character-sets.xhtml > > From the same page we have the followings for 8859-15 > ISO_8859-15 > Latin-9 > csISO885915 > > It appears only the first one is listed in our alias list (explicitly > commented). > Don't have the memory for the history (8859-15 was added a long time ago), > not sure whether the other two were missed at the very beginning or were > added later in the iana. Since we are here, I added them in. > > > Fine! > > http://cr.openjdk.java.net/~sherman/8186751/webrev > (the old one is copied to webrev.00) > > > - I think, the comment should now be in plural: # IANA alias > *es *- Also I think, it would make sense to at least add "ISO8859-16" and > "ISO8859_16" as # Other aliases > - "ISO-8859-15" must not be listed as alias, as it already is the > canonical name. > - Not sure, where "LATIN0" and "csISOlatin0" come from. > > -Ulf > > -- [image: Nuxeo] Florent Guillaume Head of R&D Twitter: @efge From jonathan.gibbons at oracle.com Fri Sep 1 20:34:30 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 01 Sep 2017 13:34:30 -0700 Subject: RFR: JDK-8186947: Fix accessibility and other issues in the java.xml.ws module Message-ID: <59A9C456.40809@oracle.com> Please review this small change to fix a broken link in the API docs for the java.xml.ws module. Although I've also posted a full webrev and API, the change is more easily seen here: --- old/src/java.xml.ws/share/classes/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.java 2017-09-01 12:29:36.684932965 -0700 +++ new/src/java.xml.ws/share/classes/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.java 2017-09-01 12:29:36.596929123 -0700 @@ -145,7 +145,7 @@ * Sets the {@code endpointName} as * {@code wsam:ServiceName/@EndpointName} in the * {@code wsa:Metadata} element. This method can only be called - * after the {@link #serviceName} method has been called. + * after the {@link #serviceName(QName)} method has been called. *

* See * 2.1 Referencing WSDL Metadata from an EPR for more details. JBS: https://bugs.openjdk.java.net/browse/JDK-8186947 Webrev: http://cr.openjdk.java.net/~jjg/8186947/webrev/index.html API: http://cr.openjdk.java.net/~jjg/8186947/api/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.html -- Jon From jonathan.gibbons at oracle.com Fri Sep 1 20:34:33 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 01 Sep 2017 13:34:33 -0700 Subject: RFR: JDK-8186946: Fix accessibility and other issues in the java.xml.bind module Message-ID: <59A9C459.5080208@oracle.com> Please review these changes to address accessibility and other minor issues in the API docs for the java.xml.bind module. JBS: https://bugs.openjdk.java.net/browse/JDK-8186946 Webrev: http://cr.openjdk.java.net/~jjg/8186946/webrev API: http://cr.openjdk.java.net/~jjg/8186946/api/java.xml.bind-summary.html -- Jon Notes on the changes follow: src/java.xml.bind/share/classes/javax/xml/bind/JAXBPermission.java Convert file to use the standard "striped" pattern for tables src/java.xml.bind/share/classes/javax/xml/bind/Marshaller.java Fix broken links that arise when relative links are inherited into the docs for subtypes src/java.xml.bind/share/classes/javax/xml/bind/Unmarshaller.java Convert file to use the standard "striped" pattern for tables Fix broken links that arise when relative links are inherited into the docs for subtypes src/java.xml.bind/share/classes/javax/xml/bind/annotation/XmlNsForm.java Convert file to use the standard "striped" pattern for tables src/java.xml.bind/share/classes/javax/xml/bind/annotation/XmlType.java Convert file to use the standard "striped" pattern for tables src/java.xml.bind/share/classes/javax/xml/bind/annotation/adapters/package-info.java Remove redundant

tags src/java.xml.bind/share/classes/javax/xml/bind/annotation/package.html Convert file to use the standard "striped" pattern for tables Replace nested layout tables with simple lists src/java.xml.bind/share/classes/javax/xml/bind/attachment/AttachmentUnmarshaller.java Convert file to use the standard "striped" pattern for tables src/java.xml.bind/share/classes/javax/xml/bind/helpers/package-info.java Remove redundant

tags src/java.xml.bind/share/classes/javax/xml/bind/util/package-info.java Remove redundant

tags The following files are removed, since they are shadowed by the corresponding package-info.java files: src/java.xml.bind/share/classes/javax/xml/bind/annotation/adapters/package.html src/java.xml.bind/share/classes/javax/xml/bind/helpers/package.html src/java.xml.bind/share/classes/javax/xml/bind/util/package.html -- Jon From joe.darcy at oracle.com Fri Sep 1 20:35:30 2017 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 1 Sep 2017 13:35:30 -0700 Subject: RFR: JDK-8186947: Fix accessibility and other issues in the java.xml.ws module In-Reply-To: <59A9C456.40809@oracle.com> References: <59A9C456.40809@oracle.com> Message-ID: <90dbbfa0-5212-56af-9cda-7fefe9b7a5f4@oracle.com> +1 -Joe On 9/1/2017 1:34 PM, Jonathan Gibbons wrote: > Please review this small change to fix a broken link in the API docs > for the java.xml.ws module. > > Although I've also posted a full webrev and API, the change is more > easily seen here: > > --- > old/src/java.xml.ws/share/classes/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.java > 2017-09-01 12:29:36.684932965 -0700 > +++ > new/src/java.xml.ws/share/classes/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.java > 2017-09-01 12:29:36.596929123 -0700 > @@ -145,7 +145,7 @@ > * Sets the {@code endpointName} as > * {@code wsam:ServiceName/@EndpointName} in the > * {@code wsa:Metadata} element. This method can only be called > - * after the {@link #serviceName} method has been called. > + * after the {@link #serviceName(QName)} method has been called. > *

> * See href="http://www.w3.org/TR/2007/REC-ws-addr-metadata-20070904/#refmetadatfromepr"> > * 2.1 Referencing WSDL Metadata from an EPR for more details. > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8186947 > Webrev: http://cr.openjdk.java.net/~jjg/8186947/webrev/index.html > API: > http://cr.openjdk.java.net/~jjg/8186947/api/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.html > > -- Jon From mandy.chung at oracle.com Fri Sep 1 20:36:05 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Sep 2017 13:36:05 -0700 Subject: RFR: JDK-8186947: Fix accessibility and other issues in the java.xml.ws module In-Reply-To: <59A9C456.40809@oracle.com> References: <59A9C456.40809@oracle.com> Message-ID: +1 Mandy On 9/1/17 1:34 PM, Jonathan Gibbons wrote: > Please review this small change to fix a broken link in the API docs > for the java.xml.ws module. > > Although I've also posted a full webrev and API, the change is more > easily seen here: > > --- > old/src/java.xml.ws/share/classes/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.java > 2017-09-01 12:29:36.684932965 -0700 > +++ > new/src/java.xml.ws/share/classes/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.java > 2017-09-01 12:29:36.596929123 -0700 > @@ -145,7 +145,7 @@ > ????? * Sets the {@code endpointName} as > ????? * {@code wsam:ServiceName/@EndpointName} in the > ????? * {@code wsa:Metadata} element. This method can only be called > -???? * after the {@link #serviceName} method has been called. > +???? * after the {@link #serviceName(QName)} method has been called. > ????? *

> ????? * See href="http://www.w3.org/TR/2007/REC-ws-addr-metadata-20070904/#refmetadatfromepr"> > ????? * 2.1 Referencing WSDL Metadata from an EPR for more details. > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8186947 > Webrev: http://cr.openjdk.java.net/~jjg/8186947/webrev/index.html > API: > http://cr.openjdk.java.net/~jjg/8186947/api/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.html > > -- Jon From lance.andersen at oracle.com Fri Sep 1 20:37:18 2017 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 1 Sep 2017 16:37:18 -0400 Subject: RFR: JDK-8186947: Fix accessibility and other issues in the java.xml.ws module In-Reply-To: <59A9C456.40809@oracle.com> References: <59A9C456.40809@oracle.com> Message-ID: <0324B78D-0580-4982-933B-8803DEE624F4@oracle.com> looks fine Jon > On Sep 1, 2017, at 4:34 PM, Jonathan Gibbons wrote: > > Please review this small change to fix a broken link in the API docs for the java.xml.ws module. > > Although I've also posted a full webrev and API, the change is more easily seen here: > > --- old/src/java.xml.ws/share/classes/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.java 2017-09-01 12:29:36.684932965 -0700 > +++ new/src/java.xml.ws/share/classes/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.java 2017-09-01 12:29:36.596929123 -0700 > @@ -145,7 +145,7 @@ > * Sets the {@code endpointName} as > * {@code wsam:ServiceName/@EndpointName} in the > * {@code wsa:Metadata} element. This method can only be called > - * after the {@link #serviceName} method has been called. > + * after the {@link #serviceName(QName)} method has been called. > *

> * See > * 2.1 Referencing WSDL Metadata from an EPR for more details. > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8186947 > Webrev: http://cr.openjdk.java.net/~jjg/8186947/webrev/index.html > API: http://cr.openjdk.java.net/~jjg/8186947/api/javax/xml/ws/wsaddressing/W3CEndpointReferenceBuilder.html > > -- Jon Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From mandy.chung at oracle.com Fri Sep 1 20:50:26 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Sep 2017 13:50:26 -0700 Subject: RFR: JDK-8186946: Fix accessibility and other issues in the java.xml.bind module In-Reply-To: <59A9C459.5080208@oracle.com> References: <59A9C459.5080208@oracle.com> Message-ID: <74bd6026-22c9-7f42-da16-2b4c31109a9a@oracle.com> Typo in the href link: e.g.See should be {@docRoot}/javax/xml/bind/XXX.html Mandy On 9/1/17 1:34 PM, Jonathan Gibbons wrote: > Please review these changes to address accessibility and other minor > issues in > the API docs for the java.xml.bind module. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8186946 > Webrev: http://cr.openjdk.java.net/~jjg/8186946/webrev > API: > http://cr.openjdk.java.net/~jjg/8186946/api/java.xml.bind-summary.html > > -- Jon > > Notes on the changes follow: > > src/java.xml.bind/share/classes/javax/xml/bind/JAXBPermission.java > ??? Convert file to use the standard "striped" pattern for tables > > src/java.xml.bind/share/classes/javax/xml/bind/Marshaller.java > ??? Fix broken links that arise when relative links are inherited into > the docs for subtypes > > src/java.xml.bind/share/classes/javax/xml/bind/Unmarshaller.java > ??? Convert file to use the standard "striped" pattern for tables > ??? Fix broken links that arise when relative links are inherited into > the docs for subtypes > > src/java.xml.bind/share/classes/javax/xml/bind/annotation/XmlNsForm.java > ??? Convert file to use the standard "striped" pattern for tables > > src/java.xml.bind/share/classes/javax/xml/bind/annotation/XmlType.java > ??? Convert file to use the standard "striped" pattern for tables > > src/java.xml.bind/share/classes/javax/xml/bind/annotation/adapters/package-info.java > > ??? Remove redundant

tags > > src/java.xml.bind/share/classes/javax/xml/bind/annotation/package.html > ??? Convert file to use the standard "striped" pattern for tables > ??? Replace nested layout tables with simple lists > > src/java.xml.bind/share/classes/javax/xml/bind/attachment/AttachmentUnmarshaller.java > > ??? Convert file to use the standard "striped" pattern for tables > > src/java.xml.bind/share/classes/javax/xml/bind/helpers/package-info.java > ??? Remove redundant

tags > > src/java.xml.bind/share/classes/javax/xml/bind/util/package-info.java > ??? Remove redundant

tags >> >> src/java.xml.bind/share/classes/javax/xml/bind/annotation/package.html >> Convert file to use the standard "striped" pattern for tables >> Replace nested layout tables with simple lists >> >> src/java.xml.bind/share/classes/javax/xml/bind/attachment/AttachmentUnmarshaller.java >> Convert file to use the standard "striped" pattern for tables >> >> src/java.xml.bind/share/classes/javax/xml/bind/helpers/package-info.java >> Remove redundant

tags >> >> src/java.xml.bind/share/classes/javax/xml/bind/util/package-info.java >> Remove redundant

tags >> >> The following files are removed, since they are shadowed by the corresponding package-info.java >> files: >> >> src/java.xml.bind/share/classes/javax/xml/bind/annotation/adapters/package.html >> src/java.xml.bind/share/classes/javax/xml/bind/helpers/package.html >> src/java.xml.bind/share/classes/javax/xml/bind/util/package.html >> >> >> -- Jon > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From jonathan.gibbons at oracle.com Fri Sep 1 21:42:26 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 01 Sep 2017 14:42:26 -0700 Subject: Update: Re: RFR: JDK-8186946: Fix accessibility and other issues in the java.xml.bind module In-Reply-To: <59A9C459.5080208@oracle.com> References: <59A9C459.5080208@oracle.com> Message-ID: <59A9D442.4090303@oracle.com> Sigh. The bad news is that I accidentally committed and pushed the changes for this issue (JDK-8186946) before they had been approved, along with the changes for JDK-8186947, which had been approved. The good news is that I had already addressed and verified Mandy's comments regarding broken links. So the net result is that the source code in the repo is good, but the metadata for these two issues is a bit messed up. I'll update the JBS bugs to document this. If anyone should find any additional issues in this review, I will of course address them and file an additional bug if necessary. -- Jon On 09/01/2017 01:34 PM, Jonathan Gibbons wrote: > Please review these changes to address accessibility and other minor > issues in > the API docs for the java.xml.bind module. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8186946 > Webrev: http://cr.openjdk.java.net/~jjg/8186946/webrev > API: > http://cr.openjdk.java.net/~jjg/8186946/api/java.xml.bind-summary.html > > -- Jon > > Notes on the changes follow: > > src/java.xml.bind/share/classes/javax/xml/bind/JAXBPermission.java > Convert file to use the standard "striped" pattern for tables > > src/java.xml.bind/share/classes/javax/xml/bind/Marshaller.java > Fix broken links that arise when relative links are inherited into > the docs for subtypes > > src/java.xml.bind/share/classes/javax/xml/bind/Unmarshaller.java > Convert file to use the standard "striped" pattern for tables > Fix broken links that arise when relative links are inherited into > the docs for subtypes > > src/java.xml.bind/share/classes/javax/xml/bind/annotation/XmlNsForm.java > Convert file to use the standard "striped" pattern for tables > > src/java.xml.bind/share/classes/javax/xml/bind/annotation/XmlType.java > Convert file to use the standard "striped" pattern for tables > > src/java.xml.bind/share/classes/javax/xml/bind/annotation/adapters/package-info.java > > Remove redundant

tags > > src/java.xml.bind/share/classes/javax/xml/bind/annotation/package.html > Convert file to use the standard "striped" pattern for tables > Replace nested layout tables with simple lists > > src/java.xml.bind/share/classes/javax/xml/bind/attachment/AttachmentUnmarshaller.java > > Convert file to use the standard "striped" pattern for tables > > src/java.xml.bind/share/classes/javax/xml/bind/helpers/package-info.java > Remove redundant

tags > > src/java.xml.bind/share/classes/javax/xml/bind/util/package-info.java > Remove redundant

tags >> >> src/java.xml.bind/share/classes/javax/xml/bind/annotation/package.html >> ??? Convert file to use the standard "striped" pattern for tables >> ??? Replace nested layout tables with simple lists >> >> src/java.xml.bind/share/classes/javax/xml/bind/attachment/AttachmentUnmarshaller.java >> >> ??? Convert file to use the standard "striped" pattern for tables >> >> src/java.xml.bind/share/classes/javax/xml/bind/helpers/package-info.java >> ??? Remove redundant

tags >> >> src/java.xml.bind/share/classes/javax/xml/bind/util/package-info.java >> ??? Remove redundant

tags >> >> The following files are removed, since they are shadowed by the >> corresponding package-info.java >> files: >> >> src/java.xml.bind/share/classes/javax/xml/bind/annotation/adapters/package.html >> >> src/java.xml.bind/share/classes/javax/xml/bind/helpers/package.html >> src/java.xml.bind/share/classes/javax/xml/bind/util/package.html >> >> >> -- Jon > From xueming.shen at oracle.com Fri Sep 1 23:17:54 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 01 Sep 2017 16:17:54 -0700 Subject: RFR JDK-8186464: ZipFile cannot read some InfoZip ZIP64 zip files In-Reply-To: References: <599B8A4C.3050904@oracle.com> <599CAFDB.9020902@oracle.com> Message-ID: <59A9EAA2.3020102@oracle.com> On 8/22/17, 5:54 PM, Martin Buchholz wrote: > > > On Tue, Aug 22, 2017 at 3:27 PM, Xueming Shen > wrote: > > > How about to add an option to our zipfs to force the ZIP64 end > record when > enabled. Harmless if not specified. > > > I agree that adding more options for testability is good. Since our > users are likely to need such things as well, I'd also like to see > such features become public, analogous to zip's -fz- flag. > > OTOH, interoperability testing is very valuable, so despite the > troubles involved writing tests that involve both jdk and zipinfo > code, I think we should do it. > Martin, I have added a test case that tests "echo hello | zip infozip.zip -" via pb.startPipeline() in ReadZip.java, which should add some coverage for the "interoperability testing". Would you please help take a look? http://cr.openjdk.java.net/~sherman/8186464/webrev/ Thanks, Sherman From hboehm at google.com Sat Sep 2 02:53:42 2017 From: hboehm at google.com (Hans Boehm) Date: Fri, 1 Sep 2017 19:53:42 -0700 Subject: RFR: 8187033: [PPC] Imporve performance of ObjectStreamClass.getClassDataLayout() In-Reply-To: References: <9cee904a-aa3a-433f-0763-618bf385088d@redhat.com> Message-ID: The problem is that all solutions that put all overhead on the writing side are inherently not correct with respect to the Java memory model. They rely on the necessary reader-side order (dataLayout read not reordered after read of ClassDataSlots that use the dataLayout value) to be enforced (as you would expect) by the data dependency. The Java memory model only guarantees that (under the right conditions) for final fields. (This is controversial, but "fixing" it opens another can of worms.) Here you would be relying on it for dataLayout, a non-final field. I believe current implementations in fact enforce the necessary ordering. But since it (surprisingly) appears to not be completely free to have dependencies of this kind enforce memory visibility ordering (DEC Alpha famously didn't), I wouldn't bet large sums of money on this statement remaining true indefinitely on all relevant architectures. On Thu, Aug 31, 2017 at 11:53 PM, Kazunori Ogata wrote: > Hi Aleksey and Hans, > > Thank you for your comments. I'll try to see how much Unsafe approach > improves performance. > > I'm now thinking the approach to use final that Aleksey mentioned in the > first reply is a good one. I checked the JIT generated code. It puts > MEMBAR-store-store and MEMBAR-release before storing ClassDataSlot array > to the final field "slots", and there is no MEMBAR when reading dataLayout > or dataLayout.slots. Since dataLayout is heavily read than written, I > think it is preferable if we can put all overhead into the writing side. > But I'll see how performance changes with lwsync on reading it. > > > private DataLayout dataLayout; > > /** > * Class to ensure elements of ClassDataSlot[] are visible to other > * threads. The "final" qualifier of the variable slots is necessary. > */ > private static class DataLayout { > final ClassDataSlot[] slots; > DataLayout(ClassDataSlot[] s) { > slots = s; > } > } > > ClassDataSlot[] getClassDataLayout() throws InvalidClassException { > // REMIND: synchronize instead of relying on volatile? > if (dataLayout == null) { > ClassDataSlot[] slots = getClassDataLayout0(); > dataLayout = new DataLayout(slots); > } > return dataLayout.slots; > } > > > Regards, > Ogata > > > > From: Aleksey Shipilev > To: Hans Boehm > Cc: Kazunori Ogata , core-libs-dev > > Date: 2017/09/01 15:14 > Subject: Re: RFR: 8187033: [PPC] Imporve performance of > ObjectStreamClass.getClassDataLayout() > > > > On 08/31/2017 09:39 PM, Hans Boehm wrote: > >> I guess you can make VarHandle.fullFence() between > getClassDataLayout0() and storing it to the > >> non-volatile field... > > > > ... with the usual warning about this sort of thing: > > > > According to the JMM, it's not guaranteed to work, because the > reader-side guarantees are not > > there. In practice, you're relying on dependency-based ordering, which > the compiler is currently > > unlikely to break in this case. But future implementations may. > > Right. > > > I presume the real concern here is the cost on the reader side? > Presumably that could be reduced > > with a VarHandle getAcquire(). I believe that eliminates the > heavy-weight sync, and just keeps > > an lwsync. Imperfect, but better. > Oh right! This is exactly why acq/rel exist. Since OSC is a heavily used > class, the Unsafe > counterparts might be better: > > private static final Unsafe U = ...; > private static final long CDS_OFFSET = U.objectFieldOffset(...); > > private volatile ClassDataSlot[] dataLayout; // volatile for safety of > naked reads > > ClassDataSlot[] getClassDataLayout() throws InvalidClassException { > ClassDataSlot[] slots = U.getObjectAcquire(this, CDS_OFFSET); > if (slots == null) { > slots = getClassDataLayout0(); > U.putObjectRelease(this, CDS_OFFSET, slots); > } > return slots; > } > > Ogata, please try if that indeed helps on POWER? > > Thanks, > -Aleksey > > [attachment "signature.asc" deleted by Kazunori Ogata/Japan/IBM] > > > From aph at redhat.com Sat Sep 2 08:26:42 2017 From: aph at redhat.com (Andrew Haley) Date: Sat, 2 Sep 2017 09:26:42 +0100 Subject: RFR: 8187033: [PPC] Imporve performance of ObjectStreamClass.getClassDataLayout() In-Reply-To: References: <9cee904a-aa3a-433f-0763-618bf385088d@redhat.com> Message-ID: On 02/09/17 03:53, Hans Boehm wrote: > The problem is that all solutions that put all overhead on the writing side > are inherently not correct with respect to the Java memory model. They rely > on the necessary reader-side order (dataLayout read not reordered after > read of ClassDataSlots that use the dataLayout value) to be enforced (as > you would expect) by the data dependency. The Java memory model only > guarantees that (under the right conditions) for final fields. (This is > controversial, but "fixing" it opens another can of worms.) Here you would > be relying on it for dataLayout, a non-final field. > > I believe current implementations in fact enforce the necessary ordering. > But since it (surprisingly) appears to not be completely free to have > dependencies of this kind enforce memory visibility ordering (DEC Alpha > famously didn't), I wouldn't bet large sums of money on this statement > remaining true indefinitely on all relevant architectures. Indeed, and I have my doubts about RISC V. But it seems to me that any processor architecture which broke dependency ordering in such a way would also require fences for every read, volatile or not, of reference type, would it not? Otherwise the basic security guarantees of the platform would not hold. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From peter.levart at gmail.com Sat Sep 2 09:57:16 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 2 Sep 2017 11:57:16 +0200 Subject: Review Request JDK-8186050: StackFrame should provide the method signature In-Reply-To: <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> References: <5ad2e4a3-3fd3-0df4-c6c4-0b9ca8068a4e@oracle.com> <1207001620.848399.1504077914483.JavaMail.zimbra@u-pem.fr> <6717ed0c-17d7-66f2-a73b-0a96b4a97998@oracle.com> Message-ID: <778ffb8d-14dc-e3f3-5f77-562638d6fdef@gmail.com> Hi Mandy, The API looks fine to me. Note that there is an opportunity for a follow-up optimization of the StackFrameInfo::getDescriptor() case. When MemberName's 'type' field is filled by native expandFromVM() it is usually filled with the descriptor string. MemberName::getMethodType() then parses this string into a MemberType, resolving all the types. So when StackFrameInfo::getDescriptor() is called, the descriptor string is 1st parsed into MethodType and then formatted back to the descriptor. By introducing new method into package-private MemberName - say getMethodDescriptorString(), this intermediate conversion could often be avoided (for example, if getMethodDescriptorString() was called before getMethodType() on an instance of MethodName). Regards, Peter On 09/01/2017 07:39 AM, mandy chung wrote: > Updated webrev: > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.01/index.html > > > This introduces two new methods, StackFrame::getMethodType and > StackFrame::getDescriptor. > > Mandy > > On 8/30/17 12:25 AM, Remi Forax wrote: >> Hi Mandy, >> thanks for taking care of this. >> >> In my opinion, we should provide both getMethodType() and >> getDescriptor(), >> getDescriptor() is handy for logging (finding the right overload when >> line numbers are not present) and getMethodType() is the one you >> whant if you want to inspect the runtime view of the stack frames >> (and by example interact with java.lang.invoke). For me, it's the >> same reason that give us getDeclaringClass() and getClassName() in >> the current API. >> >> So getDescriptor() can be called with no restriction but >> getMethodType() requires RETAIN_CLASS_REFERENCE. >> >> regards, >> R?mi >> >> ----- Mail original ----- >>> De: "mandy chung" >>> ?: "core-libs-dev" >>> Envoy?: Mardi 29 Ao?t 2017 00:57:28 >>> Objet: Review Request JDK-8186050: StackFrame should provide the >>> method signature >>> Method signature is missing in the StackFrame API. This proposes to add >>> StackFrame::getMethodDescriptor method to return the method descriptor >>> in a stack frame. >>> >>> Webrev at: >>> http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8186050/webrev.00/index.html >>> >>> >>> There are a couple options how to present the method signature in the >>> API level: >>> 1. Class[] getParameterTypes() and Class getReturnTypes() >>> similiar >>> to what java.lang.reflect.Method has. >>> 2. java.lang.invoke.MethodType >>> 3. a String representation (i) comma-separated list of the method's >>> formal parameter types (ii) bytecode method descriptor as specified >>> in JVMS >>> >>> Returning Class instance should require to add a new StackWalker >>> option to access to the parameter types and return type for option #1 >>> and #2. StackFrame::getDeclaringClass requires the stack walker to have >>> the RETAIN_CLASS_REFERENCE capability. >>> >>> Option #2 returning MethodType is handy while java.lang would reference >>> a type in java.lang.invoke. >>> >>> Option #3 requires the caller to parse the return string and call >>> Class.forName to get the Class instance. OTOH >>> MethodType::fromMethodDescriptorString method that returns MethodType >>> from a bytecode method descriptor string. >>> >>> Method signature is for information for typical cases. Getting Class >>> for the parameter types and return type would be a niche case. I think >>> returning the method descriptor string is a good option - keep the API >>> simple and can use MethodType::fromMethodDescriptorString to get back >>> the types if needed. >>> >>> thanks >>> Mandy > From Alan.Bateman at oracle.com Sat Sep 2 12:06:39 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 2 Sep 2017 13:06:39 +0100 Subject: RFR: JDK-8186946: Fix accessibility and other issues in the java.xml.bind module In-Reply-To: <59A9C459.5080208@oracle.com> References: <59A9C459.5080208@oracle.com> Message-ID: On 01/09/2017 21:34, Jonathan Gibbons wrote: > : > > src/java.xml.bind/share/classes/javax/xml/bind/util/package-info.java > ??? Remove redundant

, they are unnecessary. > - I think, similar to TCP_NODELAY, the spec should say that the options is TCP specific. From TCP_NODELAY: "The socket option is specific to stream-oriented sockets using the TCP/IP protocol.? > - "In TCP_QUICKACK mode?, maybe ?When the option is enabled?? > > -Chris. > > From peter.levart at gmail.com Wed Sep 27 09:31:12 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 27 Sep 2017 11:31:12 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <59CB46AE.2000701@oracle.com> References: <59CB46AE.2000701@oracle.com> Message-ID: <23d51a9d-6b25-4324-c293-556215169401@gmail.com> Hi Sherman, At first I checked the Deflater/Inflater changes. I'll come back with ZipFile later. I think the code is mostly correct, but I have a concern. If clean() is invoked via Deflater.end(), then the Deflater instance is still alive and synchronization is necessary as other threads might be concurrently invoking other methods. If clean() is invoked via Cleaner thread, then Deflater instance is already phantom reachable and no thread may be accessing it or be in the middle of executing any of its instance methods. How is this guaranteed? Critical Deflater methods synchronize on the zsRef instance, so at least at the beginning of the synchronized block, the Deflater instance is strongly reachable. Take for example the following Deflater instance method: ?254???? public void setDictionary(byte[] b, int off, int len) { ?255???????? if (b == null) { ?256???????????? throw new NullPointerException(); ?257???????? } ?258???????? if (off < 0 || len < 0 || off > b.length - len) { ?259???????????? throw new ArrayIndexOutOfBoundsException(); ?260???????? } ?261???????? synchronized (zsRef) { ?262???????????? ensureOpen(); ?263???????????? setDictionary(zsRef.address(), b, off, len); ?264???????? } ?265???? } Up to a point where 'this' is dereferenced to obtain the 'zsRef' value (line 261), the Deflater instance is reachable. But after that, even ensureOpen() may be inlined and 'this' is not needed any more. After that point, obtaining zsRef.address() and calling setDictionaly on the obtained address may be racing with Cleaner thread invoking ZStreamRef.run(): ? 49???? public void run() { ? 50???????? long addr = address; ? 51???????? address = 0; ? 52???????? if (addr != 0) { ? 53???????????? end.accept(addr); ? 54???????? } ? 55???? } Possible program execution is therefore: Thread1: zsRef.address() - returning non-zero address Thread2: ZStreamRef.run() - invoking Deflater.end(address) with non-zero address via the passed-in lambda. Thread1: setDictionary(, b, off, len) To fix this, you could sprinkle Reference.reachabilityFence(this) at appropriate places in Deflater, but it is simpler to just add synchronized modifier to ZStreamRef.run() method. There's no danger of deadlock(s) since Cleaner ensures the run() method is invoked at most once. The same reasoning applies to Inflater to as code is similar. A nit (Deflater and similar to Inflater): ?187???????? ZStreamRef ref = new ZStreamRef(init(level, DEFAULT_STRATEGY, nowrap), ?188???????????????????????????????????????? addr -> end(addr)); You could use a method reference there. Regards, Peter On 09/27/2017 08:35 AM, Xueming Shen wrote: > Hi, > > Please help review the change for JDK-8185582. > > issue:??? https://bugs.openjdk.java.net/browse/JDK-8185582 > webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev/ > csr:?????? https://bugs.openjdk.java.net/browse/JDK-8187485 > > The proposed change here is to replace the finalize() cleanup > mechanism with > the newly introduced java.lang.ref.Cleaner for java.util.zip package > (Deflater, > Inflater, ZipFile and > ZipFile.ZipFileInputStrean/ZipFileInflaterInputStream). > > Since it's an "incompatible" change, both behavioral and source > incompatibility, > to remove the public finalize() from these classes, a corresponding > CSR is also > created and need review as well. > > Thanks, > Sherman > From david.holmes at oracle.com Wed Sep 27 09:52:42 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Sep 2017 19:52:42 +1000 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <23d51a9d-6b25-4324-c293-556215169401@gmail.com> References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> Message-ID: Hi Peter, I thought Cleaner was supposed to avoid all these unpredictable reachability races? Otherwise why switch from using a finalizer ?? David On 27/09/2017 7:31 PM, Peter Levart wrote: > Hi Sherman, > > At first I checked the Deflater/Inflater changes. I'll come back with > ZipFile later. > > I think the code is mostly correct, but I have a concern. If clean() is > invoked via Deflater.end(), then the Deflater instance is still alive > and synchronization is necessary as other threads might be concurrently > invoking other methods. If clean() is invoked via Cleaner thread, then > Deflater instance is already phantom reachable and no thread may be > accessing it or be in the middle of executing any of its instance > methods. How is this guaranteed? Critical Deflater methods synchronize > on the zsRef instance, so at least at the beginning of the synchronized > block, the Deflater instance is strongly reachable. Take for example the > following Deflater instance method: > > > ?254???? public void setDictionary(byte[] b, int off, int len) { > ?255???????? if (b == null) { > ?256???????????? throw new NullPointerException(); > ?257???????? } > ?258???????? if (off < 0 || len < 0 || off > b.length - len) { > ?259???????????? throw new ArrayIndexOutOfBoundsException(); > ?260???????? } > ?261???????? synchronized (zsRef) { > ?262???????????? ensureOpen(); > ?263???????????? setDictionary(zsRef.address(), b, off, len); > ?264???????? } > ?265???? } > > > Up to a point where 'this' is dereferenced to obtain the 'zsRef' value > (line 261), the Deflater instance is reachable. But after that, even > ensureOpen() may be inlined and 'this' is not needed any more. After > that point, obtaining zsRef.address() and calling setDictionaly on the > obtained address may be racing with Cleaner thread invoking > ZStreamRef.run(): > > ? 49???? public void run() { > ? 50???????? long addr = address; > ? 51???????? address = 0; > ? 52???????? if (addr != 0) { > ? 53???????????? end.accept(addr); > ? 54???????? } > ? 55???? } > > > Possible program execution is therefore: > > Thread1: > > zsRef.address() - returning non-zero address > > Thread2: > > ZStreamRef.run() - invoking Deflater.end(address) with non-zero address > via the passed-in lambda. > > Thread1: > > setDictionary(, b, off, len) > > > To fix this, you could sprinkle Reference.reachabilityFence(this) at > appropriate places in Deflater, but it is simpler to just add > synchronized modifier to ZStreamRef.run() method. There's no danger of > deadlock(s) since Cleaner ensures the run() method is invoked at most once. > > The same reasoning applies to Inflater to as code is similar. > > A nit (Deflater and similar to Inflater): > > ?187???????? ZStreamRef ref = new ZStreamRef(init(level, > DEFAULT_STRATEGY, nowrap), > ?188???????????????????????????????????????? addr -> end(addr)); > > You could use a method reference there. > > Regards, Peter > > > > On 09/27/2017 08:35 AM, Xueming Shen wrote: >> Hi, >> >> Please help review the change for JDK-8185582. >> >> issue:??? https://bugs.openjdk.java.net/browse/JDK-8185582 >> webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev/ >> csr:?????? https://bugs.openjdk.java.net/browse/JDK-8187485 >> >> The proposed change here is to replace the finalize() cleanup >> mechanism with >> the newly introduced java.lang.ref.Cleaner for java.util.zip package >> (Deflater, >> Inflater, ZipFile and >> ZipFile.ZipFileInputStrean/ZipFileInflaterInputStream). >> >> Since it's an "incompatible" change, both behavioral and source >> incompatibility, >> to remove the public finalize() from these classes, a corresponding >> CSR is also >> created and need review as well. >> >> Thanks, >> Sherman >> > From uschindler at apache.org Wed Sep 27 10:31:42 2017 From: uschindler at apache.org (Uwe Schindler) Date: Wed, 27 Sep 2017 12:31:42 +0200 Subject: Invoke MR-JAR file verifier on command line without actually packaging Message-ID: <04c401d3377b$d1ee3230$75ca9690$@apache.org> Hi, I am building a multi-release JAR file using Apache Ant (for Lucene, see https://issues.apache.org/jira/browse/LUCENE-7966). As build tools like Ant/Gradle/Maven are packaging JAR files on their own, not using the official JAR tool, I wanted to do a quick check if the resulting JAR file is valid. https://bugs.openjdk.java.net/browse/JDK-8158295 added some checks to compare method signatures and class file between the base and the META-INF/versions occurrences. Is there a way to invoke those checks on an already packaged JAR file? Uwe P.S.: Just a bit of background for those who are interested! This is a totally new way to create a MR JAR by just pacthing the already compiled class files before packaging as JAR file. The use case here is: - We want to use java.util.Arrays / java.util.Objects static methods throughout Lucene, especially to improve bounds checks by having Java 9 use the intrinsics when accessing arrays/ByteBuffers. We have already seen a huge speed improvement for our implementation of the LZ4 compression! - We need to be compatible with Java 8, but improve Java 9. - So we compile all our Lucene classes against a "custom" variant of the interesting methods, located in a fallback classes we have implemented in our own org.apache.lucene.future.FutureArrays/FutureObjects. - We do not want to duplicate our code by creating clonesof all .java files that use these methods and we still want to still compile against Java 8 - as this is the release JDK version! We want it automatic! So our trick is new and also way cool: During packaging the JAR file we patch all class files using ASM's ClassRemapper and replace the Type org.apache.lucene.future.FutureArrays by java.util.Arrays, only for those that actually contain a reference to our own replacement methods. This is a 74-liner in Groovy! After patching we also place the patched files in the MR part of JAR file. Patch can be seen here: https://github.com/apache/lucene-solr/compare/master...uschindler:jira/LUCENE-7966-v2 Especially this is how we patch our class files, way cool: https://github.com/uschindler/lucene-solr/blob/d67eb6274d4c0b8315848db3f09809fbc0797f6e/lucene/core/src/tools/groovy/patch-mrjar-classes.groovy ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ From peter.levart at gmail.com Wed Sep 27 10:36:20 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 27 Sep 2017 12:36:20 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> Message-ID: <2139d434-c550-af36-b92c-3991535e1de6@gmail.com> Hi David, On 09/27/2017 11:52 AM, David Holmes wrote: > Hi Peter, > > I thought Cleaner was supposed to avoid all these unpredictable > reachability races? Otherwise why switch from using a finalizer ?? Unfortunately, there is no hidden magic in Cleaner. It is better than finalize() mostly because the clean-up function may be invoked by the user, which also de-registers it, allowing GC to collect it rather than leaving it to reference-processing pipeline to process it for no reason. And of course, it guarantees that the tracked referent can not be resurrected as a result of cleanup code execution. What remains unchanged with Cleaner is the point in program execution when the tracked referent is determined to be phantom-reachable (finalizable) and therefore its associated PhantomReference(s) eligible for processing (finalize() invoked). Regards, Peter > > David > > On 27/09/2017 7:31 PM, Peter Levart wrote: >> Hi Sherman, >> >> At first I checked the Deflater/Inflater changes. I'll come back with >> ZipFile later. >> >> I think the code is mostly correct, but I have a concern. If clean() >> is invoked via Deflater.end(), then the Deflater instance is still >> alive and synchronization is necessary as other threads might be >> concurrently invoking other methods. If clean() is invoked via >> Cleaner thread, then Deflater instance is already phantom reachable >> and no thread may be accessing it or be in the middle of executing >> any of its instance methods. How is this guaranteed? Critical >> Deflater methods synchronize on the zsRef instance, so at least at >> the beginning of the synchronized block, the Deflater instance is >> strongly reachable. Take for example the following Deflater instance >> method: >> >> >> ??254???? public void setDictionary(byte[] b, int off, int len) { >> ??255???????? if (b == null) { >> ??256???????????? throw new NullPointerException(); >> ??257???????? } >> ??258???????? if (off < 0 || len < 0 || off > b.length - len) { >> ??259???????????? throw new ArrayIndexOutOfBoundsException(); >> ??260???????? } >> ??261???????? synchronized (zsRef) { >> ??262???????????? ensureOpen(); >> ??263???????????? setDictionary(zsRef.address(), b, off, len); >> ??264???????? } >> ??265???? } >> >> >> Up to a point where 'this' is dereferenced to obtain the 'zsRef' >> value (line 261), the Deflater instance is reachable. But after that, >> even ensureOpen() may be inlined and 'this' is not needed any more. >> After that point, obtaining zsRef.address() and calling setDictionaly >> on the obtained address may be racing with Cleaner thread invoking >> ZStreamRef.run(): >> >> ?? 49???? public void run() { >> ?? 50???????? long addr = address; >> ?? 51???????? address = 0; >> ?? 52???????? if (addr != 0) { >> ?? 53???????????? end.accept(addr); >> ?? 54???????? } >> ?? 55???? } >> >> >> Possible program execution is therefore: >> >> Thread1: >> >> zsRef.address() - returning non-zero address >> >> Thread2: >> >> ZStreamRef.run() - invoking Deflater.end(address) with non-zero >> address via the passed-in lambda. >> >> Thread1: >> >> setDictionary(, b, off, len) >> >> >> To fix this, you could sprinkle Reference.reachabilityFence(this) at >> appropriate places in Deflater, but it is simpler to just add >> synchronized modifier to ZStreamRef.run() method. There's no danger >> of deadlock(s) since Cleaner ensures the run() method is invoked at >> most once. >> >> The same reasoning applies to Inflater to as code is similar. >> >> A nit (Deflater and similar to Inflater): >> >> ??187???????? ZStreamRef ref = new ZStreamRef(init(level, >> DEFAULT_STRATEGY, nowrap), >> ??188???????????????????????????????????????? addr -> end(addr)); >> >> You could use a method reference there. >> >> Regards, Peter >> >> >> >> On 09/27/2017 08:35 AM, Xueming Shen wrote: >>> Hi, >>> >>> Please help review the change for JDK-8185582. >>> >>> issue:??? https://bugs.openjdk.java.net/browse/JDK-8185582 >>> webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev/ >>> csr:?????? https://bugs.openjdk.java.net/browse/JDK-8187485 >>> >>> The proposed change here is to replace the finalize() cleanup >>> mechanism with >>> the newly introduced java.lang.ref.Cleaner for java.util.zip package >>> (Deflater, >>> Inflater, ZipFile and >>> ZipFile.ZipFileInputStrean/ZipFileInflaterInputStream). >>> >>> Since it's an "incompatible" change, both behavioral and source >>> incompatibility, >>> to remove the public finalize() from these classes, a corresponding >>> CSR is also >>> created and need review as well. >>> >>> Thanks, >>> Sherman >>> >> From david.holmes at oracle.com Wed Sep 27 12:49:10 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Sep 2017 22:49:10 +1000 Subject: Review Request JDK-8164512: Replace ClassLoader use of finalizer with phantom reference to unload native library In-Reply-To: <0c6802f2-678b-977d-e827-ef03c7f376a2@oracle.com> References: <1643128c-8714-4d6d-253a-b7413a5eb8ef@oracle.com> <9b5b908b-e54c-67c4-13a4-250d2087241f@oracle.com> <42cd522c-4a1f-f231-3323-6bb4a0183a1e@oracle.com> <3a0fe286-f38a-f7f1-4a23-15c43bb42a80@oracle.com> <3d52390c-d468-1f74-6e1e-4a9bc960b681@oracle.com> <1e71b112-cdd5-ba0d-d519-5d54e5ace549@oracle.com> <0c6802f2-678b-977d-e827-ef03c7f376a2@oracle.com> Message-ID: <22d101c6-5255-b9b7-606c-6f5519a5c597@oracle.com> Corrections ... On 27/09/2017 2:36 PM, David Holmes wrote: > On 27/09/2017 1:36 PM, mandy chung wrote: >> On 9/26/17 7:35 PM, David Holmes wrote: >>> On 27/09/2017 12:11 PM, mandy chung wrote: >>>> On 9/26/17 7:06 PM, David Holmes wrote: >>>>> >>>>>> It is not tied with the Cleaner change. Instead, the FindClass bug >>>>>> blocks the finalizer to Cleaner change. >>>>>> >>>>>> FindClass bug is uncovered when I implemented the change from >>>>>> finalizer to Cleaner (or phantom reference).?? There is a test >>>>>> calling FindClass to look for a class defined by the class loader >>>>>> being unloaded, say L. L is not Gc'ed and so FindClass >>>>>> successfully finds the class (which resurrect the class loader >>>>>> which was marked finalizable). >>>>>> >>>>>> Is that clearer? >>>>> >>>>> So the issue is only that this test breaks?? >>>> >>>> No.? The test reveals a bug in JNI_FindClass that uses a class >>>> loader being finalized as the context when NativeLibrary is the caller. >>>>> And you want to change the FindClass spec to make it clear the test >>>>> is what needs to be changed? >>>> No.?? It is a bug in the hotspot implementation. ? The JNI spec says >>>> that the context of JNI_OnUnload being called is unknown. The >>>> hotspot implementation of FindClass uses the class loader associated >>>> with that native library as the context when invoked from >>>> JNI_OnUnload which is wrong. >>> >>> I'm not sure I agree it is wrong. As I've said elsewhere there's a >>> good chance that if you are trying to load classes via FindClass as >>> part of a unload hook (which implies you are using custom >>> classloaders), then it may be only the current loader or a parent >>> (still custom) can load that class. But we're on the fringe of >>> realistic expectations here as the context is specified as being >>> "unknown". >>> >> For a native unload hook to access some class defined by this class >> loader, definitely it should not write to any fields since the class >> and class loader are not strongly reachable.?? Reading the current >> state stored in the class can be done by writing to the native fields. > > Yes that is a good point - but as the spec says due to the unknown > context the hook has to be very careful about what it tries to do. I > agree it is doubtful that anyone can, or should, be relying on the > direct use of the classloader that has become unreachable, but ... > >> I'd like to know what other use cases that FindClass must ressurrect a >> class defined by this class loader or find a class defined by its >> ancestor if you have any in mind that the existing code can't be >> replaced due to the proposed change. > > ... I can easily imagine a subsystem that runs under a custom loader and > which then instantiates further execution contexts (per connection for > example) each with their own classloader and which can be reclaimed > after the request is complete. I can then easily imagine that they use > an unload hook to record statistics about native library use, and that > the statistics classes are in the top-level custom loader, and not > locatable from the system loader. > > While the spec makes no guarantees this will work it only says > programmers "should be conservative in their use of VM services" which > strongly suggests to me a "try it and see if it works" approach. In the > current code while loading from the loader being reclaimed is highly > dubious, delegating through that loader seems fairly reasonable to me. > >>> That said given the spec says "unknown" the behaviour of the VM could >>> change and still be in spec. >>> >>> I presume that when using a cleaner the current classloader that >>> would be used by FindClass is the system loader? Hence the observed >>> behaviour of FindClass "changes" if you switch to the cleaner from >>> the finalizer - and can't be reverted to the old behaviour by using a >>> command-line flag. Hence if we want to be able to revert we have to >>> do that in a FindClass-only change first. Then drop-in the cleaner >>> update and remove the flag. >>> >>>> I will file a separate JBS issue to separate this JNI bug. >>> >>> Okay. I see this as a RFE not a bug per-se: change from "unknown >>> context" to a specific well known context. >> This case is arguable whether it's considered as a RFE or a bug >> because the current spec of JNI_OnUnload and JNI_FindClass are not >> aligned.? I lean toward a bug.??? The bottom line:? do you agree with >> this proposed JNI spec change? > > I don't think the spec _has_ to change because I disagree that there is > a misalignment between JNI_OnUnload and JNI_FindClass. FindClass clearly > states it uses the current loader or else the system loader if there is That is not accurate - sorry. FindClass doesn't actually address the possibility of being called via these load/unload hooks. See more below. > no notion of a current loader. OnUnload says it runs in an unknown > context, so you don't know what the current loader may be, or even if > there is one. But regardless a call to FindClass from OnUnload should > use the current loader if it exists, or else the system loader. The fact > it may be dubious to use the current loader when it is itself in the > process of being unloaded does not impinge on the voracity of the spec > in my opinion. > > So you can change to using a Cleaner instead of a finalizer and while it > will behave differently, that change in behaviour does not violate the > spec in any way - again in my opinion. > > Now if you want to pave the way for a future switch to Cleaner by > changing the spec for JNI_OnUnload such that it must be executed in a > context where (equivalently) there either is no current loader or the > current loader is the system loader, then I do not oppose that. But the > only purpose that serves is to allow a migration path to the new > behaviour - and then forever locks us in. > > Note however I would not want to see the implementation of FindClass > having to special case this - I would hope it just happens naturally if > the Cleaner thread reports the current class loader as the system > loader. Does it? I missed the fact that we already special case this for JNI_OnLoad and JNI_OnUnload. I would have thought that in the OnLoad case we would find the classloader of the class loading the native library without any need to resort to the NativeLibrary support code in ClassLoader. I guess that this: // Find calling class Klass* k = thread->security_get_caller_class(0); does not find the "caller" that I would have expected, but instead finds java.lang.System because we're executing System.loadLibrary - and hence finds the boot loader not the actual loader required. But the fact we jump through all these hoops is in itself questionable because the specification for JNI_FindClass does not indicate this will happen. It only accounts for two cases: 1. A JNI call from a declared native method - which uses the loader of the class that defines the method 2. A JNI call "through the Invocation Interface" which I interpret as being a JNI call from C code, from an attached thread, with no Java frames on the stack. In which case the system loader is used. A call from JNI_OnLoad (or OnUnload) does not, to me, fit either of these cases; nor does JNI_OnLoad say anything about the context in which it executes. So it seems we have presumed that this case should mean "use the loader of the class which loaded the native library". A very reasonable approach, but not one defined by the specification as far as I can see. But given this, it is not unreasonable to also use the same interpretation for JNI_OnUnload. So there is a gap in the specification regarding the execution context of the library lifecycle function hooks - other than onUnload being an "unknown context". David ----- > > Thanks, > David > >> >> Mandy From xueming.shen at oracle.com Wed Sep 27 14:56:43 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 27 Sep 2017 07:56:43 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <23d51a9d-6b25-4324-c293-556215169401@gmail.com> References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> Message-ID: <59CBBC2B.7030800@oracle.com> Hi Peter, Sorry, I might not understand your use scenario correctly. Let me try :-) If clean() is invoked via Deflater.end() first, my reading of the Cleaner code suggests that the clean()->ZStreamRef.run() is run by the thread calling deflater.end() directly, not the Cleaner thread. In this scenario, since we are still inside the synchronized(zsRef) block, all other "zsRef" sensitive deflater methods should be blocked by the same synchronzied(zeRef) ? I would assume we don't have problem in this use scenario? 567 public void end() { 568 synchronized (zsRef) { 569 cleanable.clean(); 570 buf = null; 571 } 572 } If some other method is invoked first, for example the setDictionary(...) as in your sample, any direct invocation of delfater.end() from other thread should be blocked because of "synchronized(zsRef)". So I would assume this is not a concern. It seems the concern is that the "Deflater.this" object itself is no longer needed and becomes unreachable after line#261, therefore this deflater becomes phantom-reachable and the ZStreamRef.run() is called by the Cleaner thread, then we have a race condition between the thread calls the setDictionary() and the Cleaner thread? I guess I might have a wrong assumption here, is it true that if someone/thread is calling deflater.setDictionary(), that someone/thread should still hold a reference to the deflater and therefore prevent it from becoming phantom-reachable? Or you are saying it is possible under the hotspot optimization the deflater.setDictionary() (and its ensureOpen() included) is being inline-ed and therefore there is no more reference to that deflater even we are still inside that deflater.setDictionary(), so that "after ine#261" scenario becomes possible? Thanks, Sherman On 9/27/17, 2:31 AM, Peter Levart wrote: > Hi Sherman, > > At first I checked the Deflater/Inflater changes. I'll come back with > ZipFile later. > > I think the code is mostly correct, but I have a concern. If clean() > is invoked via Deflater.end(), then the Deflater instance is still > alive and synchronization is necessary as other threads might be > concurrently invoking other methods. If clean() is invoked via Cleaner > thread, then Deflater instance is already phantom reachable and no > thread may be accessing it or be in the middle of executing any of its > instance methods. How is this guaranteed? Critical Deflater methods > synchronize on the zsRef instance, so at least at the beginning of the > synchronized block, the Deflater instance is strongly reachable. Take > for example the following Deflater instance method: > > > 254 public void setDictionary(byte[] b, int off, int len) { > 255 if (b == null) { > 256 throw new NullPointerException(); > 257 } > 258 if (off < 0 || len < 0 || off > b.length - len) { > 259 throw new ArrayIndexOutOfBoundsException(); > 260 } > 261 synchronized (zsRef) { > 262 ensureOpen(); > 263 setDictionary(zsRef.address(), b, off, len); > 264 } > 265 } > > > Up to a point where 'this' is dereferenced to obtain the 'zsRef' value > (line 261), the Deflater instance is reachable. But after that, even > ensureOpen() may be inlined and 'this' is not needed any more. After > that point, obtaining zsRef.address() and calling setDictionaly on the > obtained address may be racing with Cleaner thread invoking > ZStreamRef.run(): > > 49 public void run() { > 50 long addr = address; > 51 address = 0; > 52 if (addr != 0) { > 53 end.accept(addr); > 54 } > 55 } > > > Possible program execution is therefore: > > Thread1: > > zsRef.address() - returning non-zero address > > Thread2: > > ZStreamRef.run() - invoking Deflater.end(address) with non-zero > address via the passed-in lambda. > > Thread1: > > setDictionary(, b, off, len) > > > To fix this, you could sprinkle Reference.reachabilityFence(this) at > appropriate places in Deflater, but it is simpler to just add > synchronized modifier to ZStreamRef.run() method. There's no danger of > deadlock(s) since Cleaner ensures the run() method is invoked at most > once. > > The same reasoning applies to Inflater to as code is similar. > > A nit (Deflater and similar to Inflater): > > 187 ZStreamRef ref = new ZStreamRef(init(level, > DEFAULT_STRATEGY, nowrap), > 188 addr -> end(addr)); > > You could use a method reference there. > > Regards, Peter > > > > On 09/27/2017 08:35 AM, Xueming Shen wrote: >> Hi, >> >> Please help review the change for JDK-8185582. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8185582 >> webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev/ >> csr: https://bugs.openjdk.java.net/browse/JDK-8187485 >> >> The proposed change here is to replace the finalize() cleanup >> mechanism with >> the newly introduced java.lang.ref.Cleaner for java.util.zip package >> (Deflater, >> Inflater, ZipFile and >> ZipFile.ZipFileInputStrean/ZipFileInflaterInputStream). >> >> Since it's an "incompatible" change, both behavioral and source >> incompatibility, >> to remove the public finalize() from these classes, a corresponding >> CSR is also >> created and need review as well. >> >> Thanks, >> Sherman >> > From paul.sandoz at oracle.com Wed Sep 27 17:07:17 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 27 Sep 2017 10:07:17 -0700 Subject: Invoke MR-JAR file verifier on command line without actually packaging In-Reply-To: <04c401d3377b$d1ee3230$75ca9690$@apache.org> References: <04c401d3377b$d1ee3230$75ca9690$@apache.org> Message-ID: Hi Uwe, Unfortunately i don?t recall there being a publicly supported way to validate an existing Mr.Jar (except by recreating it), either by the jar tool itself nor by a programatic API. In a more general sense the problem is one of given two jars validate that the class file bytes in each expose a compatible public API, with a valid Mr.Jar being more restrictive that no new public artifacts should be introduced. I don?t know of any tooling outside of the JDK that might do such checks. > On 27 Sep 2017, at 03:31, Uwe Schindler wrote: > > Hi, > > I am building a multi-release JAR file using Apache Ant (for Lucene, see https://issues.apache.org/jira/browse/LUCENE-7966). > > As build tools like Ant/Gradle/Maven are packaging JAR files on their own, not using the official JAR tool, I wanted to do a quick check if the resulting JAR file is valid. https://bugs.openjdk.java.net/browse/JDK-8158295 added some checks to compare method signatures and class file between the base and the META-INF/versions occurrences. > > Is there a way to invoke those checks on an already packaged JAR file? > > Uwe > > P.S.: Just a bit of background for those who are interested! This is a totally new way to create a MR JAR by just pacthing the already compiled class files before packaging as JAR file. The use case here is: > - We want to use java.util.Arrays / java.util.Objects static methods throughout Lucene, especially to improve bounds checks by having Java 9 use the intrinsics when accessing arrays/ByteBuffers. We have already seen a huge speed improvement for our implementation of the LZ4 compression! Good know! > - We need to be compatible with Java 8, but improve Java 9. > - So we compile all our Lucene classes against a "custom" variant of the interesting methods, located in a fallback classes we have implemented in our own org.apache.lucene.future.FutureArrays/FutureObjects. > - We do not want to duplicate our code by creating clonesof all .java files that use these methods and we still want to still compile against Java 8 - as this is the release JDK version! We want it automatic! > > So our trick is new and also way cool: During packaging the JAR file we patch all class files using ASM's ClassRemapper and replace the Type org.apache.lucene.future.FutureArrays by java.util.Arrays, only for those that actually contain a reference to our own replacement methods. This is a 74-liner in Groovy! After patching we also place the patched files in the MR part of JAR file. > > Patch can be seen here: > https://github.com/apache/lucene-solr/compare/master...uschindler:jira/LUCENE-7966-v2 > > Especially this is how we patch our class files, way cool: > https://github.com/uschindler/lucene-solr/blob/d67eb6274d4c0b8315848db3f09809fbc0797f6e/lucene/core/src/tools/groovy/patch-mrjar-classes.groovy > Very neat. Paul. From hboehm at google.com Wed Sep 27 17:55:13 2017 From: hboehm at google.com (Hans Boehm) Date: Wed, 27 Sep 2017 10:55:13 -0700 Subject: Phantom Referencesvs finalizers (was: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers) Message-ID: "And of course, it guarantees that the tracked referent can not be resurrected as a result of cleanup code execution." True, but it seems to me that the real property you want is "it guarantees that the tracked referent can not be resurrected". Full stop. Actually, the property I really need is: "The referent of an enqueued PhantomReference cannot subsequently be accessed." Unfortunately, neither of those last two appear to be true due to the weakly specified behavior of JNI WeakGlobalRefs. The spec says: "Interactions between weak global references and PhantomReferences are undefined. In particular, implementations of a Java VM may (or may not) process weak global references after PhantomReferences, and it may (or may not) be possible to use weak global references to hold on to objects which are also referred to by PhantomReference objects. This undefined use of weak global references should be avoided." This allows a WeakGlobalRef to be converted to a strong JNI ref after a PhantomReference to the same object has been enqueued, thus resurrecting the referent, and allowing access. The advice in the last sentence seems to be both essential and completely impractical, since the WeakGlobalRef may point to another object, which may then indirectly point ot the PhantomReference referent. Hence you can't prevent this interaction without global, whole program knowledge of which objects may indirectly reference PhantomReference referents. This is essentially the same problem we had with (unordered) finalization, only now confined to programs that use WeakGlobalRefs. Am I reading this correctly? Is it intended? It seems like it should be fixable by giving WeakGlobalRefs essentially WeakReference behavior, as the name implies? On Wed, Sep 27, 2017 at 3:36 AM, Peter Levart wrote: > Hi David, > > On 09/27/2017 11:52 AM, David Holmes wrote: > >> Hi Peter, >> >> I thought Cleaner was supposed to avoid all these unpredictable >> reachability races? Otherwise why switch from using a finalizer ?? >> > > Unfortunately, there is no hidden magic in Cleaner. It is better than > finalize() mostly because the clean-up function may be invoked by the user, > which also de-registers it, allowing GC to collect it rather than leaving > it to reference-processing pipeline to process it for no reason. And of > course, it guarantees that the tracked referent can not be resurrected as a > result of cleanup code execution. What remains unchanged with Cleaner is > the point in program execution when the tracked referent is determined to > be phantom-reachable (finalizable) and therefore its associated > PhantomReference(s) eligible for processing (finalize() invoked). > > Regards, Peter > > > >> David >> >> On 27/09/2017 7:31 PM, Peter Levart wrote: >> >>> Hi Sherman, >>> >>> At first I checked the Deflater/Inflater changes. I'll come back with >>> ZipFile later. >>> >>> I think the code is mostly correct, but I have a concern. If clean() is >>> invoked via Deflater.end(), then the Deflater instance is still alive and >>> synchronization is necessary as other threads might be concurrently >>> invoking other methods. If clean() is invoked via Cleaner thread, then >>> Deflater instance is already phantom reachable and no thread may be >>> accessing it or be in the middle of executing any of its instance methods. >>> How is this guaranteed? Critical Deflater methods synchronize on the zsRef >>> instance, so at least at the beginning of the synchronized block, the >>> Deflater instance is strongly reachable. Take for example the following >>> Deflater instance method: >>> >>> >>> 254 public void setDictionary(byte[] b, int off, int len) { >>> 255 if (b == null) { >>> 256 throw new NullPointerException(); >>> 257 } >>> 258 if (off < 0 || len < 0 || off > b.length - len) { >>> 259 throw new ArrayIndexOutOfBoundsException(); >>> 260 } >>> 261 synchronized (zsRef) { >>> 262 ensureOpen(); >>> 263 setDictionary(zsRef.address(), b, off, len); >>> 264 } >>> 265 } >>> >>> >>> Up to a point where 'this' is dereferenced to obtain the 'zsRef' value >>> (line 261), the Deflater instance is reachable. But after that, even >>> ensureOpen() may be inlined and 'this' is not needed any more. After that >>> point, obtaining zsRef.address() and calling setDictionaly on the obtained >>> address may be racing with Cleaner thread invoking ZStreamRef.run(): >>> >>> 49 public void run() { >>> 50 long addr = address; >>> 51 address = 0; >>> 52 if (addr != 0) { >>> 53 end.accept(addr); >>> 54 } >>> 55 } >>> >>> >>> Possible program execution is therefore: >>> >>> Thread1: >>> >>> zsRef.address() - returning non-zero address >>> >>> Thread2: >>> >>> ZStreamRef.run() - invoking Deflater.end(address) with non-zero address >>> via the passed-in lambda. >>> >>> Thread1: >>> >>> setDictionary(, b, off, len) >>> >>> >>> To fix this, you could sprinkle Reference.reachabilityFence(this) at >>> appropriate places in Deflater, but it is simpler to just add synchronized >>> modifier to ZStreamRef.run() method. There's no danger of deadlock(s) since >>> Cleaner ensures the run() method is invoked at most once. >>> >>> The same reasoning applies to Inflater to as code is similar. >>> >>> A nit (Deflater and similar to Inflater): >>> >>> 187 ZStreamRef ref = new ZStreamRef(init(level, >>> DEFAULT_STRATEGY, nowrap), >>> 188 addr -> end(addr)); >>> >>> You could use a method reference there. >>> >>> Regards, Peter >>> >>> >>> >>> On 09/27/2017 08:35 AM, Xueming Shen wrote: >>> >>>> Hi, >>>> >>>> Please help review the change for JDK-8185582. >>>> >>>> issue: https://bugs.openjdk.java.net/browse/JDK-8185582 >>>> webrev: http://cr.openjdk.java.net/~sherman/8185582/webrev/ >>>> csr: https://bugs.openjdk.java.net/browse/JDK-8187485 >>>> >>>> The proposed change here is to replace the finalize() cleanup mechanism >>>> with >>>> the newly introduced java.lang.ref.Cleaner for java.util.zip package >>>> (Deflater, >>>> Inflater, ZipFile and ZipFile.ZipFileInputStrean/Zip >>>> FileInflaterInputStream). >>>> >>>> Since it's an "incompatible" change, both behavioral and source >>>> incompatibility, >>>> to remove the public finalize() from these classes, a corresponding CSR >>>> is also >>>> created and need review as well. >>>> >>>> Thanks, >>>> Sherman >>>> >>>> >>> > From paul.sandoz at oracle.com Wed Sep 27 18:38:30 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 27 Sep 2017 11:38:30 -0700 Subject: RFR: jsr166 jdk10 integration wave 3 In-Reply-To: <7A0F7B6A-6B10-4BCB-9C2A-BC2135D936A1@oracle.com> References: <1b83cdc7-7c23-0534-650b-fa4e343e321d@oracle.com> <7A0F7B6A-6B10-4BCB-9C2A-BC2135D936A1@oracle.com> Message-ID: <9F0F9A64-1F0C-4EB2-A087-9898B933A447@oracle.com> > On 21 Sep 2017, at 10:29, Paul Sandoz wrote: > > Looks good, i just need to look a little more closely at the ConcurrentSkipListMap changes. > 328 * Note that, with VarHandles, a boolean result of 329 * compareAndSet must be declared even if not used. This should not be necessary, since for such methods the return is not polymorphic. Did you observe some performance or code generation issues? Paul. From mandy.chung at oracle.com Wed Sep 27 18:58:22 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 27 Sep 2017 11:58:22 -0700 Subject: Review Request JDK-8164512: Replace ClassLoader use of finalizer with phantom reference to unload native library In-Reply-To: <22d101c6-5255-b9b7-606c-6f5519a5c597@oracle.com> References: <1643128c-8714-4d6d-253a-b7413a5eb8ef@oracle.com> <9b5b908b-e54c-67c4-13a4-250d2087241f@oracle.com> <42cd522c-4a1f-f231-3323-6bb4a0183a1e@oracle.com> <3a0fe286-f38a-f7f1-4a23-15c43bb42a80@oracle.com> <3d52390c-d468-1f74-6e1e-4a9bc960b681@oracle.com> <1e71b112-cdd5-ba0d-d519-5d54e5ace549@oracle.com> <0c6802f2-678b-977d-e827-ef03c7f376a2@oracle.com> <22d101c6-5255-b9b7-606c-6f5519a5c597@oracle.com> Message-ID: On 9/27/17 5:49 AM, David Holmes wrote: > > I missed the fact that we already special case this for JNI_OnLoad and > JNI_OnUnload. Yes this is buried in JNI_FindClass method. > I would have thought that in the OnLoad case we would find the > classloader of the class loading the native library without any need > to resort to the NativeLibrary support code in ClassLoader. I guess > that this: > > ? // Find calling class > ? Klass* k = thread->security_get_caller_class(0); > > does not find the "caller" that I would have expected, but instead > finds java.lang.System because we're executing System.loadLibrary - > and hence finds the boot loader not the actual loader required. > In the current implementation (without this change), NativeLibrary.load and NativeLibrary.unload native methods are the caller calling JNI_OnLoad and JNI_OnUnload respectively. > But the fact we jump through all these hoops is in itself questionable > because the specification for JNI_FindClass does not indicate this > will happen. It only accounts for two cases: > > 1. A JNI call from a declared native method - which uses the loader of > the class that defines the method > > 2. A JNI call "through the Invocation Interface" which I interpret as > being a JNI call from C code, from an attached thread, with no Java > frames on the stack. In which case the system loader is used. > > A call from JNI_OnLoad (or OnUnload) does not, to me, fit either of > these cases; nor does JNI_OnLoad say anything about the context in > which it executes. So it seems we have presumed that this case should > mean "use the loader of the class which loaded the native library". A > very reasonable approach, but not one defined by the specification as > far as I can see. That's the whole point of this discussion and the spec needs clarification. > But given this, it is not unreasonable to also use the same > interpretation for JNI_OnUnload. > That might be how it ends up the current implementation. > So there is a gap in the specification regarding the execution context > of the library lifecycle function hooks - other than onUnload being an > "unknown context". This suggests that we should clarify in JNI_OnLoad spec to specify the context. FYI.? I file a separate issue: https://bugs.openjdk.java.net/browse/JDK-8188052 Mandy From paul.sandoz at oracle.com Wed Sep 27 19:18:05 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 27 Sep 2017 12:18:05 -0700 Subject: Review Request: JDK-6373396: (cl spec) clarify spec of ClassLoader.getPackages() about run-time package In-Reply-To: <902c2df2-c8ae-fd8e-f830-8be4e1b8d86c@oracle.com> References: <902c2df2-c8ae-fd8e-f830-8be4e1b8d86c@oracle.com> Message-ID: <8855126C-BA96-49C3-999E-7CBDD021705B@oracle.com> > On 26 Sep 2017, at 13:13, mandy chung wrote: > > Minor update to the spec of ClassLoader::getPackages and other relevant methods to clarify that the Package(s) are the packages that *have been* defined. Also include a link to JVMS 5.3 about run-time packages, as in Package class spec. > > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/6373396/webrev.00/ > +1 Paul. From mandy.chung at oracle.com Wed Sep 27 20:41:20 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 27 Sep 2017 13:41:20 -0700 Subject: Phantom Referencesvs finalizers (was: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers) In-Reply-To: References: Message-ID: On 9/27/17 10:55 AM, Hans Boehm wrote: > "And of course, it guarantees that the tracked referent can not be > resurrected as a result of cleanup code execution." > > True, but it seems to me that the real property you want is "it guarantees > that the tracked referent can not be resurrected". Full stop. Actually, the > property I really need is: "The referent of an enqueued PhantomReference > cannot subsequently be accessed." > > Unfortunately, neither of those last two appear to be true due to the > weakly specified behavior of JNI WeakGlobalRefs. The spec says: > > "Interactions between weak global references and PhantomReferences are > undefined. In particular, implementations of a Java VM may (or may not) > process weak global references after PhantomReferences, and it may (or may > not) be possible to use weak global references to hold on to objects which > are also referred to by PhantomReference objects. This undefined use of > weak global references should be avoided." > > This allows a WeakGlobalRef to be converted to a strong JNI ref after a > PhantomReference to the same object has been enqueued, thus resurrecting > the referent, and allowing access. > > The advice in the last sentence seems to be both essential and completely > impractical, since the WeakGlobalRef may point to another object, which may > then indirectly point ot the PhantomReference referent. Hence you can't > prevent this interaction without global, whole program knowledge of which > objects may indirectly reference PhantomReference referents. This is > essentially the same problem we had with (unordered) finalization, only now > confined to programs that use WeakGlobalRefs. > > Am I reading this correctly? Is it intended? It seems like it should be > fixable by giving WeakGlobalRefs essentially WeakReference behavior, as the > name implies? > Thanks for pointing this out.?? I create a JBS issue to follow up this: ?? https://bugs.openjdk.java.net/browse/JDK-8188066 Mandy From dl at cs.oswego.edu Wed Sep 27 20:44:09 2017 From: dl at cs.oswego.edu (Doug Lea) Date: Wed, 27 Sep 2017 16:44:09 -0400 Subject: RFR: jsr166 jdk10 integration wave 3 In-Reply-To: <9F0F9A64-1F0C-4EB2-A087-9898B933A447@oracle.com> References: <1b83cdc7-7c23-0534-650b-fa4e343e321d@oracle.com> <7A0F7B6A-6B10-4BCB-9C2A-BC2135D936A1@oracle.com> <9F0F9A64-1F0C-4EB2-A087-9898B933A447@oracle.com> Message-ID: On 09/27/2017 02:38 PM, Paul Sandoz wrote: >> Looks good, i just need to look a little more closely at the ConcurrentSkipListMap changes. >> > > > 328 * Note that, with VarHandles, a boolean result of > 329 * compareAndSet must be declared even if not used. > This should not be necessary, since for such methods the return is not polymorphic. Did you observe some performance or code generation issues? > It was necessary at the time I started rewriting this in June, but apparently not now. I'll get rid of the annoying unnecessary declarations. Thanks to whoever fixed this! -Doug From kim.barrett at oracle.com Wed Sep 27 21:12:37 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 27 Sep 2017 17:12:37 -0400 Subject: Phantom Referencesvs finalizers (was: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers) In-Reply-To: References: Message-ID: <73D2ABF1-0E00-4544-B456-BFE6CF9DF92E@oracle.com> > On Sep 27, 2017, at 1:55 PM, Hans Boehm wrote: > > "And of course, it guarantees that the tracked referent can not be > resurrected as a result of cleanup code execution." > > True, but it seems to me that the real property you want is "it guarantees > that the tracked referent can not be resurrected". Full stop. Actually, the > property I really need is: "The referent of an enqueued PhantomReference > cannot subsequently be accessed." > > Unfortunately, neither of those last two appear to be true due to the > weakly specified behavior of JNI WeakGlobalRefs. The spec says: > > "Interactions between weak global references and PhantomReferences are > undefined. In particular, implementations of a Java VM may (or may not) > process weak global references after PhantomReferences, and it may (or may > not) be possible to use weak global references to hold on to objects which > are also referred to by PhantomReference objects. This undefined use of > weak global references should be avoided." > > This allows a WeakGlobalRef to be converted to a strong JNI ref after a > PhantomReference to the same object has been enqueued, thus resurrecting > the referent, and allowing access. > > The advice in the last sentence seems to be both essential and completely > impractical, since the WeakGlobalRef may point to another object, which may > then indirectly point ot the PhantomReference referent. Hence you can't > prevent this interaction without global, whole program knowledge of which > objects may indirectly reference PhantomReference referents. This is > essentially the same problem we had with (unordered) finalization, only now > confined to programs that use WeakGlobalRefs. > > Am I reading this correctly? Is it intended? It seems like it should be > fixable by giving WeakGlobalRefs essentially WeakReference behavior, as the > name implies? That quote from the JNI specification is, I think, left over from before JDK-8071507. Now that PhantomReferences are cleared when their referent is no longer referenced, the effective strength of phantoms and JNI weak references are equivalent, at least in Hotspot. The java.lang.ref and JNI specifications seem to allow the possibility that an implementation will clear references at a given strength without clearing references of a weaker strength, even if that would allow the referent of one of those stronger references to be obtained via one of those weaker references. (And indeed, that must be possible for finalize.) I'm not sure that's intentional though (except for finalize), and I've seen discussions [1] suggesting that it would be at least surprising to behave that way; Hotspot certainly doesn't, and those discussions suggested that neither do any of the other implementations. This suggests some specification tightening is called for, in order to address these concerns around post-Cleaner cleanup resurrection. [2] Giving WeakGlobalRefs essentially WeakReference behavior has been tried in the past, and doing so broke some code. [3] Whether that's still relevant is probably moot now, because of JDK-8071507. [1] http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-January/016223.html [2] https://bugs.openjdk.java.net/browse/JDK-8188066 [3] https://bugs.openjdk.java.net/browse/JDK-4126360 From aleksej.efimov at oracle.com Wed Sep 27 21:28:28 2017 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Wed, 27 Sep 2017 22:28:28 +0100 Subject: [8u-dev][JAXB] Request for review and approval: 8159240: XSOM parser incorrectly processes type names with whitespaces Message-ID: <008ef343-c8f7-bb3d-8c51-5acec3e77a72@oracle.com> Hi, Can I, please, ask for review and approval of JDK-8159240 backport to JDK8? This bug was resolved in JDK9 as part of JAXWS-RI sync changeset (JDK-8164479) therefore separate review for the partially backported changes is needed: http://cr.openjdk.java.net/~aefimov/8159240/8/00/ File changes partially backported to JDK8 from JDK9 sync changeset: ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l37.1 ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l38.1 ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l39.1 ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l40.1 ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l41.1 ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l42.1 ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l43.1 ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l44.1 ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052#l45.1 Testing: ??? New regression test and all jaxb/ws related JCK/JTREG tests shows no failures. JBS bug: ??? https://bugs.openjdk.java.net/browse/JDK-8159240 JAXWS-RI sync bug: ??? https://bugs.openjdk.java.net/browse/JDK-8164479 JDK9 JAXWS-RI sync changeset: ??? http://hg.openjdk.java.net/jdk9/dev/jaxws/rev/26c9b9c51052 JDK9 JAXWS-RI sync review threads: http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-September/043724.html http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-October/043920.html http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-November/044513.html JAXWS-RI changes for XSOM parser: https://github.com/javaee/jaxb-v2/commit/cc9c09a71d739989d1e250d9db8c1e011215abfd#diff-796464168dc0014203b00c9728d41de8 With Best Regards, Aleksei From mandy.chung at oracle.com Wed Sep 27 22:08:39 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 27 Sep 2017 15:08:39 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <59CB46AE.2000701@oracle.com> References: <59CB46AE.2000701@oracle.com> Message-ID: Comment on the CSR: On 9/26/17 11:35 PM, Xueming Shen wrote: > > csr:?????? https://bugs.openjdk.java.net/browse/JDK-8187485 > I think the @apiNote can be simpler. Deflater (similar comment for Inflater) |* @apiNote * In earlier versions, the {@link Object#finalize} method was overridden * to call the {@link #end} method when a {@code Deflater} object * becomes unreachable. * The {@link #finalize} method is no longer defined. If a subclass * overrides ||the {@code end} method, the overridden {@code end} method * is no longer invoked. ?*

* It is strongly recommended to explicitly call {@code end} to ||* discard any unprocessed input promptly to free up resources |* when |||the compressor |is no longer in use.| |ZipFile * @apiNote |* In earlier versions, the {@link Object#finalize} method was overridden * to call the {@link #close} method when a {@code ZipFile} object * becomes unreachable.| |* The {@link #finalize} method is no longer defined. If a subclass * overrides ||the {@code close} method, the overridden {@code close} method * is no longer invoked.| *

658 * Since the time when the resources held by this object will be released 659 * is undetermined, if this method is not invoked explicitly, it is strongly 660 * recommended that applications invoke this method as soon they have 661 * finished accessing this {@code ZipFile}. This will prevent holding up 662 * system resources for an undetermined length of time. 663 *

I would suggest to drop this paragraph. @apiNote and @implNote in class spec cover that. Mandy || From xueming.shen at oracle.com Wed Sep 27 23:41:25 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 27 Sep 2017 16:41:25 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: References: <59CB46AE.2000701@oracle.com> Message-ID: <59CC3725.7080505@oracle.com> Thanks Mandy! I removed the ln#657-#663, and updated the @apiNote in deflter, inflater and zipfile accordingly, mainly combined your comment and the approach for the fis/fo. they are "simpler" and straightforward now, at least for me. https://bugs.openjdk.java.net/browse/JDK-8187485 http://cr.openjdk.java.net/~sherman/8185582/webrev -Sherman On 9/27/17, 3:08 PM, mandy chung wrote: > > Comment on the CSR: > > On 9/26/17 11:35 PM, Xueming Shen wrote: >> >> csr: https://bugs.openjdk.java.net/browse/JDK-8187485 >> > I think the @apiNote can be simpler. > > Deflater (similar comment for Inflater) > | * @apiNote > * In earlier versions, the {@link Object#finalize} method was overridden > * to call the {@link #end} method when a {@code Deflater} object > * becomes unreachable. > * The {@link #finalize} method is no longer defined. If a subclass > * overrides||the {@code end} method, the overridden {@code end} method > * is no longer invoked. > *

> * It is strongly recommended to explicitly call {@code end} to > || * discard any unprocessed input promptly to free up resources > | * when|||the compressor|is no longer in use.| > > > |ZipFile > > * @apiNote > | * In earlier versions, the {@link Object#finalize} method was overridden > * to call the {@link #close} method when a {@code ZipFile} object > * becomes unreachable.| > | * The {@link #finalize} method is no longer defined. If a subclass > * overrides||the {@code close} method, the overridden {@code close} method > * is no longer invoked.| > *

> | * The recommended cleanup for|||{@code ZipFile}| is to explicitly call {@code close} > * or use try-with-resources.| > > 657 *

> 658 * Since the time when the resources held by this object will be released > 659 * is undetermined, if this method is not invoked explicitly, it is strongly > 660 * recommended that applications invoke this method as soon they have > 661 * finished accessing this {@code ZipFile}. This will prevent holding up > 662 * system resources for an undetermined length of time. > 663 *

wrote: > On 09/27/2017 02:38 PM, Paul Sandoz wrote: > >> Looks good, i just need to look a little more closely at the > ConcurrentSkipListMap changes. > >> > > > > > > 328 * Note that, with VarHandles, a boolean result of > > 329 * compareAndSet must be declared even if not used. > > This should not be necessary, since for such methods the return is not > polymorphic. Did you observe some performance or code generation issues? > > > > It was necessary at the time I started rewriting this in June, > but apparently not now. I'll get rid of the annoying unnecessary > declarations. Thanks to whoever fixed this! The latest webrev no longer has the unneeded boolean locals. From Roger.Riggs at Oracle.com Fri Sep 29 17:17:34 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 29 Sep 2017 13:17:34 -0400 Subject: RFR 8080225: FileInputStream cleanup should be improved Message-ID: Replacing finalizers in FileInputStream, FileOutputStream, and adding cleanup to RandomAccessFile and NIO File Channels with Cleaner based implementations required changes to FileDescriptor. The specification of FileInputStream and FileOutputStream is changed to remove the finalizer behavior that required their respective close methods to be called. This change to the behavior is tracked with CSR 8187325 [3]. The FileDescriptor implementations (Unix and Windows) provide a cleanup function that is now used by FIS, FOS, RAF, and async and normal FileChannels.? Each requests FileDescriptor to register a cleanup function when the fd or handle is assigned and delegates to FileDescriptor.close.? If the respective FileDescriptor.close method is not called, the fd or handle will be closed when the FileDescriptor is determined to be phantom reachable. The other uses of FileDescriptor are not intended to be changed, for example in sockets and datagrams. Some tests were modified to not rely on finalization; new tests are provided. Comments are appreciated on both the CSR [3] and the implementation [1]. [1] webrev: http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ [2] Issue:?? https://bugs.openjdk.java.net/browse/JDK-8080225 [3] CSR:? 8187325? FileInputStream/FileOutputStream should use the Cleaner instead of finalize ? https://bugs.openjdk.java.net/browse/JDK-8187325 Thanks, Roger From brian.burkhalter at oracle.com Fri Sep 29 20:02:41 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 29 Sep 2017 13:02:41 -0700 Subject: RFR 8080225: FileInputStream cleanup should be improved In-Reply-To: References: Message-ID: <88B9DF82-DEFB-42FB-9047-4F44B2D0A35A@oracle.com> Hi Roger, Looks good overall; total nitpicks here: FileInputStream (similar story in FileOutputStream) 48-49: ?explicitly" is used twice 53: could probably drop ?explicitly? here altogether. The ?Shd? in a couple of test names is kind of annoying; perhaps s/Shd/Should/ ? Copyright dates are not update in a few places. Thanks, Brian On Sep 29, 2017, at 10:17 AM, Roger Riggs wrote: > Comments are appreciated on [?] the implementation [1]. > > [1] webrev: http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ From peter.levart at gmail.com Fri Sep 29 20:18:59 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 29 Sep 2017 22:18:59 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <59CC3725.7080505@oracle.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> Message-ID: <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> Hi Sherman, I looked into ZipFile as promised. One thing I noticed is that FinalizeZipFile.java test fails compilation: test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: unreported exception Throwable; must be caught or declared to be thrown super.finalize(); ^ (the overridden finalize() in InstrumentedZipFile should now declare throws Throwable, since it overrides Object.finalize() and not ZipFile.finalize() which is gone). The other thing I noticed is that Releaser 1st closes the streams (that are still reachable via streams WeakHashMap) and also ends the associated inflaters. But closing the stream will already release the inflater (in case it is a ZipFileInflaterInputStream) into the inflaters cache and the cache is scanned and inflaters ended later. So we don't need a stream -> inflater association outside the stream in the form of WeekHashMap. But we still need to keep the set of input streams weakly reachable from ZipFile in case we want to close the ZipFile explicitly (and there is no harm although useless if this also happens as a result of automatic ZipFile cleaner processing). This could be implemented in a form of: final Set streams = Collections.newSetFromMap(new WeakHashMap<>()); I also noticed that it is useless to test whether the inflater is ended() when obtaining it from or releasing it into cache if the code keeps the invariant that it never ends inflaters while they are still in cache or associated with the open stream (the only place where inflaters should be ended explicitly is in the Releaser). To make this even more obvious, it might be good to move the obtaining/releasing logic directly into the ZipFileInflaterInputStream constructor which would be passed a reference to the inflatersCache instead of the Inflater instance. Here's what I have in mind (I cahnged just the ZipFile and FinalizeZipFile): http://cr.openjdk.java.net/~plevart/jdk10-dev/8185582_ZIP.cleaner/webrev.01/ What do you think? Because Inflaters used in ZipFile will never be automatically ended by their own cleaners (they are kept strongly reachable in the cache of inflaters, which is strongly reachable from the registered ZipFile cleanup function), it might be useful to add a special package-private constructor to Inflater which would not register its own cleaner. But this could be left as an exercise for some later time. Regards, Peter On 09/28/17 01:41, Xueming Shen wrote: > Thanks Mandy! > > I removed the ln#657-#663, and updated the @apiNote in deflter, inflater > and zipfile accordingly, mainly combined your comment and the approach > for the fis/fo. they are "simpler" and straightforward now, at least > for me. > > https://bugs.openjdk.java.net/browse/JDK-8187485 > http://cr.openjdk.java.net/~sherman/8185582/webrev > > -Sherman > > On 9/27/17, 3:08 PM, mandy chung wrote: >> >> Comment on the CSR: >> >> On 9/26/17 11:35 PM, Xueming Shen wrote: >>> >>> csr: https://bugs.openjdk.java.net/browse/JDK-8187485 >>> >> I think the @apiNote can be simpler. >> >> Deflater (similar comment for Inflater) >> |? * @apiNote >> ? * In earlier versions, the {@link Object#finalize} method was >> overridden >> ? * to call the {@link #end} method when a {@code Deflater} object >> ? * becomes unreachable. >> ? * The {@link #finalize} method is no longer defined.? If a subclass >> ? * overrides||the {@code end} method, the overridden {@code end} method >> ? * is no longer invoked. >> ? *

>> ? * It is strongly recommended to explicitly call {@code end} to >> ||? * discard any unprocessed input promptly to free up resources >> |? * when|||the compressor|is no longer in use.| >> >> >> |ZipFile >> ? ? * @apiNote >> |? * In earlier versions, the {@link Object#finalize} method was >> overridden >> ? * to call the {@link #close} method when a {@code ZipFile} object >> ? * becomes unreachable.| >> |? * The {@link #finalize} method is no longer defined.? If a subclass >> ? * overrides||the {@code close} method, the overridden {@code close} >> method >> ? * is no longer invoked.| >> ? *

>> |? * The recommended cleanup for|||{@code ZipFile}|? is to explicitly >> call {@code close} >> ? * or use try-with-resources.| >> >> ? 657????? *

>> ? 658????? * Since the time when the resources held by this object >> will be released >> ? 659????? * is undetermined, if this method is not invoked >> explicitly, it is strongly >> ? 660????? * recommended that applications invoke this method as soon >> they have >> ? 661????? * finished accessing this {@code ZipFile}. This will >> prevent holding up >> ? 662????? * system resources for an undetermined length of time. >> ? 663????? *

>> >> I would suggest to drop this paragraph.? @apiNote and @implNote in >> class spec cover that. >> >> Mandy >> || > From mandy.chung at oracle.com Fri Sep 29 20:34:52 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 29 Sep 2017 13:34:52 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <23d51a9d-6b25-4324-c293-556215169401@gmail.com> References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> Message-ID: On 9/27/17 2:31 AM, Peter Levart wrote: > > Up to a point where 'this' is dereferenced to obtain the 'zsRef' value > (line 261), the Deflater instance is reachable. But after that, even > ensureOpen() may be inlined and 'this' is not needed any more. After > that point, obtaining zsRef.address() and calling setDictionaly on the > obtained address may be racing with Cleaner thread invoking > ZStreamRef.run(): What about making the native setDictionary method as an instance method (currently it's a static method) so that this object remains strongly reachable until the method returns? Mandy From xueming.shen at oracle.com Fri Sep 29 20:49:04 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 29 Sep 2017 13:49:04 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> Message-ID: <59CEB1C0.2030502@oracle.com> On 9/29/17, 1:18 PM, Peter Levart wrote: > Hi Sherman, > > I looked into ZipFile as promised. > > One thing I noticed is that FinalizeZipFile.java test fails compilation: > > test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: unreported exception Throwable; must be caught or declared to be thrown > super.finalize(); > ^ > (the overridden finalize() in InstrumentedZipFile should now declare > throws Throwable, since it overrides Object.finalize() and not > ZipFile.finalize() which is gone). > > Yes, it's the expected source incompatibility issue specified in the CSR request. I think I had it changed somewhere but obviously it's not in the webrev. Thanks for catching it. Yes, the test needs to update to be catch the throwable. Will return to the other comments later. Thanks! -sherman From forax at univ-mlv.fr Fri Sep 29 20:49:32 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 29 Sep 2017 22:49:32 +0200 (CEST) Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> Message-ID: <42344256.6227374.1506718172110.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "mandy chung" > ?: "Peter Levart" , "Xueming Shen" , "core-libs-dev" > > Envoy?: Vendredi 29 Septembre 2017 22:34:52 > Objet: Re: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers > On 9/27/17 2:31 AM, Peter Levart wrote: >> >> Up to a point where 'this' is dereferenced to obtain the 'zsRef' value >> (line 261), the Deflater instance is reachable. But after that, even >> ensureOpen() may be inlined and 'this' is not needed any more. After >> that point, obtaining zsRef.address() and calling setDictionaly on the >> obtained address may be racing with Cleaner thread invoking >> ZStreamRef.run(): > What about making the native setDictionary method as an instance method > (currently it's a static method) so that this object remains strongly > reachable until the method returns? Mandy, unlike in C or C++, in Java a reference is garbage collected as soon as you do not need it anymore, so using an instance method will not change the issue here. one way to be sure that a referenced object is not garbage collected is to use http://docs.oracle.com/javase/9/docs/api/java/lang/ref/Reference.html#reachabilityFence-java.lang.Object- > > Mandy R?mi From mandy.chung at oracle.com Fri Sep 29 20:50:59 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 29 Sep 2017 13:50:59 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <59CEB1C0.2030502@oracle.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> <59CEB1C0.2030502@oracle.com> Message-ID: <9cce9c8f-7315-29a9-8e1b-ff9bcc7e9eda@oracle.com> On 9/29/17 1:49 PM, Xueming Shen wrote: > On 9/29/17, 1:18 PM, Peter Levart wrote: >> Hi Sherman, >> >> I looked into ZipFile as promised. >> >> One thing I noticed is that FinalizeZipFile.java test fails compilation: >> >> test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: unreported exception Throwable; must be caught or declared to be thrown >> super.finalize(); >> ^ >> (the overridden finalize() in InstrumentedZipFile should now declare >> throws Throwable, since it overrides Object.finalize() and not >> ZipFile.finalize() which is gone). >> >> > Yes, it's the expected source incompatibility issue specified in the > CSR request. > I think I had it changed somewhere but obviously it's not in the > webrev. Thanks > for catching it. Yes, the test needs to update to be catch the throwable. > I would suggest to update InstrumentedZipFile to migrate away from the finalizer. ? I can override the close method instead of the finalize method.?? It can test explicitly calling close (that's what the test is currently doing) and try-with-resource. Mandy From peter.levart at gmail.com Fri Sep 29 20:53:01 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 29 Sep 2017 22:53:01 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> Message-ID: <2dc6c340-aa71-ce41-de81-b7cb2433588a@gmail.com> On 09/29/17 22:34, mandy chung wrote: > > > On 9/27/17 2:31 AM, Peter Levart wrote: >> >> Up to a point where 'this' is dereferenced to obtain the 'zsRef' >> value (line 261), the Deflater instance is reachable. But after that, >> even ensureOpen() may be inlined and 'this' is not needed any more. >> After that point, obtaining zsRef.address() and calling setDictionaly >> on the obtained address may be racing with Cleaner thread invoking >> ZStreamRef.run(): > What about making the native setDictionary method as an instance > method (currently it's a static method) so that this object remains > strongly reachable until the method returns? > > Mandy This is a possibility too, yes. In general there might be other places where the same could be performed. It is equivalent to puting Reference.reachabilityFence(this) after the invocation of setDictionary() static native method. But this would only fix public setDictionary() instance method. There might be other public methods with similar problems. Synchronizing the ZStreamRef.run() method fixes all of them in one place. Regards, Peter From mandy.chung at oracle.com Fri Sep 29 20:54:02 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 29 Sep 2017 13:54:02 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <42344256.6227374.1506718172110.JavaMail.zimbra@u-pem.fr> References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> <42344256.6227374.1506718172110.JavaMail.zimbra@u-pem.fr> Message-ID: On 9/29/17 1:49 PM, Remi Forax wrote: >> On 9/27/17 2:31 AM, Peter Levart wrote: >>> Up to a point where 'this' is dereferenced to obtain the 'zsRef' value >>> (line 261), the Deflater instance is reachable. But after that, even >>> ensureOpen() may be inlined and 'this' is not needed any more. After >>> that point, obtaining zsRef.address() and calling setDictionaly on the >>> obtained address may be racing with Cleaner thread invoking >>> ZStreamRef.run(): >> What about making the native setDictionary method as an instance method >> (currently it's a static method) so that this object remains strongly >> reachable until the method returns? > Mandy, > unlike in C or C++, in Java a reference is garbage collected as soon as you do not need it anymore, > so using an instance method will not change the issue here. > The case that Peter observed is when "this" is being optimized out and becomes unreachable before setDictionary is called.? Since setDictionary is a JNI function, the caller has to pass this (as jobject) to the native function.?? Would that cover this special case? Mandy From Roger.Riggs at Oracle.com Fri Sep 29 20:55:43 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 29 Sep 2017 16:55:43 -0400 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <59CEB1C0.2030502@oracle.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> <59CEB1C0.2030502@oracle.com> Message-ID: <19fe961c-756e-ff24-7da7-475f1cf8e70e@Oracle.com> fyi, The proposed[1]? changes to FileInputStream and FileOutputStream remove the finalize method exposing Object.finalize (throws Throwable) to subclasses.? We may need retain the finalize methods (with empty bodies) to mitigate source compatibility. Roger [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-September/049351.html On 9/29/2017 4:49 PM, Xueming Shen wrote: > On 9/29/17, 1:18 PM, Peter Levart wrote: >> Hi Sherman, >> >> I looked into ZipFile as promised. >> >> One thing I noticed is that FinalizeZipFile.java test fails compilation: >> >> test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: >> unreported exception Throwable; must be caught or declared to be thrown >> ???????????? super.finalize(); >> ?????????????????????????? ^ >> (the overridden finalize() in InstrumentedZipFile should now declare >> throws Throwable, since it overrides Object.finalize() and not >> ZipFile.finalize() which is gone). >> >> > Yes, it's the expected source incompatibility issue specified in the > CSR request. > I think I had it changed somewhere but obviously it's not in the > webrev. Thanks > for catching it. Yes, the test needs to update to be catch the throwable. > > Will return to the other comments later. > > Thanks! > -sherman > > From peter.levart at gmail.com Fri Sep 29 20:56:26 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 29 Sep 2017 22:56:26 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <42344256.6227374.1506718172110.JavaMail.zimbra@u-pem.fr> References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> <42344256.6227374.1506718172110.JavaMail.zimbra@u-pem.fr> Message-ID: <52195fe2-29cc-d433-9031-84f3efeec109@gmail.com> Hi Remi, On 09/29/17 22:49, Remi Forax wrote: > ----- Mail original ----- >> De: "mandy chung" >> ?: "Peter Levart" , "Xueming Shen" , "core-libs-dev" >> >> Envoy?: Vendredi 29 Septembre 2017 22:34:52 >> Objet: Re: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers >> On 9/27/17 2:31 AM, Peter Levart wrote: >>> Up to a point where 'this' is dereferenced to obtain the 'zsRef' value >>> (line 261), the Deflater instance is reachable. But after that, even >>> ensureOpen() may be inlined and 'this' is not needed any more. After >>> that point, obtaining zsRef.address() and calling setDictionaly on the >>> obtained address may be racing with Cleaner thread invoking >>> ZStreamRef.run(): >> What about making the native setDictionary method as an instance method >> (currently it's a static method) so that this object remains strongly >> reachable until the method returns? > Mandy, > unlike in C or C++, in Java a reference is garbage collected as soon as you do not need it anymore, > so using an instance method will not change the issue here. I might be wrong, but native instance method is an exception. It can't be inlined. The preparation for native method invocation makes sure 'this' is kept reachable because it can be dereferenced from the native code then and native code is out-of-bounds for JIT optimization. Regards, Peter > one way to be sure that a referenced object is not garbage collected is to use > http://docs.oracle.com/javase/9/docs/api/java/lang/ref/Reference.html#reachabilityFence-java.lang.Object- > >> Mandy > R?mi From mandy.chung at oracle.com Fri Sep 29 20:58:27 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 29 Sep 2017 13:58:27 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <59CC3725.7080505@oracle.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> Message-ID: On 9/27/17 4:41 PM, Xueming Shen wrote: > Thanks Mandy! > > I removed the ln#657-#663, and updated the @apiNote in deflter, inflater > and zipfile accordingly, mainly combined your comment and the approach > for the fis/fo. they are "simpler" and straightforward now, at least > for me. > > https://bugs.openjdk.java.net/browse/JDK-8187485 > http://cr.openjdk.java.net/~sherman/8185582/webrev > || 76 * specified to call the {@code end} method to close the {@code deflater} and s/deflater/Deflater 80 * The recommended cleanup for compressor is to explicitly call {@code end} 81 * method when it is no longer in use. Existing subclasses of {@code Deflater} 82 * that override {@code end} and require {@code end} to be invoked when the 83 * instance is unreachable should explicitly override {@link Object#finalize} 84 * and call {@code end}. I suggest not to recommend "explicitly override Object.finalize" (in fact, we should discourage it) and the overridden end method should be called explicitly.? This was what I suggested in the previous mail:|| |* It is strongly recommended to explicitly call {@code end} to ||* discard any unprocessed input promptly to free up resources |* when |||the compressor |is no longer in use.| ||Same comment applies to Inflater. 75 * specified to call the {@code end} method to close the {@code inflater} and | s/inflater/Inflater FinalizeZipFile.zip (I have mentioned this in the other mail): I suggest to update InstrumentedZipFile to migrate away from the finalizer. ? I can override the close method instead of the finalize method.?? It can test explicitly calling close (that's what the test is currently doing) and try-with-resource. TestCleaner.java ?130 throw new RuntimeException("'ZipFile.Source.zfile' is not accesible"); s/accesible/accessible Mandy From forax at univ-mlv.fr Fri Sep 29 21:03:56 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 29 Sep 2017 23:03:56 +0200 (CEST) Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <52195fe2-29cc-d433-9031-84f3efeec109@gmail.com> References: <59CB46AE.2000701@oracle.com> <23d51a9d-6b25-4324-c293-556215169401@gmail.com> <42344256.6227374.1506718172110.JavaMail.zimbra@u-pem.fr> <52195fe2-29cc-d433-9031-84f3efeec109@gmail.com> Message-ID: <47490564.6243100.1506719036146.JavaMail.zimbra@u-pem.fr> > De: "Peter Levart" > ?: "Remi Forax" , "mandy chung" > Cc: "Xueming Shen" , "core-libs-dev" > > Envoy?: Vendredi 29 Septembre 2017 22:56:26 > Objet: Re: RFR JDK-8185582, Update Zip implementation to use Cleaner, not > finalizers > Hi Remi, Hi Peter, > On 09/29/17 22:49, Remi Forax wrote: >> ----- Mail original ----- >>> De: "mandy chung" [ mailto:mandy.chung at oracle.com | ] >>> ?: "Peter Levart" [ mailto:peter.levart at gmail.com | ] >>> , "Xueming Shen" [ mailto:xueming.shen at oracle.com | ] >>> , "core-libs-dev" [ mailto:core-libs-dev at openjdk.java.net | >>> ] Envoy?: Vendredi 29 Septembre 2017 22:34:52 >>> Objet: Re: RFR JDK-8185582, Update Zip implementation to use Cleaner, not >>> finalizers >>> On 9/27/17 2:31 AM, Peter Levart wrote: >>>> Up to a point where 'this' is dereferenced to obtain the 'zsRef' value >>>> (line 261), the Deflater instance is reachable. But after that, even >>>> ensureOpen() may be inlined and 'this' is not needed any more. After >>>> that point, obtaining zsRef.address() and calling setDictionaly on the >>>> obtained address may be racing with Cleaner thread invoking >>>> ZStreamRef.run(): >>> What about making the native setDictionary method as an instance method >>> (currently it's a static method) so that this object remains strongly >>> reachable until the method returns? >> Mandy, >> unlike in C or C++, in Java a reference is garbage collected as soon as you do >> not need it anymore, >> so using an instance method will not change the issue here. > I might be wrong, but native instance method is an exception. It can't be > inlined. The preparation for native method invocation makes sure 'this' is kept > reachable because it can be dereferenced from the native code then and native > code is out-of-bounds for JIT optimization. I do not think that you are wrong now but this is a property (a side effect) of the current implementation, Panama and Metropolis may change that. I think it's better to keep the reference available with a reachability fence. > Regards, Peter R?mi >> one way to be sure that a referenced object is not garbage collected is to use [ >> http://docs.oracle.com/javase/9/docs/api/java/lang/ref/Reference.html#reachabilityFence-java.lang.Object >> | >> http://docs.oracle.com/javase/9/docs/api/java/lang/ref/Reference.html#reachabilityFence-java.lang.Object >> ] - >>> Mandy >> R?mi From peter.levart at gmail.com Fri Sep 29 21:11:50 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 29 Sep 2017 23:11:50 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <19fe961c-756e-ff24-7da7-475f1cf8e70e@Oracle.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> <59CEB1C0.2030502@oracle.com> <19fe961c-756e-ff24-7da7-475f1cf8e70e@Oracle.com> Message-ID: <773b32d5-a26d-ab44-551e-53b466d46bef@gmail.com> Hi Roger, On 09/29/17 22:55, Roger Riggs wrote: > fyi, > > The proposed[1]? changes to FileInputStream and FileOutputStream > remove the finalize method > exposing Object.finalize (throws Throwable) to subclasses.? We may > need retain > the finalize methods (with empty bodies) to mitigate source > compatibility. The problem is that empty finalize() method that throws anything other than Throwable will not compile. Regards, Peter > > Roger > > > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-September/049351.html > > On 9/29/2017 4:49 PM, Xueming Shen wrote: >> On 9/29/17, 1:18 PM, Peter Levart wrote: >>> Hi Sherman, >>> >>> I looked into ZipFile as promised. >>> >>> One thing I noticed is that FinalizeZipFile.java test fails >>> compilation: >>> >>> test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: >>> unreported exception Throwable; must be caught or declared to be thrown >>> ???????????? super.finalize(); >>> ?????????????????????????? ^ >>> (the overridden finalize() in InstrumentedZipFile should now declare >>> throws Throwable, since it overrides Object.finalize() and not >>> ZipFile.finalize() which is gone). >>> >>> >> Yes, it's the expected source incompatibility issue specified in the >> CSR request. >> I think I had it changed somewhere but obviously it's not in the >> webrev. Thanks >> for catching it. Yes, the test needs to update to be catch the >> throwable. >> >> Will return to the other comments later. >> >> Thanks! >> -sherman >> >> > From peter.levart at gmail.com Fri Sep 29 21:14:33 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 29 Sep 2017 23:14:33 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <773b32d5-a26d-ab44-551e-53b466d46bef@gmail.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> <59CEB1C0.2030502@oracle.com> <19fe961c-756e-ff24-7da7-475f1cf8e70e@Oracle.com> <773b32d5-a26d-ab44-551e-53b466d46bef@gmail.com> Message-ID: On 09/29/17 23:11, Peter Levart wrote: > Hi Roger, > > On 09/29/17 22:55, Roger Riggs wrote: >> fyi, >> >> The proposed[1]? changes to FileInputStream and FileOutputStream >> remove the finalize method >> exposing Object.finalize (throws Throwable) to subclasses.? We may >> need retain >> the finalize methods (with empty bodies) to mitigate source >> compatibility. > > The problem is that empty finalize() method that throws anything other > than Throwable will not compile. Correction - it will compile (I was thinking about a method calling just super.finalize() which is not empty, of course). Yes, this is the solution to source compatibility. > > Regards, Peter > >> >> Roger >> >> >> [1] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-September/049351.html >> >> On 9/29/2017 4:49 PM, Xueming Shen wrote: >>> On 9/29/17, 1:18 PM, Peter Levart wrote: >>>> Hi Sherman, >>>> >>>> I looked into ZipFile as promised. >>>> >>>> One thing I noticed is that FinalizeZipFile.java test fails >>>> compilation: >>>> >>>> test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: >>>> unreported exception Throwable; must be caught or declared to be >>>> thrown >>>> ???????????? super.finalize(); >>>> ?????????????????????????? ^ >>>> (the overridden finalize() in InstrumentedZipFile should now >>>> declare throws Throwable, since it overrides Object.finalize() and >>>> not ZipFile.finalize() which is gone). >>>> >>>> >>> Yes, it's the expected source incompatibility issue specified in the >>> CSR request. >>> I think I had it changed somewhere but obviously it's not in the >>> webrev. Thanks >>> for catching it. Yes, the test needs to update to be catch the >>> throwable. >>> >>> Will return to the other comments later. >>> >>> Thanks! >>> -sherman >>> >>> >> > From xueming.shen at oracle.com Fri Sep 29 21:14:49 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 29 Sep 2017 14:14:49 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> Message-ID: <59CEB7C9.8050104@oracle.com> On 9/29/17, 1:58 PM, mandy chung wrote: > > > On 9/27/17 4:41 PM, Xueming Shen wrote: >> Thanks Mandy! >> >> I removed the ln#657-#663, and updated the @apiNote in deflter, inflater >> and zipfile accordingly, mainly combined your comment and the approach >> for the fis/fo. they are "simpler" and straightforward now, at least >> for me. >> >> https://bugs.openjdk.java.net/browse/JDK-8187485 >> http://cr.openjdk.java.net/~sherman/8185582/webrev >> || > > 76 * specified to call the {@code end} method to close the {@code deflater} and > > s/deflater/Deflater > 80 * The recommended cleanup for compressor is to explicitly call {@code end} > 81 * method when it is no longer in use. Existing subclasses of {@code Deflater} > 82 * that override {@code end} and require {@code end} to be invoked when the > 83 * instance is unreachable should explicitly override {@link Object#finalize} > 84 * and call {@code end}. > I suggest not to recommend "explicitly override Object.finalize" (in > fact, we should discourage it) and the overridden end method should be > called explicitly. This was what I suggested in the previous mail: "calling end() directly/explicitly to release the resource" is being recommended at the first sentence. The second sentence here is meant to provide a possible alternative if any existing subclass is implemented the way that it has the dependency on the old mechanism that its own "end()" being called, for situation that the Cleaner is not possible, or difficult to implement. No value at all? -Sherman > || > | * It is strongly recommended to explicitly call {@code end} to > || * discard any unprocessed input promptly to free up resources > | * when|||the compressor|is no longer in use.| > > ||Same comment applies to Inflater. > > 75 * specified to call the {@code end} method to close the {@code inflater} and > | > From forax at univ-mlv.fr Fri Sep 29 21:23:21 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 29 Sep 2017 23:23:21 +0200 (CEST) Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> <59CEB1C0.2030502@oracle.com> <19fe961c-756e-ff24-7da7-475f1cf8e70e@Oracle.com> <773b32d5-a26d-ab44-551e-53b466d46bef@gmail.com> Message-ID: <335122608.6249049.1506720201183.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Peter Levart" > ?: "Roger Riggs" , "core-libs-dev" > Envoy?: Vendredi 29 Septembre 2017 23:14:33 > Objet: Re: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers > On 09/29/17 23:11, Peter Levart wrote: >> Hi Roger, >> >> On 09/29/17 22:55, Roger Riggs wrote: >>> fyi, >>> >>> The proposed[1]? changes to FileInputStream and FileOutputStream >>> remove the finalize method >>> exposing Object.finalize (throws Throwable) to subclasses.? We may >>> need retain >>> the finalize methods (with empty bodies) to mitigate source >>> compatibility. >> >> The problem is that empty finalize() method that throws anything other >> than Throwable will not compile. > > Correction - it will compile (I was thinking about a method calling just > super.finalize() which is not empty, of course). Yes, this is the > solution to source compatibility. > Overriding finalize() will make the object not to be GCed as soon as it should. > >> >> Regards, Peter >> R?mi >>> >>> Roger >>> >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-September/049351.html >>> >>> On 9/29/2017 4:49 PM, Xueming Shen wrote: >>>> On 9/29/17, 1:18 PM, Peter Levart wrote: >>>>> Hi Sherman, >>>>> >>>>> I looked into ZipFile as promised. >>>>> >>>>> One thing I noticed is that FinalizeZipFile.java test fails >>>>> compilation: >>>>> >>>>> test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: >>>>> unreported exception Throwable; must be caught or declared to be >>>>> thrown >>>>> ???????????? super.finalize(); >>>>> ?????????????????????????? ^ >>>>> (the overridden finalize() in InstrumentedZipFile should now >>>>> declare throws Throwable, since it overrides Object.finalize() and >>>>> not ZipFile.finalize() which is gone). >>>>> >>>>> >>>> Yes, it's the expected source incompatibility issue specified in the >>>> CSR request. >>>> I think I had it changed somewhere but obviously it's not in the >>>> webrev. Thanks >>>> for catching it. Yes, the test needs to update to be catch the >>>> throwable. >>>> >>>> Will return to the other comments later. >>>> >>>> Thanks! >>>> -sherman >>>> >>>> >>> From peter.levart at gmail.com Fri Sep 29 21:38:22 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 29 Sep 2017 23:38:22 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <335122608.6249049.1506720201183.JavaMail.zimbra@u-pem.fr> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> <59CEB1C0.2030502@oracle.com> <19fe961c-756e-ff24-7da7-475f1cf8e70e@Oracle.com> <773b32d5-a26d-ab44-551e-53b466d46bef@gmail.com> <335122608.6249049.1506720201183.JavaMail.zimbra@u-pem.fr> Message-ID: <8e22caab-0ee1-97e0-7aa5-8018f56c5b36@gmail.com> On 09/29/17 23:23, Remi Forax wrote: > ----- Mail original ----- >> De: "Peter Levart" >> ?: "Roger Riggs" , "core-libs-dev" >> Envoy?: Vendredi 29 Septembre 2017 23:14:33 >> Objet: Re: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers >> On 09/29/17 23:11, Peter Levart wrote: >>> Hi Roger, >>> >>> On 09/29/17 22:55, Roger Riggs wrote: >>>> fyi, >>>> >>>> The proposed[1]? changes to FileInputStream and FileOutputStream >>>> remove the finalize method >>>> exposing Object.finalize (throws Throwable) to subclasses.? We may >>>> need retain >>>> the finalize methods (with empty bodies) to mitigate source >>>> compatibility. >>> The problem is that empty finalize() method that throws anything other >>> than Throwable will not compile. >> Correction - it will compile (I was thinking about a method calling just >> super.finalize() which is not empty, of course). Yes, this is the >> solution to source compatibility. >> > Overriding finalize() will make the object not to be GCed as soon as it should. I think the hotspot has an optimization and detects that finalize() has no bytecodes and treats the object as not needing finalization. Regards, Peter >>> Regards, Peter >>> > R?mi > >>>> Roger >>>> >>>> >>>> [1] >>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-September/049351.html >>>> >>>> On 9/29/2017 4:49 PM, Xueming Shen wrote: >>>>> On 9/29/17, 1:18 PM, Peter Levart wrote: >>>>>> Hi Sherman, >>>>>> >>>>>> I looked into ZipFile as promised. >>>>>> >>>>>> One thing I noticed is that FinalizeZipFile.java test fails >>>>>> compilation: >>>>>> >>>>>> test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: >>>>>> unreported exception Throwable; must be caught or declared to be >>>>>> thrown >>>>>> ???????????? super.finalize(); >>>>>> ?????????????????????????? ^ >>>>>> (the overridden finalize() in InstrumentedZipFile should now >>>>>> declare throws Throwable, since it overrides Object.finalize() and >>>>>> not ZipFile.finalize() which is gone). >>>>>> >>>>>> >>>>> Yes, it's the expected source incompatibility issue specified in the >>>>> CSR request. >>>>> I think I had it changed somewhere but obviously it's not in the >>>>> webrev. Thanks >>>>> for catching it. Yes, the test needs to update to be catch the >>>>> throwable. >>>>> >>>>> Will return to the other comments later. >>>>> >>>>> Thanks! >>>>> -sherman >>>>> >>>>> From mandy.chung at oracle.com Fri Sep 29 21:45:18 2017 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 29 Sep 2017 14:45:18 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <8e22caab-0ee1-97e0-7aa5-8018f56c5b36@gmail.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> <59CEB1C0.2030502@oracle.com> <19fe961c-756e-ff24-7da7-475f1cf8e70e@Oracle.com> <773b32d5-a26d-ab44-551e-53b466d46bef@gmail.com> <335122608.6249049.1506720201183.JavaMail.zimbra@u-pem.fr> <8e22caab-0ee1-97e0-7aa5-8018f56c5b36@gmail.com> Message-ID: <0628f0e3-1fa1-16c0-8f27-d4990c7b3ed1@oracle.com> On 9/29/17 2:38 PM, Peter Levart wrote: > I think the hotspot has an optimization and detects that finalize() > has no bytecodes and treats the object as not needing finalization. http://hg.openjdk.java.net/jdk10/master/file/7d67bb6b0599/src/hotspot/share/classfile/classFileParser.cpp#l4250 ? // Check if this klass has an empty finalize method (i.e. one with return bytecode only), ? // in which case we don't have to register objects as finalizable ? if (!_has_empty_finalizer) { ??? if (_has_finalizer || ??????? (super != NULL && super->has_finalizer())) { ????? ik->set_has_finalizer(); ??? } ? } Mandy From forax at univ-mlv.fr Fri Sep 29 22:00:23 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 30 Sep 2017 00:00:23 +0200 (CEST) Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <0628f0e3-1fa1-16c0-8f27-d4990c7b3ed1@oracle.com> References: <59CB46AE.2000701@oracle.com> <59CEB1C0.2030502@oracle.com> <19fe961c-756e-ff24-7da7-475f1cf8e70e@Oracle.com> <773b32d5-a26d-ab44-551e-53b466d46bef@gmail.com> <335122608.6249049.1506720201183.JavaMail.zimbra@u-pem.fr> <8e22caab-0ee1-97e0-7aa5-8018f56c5b36@gmail.com> <0628f0e3-1fa1-16c0-8f27-d4990c7b3ed1@oracle.com> Message-ID: <1971601616.6254974.1506722423732.JavaMail.zimbra@u-pem.fr> Very cool, i learn something today :) R?mi ----- Mail original ----- > De: "mandy chung" > ?: "core-libs-dev" > Envoy?: Vendredi 29 Septembre 2017 23:45:18 > Objet: Re: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers > On 9/29/17 2:38 PM, Peter Levart wrote: >> I think the hotspot has an optimization and detects that finalize() >> has no bytecodes and treats the object as not needing finalization. > http://hg.openjdk.java.net/jdk10/master/file/7d67bb6b0599/src/hotspot/share/classfile/classFileParser.cpp#l4250 > > ? // Check if this klass has an empty finalize method (i.e. one with > return bytecode only), > ? // in which case we don't have to register objects as finalizable > ? if (!_has_empty_finalizer) { > ??? if (_has_finalizer || > ??????? (super != NULL && super->has_finalizer())) { > ????? ik->set_has_finalizer(); > ??? } > ? } > > Mandy From xueming.shen at oracle.com Fri Sep 29 22:18:08 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 29 Sep 2017 15:18:08 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> Message-ID: <59CEC6A0.9000601@oracle.com> On 9/29/17, 1:18 PM, Peter Levart wrote: > Hi Sherman, > putStream> streams = Collections.newSetFromMap(new WeakHashMap<>()); > > I also noticed that it is useless to test whether the inflater is > ended() when obtaining it from or releasing it into cache if the code > keeps the invariant that it never ends inflaters while they are still > in cache or associated with the open stream (the only place where > inflaters should be ended explicitly is in the Releaser). To make this > even more obvious, it might be good to move the obtaining/releasing > logic directly into the ZipFileInflaterInputStream constructor which > would be passed a reference to the inflatersCache instead of the > Inflater instance. > If my memory serves me correctly, this "ended()" check was for the situation that the ZipfileInflaterStream is getting finalized (close() is not called explicitly/directly) and its inflater is also getting finalized at the same time/pass, and the inflater.end() gets called (by its finalize()) first, so when the zfis tries to release the inflater back to the cache, it has been "ended" already. We had some nasty finalize() and object resurrection timing issue in earlier releases, that probably is part of the fix (since it's a timing issue, we also do the same "ended()" check in getInflater() when lease it out) So my question is that do we have the similar situation under the Cleaner? mean the zfis and its inflater both are phantom reachable and the inflater's cleaner gets called first? I'm still reading your proposed change to remove the "inflater" from the stream->inflater mapping to see if there is any loophole, will reply that separately later. -sherman From xueming.shen at oracle.com Sat Sep 30 00:02:46 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 29 Sep 2017 17:02:46 -0700 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> Message-ID: <59CEDF26.60301@oracle.com> On 9/29/17, 1:18 PM, Peter Levart wrote: > Hi Sherman, > > I looked into ZipFile as promised. > > One thing I noticed is that FinalizeZipFile.java test fails compilation: > > test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: unreported exception Throwable; must be caught or declared to be thrown > super.finalize(); > ^ > (the overridden finalize() in InstrumentedZipFile should now declare > throws Throwable, since it overrides Object.finalize() and not > ZipFile.finalize() which is gone). > > > The other thing I noticed is that Releaser 1st closes the streams > (that are still reachable via streams WeakHashMap) and also ends the > associated inflaters. But closing the stream will already release the > inflater (in case it is a ZipFileInflaterInputStream) into the > inflaters cache and the cache is scanned and inflaters ended later. > > So we don't need a stream -> inflater association outside the stream > in the form of WeekHashMap. But we still need to keep the set of input > streams weakly reachable from ZipFile in case we want to close the > ZipFile explicitly (and there is no harm although useless if this also > happens as a result of automatic ZipFile cleaner processing). > > This could be implemented in a form of: > > final Set streams = Collections.newSetFromMap(new > WeakHashMap<>()); > > I also noticed that it is useless to test whether the inflater is > ended() when obtaining it from or releasing it into cache if the code > keeps the invariant that it never ends inflaters while they are still > in cache or associated with the open stream (the only place where > inflaters should be ended explicitly is in the Releaser). To make this > even more obvious, it might be good to move the obtaining/releasing > logic directly into the ZipFileInflaterInputStream constructor which > would be passed a reference to the inflatersCache instead of the > Inflater instance. > > Here's what I have in mind (I cahnged just the ZipFile and > FinalizeZipFile): > > http://cr.openjdk.java.net/~plevart/jdk10-dev/8185582_ZIP.cleaner/webrev.01/ > > What do you think? > Peter, I read those old emails back to 2011 on why we put the "Inflater" into the streams as the value of the map entry. It appears the main (only) purpose is to keep the finalization of in order. As I explained in previous email, stream and its inflater can be phantom reachable at same round, if inflater gets finalized/ended first, then it can no longer be reused/cached and has to be thrown away, which means the caching mechanism is actually broken/not functioning. We do have use scenario depends on it to avoid too many "new Inflater()' that might pressure the memory usage. Putting the pair in a weakhashmap appears to solve this problem back then (the ended() checks are still there just in case there is rare case, the inflater still gets cleaned up first). The situation might be changed now in Cleaner case, as the cleaner object itself now holds a strong reference, which in theory will/should prevent the inflater from being phantom reachable before stream is cleaned, so we might no longer need this pair in a map to have a "ordered" cleanup. Just wanted to make sure my assumption is correct here. sherman From zheng.jun.li at oracle.com Sat Sep 30 02:55:36 2017 From: zheng.jun.li at oracle.com (Jack Li) Date: Sat, 30 Sep 2017 10:55:36 +0800 Subject: RFR: 8187954 Update JAX-WS RI integration to latest version Message-ID: <12668CC3-A8DB-4BDC-9F21-5068CA559DA8@oracle.com> Hi, Please review standalone JAXB/JAXWS changes, synced to jdk/jaxws repo. JBS: https://bugs.openjdk.java.net/browse/JDK-8187954 Webrev: http://cr.openjdk.java.net/~aefimov/jaxws-integrations/8187954/10/00/ Summary of changes: jaxws/src/java.xml.bind/share/classes/javax/xml/bind/* JDK-8186946 - Fix accessibility and other issues in the java.xml.bind module jaxws/src/java.xml.ws/share/classes/com/sun/xml/internal/messaging/saaj/** JDK-8186314 - code at c.s.x.i.m.saaj.soap.MessageImpl must be modified to avoid crash after javac change And also contains the fixes for importing nodes for SOAPDocumentFragment Patch also contains several small bugfixes, not tracked in JBS. ---------------- Best regards Jack Li From ecki at zusammenkunft.net Sat Sep 30 04:34:51 2017 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Sat, 30 Sep 2017 04:34:51 +0000 Subject: RFR 8080225: FileInputStream cleanup should be improved In-Reply-To: References: Message-ID: Hello, if UnreferencedRAFClosesFd.java is supposed to test the behavior of unreachable ?raf? I wonder how it works as it hold on raf with a reachabilityFence at the end of the main method? Maybe that asks for an explanative comment? Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: core-libs-dev on behalf of Roger Riggs Sent: Friday, September 29, 2017 7:17:34 PM To: Core-Libs-Dev Subject: RFR 8080225: FileInputStream cleanup should be improved Replacing finalizers in FileInputStream, FileOutputStream, and adding cleanup to RandomAccessFile and NIO File Channels with Cleaner based implementations required changes to FileDescriptor. The specification of FileInputStream and FileOutputStream is changed to remove the finalizer behavior that required their respective close methods to be called. This change to the behavior is tracked with CSR 8187325 [3]. The FileDescriptor implementations (Unix and Windows) provide a cleanup function that is now used by FIS, FOS, RAF, and async and normal FileChannels. Each requests FileDescriptor to register a cleanup function when the fd or handle is assigned and delegates to FileDescriptor.close. If the respective FileDescriptor.close method is not called, the fd or handle will be closed when the FileDescriptor is determined to be phantom reachable. The other uses of FileDescriptor are not intended to be changed, for example in sockets and datagrams. Some tests were modified to not rely on finalization; new tests are provided. Comments are appreciated on both the CSR [3] and the implementation [1]. [1] webrev: http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ [2] Issue: https://bugs.openjdk.java.net/browse/JDK-8080225 [3] CSR: 8187325 FileInputStream/FileOutputStream should use the Cleaner instead of finalize https://bugs.openjdk.java.net/browse/JDK-8187325 Thanks, Roger From peter.levart at gmail.com Sat Sep 30 07:41:41 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 30 Sep 2017 09:41:41 +0200 Subject: RFR JDK-8185582, Update Zip implementation to use Cleaner, not finalizers In-Reply-To: <59CEDF26.60301@oracle.com> References: <59CB46AE.2000701@oracle.com> <59CC3725.7080505@oracle.com> <32f3d1cc-0423-f28e-279f-900a8df564e2@gmail.com> <59CEDF26.60301@oracle.com> Message-ID: Hi Sherman, On 09/30/17 02:02, Xueming Shen wrote: > On 9/29/17, 1:18 PM, Peter Levart wrote: >> Hi Sherman, >> >> I looked into ZipFile as promised. >> >> One thing I noticed is that FinalizeZipFile.java test fails compilation: >> >> test/jdk/java/util/zip/ZipFile/FinalizeZipFile.java:49: error: unreported exception Throwable; must be caught or declared to be thrown >> super.finalize(); >> ^ >> (the overridden finalize() in InstrumentedZipFile should now declare >> throws Throwable, since it overrides Object.finalize() and not >> ZipFile.finalize() which is gone). >> >> >> The other thing I noticed is that Releaser 1st closes the streams >> (that are still reachable via streams WeakHashMap) and also ends the >> associated inflaters. But closing the stream will already release the >> inflater (in case it is a ZipFileInflaterInputStream) into the >> inflaters cache and the cache is scanned and inflaters ended later. >> >> So we don't need a stream -> inflater association outside the stream >> in the form of WeekHashMap. But we still need to keep the set of >> input streams weakly reachable from ZipFile in case we want to close >> the ZipFile explicitly (and there is no harm although useless if this >> also happens as a result of automatic ZipFile cleaner processing). >> >> This could be implemented in a form of: >> >> final Set streams = Collections.newSetFromMap(new >> WeakHashMap<>()); >> >> I also noticed that it is useless to test whether the inflater is >> ended() when obtaining it from or releasing it into cache if the code >> keeps the invariant that it never ends inflaters while they are still >> in cache or associated with the open stream (the only place where >> inflaters should be ended explicitly is in the Releaser). To make >> this even more obvious, it might be good to move the >> obtaining/releasing logic directly into the >> ZipFileInflaterInputStream constructor which would be passed a >> reference to the inflatersCache instead of the Inflater instance. >> >> Here's what I have in mind (I cahnged just the ZipFile and >> FinalizeZipFile): >> >> http://cr.openjdk.java.net/~plevart/jdk10-dev/8185582_ZIP.cleaner/webrev.01/ >> >> What do you think? >> > Peter, > > I read those old emails back to 2011 on why we put the "Inflater" into > the streams as > the value of the map entry. It appears the main (only) purpose is to > keep the finalization > of in order. As I explained in previous email, > stream and its inflater > can be phantom reachable at same round, if inflater gets > finalized/ended first, then it > can no longer be reused/cached and has to be thrown away, which means > the caching > mechanism is actually broken/not functioning. We do have use scenario > depends on it > to avoid too many "new Inflater()' that might pressure the memory > usage. Putting the > pair in a weakhashmap appears to solve this problem back then (the > ended() checks are > still there just in case there is rare case, the inflater still gets > cleaned up first). > > The situation might be changed now in Cleaner case, as the cleaner > object itself now > holds a strong reference, which in theory will/should prevent the > inflater from being > phantom reachable before stream is cleaned, so we might no longer need > this > pair in a map to have a "ordered" cleanup. Just > wanted to make sure > my assumption is correct here. > > sherman > Right, the Inflater is captured by the cleaning function for the ZipFileInflaterInputStream and is kept strongly reachable until the function fires, which may happen automatically or manually. At that time, it is returned to the inflaters cache, which is rooted at the ZipFile instance and also captured by the ZipFile cleaning function - the Releaser, which is strongly reachable until fired. So I claim that there is no possibility for Inflaters used in ZipFile to ever get cleaned-up (ended) automatically by their cleaner functions as a result of getting phantom reachable. The situation has changed. It could even be beneficial to create a package-private Inflater constructor for this case, which creates Inflaters without cleaner registration. Regards, Peter From Roger.Riggs at Oracle.com Sat Sep 30 15:55:32 2017 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Sat, 30 Sep 2017 11:55:32 -0400 Subject: RFR 8080225: FileInputStream cleanup should be improved In-Reply-To: References: Message-ID: Hi Bernd, The reachabilityFence only ensures that the local variable 'ref' is not prematurely deemed to be out of scope? by the hotspot compiler. The value of 'raf' is set to null before the gc is triggered (usually) on the first iteration of the loop. I'll add a comment. Regards, Roger On 9/30/2017 12:34 AM, Bernd Eckenfels wrote: > Hello, > > if UnreferencedRAFClosesFd.java is supposed to test the behavior of > unreachable ?raf? I wonder how it works as it hold on raf with a > reachabilityFence at the end of the main method? Maybe that asks for > an explanative comment? > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ------------------------------------------------------------------------ > *From:* core-libs-dev on > behalf of Roger Riggs > *Sent:* Friday, September 29, 2017 7:17:34 PM > *To:* Core-Libs-Dev > *Subject:* RFR 8080225: FileInputStream cleanup should be improved > Replacing finalizers in FileInputStream, FileOutputStream, and adding > cleanup to RandomAccessFile > and NIO File Channels with Cleaner based implementations required > changes to FileDescriptor. > > The specification of FileInputStream and FileOutputStream is changed to > remove the finalizer > behavior that required their respective close methods to be called. > This change to the behavior is tracked with CSR 8187325 [3]. > > The FileDescriptor implementations (Unix and Windows) provide a cleanup > function that is now used by > FIS, FOS, RAF, and async and normal FileChannels.? Each requests > FileDescriptor to register a cleanup function > when the fd or handle is assigned and delegates to > FileDescriptor.close.? If the respective > FileDescriptor.close method is not called, the fd or handle will be > closed when the FileDescriptor > is determined to be phantom reachable. > > The other uses of FileDescriptor are not intended to be changed, for > example in sockets and datagrams. > > Some tests were modified to not rely on finalization; new tests are > provided. > > Comments are appreciated on both the CSR [3] and the implementation [1]. > > [1] webrev: > http://cr.openjdk.java.net/~rriggs/webrev-fis-cleanup-8080225/ > > > [2] Issue: https://bugs.openjdk.java.net/browse/JDK-8080225 > > [3] CSR:? 8187325? FileInputStream/FileOutputStream should use the > Cleaner instead of finalize > https://bugs.openjdk.java.net/browse/JDK-8187325 > > Thanks, Roger > >